提问人:DaminouTav 提问时间:3/24/2022 最后编辑:DaminouTav 更新时间:3/27/2022 访问量:126
Mongodb大时间序列抽取/查询改进
Mongodb large time series decimation / query improvments
问:
我有一个传感器数据的时间序列集合,例如:
{"date":{"$date":"2018-01-01T00:00:05.045Z"},"metaField":{"assets":{"city":"Meylan","couleur":"BlougeCeyBienBlouge","country":"France","machine_name":"totomachine","monitoring_id":666},"input_name":"inputname1","results":{"head":{"dimension":1,"id":"HEAD","is_complex":0,"magnitude_key":"Acceleration","measure_status":2,"name":"FFT 1: FFT1: OvPwre [1]-Input 1","overall_level_status":0,"physical_quantity":"","result_type":1,"size":9,"unit_label":null,"unit_name":null,"version":5,"weighted_ol_status":0,"weighting_window":0},"info":{"global_level":0,"id":"INFO","info_mask":0,"tacho":0,"version":1,"weighted_global_level":0},"module_id":10,"process_id":21,"scal":{"id":"SCAL","imag":0,"version":2}}},"_id":{"$oid":"623b3c53f492b4b87a47f0d6"},"value":51}
{"date":{"$date":"2018-01-01T00:00:10.122Z"},"metaField":{"assets":{"city":"Meylan","couleur":"BlougeCeyBienBlouge","country":"France","machine_name":"totomachine","monitoring_id":666},"input_name":"inputname1","results":{"head":{"dimension":1,"id":"HEAD","is_complex":0,"magnitude_key":"Acceleration","measure_status":2,"name":"FFT 1: FFT1: OvPwre [1]-Input 1","overall_level_status":0,"physical_quantity":"","result_type":1,"size":9,"unit_label":null,"unit_name":null,"version":5,"weighted_ol_status":0,"weighting_window":0},"info":{"global_level":0,"id":"INFO","info_mask":0,"tacho":0,"version":1,"weighted_global_level":0},"module_id":10,"process_id":21,"scal":{"id":"SCAL","imag":0,"version":2}}},"_id":{"$oid":"623b3c53f492b4b87a47f0dc"},"value":77}
{"date":{"$date":"2018-01-01T00:00:15.165Z"},"metaField":{"assets":{"city":"Meylan","couleur":"BlougeCeyBienBlouge","country":"France","machine_name":"totomachine","monitoring_id":666},"input_name":"inputname1","results":{"head":{"dimension":1,"id":"HEAD","is_complex":0,"magnitude_key":"Acceleration","measure_status":2,"name":"FFT 1: FFT1: OvPwre [1]-Input 1","overall_level_status":0,"physical_quantity":"","result_type":1,"size":9,"unit_label":null,"unit_name":null,"version":5,"weighted_ol_status":0,"weighting_window":0},"info":{"global_level":0,"id":"INFO","info_mask":0,"tacho":0,"version":1,"weighted_global_level":0},"module_id":10,"process_id":21,"scal":{"id":"SCAL","imag":0,"version":2}}},"_id":{"$oid":"623b3c53f492b4b87a47f0e2"},"value":100}
索引为:日期 -1、input_name 1、metaField.results.module_id 1 和 metaField.results.process_id 1。
可能是一百个传感器,每 10 秒推送一次样本。我想从 input_name、module_id 和 process_id 查询数组。如果范围日期很大,则需要抽取。
我首先尝试聚合每年/每月/每天/小时的值(如果范围时间不太大,则为分钟):
{"$match": {"metaField.input_name": "inputname1", "metaField.results.module_id": 10, "metaField.results.process_id": 21, "date:"{"$gte": new ISODate("2018-01-01T01:01:01Z"), "$lt": new ISODate("2020-01-01T01:01:01Z")}},
"$group": {"_id": {
"input_name": "$metaField.input_name",
"hour": {"$hour": "$date"},
"day": {"$dayOfMonth": "$date"},
"month": {"$month": "$date"},
"year": {"$year": "$date"}},
"date": {"$first": "$date"},
"value": {"$first": "$value"}},
"$sort": {"date": 1},
"$group": {"_id": "$_id.input_name",
"data": {"$push": "$value"},
"date": {"$push": "$date"}}}
但是对于巨大的日期范围来说,性能是不可接受的......(30 秒,持续 3 年......
你有什么更好的主意,或者有一种方法可以不基于日期进行抽取吗?比如检索 1/10 值?
答: 暂无答案
评论
$match