Wenqiang Yue
ABSTRACT
With the development of environmental sensor technology, more and more environmental sensors are widely used in various fields, such as environmental monitoring, smart home, smart city and so on. However, due to the complexity and diversity of environmental sensor data, there are a high number of anomalous data in sensor data, which can lead to misjudgment and false positives, compromising data reliability and accuracy. Therefore, the detection method of abnormal data in different dimensions of environmental sensors based on machine learning algorithms were studied in this paper. Firstly, the environmental sensor data of IBRL (Intel Berkeley Research Lab) data set was preprocessed, including data cleaning, data normalization and feature extraction. Then, the simulation experiment was completed using the data set, and the data of different dimensions was detected using machine learning algorithms such as OneClass Support Vector Machine algorithm, Local Outlier Factor algorithm, and Isolation Forest algorithm. Finally, through experimental comparison and analysis of the application effects of the above three algorithms in anomaly detection of environmental data in different dimensions, the Isolation Forest algorithm with the best comprehensive detection effect was finally selected for practical engineering projects. In practical engineering projects, maintenance costs can be effectively reduced, project safety and reliability can be improved, and the purpose of improving project efficiency can ultimately be achieved by the proposed method.
- Hao Z J, Xu H W, Dang X C, 2020. Method for Patching Three-Dimensional Surface Coverage Loopholes of Hybrid Nodes in Wireless Sensor Nerworkst. Journal of Sensors, no.1. https://doi.org/10.1155/2020/6492457Google Scholar
Cross Ref
- Xiong Juxia, Wu Jinzhao. 2020. Dynamic tracking simulation of abnormal nodes in high-dimensional data stream, Computer simulation, vol.37, no.10, 445-449.Google Scholar
- Zou Chengming, Chen De. 2021. Unsupervised anomaly detection method for high-dimensional big data analysis, Computer Science, vol.48, no.2, 121-127.Google Scholar
- Xi Liang, Wang Ruidong, 2021. Unsupervised deep anomaly detection model based on sample association perception, Journal of Computer Science, vol.44, no.11, 2317-2331.Google Scholar
- Kim D, Lee S. 2020. An Ensemble-Based Approach to Anomaly Detection in Marine Engine Sensor Streams for Efficient Condition Monitoring and Analysis, Sensors, vol.20, no.24. https://doi.org/10.3390/s20247285Google ScholarCross Ref
- Xiao Zheng. 2020. Applied Research on Anomaly Detection of Support Vector Machine, Journal of Yellow River University of Science and Technology, vol.22, no.8, 67-69.Google Scholar
- V Erik, B Andreas. 2019. Unsupervised anomaly detection based on clustering methods and sensor data on a marine diesel engine, Journal of Marine Engineering & Technology, vol.20, no.4, 1-18. https://doi.org/10.1080/20464177.2019.1633223Google ScholarCross Ref
- Liu Yi, Lan Shaohua. 2017. Network time covert channel detection method based on One-classSVM, Computer and Modernization, vol.6, no.5, 108-111.Google Scholar
- Peng Chao, Tang Xianghong, Lu Jianguang. 2020. Bearing fault diagnosis based on edge computing, Combined machine tool and automatic machining technology, vol.562, no.12, 52-55.Google Scholar
- BREUNIG M M, KRIEGEL H P, NG R T, 2000. LOF: identifying density-based local outliers, Sigmod Record, vol.29, no.2, 93-104. https://doi.org/10.1145/342009.335388Google ScholarDigital Library
- KNORR E M, NG R T, TUCAKOV V. 2000. Distance-based outliers: algorithms and applications. VLDB Journal, vol.8, no.3, 237-253. https://doi.org/10.1007/s007780050006Google ScholarDigital Library
- Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation Forest, IEEE International Conference on Data Mining(ICDM), 413-422. https://doi.org/10.1109/ICDM.2008.17Google ScholarDigital Library
- Wang Hailong, Li Dongbo, Wu Shaofeng. 2022. Water quality anomaly data detection based on parallel deep Isolation forest algorithm, Mechanical design and manufacturing engineering, vol.51, no.8, 83-88.Google Scholar
- Rajasegarar S, Bezdek J C, Leckie C, 2010. Elliptical Anomalies inWireless Sensor Networks, ACM Transactions on Sensor Networks, vol.6, no.1, 1-28. https://doi.org/10.1145/1653760.1653767Google ScholarDigital Library
- Yi X, Zheng Y, Zhang J, 2016. ST-MVL: Filling Missing Values in Geo-Sensory Time Series Data, The Twenty-Fifth International Joint Conference on Artificial Intelligence, 2704-2710.Google Scholar
- Tang Haixian, Li Guanghui. 2021. Semi-supervised online anomaly detection algorithm for sensor data stream based on C-LSTM, Journal of Sensor Technology, vol.34, no.3, 330-339.Google Scholar
Index Terms
- Research on Machine Learning Algorithm-Based Approach for Detecting Abnormal Data from Environmental Sensors in Different Dimensions
Recommendations
Research on detection method of abnormal capital transfer in electronic commerce based on machine learning
In order to overcome the problems of long detection time, low detection efficiency and high false alarm rate, a new method based on machine learning is proposed. Data mining in e-commerce platform. The improved k-means algorithm was used to cluster the ...
Probabilistic distance based abnormal pattern detection in uncertain series data
Abnormal pattern detection is an important task in series data anomaly detection. Because of the noise interference, the accuracy of abnormal detection method based on deterministic value is decreased. Whereas, most recent studies aimed at solving the ...
The research of Wireless Edge Computing Gateway with Anomaly Detection
AbstractTo the superiority of low cost and convenience, large-scale Internet of things has been deployed for various monitoring pervasively. Typically cloud platform is utilized as a remote data and control center. However, it is a challenge to the ...
Comments