Skip to main content
SpringerLink
Log in

A Soft Sensor with Light and Efficient Multi-scale Feature Method for Multiple Sampling Rates in Industrial Processing

  • Research Article
  • Published:
Machine Intelligence Research Aims and scope Submit manuscript

Abstract

In industrial process control systems, there is overwhelming evidence corroborating the notion that economic or technical limitations result in some key variables that are very difficult to measure online. The data-driven soft sensor is an effective solution because it provides a reliable and stable online estimation of such variables. This paper employs a deep neural network with multiscale feature extraction layers to build soft sensors, which are applied to the benchmarked Tennessee-Eastman process (TEP) and a real wind farm case. The comparison of modelling results demonstrates that the multiscale feature extraction layers have the following advantages over other methods. First, the multiscale feature extraction layers significantly reduce the number of parameters compared to the other deep neural networks. Second, the multiscale feature extraction layers can powerfully extract dataset characteristics. Finally, the multiscale feature extraction layers with fully considered historical measurements can contain richer useful information and improved representation compared to traditional data-driven models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. T. Tham, G. A. Montague, A. J. Morris, P. A. Lant. Soft-sensors for process estimation and inferential control. Journal of Process Control, vol. 1, no. 1, pp. 3–14, 1991. DOI: https://doi.org/10.1016/0959-1524(91)87002-F.

    Article  CAS  Google Scholar 

  2. Y. C. Jiang, S. Yin, J. W. Dong, O. Kaynak. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, vol. 21, no. 11, pp. 12868–12881, 2021. DOI: https://doi.org/10.1109/JSEN.2020.3033153.

    Article  ADS  Google Scholar 

  3. C. Shang, F. Yang, D. X. Huang, W. X. Lyu. Data-driven soft sensor development based on deep learning technique. Journal of Process Control, vol. 24, no. 3, pp. 223–233, 2014. DOI: https://doi.org/10.1016/j.jprocont.2014.01.012.

    Article  CAS  Google Scholar 

  4. H. Haimi, M. Mulas, F. Corona, R. Vahala. Data-derived soft-sensors for biological wastewater treatment plants: An overview. Environmental Modelling & Software, vol. 47, pp. 88–107, 2013. DOI: https://doi.org/10.1016/j.envsoft.2013.05.009.

    Article  Google Scholar 

  5. F. Curreri, S. Graziani, M. G. Xibilia. Input selection methods for data-driven Soft sensors design: Application to an industrial process. Information Sciences, vol. 537, pp. 1–17, 2020. DOI: https://doi.org/10.1016/j.ins.2020.05.028.

    Article  MathSciNet  Google Scholar 

  6. J. J. Su, D. Z. Wang, Y. N. Zhang, F. Yang, Y. Zhao, X. K. Pang. Capturing causality for fault diagnosis based on multi-valued alarm series using transfer entropy. Entropy, vol. 19, no. 12, Article number 663, 2017. DOI: https://doi.org/10.3390/e19120663.

  7. P. Kadlec, B. Gabrys, S. Strandt. Data-driven soft sensors in the process industry. Computers & Chemical Engineering, vol. 33, no. 4, pp. 795–814, 2009. DOI: https://doi.org/10.1016/j.compchemeng.2008.12.012.

    Article  CAS  Google Scholar 

  8. X. F. Yuan, Y. J. Gu, Y. L. Wang, C. H. Yang, W. H. Gui. A deep supervised learning framework for data-driven soft sensor modeling of industrial processes. IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4737–4746, 2020. DOI: https://doi.org/10.1109/TNNLS.2019.2957366.

    Article  PubMed  Google Scholar 

  9. Z. Liu, T. Y. Chai, W. Yu, J. Tang. Multi-frequency signal modeling using empirical mode decomposition and PCA with application to mill load estimation. Neurocomputing, vol. 169, pp. 392–402, 2015. DOI: https://doi.org/10.1016/j.neucom.2014.08.087.

    Article  Google Scholar 

  10. J. V. Kresta, T. E. Marlin, J. F. MacGregor. Development of inferential process models using PLS. Computers & Chemical Engineering, vol. 18, no. 7, pp. 597–611, 1994. DOI: https://doi.org/10.1016/0098-1354(93)E0006-U.

    Article  CAS  Google Scholar 

  11. D. A. Pisner, D. M. Schnyer. Support vector machine. Machine Learning: Methods and Applications to Brain Disorders, A. Mechelli, S. Vieira, Eds., London, UK: Academic Press, pp.101–121, 2020. DOI: https://doi.org/10.1016//B788-0-22-815739-8.00006-7.

    Google Scholar 

  12. M. Aljanabi, M. Shkoukani, M. Hijjawi. Ground-level ozone prediction using machine learning techniques: A case study in Amman, Jordan. International Journal of Automation and Computing, vol. 17, no. 5, pp. 667–677, 2020. DOI: https://doi.org/10.1007/s11633-020-1233-4.

    Article  Google Scholar 

  13. Y. T. Zhao, B. C. Ding, Y. L. Zhang, L. M. Yang, X. C. Hao. Online cement clinker quality monitoring: A soft sensor model based on multivariate time series analysis and CNN. ISA Transactions, vol. 117, pp. 180–195, 2021. DOI: https://doi.org/10.1016/j.isatra.2021.01.058.

    Article  PubMed  Google Scholar 

  14. J. Zhang, K. F. Man. Time series prediction using RNN in multi-dimension embedding phase space. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, pp. 1868–1873, 1998. DOI: https://doi.org/10.1109/ICSMC.1998.728168.

  15. Q. Liu, M. W. Jia, Z. L. Gao, L. F. Xu, Y. Liu. Correntropy long short term memory soft sensor for quality prediction in industrial polyethylene process Chemometrics and Intelligent Laboratory Systems, vol. 231, Article number 104678, 2022. DOI: https://doi.org/10.1016/j.chemolab.2022.104678.

  16. Z. H. Liu, X. D. Meng, H. L. Wei, L. Chen, B. L. Lu, Z. H. Wang, L. Chen. A regularized LSTM method for predicting remaining useful life of rolling bearings. International Journal of Automation and Computing, vol. 18, no. 4, pp. 581–593, 2021. DOI: https://doi.org/10.1007/s11633-020-1276-6.

    Article  Google Scholar 

  17. X. C. Li, X. F. Ma, F. C. Xiao, C. Xiao, F. Wang, S. C. Zhang. Time-series production forecasting method based on the integration of bidirectional gated recurrent unit (Bi-GRU) network and sparrow search algorithm (SSA). Journal of Petroleum Science and Engineering, vol. 208, Article number 109309, 2022. DOI: https://doi.org/10.1016/j.petrol.2021.109309.

  18. G. Lai, W. C. Chang, Y. Yang, et al. Modeling long- and short-term temporal patterns with deep neural networks. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, USA, pp. 95–104, 2018. DOI: https://doi.org/10.1145/3209978.3210006.

  19. X. J. Shi, Z. R. Chen, H. Wang, D. Y. Yeung, W. K. Wong, E. C. Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, Canada, vol. 1, pp. 802–810, 2015.

    Google Scholar 

  20. Y. T. Xiao, H. S. Yin, Y. D. Zhang, H. G. Qi, Y. D. Zhang, Z. Y. Liu. A dual-stage attention-based Conv-LSTM network for spatio-temporal correlation and multivariate time series prediction. International Journal of Intelligent Systems, vol. 36, no. 5, pp. 2036–2057, 2021. DOI: https://doi.org/10.1002/int.22370.

    Article  Google Scholar 

  21. P. Hewage, A. Behera, M. Trovati, E. Pereira, M. Ghahremani, F. Palmieri, Y. H. Liu. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, vol. 24, no. 21, pp. 16453–16482, 2020. DOI: https://doi.org/10.1007/s00500-020-04954-0.

    Article  Google Scholar 

  22. B. H. Mevik, H. R. Cederkvist. Mean squared error of prediction (MSEP) estimates for principal component regression (PCR) and partial least squares regression (PLSR)[J]. Journal of Chemometrics. Journal of Chemometrics, vol. 18, no. 9, pp. 422–429, 2004. DOI: https://doi.org/10.1002/cem.887.

    Article  CAS  Google Scholar 

  23. J. H. Zheng, Z. H. Song. Semisupervised learning for probabilistic partial least squares regression model and soft sensor application. Journal of Process Control, vol. 64, pp. 123–131, 2018. DOI: https://doi.org/10.1016/j.jprocont.2018.01.008.

    Article  CAS  Google Scholar 

  24. H. M. Zhao, J. J. Zheng, J. J. Xu, W. Deng. Fault diagnosis method based on principal component analysis and broad learning system. IEEE Access, vol. 7, pp. 99263–99272, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2929094.

    Article  Google Scholar 

  25. J. Tang, T. Y. Chai, L. J. Zhao, W. Yu, H. Yue. Soft sensor for parameters of mill load based on multi-spectral segments PLS sub-models and on-line adaptive weighted fusion algorithm. Neurocomputing, vol. 78, no. 1, pp. 38–47, 2012. DOI: https://doi.org/10.1016/j.neucom.2011.05.028.

    Article  Google Scholar 

  26. W. W. Yan, H. H. Shao, X. F. Wang. Soft sensing modeling based on support vector machine and Bayesian model selection. Computers & Chemical Engineering, vol. 28, no. 8, pp. 1489–1498, 2004. DOI: https://doi.org/10.1016/j.compchemeng.2003.11.004.

    Article  CAS  Google Scholar 

  27. C. Shang, X. Gao, F. Yang, D. Huang. Novel Bayesian framework for dynamic soft sensor based on support vector machine with finite impulse response. IEEE Transactions on Control Systems Technology, vol. 22, no. 4, pp. 1550–1557, 2014. DOI: https://doi.org/10.1109/TCST.2013.2278412.

    Article  Google Scholar 

  28. X. F. Yuan, C. Ou, Y. L. Wang, C. H. Yang, W. H. Gui. Deep quality-related feature extraction for soft sensing modeling: A deep learning approach with hybrid VW-SAE. Neurocomputing, vol. 396, pp. 375–382, 2020. DOI: https://doi.org/10.1016/j.neucom.2018.11.107.

    Article  Google Scholar 

  29. G. M. Wang, Q. S. Jia, M. C. Zhou, J. Bi, J. F. Qiao. Soft-sensing of wastewater treatment process via deep belief network with event-triggered learning. Neurocomputing, vol. 436, pp. 103–113, 2021. DOI: https://doi.org/10.1016/j.neucom.2020.12.108.

    Article  Google Scholar 

  30. K. Hornik, M. Stinchcombe, H. White. Multilayer feedforward networks are universal approximators. Neural Networks, vol. 2, no. 5, pp. 359–366, 1989. DOI: https://doi.org/10.1016/0893-6080(89)90020-8.

    Article  Google Scholar 

  31. Z. Lu, H. M. Pu, F. C. Wang, Z. Q. Hu, L. W. Wang. The expressive power of neural networks: A view from the width. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, pp. 6232–6240, 2017.

  32. X. Wang, H. Liu. Data supplement for a soft sensor using a new generative model based on a variational autoencoder and Wasserstein GAN. Journal of Process Control, vol. 85, pp. 91–99, 2020. DOI: https://doi.org/10.1016/j.jprocont.2019.11.004.

    Article  CAS  Google Scholar 

  33. Y. Liu, C. Yang, Z. L. Gao, Y. Yao. Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes. Chemometrics and Intelligent Laboratory Systems, vol. 174, pp. 15–21, 2018. DOI: https://doi.org/10.1016/j.chemolab.2018.01.008.

    Article  CAS  Google Scholar 

  34. W. Cheng, Y. Wang, Z. Peng, X. D. Ren, Y. B. Shuai, S. Y. Zang, H. Liu, H. Cheng, J. G. Wu. High-efficiency chaotic time series prediction based on time convolution neural network. Chaos, Solitons & Fractals, vol. 152, Article number 111304, 2021. DOI: https://doi.org/10.1016/j.chaos.2021.111304.

  35. S. Gao, Y. Dai, Y. J. Li, Y. X. Jiang, Y. Liu. Augmented flame image soft sensor for combustion oxygen content prediction. Measurement Science and Technology, vol. 34, no. 1, Article number 15401, 2023. DOI: https://doi.org/10.1088/1361-6501/ac95b5.

  36. A. N. Shiryaev. Mathematical foundations of probability theory. Probability-1, A. N. Shiryaev, Ed., New York, USA: Springer, pp. 159–371, 2016. DOI: https://doi.org/10.1007/978-0-387-72206-1_2.

    Chapter  Google Scholar 

  37. M. O. Alassafi, M. Jarrah, R. Alotaibi. Time series predicting of COVID-19 based on deep learning. Neurocomputing, vol. 468, pp. 335–344, 2022. DOI: https://doi.org/10.1016/j.neucom.2021.10.035.

    Article  PubMed  Google Scholar 

  38. X. Zhang, Y. Y. Zou, S. Y. Li, S. H. Xu. A weighted auto regressive LSTM based approach for chemical processes modeling. Neurocomputing, vol. 367, pp. 64–74, 2019. DOI: https://doi.org/10.1016/j.neucom.2019.08.006.

    Article  Google Scholar 

  39. M. H. Liu, A. L. Zeng, M. X. Chen, Z. J. Xu, Q. X. Lai, L. N. Ma, Q. Xu. SCINet: Time Series Modeling and forecasting with sample convolution and interaction. [Online], Available: https://arxiv.org/abs/2106.09305, 2021. DOI: https://doi.org/10.48550/arXiv.2106.09305.

  40. E. Fu, Y. N. Zhang, F. Yang, S. Y. Wang. Temporal self-attention-based Conv-LSTM network for multivariate time series prediction. Neurocomputing, vol. 501, pp. 162–173, 2022. DOI: https://doi.org/10.1016/j.neucom.2022.06.014.

    Article  Google Scholar 

  41. A. N. Shiryaev. Probability-1, New York, USA: Springer, 2016. DOI: https://doi.org/10.1007/978-0-387-72206-1.

    Book  Google Scholar 

  42. C. Guo, W. K. Hu, F. Yang, D. X. Huang. Deep learning technique for process fault detection and diagnosis in the presence of incomplete data. Chinese Journal of Chemical Engineering, vol. 28, no. 9, pp. 2358–2367, 2020. DOI: https://doi.org/10.1016/j.cjche.2020.06.015.

    Article  CAS  Google Scholar 

  43. B. Hanin, M. Sellke. Approximating continuous functions by ReLU nets of minimal width. [Online], Available: https://arxiv.org/abs/1710.11278, 2017.

  44. J. F. Kolen, S. C. Kremer. Gradient flow in recurrent nets: The difficulty of learning LongTerm dependencies. A Field Guide to Dynamical Recurrent Networks, J. F. Kolen, S. C. Kremer, Eds., New York, USA: IEEE Press, pp. 237–243, 2001. DOI: https://doi.org/10.1109/9780470544037.ch14.

    Google Scholar 

  45. X. Glorot, A. Bordes, Y. Bengio. Deep sparse rectifier neural networks. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, pp. 315–323, 2011.

  46. D. P. Kingma, J. Ba. Adam: A method for stochastic optimization. [Online], Available: https://arxiv.org/abs/1412.6980, 2014.

  47. J. J. Downs, E. F. Vogel. A plant-wide industrial process control problem. Computer & Chemical Engineering, vol. 17, no. 3, pp. 245–255, 1993. DOI: https://doi.org/10.1016/0098-1354(93)80018-I.

    Article  CAS  Google Scholar 

  48. N. L. Ricker. Decentralized control of the Tennessee Eastman challenge process. Journal of Process Control, vol. 6, no. 4, pp. 205–221, 1996. DOI: https://doi.org/10.1016/0959-1524(96)00031-5.

    Article  ADS  CAS  Google Scholar 

  49. F. Yang, C. Guo, G. Y. Ke, D. Z. Wang, H. Ye. Efficiency improvement performance assessment based on partial least squares for wind generation turbine. Process Automation Instrumentation, vol. 40, no. 10, pp. 29–34, 2019. DOI: https://doi.org/10.16086/j.cnki.issn1000-0380.2018120056. (in Chinese)

    Google Scholar 

  50. P. H. Borghi, O. Zakordonets, J. P. Teixeira. A COVID-19 time series forecasting model based on MLP ANN. Procedia Computer Science, vol. 181, pp. 940–947, 2021. DOI: https://doi.org/10.1016/j.procs.2021.01.250.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 61873142), the Science and Technology Research Program of the Chongqing Municipal Education Commission, China (Nos. KJZD-K20220 1901, KJQN202201109, KJQN202101904, KJQN20200 1903 and CXQT21035), the Scientific Research Foundation of Chongqing University of Technology, China (No. 2019ZD76), the Scientific Research Foundation of Chongqing Institute of Engineering, China (No. 2020xzky05), and the Chongqing Municipal Natural Science Foundation, China (No. cstc2020jcyj-msxmX0666).

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Dezheng Wang

  2. Software and Artificial Intelligence College, Chongqing Institute of Engineering, Chongqing, 400056, China

    Yinglong Wang, Yiran Chen & Ning Liao

  3. Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University, Beijing, 100084, China

    Fan Yang

  4. Liangjiang International College, Chongqing University of Technology, Chongqing, 401135, China

    Liyang Xu

  5. Smart City College, Beijing Union University, Beijing, 100101, China

    Yinong Zhang

Authors
  1. Dezheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinglong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liyang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yinong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yiran Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ning Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liyang Xu.

Ethics declarations

The authors declared that they have no conflicts of interest to this work.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Dezheng Wang received the M. Sc. degree in software engineering from Beijing Union University, China in 2018. He is currently a Ph. D. degree candidate with School of Automation, Southeast University, China. After working as an engineering with SAIC General Motors (SGM), he joined College of Big Data and Artificial intelligence, Chongqing Institute of Engineering, China from 2019 to 2022, where he was a lecturer.

His research interests include fault diagnosis, process data analytics and pattern recognition.

Yinglong Wang received the B. Sc. degree in electronic information science and technology from Hubei Normal University, China in 2010, and the M. Sc. degree in computational mathematics from Chongqing University, China in 2014. After working as an engineer with ASM Pacific Technology Ltd. (ASMPT), he joined College of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, China in 2018.

His research interests include computational mathematics and image processing.

Fan Yang received the B. Eng. degree in automation and the Ph. D. degree in control science and engineering from Tsinghua University, China in 2002 and 2008, respectively. After working as a postdoctoral fellow at University of Alberta, he joined the Department of Automation, Tsinghua University, China in 2011, where he is currently a professor. He was a recipient of the Young Research Paper Award from the IEEE Control Systems Society Beijing Chapter in 2006, the Science and Technology Progress Award from the Chinese Association of Automation in 2018, the Zhang Zhong Jun’s Excellent Paper Award in 2019, and the Teaching Achievement Awards from Tsinghua University in 2012, 2014 and 2016, and from the Chinese Association of Automation in 2016.

His research interests include topology modelling of large-scale processes, abnormal event monitoring, process hazard analysis and smart alarm management.

Liyang Xu received the B. Eng. and Ph. D. degrees in materials processing engineering from Sichuan University, China in 2013 and 2018, respectively. She joined Liang Jiang International College, Chongqing University of Technology, China in 2018.

Her research interests include topology modelling of large-scale processes, machine learning and polymer functional film.

Yinong Zhang received the B. Sc. degree in electrical automation from Inner Mongolia University of Technology, China in 1990, and the M. Sc. degree in automation from Tsinghua University, China in 2004. She served as an associate professor at Smart City College, Beijing Union University, China in 2004 and was promoted to professor in 2014. She has published more than 20 academic papers, completed more than 30 scientific research projects, and obtained 15 patents.

Her research interests include complex process modelling, intelligent control, fault diagnosis and safety evaluation.

Yiran Chen received the B. Sc. and M. Sc. degrees in computer science and technology from College of Computer and Information Engineering, Chongqing University, China in 2007 and 2015, respectively. She is currently a Ph. D. degree candidate in computer science at Artificial Intelligence, Chongqing University, China. Meanwhile, she is an associate professor at College of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, China.

Her research interests include computational intelligence, deep learning and computer vision.

Ning Liao received the M. Sc. degree in computational chemistry from Peking University, China in 2000, and studied at State College, Pennsylvania State University, USA, from 2000 to 2005. From 2005 to 2011, he was a research assistant in the field of high-performance computing of protein-environment interaction with the Quantum Theory Project, University of Florida, USA. In 2012, he joined the Chongqing Institute of Engineering, China. Now he is an associate professor and vice dean of College of Big Data and Artificial Intelligence, China. He is the author of a book of Machine Learning and more than 20 articles.

His research interests include big data algorithms and applications in commercial credit investigation, computer vision based on deep learning, and intelligent computing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Wang, Y., Yang, F. et al. A Soft Sensor with Light and Efficient Multi-scale Feature Method for Multiple Sampling Rates in Industrial Processing. Mach. Intell. Res. 21, 400–410 (2024). https://doi.org/10.1007/s11633-022-1401-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-022-1401-9

Keywords

Search

Navigation