Abstract
The manufacturing industry has been undergoing a paradigm shift toward the concept of a smart factory. To stay abreast of this paradigm shift, extensive research, particularly in the chemical engineering manufacturing field, has been focused on analyzing newly acquired image data. Consequently, this study proposes a novel method to analyze the nozzle spray patterns of feedstock oil inside a carbon black reactor by analyzing images acquired from a machine vision system. To replace conventional methods making use of naked eye measurements, the images inside a reactor were acquired and processed using three different methods. Several models to detect irregular nozzle spray patterns in processed images have been developed through transfer learning. We combined these individual models to develop an ensemble model that exhibited better performance than the individual models. The effect of the ensemble was verified through gradient-weighted class activation mapping analysis. Using the proposed ensemble model, a test dataset accuracy of 98.5% was obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams, S., Crannell, G., Bolcavage, A., McIntyre, R., & Beling, P. A. (2018). A condition monitoring system for low vacuum plasma spray using computer vision. In IEEE International Conference on Prognostics and Health Management, Seattle, WA, USA, pp. 1–7. https://doi.org/10.1109/ICPHM.2018.8448464
Aminzadeh, M., & Kurfess, T. R. (2018). Online quality inspection using Bayesian classification in powder-bed additive manufacturing from high-resolution visual camera images. Journal of Intelligent Manufacturing, 30(6), 2505–2523. https://doi.org/10.1007/s10845-018-1412-0
Badmos, O., Kopp, A., Bernthaler, T., & Schneider, G. (2019). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing, 31(4), 885–897. https://doi.org/10.1007/s10845-019-01484-x
Bermejo-Pelaez, D., Ash, S. Y., Washko, G. R., Jose Estepar, S., R., & Ledesma-Carbayo, M. J. (2020). Classification of interstitial lung abnormality patterns with an ensemble of deep convolutional neural networks. Scientific Reports, 10(1), 338. https://doi.org/10.1038/s41598-019-56989-5
Bustillo, A., Reis, R., Machado, A. R., & Pimenov, D. Y. (2020). Improving the accuracy of machine-learning models with data from machine test repetitions. Journal of Intelligent Manufacturing, 33(1), 203–221. https://doi.org/10.1007/s10845-020-01661-3
Casperson, J. R. (1982). Feedstock nozzle and use in carbon black reactor. US Patent No. 4360497.
Cha, D., Pae, C., Seong, S. B., Choi, J. Y., & Park, H. J. (2019). Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database. EBioMedicine, 45, 606–614. https://doi.org/10.1016/j.ebiom.2019.06.050
Cheng, P. J., & Mills, K. L. (1982). Method for producing carbon black.US Patent No.4328199.
Chengli, Z., Xuezhang, H., & Zhiqiang, S. (2011). Application research on digital image technology in the measurement of nozzle spray cone angle. In IEEE International Instrumentation and Measurement Technology Conference, Hangzhou, China, pp. 1–5. https://doi.org/10.1109/IMTC.2011.5944077
Chiang, L., Lu, B., & Castillo, I. (2017). Big data analytics in chemical engineering. Annual Review of Chemical and Biomolecular, 8, 63–85. https://doi.org/10.1146/annurev-chembioeng-060816-101555
Deng, J., Dong, W., Socher, R., Li, L., Kai, L., & Li, F. F. (2009). ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2019). Artificial intelligence for decision making in the era of Big Data – evolution, challenges and research agenda. International Journal of Information Management, 48, 63–71. https://doi.org/10.1016/j.ijinfomgt.2019.01.021
Ekbal, A., & Saha, S. (2011). Weighted vote-based classifier ensemble for named entity recognition. ACM Transactions on Asian Language Information Processing, 10(2), 9. https://doi.org/10.1145/1967293.1967296
Fan, Y., Fowler, G. D., & Zhao, M. (2020). The past, present and future of carbon black as a rubber reinforcing filler – A review. Journal of Cleaner Production, 247, 119115. https://doi.org/10.1016/j.jclepro.2019.119115
Farahani, B. V., Barros, F., Sousa, P. J., Tavares, P. J., & Moreira, P. M. G. P. (2020). A railway tunnel structural monitoring methodology proposal for predictive maintenance. Structural Control and Health Monitoring, 27(8), e2587. https://doi.org/10.1002/stc.2587
Ghafoor, M., Bajwa, U. I., & Taj, I. A. (2011). Fuel nozzle spray pattern classifier. In Frontiers of Information Technology, Islamabad, Pakistan, pp. 303–307. https://doi.org/10.1109/fit.2011.63
Gokalp, O., & Tasci, E. (2019). Weighted voting based ensemble classification with hyper-parameter optimization. In Innovations in Intelligent Systems and Applications Conference, pp. 1–4. https://doi.org/10.1109/ASYU48272.2019.8946373
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
Hsu, C. Y., & Chien, J. C. (2020). Ensemble convolutional neural networks with weighted majority for wafer bin map pattern classification. Journal of Intelligent Manufacturing, 33(3), 831–844. https://doi.org/10.1007/s10845-020-01687-7
Huang, G., Liu, Z., Maaten, L. V. D., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv preprint arXiv:1602.07360.
Javadi, A., Soltanieh, M., Sahebdelfar, S., Bastani, D., & Javadi, K. (2006). Estimation of temperature and residence time of carbon black oil furnace industrial reactors. In The ASME International Mechanical Engineering Congress and Exposition, Chicago, Illinois, USA, pp. 69–77. https://doi.org/10.1115/imece2006-15671
Kang, D., Gweon, H. M., Eun, N. L., Youk, J. H., Kim, J. A., & Son, E. J. (2021). A convolutional deep learning model for improving mammographic breast-microcalcification diagnosis. Scientific Reports, 11(1), 23925. https://doi.org/10.1038/s41598-021-03516-0
Kausar, A., Ishtiaq, M., Jaffar, M. A., & Mirza, A. M. (2010). Optimization of ensemble based decision using PSO. In The World Congress on Engineering, London, pp. 671–676.
Keys, R. (1981). Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 29(6), 1153–1160. https://doi.org/10.1109/TASSP.1981.1163711
Kim, M., Lee, M., An, M., & Lee, H. (2019). Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. Journal of Intelligent Manufacturing, 31(5), 1165–1174. https://doi.org/10.1007/s10845-019-01502-y
Krishnamurthy, V., Nezafati, K., & Singh, V. (2019). Application of machine learning and spatial bootstrapping to image processing for predictive maintenance. In IEEE International Conference on Big Data, pp. 4395–4401. https://doi.org/10.1109/BigData47090.2019.9006439
Lee, G. Y., Kim, M., Quan, Y. J., Kim, M. S., Kim, T. J. Y., Yoon, H. S., et al. (2018). Machine health management in smart factory: A review. Journal of Mechanical Science and Technology, 32(3), 987–1009. https://doi.org/10.1007/s12206-018-0201-1
Li, H., Cryer, S., Acharya, L., & Raymond, J. (2020). Video and image classification using atomisation spray image patterns and deep learning. Biosystems Engineering, 200, 13–22. https://doi.org/10.1016/j.biosystemseng.2020.08.016
Lins, R. G., de Araujo, P. R. M., & Corazzim, M. (2020). In-process machine vision monitoring of tool wear for Cyber-Physical Production Systems. Robotics and Computer-Integrated Manufacturing, 61, 101859. https://doi.org/10.1016/j.rcim.2019.101859
Longadge, R., Dongre, S., & Latesch, M. (2013). Class imbalance problem in data mining review. arXiv preprint arXiv:1305.1707.
MATLAB (2020). MATLAB, 2020. version 9.8.0 (R2020a Update 6), Natick, Massachusetts: The MathWorks Inc.
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015). Deep learning applications and challenges in big data analytics. Journal of Big Data, 2(1), 1. https://doi.org/10.1186/s40537-014-0007-7
Niu, S., Chen, Q., de Sisternes, L., Ji, Z., Zhou, Z., & Rubin, D. L. (2017). Robust noise region-based active contour model via local similarity factor for image segmentation. Pattern Recognition, 61, 104–119. https://doi.org/10.1016/j.patcog.2016.07.022
Oh, S. M., Park, J., Yang, J., Oh, Y. G., & Yi, K. W. (2021). Image processing for analysis of carbon black pellet size distribution during pelletizing. Measurement, 174, 108963. https://doi.org/10.1016/j.measurement.2021.108963
O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., et al. (2019). Deep learning vs. Traditional computer vision. In The Computer Vision Conference, pp. 128–144. https://doi.org/10.1007/978-3-030-17795-9_10
Ono, K., Yanaka, M., Tanaka, S., Saito, Y., Aoki, H., Fukuda, O., et al. (2012). Influence of furnace temperature and residence time on configurations of carbon black. Chemical Engineering Journal, 200–202, 541–548. https://doi.org/10.1016/j.cej.2012.06.061
Panwar, H., Gupta, P. K., Siddiqui, M. K., Morales-Menendez, R., Bhardwaj, P., & Singh, V. (2020). A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images. Chaos Solitons Fractals, 140, 110190. https://doi.org/10.1016/j.chaos.2020.110190
Prati, R. C., Batista, G. E. A. P. A., & Silva, D. F. (2014). Class imbalance revisited: A new experimental setup to assess the performance of treatment methods. Knowledge and Information Systems, 45(1), 247–270. https://doi.org/10.1007/s10115-014-0794-3
Rokach, L. (2009). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), 1–39. https://doi.org/10.1007/s10462-009-9124-7
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. https://doi.org/10.1109/CVPR.2018.00474
Scime, L., & Beuth, J. (2018). Anomaly detection and classification in a laser powder bed additive manufacturing process using a trained computer vision algorithm. Additive Manufacturing, 19, 114–126. https://doi.org/10.1016/j.addma.2017.11.009
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision, pp. 618–626. https://doi.org/10.1109/ICCV.2017.74
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556
Singh, R. K., Pandey, R., & Babu, R. N. (2021). COVIDScreen: Explainable deep learning framework for differential diagnosis of COVID-19 using chest X-rays. Neural Computing and Applications. https://doi.org/10.1007/s00521-020-05636-6
Smith, C. M., & Hoehle, M. S. (2018). Imaging through fire using narrow-spectrum illumination. Fire Technology, 54, 1705–1723. https://doi.org/10.1007/s10694-018-0756-5
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826. https://doi.org/10.1109/CVPR.2016.308
Tasci, E. (2020). Voting combinations-based ensemble of fine-tuned convolutional neural networks for food image recognition. Multimedia Tools and Applications, 79(41–42), 30397–30418. https://doi.org/10.1007/s11042-020-09486-1
Thabtah, F., Hammoud, S., Kamalov, F., & Gonsalves, A. (2020). Data imbalance in classification: Experimental evaluation. Information Sciences, 513, 429–441. https://doi.org/10.1016/j.ins.2019.11.004
Truong, L. P., Pham, B. D., & Vu, Q. H. (2020). A mobile deep convolutional neural network combined with grad-CAM visual explanations for real time tomato quality classification system. In 5th International Conference on Green Technology and Sustainable Development, pp. 321–325. https://doi.org/10.1109/GTSD50082.2020.9303079
Vulgarakis-Minov, S., Cointault, F., Vangeyte, J., Pieters, J., & Nuyttens, D. (2014). Spray nozzle characterization using a backlighted high speed imaging technique. Aspects of Applied Biology, 122, 353–361.
Waltz, R. A., Morales, J. L., Nocedal, J., & Orban, D. (2005). An interior algorithm for nonlinear optimization that combines line search and trust region steps. Mathematical Programming, 107(3), 391–408. https://doi.org/10.1007/s10107-004-0560-5
Xia, C., Pan, Z., Fei, Z., Zhang, S., & Li, H. (2020). Vision based defects detection for Keyhole TIG welding using deep learning with visual explanation. Journal of Manufacturing Processes, 56, 845–855. https://doi.org/10.1016/j.jmapro.2020.05.033.
Xue, D., Zhou, X., Li, C., Yao, Y., Rahaman, M. M., Zhang, J., et al. (2020). An application of transfer learning and ensemble learning techniques for cervical histopathology image classification. IEEE Access, 8, 104603–104618. https://doi.org/10.1109/access.2020.2999816
Yang, X., Han, M., Tang, H., Li, Q., & Luo, X. (2020). Detecting defects with support vector machine in logistics packaging boxes for edge computing. IEEE Access, 8, 64002–64010. https://doi.org/10.1109/access.2020.2984539
Yarpiz (2022). Differential evolution (DE) (https://www.mathworks.com/matlabcentral/fileexchange/52897-differential-evolution-de), MATLAB Central File Exchange. Retrieved January 5, 2022.
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018). ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6848–6856. https://doi.org/10.1109/CVPR.2018.00716
Zhang, Y., Zhang, H., Cai, J., & Yang, B. (2014). A weighted voting classifier based on differential evolution. Abstract and Applied Analysis, 2014, 1–6. https://doi.org/10.1155/2014/376950
Zhou, Q., Chen, R., Huang, B., Liu, C., Yu, J., & Yu, X. (2019). An automatic surface defect inspection system for automobiles using machine vision methods. Sensors, 19(3), 644. https://doi.org/10.3390/s19030644
Acknowledgements
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Oh, SM., Park, J., Yang, J. et al. Smart classification method to detect irregular nozzle spray patterns inside carbon black reactor using ensemble transfer learning. J Intell Manuf 34, 2729–2745 (2023). https://doi.org/10.1007/s10845-022-01951-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-01951-y