Skip to main content
SpringerLink
Log in

Multiscale fire image detection method based on CNN and Transformer

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fire is one of the most harmful hazards that affect daily life. The existing fire detection methods have the problems of large computation, slow detection speed, and low detection accuracy to varying degrees, and do not achieve a better trade-off between model complexity, accuracy, and detection speed. In this paper, a multiscale fire image detection method combining Convolutional Neural Network(CNN) and Transformer is proposed. In the shallow layer of the model, the CNN-based multiscale feature extraction module is used to obtain rich fire image information. In the deep layers of the model, the powerful global learning ability of the Transformer is used to carry out overall perception and macroscopic understanding of images. The experimental results show that the best detection accuracy of the model can reach 94.62%, and the fastest detection speed can reach 158.12FPS, F1 score is stable at around 94%, which is fully capable of real-time and accurate detection of fire. Compared with the existing detection methods, this method has higher detection accuracy under similar model complexity and detection speed. With similar detection accuracy, our method has a faster detection speed. The proposed method achieves a better balance between model complexity, detection speed, and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author on reasonable request. Specific sources of the datasets used in this article:

Dataset-1 in https://mivia.unisa.it/datasets/video-analysis-datasets/fire-detection-dataset/.

Dataset-2 in https://cvpr.kmu.ac.kr/.

Dataset-3 mainly sourced in https://www.kaggle.com/datasets/phylake1337/fire-datasethttps://www.kaggle.com/datasets/atulyakumar98/test-dataset and https://www.kaggle.com/datasets/studioai/fire1500.

Our dataset has been uploaded to Kaggle https://www.kaggle.com/datasets/firedetect/fire-dataset-10000 and will be open-sourced in due course.

References

  1. National Fire and Rescue Administration (2023), Retrieved August 2023 from: https://www.119.gov.cn/qmxfxw/xfyw/2023/36210.shtml

  2. Chen TH, Wu PH, Chiou YC (2004) An early fire-detection method based on image processing. In: 2004 International Conference on Image Processing (ICIP), pp 1707–1710

  3. Marbach G, Loepfe M, Brupbacher T (2006) An image processing technique for fire detection in video images. Fire Saf 41(4):285–289

    Article  Google Scholar 

  4. Celik T, Demirel H (2009) Fire detection in video sequences using a generic color model. Fire Saf 44(2):147–158

    Article  Google Scholar 

  5. Celik T (2010) Fast and efficient method for fire detection using image processing. ETRI J 32(6):881–890

    Article  Google Scholar 

  6. Celik T, Ozkaramanli H, Demirel H (2007) Fire pixel classification using fuzzy logic and statistical color model. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 1205-1208

  7. Rinsurongkawong S, Ekpanyapong M, Dailey MN (2012) Fire detection for early fire alarm based on optical flow video processing. In: 2012 9th International Conference on electrical engineering/electronics, computer, telecommunications and information technology, pp 1–4

  8. Piri J, Mohapatra P, Pradhan MR et al (2021) A binary multi-objective chimp optimizer with dual archive for feature selection in the healthcare domain. IEEE Access 10:1756–1774

    Article  Google Scholar 

  9. Das H, Chakraborty S, Acharya B et al. (2020) Optimal selection of features using teaching-learning-based optimization algorithm for classification. Appl Intell Decis Making Machine Learn 213–227

  10. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–9

  11. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  12. Muhammad K, Ahmad J, Mehmood I, Rho S, Baik SW (2018) Convolutional neural networks based fire detection in surveillance videos. IEEE Access 6:18174–18183

    Article  Google Scholar 

  13. Muhammad K, Ahmad J, Baik SW (2018) Early Fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288:30–42

    Article  Google Scholar 

  14. Wu H, Xiao B, Codella N et al (2021) Cvt: introducing convolutions to vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp 22–31

  15. Töreyin BU, Dedeoğlu Y, Güdükbay U, Cetin AE (2006) Computer vision-based method for real-time fire and flame detection. Pattern Recognit Lett 27(1):49–58

    Article  Google Scholar 

  16. Ha C, Hwang U, Jeon G, Cho J, Jeong J (2012) Vision-based fire detection algorithm using optical flow. In: 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems, pp 526–530

  17. Chunyu Y, Jun F, Jinjun W, Yongming Z (2010) Video fire smoke detection using motion and color features. Fire Technol 46:651–663

    Article  Google Scholar 

  18. Qiu T, Yan Y, Lu G (2011) An auto adaptive edge-detection algorithm for flame and Fire image processing. IEEE Trans Instrum Meas 61(5):1486–1493

    Article  Google Scholar 

  19. Ko BC, Cheong KH, Nam JY (2009) Fire detection based on vision sensor and support vector machines. Fire Saf J 44(3):322–329

    Article  Google Scholar 

  20. Frizzi S, Kaabi R, Bouchouicha M, Ginoux JM, Moreau E, Fnaiech F (2016) Convolutional neural network for video fire and smoke detection. In: IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, pp 877–882

  21. Mao W, Wang W, Dou Z, Li Y (2018) Fire recognition based on multi-channel convolutional neural network. Fire Technol 54:531–554

    Article  Google Scholar 

  22. Muhammad K, Ahmad J, Lv Z, Bellavista P, Yang P, Baik SW (2018) Efficient deep CNN-based Fire detection and localization in video surveillance applications. IEEE Trans Syst Man Cybern: Syst 49(7):1419–1434

    Article  Google Scholar 

  23. Muhammad K, Khan S, Elhoseny M, Ahmed SH, Baik SW (2019) Efficient fire detection for uncertain surveillance environment. IEEE Trans Industr Inf 15(5):3113–3122

    Article  Google Scholar 

  24. Li S, Yan Q, Liu P (2020) An efficient fire detection method based on multiscale feature extraction, implicit deep supervision and channel attention mechanism. IEEE Trans Image Process 29:8467–8475

    Article  Google Scholar 

  25. Jeon M, Choi HS, Lee J, Kang M (2021) Multi-scale prediction for fire detection using convolutional neural network. Fire Technol 57(5):2533–2551

    Article  Google Scholar 

  26. Ghosh R, Kumar A (2022) A hybrid deep learning model by combining convolutional neural network and recurrent neural network to detect forest fire. Multimed Tools Appl 81(27):38643–38660

    Article  Google Scholar 

  27. Jianxin Z, Siwen G, Guolan Z, Lin T (2021) Fire detection model based on multi-scale feature fusion. J Zhengzhou Univ (Engineering Science) 42(5):13–18

    Google Scholar 

  28. Abdusalomov A, Baratov N, Kutlimuratov A, Whangbo TK (2021) An improvement of the fire detection and classification method using YOLOv3 for surveillance systems. Sensors 21(19):6519

    Article  Google Scholar 

  29. Xu H, Li B, Zhong F (2022) Light-YOLOv5: a Lightweight Algorithm for Improved YOLOv5 in Complex Fire scenarios. Appl Sci 12(23):12312

    Article  Google Scholar 

  30. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929

  31. Li Y, Zhang W, Liu Y, Jing R, Liu C (2022) An efficient fire and smoke detection algorithm based on an end-to-end structured network. Eng Appl Artif Intell 116:105492

    Article  Google Scholar 

  32. Mingliang Y, Huiying Z, Jianjun L (2022) Forest fire detection algorithm based on an improved swin transformer. J Cent South Univ Forestry Technol 42(08):101–110

    Google Scholar 

  33. Cao J, Li Y, Sun M et al (2022) Do-conv: depthwise over-parameterized convolutional layer. IEEE Trans Image Process 31:3726–3736

    Article  Google Scholar 

  34. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 7132–7141

  35. Foggia P, Saggese A, Vento M (2015) Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE Trans Circ Syst Video Technol 25(9):1545–1556

    Article  Google Scholar 

  36. Ko BC, Ham SJ, Nam JY (2011) Modeling and formalization of fuzzy finite automata for detection of irregular Fire flames. IEEE Trans Circ Syst Video Technol 21(12):1903–1912

    Article  Google Scholar 

  37. Gomes P, Santana P, Barata J (2014) A vision-based approach to fire detection. Int J Adv Robot Syst 11(9):1–12

    Article  Google Scholar 

  38. Zhang D, Han S, Zhao J et al (2009) Image based forest fire detection using dynamic characteristics with artificial neural networks. International Joint Conference on Artificial Intelligence. IEEE, pp 290–293

  39. Sharma J, Granmo OC, Goodwin M, Fidje JT (2017) Deep convolutional neural networks for fire detection in images. In: Proceedings of Engineering Applications of Neural Networks: 18th International Conference (EANN), pp 183–193

  40. Park M, Tran DQ, Jung D, Park S (2020) Wildfire-detection method using DenseNet and cycleGAN Data Augmentation-based remote Camera Imagery. Remote Sens 12:3715

    Article  Google Scholar 

  41. Wang W, Xie E, Li X, Fan DP, Song K, Liang, et al (2021) Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp 568–578

  42. Graham B, El-Nouby A, Touvron H, Stock P, Joulin A, Jégou H, Douze M (2021) Levit: a vision transformer in convnet’s clothing for faster inference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp 12259–12269

Download references

Acknowledgements

This research was supported by the Major scientific and technological project of Hubei Province, China [grant number 2021AAA007].

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Wuhan University of Technology, Wuhan, 430070, People’s Republic of China

    Shengbao Wu, Buyun Sheng, Gaocai Fu & Yuchao Jian

  2. School of Mechanical Engineering, Hubei University of Technology, Wuhan, 430068, People’s Republic of China

    Buyun Sheng & Daode Zhang

Authors
  1. Shengbao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Buyun Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gaocai Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daode Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuchao Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaocai Fu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, S., Sheng, B., Fu, G. et al. Multiscale fire image detection method based on CNN and Transformer. Multimed Tools Appl 83, 49787–49811 (2024). https://doi.org/10.1007/s11042-023-17482-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17482-4

Keywords

Search

Navigation