Skip to main content
SpringerLink
Log in

Study on defect detection of metal castings based on supervised enhancement and attention distillation

  • Research
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Metal castings are essential parts and their quality greatly impacts the product’s performance. Current techniques for identifying surface defects in metal castings usually suffer from issues like inadequate accuracy in defect recognition. In comparison to other models, the performance of the Deformable DETR model in detecting defects in metal castings is commendable, yet its attentional mechanism inherently demands a larger sample size and exhibits a slower convergence rate. However, the dataset collected in this paper for metal castings possesses a limited number of samples and imbalanced data, which consequently limits the overall effectiveness of the Deformable DETR model on this task. Responding to the above, a supervised enhancement algorithm for one-to-many assignment based on the Deformable DETR was proposed in this paper. This algorithm accelerates convergence speed and reduces data requirements while eliminating Non-Maximum Suppression post-processing. Applying the supervised enhancement algorithm to DETR on this dataset also improves defect recall. In addition, attention distillation was applied to the deformable attention mechanism to reduce time and space complexity to O(Llog(L)). It creates favourable conditions for the generation of attention weights, which are changed from a linear transformation of query to query-key interaction, and emphasises casting defects while ensuring global attention. Furthermore, data processing methods such as image slicing and data augmentation are also used to enhance casting defect detection ability in this paper. Finally, the recall rate and precision rate of metal casting defects have been improved, increasing to 97.7% and 85.0%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data Availability

The data will be provided based on request.

References

  1. Zuo, B., Wang, F.: Surface cutting defect detection of magnet using Fourier image reconstruction. Comput. Eng. Appl. 52(3), 256–260 (2016)

    Google Scholar 

  2. Peng, X., Chen, Y., Yu, W., Zhou, Z., Sun, G.: An online defects inspection method for float glass fabrication based on machine vision. Int. J. Adv. Manuf. Technol. 39, 1180–1189 (2008)

    Article  Google Scholar 

  3. Shi, Y., Cui, L., Qi, Z., Meng, F., Chen, Z.: Automatic road crack detection using random structured forests. IEEE Trans. Intell. Transp. Syst. 17(12), 3434–3445 (2016)

    Article  Google Scholar 

  4. Tao, X., Zhang, D., Wang, Z., Liu, X., Zhang, H., Xu, D.: Detection of power line insulator defects using aerial images analyzed with convolutional neural networks. IEEE Trans. Syst. Man Cybern.: Syst. 50(4), 1486–1498 (2018)

    Article  Google Scholar 

  5. Zhang, C., Chang, C.-C., Jamshidi, M.: Concrete bridge surface damage detection using a single-stage detector. Comput.-Aid. Civil Infrastruct. Eng. 35(4), 389–409 (2020)

    Article  Google Scholar 

  6. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement (2018). arXiv:1804.02767

  7. Chen, S.-H., Tsai, C.-C.: Smd led chips defect detection using a yolov3-dense model. Adv. Eng. Inform. 47, 101255 (2021)

    Article  Google Scholar 

  8. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  9. Sabokrou, M., Khalooei, M., Fathy, M., Adeli, E.: Adversarially learned one-class classifier for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3379–3388 (2018)

  10. Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised anomaly segmentation in brain mr images. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I 4, pp. 161–169. Springer (2019)

  11. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14, pp. 622–637. Springer (2019)

  12. Yang, H., Zhou, Q., Song, K., Yin, Z.: An anomaly feature-editing-based adversarial network for texture defect visual inspection. IEEE Trans. Industr. Inf. 17(3), 2220–2230 (2020)

    Article  Google Scholar 

  13. Akyon, F.C., Altinuc, S.O., Temizel, A.: Slicing aided hyper inference and fine-tuning for small object detection. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 966–970. IEEE (2022)

  14. Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.-Y., Cubuk, E.D., Le, Q.V., Zoph, B.: Simple copy-paste is a strong data augmentation method for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2918–2928 (2021)

  15. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection (2020). arXiv:2010.04159

  16. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229. Springer (2020)

  17. Jocher, G.: yolov5 (2020). https://github.com/ultralytics/yolov5/. Accessed March 9, 2022

  18. Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7464–7475 (2023)

  19. Cai, Z., Vasconcelos, N.: Cascade r-cnn: Delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162 (2018)

  20. Pernkopf, F.: Detection of surface defects on raw steel blocks using Bayesian network classifiers. Pattern Anal. Appl. 7, 333–342 (2004)

    Article  MathSciNet  Google Scholar 

  21. Yue, B., Wang, Y., Min, Y., Zhang, Z., Wang, W., Yong, J.: Rail surface defect recognition method based on adaboost multi-classifier combination. In: 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 391–396. IEEE (2019)

  22. Fang, J., Tan, X., Wang, Y.: Acrm: attention cascade r-cnn with mix-nms for metallic surface defect detection. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 423–430. IEEE (2021)

  23. Xu, Y., Wang, X., Li, S.: Track surface defect detection based on efficientdet. In: International Conference on Electrical and Information Technologies for Rail Transportation, pp. 56–66. Springer (2021)

  24. Usamentiaga, R., Lema, D.G., Pedrayes, O.D., Garcia, D.F.: Automated surface defect detection in metals: a comparative review of object detection and semantic segmentation using deep learning. IEEE Trans. Ind. Appl. 58(3), 4203–4213 (2022)

    Article  Google Scholar 

  25. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  26. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2009)

    Article  Google Scholar 

  27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: Ssd: Single shot multibox detector. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp. 21–37. Springer (2016)

  28. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Adv. Neural Inform. Process. Syst. 30 (2017)

  29. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  30. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inform. Process. Syst. 28 (2015)

  31. Sun, P., Zhang, R., Jiang, Y., Kong, T., Xu, C., Zhan, W., Tomizuka, M., Li, L., Yuan, Z., Wang, C., et al.: Sparse r-cnn: End-to-end object detection with learnable proposals. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14454–14463 (2021)

  32. Meng, D., Chen, X., Fan, Z., Zeng, G., Li, H., Yuan, Y., Sun, L., Wang, J.: Conditional detr for fast training convergence. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3651–3660 (2021)

  33. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11106–11115 (2021)

  34. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740–755. Springer (2014)

  35. Wang, J., Chen, K., Yang, S., Loy, C.C., Lin, D.: Region proposal by guided anchoring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2965–2974 (2019)

  36. Yang, T., Zhang, X., Li, Z., Zhang, W., Sun, J.: Metaanchor: Learning to detect objects with customized anchors. Adv. Neural Inform. Process. Syst. 31 (2018)

  37. Zhang, X., Wan, F., Liu, C., Ji, R., Ye, Q.: Freeanchor: Learning to match anchors for visual object detection. Adv. Neural Inform. Process. Syst. 32 (2019)

  38. Ge, Z., Liu, S., Li, Z., Yoshie, O., Sun, J.: Ota: Optimal transport assignment for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 303–312 (2021)

  39. Wang, W., Cao, Y., Zhang, J., Tao, D.: Fp-detr: Detection transformer advanced by fully pre-training. In: International Conference on Learning Representations (2021)

  40. Wang, W., Zhang, J., Cao, Y., Shen, Y., Tao, D.: Towards data-efficient detection transformers. In: European Conference on Computer Vision, pp. 88–105. Springer (2022)

  41. Chen, Q., Chen, X., Wang, J., Zhang, S., Yao, K., Feng, H., Han, J., Ding, E., Zeng, G., Wang, J.: Group detr: Fast detr training with group-wise one-to-many assignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6633–6642 (2023)

  42. Jia, D., Yuan, Y., He, H., Wu, X., Yu, H., Lin, W., Sun, L., Zhang, C., Hu, H.: Detrs with hybrid matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19702–19712 (2023)

  43. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. arXiv e-prints (2017)

  44. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. IEEE (2016)

  45. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database, pp. 248–255 (2009)

Download references

Acknowledgements

Thanks to all the authors for their contributions to this article.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Author notes

  1. Haixia Pan and Han Zhao contributed equally to this work.

Authors and Affiliations

  1. School of Software, Beihang University, Beijing, 100191, China

    Haixia Pan, Han Zhao, Xingyun Wei, Dongdong Zhang, Biao Dong & Jiahua Lan

Authors
  1. Haixia Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingyun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongdong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Biao Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiahua Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, HZ; methodology, HP, HZ, XW, DZ, BD, JL; software, HZ; validation, HZ; formal analysis, HZ; investigation, HZ; resources, HP, HZ; data curation, HZ; writing—original draft preparation, HZ; writing—review and editing, HP, XW, DZ, BD, JL; visualization, HZ; supervision, HP; project administration, HP All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Haixia Pan.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Supervised enhancement algorithm pseudocode

The labelled truth values are replicated n times before the Hungarian matching is executed. Pseudocode is an example of copying 2x.

Algorithm 1
figure a

Algorithm of Supervised Enhancement

Full size image

Appendix B: Attention distillation algorithm pseudocode

Algorithm 2
figure b

Algorithm of Attention Distillation

Full size image

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, H., Zhao, H., Wei, X. et al. Study on defect detection of metal castings based on supervised enhancement and attention distillation. Machine Vision and Applications 35, 57 (2024). https://doi.org/10.1007/s00138-024-01536-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-024-01536-0

Keywords

Search

Navigation