Skip to main content
SpringerLink
Log in

An adaptive feature reconstruction network for the precise segmentation of surface defects on printed circuit boards

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Segmentation networks based on deep learning are widely used in the field of industrial vision inspection, including for the precise segmentation of surface defects on printed circuit boards (PCBs). However, most previous studies have focused only on the utilization of defect samples with visible defects and underestimated the value of template samples without surface defects. In fact, template samples can provide sufficient prior information to identify defects and are not difficult to obtain in many manufacturing scenarios. Therefore, an adaptive feature reconstruction network (AFRNet) is proposed in this paper to utilize these two types of samples. Specifically, AFRNet consists of two main components: a Siamese encoder with shared parameters for extracting features from the input sample pair, and a symmetrical feature reconstruction module for adaptively fusing these extracted features. Similar image-level and feature-level fusion schemes, as well as spatial misalignment caused by unaligned sample pairs have been carefully studied. Extensive experiments on a real-world PCB surface-defect dataset confirm the effectiveness of the proposed method, demonstrating that it can significantly improve the segmentation performance of multiple baselines, such as DANet, PSPNet and DeepLabv3.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The PCB surface-defect dataset is publicly available at https://github.com/youtang1993/MeiweiPCB.

  2. https://pytorch.org/.

  3. https://github.com/open-mmlab/mmsegmentation.

References

  • Bromley, J., Guyon, I., LeCun, Y., et al. (1993). Signature verification using a siamese time delay neural network. In Advances in Neural Information Processing Systems 6, [7th NIPS Conference, Denver, Colorado, USA, 1993] (pp. 737–744). Morgan Kaufmann.

  • Çelik, A., Kucukmanisa, A., Sumer, A., et al. (2020). A real-time defective pixel detection system for LCDs using deep learning based object detectors. Journal of Intelligent Manufacturing, 1, 1–10. https://doi.org/10.1007/s10845-020-01704-9

    Article  Google Scholar 

  • Cen, M., & Jung, C. (2018). Fully convolutional siamese fusion networks for object tracking. In 2018 25th IEEE International Conference on Image Processing (ICIP) (pp. 3718–3722) https://doi.org/10.1109/ICIP.2018.8451102.

  • Chen, H., Pang, Y., Hu, Q., et al. (2020). Solar cell surface defect inspection based on multispectral convolutional neural network. Journal of Intelligent Manufacturing, 31, 453–468. https://doi.org/10.1007/s10845-018-1458-z

    Article  Google Scholar 

  • Chen, J., Yuan, Z., Peng, J., et al. (2021). DASNet: Dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 1194–1206. https://doi.org/10.1109/JSTARS.2020.3037893

    Article  Google Scholar 

  • Chen, L., Papandreou, G., Kokkinos, I., et al. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  • Chen, L., Zhu, Y., Papandreou, G., et al. (2018b). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Computer Vision—ECCV 2018—15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VII, Lecture Notes in Computer Science (Vol. 11211, pp. 833–851). Springer https://doi.org/10.1007/978-3-030-01234-2_49.

  • Chen, L. C., Papandreou, G., Schroff, F., et al. (2017). Rethinking atrous convolution for semantic image segmentation. CoRR abs/1706.05587. arXiv: 1706.05587.

  • Deng, J., Dong, W., Socher, R., et al. (2009). ImageNet: a large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20–25 June 2009, Miami, Florida, USA. IEEE Computer Society (pp. 248–255). https://doi.org/10.1109/CVPR.2009.5206848.

  • Foumani, M., Ibrahim, M. Y., Gunawan, I. (2013). Scheduling dual gripper robotic cells with a hub machine. In 22nd IEEE International Symposium on Industrial Electronics, ISIE 2013, Taipei, Taiwan, May 28–31, 2013. IEEE (pp. 1–6). https://doi.org/10.1109/ISIE.2013.6563748.

  • Fu, J., Liu, J., Tian, H., et al. (2019). Dual attention network for scene segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Computer Vision Foundation/IEEE (pp. 3141–3149) https://doi.org/10.1109/CVPR.2019.00326.

  • Gao, Y., Li, X., Wang, X. V., et al. (2021). A review on recent advances in vision-based defect recognition towards industrial intelligence. Journal of Manufacturing Systems, 1, 1–1. https://doi.org/10.1016/J.JMSY.2021.05.008

    Article  Google Scholar 

  • Hadsell, R., Chopra, S., LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006) 17–22 June 2006, New York, NY, USA (pp. 1735–1742). https://doi.org/10.1109/CVPR.2006.100.

  • He, K., Zhang, X., Ren, S., et al. (2014). Spatial pyramid pooling in deep convolutional networks for visual recognition. In Computer Vision—ECCV 2014—13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part III, Lecture Notes in Computer Science (Vol. 8691, pp. 346–361). Springer. https://doi.org/10.1007/978-3-319-10578-9/23.

  • He, K., Zhang, X., Ren, S., et al. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016 (pp. 770–778). IEEE Computer Society. https://doi.org/10.1109/CVPR.2016.90.

  • He, T., Zhang, Z., Zhang, H., et al. (2019). Bag of tricks for image classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16–20, 2019. Computer Vision Foundation/IEEE (pp. 558–567). https://doi.org/10.1109/CVPR.2019.00065.

  • Huang, Z., Wang, X., Wei, Y., et al. (2020). CCNet: Criss-cross attention for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 603–612. https://doi.org/10.1109/TPAMI.2020.3007032

    Article  Google Scholar 

  • Kirillov, A., Wu, Y., He, K., et al. (2020). PointRend: Image segmentation as rendering. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13–19, 2020 (pp. 9796–9805). IEEE. https://doi.org/10.1109/CVPR42600.2020.00982.

  • LeCun, Y., Boser, B. E., Denker, J. S., et al. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4), 541–551. https://doi.org/10.1162/neco.1989.1.4.541

    Article  Google Scholar 

  • Lin, T., Goyal, P., Girshick, R. B., et al. (2020). Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 318–327. https://doi.org/10.1109/TPAMI.2018.2858826

    Article  Google Scholar 

  • Liu, R., Sun, Z., Wang, A., et al. (2020). Real-time defect detection network for polarizer based on deep learning. Journal of Intelligent Manufacturing, 31(8), 1813–1823. https://doi.org/10.1007/s10845-020-01536-7

    Article  Google Scholar 

  • Liu, S., Huang, D., Wang, Y. (2018). Receptive field block net for accurate and fast object detection. In Computer Vision—ECCV 2018—15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XI, Lecture Notes in Computer Science. (Vol. 11215, pp. 404–419). Springer. https://doi.org/10.1007/978-3-030-01252-6/24.

  • Luo, Q., Fang, X., Liu, L., et al. (2020). Automated visual defect detection for flat steel surface: A survey. IEEE Transactions on Instrumentation and Measurement, 69(3), 626–644. https://doi.org/10.1109/TIM.2019.2963555

    Article  Google Scholar 

  • Miao, Y., Jeon, J., & Park, G. (2020). An image processing-based crack detection technique for pressed panel products. Journal of Manufacturing Systems, 57, 287–297. https://doi.org/10.1016/j.jmsy.2020.10.004

    Article  Google Scholar 

  • Schwendemann, S., Amjad, Z., & Sikora, A. (2021). A survey of machine-learning techniques for condition monitoring and predictive maintenance of bearings in grinding machines. Computers in Industry, 125(103), 380. https://doi.org/10.1016/j.compind.2020.103380

    Article  Google Scholar 

  • Shelhamer, E., Long, J., & Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  • Shrivastava, A., Gupta, A., Girshick, R. B. (2016). Training region-based object detectors with online hard example mining. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016 (pp. 761–769). IEEE Computer Society. https://doi.org/10.1109/CVPR.2016.89.

  • Smith, M. L., Smith, L. N., & Hansen, M. F. (2021). The quiet revolution in machine vision—A state-of-the-art survey paper, including historical review, perspectives, and future directions. Computers in Industry, 130(103), 472. https://doi.org/10.1016/j.compind.2021.103472

    Article  Google Scholar 

  • Song, K., & Yan, Y. (2013). A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 285, 858–864. https://doi.org/10.1016/J.APSUSC.2013.09.002

    Article  Google Scholar 

  • Tabernik, D., Sela, S., Skvarc, J., et al. (2020). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3), 759–776. https://doi.org/10.1007/s10845-019-01476-x

    Article  Google Scholar 

  • Tsai, D. M., & Rivera Molina, D. E. (2019). Morphology-based defect detection in machined surfaces with circular tool-mark patterns. Measurement, 134, 209–217. https://doi.org/10.1016/j.measurement.2018.10.079

    Article  Google Scholar 

  • Wang, X., Girshick, R. B., Gupta, A., et al. (2018). Non-local neural networks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18–22, 2018 (pp. 7794–7803). IEEE Computer Society. https://doi.org/10.1109/CVPR.2018.00813.

  • Wang, Z., & Zhu, D. (2019). An accurate detection method for surface defects of complex components based on support vector machine and spreading algorithm. Measurement, 147(106), 886. https://doi.org/10.1016/J.MEASUREMENT.2019.106886

    Article  Google Scholar 

  • Wei, X., Yang, Z., Liu, Y., et al. (2019). Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Engineering Applications of Artificial Intelligence, 80, 66–81. https://doi.org/10.1016/j.engappai.2019.01.008

    Article  Google Scholar 

  • Wu, N., Chu, F., Chu, C., et al. (2011). Petri net-based scheduling of single-arm cluster tools with reentrant atomic layer deposition processes. IEEE Transactions on Automation Science and Engineering, 8(1), 42–55. https://doi.org/10.1109/TASE.2010.2046736

    Article  Google Scholar 

  • Xiao, T., Liu, Y., Zhou, B., et al. (2018). Unified perceptual parsing for scene understanding. In Computer Vision—ECCV 2018—15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V, Lecture Notes in Computer Science (Vol. 11209, pp. 432–448). Springer. https://doi.org/10.1007/978-3-030-01228-1/26.

  • Xing, J., & Jia, M. (2021). A convolutional neural network-based method for workpiece surface defect detection. Measurement, 176(109), 185. https://doi.org/10.1016/J.MEASUREMENT.2021.109185

    Article  Google Scholar 

  • Yang, Y., Yang, R., Pan, L., et al. (2020). A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery. Computers in Industry, 123(103), 306. https://doi.org/10.1016/j.compind.2020.103306

    Article  Google Scholar 

  • Yun, J., Shin, W., Koo, G., et al. (2020). Automated defect inspection system for metal surfaces based on deep learning and data augmentation. Journal of Manufacturing Systems, 55, 317–324. https://doi.org/10.1016/j.jmsy.2020.03.009

  • Zhang, H., Dana, K. J., Shi, J., et al. (2018). Context encoding for semantic segmentation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18–22, 2018 (pp. 7151–7160). IEEE Computer Society. https://doi.org/10.1109/CVPR.2018.00747.

  • Zhang, H., Jiang, L., & Li, C. (2021). CS-ResNet: Cost-sensitive residual convolutional neural network for PCB cosmetic defect detection. Expert Systems with Applications, 185(1), 115–673. https://doi.org/10.1016/j.eswa.2021.115673

  • Zhao, H., Shi, J., Qi, X., et al. (2017). Pyramid scene parsing network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017 (pp. 6230–6239). IEEE Computer Society. https://doi.org/10.1109/CVPR.2017.660.

  • Zhu, Z., Xu, M., Bai, S., et al. (2019). Asymmetric non-local neural networks for semantic segmentation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 593–602). IEEE https://doi.org/10.1109/ICCV.2019.00068.

Download references

Acknowledgements

This work was partially supported by the Key Areas Research and Development Program of Guangdong Province under Grant 2018B010109007 and the Key-Area Research and Development Program of Guangzhou under Grant 202007030004.

Author information

Authors and Affiliations

  1. Sun Yat-Sen University, Guangzhou, 510006, China

    Danqing Kang, Jianhuang Lai & Yu Han

  2. School of Information Science, Guangzhou Xinhua University, Guangzhou, 510520, China

    Jianhuang Lai

  3. Guangdong Provincial Key Laboratory of Fire Science and Technology, Guangzhou, 510006, China

    Yu Han

  4. Intelligence Eyes Co., Ltd, Guangzhou, 510006, China

    Junyong Zhu

Authors
  1. Danqing Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhuang Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhuang Lai.

Ethics declarations

Conflict of interest

There are no conflicts of interest to declare.

Ethical approval

We confirm that the content of the manuscript has not been published or submitted for publication elsewhere. All authors have seen the manuscript and approved it for submission to the journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, D., Lai, J., Zhu, J. et al. An adaptive feature reconstruction network for the precise segmentation of surface defects on printed circuit boards. J Intell Manuf 34, 3197–3214 (2023). https://doi.org/10.1007/s10845-022-02008-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-02008-w

Keywords

Search

Navigation