Skip to main content
SpringerLink
Log in

Enhanced detection of diverse defects by developing lighting strategies using multiple light sources based on reinforcement learning

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Traditional inspection systems with a single light source are not efficient at detecting a few particular defects with a single inspection. Unlike before, the multi-light source inspection environment allows us to extract more different defects in a piece of work, depending on what we are working on with varying sources of light. We proposed the formulation of the multi-lights source lighting strategy to improve the inspection capability of Automated Optical Inspection (AOI). The process of developing this study not only utilizes the ubiquitous image processing to extract defects but also imports the design of generalized defect sample and reinforcement learning, dealing with diverse defects under in-depth inspection by cascading both light and camera parameters. As a result, the AOI system emphasized that the inspection parameters can be intelligently adjusted to appropriate values based on various defects, maximizing the detection of diverse defects. From the perspective of intelligent AOI results, there are two outstanding outcomes for a multi-light source lighting strategy. One is an efficient learning process, which facilitates us to obtain the strategy needed in 40 to 50 min, depending on the reward function designed. The other is an advanced inspection function that can extract 37% more defects than conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Data availability

The datasets and materials generated during and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

Code availability

The Code during the present study is not publicly available but is available from the corresponding author on request.

References

  • Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), 679–698.

    Article  Google Scholar 

  • Chang, S. G., Yu, B., & Vetterli, M. (2000). Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9), 1532–1546.

    Article  Google Scholar 

  • Chang, M., Chen, B. C., Gabayno, J. L., & Chen, M. F. (2016). Development of an optical inspection platform for surface defect detection in touch panel glass. International Journal of Optomechatronics, 10(2), 63–72.

    Article  Google Scholar 

  • Chen, T. D., & Perng, D. B. (2005). Design and development of a CMOS glass Lid AOI Machine. Mster Thesis, Department of Industrial Engineering and Managemente, National Chiao Tung University, Hsinchu, Taiwan.

  • Chen, S., & Perng, D. (2016). Automatic optical inspection system for IC molding surface. Journal of Intelligent Manufacturing, 27, 915–926.

    Article  Google Scholar 

  • Chen, M. F., Chen, B. C., Chen, C. W., Weng, R. C., & Chang, M. (2015). Design and implementation of an illumination device for optical inspection of defects in glass substrates. Ninth International Symposium on Precision Engineering Measurement and Instrumentation (ISPEMI), 9446, 94464B.

    Google Scholar 

  • Choi, M. G., Jung, J. H., & Jeon, J. W. (2009). No-reference image quality assessment using blur and noise. International Journal of Computer Science and Engineering, 3(2), 76–80.

    Google Scholar 

  • Dulac-Arnold, G., Evans, R., Hasselt, H. V., Sunehag, P., Lillicrap, T., Hunt, J. et al. (2015). Deep reinforcement learning in large discrete action spaces. arXiv preprint, arXiv:1512.07679.

  • Huang, C., Liao, C., Huang, A., & Tarng, Y. (2008). An automatic optical inspection of drill point defects for micro-drilling. The International Journal of Advanced Manufacturing Technology, 37(11–12), 1133–1145.

    Article  Google Scholar 

  • Idesawa, M. (1989). High-precision image position sensing methods suitable for 3-D measurement. Optics and Lasers in Engineering, 10(3–4), 191–204.

    Article  Google Scholar 

  • Jeon, Y. J., Choi, D. C., Lee, S. J., Yun, J. P., & Kim, S. W. (2016). Steel-surface defect detection using a switching-lighting scheme. Applied Optics, 55(1), 47–57.

    Article  Google Scholar 

  • Kuo, C. J., Fang, T. Y., Lee, C. L., & Wu, H. C. (2019). Automated optical inspection system for surface mount device light emitting diodes. Journal of Intelligent Manufacturing, 30, 641–655.

    Article  Google Scholar 

  • Li, Y., Li, Y. F., Wang, Q. L., Xu, D., & Tan, M. (2010). Measurement and defect detection of the weld bead based on online vision inspection. IEEE Transactions on Instrumentation and Measurement, 59(7), 1841–1849.

    Article  Google Scholar 

  • Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T. Tassa, Y. et al. (2015). Continuous control with deep reinforcement learning. arXiv:1509.02971.

  • Lin, H. D. (2008). Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques. Image and Vision Computing, 26(5), 603–621.

    Article  Google Scholar 

  • Mehle, A., Bukovec, M., Likar, B., & Tomaževič, D. (2016). Print registration for automated visual inspection of transparent pharmaceutical capsules. Machine Vision and Applications, 27(7), 1087–1102.

    Article  Google Scholar 

  • Mehta, S., Patel, A., & Mehta, J. (2015). CCD or CMOS Image sensor for photography. In International conference on communications and signal processing (ICCSP) (pp. 0291–0294). https://doi.org/10.1109/ICCSP.2015.7322890.

  • Newman, T. S., & Jain, A. K. (1995). A survey of automated visual inspection. Computer Vision and Image Understanding, 61(2), 231–262.

    Article  Google Scholar 

  • Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66.

    Article  Google Scholar 

  • Schechner, Y., Belhumeur, P., & Nayar, S. (2007). Multiplexing for optimal lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(08), 1339–1354.

    Article  Google Scholar 

  • Simonyan, K. & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv:1409.1556.

  • Spence, A. D., & Chantler, M. J. (2006). Optimal illumination for three-image photometric stereo using sensitivity analysis. IEE Proceedings-Vision, Image and Signal Processing, 153(2), 149–159. https://doi.org/10.1049/ip-vis:20050229

    Article  Google Scholar 

  • Uhlenbeck, G. E., & Ornstein, L. S. (1930). On the theory of the brownian motion. Physical Review, 36(5), 823.

    Article  Google Scholar 

  • Uusitalo, J., & Tuokko, R. (2007). Setting up task-optimal illumination automatically for inspection purposes. International Society for Optics and Photonics (SPIE), 6503, 65030K.

    Google Scholar 

  • Varun, A.V. (2011). Adaptive lighting for machine vision applications. In 2011 Canadian conference on computer and robot vision (pp. 140–145). https://doi.org/10.1109/CRV.2011.25.

  • Vitabile, S., Pollaccia, G., Pilato, G. & Sorbello, F. (2001). Road signs recognition using a dynamic pixel aggregation technique in the HSV color space. In International conference on image analysis and processing (ICIAP) (p. 572).

  • Wang, S., Mu, P., Dai, S., & Liu, W. (2007). Study on automatic measurement of tiny-size. In: 3rd international symposium on advanced optical manufacturing and testing technologies: optical test and measurement technology and equipment (vol. 6723, p. 67232E).

  • Xu, L. M., Yang, Z. Q., Jiang, Z. H., et al. (2017). Light source optimization for automatic visual inspection of piston surface defects. The International Journal of Advanced Manufacturing Technology, 91, 2245–2256.

    Article  Google Scholar 

  • Ye, R., Chang, M. C., Pan, S., Chiang, C. A., & Gabayno, J. L. (2018). High-resolution optical inspection system for fast detection and classification of surface defects. International Journal of Optomechatronics, 12(1), 1–10.

    Article  Google Scholar 

Download references

Funding

The authors would like to acknowledge the support of this work from Ministry of Science and Technology, Taiwan R.O.C. (MOST 108-2221-E-007-082-MY3).

Author information

Authors and Affiliations

  1. Department of Power Mechanical Engineering, National Tsing Hua University, 30013, Hsinchu, Taiwan ROC

    Chih-Kai Cheng & Hung-Yin Tsai

Authors
  1. Chih-Kai Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hung-Yin Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hung-Yin Tsai.

Ethics declarations

Conflict of interest

All authors have no conflicts of interest or competing interests to report.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, CK., Tsai, HY. Enhanced detection of diverse defects by developing lighting strategies using multiple light sources based on reinforcement learning. J Intell Manuf 33, 2357–2369 (2022). https://doi.org/10.1007/s10845-021-01800-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01800-4

Keywords

Search

Navigation