Skip to main content
SpringerLink
Log in

PreAugNet: improve data augmentation for industrial defect classification with small-scale training data

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

With the prevalence of deep learning and convolutional neural network (CNN), data augmentation is widely used for enriching training samples to gain model training improvement. Data augmentation is important when training samples are scarce. This work focuses on improving data augmentation for training an industrial steel surface defect classification network, where the performance is largely depending on the availability of high-quality training samples. It is very difficult to find a sufficiently large dataset for this application in real-world settings. When it comes to synthetic data augmentation, the performance is often degraded by incorrect class labels, and a large effort is required to generate high-quality samples. This paper introduces a novel off-line pre-augmentation network (PreAugNet) which acts as a class boundary classifier that can effectively screen the quality of the augmented samples and improve image augmentation. This PreAugNet can generate augmented samples and update decision boundaries via an independent support vector machine (SVM) classifier. New samples are automatically distributed and combined with the original data for training the target network. The experiments show that these new augmentation samples can improve classification without changing the target network architecture. The proposed method for steel surface defect inspection is evaluated on three real-world datasets: AOI steel defect dataset, MT, and NEU datasets. PreAugNet significantly increases the accuracy by 3.3% (AOI dataset), 6.25% (MT dataset) and 2.1% (NEU dataset), respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets analysed during the current study are publicly available.

Code availability

Not applicable.

Notes

  1. https://aidea-web.tw/topic/701e1e79-84ff-49a5-86ee-a7f01c24c6f7.

References

  • Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., & Arshad, H. (2018). State-of-the-art in artificial neural network applications: a survey. Heliyon, 4(11), e00938.

    Article  PubMed  PubMed Central  Google Scholar 

  • Abu, M., Amir, A., Lean, Y., Zahri, N., & Azemi, S. (2021). The performance analysis of transfer learning for steel defect detection by using deep learning. Journal of Physics: Conference Series, 1755(1), 012041.

    CAS  Google Scholar 

  • Bowles, C., Chen, L., Guerrero, R., Bentley, P., Gunn, R., Hammers, A., Dickie, D. A., Hernández, M. V., Wardlaw, J., & Rueckert, D. (2018). Gan augmentation: Augmenting training data using generative adversarial networks. Preprint retrieved from https://arxiv.org/abs/1810.10863.

  • Burrascano, P. (1991). Learning vector quantization for the probabilistic neural network. IEEE Transactions on Neural Networks, 2(4), 458–461.

    Article  CAS  PubMed  Google Scholar 

  • Buslaev, A., Iglovikov, V. I., Khvedchenya, E., Parinov, A., Druzhinin, M., & Kalinin, A. A. (2020). Albumentations: Fast and flexible image augmentations. Information, 11(2), 125.

    Article  Google Scholar 

  • Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar cell surface defect inspection based on multispectral convolutional neural network. Journal of Intelligent Manufacturing, 31, 453–468.

    Article  Google Scholar 

  • Cheon, S., Lee, H., Kim, C. O., & Lee, S. H. (2019). Convolutional neural network for wafer surface defect classification and the detection of unknown defect class. IEEE Transactions on Semiconductor Manufacturing, 32(2), 163–170.

    Article  Google Scholar 

  • Choe, J., Lee, S., & Shim, H. (2020). Attention-based dropout layer for weakly supervised single object localization and semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12), 4256–4271.

    Article  Google Scholar 

  • Czimmermann, T., Ciuti, G., Milazzo, M., Chiurazzi, M., Roccella, S., Oddo, C. M., & Dario, P. (2020). Visual-based defect detection and classification approaches for industrial applications—A survey. Sensors, 20(5), 1459.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • DeVries, T., & Taylor, G. W. (2017). Improved regularization of convolutional neural networks with cutout. Preprint retrived from https://arxiv.org/abs/1708.04552.

  • Elleuch, M., Maalej, R., & Kherallah, M. (2016). A new design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition. Procedia Computer Science, 80, 1712–1723.

    Article  Google Scholar 

  • Farady, I., Lin, C. Y., Akhyar, F., Roshini, R., & Alex, J. S. R. (2021). Evaluation of data augmentation on surface defect detection. 2021 IEEE international conference on consumer electronics-Taiwan (ICCE-TW). IEEE.

    Google Scholar 

  • Farady, I., Sarkar, M. D., Chang, W. T., & Lin, C. Y. (2022). Evaluation of additional augmented images for steel surface defect detection. 2022 IEEE international conference on consumer electronics-Taiwan (pp. 1–2). IEEE.

    Google Scholar 

  • Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018). GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 321, 321–331.

    Article  Google Scholar 

  • Ghiasi, G., Lin, T. Y., & Le, Q. V. (2018). Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems, 31, 1–10.

    Google Scholar 

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139–144.

    Article  MathSciNet  Google Scholar 

  • He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask r-cnn. Proceedings of the IEEE international conference on computer vision (pp. 2961–2969). IEEE.

    Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceeding of the IEEE conference on computer vision and pattern recognition (pp. 770–778). IEEE.

    Google Scholar 

  • He, Y., Song, K., Dong, H., & Yan, Y. (2019a). Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network. Optics and Lasers in Engineering, 122, 294–302.

    Article  ADS  Google Scholar 

  • He, Y., Song, K., Meng, Q., & Yan, Y. (2019b). An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 69(4), 1493–1504.

    Article  ADS  Google Scholar 

  • Hernández-García, A., & König, P. (2018). Data augmentation instead of explicit regularization. Preprint retrieved from https://arxiv.org/abs/1806.03852.

  • Huang, Y., Qiu, C., Wang, X., Wang, S., & Yuan, K. (2020a). A compact convolutional neural network for surface defect inspection. Sensors, 20(7), 1974.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Huang, Y., Qiu, C., & Yuan, K. (2020b). Surface defect saliency of magnetic tile. The Visual Computer, 36, 85–96.

    Article  CAS  Google Scholar 

  • Jain, S., Seth, G., Paruthi, A., Soni, U., & Kumar, G. (2022). Synthetic data augmentation for surface defect detection and classification using deep learning. Journal of Intelligent Manufacturing, 2022, 1–14.

    Google Scholar 

  • Joshi, K. D., Chauhan, V., & Surgenor, B. (2020). A flexible machine vision system for small part inspection based on a hybrid SVM/ANN approach. Journal of Intelligent Manufacturing, 31, 103–125.

    Article  Google Scholar 

  • Kang, J., Park, Y. J., Lee, J., Wang, S. H., & Eom, D. S. (2017). Novel leakage detection by ensemble CNN-SVM and graph-based localization in water distribution systems. IEEE Transactions on Industrial Electronics, 65(5), 4279–4289.

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.

    Article  Google Scholar 

  • Kumar, A. (2008). Computer-vision-based fabric defect detection: a survey. IEEE Transactions on Industrial Electronics, 55(1), 348–363.

    Article  Google Scholar 

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  • Li, C., Xu, T., Zhu, J., & Zhang, B. (2017). Triple generative adversarial nets. Advances in Neural Information Processing Systems, 30, 1.

    Google Scholar 

  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). Ssd: Single shot multibox detector. Computer Vision–ECCV 2016: 14th European Conference, The Netherlands, October 11–14, 2016.

  • Liu, M. Y., & Tuzel, O. (2016). Coupled generative adversarial networks. Advances in Neural Information Processing Systems, 29, 1.

    CAS  Google Scholar 

  • Luo, Q., Fang, X., Liu, L., Yang, C., & Sun, Y. (2020). Automated visual defect detection for flat steel surface: A survey. IEEE Transactions on Instrumentation and Measurement, 69(3), 626–644.

    Article  ADS  CAS  Google Scholar 

  • Mao, K. Z., Tan, K. C., & Ser, W. (2000). Probabilistic neural-network structure determination for pattern classification. IEEE Transactions on Neural Networks, 11(4), 1009–1016.

    Article  CAS  PubMed  Google Scholar 

  • Marino, S., Beauseroy, P., & Smolarz, A. (2020). Unsupervised adversarial deep domain adaptation method for potato defects classification. Computers and Electronics in Agriculture, 174, 105501.

    Article  Google Scholar 

  • Meireles, M. R., Almeida, P. E., & Simões, M. G. (2003). A comprehensive review for industrial applicability of artificial neural networks. IEEE Transactions on Industrial Electronics, 50(3), 585–601.

    Article  Google Scholar 

  • Ngan, H. Y., Pang, G. K., & Yung, N. H. (2011). Automated fabric defect detection—a review. Image and Vision Computing, 29(7), 442–458.

    Article  Google Scholar 

  • Niu, X. X., & Suen, C. Y. (2012). A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recognition, 45(4), 1318–1325.

    Article  ADS  Google Scholar 

  • Pan, H., Pang, Z., Wang, Y., Wang, Y., & Chen, L. (2020). A new image recognition and classification method combining transfer learning algorithm and mobilenet model for welding defects. Ieee Access, 8, 119951–119960.

    Article  Google Scholar 

  • Park, J. K., Kwon, B. K., Park, J. H., & Kang, D. J. (2016). Machine learning-based imaging system for surface defect inspection. International Journal of Precision Engineering and Manufacturing-Green Technology, 3, 303–310.

    Article  Google Scholar 

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: unified, real-time object detection. Proceedings of the EEE conference on computer vision and pattern recognition (pp. 779–788). IEEE.

    Google Scholar 

  • Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28, 1.

    Google Scholar 

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234-241

  • Saito, K., Kim, D., Sclaroff, S., & Saenko, K. (2020). Universal domain adaptation through self supervision. Advances in Neural Information Processing Systems, 33, 16282–16292.

    Google Scholar 

  • Saritas, M. M., & Yasar, A. (2019). Performance analysis of ANN and Naive Bayes classification algorithm for data classification. International Journal of Intelligent Systems and Applications in Engineering, 7(2), 88–91.

    Article  Google Scholar 

  • Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1–48.

    Article  Google Scholar 

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint retrived from https://arxiv.org/abs/1409.1556.

  • Singh, K. K., & Lee, Y. J. (2017). Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization. IEEE International Conference on Computer Vision (ICCV), 2017, 3544–3553.

    Article  Google Scholar 

  • Sun, X., Liu, L., Li, C., Yin, J., Zhao, J., & Si, W. (2019). Classification for remote sensing data with improved CNN-SVM method. Ieee Access, 7, 164507–164516.

    Article  Google Scholar 

  • Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818–2826). IEEE.

    Google Scholar 

  • Tan, M., & Le, Q. (2019). Efficientnet: rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, 2019, 6105–6144.

    Google Scholar 

  • Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5), 988–999.

    Article  CAS  PubMed  Google Scholar 

  • Wong, S. C., Gatt, A., Stamatescu, V., & McDonnell, M. D. (2016). Understanding data augmentation for classification: when to warp? 2016 international conference on digital image computing: techniques and applications (DICTA) (pp. 1–6). IEEE.

    Google Scholar 

  • Xue, D. X., Zhang, R., Feng, H., & Wang, Y. L. (2016). CNN-SVM for microvascular morphological type recognition with data augmentation. Journal of Medical and Biological Engineering, 36, 755–764.

    Article  PubMed  PubMed Central  Google Scholar 

  • Yang, S., Wang, Y., Van De Weijer, J., Herranz, L., & Jui, S. (2021). Generalized source-free domain adaptation. Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 8978–8987). IEEE.

    Google Scholar 

  • Yun, J. P., Shin, W. C., Koo, G., Kim, M. S., Lee, C., & Lee, S. J. (2020). Automated defect inspection system for metal surfaces based on deep learning and data augmentation. Journal of Manufacturing Systems, 55, 317–324.

    Article  Google Scholar 

  • Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., & Yoo, Y. (2019). Cutmix: Regularization strategy to train strong classifiers with localizable features. Proceeding of the IEEE/CVF international conference on computer vision (pp. 6023–6032). IEEE.

    Google Scholar 

  • Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. Preprint retrived from https://arxiv.org/abs/1710.09412.

  • Zhang, S., Zhang, Q., Gu, J., Su, L., Li, K., & Pecht, M. (2021a). Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network. Mechanical Systems and Signal Processing, 153, 107541.

    Article  Google Scholar 

  • Zhang, W., Li, X., Ma, H., Luo, Z., & Li, X. (2021b). Transfer learning using deep representation regularization in remaining useful life prediction across operating conditions. Reliability Engineering & System Safety, 211, 107556.

    Article  Google Scholar 

  • Zhang, W., Li, X., Ma, H., Luo, Z., & Li, X. (2021c). Universal domain adaptation in fault diagnostics with hybrid weighted deep adversarial learning. IEEE Transactions on Industrial Informatics, 17(12), 7957–7967.

    Article  Google Scholar 

  • Zhang, Y., Wang, Y., Jiang, Z., Zheng, L., Chen, J., & Lu, J. (2022). Tire Defect Detection by Dual-Domain Adaptation-Based Transfer Learning Strategy. IEEE Sensors Journal, 22(19), 18804–18814.

    Article  ADS  Google Scholar 

  • Zhong, J., Liu, X., & Hsieh, C.J. (2020a). Improving the speed and quality of gan by adversarial training. Preprint retrieved from https://arxiv.org/abs/2008.03364.

  • Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020b). Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 13(7), 13001–13008.

    Article  Google Scholar 

  • Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision (pp. 2223–2232). IEEE.

    Google Scholar 

Download references

Funding

This study was funded by Ministry of Science and Technology, Taiwan (MOST 110-2221-E-155-039-MY3).

Author information

Authors and Affiliations

  1. Electrical Engineering, Yuan Ze University, Taoyuan, Taiwan

    Isack Farady

  2. Electrical Engineering, Mercu Buana University, West Jakarta, Indonesia

    Isack Farady

  3. Mechanical Engineering, National Central University, Taoyuan, Taiwan

    Chih-Yang Lin

  4. Computer Science, University at Albany, Albany, USA

    Ming-Ching Chang

Authors
  1. Isack Farady
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chih-Yang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming-Ching Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the design and implementation of the research. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chih-Yang Lin.

Ethics declarations

Competing interest

The authors declare that they have no competing interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farady, I., Lin, CY. & Chang, MC. PreAugNet: improve data augmentation for industrial defect classification with small-scale training data. J Intell Manuf 35, 1233–1246 (2024). https://doi.org/10.1007/s10845-023-02109-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-023-02109-0

Keywords

Search

Navigation