Elsevier

Measurement

Volume 170, January 2021, 108736
Measurement

Casting defect detection in X-ray images using convolutional neural networks and attention-guided data augmentation

https://doi.org/10.1016/j.measurement.2020.108736Get rights and content

Highlights

  • A weakly supervised convolutional neural network for classification is proposed.

  • The defective castings with the X-ray casting images are classified automatically.

  • The attention-guided data augmentation enlarges the dataset effectively.

  • The mutual-channel loss focuses on subtle discriminative details in two classes.

Abstract

Aluminum alloy castings have a high utilization rate in the automotive industry, and its quality directly affects the safety performance of the mechanical components. Hence, casting quality management is vital during the casting production process. This paper presents a weakly-supervised Convolutional Neural Network model to recognize defects based on casting X-ray images. These images are divided into two classes including defective and non-defective. Firstly, attention maps are generated to represent the defective parts by weakly-supervised learning for each image. Then mutual-channel loss combined with the cross-entropy loss function encourage the network to focus on discriminative features. Simultaneously, a novel data-augmentation methods guided by these attention maps is proposed to enlarge the dataset. The test accuracy achieves 95.5%, and the recall is 96.0%, which means our model is accurate and robust. The efficiency of the proposed approach is verified by comparing the state-of-art approaches and the ablation experiments.

Introduction

With the development of the lightweight automotive technology, the demand for aluminum alloy castings rapid growth. During the casting process, various defects such as holes and flaws [1] are inevitably generated due to the limitation of the manufacturing technique. If these defective castings are still used in production, they will lead to catastrophic failure of critical mechanical components [2]. Hence, it is necessary and important to guarantee the quality of the aluminum components. Nowadays, inspection systems based on vision have been widely used in various industries [3], which not only greatly enhances the recognition accuracy, but also greatly decreases the cost of manual inspecting, substantially. Conventional computer vision methods need the designed features and statistics-based machine learning models [4], [5]. Specifically, image processing algorithms are used to generate feature vectors; then, the researchers rely on machine learning-based methods to form inspection systems [6]. However, the handcrafted feature vectors, which are designed in a specific scenario, can hardly satisfy the requirements in a complex real industrial environment. What is more, classification method like pixel-wise classification [7] is time-consuming and the context information is limited by the image blocks.

As the deep learning technology develops [8], convolutional neural network(CNN) inspired from the visual cortex of the brain are used to solve difficult image-driven pattern recognition missions [9], [10], and it has achieved the highest accuracy in X-ray images than sparse representations, local descriptors and texture features [11]. CNN is capable of automatically extracting features, which can save time and cost of designing features manually. The great performances have achieved in defects recognition [12], [13] and detection [14], [15] in many industrial fields.

The defective castings recognition can be framed as an fine-grained image classification task. The casting X-ray images are divided into defective and non-defective images. Casting images in the same class may appear to be very different in appearance and pose among the different types of castings components; In addition, the discriminative difference between the two classes is only fine-grained defects. The task is much harder compared to conventional image classification [16] since the differences between defective and non-defective castings are subtle and deeply embedded within local discriminative parts. What is more, labeling each defect requires expensive expert knowledge. Hence, we utilize weakly supervised learning (WSL) [17] to locate the discriminative defective parts only by image-level annotation. WSL greatly saves the time and cost of image pre-processing. However, an image-level annotation has not enough position information of the defects. To overcome the challenges, defective parts or visual pattern are presented by attention maps which is able to select more attributes of input data.

The loss function is very important in machine learning because the process of training model is actually the process of optimizing the loss function. Any function that can measure the difference between the predicted value and the real value of the model can be called the loss function. The loss function is used to measure the quality of the model. The smaller the loss function is, the more consistent the model and parameters are to the training samples. Therefore, the main task of a model training is to use optimization methods to find the model parameters corresponding to loss function minimization. Unlike the common loss function in the traditional image classification tasks, it is necessary to pay more attention to reducing the intra-class differences and enlarging the inter-class differences in our defective casting classification.

For deep supervised networks, a large dataset is needed to achieve high accuracy. Fortunately, data augmentation (DA) [18] methods such as spatial transformation, color distortion, and information dropping. These method improve the iteration of deep models through increasing the number of the training data. DA is a frequently-used strategy by introducing more data variances. Because the available samples of the defective castings is smaller than the non-defective castings. However, a large number of samples of each class is necessary during the process of training a CNN model. Therefore, we propose a novel data augmentation method to increase the number of training images.

In this paper, a comprehensive study based on the deep neural network consisting of advanced training technologies is proposed to classify the defective castings. In the proposed study, VGG-16 (Visual Geometry Group) based CNN model is used for the feature extraction [19]. This model is trained with the transfer learning [20] approach which achieves high performance on the ImageNet dataset. In the next step, a 11 filter is operated on the selected feature layer to generate the attention maps. These attention maps will guide the data augmentation block to create new training data. At the same time, two loss function viz., cross-entropy loss and mutual channel loss are combined to optimize the proposed model.

The main contributions of this paper are: (1) We propose a novel modified CNN based model to classify the defective castings in x-ray images. The proposed network is able to extract features automatically merely on the image-level annotations and learn the attention maps of the castings. (2) We test the effectiveness of the proposed attention-guided data augmentation and mutual-channel loss through the ablation experiment. The results show that the proposed model increase the performance of the baseline model to a large extent.

The research paper is organized into five sections: Section 1 introduces the topic and explains its importance. Section 2 explores work related to the proposed model. Section 3 explains the methodology of the paper, explaining the dataset and the specific CNN used in this paper. Section 4 presents the experimental results achieved by this model and explain the details. Section 5 concludes the research paper.

Section snippets

Related work

The task of classifying casting x-ray images into defective and non-defective classes has been significantly explored. In the early works, the effectiveness of CNN based models used to detect the defects are proved by researchers. Mery Domingo [11] developed an Xnet model which simplifies the CNN for the defect detection task. It only has a five weight-layer net and ten layers in total. Additionally, they included a dropout block that randomly turns off connections of the neural network during

Methodology

In this section, we introduce a modified CNN architecture combined with attention-guided data augmentation and mutual-channel (MC) loss function. As shown in Fig. 1, after obtaining the feature maps, we add a 1 × 1 convolution layer to generate the attention maps. Then for each training image, we randomly choose one of its attention maps to guide the data augmentation process. At last, the obtained attention maps enter into two streams of the network. There are two different sub-loss tailored

Experiments

In this section, our model is trained and tested on the x-Ray Images (Casting) dataset consisting of 6992 images for training, 1000 images for validation, and 1000 images for testing the model (Table 2). Accuracy, Recall, Precision, and F1 score are used to measure the performance of the model. It is highly significant to select an appropriate evaluation criterion for the classification task. Accuracy is the classification accuracy of the model. Recall measures the performance evaluation in the

Conclusion

This research paper presents a weakly supervised CNN model (MC-DAN) which combines the attention-guided data augmentation and mutual-channel loss function. Weakly supervised learning provides the spatial distribution of the defects for data augmentation which encourages the network to learn from multiple discriminative features. Simultaneously, the added mutual-channel loss focuses more on the discriminative features through diving into the feature channels before they fused. The image-level

CRediT authorship contribution statement

Lili Jiang: Conceptualization, Methodology, Software, Validation, Writing - original draft, Writing - review & editing, Visualization. Yongxiong Wang: Writing - review & editing, Supervision. Zhenhui Tang: Data curation, Writing - review & editing, Visualization. Yinlong Miao: Supervision. Shuyi Chen: Resources.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This paper is financially supported by the National Science Foundation of China (No. 61673276).

References (36)

  • MaoWentao et al.

    Online detection for bearing incipient fault based on deep transfer learning

    Measurement

    (2020)
  • KolarZdenek et al.

    Transfer learning and deep convolutional neural networks for safety guardrail detection in 2d images

    Autom. Constr.

    (2018)
  • IngleVaibhav et al.

    Defects, root causes in casting process and their remedies: Review

    Int. J. Eng. Res. Appl.

    (2017)
  • FergusonMax et al.

    Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning

    Comput. Vis. Pattern Recognit.

    (2018)
  • AndrushiaA. Diana et al.

    Rts-elm: an approach for saliency-directed image segmentation with ripplet transform

    Pattern Anal. Appl.

    (2020)
  • CiresanDan Claudio et al.

    High-performance neural networks for visual object classification

    Artificial Intelligence

    (2011)
  • MeryDomingo et al.

    Automatic defect recognition in x-ray testing using computer vision

  • FergusonMax et al.

    Automatic localization of casting defects with convolutional neural networks

  • Cited by (36)

    • ADNet: Anti-noise dual-branch network for road defect detection

      2024, Engineering Applications of Artificial Intelligence
    • GDALR: Global Dual Attention and Local Representations in transformer for surface defect detection

      2024, Measurement: Journal of the International Measurement Confederation
    View all citing articles on Scopus
    View full text