Casting defect detection in X-ray images using convolutional neural networks and attention-guided data augmentation
Graphical abstract
Introduction
With the development of the lightweight automotive technology, the demand for aluminum alloy castings rapid growth. During the casting process, various defects such as holes and flaws [1] are inevitably generated due to the limitation of the manufacturing technique. If these defective castings are still used in production, they will lead to catastrophic failure of critical mechanical components [2]. Hence, it is necessary and important to guarantee the quality of the aluminum components. Nowadays, inspection systems based on vision have been widely used in various industries [3], which not only greatly enhances the recognition accuracy, but also greatly decreases the cost of manual inspecting, substantially. Conventional computer vision methods need the designed features and statistics-based machine learning models [4], [5]. Specifically, image processing algorithms are used to generate feature vectors; then, the researchers rely on machine learning-based methods to form inspection systems [6]. However, the handcrafted feature vectors, which are designed in a specific scenario, can hardly satisfy the requirements in a complex real industrial environment. What is more, classification method like pixel-wise classification [7] is time-consuming and the context information is limited by the image blocks.
As the deep learning technology develops [8], convolutional neural network(CNN) inspired from the visual cortex of the brain are used to solve difficult image-driven pattern recognition missions [9], [10], and it has achieved the highest accuracy in X-ray images than sparse representations, local descriptors and texture features [11]. CNN is capable of automatically extracting features, which can save time and cost of designing features manually. The great performances have achieved in defects recognition [12], [13] and detection [14], [15] in many industrial fields.
The defective castings recognition can be framed as an fine-grained image classification task. The casting X-ray images are divided into defective and non-defective images. Casting images in the same class may appear to be very different in appearance and pose among the different types of castings components; In addition, the discriminative difference between the two classes is only fine-grained defects. The task is much harder compared to conventional image classification [16] since the differences between defective and non-defective castings are subtle and deeply embedded within local discriminative parts. What is more, labeling each defect requires expensive expert knowledge. Hence, we utilize weakly supervised learning (WSL) [17] to locate the discriminative defective parts only by image-level annotation. WSL greatly saves the time and cost of image pre-processing. However, an image-level annotation has not enough position information of the defects. To overcome the challenges, defective parts or visual pattern are presented by attention maps which is able to select more attributes of input data.
The loss function is very important in machine learning because the process of training model is actually the process of optimizing the loss function. Any function that can measure the difference between the predicted value and the real value of the model can be called the loss function. The loss function is used to measure the quality of the model. The smaller the loss function is, the more consistent the model and parameters are to the training samples. Therefore, the main task of a model training is to use optimization methods to find the model parameters corresponding to loss function minimization. Unlike the common loss function in the traditional image classification tasks, it is necessary to pay more attention to reducing the intra-class differences and enlarging the inter-class differences in our defective casting classification.
For deep supervised networks, a large dataset is needed to achieve high accuracy. Fortunately, data augmentation (DA) [18] methods such as spatial transformation, color distortion, and information dropping. These method improve the iteration of deep models through increasing the number of the training data. DA is a frequently-used strategy by introducing more data variances. Because the available samples of the defective castings is smaller than the non-defective castings. However, a large number of samples of each class is necessary during the process of training a CNN model. Therefore, we propose a novel data augmentation method to increase the number of training images.
In this paper, a comprehensive study based on the deep neural network consisting of advanced training technologies is proposed to classify the defective castings. In the proposed study, VGG-16 (Visual Geometry Group) based CNN model is used for the feature extraction [19]. This model is trained with the transfer learning [20] approach which achieves high performance on the ImageNet dataset. In the next step, a filter is operated on the selected feature layer to generate the attention maps. These attention maps will guide the data augmentation block to create new training data. At the same time, two loss function viz., cross-entropy loss and mutual channel loss are combined to optimize the proposed model.
The main contributions of this paper are: (1) We propose a novel modified CNN based model to classify the defective castings in x-ray images. The proposed network is able to extract features automatically merely on the image-level annotations and learn the attention maps of the castings. (2) We test the effectiveness of the proposed attention-guided data augmentation and mutual-channel loss through the ablation experiment. The results show that the proposed model increase the performance of the baseline model to a large extent.
The research paper is organized into five sections: Section 1 introduces the topic and explains its importance. Section 2 explores work related to the proposed model. Section 3 explains the methodology of the paper, explaining the dataset and the specific CNN used in this paper. Section 4 presents the experimental results achieved by this model and explain the details. Section 5 concludes the research paper.
Section snippets
Related work
The task of classifying casting x-ray images into defective and non-defective classes has been significantly explored. In the early works, the effectiveness of CNN based models used to detect the defects are proved by researchers. Mery Domingo [11] developed an Xnet model which simplifies the CNN for the defect detection task. It only has a five weight-layer net and ten layers in total. Additionally, they included a dropout block that randomly turns off connections of the neural network during
Methodology
In this section, we introduce a modified CNN architecture combined with attention-guided data augmentation and mutual-channel (MC) loss function. As shown in Fig. 1, after obtaining the feature maps, we add a 1 × 1 convolution layer to generate the attention maps. Then for each training image, we randomly choose one of its attention maps to guide the data augmentation process. At last, the obtained attention maps enter into two streams of the network. There are two different sub-loss tailored
Experiments
In this section, our model is trained and tested on the x-Ray Images (Casting) dataset consisting of 6992 images for training, 1000 images for validation, and 1000 images for testing the model (Table 2). Accuracy, Recall, Precision, and F1 score are used to measure the performance of the model. It is highly significant to select an appropriate evaluation criterion for the classification task. Accuracy is the classification accuracy of the model. Recall measures the performance evaluation in the
Conclusion
This research paper presents a weakly supervised CNN model (MC-DAN) which combines the attention-guided data augmentation and mutual-channel loss function. Weakly supervised learning provides the spatial distribution of the defects for data augmentation which encourages the network to learn from multiple discriminative features. Simultaneously, the added mutual-channel loss focuses more on the discriminative features through diving into the feature channels before they fused. The image-level
CRediT authorship contribution statement
Lili Jiang: Conceptualization, Methodology, Software, Validation, Writing - original draft, Writing - review & editing, Visualization. Yongxiong Wang: Writing - review & editing, Supervision. Zhenhui Tang: Data curation, Writing - review & editing, Visualization. Yinlong Miao: Supervision. Shuyi Chen: Resources.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This paper is financially supported by the National Science Foundation of China (No. 61673276).
References (36)
- et al.
Non-destructive image processing based system for assessment of rice quality and defects for classification according to inferred commercial value
Measurement
(2019) - et al.
Multi-site structural damage identification using a multi-label classification scheme of machine learning
Measurement
(2020) - et al.
Detection and classification of surface defects of gun barrels using computer vision and machine learning
Measurement
(2015) - et al.
Automated detection of welding defects in pipelines from radiographic images dwdi
NDT E Int.
(2017) - et al.
Optimized unsupervised deep learning assisted reconstructed coder in the on-nodule wearable sensor for human activity recognition
Measurement
(2020) - et al.
Classification of flower species by using features extracted from the intersection of feature selection methods in convolutional neural network models
Measurement
(2020) - et al.
Railway subgrade defect automatic recognition method based on improved faster r-cnn
Sci. Program.
(2018) - et al.
Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques
Autom. Constr.
(2018) - et al.
Evaluation of turned and milled surfaces roughness using convolutional neural network
Measurement
(2020) - et al.
Classification of mammogram for early detection of breast cancer using svm classifier and hough transform
Measurement
(2019)
Online detection for bearing incipient fault based on deep transfer learning
Measurement
Transfer learning and deep convolutional neural networks for safety guardrail detection in 2d images
Autom. Constr.
Defects, root causes in casting process and their remedies: Review
Int. J. Eng. Res. Appl.
Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning
Comput. Vis. Pattern Recognit.
Rts-elm: an approach for saliency-directed image segmentation with ripplet transform
Pattern Anal. Appl.
High-performance neural networks for visual object classification
Artificial Intelligence
Automatic defect recognition in x-ray testing using computer vision
Automatic localization of casting defects with convolutional neural networks
Cited by (36)
ADNet: Anti-noise dual-branch network for road defect detection
2024, Engineering Applications of Artificial IntelligenceGDALR: Global Dual Attention and Local Representations in transformer for surface defect detection
2024, Measurement: Journal of the International Measurement ConfederationMachine learning method for estimating the defect-related mechanical properties of additive manufactured alloys
2023, Engineering Fracture MechanicsTeacher–student network for 3D point cloud anomaly detection with few normal samples
2023, Expert Systems with Applications