Super-resolution visualization of subwavelength defects via deep learning-enhanced ultrasonic beamforming: A proof-of-principle study
Introduction
Ultrasonic testing has proven during the last decades to be an effective nondestructive evaluation (NDE) tool to inspect various engineering materials and structural elements owing to the sensitivity of ultrasonic waves to mechanical condition changes [[1], [2], [3], [4]]. An effective way to deliver ultrasonic inspection results is an image of the inspected structure, where local mechanical inhomogeneities such as damage are identified and located. Various ultrasonic imaging methods have been investigated including tomography [[5], [6], [7], [8]], array imaging [1,[9], [10], [11], [12], [13], [14], [15]] and wavefield imaging [[16], [17], [18], [19], [20]]. Among various imaging methods, the ultrasonic array imaging is the particular interest of this study due to its ability to image a large inspection area with limited access.
Ultrasonic array imaging can be carried out by beamforming where the array elements are excited in a predefined sequence with time delays (termed as delay law) to form an ultrasonic beam; the same delay law is typically applied when receiving signals [13]. The received signals are then summed to form an image in which inhomogeneities are indicated as high pixel intensities. Alternatively, post-processing type beamforming can be carried out by applying the same delay and sum (DAS) principle to the full matrix of ultrasonic transmit-receive array data [9]. The most widely used DAS-based post-processing algorithm is arguably the total focusing method (TFM), where synthetical ultrasonic beam focusing is performed every pixel within an imaging plane [9,10]. The TFM provides higher resolution than other beamforming algorithms and robust performance against noise. Despite the advantages, the resolution of the TFM is limited, as with other imaging methods, by the Rayleigh criterion (diffraction limit) according to which the minimum resolvable size is in the order of the wavelength of a propagating wave. To image smaller defects, short wavelengths (or high frequencies) need to be employed. However, when the frequency is increased to employ a shorter wavelength, the propagating wave suffers higher attenuation that limits the imaging range that can be covered. Implementing high-power equipment that increases the excitation energy and thus imaging range is costly and sometimes risky. Moreover, as the excitation frequency increases, the coherent noise due to material backscattering and the emergence of multiple scattering limits the upper bound of frequency. Therefore, subwavelength imaging that could somehow overcome the diffraction limit is an important problem in modern ultrasonic array imaging approaches.
Research has been conducted to overcome the diffraction limit and achieve super-resolution through hardware enhancements or computational approaches. In the former case, novel ultrasonic transducers have also been developed; for example, recent advances in hyperlens technology powered by engineered metamaterials have enabled overcoming the diffraction limit and achieving super-resolution imaging [[21], [22], [23], [24], [25]]. High wavenumber evanescent waves that carry the subwavelength information, not measurable using the traditional array technology, are suitably detected using metamaterial-based acoustic hyperlens. Notably, it has been demonstrated that metamaterial-based hyperlens can achieve super-resolution down to a feature size of 1/50 of wavelength [23]. Despite the high potential, the practicality of the hyperlens technology is yet to be investigated for industrial NDE applications, in part because of its complexity imposed by designing and manufacturing metamaterials for hyperlenses as well as the need of hardware configuration change from existing industry standard setup to incorporate hyperlenses. On the other hand, computational approaches, such as the time-reversal methods combined with the MUSIC (multiple signal classification) algorithm, known as TR MUSIC, have shown to be effective in imaging subwavelength inhomogeneities [[26], [27], [28], [29]]. A significant advantage of the TR MUSIC approach is that the same form of ultrasonic array data used for the TFM can be used, and super-resolution imaging is achieved by post-processing: no modification in the array data acquisition system is further needed. However, the TR MUSIC approach is known to be susceptible to noise, and its subwavelength imaging performance is compromised under low signal-to-noise ratio (SNR) [10,30].
In this study we leverage the recent breakthroughs in the deep learning to computationally achieve super-resolution ultrasonic visualization of subwavelength defects. Deep learning is a computational methodology to learn the complicated nonlinear mapping between inputs and desired output pairs using multilayered artificial neural networks or their variants [31]. With its superior modeling ability, deep learning enables us to build a new ultrasonic array data processing framework that maps a low-resolution array image to a super-resolution output image that renders visualization of subwavelength defects. In optical and biomedical imaging fields, deep learning techniques, for example, have been applied to resolution improvement in fluorescence microscopy [32], high-resolution X-ray tomography [33], photoacoustic tomography [34] and super-resolution ultrasound localization microscopy [35]. Recently, applying deep learning technology to NDE areas (non-super-resolution) has also been demonstrated [[36], [37], [38], [39]]. Despite wide successful implementations of deep learning in other imaging-related fields, to the authors’ best knowledge, deep learning-based super-resolution ultrasonic array imaging for subwavelength defects has not studied yet in NDE fields.
In this proof-of-concept study, we present a deep learning-enhanced super-resolution ultrasonic beamforming approach for visualization of subwavelength defects. It is a generic array image processing framework that can be applied to both bulk wave and guided wave applications. Specifically, the proposed super-resolution approach is a novel hierarchical, multi-scale visualization methodology that combines two distinct convolutional neural networks (CNNs). The first network (named as the detection network) globally detects defective regions from a raw TFM image, and subsequently, the detected defective image is fed to the second network (the super-resolution network) that locally resolves subwavelength-scale fine details of the detected defects. The proposed approach provides the following unique advantages: (1) no modification to existing ultrasonic array data acquisition hardware is needed to achieve super-resolution; (2) a robust super-resolution performance against noise is achieved; and (3) a large volume of datasets for training CNNs are easily generated with a few array beamforming images.
The remainder of this paper is organized as follows. Section 2 describes the TFM and its achievable imaging resolution. Section 3 describes details of the proposed deep learning-enhanced super-resolution ultrasonic beamforming approach and training strategies. Section 4 provides numerical (bulk waves) and laboratory experimental (guided waves) validations of the proposed approach. In Section 5, important aspects of the proposed approach such as performance comparison with an existing super-resolution approach are discussed.
Section snippets
Total focusing method (TFM) and its achievable imaging resolution
In this study, the classic TFM is used as the preprocessing step to create a raw defect image. The TFM is a DAS-based post-processing array imaging algorithm, where an ultrasound beam is synthetically focused on each image pixel in the imaging region. Fig. 1 shows the schematic illustration of the TFM. First, ultrasonic array data is obtained through FMC, where an ultrasonic transducer from a transducer array is individually excited to generates incident ultrasonic waves while all the
Deep learning-enhanced super-resolution ultrasonic beamforming approach
In this section, the proposed deep learning-enhanced super-resolution ultrasonic beamforming approach to characterize subwavelength defects is described.
Numerical simulations and laboratory experiments for dataset generation
Two separate ultrasonic array datasets were configured to validate the detection and super-resolution networks described in the previous section: a bulk wave array dataset generated by numerical simulations and guided wave array dataset by laboratory experiments. Without losing validity, an experimental setup that is different from the numerical simulation setup is used in this study mainly due to limited experimental resources.
Comparison with an existing super-resolution approach
In Section 3, numerical simulation and experimental results demonstrate that super-resolution ultrasonic beamforming approach that overcomes the diffraction limit can be achieved through the a hierarchical, multi-scale deep learning approach. Here, we compare the performance of the proposed deep learning-based super-resolution approach with the TR MUSIC that is arguably one of the most widely accepted super-resolution array imaging approaches. Numerical simulation data obtained from the model
Conclusions and future work
This study presents a deep learning-enhanced super-resolution ultrasonic beamforming approach to visualize subwavelength defects. Based on the numerical and experimental validation results, the following conclusions are drawn:
- (1)
The proposed deep learning-enhanced ultrasonic beamforming approach enables computationally overcoming the diffraction limit and hence achieves super-resolution in the defect image.
- (2)
The proposed approach outputs a super-resolution defect image with high contrast between an
CRediT authorship contribution statement
Homin Song: Methodology, Visualization, Investigation, Formal analysis, Data curation, Software, Validation, Writing - original draft. Yongchao Yang: Methodology, Investigation, Conceptualization, Supervision, Project administration, Funding acquisition, Writing - review & editing.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research was partially funded by the Physics of Artificial Intelligence Program of Defense Advanced Research Projects Agency (DARPA).
References (46)
- et al.
Ultrasonic arrays for non-destructive evaluation: a review
NDT E Int
(2006) Structural health monitoring with piezoelectric wafer active sensors
(2008)- et al.
Guided Lamb waves for identification of damage in composite structures: a review
J Sound Vib
(2006) - et al.
Noncontact ultrasonic nondestructive techniques: state of the art and their use in civil engineering
J Infrastruct Syst
(2016) - et al.
High-resolution guided wave tomography
Wave Motion
(2013) - et al.
NDE application of ultrasonic tomography to a full-scale concrete structure
IEEE Trans Ultrason Ferroelectrics Freq Contr
(2015) - et al.
Guided wave diffraction tomography within the born approximation
IEEE Trans Ultrason Ferroelectrics Freq Contr
(2010) - et al.
Guided wave tomography based on least-squares reverse-time migration
Struct Health Monit
(2019) - et al.
Post-processing of the full matrix of ultrasonic transmit-receive array data for non-destructive evaluation
NDT E Int
(2005) - et al.
A comparison between ultrasonic array beamforming and super resolution imaging algorithms for non-destructive evaluation
Ultrasonics
(2014)
In situ 2-D piezoelectric wafer active sensors arrays for guided wave damage detection
Ultrasonics
Pulsed laser-scanning laser Doppler vibrometer (PL-SLDV) phased arrays for damage detection in aluminum plates
Mech Syst Signal Process
Ultrasonic arrays: a comparison between medical and NDE requirements
AIP Conf Proc
Damage localisation in plate-like structures based on PZT sensors
Mech Syst Signal Process
Experimental and numerical validation of guided wave phased arrays integrated within standard data acquisition systems for structural health monitoring
Struct Contr Health Monit
Frequency-wavenumber domain filtering for improved damage visualization
Smart Mater Struct
Frequency-wavenumber domain methods for analysis of incident and scattered guided wave fields
Heal Monit Struct Biol Syst
Structural imaging through local wavenumber estimation of guided waves
NDT E Int
Complete noncontact laser ultrasonic imaging for automated crack visualization in a plate
Smart Mater Struct
Identification of cracks in thin-walled structures by means of wavenumber filtering
Mech Syst Signal Process
Superlenses to overcome the diffraction limit
Nat Mater
Hyperlenses and metalenses for far-field super-resolution imaging
Nat Commun
A holey-structured metamaterial for acoustic deep-subwavelength imaging
Nat Phys
Cited by (35)
Metamaterial enhanced subwavelength imaging of inaccessible defects in guided ultrasonic wave inspection
2024, NDT and E InternationalGreen function based ultrasonic super resolution imaging for defect location and characterization
2024, Mechanical Systems and Signal ProcessingLamb wave phased array imaging based on phase-amplitude compounding algorithm
2023, Mechanical Systems and Signal ProcessingOn the ultrasonic characterization of the stacking sequence of CFRP laminates
2023, NDT and E International