Super-resolution visualization of subwavelength defects via deep learning-enhanced ultrasonic beamforming: A proof-of-principle study

Abstract

Detecting small, subwavelength defect has known to be a challenging task mainly due to the diffraction limit, according to which the minimum resolvable size is in the order of the wavelength of a propagating wave. In this proof-of-concept study, we present a deep learning-enhanced super-resolution ultrasonic beamforming approach that computationally exceeds the diffraction limit and visualizes subwavelength defects. The proposed super-resolution approach is a novel subwavelength beamforming methodology enabled by a hierarchical deep neural network architecture. The first network (the detection network) globally detects defective regions from an ultrasonic beamforming image. Subsequently, the second network (the super-resolution network) locally resolves subwavelength-scale fine details of the detected defects. We validate the proposed approach using two independent datasets: a bulk wave array dataset generated by numerical simulations and guided wave array dataset generated by laboratory experiments. The results demonstrate that our deep learning super-resolution ultrasonic beamforming approach not only enables visualization of fine structural features of subwavelength defects, but also outperforms the existing widely-accepted super-resolution algorithm (time-reversal MUSIC). We also study key factors of the performance of our approach and discuss its applicability and limitations.

Introduction

Ultrasonic testing has proven during the last decades to be an effective nondestructive evaluation (NDE) tool to inspect various engineering materials and structural elements owing to the sensitivity of ultrasonic waves to mechanical condition changes [[1], [2], [3], [4]]. An effective way to deliver ultrasonic inspection results is an image of the inspected structure, where local mechanical inhomogeneities such as damage are identified and located. Various ultrasonic imaging methods have been investigated including tomography [[5], [6], [7], [8]], array imaging [1,[9], [10], [11], [12], [13], [14], [15]] and wavefield imaging [[16], [17], [18], [19], [20]]. Among various imaging methods, the ultrasonic array imaging is the particular interest of this study due to its ability to image a large inspection area with limited access.

Ultrasonic array imaging can be carried out by beamforming where the array elements are excited in a predefined sequence with time delays (termed as delay law) to form an ultrasonic beam; the same delay law is typically applied when receiving signals [13]. The received signals are then summed to form an image in which inhomogeneities are indicated as high pixel intensities. Alternatively, post-processing type beamforming can be carried out by applying the same delay and sum (DAS) principle to the full matrix of ultrasonic transmit-receive array data [9]. The most widely used DAS-based post-processing algorithm is arguably the total focusing method (TFM), where synthetical ultrasonic beam focusing is performed every pixel within an imaging plane [9,10]. The TFM provides higher resolution than other beamforming algorithms and robust performance against noise. Despite the advantages, the resolution of the TFM is limited, as with other imaging methods, by the Rayleigh criterion (diffraction limit) according to which the minimum resolvable size is in the order of the wavelength of a propagating wave. To image smaller defects, short wavelengths (or high frequencies) need to be employed. However, when the frequency is increased to employ a shorter wavelength, the propagating wave suffers higher attenuation that limits the imaging range that can be covered. Implementing high-power equipment that increases the excitation energy and thus imaging range is costly and sometimes risky. Moreover, as the excitation frequency increases, the coherent noise due to material backscattering and the emergence of multiple scattering limits the upper bound of frequency. Therefore, subwavelength imaging that could somehow overcome the diffraction limit is an important problem in modern ultrasonic array imaging approaches.

Research has been conducted to overcome the diffraction limit and achieve super-resolution through hardware enhancements or computational approaches. In the former case, novel ultrasonic transducers have also been developed; for example, recent advances in hyperlens technology powered by engineered metamaterials have enabled overcoming the diffraction limit and achieving super-resolution imaging [[21], [22], [23], [24], [25]]. High wavenumber evanescent waves that carry the subwavelength information, not measurable using the traditional array technology, are suitably detected using metamaterial-based acoustic hyperlens. Notably, it has been demonstrated that metamaterial-based hyperlens can achieve super-resolution down to a feature size of 1/50 of wavelength [23]. Despite the high potential, the practicality of the hyperlens technology is yet to be investigated for industrial NDE applications, in part because of its complexity imposed by designing and manufacturing metamaterials for hyperlenses as well as the need of hardware configuration change from existing industry standard setup to incorporate hyperlenses. On the other hand, computational approaches, such as the time-reversal methods combined with the MUSIC (multiple signal classification) algorithm, known as TR MUSIC, have shown to be effective in imaging subwavelength inhomogeneities [[26], [27], [28], [29]]. A significant advantage of the TR MUSIC approach is that the same form of ultrasonic array data used for the TFM can be used, and super-resolution imaging is achieved by post-processing: no modification in the array data acquisition system is further needed. However, the TR MUSIC approach is known to be susceptible to noise, and its subwavelength imaging performance is compromised under low signal-to-noise ratio (SNR) [10,30].

In this study we leverage the recent breakthroughs in the deep learning to computationally achieve super-resolution ultrasonic visualization of subwavelength defects. Deep learning is a computational methodology to learn the complicated nonlinear mapping between inputs and desired output pairs using multilayered artificial neural networks or their variants [31]. With its superior modeling ability, deep learning enables us to build a new ultrasonic array data processing framework that maps a low-resolution array image to a super-resolution output image that renders visualization of subwavelength defects. In optical and biomedical imaging fields, deep learning techniques, for example, have been applied to resolution improvement in fluorescence microscopy [32], high-resolution X-ray tomography [33], photoacoustic tomography [34] and super-resolution ultrasound localization microscopy [35]. Recently, applying deep learning technology to NDE areas (non-super-resolution) has also been demonstrated [[36], [37], [38], [39]]. Despite wide successful implementations of deep learning in other imaging-related fields, to the authors’ best knowledge, deep learning-based super-resolution ultrasonic array imaging for subwavelength defects has not studied yet in NDE fields.

In this proof-of-concept study, we present a deep learning-enhanced super-resolution ultrasonic beamforming approach for visualization of subwavelength defects. It is a generic array image processing framework that can be applied to both bulk wave and guided wave applications. Specifically, the proposed super-resolution approach is a novel hierarchical, multi-scale visualization methodology that combines two distinct convolutional neural networks (CNNs). The first network (named as the detection network) globally detects defective regions from a raw TFM image, and subsequently, the detected defective image is fed to the second network (the super-resolution network) that locally resolves subwavelength-scale fine details of the detected defects. The proposed approach provides the following unique advantages: (1) no modification to existing ultrasonic array data acquisition hardware is needed to achieve super-resolution; (2) a robust super-resolution performance against noise is achieved; and (3) a large volume of datasets for training CNNs are easily generated with a few array beamforming images.

The remainder of this paper is organized as follows. Section 2 describes the TFM and its achievable imaging resolution. Section 3 describes details of the proposed deep learning-enhanced super-resolution ultrasonic beamforming approach and training strategies. Section 4 provides numerical (bulk waves) and laboratory experimental (guided waves) validations of the proposed approach. In Section 5, important aspects of the proposed approach such as performance comparison with an existing super-resolution approach are discussed.