Assessment methodology for defect characterisation using ultrasonic arrays. NDT

There has been a rapid increase in the use of ultrasonic arrays for non-destructive evaluation in recent years and new methods for defect characterisation are now emerging. However, it is also known that defects can show a very different re ﬂ ectivity depending on their relative location with respect to the array. In this paper, a mapping approach is introduced to evaluate the spatial performance of characterisation methods against a range of key variables including crack size and orientation, as well as to explore the in ﬂ uence of structural noise. This spatial method takes advantage of computer power and fast hybrid modelling techniques to simulate crack-like defects at different locations on a mesh-grid in front of the array and apply the characterisation method of interest to each simulated defect separately. As a case study, the spatial mapping procedure is applied to a characterisation method based on the measurement of the scattering matrices and comparison with a pre-computed database. Dramatic spatial performance variations are observed in the simulations and this is corroborated experimentally. These performance varia- tions are explained by a combination of the defect signal-to-noise-ratio (SNR) and the feature density of the scattering matrix (S-matrix) of the defect. Optimal characterisation performance is achieved when both the SNR and the S-matrix feature density are high.


Introduction
When solid structures go through normal operating conditions such as temperature and humidity fluctuations and fatigue loading, defects such as cracks can be produced that can eventually cause structural failure [1]. Therefore, reliable inspection and maintenance during manufacture and in-service operation is crucial to ensure safety and to protect the environment against catastrophic failures [2]. Detection and characterisation of crack-like defects can also enhance the prediction of the remaining life of the component in structural integrity assessments.
When ultrasonic waves illuminate a defect, the scattered signals potentially contain information relating to the location, shape, size and orientation angle [3]. This information can be captured either by using multiple conventional monolithic transducers from different inspection configurations [4][5][6][7] or with ultrasonic arrays [8][9][10][11][12][13]. In recent years, due to their flexibility and greater imaging performance compared to conventional monolithic transducers, ultrasonic arrays have seen a rapid increase in their non-destructive evaluation (NDE) applications and are now regularly used in industry [8]. A single linear ultrasonic array can be used to undertake several different inspections and produce real-time 2D images of the test structures. Ultrasonic arrays were originally used in the field of medical diagnostics and then implemented in the NDE field [14]. Flexible and high temperature arrays are also being developed for complex geometries and extreme conditions [15,16]. Current array controller systems are designed to excite the elements with programmable time delays and hence simulate multiple monolithic transducers, such as plane beams, steered angled beams and focused beams [17,18]. Enhanced resolution has been shown with super resolution techniques developed for radar [19], inverse wave field extrapolation from the seismic field [20], synthetic aperture focusing from the sonar and medical fields [21,22] and total focusing method (TFM) [23]. In an inspection using the TFM imaging algorithm, the full array data set, termed the full matrix capture (FMC) [23], is first recorded and then post-processed to create an image to detect a defect.
There have been variety of methods introduced to characterise cracklike defects using ultrasonic arrays, such as image-based characterisations (for larger defects) [24] and characterisation using the scattering coefficient matrix (S-matrix) (for smaller defects) [25][26][27]. However, the robustness of these techniques is currently unknown and there is no agreed methodology for assessing their performance. Instead, these methods are typically assessed by simply applying them to a few specific defects located at specific positions and it is assumed that the performance is similar for other defects at other locations [26][27][28].
This paper aims to introduce a spatial assessment method to map the performance of characterisation algorithms against the key variables that affect their accuracy, such as the location, size and orientation angle of crack-like defects, as well as the extent of any structural noise. This will enable us to understand where the methods work best and where they fail as well as providing a basis for quantifying future developments.
As a case study, a recently introduced characterisation method [26], developed to characterise small crack-like defects by measuring the S-matrix of a defect and comparing it to a large database of pre-computed S-matrices for varying crack-like defects (termed as the database similarity metric method), was assessed and the key factors governing its performance were identified. Finally, the simulations are verified by comparing the characterisation performance of simulated crack-like defects with experimental measurements.

Hybrid forward wave scattering model
In order to investigate the performance of a given characterisation method, a fast hybrid forward model is used to simulate a defect at any given location relative to the array. As shown in Fig. 1, the simulation assumes a two-dimensional (2-D) geometry and planar crack-like defects of negligible width, with two key characteristics of size (length) and orientation angle (respectively l and α as shown in Fig. 1). The orientation angle of the crack-like defect refers to the anti-clockwise angle from the horizontal and can vary from Àπ = 2 to π = 2. The figure also shows the wave path from an array transmitter element to the defect and its return path to a receiver element. Assuming the defect is in the far field of the array elements and the array elements are in the far field of the defect, the received signal in the frequency-domain, transmitted from the i th element and received by the j th element can be expressed as [29]: where F 0 is the frequency spectrum of the signal transmitted into the test sample, λ is the wavelength, R T and R R are the distances from the defect to the transmitting and receiving elements respectively, D f is the product of the impulse response function and the element directivity function [8,30], S is the defect S-matrix, θ in and θ out are incident and scattering angles to and from the crack-like defect and τ is the traveling time of the signal from transmitter to the defect centre and back to the receiver. The model is termed hybrid as the S-matrix is calculated by a 2-D semi-analytical boundary integral method developed in [31]. Note that equation (1) is a general model and hence true for any wave mode and mode conversions, however, in this paper only the longitudinal wave components (i.e. L-L scattering) are used.
An inverse Fourier transform is then applied to the result of equation (1) to obtain the time-domain signal f ij ðtÞ. This procedure is then carried out for every pair of elements to create a FMC array data set for a given defect location. A linear ultrasonic array is used throughout with parameters given in Table 1. Fig. 2 shows TFM images of two 30 , 1.5 mm length crack-like defects simulated at two different locations with respect to the array. The model is fast and the FMC data set required for Fig. 2 was computed in 2s on a standard desktop PC. From the image, it is apparent that the defect response is strongly dependent on location and this makes reliable characterisation challenging and is at the root of the spatial characterisation performance variations discussed in this paper.

Spatial assessment of characterisation methods
An assessment method is now developed to spatially map how any given characterisation algorithm performs on a specific type of defect. Firstly, a given defect is simulated, located in turn on the nodes of a 2-D grid in front of the array (as shown in Fig. 1). At each grid-node, a FMC array data set is generated, the characterisation algorithm is applied, and the result is compared to the known true defect, creating a performance map. Maps can then be generated for different conditions such as defect type, material noise and array type to see the effect of each variable on the performance. This assessment methodology is now introduced and  (1). 2D space in front of the array is discretised into a grid.  demonstrated using a case study on the database similarity metric method [26] which is a characterisation method that has shown promise for small crack-like defects.

S-matrix extraction
In the database similarity metric method, the S-matrix is extracted from FMC array data set in the time domain using the subarray approach [23,25]. It is crucial that the S-matrix extraction is as reliable, accurate and fast.
Prior to S-matrix extraction, a de-noising band pass filter is applied to the FMC data. Then array elements are grouped as subarrays, each consisting of 8 elements. For every pair of subarrays, the arrival time of the signal between the subarray centres and the point of interest is calculated and the corresponding value in f ij ðtÞ is taken. Knowing the location of the point where S-matrix is being measured from (typically a maximum in the TFM image), every transmitting subarray then corresponds to an incident angle and likewise, every receiving subarray corresponds to a scattering angle. It should be noted that the size of the subarray (i.e. the number of elements in each subarray) is a compromise between spatial resolution (i.e. a larger subarray leads to better focusing) and an angular blurring effect (i.e. a larger subarray averages the S-matrix over a wider range of angles). The signals received are normalised to compensate for the effects of the directivity functions, D f ðφ T Þ and D f ðφ R Þ and beam spread, 1 ffiffiffiffiffiffiffiffi RT RR p and the measured S-matrix value is taken as the magnitude of the subarray TFM image at the defect (see Ref. [25] for fuller details).

Spatial mapping assessment method
In order to assess the S-matrix extraction algorithm, firstly simulations are performed in a noise free medium using equation (1). The measured S-matrix, S m is extracted as explained in section 3.1 from this noise free FMC data set at the true centre of the simulated crack-like defect. Then S m is compared to the true S-matrix in the centre frequency (S t ) of the simulated defect using correlation coefficient [26]: where S 1 and S 2 are the two S-matrices to be compared, N is the total number of corresponding incident/scattering angles (i.e. number of subarrays), S 1 and S 2 are of size N Â N, and S 1 and S 2 are the mean values of the matrices S 1 and S 2 respectively. By using equation (2) and replacing S 1 and S 2 with S m and S t , a correlation coefficient (CC) can be calculated for each grid point in front the array. Fig. 3 demonstrates spatial maps of the CC between S m and S t for a simulated crack-like defect with a size of 1 mm and orientation angle of 30 . In Fig. 3 (a-d), Hanning filters of different bandwidths are used, from which it can be seen that a narrower bandwidth, results in better correlation, which would lead to better characterisation. This is due to the fact that the S-matrices in the database are calculated at a single central frequency and so, as the bandwidth of the simulated measurement narrows, the extracted S-matrix tends to the single frequency case. From Fig. 3, it is also shown that whilst the filter bandwidth has a major impact on a specific area, much of the region in front of the array is unaffected. We note that the region of low correlation is dependent on the orientation angle of the defect, presumably because the bandwidth has an effect on specific regions of the S-matrix. For the remainder of the paper, a bandwidth of 15% was used as a good compromise between imaging resolution and characterisation performance.

Assessment of characterisation methods
Spatial mapping can be used to quantitatively assess performance of characterisation methods, in which case the measured characteristics of each simulated defect (such as size and orientation angle) are compared with the true characteristics and the error between them is used in the mapping. The error in size of a crack-like defect is given by: where l m is the measured length of the crack-like defect using a given characterisation algorithm and l t is the true length. Similarly, the error in orientation angle of a crack-like defect is given by: where α m is the measured orientation angle of the crack-like defect using a characterisation algorithm and α t is the true orientation angle. e α is always the smallest angular difference between the true and characterised orientation angles, which is necessarily equal or less than π = 2.
This assessment can also be performed in the presence of various forms of noise, such as coherent grain scattering noise, to assess the sensitivity of characterisation methods to the noise. Here, we add experimentally measured grain noise to the simulated FMC data in the time domain [32]. This results in a FMC data set that contains a simulated defect in a noisy medium. We note that the simple addition approach used here inherently ignores multiple scattering between the defect and the grains.
The database similarity metric method [26] requires the formation of a database of analytically modelled S-matrices [31] for a range of different defects with varying sizes (from 0.1 mm to 3.0 mm) and orientation angles (from Àπ   5. Compare the S-matrix from step 3 to all S-matrices in the database created in step 4 using equation (2). This gives a CC value for every defect in the database. 6. Characterise defect as the crack-like defect in database, with which the measured S-matrix has the maximum CC. Fig. 4 illustrates the spatial performance of the database similarity metric method. Fig. 4 (a) and (b) show the error map in size and orientation angle for a simulated crack-like defect (l ¼ 1 mm, α ¼ 60 ) respectively. Fig. 4 (c) and 4 (d) illustrate the same defects with presence of grain noise extracted from bright mild steel (080A15). By comparing 4 (a) with 4 (c) and 4 (b) with 4 (d), the effect of noise at different locations can be observed.
The effect of noise on the defect characterisation performance is further investigated through a range of noise levels by considering the noise intensity distribution (measured from the TFM images of specimens without any defects) of four different materials: bright mild steel (080A15), austenitic stainless steel, mild steel (EN24) and Waspaloy (UNS N07001). For each material, FMC data at six different locations were recorded. It was found that the shape of the TFM intensity distribution was similar for these different materials and so in the remainder of this paper, the noise was first measured from bright mild steel (080A15) and then multiplied by various amplification factors to investigate the effect of reduced/increased noise levels on characterisation performance. Spatial error maps were created for the six different realisations of noise and then an average of these error maps was taken. Fig. 5 shows the spatial error in defect orientation angle in presence of a range of noise levels and for different crack sizes and orientation angles. Each column represents a specific level of grain noise starting from 0, where simulated defects are noise free. Noise level of 1 represents the grain noise from a bright mild steel (080A15). The other two noise levels show the noise amplitude multiplication factor. In Fig. 5 (a), each row represents a specific orientation angle and the size was kept constant at 1 mm. In Fig. 5 (b) however, the orientation angle is kept constant at 30 and each row represents different crack lengths. Fig. 6 is similar to Fig. 5, except that it shows performance in characterising the size of crack-like defects. Taken together, Figs. 5 and 6 give an overall spatial view of the performance of the database similarity metric method against structural noise and defect type. The next section seeks to understand the reasons for the form of these spatial performance maps.

Discussion
As it can be seen from Figs. 5 and 6, the patterns of maps show distinctive shapes. For example, by looking at Figs. 5b and 6b, the characterisation performance is better for larger crack-like defects, which can be expected as the reflected signals will be larger and therefore the characterisation becomes less sensitive to noise. However, it can also be seen that the characterisation performance of the larger defects is still sensitive to the location of defect. This can be explained by considering the SNR for each point in the spatial map (x; z) which can be written as [33]: where qðx; zÞ is the maximum TFM amplitude from the defect (i.e.  Fig. 7(a) to (d) show the spatial maps of SNR, calculated from equation (5), for crack-like defects of size 1 mm and various orientation angles. Fig. 7 suggests that the general patterns of the error maps in Figs. 5a   and 6a, with high levels of noise, can be explained by the defect SNR. This will, for example, mean that crack-like defects are better characterised when the array receives the high amplitude (and hence high SNR) specular reflection for the crack face. This is most obvious when comparing the columns on the right-hand side of Figs. 5a and 6a, where noise level is highest, with Fig. 7.
It is also important that the measured S-matrices correlate well with a single element of the database and poorly with the other elements. We hypothesise that this is most likely to happen when the S-matrices are information rich, which means that the S-matrix varies (i.e. has distinct features) over the measured angular range. Conversely, if the measured S-matrix is constant over the angular range, this would be information poor and hence it is likely to show a high correlation with other similar shaped S-matrices, including the correct S-matrix in database. This is analogous to the situation in voice recognition and fingerprinting, where the recorded voice or the scanned fingerprint is checked with a large database to find the best match [35][36][37][38]. The more features in the captured data, the more likely to find the right match. One simple way to quantify this uniqueness, is to see how many S-matrices in the database have a CC with the measured S-matrix equal or greater than a specific threshold in a noise-free condition. The more database S-matrices above this threshold, the more likely the characterisation method is to fail in presence of noise. Fig. 8 illustrates this by taking the number of S-matrices in database whose CC with the measured S-matrix is above 0.99 of the maximum CC with the entire database (or within 1% from the max CC). This metric (termed the 1% population) can be used to measure uncertainty in both size and orientation angle characterisation. The left column of Fig. 8 shows uncertainty in size characterisation (corresponding to Fig. 6 (a)), and the right column demonstrates uncertainty in orientation angle characterisation (corresponding to Fig. 5 (a)). Fig. 8 suggests that characterisation in the regions with large 1% population (blue regions) should be more sensitive to the material noise. This information uniqueness metric again favours crack-like defects located such that the array receives the specular reflection, however, other regions also have high uniqueness leading to features, such as the three-pronged shapes seen in Figs. 5a and 6a.

Experimental results
The proposed spatial mapping assessment method is now validated through experimental measurements. Fig. 9 shows a comparison between the error in characterising simulated crack-like defects and characterising real experimental crack-like defects (EDM notches cut with a 0.1 mm thick wire) in bright mild steel (080A15). The details of the experimental defects are shown in Table 2. It should be noted that it is difficult to manufacture real cracks with different orientation angles at different locations. Instead, EDM notches were used as the closest manufactured crack-like defects. In order to plot the error in characterising simulated crack-like defects, 16 coherent noise realisations have been used for each location along x-axis and the maximum error (dashed lines), minimum error (dotted lines) and mean error (solid lines) are plotted in increments of 3 mm along x-axis. The experiment setup was as shown in Fig. 1 (array parameters in Table 1) and measurements along the x-axis were taken in increments of 15 mm by moving the array from the far right (such that the defect is located at À45 mm in x-axis from the centre of the array) to far left (such that the defect is located at 45 mm in x-axis from the centre of the array). Each error bar represents 5 measurements along the y-direction (in thickness direction) and shows the mean error of the 5 measurements and the maximum and minimum errors. The left and right columns in Fig. 9 show error in size and orientation angle respectively. Rows one to four from the top represent defects A to D respectively (see Table 2). Fig. 9 suggests a good overall agreement between the experimental and simulated characterisation errors. The small differences seen, are thought to be due to the assumption in simulations that crack-like defects have zero width. So, the diffraction from experimental notch tip would be slightly different from the zero-width simulated crack-like defect tips. Experimental results can also be affected by the steel block's surface and coupling conditions at specific locations and the spatial variation in grain noise.

Conclusions
A spatial mapping approach is introduced to assess the performance of characterisation methods in the imaging plane of a linear array. This method takes advantage of modern computer power to rapidly simulate the FMC data from arbitrary defects at arbitrary locations and then apply the characterisation method of interest to each simulated defect. As a case study, the performance of the database similarity metric method for the characterisation of small crack-like defects has been examined. This characterisation method works by measuring the S-matrices from defects and comparing them to a database of pre-computed S-matrices. Using spatial assessment, the key factors impacting the performance of the database similarity metric method have been identified and it is shown that grain noise can have a significant effect on particular regions whilst leaving other regions relatively unaffected. It is further shown that the location of regions of good performance can vary significantly depending on the size and orientation of the crack. It was shown that the defect SNR and the S-matrix uniqueness govern this spatial distribution characterisation performance. Simple models were developed that allow the regions of good performance to be predicted. Finally, the spatial error maps have been verified by comparing the characterisation errors found in simulation with those measured in experiments on steel samples with EDM notches. It is also worth noting that the proposed assessment methodology could be extended to produce a 3D spatial map, in order to assess more characterisation methods that would be possible with 2D arrays.