A robust bidirectional motion-compensated interpolation algorithm to enhance temporal resolution of 3D echocardiography
Introduction
Three-dimensional echocardiography (3DE) is fundamental to the understanding of the cardiac structures and function. It offers a real-time representation of a beating heart in three dimensions which assists echocardiographers in assessing cardiac anatomy, atrial and ventricular functions and valvular diseases [1].
3D ultrasound images are constructed by transmitting, receiving and processing ultrasonic pulses. The acquisition time of each ultrasound image is determined by the imaging field of view, the number of transmitted beams and the speed of sound in the imaging medium. Since the speed of sound in tissue is roughly 1540 m/s [2], the acquisition frame rate of 3DE is limited and controlled by the imaging volume and the number of fired beams. This, therefore, creates a trade-off between spatial and temporal resolution. Comparing to 2DE, this trade-off is more pronounced in 3DE since the field of view spans a volumetric region instead of a plane. Hence, the displayed time-sequence of 3DE data appears jerky and not smooth.
To interpret the cardiac dynamic using the 3DE, it is instrumental to have a sufficiently high temporal resolution (frame rate) as well as spatial resolution. Spending time to increase the spatial resolution of each image hinders the evaluation of moving structures like the myocardium and the cardiac valves. Low frame rate 3DE, for instance, degrades the feasibility of 3D chamber quantification [3], [4] and speckle-tracking echocardiography [5].
It is also essential to have a reasonably high frame rate 3DE to model the behavior of the mitral valve in mitral valve repair surgeries [6]. The rapid motion of the mitral valve leaflet makes it difficult to study the valve's behavior given a low frame rate 3DE. Volumetric color Doppler imaging is another example in 3DE that faces a challenge with low frame rate acquisition [7]. In this technique, some of the ultrasonic pulses are deployed to construct the Doppler signals. Hence, the frame rate of 3D B-mode images used to guide the Doppler region of interest decreases. This potentially leads to misplacement of the Doppler signals.
Methodologies to enhance the temporal resolution of 3DE can be categorized into two distinct groups. The first group aims to improve the temporal resolution by altering the acquisition systems. Techniques like sub-volume stitching over several heart beats [8], utilizing parallel acquisition systems [9], coherent plane-wave compounding [10], emanating diverging waves [11] and multi-line transmission [12] belong to this group. The salient limitation of these approaches is that they normally degrade the spatial resolution.
The second group addresses the problem of low temporal resolution by developing post-acquisition strategies. The objective of this group is to temporally upsample a time series of spatially high resolution (but low frame rate) 3DE images. Techniques based on frame reordering [13], two time-delayed acquisitions and image registration [14], interpolation using intensity variation of voxels over time [15], [16], [17] and temporal morphing using tissue Doppler data [18] have been deployed to improve the temporal resolution of 2DE and 3DE images. These approaches are either computationally too intensive or require additional helper data to enhance the temporal resolution, and hence not suitable for a real-time application.
Motion-compensated interpolation offers an alternative post-acquisition strategy to enhance the temporal resolution of echocardiographic data which has a potential to be applied during live-scanning [19], [20], [21]. In this technique, a dense motion field between two temporally consecutive ultrasound images is estimated and then a new image is interpolated along the motion trajectory. The quality of the interpolated data by this approach is highly influenced by the accuracy of the motion estimator.
Lee et al. [19] developed a motion-compensated interpolation technique based on 3D speckle tracking (3DST) algorithm to enhance the temporal resolution of fetal ultrasound images. Nam et al. [20], on the other hand, utilized a motion estimation algorithm based on optical flow techniques to achieve higher temporal resolution in 2D ultrasound data. The type of motion estimators in these methods were unidirectional estimators. In our previous work [21], we demonstrated that the quality of motion-compensated frame interpolation in 2DE can be significantly improved by utilizing a bidirectional motion estimator instead. Extending the work in [21] to 3DE is straight forward. However, achieving a real-time performance cannot be easily realized in today's medical ultrasound scanners.
Among the above-mentioned motion-compensated interpolation algorithms, the method in [19] proved to be a feasible solution for real-time processing in 3D ultrasound applications. The main reason for this achievement is the deployment of 3DST. 3DST is a simple and parallel processing friendly algorithm that is widely used in the echocardiographic community [22]. However, the performance of [19] deteriorates in low frame rate echocardiography especially near fast-moving cardiac structures like the cardiac valves [21]. There are two reasons for this limitation. First, the decorrelation of the speckle pattern in low frame rate echocardiography is high. Therefore, 3DST fails to capture the true motion in low frame data [23]. Moreover, since [19] is a single-scale 3DST approach, one has to increase the search region while addressing the large motions, which is ubiquitous is low frame rate echocardiography. The larger the search region, the more likely it is to capture a false positive by speckle tracking. Second, since the motion estimator in [19] is unidirectional, the chance of obtaining converging or diverging motion vectors in low frame rate echocardiography is high. The consequence of this defect is interpolation artifacts during the motion-compensated interpolation process.
In order to benefit from the low computational complexity of 3DST and, at the same time, improve its performance in motion-compensated 3D image interpolation, we propose a robust multiscale bidirectional 3DST algorithm to better handle large and small motions of the cardiac structures. The bidirectional and multiscale settings of the proposed model allows it to overcome the above limitations of the existing 3DST-based interpolation algorithm. In a multiscale motion estimation framework, it is crucial to have a reliable estimate of the coarse-scale tracking. To achieve this goal, we introduce a motion estimator which only favors those bidirectional motion estimates that have agreeable forward and backward motions at the coarsest level. Moreover, to reduce the interpolation artifacts caused by inaccurate motion estimates, we propose a new adaptive variational interpolation model based on robust statistics [24]. This model differentiates between poor and good motion estimates by employing a confidence map based on the structural similarity index [25]. This discrimination reduces the effect of poor motion estimates in the interpolation process. Experimental results show that, compared to [19], the proposed algorithm significantly improves the quality of 3DST-based motion-compensated interpolation in 3DE data.
Section snippets
Multiscale bidirectional speckle tracking
Typically, a 3D speckle tracking algorithm extracts a cubical region around a given speckle from the reference volume and searches for the displaced speckle in a region of interest in the target volume by utilizing a block matching algorithm [26]. This process is illustrated in Fig. 1.
The block diagram of the proposed 3DST algorithm is shown in Fig. 2. Given two consecutive ultrasound volumes Ii−1 and Ii+1, the objective of the algorithm in Fig. 2(c) is to find a dense displacement field for
Data
The experiments were carried out using twenty one 3D cardiac ultrasound scans acquired using GE Vivid E95 ultrasound scanners (GE Vingmed Ultrasound AS, Horten, Norway) using transoesophageal and transthoracic echocardiography (TTE and TEE) phased array probes. Each scan consisted of a timed sequence of cardiac ultrasound volumes acquired at different time points of a complete heart cycle. Acquisition frame rates ranged from 18 to 30 volumes per second.
Experimental setup
We conducted several experiments to assess
Discussion
Three-dimensional echocardiography, with sufficient spatio-temporal resolution, provides a better representation of the cardiac structures and their dynamics compared to its 2D counterpart [33]. However, due to the finite velocity of sound in tissue and relatively larger field of views in 3D, the acquisition frame rate is normally low. Hence, the temporal resolution and consequently the viewing experience of the cardiac dynamics are degraded in 3DE data. Motion-compensated interpolation
Conclusion
In this work, a new 3DST-based motion-compensated interpolation algorithm was proposed to enhance the temporal resolution of 3DE. The proposed 3DST, which is a multiscale bidirectional motion estimator, was shown to be faster and more robust in estimating the displacements of both the myocardium and the cardiac valves in comparison to a unidirectional single-scale 3DST approach. To reduce the interpolation artifacts caused by inaccurate displacement estimates, a new adaptive interpolation
Authors’ contributions
Hani Nozari Mirarkolaei: Investigation, Methodology, Software, Writing – Original draft preparation.
Sten Roar Snare: Supervision, Resources, Writing – Reviewing and Editing.
Anne H Schistad Solberg: Supervision, Reviewing and Editing.
Acknowledgement
This research has been funded by the Norwegian Research Counsel (grant number 219277).
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (34)
- et al.
Three-dimensional echocardiography: the benefits of the additional dimension
J. Am. Coll. Cardiol.
(2006) - et al.
Normal values of left ventricular mass index assessed by transthoracic three-dimensional echocardiography
J. Am. Soc. Echocardiogr.
(2016) - et al.
Real-time 3d echocardiographic quantification of left atrial volume: multicenter study for validation with cmr
JACC Cardiovasc. Imaging
(2012) - et al.
Volume stitching in three-dimensional echocardiography: distortion analysis and extension to real time
Ultrasound Med. Biol.
(2007) - et al.
Temporal enhancement of 3d echocardiography by frame reordering
JACC Cardiovasc. Imaging
(2012) - et al.
Temporal super resolution of ultrasound images using compressive sensing
Biomed. Signal Process.
(2019) - et al.
Temporal super-resolution of 2d/3d echocardiography using cubic b-spline interpolation
Biomed. Signal Process.
(2020) - et al.
Frame rate up-conversion in cardiac ultrasound
Biomed. Signal Proces.
(2020) Diagnostic Ultrasound Imaging: Inside Out
(2004)- et al.
Effect of through-plane and twisting motion on left ventricular strain calculation: direct comparison between two-dimensional and three-dimensional speckle-tracking echocardiography
J. Am. Soc. Echocardiogr.
(2013)