A robust bidirectional motion-compensated interpolation algorithm to enhance temporal resolution of 3D echocardiography

https://doi.org/10.1016/j.bspc.2020.102384Get rights and content

Highlights

  • Increasing frame rate of 3D echocardiography by motion-compensated frame interpolation.

  • Developing a multiscale bidirectional 3D speckle tracker in low frame rate acquisitions.

  • Developing an adaptive temporal image interpolation scheme based on robust statistic.

  • Enhancing visualization of the cardiac valves in 3D transthoracic and transesophageal acquisitions.

Abstract

High frame rate three-dimensional echocardiography (3DE) improves the assessment of the cardiac dynamics by providing a realistic view of the cardiac structures. However, since the field of view in 3D is large and the speed of sound in tissue is limited, one must sacrifice some spatial resolution to achieve a temporally smoothed sequence of 3DE data. Motion-compensated frame interpolation provides a means to enhance perceived temporal resolution without degrading the spatial resolution. Among the existing interpolation techniques, motion-compensated interpolation based on 3D speckle tracking (3DST) is very simple to implement and can be applied in real time since it is well suited for parallel processing. However, the performance of this approach deteriorates in low frame rate 3DE due to large inter-frame displacements. In this paper, we propose a new 3DST-based motion-compensated interpolation algorithm based on a multiscale bidirectional motion estimator. The proposed estimator improves the reliability of the motion estimate at the coarsest scale by utilizing a multidirectional speckle tracker. Moreover, the proposed 3DST algorithm requires smaller search regions for an equivalent search effect compared to the existing algorithms; therefore, it reduces the computation time of 3DST. To reduce the interpolation artifacts caused by decorrelation of the speckle pattern and inaccurate motion estimates, we develop an adaptive interpolation scheme based on robust statistics and the structural similarity index. Experimental results demonstrate that the proposed algorithm, compared to an existing 3DST-based algorithm, greatly reduces the interpolation artifacts such as ghosting and blurring, especially near fast-moving cardiac structures.

Introduction

Three-dimensional echocardiography (3DE) is fundamental to the understanding of the cardiac structures and function. It offers a real-time representation of a beating heart in three dimensions which assists echocardiographers in assessing cardiac anatomy, atrial and ventricular functions and valvular diseases [1].

3D ultrasound images are constructed by transmitting, receiving and processing ultrasonic pulses. The acquisition time of each ultrasound image is determined by the imaging field of view, the number of transmitted beams and the speed of sound in the imaging medium. Since the speed of sound in tissue is roughly 1540 m/s [2], the acquisition frame rate of 3DE is limited and controlled by the imaging volume and the number of fired beams. This, therefore, creates a trade-off between spatial and temporal resolution. Comparing to 2DE, this trade-off is more pronounced in 3DE since the field of view spans a volumetric region instead of a plane. Hence, the displayed time-sequence of 3DE data appears jerky and not smooth.

To interpret the cardiac dynamic using the 3DE, it is instrumental to have a sufficiently high temporal resolution (frame rate) as well as spatial resolution. Spending time to increase the spatial resolution of each image hinders the evaluation of moving structures like the myocardium and the cardiac valves. Low frame rate 3DE, for instance, degrades the feasibility of 3D chamber quantification [3], [4] and speckle-tracking echocardiography [5].

It is also essential to have a reasonably high frame rate 3DE to model the behavior of the mitral valve in mitral valve repair surgeries [6]. The rapid motion of the mitral valve leaflet makes it difficult to study the valve's behavior given a low frame rate 3DE. Volumetric color Doppler imaging is another example in 3DE that faces a challenge with low frame rate acquisition [7]. In this technique, some of the ultrasonic pulses are deployed to construct the Doppler signals. Hence, the frame rate of 3D B-mode images used to guide the Doppler region of interest decreases. This potentially leads to misplacement of the Doppler signals.

Methodologies to enhance the temporal resolution of 3DE can be categorized into two distinct groups. The first group aims to improve the temporal resolution by altering the acquisition systems. Techniques like sub-volume stitching over several heart beats [8], utilizing parallel acquisition systems [9], coherent plane-wave compounding [10], emanating diverging waves [11] and multi-line transmission [12] belong to this group. The salient limitation of these approaches is that they normally degrade the spatial resolution.

The second group addresses the problem of low temporal resolution by developing post-acquisition strategies. The objective of this group is to temporally upsample a time series of spatially high resolution (but low frame rate) 3DE images. Techniques based on frame reordering [13], two time-delayed acquisitions and image registration [14], interpolation using intensity variation of voxels over time [15], [16], [17] and temporal morphing using tissue Doppler data [18] have been deployed to improve the temporal resolution of 2DE and 3DE images. These approaches are either computationally too intensive or require additional helper data to enhance the temporal resolution, and hence not suitable for a real-time application.

Motion-compensated interpolation offers an alternative post-acquisition strategy to enhance the temporal resolution of echocardiographic data which has a potential to be applied during live-scanning [19], [20], [21]. In this technique, a dense motion field between two temporally consecutive ultrasound images is estimated and then a new image is interpolated along the motion trajectory. The quality of the interpolated data by this approach is highly influenced by the accuracy of the motion estimator.

Lee et al. [19] developed a motion-compensated interpolation technique based on 3D speckle tracking (3DST) algorithm to enhance the temporal resolution of fetal ultrasound images. Nam et al. [20], on the other hand, utilized a motion estimation algorithm based on optical flow techniques to achieve higher temporal resolution in 2D ultrasound data. The type of motion estimators in these methods were unidirectional estimators. In our previous work [21], we demonstrated that the quality of motion-compensated frame interpolation in 2DE can be significantly improved by utilizing a bidirectional motion estimator instead. Extending the work in [21] to 3DE is straight forward. However, achieving a real-time performance cannot be easily realized in today's medical ultrasound scanners.

Among the above-mentioned motion-compensated interpolation algorithms, the method in [19] proved to be a feasible solution for real-time processing in 3D ultrasound applications. The main reason for this achievement is the deployment of 3DST. 3DST is a simple and parallel processing friendly algorithm that is widely used in the echocardiographic community [22]. However, the performance of [19] deteriorates in low frame rate echocardiography especially near fast-moving cardiac structures like the cardiac valves [21]. There are two reasons for this limitation. First, the decorrelation of the speckle pattern in low frame rate echocardiography is high. Therefore, 3DST fails to capture the true motion in low frame data [23]. Moreover, since [19] is a single-scale 3DST approach, one has to increase the search region while addressing the large motions, which is ubiquitous is low frame rate echocardiography. The larger the search region, the more likely it is to capture a false positive by speckle tracking. Second, since the motion estimator in [19] is unidirectional, the chance of obtaining converging or diverging motion vectors in low frame rate echocardiography is high. The consequence of this defect is interpolation artifacts during the motion-compensated interpolation process.

In order to benefit from the low computational complexity of 3DST and, at the same time, improve its performance in motion-compensated 3D image interpolation, we propose a robust multiscale bidirectional 3DST algorithm to better handle large and small motions of the cardiac structures. The bidirectional and multiscale settings of the proposed model allows it to overcome the above limitations of the existing 3DST-based interpolation algorithm. In a multiscale motion estimation framework, it is crucial to have a reliable estimate of the coarse-scale tracking. To achieve this goal, we introduce a motion estimator which only favors those bidirectional motion estimates that have agreeable forward and backward motions at the coarsest level. Moreover, to reduce the interpolation artifacts caused by inaccurate motion estimates, we propose a new adaptive variational interpolation model based on robust statistics [24]. This model differentiates between poor and good motion estimates by employing a confidence map based on the structural similarity index [25]. This discrimination reduces the effect of poor motion estimates in the interpolation process. Experimental results show that, compared to [19], the proposed algorithm significantly improves the quality of 3DST-based motion-compensated interpolation in 3DE data.

Section snippets

Multiscale bidirectional speckle tracking

Typically, a 3D speckle tracking algorithm extracts a cubical region around a given speckle from the reference volume and searches for the displaced speckle in a region of interest in the target volume by utilizing a block matching algorithm [26]. This process is illustrated in Fig. 1.

The block diagram of the proposed 3DST algorithm is shown in Fig. 2. Given two consecutive ultrasound volumes Ii−1 and Ii+1, the objective of the algorithm in Fig. 2(c) is to find a dense displacement field for

Data

The experiments were carried out using twenty one 3D cardiac ultrasound scans acquired using GE Vivid E95 ultrasound scanners (GE Vingmed Ultrasound AS, Horten, Norway) using transoesophageal and transthoracic echocardiography (TTE and TEE) phased array probes. Each scan consisted of a timed sequence of cardiac ultrasound volumes acquired at different time points of a complete heart cycle. Acquisition frame rates ranged from 18 to 30 volumes per second.

Experimental setup

We conducted several experiments to assess

Discussion

Three-dimensional echocardiography, with sufficient spatio-temporal resolution, provides a better representation of the cardiac structures and their dynamics compared to its 2D counterpart [33]. However, due to the finite velocity of sound in tissue and relatively larger field of views in 3D, the acquisition frame rate is normally low. Hence, the temporal resolution and consequently the viewing experience of the cardiac dynamics are degraded in 3DE data. Motion-compensated interpolation

Conclusion

In this work, a new 3DST-based motion-compensated interpolation algorithm was proposed to enhance the temporal resolution of 3DE. The proposed 3DST, which is a multiscale bidirectional motion estimator, was shown to be faster and more robust in estimating the displacements of both the myocardium and the cardiac valves in comparison to a unidirectional single-scale 3DST approach. To reduce the interpolation artifacts caused by inaccurate displacement estimates, a new adaptive interpolation

Authors’ contributions

Hani Nozari Mirarkolaei: Investigation, Methodology, Software, Writing – Original draft preparation.

Sten Roar Snare: Supervision, Resources, Writing – Reviewing and Editing.

Anne H Schistad Solberg: Supervision, Reviewing and Editing.

Acknowledgement

This research has been funded by the Norwegian Research Counsel (grant number 219277).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (34)

  • A. Macnab et al.

    Three-dimensional echocardiography is superior to multiplane transoesophageal echo in the assessment of regurgitant mitral valve morphology

    Eur. J. Echocardiogr.

    (2004)
  • L. Sugeng et al.

    Dynamic three-dimensional color flow doppler: an improved technique for the assessment of mitral regurgitation

    Echocardiography

    (2003)
  • D.P. Shattuck et al.

    Explososcan: a parallel processing technique for high speed ultrasound imaging with linear phased arrays

    J. Acoust. Soc. Am.

    (1984)
  • G. Montaldo et al.

    Coherent plane-wave compounding for very high frame rate ultrasonography and transient elastography

    IEEE Trans. Ultrason. Ferroelectr. Freq. Control

    (2009)
  • M. Couade et al.

    Ultrafast imaging of the heart using circular wave synthetic imaging with phased arrays

  • B. Denarie et al.

    Multi-line transmission in 3-d with reduced crosstalk artifacts: a proof of concept study

    IEEE Trans. Ultrason. Ferroelectr. Freq. Control

    (2013)
  • F. Contijoch et al.

    Increasing temporal resolution of 3d transesophageal ultrasound by rigid body registration of sequential, temporally offset sequences

  • Cited by (0)

    View full text