Joint edge detection and motion estimation of cardiac MR image sequence by a phase field method
Introduction
The breath-held Cine MRI technique provides a detailed viewing of the beating heart consistent with the standard echocardiography scenes. It is used for accurate estimation of the ejection fraction and the stroke volume, investigation of regional wall motion abnormalities, as well as scar and infarct imaging. ECG gated and fast imaging technique of Cine MRI along with its ability in representing soft tissues has made it an important tool for inspection of cardiovascular system [1], [2], [3], [4].
Cardiac motion estimation and segmentation are two intensively addressed topics in the biomedical literature. Inspection of heart deformation and strain requires the motion of all pixels (i.e. dense motion estimation), that can be estimated by using global motion estimation methods such as optical flow techniques. A popular optical flow algorithm is the Lucas–Kanade method, which estimates the motion locally, assuming that the velocity field is constant within a window [5]. The introduction of a global variational approach for the optical flow problem by Horn and Schunck [6] was followed by developments which successfully resolved issues such as smoothing of discontinuities and computational cost [7], [8], [9]. Mailloux et al. extended the optical flow algorithm by adding a linearity constraint to the motion field [10]. Later, Zini et al. added an additional incompressibility constraint [11].
Besides dense motion estimation, fine segmentation of cardiac chambers is of much interest in cardiac diagnosis [12], [13], [14]. Various methods such as active appearance and deformable models [12], [15], [16], [17], [18], fuzzy connectedness [19], and hybrid methods combining edge, region and shape information [20], [21] are applied for segmentation and tracking of heart boundaries. Generally, employing priori information in the statistical approaches restricts the performance due to the dependence on sample size and training set comprehensiveness. Hence, knowledge free methods are considered in many applications and studies like this paper.
Two-phase approaches with cascade processing, where either segmentation or motion estimation is performed first, are applied to the cardiac deformation analysis through various modalities in [22], [23], [24], [25]. Theoretically, segmentation associates with discontinuity extraction in either the motion field or the image field. Moving objects with smooth boundary and distinctive objects without motion cause each one of the above two phase approaches to fail. Consequently, all these approaches suffer from an unreliable decision on extracting discontinuities either on the motion or the image fields.
Alternatively, joint processing of these tasks eliminates the necessity of such a decision. The idea of joint processing is proposed to cope with the ambiguity of the tasks’ priority in applications. It has been applied to mathematical image and motion processing such as joint segmentation and registration or motion estimation. Simultaneous segmentation and registration are recently studied by Droske and Ring in [26], [27]. Kornprobst et al. considered piecewise smooth motion patterns as well as piecewise smooth objects on image sequences [28], [29], [30]. Their work focuses on the functions of bounded variation (BV) and proposes effective numerical implementation.
Basically, two different measurements are employed in motion estimation. One is the mean square error (MSE) distortion between two frames after motion compensation (i.e. intensity matching). This is based on intensity constancy assumption, which depicts motion as the only source for intensity variation. A functional based on this measurement implements the warping theory also used in registration. Brox et al. minimized a functional of constancy measure (intensity matching) and segmentation approximated by level set method for joint segmentation and motion estimation [31]. Alternatively, in optical flow theory, by using the Taylor's series expansion, the constancy measure is replaced with its first order approximation. As a consequence of this approximation, a minimization of the functional leads to linear partial differential equations that can be solved with finite element methods and numerical linear algebra, which is much faster than gradient descent methods required for solving PDEs associated with warping based functionals.
Nesi related Mumford–Shah segmentation with the optical flow motion estimation in [32] and Keeling and Ring investigated the optical flow extraction with respect to optimization in [33]. Papenberg et al. recently proposed a total variation regularization of the optical flow involving higher order gradients [34].
In this paper an accurate and reliable knowledge-free method is developed in the spirit of [7] and applied to Cine MRI sequences. The functional for joint motion estimation and segmentation is developed based on Mumford–Shah segmentation and optical flow motion estimation. The minimization of the functional is accomplished by employing a phase-field approximation by deriving partial differential equations through the mathematical calculus of variation [35], [36]. The minimizer corresponds to the optimal motion field and edge-set, considering discontinuities in both image and motion field. The phase-field approximation to the joint functional is inherited from Ambrosio and Tortorelli's approach to the Mumford–Shah functional [37]. By exploiting a finite element method for solving the resulting partial differential equations, the whole process converges in just a few iterations.
The paper is organized as follows: the next section provides a brief insight into the Mumford–Shah functional for image segmentation, as well as the mathematical principles for the joint segmentation and motion estimation approach. While the formulas are derived for 2D+T case, they can be easily generalized to 3D+T. Section 4 discusses experimental results of applying the joint tasks method to both synthetic and clinical data. It also reports process features such as convergence rate and parameter selection considerations. The paper will be concluded in Section 5, while the appendix provides a concise definition of the matrices for an implementation of the method with finite elements.
Section snippets
Joint segmentation and motion estimation functional
Image and motion segmentation by global optimization approaches comprises two major issues: mathematical description of the problem as an energy functional or probabilistic framework (Bayes or minimum description length) and its numerical solution. Typical probabilistic criteria involve conditional probability of segmentation that usually require powerful optimization algorithms such as simulated annealing and graduated non-convexity [38], [39], [40]. Shape prior for variational segmentation is
Numerical solution
Let denote the variation of the energy functional E in direction g with respect to f. Assuming sufficiently smooth I, , and w the variations of our functional (5) with respect to the unknowns areAll , , and are scalar test functions and the functional (5) is rewritten expanding
Experimental results
The proposed joint segmentation and motion estimation model is implemented by using an Intel® Core2TM Duo processor (T7300) at 2.0 GHz with a MATLAB platform and a Windows VistaTM operating system. The implemented solution is based on just one scale parameter instead of the multiscale strategy mentioned before, due to the small size of images in the clinical data. In the small frames of clinical Cine MRI, the anatomical objects are so close that they may be merged by choosing large scale
Conclusion
In this paper, a mathematical method for the simultaneous segmentation and motion estimation of cardiac MR image sequences is proposed. The method is based on the calculus of variation and minimizes a functional defined by using a combination of an optical flow functional and the Mumford–Shah segmentation functional. It is shown that finite element discretization and phase-field approximation provide an appropriate solution for the joint segmentation and motion estimation problem due to the
Conflict of interest statement
None declared.
Acknowledgments
The authors would like to thank Mr. Alinaghizadeh and the Nour clinic for providing Cine MRI sequences and Dr. Mohebbi for delineation and tracking of heart chambers. The authors would also in particular like to thank Dr. Razvan for giving mathematical insight into variational theory.
References (52)
- et al.
Clinical indications for cardiovascular magnetic resonance (CMR): consensus panel report
Elsevier Eur. Heart J.
(2004) - ...
- ...
- ...
- B. Lucas, T. Kanade, An iterative image restoration technique with an application to stereo vision, in: Proceedings of...
- et al.
Determining optical flow
Artif. Intell.
(1981) - T. Preusser, M. Droske, C.S. Garbe, A. Telea, M. Rumpf, A phase field method for joint denoising, edge detection, and...
- et al.
An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences
IEEE Trans. Pattern Anal. Mach. Intell.
(1986) - et al.
A confidence measure for variational optic flow methods
- et al.
Restoration of the velocity field of the heart from two-dimensional echocardiograms
IEEE Trans. Med. Imag.
(1989)
Application of continuum theory and multi-grid methods to motion evaluation from 3D echocardiography
IEEE Trans. Ultrason. Ferroelect. Freq. Control
Segmentation of the left ventricle from MR images via snake models incorporating shape similarities
IEEE Int. Conf. Image Process.
Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images
IEEE Trans. Med. Imag.
Automated segmentation of the left ventricle in cardiac MRI
Med. Image Anal.
Statistical shape model of atria, ventricles and epicardium from short- and long-axis MR images
Med. Image Anal.
Automated left ventricular segmentation in cardiac MRI
IEEE Trans. Biomed. Eng.
Shape-based segmentation and tracking in cardiac image analysis
IEEE Trans. Med. Imag.
Automatic segmentation of echocardiographic sequences by active appearance motion models
IEEE Trans. Med. Imag.
Magnetic resonance image segmentation and heart motion tracking with an active mesh based system
Comput. Cardiol.
Myocardial motion analysis from B-mode echocardiograms
IEEE Trans. Image Process.
A Mumford–Shah level-set approach for geometric image registration
SIAM J. Appl. Math.
Cited by (5)
Segmentation by retrieval with guided random walks: Application to left ventricle segmentation in MRI
2013, Medical Image AnalysisCitation Excerpt :A major advantage of explicit deformable models is their direct extensions to 3D, i.e., changing the representation from a 2D curve to a 3D mesh surface. These methods not only allow 3D LV segmentation (Heiberg et al., 2005; Montagnat and Delingette, 2005), but also 3D segmentation of all cardiac phases over time (3D+t) (Lynch et al., 2008; Eslami et al., 2010). Furthermore, the segmentation can be improved by coupling the 3D segmentation with biomechanical models of the heart to regularize the segmentation (Papademetris et al., 2002; Yan et al., 2007; Billet et al., 2009).
Magnetic resonance-based motion correction for positron emission tomography imaging
2013, Seminars in Nuclear MedicineCitation Excerpt :For example, the radial fast low-angle shot MR imaging can be used to obtain a series of image volumes that include nodules and boundary of the lungs. The motion fields of lungs can then be obtained from the MR cine images using image registration.38-40 The estimated MR-measured motion fields can be integrated into the system modeling for PET reconstruction.
A multiscale phase field method for joint segmentation-rigid registration application to motion estimation of human knee joint
2011, Biomedical Engineering - Applications, Basis and CommunicationsTracking pylorus in ultrasonic image sequences with edge-based optical flow
2012, IEEE Transactions on Medical ImagingManifold learning for shape guided segmentation of Cardiac boundaries: Application to 3Dt Cardiac MRI
2011, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS