Automatized Evaluation of the Left Ventricular Ejection Fraction from Echocardiographic Images Using Graph Cut

. In this paper, we present a fast and interactive graph cut method for 3D segmentation of the endocardial wall of the left ventricle (LV) given 3D echocardiographic images. This is a challenging task due to the poor contrast and the low signal-to-noise ratio typical of echocardiographic images. The method is carried out in 3 steps. First, 3D sampling of the LV cavity is made in a spherical-cylindrical co-ordinate system. Then, a gradient-based energy term is assigned to each voxel, some of which being given an inﬁnite energy to make sure the resulting volume passes through key anatomical points. Then, a graph-cut procedure provides delineation of the endocardial surface. Results obtained on 30 exams from the 2014 CETUS MIC-CAI challenge dataset reveal that our method takes between 5 and 10 seconds to segment a 3D volume with an overall mean surface distance lower than 2.3 mm and an ejection fraction error of less than 5% compared to a manual tracing by an expert.


Introduction
Echocardiography is a very common diagnostic imaging tool for cardiovascular diseases.It provides real-time imaging of the left ventricle (LV) throughout the cardiac cycle.This technique is quick, painless, reliable, non-invasive and cheap.Despite obvious advantages, visual evaluation of the LV is not always accurate due to the large amount of noise, the lack of contrast between structures and the orientation plane (and more generally the acquisition protocol) that is user-dependent.In 2D, the image orientation results from the transducer orientation strategy.The standard image planes are the long axis plane, the four chamber plane and the short axis plane.Nowadays, computer and transducer technologies permit real-time 3D echocardiographic acquisition and presentation of cardiac structures from any spatial point of view [6].The volumetric dataset can be sliced to obtain multiple 2D images.Then, the limitations of acoustic imaging with conventional 2D imaging can be overcome, by providing the acquisition of different planes from virtually any acoustic window.
The cardiac cavity volumes in diastole and systole and the ejection fraction (EF) are the most common LV functional metrics.In order to calculate functional parameters such as the stroke volume (defined as the LV diastolic volume minus the LV systolic volume) and the EF, one has to segment the endocardial wall.Since manual contouring of the LV is far too slow in practice, one has to use an automatic or semi-automatic method.In this paper, we propose a fast semi-automatic 3D segmentation method of the endocardial wall.
Our method has the following key advantages: 1. it segments a 3D endocardium in 5 to 10 seconds on a mid-end laptop computer; 2. the result is guaranteed to have an anatomically plausible U shape which starts from the mitral valves all the way down to the apex; 3. as opposed to other methods [14], in areas where the LV has a signal dropout, our resulting shape is guaranteed not to have any holes.And as opposed to some active contour methods, our approach cannot "bleed" in these areas; 4. our semi-automatic method requires only 4 clicks from the user.It also relies on only 1 intuitive parameter which determines the smoothness of the resulting shape.
Our method was evaluated on 30 sequences of 3D ultrasound volumes in the framework of the CETUS MICCAI challenge [1].For each case, the provided segmentation was compared to meshes manually outlined by three expert cardiologists (at end-diastolic and end-systolic instances).The aim of the evaluation metrics are, firstly, the measure of degree of accuracy of the segmentation (with the modified Dice similarity index, the Hausdorff distance (HD), the mean surface distance (MSD), the minimum error and the maximum error) and secondly the degree of accuracy of the derived clinical metrics (with the calculation of the ejection fraction and the stroke volume).

Previous Work
Echocardiographic images are well known for having a low signal-to-noise ratio with signal dropouts in some sections.Furthermore, since those images not only contain the LV but also other sections of the heart and nearby organs, many segmentation methods require a manual initialization of some sort.These methods are usually called "semi-automatic".
Several semi-automatic approaches use deformable models like active contours and level sets according to which the user has to draw an initial contour close to the LV [12].Multiple methods of this family combine active contours with other strategies such as K-means clustering or probability density functions [10,11].Such methods are efficient to detect the boundaries of an object in 2D acquisitions, but does not always generalize well to 3D [15] especially when they work along the short axis view [12].Furthermore, active contour methods are error prone everywhere the endocardium border is poorly defined such as close to the apex where the amount of noise is large or in areas where the papillary muscles are agglomerated in a bundle.And last, active contours may very well "bleed" in areas suffering from a signal dropout.In order to prevent this, authors usually include a shape prior as in [15,5].This unfortunately leads to methods that are prohibatively slow as well as mathematically and algorithmically complicated.
Another family of methods that recently emerged are those using deep neural networks(see [4]), which are fully automated.A well trained system will provide highly accurate results, though it requires a huge set of training images and there is no way to apply a correction to the resulting segmentation.Furthermore, training a system that can handle both healthy and pathological cases is far from trivial.
A few methods approached the problem differently by working in an anatomical space instead of the Euclidean space.Heyde et al. [8] compared anatomically oriented B-spline free-form deformation(FFD) models with classical FFD models defined in Cartesian space.The results of the two models were competitive, but working with an anatomical grid reduced considerably the processing time.Another approach that unfolds the LV from an Euclidean space to an anatomical space, which happens to be similar to ours, has been proposed by VanStralen et al. [16].Segmentation is done with a dynamic programming procedure applied in the anatomial space.But as opposed to our approach, their method requires the user to manually outline the endocardial contour of 2 perpendicular long-axis slices.Furthermore, unlike our approach which segments the 3D volume all at once, their method segments a collection of 2D slices and estimates a 3D volume afterward.Juang et al. [13] introduced an automatic method that segments the LV and the atrium with graph cut from 3D echocardiographical data represented in a cylindrical space.Unlike our graph cut approach, all their nodes are connected to the source and the drain which can create holes in the myocardium and false positives inside the LV (further details on graph cut will be given in section 3).Furthermore, their method drastically fails to segment the apex of the LV (see Fig. 3 of [13]) and thus can hardly be used to calculate accurate LV measures.
More details on previous echocardiographic segmentation methods can be found in [7,15].

Proposed Method
This section presents the three main steps of our method, i.e.

Spherical-Cylindrical Space
Unlike most other methods [4,5,14], our approach does not work on the native (X,Y,Z) Euclidean space.Instead, our method works in a spherical-cylindrical (S-C) space.In this S-C space, a plane corresponds to a 3D U shape (or cup) in the 3D Euclidean space.Thus, working in the S-C space allows to implicitly enforce a U shape prior to the segmented result.This again is rather different than most other methods whose shape prior is always carried out explicitly [5,15].In order to convert the input 3D image from the Euclidean space to the S-C space, the basal and the apex first need to be identified.To do so, the user can adjust the reference angles (as you would in a 3D software) to display the correct long axis and short axis views.Once this is done, the user identifies the center of the LV cavity from the long axis view.He then places a T-bar target at the base and extends it all the way to the apex as in Fig. 1

(b).
The T-bar has 3 control points A,B and C. A is used to localize the mitral valve (close to the endocardium), B the epicardial border and C the apex.
Once the T-bar is in place, the S-C image is generated by casting rays perpendicular to the U shape.From the T-bar target, rays are generated in cylindrical and radial directions in the 3D space (see the thin red lines in Fig. 1 (c)).Each ray is associated to a radial angle θ, a height value H or an elevation angle α as shown in Fig. 1 (a).In this way, every Euclidean voxel (x, y, z) within the cylindrical area is assigned a (R, H, θ) coordinate and those within the top hemisphere are assigned a (R, α, θ) spherical coordinate.A distance ratio is used to determine where the spherical cap begins.For the sake of the 2014 CETUS MICCAI challenge, a ratio of 0.45 (starting from the apex) was used.Grayscale integration is then performed along these rays.A sub-pixel accuracy is obtained by interpolating multiple samples per pixel (2 samples per pixel by default).Rays are then stacked up in a 3D image (see Fig. 1(d)) so that each line R in the 3D image corresponds to a sampled ray in the 3D space.The number of sampled rays for each θ orientation as well as the number of orientations can be modified by the user.A default value of 16 cylindrical rays and 8 spherical rays is used along 20 evenly distributed θ.This results into a 3D S-C image whose axes (determined by the orientation of the T-bar) are the radial axis R, the rotation angle θ and the elevation position in the cylinder or the hemisphere part H/α.The H/α axis is represented with the letter Γ (Fig. 1(d)) and the 3D S-C image by the letter I.

Graph Cut
The goal of our method is to find the endocardium surface inside the 3D S-C image.One fairly good assumption is that the endocardium is located where the gradient magnitude in the S-C image is the largest, i.e.where the grayscale slope is the heaviest.Although one can try to locate the endocardium surface by selecting the maximum gradient value along the R dimension following a maximum likelihood procedure, due to the very nature of echocardiographic images, such an approach drastically fails, producing an irregular (and yet anatomically impossible) jaggy surface.
An alternative for a 2D S-C image such as the one in Fig. 1 (d) is to use a dynamic programming procedure as in [16].In this case, the endocardial wall becomes the shortest path between the top and the bottom of the S-C image.Since dynamic programming prevents from having strong discontinuities, the resulting solution is guaranteed to be smoother than that of maximum likelihood.In the case of a 3D S-C image, the dynamic programming procedure needs to be generalized to 3D which is far from trivial.As an alternative solution, we use graph cut.
Graph cut uses a weighted graph G = {N , Σ} made of nodes N , connected by edges Σ.Each edge e ab ∈ Σ connects two nodes a ∈ N and b ∈ N and is associated to a weight (or capacity) w ab ≥ 0. The weight of an edge e ab indicates how much 'flow' can circulate between a and b.The graph also contains 2 terminal nodes : the source s and the sink t [2].The source is the node from which the flow comes out and the sink is the node into which the flow ends up.
The goal of graph cut is to divide the graph in two subsets of nodes namely S and T .To be valid, S and T must comply to the following conditions: S T = ∅, S T = N , and s ∈ S, t ∈ T .Given two subsets S and T , we also have U , the set of edges that connect nodes a and b from S to T such that a ∈ S and b ∈ T .
A concept pivotal to graph cut is the notion of 'cost'.The cost of a cut is the sum of all edges contained in U , i.e.C(S, T, U ) = e ab ∈U w ab .The goal of graph cut is to find the minimum cut, i.e.

S, T, U = arg min
Finding the minimum cut calls for an optimization function.In our case, we use alphaexpansion [17].For more details on graph cut, please refer to [3,9].

Energy Function
The result of graph cut depends on how the graph is built and which edge capacities are used.In our case, we use a 3D graph for which each node a ∈ N is associated to a voxel (R, Γ, θ) in the S-C image.Also, each node a in the graph is connected to its 6 nearest neighbors corresponding to voxel (R + 1, Γ, θ), (R − 1, Γ, θ), (R, Γ + 1, θ), (R, Γ − 1, θ), (R, Γ, θ + 1), and (R, Γ, θ − 1) in the S-C space.Also, as shown in Fig. 1 (d), the source node s is connected to the first row of nodes (those for which R = 0) and the drain is connected to the last row of nodes (those for which R = R max ).
Once the graph is built, we set a weight to each edge.The weight should be carefully selected such that the resulting cut corresponds to large gradient values, i.e.where the endocardium is most likely to be.These intensity values correspond to the bright vertical blurry shape in Fig. 1(d).
Since the endocardium surface cannot go outside the S-C image, we set to infinity the capacity of the source and drain edges w sa and w at .By doing so, we make sure that these edges are never cut.For the other edges, we distinguish between horizontal edges e h ab and vertical edges e v ab .As shown in Fig. 1(d), horizontal edges e h ab are parallel to the R axis, i.e. those for which a = (R, Γ, θ) and b = (R + 1, Γ, θ).Vertical edges are the ones perpendicular to e h ab (i.e.along the Γ and θ axis).Horizontal edges are assigned the following weight where d(I(a)) is the gradient magnitude of the S − C image evaluated at voxel a which we implement with the following numerical approximation: d(I(a)) = (I(R + 1, Γ, θ) − I(R, Γ, θ)) 2 .σ is a noise parameter that we set to 0.6 for all our experiments.As for the vertical edges, their capacity is set to a constant value β that we call the smoothness term.Since graph cut always finds the minimum cut, the resulting solution is the one for which S, T, U = arg min Equation ( 3) contains a data term and a smoothness term.The data term makes sure that the center line of the endocardium falls onto voxels with a large gradient value d(I(a) while the smoothness term makes sure that the endocardium surface is smooth and has no strong discontinuities.With β = 0, our method ends up computing a maximum likelihood along each horizontal line R which, in turn, is very noisy.With an increasing value of β, the resulting solution becomes smoother.At the limit, with β → ∞, the resulting value is a straight line (like the dotted line in Fig. 1(d)).Since that line is in the S-C space, it converts into a perfect U shape in the Euclidean space.In other words, the more vertically straight a myocardium center line is, the smoother the resulting 3D U shape will be.That is why we say that our method implicitly enforces a U-shape prior.The result of graph cut is shown as a red line in Fig. 1(e).Although the result is shown as a line, the actual result of graph cut is a 3D manifold in the 3D S-C space, and thus a 3D volume in the Euclidean space.

Further Improvements
Empirical validation made us realize that although minimizing Eq.( 3) with graph cut provides good results, it nonetheless has some limitations.The first and foremost limitation is segmentation at the apex.Due to poor resolution of echocardiographic images at the tip of the heart, our method has a tendency of underestimating (and yet cutting) the endocardium's nib thus leading sometimes to an Hausdorff error of 12 to 15 mm.The second limitation is due to the U-Shape prior which leaves the base of the heart wide open instead of being curled in as it should be next to the mitral valves.
We solve these two limitations by forcing to infinity the grayscale values associated to the basal control point A, a symmetrical point of A on the base line and the apex control point C and to zero the other pixels along these lines.This is shown in Fig. 1(e) where 3 lines have zero values everywhere and a white dot on one pixel.This simple modification drives to zero the energy function and thus forces graph cut to start, finish and pass by the three control points on the T-bar.
Another limitation that we observed is due to the varying contrast from one image to another.We found that the mean surface distance to the groundtruth can be significantly reduced by rescaling the grayscale value between grayscale 25 and 200.Every intensity lower than 25 and higher than 200 is forced to 0 and 255 respectively, while intensities in the range [25,200] are rescaled between [0, 255].This removes some of the noise and helps graph cut stick to the endocardial wall with a minimal negative impact to the fusion of the papillary muscles to the endocardial walls.

Experimental Results
As shown in Table 1, our method works well on most cases provided for the challenge.The mean surface distance (MSD) value of 1.84 mm and 2.3 mm show that on average, the cut is roughly 2 or 3 pixels away from the true endocardial wall.Although the average max error (here Hausdorff distance -HD) is larger (7.03 mm and 8.58 mm), since the standard deviation is small (≤ 2.5 mm), we can say that these large errors only occurs on small localized areas.Also, the max error does not seem to have much of an influence on the ejection fraction (EF) whose error is below 5% both for the training and testing datasets.
Results reported in table 2 also underline the fact that our method produces accurate EF results with a correlation coefficient 'r' of more than 0.8.We also report result for bias which happens to be : bias= (minError + maxError)/2.With a bias of less than 1.5 for both EF and Stroke, it shows that our method does not systematically over or under estimate the volume.This is also the case with the volume bias reported in table 3 which is slighly positive on the training data and negative on the testing data.We also got surprisingly high correlation coefficient both on the training and testing volumes.The correlation coefficient of the stroke volume on the other hand is rather low for the training data set.This metric is heavily disrupted by 2 outlier cases; one that overestimates the stroke volume by 19.2ml and another that underestimates by 22.10ml.By omitting the 2 outlier cases, we obtain a correlation coefficient of 0.9186 for the EF and 0.7150 for the stroke volume which is a drastic improvement.Needless to say, the amount of cases in the dataset is somewhat small to have statistical results resistant to outlier cases.For more details on the metrics used in table 1, 2, and 3, please refer to [1].
In order to provide the reader with an intuitive understanding on how our results look like, we provided screenshots of positive results (Fig. 2) and less accurate results obtained on pathological cases (Fig. 3). Figure 2 shows results for 6 hearts, 3 from the training dataset and 3 from the testing dataset.As one can see, our method hugs closely the endocardial wall even in areas suffering from a severe signal dropout like on the left-hand side of Fig. 2 (a) and (f).Out of 30 patients, 21 have an EF error below 7% and 27 have a MSD below 2.5 mm which is a clear indication of the efficiency of our method.
Although our approach is successful most of the time, it nonetheless has issues handling extreme cases.For example, our method may fail when the manual tracing does not correspond to the apparent contour.This is shown in Fig. 3(a) where our result follows closely the intensity border while the ground truth cuts early near the left mitral valve.Also, since our method uses a T-bar target, our approach makes the assumption that the long axis of the LV is perpendicular to the mitral valves.Although this is usually true, in cases such as Fig. 3(b), the cavity has some kind of an angle which prevents the T-bar target from covering both the apex and the mitral valve borders.This results into a max error of 10 to 12 mm next to the mitral valves.That being said, since the max error is only localized in that area, it does not influence much the overall MSD error.
Another difficult case is with patient 26 (Fig. 3(c)).In this case, a large section of the LV is missing from the acquisition, which causes an important underestimation of the ventricle's volume.This is depicted by the yellow line on the right-hand side of the image.But surprisingly, since the volume is underestimated for both the diastole and systole phases, the resulting EF is rather accurate with 32.6% as opposed to 33.9% for the groundtruth.Also, the poor quality of the images at the apex of the heart may hinders the segmentation in that area.Fig. 3(d) illustrates such a case.Although the MSD error is 1.96 mm, the max error is 12.4 mm at the apex.But again, since the MSD is small, the resulting EF and stroke volume are rather precise with 37.9% and 67.4 ml as opposed to 36.1% and 67.1 ml for the groundtruth.
In most cases, the proposed method is able to successfully cut behind the papillary muscles (see Fig. 4(a)), especially when they are not fused with the endocardium.On the other hand, our method sometimes underestimates the endocardial wall when the papillary muscles are tightly compressed against the wall.We observed that situation in 4 patients out of 30 regardless of the intensity rescaling applied while sampling.
The processing time of our method wanders between 5 and 10 seconds per ventricle depending on the number of slices used (see Fig. 1(c)).For this study, we used between 13 and 24 slices.Again, our method requires 4 clicks, i.e. one for translating the target and 3 for positionning control points A,B and C. Also, 3 additionnal clicks might be needed to adjust the reference angles to obtain a better long axis view perspective.The smoothing parameter β was set between 0.06 and 0.3.

Conclusion
This paper presents a new semi-automatic method that allows for a fast and reliable segmentation of the endocardial wall of the LV from echocardiographic images.The novelties are the use of graph cut and to work in spherical-cylindrical space in order to take into account to the U-shape of the LV in long axis orientations.Experimental results obtained on 30 patients are very promising with an average EF error of less than 5%.

( 1 )
conversion from the Euclidean space to the spherical-cylindrical space, (2) computation of gradient-based energy values, and (3) localization of the endocardial wall with graph cut.

Fig. 1 .
Fig. 1.(a) We assume the LV has a U-shape with a cylinder at the bottom and an hemisphere at the apex.(b) Voxels (x, y, z) with the same S-C radius coordinate R fall onto the same U shape (green dotted line in (b)).The S-C image (d) is obtained by integrating grayscale values along S-C rays (c) in the Euclidean space.(d) A graph is built on top of the S-C image in which each voxel is a node connected to its 6 neighbors.The resulting cut is shown in red.

Table 1 .
Mean and standard deviation error obtained on the 2014 CETUS MICCAI challenge dataset.The first 2 rows are for the training dataset and the last 2 rows for the testing dataset.

Table 2 .
Correlation coefficient (r), bias and limit of agreement (LOA) of the ejection fraction and stroke volume obtained on the 2014 CETUS MICCAI challenge training and testing dataset.

Table 3 .
Correlation coefficient (r), bias and limit of agreement (LOA) of the end diastolic (ED) volume and the end systolic (ES) volume obtained on the 2014 CETUS MICCAI challenge training and testing dataset.