User-Guided Level Set Segmentation of Anatomical Structures with ITK-SNAP

,


Introduction
Medical image segmentation is used to address a wide range of biomedical research problems: cancer detection, brain morphology, treatment planning, robotguided surgery, etc.The spectrum of segmentation techniques available to the clinical researcher is broad, ranging from manual slice-by-slice outlining to fully automatic techniques that incorporate prior knowledge about the shape and intensity of the structures of interest.Amid this spectrum lie semiautomatic approaches that combine high efficiency, accuracy and repeatability of automatic methods with expertise and quality control that can only come from human supervision.Active contour segmentation via level set methods [2,7] is an especially elegant segmentation technique that requires the expert to provide an initialization, set control parameters, and terminate the segmentation.
Despite the advances in automatic segmentation methodology, many biomedical research labs continue to rely on manual delineation.This reluctance to accept automation may be due to insufficient reliability and high specificity of fully automatic methods and the lack of inexpensive user-friendly tools implementing semi-automatic methods.To address this shortage, ITK-SNAP 5 was developed, first as a series of student projects at UNC Chapel Hill, and later as a component of the Insight Toolkit (ITK) [4].
ITK-SNAP is an open-source application intended to bring level set active contour segmentation to the fingertips of clinical users.It was designed to make segmentation and parameter selection as easy as possible for users without a mathematical background.ITK-SNAP is more than just an implementation of a given computational method: in addition to the automatic segmentation workflow, it contains innovative tools for manual outlining and quality control.In recent years, a large number of 3D image processing toolboxes have been developed, both commercially (e.g., Analyze [6]) and in the public domain (e.g., Slicer [3]).These toolboxes tend to provide a broad range of algorithms rather than focus specifically on segmentation.In contrast, ITK-SNAP was developed specifically for segmenting anatomical structures and is considerably easier to master than the above packages due to its tightly focused functionality.
This paper gives a brief overview of the methods used in ITK-SNAP, touches on the key points of its user interface, and presents the results of validation experiments performed in the context of an ongoing child neuroimaging autism study, where the accuracy, repeatability and efficiency of ITK-SNAP segmentation of the caudate are compared to manual segmentation using multiple raters.

Methods
ITK-SNAP implements the 3D geodesic active contour method developed by Caselles et al. [1,2] as well as the region competition method by Zhu and Yuille [13].Both can be formulated in terms of contour evolution: where C(t, u) is the contour at time t parameterized by u and F are forces acting on the contour in the normal direction.The internal force modulates evolution based on the contour's geometry, while the external force incorporates information from the image being segmented.In the Caselles method, the external force is computed from the gradient magnitude of the image intensity, while in the Zhu and Yuille method it is based on voxel probability maps.Caselles's formulation is written as where α, β, and γ are modulating weights, κ is the mean curvature of the contour, N is the contour normal, and g I is a function that decreases monotonically with the gradient magnitude of the image I: where G σ * I denotes convolution with a Gaussian kernel, and ν, λ, and σ are user-supplied parameters.The first term in ( 2) is an outwards-acting external force, the second term is an internal force modulated by g I , and the third is the external advection force, which can act inwards when a contour approaches edges in the image, thus counterbalancing the outward g I -proportional forces.
In our implementation of the Zhu and Yuille method, the external force is proportional to difference of probabilities P obj = P (voxel ∈ object) and P bg = P (voxel ∈ background): In ITK-SNAP, these probabilities are estimated using fuzzy one-sided and twosided thresholds.The Zhu and Yuille approach is more appropriate when the structure of interest has a well-defined intensity range with respect to the image background.In contrast, the Caselles method is well suited for structures bounded by strong image intensity edges.
To implement contour evolution, ITK-SNAP uses the level set method [5,8], which ensures numerical stability.The contour is represented as a level set of a function φ defined on the ambient space, and the relationship N = ∇φ/ ∇φ is used to write ( 2) and ( 4) as partial differential equations in φ.The equations are solved iteratively using the extreme narrow banding method by Whitaker [10].

Software Design and User Interface
ITK-SNAP provides an interactive image viewing/labelling UI on top of which a wizard-like automatic segmentation workflow is constructed.The image viewing UI (Fig. 1) emphasizes the three-dimensional nature of images by displaying three orthogonal views that intersect at the position identified by the 3D cursor.Within each view, the cursor can be repositioned in and out of the view plane by dragging the mouse and rolling the mouse wheel.To perform manual segmentation, the user marks up the image in either of the three views.Each voxel in the image can be assigned a numerical label that corresponds to an anatomical structure.Thus, the result of manual or automatic segmentation is represented as an image rather than as a set of contours (partial labeling of voxels is not supported).The UI supports 3D rendering of the labeled structures, and a cut-plane tool allows users to relabel segmentations directly in 3D.
The challenge in developing a user-friendly level set segmentation tool lies in devising a simple way for the user to specify the values of various parameters that affect segmentation (α, β, γ, λ, ν, σ, etc.).To assist the user in setting these parameters, ITK-SNAP employs a structured wizard-like segmentation workflow (Fig. 1) and live feedback/preview mechanisms.In the first step of the workflow, the user chooses whether to base the segmentation on regions of uniform intensity (Zhu and Yuille method) or on image edges (Caselles method).Then, the user is presented with an interface, shown in Fig. 2, in which she must "convert" the input image into an image that is bright in the regions where the snake should expand, dark in the regions where it should stay put, and bright blue in the regions where it should retract (only in the Zhu and Yuille case).As the user changes the parameters, she is presented both with the graph of the function g I or P obj (I), depending on the method, and the resulting image.ITK's flexible filter architecture allowed us to apply the preprocessing only to the subset of the image that the user sees in the three slice views, making it possible to respond to parameter changes in real time.In the workflow's second step, the user initializes the segmentation by placing one or more bubbles in the structures of interest (Fig. 1).Previously computed segmentations can also be used for initialization.In the third step, the user oversees the actual segmentation process using a VCR-like UI.The user can choose between "easy" and "expert" interfaces for adjusting the values of parameters α, β and γ.The easy interface gives textual descriptions of the parameters' meanings, while the expert interface shows the formula relating the parameters.In both cases, the user sees a graphical 2-D preview of each of the internal and external forces modulated by α, β and γ, as well as their weighted sum (Fig. 2).

ITK-SNAP in Clinical Applications
The ITK-SNAP tool, with its combination of user-guided 3D active contour segmentation, manual post-editing in 2-D orthogonal slices, and 3D cutting knife capability, is increasingly replacing conventional 2-D slice editing for a variety of image segmentation tasks.ITK-SNAP is used in several large neuroimaging studies at UNC Chapel Hill, Duke University and University of Pennsylvania.Segmentation either uses the fuzzy threshold option for the definition of object and background probabilities, e.g. for the segmentation of the intracranial volume in head MRI, or makes use of existing tissue probability maps.This option is used for the segmentation of ventricles based on a cerebrospinal fluid probabilistic segmentations, or for the segmentation of caudate nucleus structures using gray matter probability maps.Also, tests of hippocampus segmentation from MRI driven by intensity edges are encouraging but not ready for use in large-scale clinical studies.In thin-slice computer tomography imaging (CT) in radiation oncology applications, ITK-SNAP proved to be useful to segment the liver, kidneys, bony structures and tumors.ITK-SNAP has been used to segment lung volumes in CT images and pulmonary vasculature in lung MR images.ITK-SNAP is used alongside deformable registration in several applications, including building a cortical atlas for template deformation morphology [11], automatic mask generation for 3D reconstruction of a murine brain from histology slices [12], and manual landmark definition.Other uses of ITK-SNAP include manual post processing for automatic brain extraction tools and 3D smoothing of slice-by-slice manual segmentations with large step-edge artifacts.

Improved Efficiency and Reliability for Caudate Segmentation
To validate ITK-SNAP against the standard of manual segmentation, we compared the two approaches in the context of an ongoing child neuroimaging autism study conducted in partnership with the UNC Chapel Hill Psychiatry Department.Segmentation of the caudate from T1-weighted MRI was chosen as the application for validation.The caudate is a gray matter structure that presents challenges for segmentation because boundaries with the adjacent gray matter structures (nucleus accumbens, putamen) are not visible in MRI (see Fig. 3).The validation study was conducted on a set of 15 MR images acquired with voxel size 0.78 2 × 1.5mm 3 and resampled to have isotropic voxels of size 1mm 3 .
Using the drawing tools in ITK-SNAP, we have developed a highly reliable manually assisted caudate segmentation procedure using slice-by-slice boundary drawing in all orthogonal orientations.Additionally to the boundary overlays, the segmentation is supported by a 3D display of the segmented structure.Coupled cursors between slices and 3D display provide additional help to significantly reduce slice-by-slice jitter as often seen in this type of segmentations.The time for manual segmentation of the left and right caudate is approximately 1.5 hours for experienced experts.
In the course of the autism study, we developed a new protocol for caudate segmentation using the level set workflow in ITK-SNAP.The distributions of T1 intensity values in the caudate regions in infant MRI show significant differences among subjects.In order to achieve optimal reliability, we estimate these distributions by using ITK-SNAP to mark regions in four axial slices of the caudate and compute the mean intensity and standard deviation in these regions.These estimates are then used to set the upper and lower fuzzy thresholds that define probability maps P obj and P bg .Parameters α and β, which determine the tradeoff between internal and external forces in contour evolution, were trained in a pilot study and kept unchanged.The regions used to estimate intensity distributions were also used to initialize the contour.Contour evolution was terminated manually once the contour started to bleed into the adjacent putamen.
In this protocol, optimal threshold selection and contour evolution require only about 5 minutes for the left and right caudate combined.In some cases, the expert may choose to apply a cut plane to correct the segmentation in 3D, which only takes a few additional seconds.In our autism project, we decided to add a precise a detailed separation from the putamen and a masking of the left and right nucleus accumbens.This is a purely manual operation since there are no visible boundaries between caudate and nucleus accumbens in MRI.This step added another 30 minutes to the whole process.Still, the total segmentation time was reduced from 1.5 hours for slice-by-slice contour drawing to 35 minutes, with the option to be reduced to only 5 minutes if simple cut-planes for inferior boundaries would be sufficient.The raters also reported that they felt much more comfortable with ITK-SNAP since they could focus their effort on a small part of the boundary that is most difficult to identify.
Table 1 reports intra-rater and inter-rater reliability scores for manual and ITK-SNAP segmentations of the caudate nucleus.The reliability scores are com-Table 1. Intra-and inter-rater reliability of caudate segmentation, measured by analyzing volumes of the segmented structures for 3 replications of 5 test datasets by two raters.Reliability values for manual segmentation by two experts (top), manual contouring versus ITK-SNAP, and ITK-SNAP segmentation by two experts show excellent reliability of both methods and excellent agreement between manual segmentation and ITK-SNAP.However, training to establish reliability was several months for the manual method and significantly shorter for ITK-SNAP.Also, ITK-SNAP reduced user time from 1.5 hours to 35 minutes.puted as the intra-class correlation coefficients [9] of the volumes of the segmented structures produced by different raters.The results show not only the excellent reliability of ITK-SNAP segmentation but also reflects the excellent reliability of the manual experts trained over several months.Therefore, reliability between methods is not different.The ITK-SNAP segmentation, which requires much less training and is significantly more efficient, is shown equivalent to the manual expert, both with respect to intra-method reliability and intermethod validity.However, this is to be compared with the significantly reduced segmentation time and short rater training time of ITK-SNAP.

Conclusions
ITK-SNAP is an open source medical image processing application that fills a specific set of biomedical research needs: it provides a combination of manual and expert-controlled automatic tools for segmentation of structures in 3D anatomical images of varying modalities and anatomical regions.To minimize the amount of training needed to use the tool, the UI is focused on the specific task of segmentation, parameter selection is simplified using live feedback, and the "creep" of features unrelated to segmentation is carefully managed.
The validation study demonstrates excellent reliability of ITK-SNAP for efficient 3D segmentation.The tool competes with highly trained raters, which explains the similar range of intra-method reliability.Besides efficiency, our experts preferred ITK-SNAP over slice contouring due to its capability to segment directly in 3D and the simple option to postprocess the segmentation results.

Fig. 1 .
Fig. 1.Left.ITK-SNAP user interface shows three orthogonal views of a volumetric image, linked by a common cursor (light blue crosshairs).A fourth panel is used to view the segmented structures in three dimensions.Right.The automatic segmentation workflow is organized as a wizard, shown here in the initialization stage.

Fig. 2 .
Fig. 2. Live feedback in ITK-SNAP.Left.A fuzzy two-sided threshold is being used to define object and background probability maps; the maps are displayed to the user of the fly.Right.Parameter selection dialog for contour evolution.The effect of the parameters on the forces acting on the contour is illustrated in a 2-D example.

Fig. 3 .
Fig. 3.The caudates viewed in 2-D and 3D: coronal MRI slice (left), overlay of segmented structures (middle), and 3D view of the caudate and putamen relative to the lateral ventricles (right).Right and left caudates are shown in green and red; left and right putamen in yellow; and the outline of the nucleus accumbens is shown in red.Note the lack of contrast at the boundary between the caudate and the nucleus accumbens, and the fine-scale cell bridges between the caudate and the putamen.