Performance comparison of 10 different classification techniques in segmenting white matter hyperintensities in aging
Introduction
White matter hyperintensities (WMHs), commonly identified as areas of increased signal in relation with the surrounding white matter regions on T2w, PD and FLAIR MRIs, are one of the non-specific yet typical and constant MRI expressions of cerebral small vessel disease (CSVD), along with lacunar infarcts and microhemorrhages (Conklin et al., 2014, Gouw et al., 2010). They have been shown to be more extensive in patients with Alzheimer's disease compared to age-matched healthy normal populations (Yoshita et al., 2005). WMHs reflect ischemic injury in the elderly and AD populations and the existence and severity of WMHs can lead to or accelerate decline in cognitive as well as executive functions (Dubois et al., 2014). As a result, the location and load of WMHs are important clinical measures, raising substantial need for their accurate quantifications. WMHs are generally detected using fluid attenuated inversion recovery (FLAIR) or T2w/PD scans. Manually labeling WMHs is challenging due to time constraints as well as inter-rater and intra-rater variabilities (Grimaud et al., 1996). As a result, automated tools that can segment WMHs robustly and with high accuracy are extremely useful, particularly in large scale studies such the Alzheimer's Disease Neuroimaging Initiative (http://www.loni.ucla.edu/ADNI/), the National Alzheimer's Coordinating Center (NACC) database (https://www.alz.washington.edu/) and others where it is desired to estimate the contribution of neurovascular disease to cognitive decline.
The heterogeneity in the distribution and patterns of WMHs makes the segmentation task intrinsically complex (Caligiuri et al., 2015). Automated segmentation tools usually integrate information from multiple complementary MRI contrasts including T1w, T2w, PD and FLAIR to reduce uncertainty and improve segmentation accuracy. Most successful fully automated WMH segmentation techniques extract a combination of location and intensity features from these images and use them as inputs to a linear or nonlinear classifier. Here we review the most commonly used linear and nonlinear classifiers in general as well as their application to the task of segmenting lesions in general or WMHs of vascular etiology specifically.
While there have been many studies attempting to segment WMHs using these classification techniques, drawing meaningful comparisons between their performances is not possible since they have been applied to different datasets and results are highly variable across different populations and imaging protocols (García-Lorenzo et al., 2013; Caligiuri et al., 2015). To our knowledge, no studies have compared the performance of these classification techniques for detecting WMHs against one another on the same datasets, especially for cases where classification is attempted without using the optimal FLAIR information. In this paper, we have extensively compared the performance of these different classification techniques in detecting WMHs with and without FLAIR information using 3 different large publicly available datasets with different scanners and acquisition protocols. This enables us to draw more generalizable conclusions regarding the performance of the classifiers. Our contributions include an extensive comparison of 10 widely used classification techniques in detecting WMHs across 4 different datasets, three of which are from multi-site and multi-scanner studies and across different combinations of imaging modalities. In addition, we make publicly available an implementation of the segmentation tool along with all the pre-trained classifiers (http://nist.mni.mcgill.ca/?p=221). The proposed tool is generalizable to data from different scanners since it has been trained on data from multiple scanners.
Section snippets
Subjects
The performances of the different classifiers were assessed based on four datasets of subjects with different ranges of WMH loads. Table 1 shows the demographic information for each dataset.
Segmentation using T1w, T2w, PD, and FLAIR
The performance of each classifier was validated through 10-fold cross validation using T1w, T2w, PD, and FLAIR images for the ADC dataset. All voxels within a brain mask that contained the cerebrum, cerebellum and brain stem were classified. Table 5 shows the average Dice Kappa, detection/outline error rates (DER/OER), ICC, TPR, and PPV values for different classifiers. Fig. 3 shows boxplot diagrams for the same results separately for subjects with small, medium and large WMH loads. Fig. 4
Discussion
In the recent years, there have been many different studies in the literature that address the challenge of automatically segmenting WMHs (Caligiuri et al., 2015, Admiraal-Behloul et al., 2005, Anbeek et al., 2004, Beare et al., 2009, De Boer et al., 2009, Dyrby et al., 2008, Ghafoorian et al., 2016a, Griffanti et al., 2016, Ithapu et al., 2014, Lao et al., 2008, Ong et al., 2012, Schmidt et al., 2012, Simões et al., 2013, Steenwijk et al., 2013, Wu et al., 2006a, Wu et al., 2006b, Yoo et al.,
Acknowledgement
We would like to acknowledge funding from the Famille Louise & André Charron. This work was also supported by grants from the Canadian Institutes of Health Research (MOP-111169), les Fonds de Research Santé Québec Pfizer Innovation fund, an NSERC CREATE grant (4140438 − 2012), the Levesque Foundation, the Douglas Hospital Research Centre and Foundation, the Government of Canada, and the Canada Fund for Innovation. This research was also supported by NIH Grants P30AG010129, K01 AG030514, and the
References (78)
- et al.
Fully automatic segmentation of white matter hyperintensities in MR images of the elderly
NeuroImage
(2005) - et al.
Segmentation of magnetic resonance brain images through discriminant analysis
J. Neurosci. Methods
(2003) - et al.
Probabilistic segmentation of white matter lesions in MR imaging
NeuroImage
(2004) - et al.
A new method for structural volume analysis of longitudinal brain MRI data and its application in studying the growth trajectories of anatomical brain structures in childhood
NeuroImage
(2013) - et al.
Development and validation of morphological segmentation of age-related cerebral white matter hyperintensities
Neuroimage
(2009) - et al.
Automatic segmentation of magnetic resonance images using a decision tree with spatial information
Comput. Med. Imaging Graph.
(2009) - et al.
MRI segmentation: methods and applications
Magn. Reson. Imaging
(1995) - et al.
White matter lesion extension to automatic brain tissue segmentation on MRI
Neuroimage
(2009) - et al.
Advancing research diagnostic criteria for Alzheimer's disease: the IWG-2 criteria
Lancet Neurol.
(2014) - et al.
Segmentation of age-related white matter changes in a clinical multi-center study
NeuroImage
(2008)
Individualized statistical learning from medical image databases: application to identification of brain lesions
Med. Image Anal.
Unbiased average age-appropriate atlases for pediatric studies
NeuroImage
Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging
Med. Image Anal.
Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images
NeuroImage
BIANCA (Brain Intensity AbNormality Classification Algorithm): a new tool for automated segmentation of white matter hyperintensities
NeuroImage
Quantification of MRI lesion load in multiple sclerosis: a comparison of three computer-assisted techniques
Magn. Reson. Imaging
Fully automatic segmentation of multiple sclerosis lesions in brain MR FLAIR images using adaptive mixtures method and Markov random field model
Comput. Biol. Med.
Simple methods for segmentation and measurement of diabetic retinopathy lesions in retinal fundus images
Comput. Methods Prog. Biomed.
Computer-assisted segmentation of white matter lesions in 3D MR images using support vector machine
Acad. Radiol.
Extra Tree forests for sub-acute ischemic stroke lesion segmentation in MR sequences
J. Neurosci. Methods
Lesion segmentation from multimodal MRI using random forest following ischemic stroke
NeuroImage
Automatic white matter lesion segmentation using an adaptive outlier detection method
Magn. Reson. Imaging
An automated tool for detection of FLAIR-hyperintense white-matter lesions in multiple sclerosis
NeuroImage
A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions
NeuroImage
Automatic segmentation of cerebral white matter hyperintensities using only 3D FLAIR images
Magn. Reson. Imaging
Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (kNN-TTPs)
NeuroImage Clin.
A fully automated method for quantifying and localizing white matter hyperintensities on MR images
Psychiatry Res. Neuroimaging
Automated segmentation of multiple sclerosis lesion subtypes with multichannel MRI
NeuroImage
Textural based SVM for MS lesion segmentation in FLAIR MRIs
Open J. Med. Imaging
Automatic segmentation and classification of multiple sclerosis in multichannel MRI
IEEE Trans. Biomed. Eng.
Adult cerebrovascular disease: role of modified rapid fluid-attenuated inversion-recovery sequences
Am. J. Neuroradiol.
An introduction to kernel and nearest-neighbor nonparametric regression
Am. Stat.
Fluid-attenuated inversion recovery magnetic resonance imaging detects cortical and juxtacortical multiple sclerosis lesions
Arch. Neurol.
Imaging of white matter lesions
Cerebrovasc. Dis.
The National Alzheimer's Coordinating center (NACC) database: an Alzheimer disease database
Alzheimer Dis. Assoc. Disord.
Bagging predictors
Mach. Learn.
Random Forests
Mach. Learn.
Automatic detection of white matter hyperintensities in healthy aging and pathology using magnetic resonance imaging: a review
Neuroinformatics
Cited by (55)
Understanding the impact of radical changes in diet and the gut microbiota on brain function and structure: rationale and design of the EMBRACE study
2023, Surgery for Obesity and Related DiseasesRacial differences in white matter hyperintensity burden in older adults
2023, Neurobiology of AgingQuantitative analysis of white matter hyperintensity: Comparison of magnetic resonance imaging image analysis software
2022, Journal of Stroke and Cerebrovascular DiseasesWhite matter hyperintensities are associated with grey matter atrophy and cognitive decline in Alzheimer's disease and frontotemporal dementia
2022, Neurobiology of AgingCitation Excerpt :For each subject, the T2-weighted, PD, and FLAIR scans were then co-registered to the T1-weighted scan of the same visit, using a rigid body registration (6 parameters: 3 translations and 3 rotations) with a mutual information cost function (Dadar et al., 2018a). Using a previously validated WMH segmentation method, the WMHs were automatically segmented for all longitudinal visits, using the preprocessed and co-registered scans (Dadar et al., 2017c). The quality of the registrations and segmentations was visually assessed, and the results that did not pass this quality control were excluded (n = 102 out of 5774 timepoints).
- 1
Part of the data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at:
http://adni.loni.usc.edu/wp-ontent/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf