http://orcid.org/0000-0003-0752-941X
Figures
Abstract
Background
Many classification methods have been proposed based on magnetic resonance images. Most methods rely on measures such as volume, the cerebral cortical thickness and grey matter density. These measures are susceptible to the performance of registration and limited in representation of anatomical structure. This paper proposes a two-stage local feature fusion method, in which deformable registration is not desired and anatomical information is represented from moderate scale.
Methods
Keypoints are firstly extracted from scale-space to represent anatomical structure. Then, two kinds of local features are calculated around the keypoints, one for correspondence and the other for representation. Scores are assigned for keypoints to quantify their effect in classification. The sum of scores for all effective keypoints is used to determine which group the test subject belongs to.
Results
We apply this method to magnetic resonance images of Alzheimer's disease and Parkinson's disease. The advantage of local feature in correspondence and representation contributes to the final classification. With the help of local feature (Scale Invariant Feature Transform, SIFT) in correspondence, the performance becomes better. Local feature (Histogram of Oriented Gradient, HOG) extracted from 16×16 cell block obtains better results compared with 4×4 and 8×8 cell block.
Citation: Li T, Li W, Yang Y, Zhang W (2017) Classification of brain disease in magnetic resonance images using two-stage local feature fusion. PLoS ONE 12(2): e0171749. https://doi.org/10.1371/journal.pone.0171749
Editor: Dewen Hu, National University of Defense Technology College of Mechatronic Engineering and Automation, CHINA
Received: October 11, 2016; Accepted: January 25, 2017; Published: February 16, 2017
Copyright: © 2017 Li et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data used in this paper are available from the publicly accessible OASIS study (http://www.oasis-brains.org/app/template/Tools.vm) and PPMI study (http://ida.loni.usc.edu/login.jsp?project=PPMI). The specific subjects we used in OASIS and PPMI are described in session Preprocessing.
Funding: This paper is supported by the National Natural Science Foundation of China grant 61432008, 81171314,61532006, 61305018 (http://www.nsfc.gov.cn/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Magnetic resonance imaging (MRI) is a powerful technique which provides rich information about anatomical structure [1]. As a non-invasive medical imaging technique, MR images have been widely used in neuroscience and brain disease research [2]. Growing interest has been focused on the accurate classification of brain disease using MR images.
Many MRI measurements have been used as features and combined with classification methods to aid the diagnosis of brain disease. The traditional measures extracted from the structural MR images include: 1) Voxel-wise density [3–16]; 2) Cortical thickness [17–22]; 3) Volume [17,19,23–31]; 4) Deformation information [32,33]. However, there are several constraints in such MRI measures: 1) A fundamental assumption underlying these statistical measures is that inter-subject registration is capable of achieving one-to-one correspondence [35–37]. As the knowledge of correspondence is ambiguous or non-existent, this assumption may be potentially unrealistic [37,38]. 2) To extract these measures precisely, the brains require nonlinear alignments to a template. For the lack of ground truth, it's hard to evaluate the performance of inter-subject registration [39]. There is the risk of misalignment or over-alignment, and the differences caused by disease may be removed in the registration process [37,40]. 3) The cortical thickness and volume information are usually averaged from some predefined region of interest (ROI). These ROIs rely on manual or semi-automatic segmentation, which is time consuming and prone to errors [36].
To overcome the above limitations, many researches attempt to represent MR images using local features instead of MRI measures. Local features calculated independently from individual subject images help to identify the same anatomical structure among different subjects. Local features are combined with feature-based morphometry (FBM) and bag of words (BOW) to represent MR images in some recent studies. In FBM, the anatomical characteristics are represented by Scale Invariant Feature Transform (SIFT) features, and group-related anatomical differences are expressed in terms of feature/group co-occurrence statistics [37]. Afterwards, many extensions of FBM are proposed. Such as Mwangi et al. [12] propose to combine FBM with voxel-based morphometry (VBM), and obtain a good classification results. Chen et al. [40] propose a novel MR analysis method based on FBM and support vector machine (SVM). BOW is originally applied in document categorization, and is gradually used to analyze MR images recently. The basic idea of this method is to represent the image visual content as a probability distribution (histogram) of local features (visual words) and collect a knowledge based from a set of images previously labeled [41]. Local features such as SIFT features [42], Laguerre Circular Harmonic Functions coefficients [35,43] are combined with BOW in the classification of brain diseases.
However, there still have several constraints in the aforementioned FBM-based and BoW-based algorithms:
- The location information of local features is ignored in BoW, thus the local features from different anatomical structure are substituted by the same visual word in the bag. The effect of local feature in correspondence would be weakened.
- In FBM, some significant features come from few subjects. In practical, as the number of sample is limited, some significant features may be unrepresentative in a larger dataset.
- In FBM, the SIFT features whose occurrence frequency are close in different groups provide little information for the final classification. As SIFT features are invariant to rotation and scale, two points which are similar with each other in SIFT descriptors may be different in the same scale.
To address the above constraints, in this paper, we aim to integrate the advantage of SIFT features in correspondence and the advantage of Histogram of Oriented Gradient (HOG) features in representation. Thus we propose a two-stage local feature fusion method. In correspondence stage, we extract keypoints and SIFT descriptor from scale-space in MR images. SIFT descriptors are used to correspond same anatomical structure among different subjects. The representative keypoints which exist in most of sample are reserved for they are more likely to appear in the test subjects. In representation stage, HOG descriptors will be calculated to demonstrate the local character around the representative keypoints from the same scale. The difference in HOG descriptors between two groups is used to weight the effect of the representative keypoints. SVM classifier is constructed for a representative keypoint in terms of HOG descriptors. For a new subject to be classified, we firstly identify matches of the representative keypoints, and then assign scores for the matches based on trained SVM classifier. The final classification result depends on the total score.
The contributions of this paper are as follows: 1) A two-stage local feature fusion method is proposed, in which SIFT features are used for correspondence and HOG features are used to character anatomical structure. 2) To reduce the calculation complexity and construct a standard keypoint set, the template brain is used as reference. 3) A quantification of representative effect of keypoints is adopted.
The rest of this paper is organized as follows. Firstly, the whole framework of classification via two-stage local feature fusion is introduced in Section 2. Then the experiments and results are presented in Section 3. Finally, the discussion and conclusion of this paper are followed in Section 4.
Method
The overall flow of proposed framework is illustrated in Fig 1. This framework can be broke up into four steps, which is summarized as follows: preprocessing, correspondence, representation, and classification.
The overall process contains the following steps: (1) the training MR images are aligned to template brain; (2) keypoints are extracted for every subject and matched among different subjects; (3) HOG descriptors are calculated for keypoints, and differences in terms of HOG is quantified. The effect of keypoint in classification is represented through assigning scores for them. (4) based on keypoints and their scores, testing subject is classified.
Preprocessing
The proposed method in this study is evaluated on public datasets in Open Access Series of Imaging Studies (OASIS) [44] for Alzheimer's disease (AD) and Parkinson's Progression Markers Initiative (PPMI) [45] for Parkinson's disease (PD).
OASIS dataset.
OASIS contains 98 normal control (NC) subjects and 99 probable AD subjects aged 60–96. For the later comparison, these subjects are subdivided into two subsets according to their ages and dementia statuses measured by Clinical Dementia Rating (CDR):
- AD-86: 86 subjects aged 60–80 years are chosen, including 20 patients with mild AD (CDR = 1) and 66 healthy subjects (CDR = 0) [40];
- AD-126: 126 subjects aged 60–96 years are chosen, including 28 patients with mild AD (CDR = 1) and 98 healthy subjects (CDR = 0) [40].
Scans in OASIS subjects are first averaged and gain-field corrected in advance to improve signal/noise ratio, and then registered to Talairach space via affine transform and the skull are masked out. Marcus et al [44] present the detailed description of the preprocessing steps for this dataset. These subjects are subdivided into two subsets according to their ages and dementia statuses measured by Clinical Dementia Rating.
PPMI dataset.
The PPMI cohort comprises more than 400 Parkinson's disease (PD) subjects and more than 200 healthy subjects acquired from 21 clinical sites with different scanner parameters [45]. We use T1-weighted MP-RAGE images whose slice thickness is 1.0mm and acquisition plane is sagittal. We found 67 NC subjects and 145 PD subjects. These subsets will be used to evaluate the usability of this method. These subjects are subdivided into two subsets according to their ages and depression statuses measured by Geriatric Depression Scale (GDS):
- PD-46: 46 subjects aged 50–65 years are chosen, including 20 healthy subjects (GDS<4) and 26 PD subjects (GDS>4).
- PD-212: 212 subjects aged 30–85 years are chosen, including 67 healthy subjects and 145 PD subjects.
Correspondence
We extract maxima and minima of scale-normalized Laplacian of Gaussian [46]. These extremal points are considered as keypoints in this paper, which stand for anatomical structures. SIFT features are widely used in registration for its invariance to image scale and rotation. In this paper, we use SIFT features to correspond same anatomical structure (keypoint) among different subjects.
In correspondence part, firstly, we extract keypoints from scale-space and calculate SIFT descriptors for them in train subjects and template brain ICBM_152. Secondly, for a keypoint in template brain, we locate matches in training subjects. The last but not the least, keypoints which find matching points in few subjects are filtered out.
Extracting keypoints and SIFT descriptors.
After transforming 3-D brain images to 2-D slices as shown in Fig 2, package vlFeat [47] is used to extract keypoints and SIFT descriptors from these slices. The SIFT descriptor is described by 128 numbers which characterize the image appearance around the keypoint in more detail. A keypoint corresponds to a slice orientation (coronal, axial or sagittal), a slice order of the 2-D slice from which the keypoint is extracted, an orientation, a central location. We extract keypoints and SIFT descriptors for every subject in patient group, normal control group and the template brain ICBM_152. To be noted, the appearance matrix is stretched into vector, and the vector is normalized to unit length.
The brain is sliced along 3 orientations to obtain 2D images, and SIFT features are extracted from 2D images.
Here, an individual feature is denoted as fi = {di, li, xi, oi, ai} where di and li are the slice orientation and slice order of the 2-D slice respectively, and xi is the center location of keypoint, and ai is a vector of image measurements representing the image appearance around keypoint. Let Fp,m(m ∈ [1, M]), Fc,n(n ∈ [1, N]) and Ft represent all keypoints extracted from the mth subject in patient group, the nth subject in control group and the template brain ICBM_152. Where M is the total number of patient subjects, and N is the total number of nomal control subjects. To illustrate this method more clearly, keypoint is expressed in different ways. In this paper, the keypoint in template brain is still called "keypoint", and the keypoint in training and testing subjects is called "candidate point". For a keypoint in template, the correspondence found in training and testing subjects is called "matching point".
Locating matching points.
To locate matching point in training subjects, a common method is clustering. However, two or more candidate points of one subject may be assigned into one cluster due to noise or clustering algorithm. In this work, we propose a concise method to accomplish this goal via template brain. We assume that the candidate points which appear in training and testing subjects can all be covered by keypoints extracted from template brain ICBM_152.
The matching algorithm is performed between one image in template brain and the image in training subjects which shares the same slice orientation and slice order. For example, suppose fi is a keypoint extracted from the brain template, and we plan to locate the matching point in the mth subject of patient group. The matching algorithm can be subdivided into three steps:
Firstly, from Fp,m, we retrieve the candidate points which share the same slice orientation and slice order with fi,
(1)
Secondly, we calculate the Euclidean distance for the invariant descriptor vector between fi and keypoint fj in Ftemp.
Thirdly, we compare the Euclidean distance of the closest neighbor to that of the second-closest neighbor. If this distance ratio is smaller than εmatch, we believe that we find the matching point in the mth subject of patient group.
The idea behind this matching algorithm is that correct matching point should have the closest neighbor significantly closer than the closest incorrect matching point to achieve reliable matching [48]. This matching algorithm is based on Lowe's work [48]. Specially, if there is just one candidate point in Ftemp, we don't believe that we have found the correct matching point.
Discarding unrepresentative keypoints.
For a keypoint fi in template brain, we identify the matching points in patient group and control group. For keypoint fi, let Si,p and Si,c be the set of matching keypoints in patient group and normal control group respectively. In this paper, we aim to find the representative keypoints (anatomical structure) which appear in most of subjects. Therefore, the keypoints which find matching points in a fraction of subjects are not what we pursue. Specifically, if |Si,p|< εrate × M or |Si,c|< εrate × N, we consider fi a unrepresentative keypoint, otherwise we believe fi is a effective keypoint. Where |⋅| is the cardinal number of a set.
After discarding unrepresentative keypoints, the total number of keypoints in template brain is reduced to K. Namely, for every effective keypoint fi in template brain, we have a patient matching set Si,p and normal control matching set Si,c.
Representation
In representation part, firstly, histogram of oriented gradients (HOG) descriptors is extracted for matching points. Then, the effect of keypoint in classification is quantified based on the difference of two groups in HOG descriptor. Finally, a SVM classifier is constructed for every keypoint.
HOG descriptors.
SIFT feature is invariant to scale, which indicates that the appearance matrix of a keypoint may be calculated in different scales. To demonstrate the differences in morphology of the same anatomical structure between different groups better, we extract local features in the same scale (the original image). A local feature similar with HOG descriptor is used to depict the gradient character of a block around the keypoint. For a matching point fi = {di, li, xi, oi, ai}, the extraction of local feature can be described as:
- To make the local feature (HOG) be rotationally invariant, the image from which keypoint fi is extracted should be rotated relative to the dominant orientation oi in advance.
- The image gradient magnitudes and orientations are sampled in a 16×16 cell block shown in Fig 3(a) around the keypoint location xi.
- Gradient magnitudes are accumulated into orientation histograms summarizing the contents over 4×4 subregions with 8 histogram channels shown in Fig 3(b) and 3(c).
- The description matrix is converted to a local feature (HOG) vector, and the vector is then normalized to unit length.
Gradient magnitudes and orientations are sampled, and then accumulated into orientation histogram with 8 channels. (a) Cell block. (b) 8 histogram channels. (c) Oriented gradients matrix.
The final local feature (HOG) vector is a geometric description of keypoint. Similar calculation is done for the matching points in Si,p and Si,c. The local features extracted for Si,p and Si,c are denoted by Qi,p and Qi,c. The dimensions of matrix Qi,p are H × |Si,p|, and the dimensions of matrix Qi,c are H × |Si,c|. Where |⋅| is the cardinal number of a set, and H is the length of a HOG feature vector.
Quantification.
In this section, we will evaluate the difference in geometry of this anatomical structure between patient group and normal control group. RV coefficient [49] will be used to implement this difference measure.
For keypoint fi, the local features (HOG) in patient group and normal control are Qi,p and Qi,c respectively. The RV coefficient for these two set can be described as:
(2)
Where tr(.) is the trace operator of square matrix, QT is the transpose of matrix Q.
The value of RV coefficient ranges from 0 to 1. If RV coefficient is 0, the two sets are independent, which means there is no correlation or similarity between the two data sets [50]. If RV coefficient is 1, the eigen components of data set Qi,p can be derived from Qi,c through a homothetic transformation, which means that there exists a rotation matrix H and a scaling factor c such that cQipH = Qic. Namely, larger RV coefficient means more similarity.
The effect of keypoint fi in classification is quantified in terms of the differences in local feature (HOG). As RV(Qi,p, Qi,c) reflects the similarity between the two data sets Qi,p and Qi,c. We use 1−RV(Qi,p, Qi,c) to represent the degree of difference between control group and patient group at keypoint fi. A keypoint is scored according to following expression:
(3)
SVM classifier.
For a keypoint fi(i ∈ [1, K]) in template brain, the local features (HOG) in patient group and normal control are Qi,p and Qi,c respectively. We train a SVM classifier for this keypoint to identify a new local feature is healthy feature or patient feature. The input samples to the SVM are Qi,p and Qi,c. The class labels are -1 and 1 if they are from healthy group and patient group respectively. A 2-class SVM implementation from libSVM [51] is used. As the number of feature is in the same order as the number of samples, linear SVM is chosen to classification. A linear SVM classifier is less prone to overfitting than a non-linear one.
To be noted, Qi,p and Qi,c are imbalanced, and a simple default strategy of guessing the majority class would give a nice classification results. To make the classifier more suitable, we use an over-sampling approach in which the minority class is over-sampled by creating "synthetic" examples [52]. SMOTE arithmetic is used in this paper.
Classification
For a testing subject to be classified, candidate points and SIFT descriptors Ftest are first extracted. Based on the effective keypoint fi(i ∈ [1, K]) in brain template, the keypoint score fiscore and corresponding SVM classifier, we classify the testing brain in following steps:
- Locating matching point. For a keypoint fi(i ∈ [1, K]) in template brain, we identify whether this anatomical structure exists in Ftest. The matching algorithm has been presented in Correspondence.
- Extracting local features. If there exists a matching point in Ftest for fi(i ∈ [1, K]), we extract HOG descriptor θi at the central location based on method presented in Representation.
- Assigning scores. For local feature θi, we label it based on the trained SVM classifier:
(4)
- Classifying new subject. The final classification result depends on the sum of scores LSsum = ∑iLS(θi) of all matched keypoints. The brain is classified as patient brain if LSsum is larger than threshold εc and is classified as normal control brain otherwise.
(5)
Ideally, the threshold is 0, however, the occurrence frequency of keypoints in patient group and control group is inconsistent. We obtain this threshold similar with Chen et al.' [40] method by classifying the training brains and then finding the threshold that minimize the difference between false negatives and false positives. Finally, we use this threshold to predict new brains.
Parameter setting
There are several thresholds including matching threshold εm, and RV threshold εrv. Matching threshold εm is set as 0.8 which follows the suggestion of Lowe's paper [48]. Rate threshold εrate is set as 0.5. The size of the cell block also affects the final result. Different sizes of cell block are tested in the following evaluation.
Ethical standards and patient consent
Datasets AD-86 and AD-126 are obtained from the OASIS database. Written informed consent is obtained from all subjects and the use of these subjects is approved by Washington University (http://www.oasis-brains.org/).
Datasets PD-46 and PD-212 are obtained from the PPMI database. PPMI database is funded by the Michael J. Fox Foundation for Parkinson’s Research and funding partners, including Abbvie, Avid, Biogen Idec, Br Bristol-Meyers Squibb, Covance, GE Health care, Genentech, GlaxoSmithKline, Lilly, Lundback, Merck, Meso Scale Discovery, Pfizer, Piramal, Roche, Servier, and UCB (www.ppmi-info.org/fundingpartners). Written informed consent is obtained from all subjects.
Experiment and results
Visualization of keypoints
FBM-based methods and BOW-based methods try to utilize the unbalance of SIFT features occurrence in different groups. However, our method tries to identify the keypoints which exist difference in anatomical characteristics from same image scale. To identify the discriminative brain regions, AAL atlas was used to map the keypoints. Examples of 4 keypoints for OASIS which appear in the same anatomical structure of left brain and right brain with a high score are shown in Fig 4.
Four keypoints are listed, which appear in the same anatomical structure of left brain and right brain with a high score.
For AD discrimination, the keypoints which have a high score are distributed mainly in thalamas, temporal lobe, calcarine and cingulum. A number of papers have reported these four regions [53–56]. Thalamas is responsible for the regulation of consciousness [57]. AD patients are usually loss of motivation and not managing self care [58]. Thalamas may be related with this symptom. The temporal lobe is very reasonable region with considerations of its important role in processing sensory input to derived meanings [59]. Temporal lobe is an import part for the appropriate retention of visual memory, language comprehension and emotion association, which is associate with the symptom (short-term memory loss, emotion swing) of AD [58,59]. Cingulum is very important to brain structure connectivity and information integration [60]. Damage in this area may result in many mental disorders.
For PD discrimination, the most significant keypoints are distributed mainly in thalamas, frontal lobe, occipital lobe which in accord with many papers [61–63]. Thalamas is responsible for the regulation of consciousness [57]. This is in consist with PD patients' symptoms such as sensory and emotional problems [64]. The frontal lobe is an very important part for voluntary movement which may cause the behavioral problems in PD patients [65].
Evaluation and performance
We perform leave-one-out cross validation for all datasets. One subject is chosen as testing subject once and the remaining subjects are used for training. The accuracy, sensitivity and specificity of the classification are computed for all datasets.
Keypoints.
Intuitively, we assume that the keypoints which present more difference between two groups contribute more to the final classification.
In this section, we take the top 25, 50,…, 225 keypoints ranking according the difference of HOG descriptors between groups. The classification results for AD-86, AD-126, PD-46 and PD-212 are illustrated in Fig 5. For AD-86 and PD-46, as the number of keypoints increases, the classification accuracy increased at first and then decreased. For AD-126 and PD-212, the method obtains best performance when taking 75 or 100 keypoints. Then the performance swings a little. From the curve in Fig 5, we find that classification results in four groups with 100 keypoints are not bad. In this paper, we take the top 100 keypoints ranking according to the difference of HOG descriptors for the classification of AD-86 and AD-126.
As the number of keypoints increases, the classification accuracy changes in AD-86, AD-126, PD-46 and PD-212.
To verify our assumption, we take another test. In this paper, we propose to quantify the effects of keypoints in terms of the difference in local feature (HOG). The simplest method to quantify keypoints is assigning same weight for all keypoints. This corresponding to setting fiscore = 1. Two types of scoring method are compared in this paper.
The comparison is done in group AD-86 and AD-126 shown in Table 1. For AD-86, PD-46 and PD-212, there is an improvement in the classification accuracy when assigning weight for keypoints in terms of the difference of local feature (HOG). However, for AD-126, assigning weights for keypoints leads to a lower classification accuracy. This may be due to the effect of keypoints which have a low score. These keypoints affect the final results.
Cell block for local feature HOG.
Local feature (HOG) is extracted from cell blocks. The size of cell blocks plays an important effect on the final classification. Three types of cell blocks are compared in this paper, and the performance is displayed in Table 2.
The results in Table 2 suggest that a bigger cell block leads to a better result in general. HOG descriptor extracted from a larger region may be more likely to demonstrate the detailed information around the keypoint. The results also suggest that the oriented gradients matrix in a small region would be hard to describe the detailed structural around the keypoint, and the difference between group would be deviated from the reality. In order to balance the speed and the accuracy, 16×16 cell blocks will be used in this paper.
Different local feature for representation.
We try different types of local feature for representation and analyze which kind of local feature is suitable to demonstrate difference between healthy control group and patient group. Three kinds of local feature are compared in this paper: gray value (GV), gray-level co-occurrence matrix (GLCM) and HOG. The performance is displayed in Table 3. From the result, we find that classification with GLCM in AD-86 and AD-126 is very poor, this may due to that sulcus is widespread exists and GLCM is difficult to describe the difference. The classification results using GLCM and gray value are similar in PD-46 and PD-212. Though worse than HOG, gray value may be able to provide effective information for the final classification.
Effect of local feature (SIFT) in Correspondence.
To demonstrate the effect of local feature (SIFT) in correspondence, we compare the classification results with and without the use of SIFT. With the help of SIFT, we extract local feature (HOG) at the keypoints extracted from scale-space in training subjects. Without the use of SIFT, based on the central location of keypoints extracted from the brain template, we extract local feature (HOG) at the same position in training subjects. The comparison is done in four groups.
The performance of four datasets with and without the effect of local feature (SIFT) in correspondence is shown in Table 4. The results suggest that with the help of local feature (SIFT) in correspondence, the performance becomes much better. Keypoints extracted from scale-space help to locate the coordinate and SIFT features help to correspond among different subjects. This improvement supports the idea that local feature help to relieve the constraint in registration.
Accuracy.
For better comparison, we compare with methods which have been tested on the OASIS dataset. The subjects in this dataset have been preprocessed, avoiding the possible noise in preprocessing.
Toews et al. [37] propose a feature-based morphometry (FBM), in which group-related anatomical differences are expressed in terms of feature/group co-occurrence statistics. Daliri [42] proposes to combine BOW with SIFT features, followed by classification with SVM. Chen et al. [40] propose a MR analysis method based on FBM and SVM. Wang et al. [66] improve correspondences of localized patterns based on FBM. These four methods are all evaluated on AD-86 and AD-126. The comparison is summarized in Table 5. Equal Error Rate (EER) accuracy is used to evaluate the performance of classification method, so we follow this criterion. EER accuracy is calculated by first choosing a threshold so that the false positives rate is equal to false negative rate and then calculating the classification accuracy with the chosen threshold [37,40].
Discussion
Local features
There are many local features such as SURF [67], LBP [68], Extended LBP [69], WHGO [70], Multi-view feature [71] and so on, it's hard to select the best one. In this paper, we mainly demonstrate one method which combines the advantage of local feature in correspondence among different subjects and representation of anatomical structure. In this paper, we choose SIFT feature for correspondence and choose HOG feature for representation of anatomical structure.
SIFT features are widely used in registration to compensate the deformation of anatomical structures and achieve effective alignment among different subjects [72]. We choose SIFT feature for correspondence mainly due to its robustness and invariance for rotation and scale. Due to individual difference, same structure in different subjects may have great differences in the same scale. As SIFT features are invariant to scale, same structure can be robustly corresponded among different subjects. The performance of AD-86, AD-126, PD-46 and PD-212 with and without the effect of local feature (SIFT) in correspondence is displayed in Table 4. The result shows that SIFT features may help to achieve more precise correspondence among subjects and help to obtain better classification results.
The other local feature we use in this paper is designed to discriminate different structural pattern. Three kinds of local feature (GV, GLCM and HOG) are used to depict the texture around the keypoints and search difference between different groups. The performance in Table 3 shows that HOG outperforms GV and GV outperforms GLCM. GLCM perform very poor which beyond our expectation. We find that the numbers in the corners of GLCM are usually much larger than the numbers in the center of GLCM. This may be due to the structure of sulcus. Sulcus is so widespread that GLCM is difficult to reflect differences between groups. The results don't mean that HOG is the most suitable local feature for representation. The results reflect that gradient information outperforms gray value adjacency information and gray value. This may result from the special structural pattern of brain sulcus.
HOG features are finally selected to demonstrate the detailed gradient information of a cell block. The cell block in which we extract HOG descriptors plays an important role in classification as shown in Table 2. Measurements extracted from a small scale are limited in representation, leading to the bad performance in 4×4 cell block. Our results show that 16×16 outperforms 4×4 and 8×8. Measurements from moderate scale can be effectively useful for the representation of anatomical characteristics.
Keypoints
The keypoints extracted from MR images are used to represent the anatomical structure. The representation of 3-D images is greatly simplified via keypoints and template brain. Moreover, the keypoints extracted from maxima and minima of scale-normalized Laplacian of Gaussian produce the most stable local image features compared to a range of other points [46]. Some studies simplify the representation of 3-D images by t-test. For example, Hinrichs et al. [34] consider each voxel's intensity value as feature, and t-test is performed to rank the features by resulting p-values. The top ranked features are seen as a simplification of original 3-D image. This simplification is based on the assumption that one-to-one correspondence can be achieved among subjects precisely. As mentioned before, this assumption is ambiguous or non-existent. Besides, the significant region appears in the form of clusters, which result in redundancy. In this paper, we simplify MR images via keypoints. On one hand, keypoints can provide precise correspondence. On the other hand, simplification through keypoints is very concise, and a 3-D MR image can be represented by only five thousands of voxels.
Scoring methods for keypoint
In this paper, RV coefficient is used to measure the similarity between two sets of local features. We assume that the keypoints which have little difference in terms of HOG descriptors play little effect on the final classification. The results in Fig 5 and Table 1 suggest this assumption is reasonable. When the RV coefficient is high, we assume that there is no difference between two local feature groups. The results using this strategy shown in Table 1 perform well in terms of accuracy.
Limitation and future work
We use a 2-D SIFT algorithm mainly due to its robustness and feasibility, as many 3D SIFT implementations couldn't achieve the full orientation invariance respect to 3 degree of rotation freedom such as the 3DSIFT Matlab package [73]. Besides, for the limitation of computing capability in our lab, the serial images are down-sampled once to reduce the computation load. From the results testing on AD-86 and AD-126, we find that RV coefficient is sensitive to the number of sample. A more suitable quantification method should be studied for the difference of local features around keypoints in the future work.
Conclusion
We propose a novel framework for the diagnosis of brain diseases from MR images based on local features. Two kinds of local features are embedded in this frame, one for correspondence and the other for representation. Keypoints are extracted to represent anatomical structure. Scores are assigned for keypoints to quantify their effect in classification. The sum of scores for all keypoints is used to determine which group the test subject belongs to.
The proposed framework is evaluated on public dataset OASIS and PPMI. As shown in Table 4, the proposed method outperforms four SIFT-based methods in terms of EER accuracy more or less. The results suggest that the proposed method can be potentially a practical means to represent anatomical characteristics and aid to diagnose brain diseases from normal controls.
Author Contributions
- Conceptualization: TL WL.
- Formal analysis: TL.
- Funding acquisition: WZ WL.
- Investigation: TL.
- Methodology: TL WL.
- Software: TL.
- Writing – original draft: TL YY.
- Writing – review & editing: TL YY WZ WL.
References
- 1.
Behera NKS, Sahoo MK, Behera HS. HOG Based Radial Basis Function Network for Brain MR Image Classification. Computational Intelligence in Data Mining. Springer India. 2015.
- 2. Vijayalaxmi , Fatahi M, Speck O. Magnetic resonance imaging (MRI): A review of genetic damage investigations. Mutation Research/Reviews in Mutation Research. 2015; 764: 51–63. pmid:26041266
- 3. Abdulkadir A, Mortamet B, Vemuri P, Jack CR, Krueger G, Kloppel S, et al. Effects of hardware heterogeneity on the performance of SVM Alzheimer's disease classifier. Neuroimage. 2011; 58(3): 785–792. pmid:21708272
- 4. Costafreda SG, Chu C, Ashburner J, Fu CHY. Prognostic and diagnostic potential of the structural neuroanatomy of depression. PloS one. 2009; 4(7): e6353. pmid:19633718
- 5. Davatzikos C, Fan Y, Wu X, Shen D. Detection of prodromal Alzheimer's disease via pattern classification of magnetic resonance imaging. Neurobiology of aging. 2008; 29(4): 514–523. pmid:17174012
- 6. Ecker C, Rocha-Rego V, Johnston P, Mourao-Miranda J, Marquand A, Daly EM, et al. Investigating the predictive value of whole-brain structural MR scans in autism: a pattern classification approach. Neuroimage. 2010; 49(1): 44–56. pmid:19683584
- 7. Fan Y, Resnick SM, Wu X, Davatzikos C. Structural and functional biomarkers of prodromal Alzheimer's disease: a high-dimensional pattern classification study. Neuroimage. 2008; 41(2): 277–285. pmid:18400519
- 8. Gong Q, Wu Q, Scarpazza C, Lui S, Jia Z, Marquand A, et al. Prognostic prediction of therapeutic response in depression using high-field MR imaging. Neuroimage. 2011; 55(4): 1497–1503. pmid:21134472
- 9. Klöppel S, Stonnington CM, Chu C, Draganski B, Scahill RI, Rohrer JD, et al. Automatic classification of MR scans in Alzheimer's disease. Brain. 2008; 131(3): 681–689.
- 10. Lao Z, Shen D, Xue Z, Karacali B, Resnick SM, Davatzikos C. Morphological classification of brains via high-dimensional shape transformations and machine learning methods. Neuroimage. 2004; 21(1): 46–57. pmid:14741641
- 11. Magnin B, Mesrob L, Kinkingnéhun S, Pélégrini-Issac M, Colliot O, Sarazin M, et al. Support vector machine-based classification of Alzheimer’s disease from whole-brain anatomical MRI. Neuroradiology. 2009; 51(2): 73–83. pmid:18846369
- 12. Mwangi B, Ebmeier KP, Matthews K, Steele JD. Multi-centre diagnostic classification of individual structural neuroimaging scans from patients with major depressive disorder. Brain. 2012; 135(5): 1508–1521.
- 13. Nouretdinov I, Costafreda SG, Gammerman A, Chervonenkis A, Vovk V, Vapnik V, et al. Machine learning classification with confidence: application of transductive conformal predictors to MRI-based diagnostic and prognostic markers in depression. Neuroimage. 2011; 56(2): 809–813. pmid:20483379
- 14. Sun D, van Erp TGM, Thompson PM, Bearden CE, Daley M, Kushan L, et al. Elucidating a magnetic resonance imaging-based neuroanatomic biomarker for psychosis: classification analysis using probabilistic brain atlas and machine learning algorithms. Biological psychiatry. 2009; 66(11): 1055–1060. pmid:19729150
- 15. Vemuri P, Gunter JL, Senjem ML, Whitwell JL, Kantarci K, Knopman DS, et al. Alzheimer's disease diagnosis in individual subjects using structural MR images: validation studies. Neuroimage. 2008; 39(3): 1186–1197. pmid:18054253
- 16. Zeng LL, Xie L, Shen H, Luo Z, Fang P, Hou Y, et al. Differentiating Patients with Parkinson’s Disease from Normal Controls Using Gray Matter in the Cerebellum. Cerebellum. 2016; 1–7.
- 17. Arimura H, Yoshiura T, Kumazawa S, Tanaka K, Koga H, Mihara F, et al. Automated method for measurement of cerebral cortical thickness for 3-D MR images. International Journal of Computer Assisted Radiology & Surgery. 2006; 1:19–20.
- 18. Desikan RS, Cabral HJ, Hess CP, Dillon WP, Glastonbury CM, Weiner MW, et al. Automated MRI measures identify individuals with mild cognitive impairment and Alzheimer's disease. Brain A Journal of Neurology. 2009; 132(8),2048–57.
- 19. Jung WB, Lee YM, Kim YH, Mun C. Automated Classification to Predict the Progression of Alzheimer's Disease Using Whole-Brain Volumetry and DTI. Psychiatry investigation. 2015; 12(1): 92–102. pmid:25670951
- 20. Lerch JP, Pruessner J, Zijdenbos AP, Collins DL, Teipel SJ, Hampel H, et al. Automated cortical thickness measurements from MRI can accurately separate Alzheimer's patients from normal elderly controls. Neurobiology of aging. 2008; 29(1): 23–30. pmid:17097767
- 21. Jr OP, Nitrini R, Busatto G, Buchpiguel C, Sato JR, Jr AE. Use of SVM methods with surface-based cortical and volumetric subcortical measurements to detect Alzheimer's disease. Journal of Alzheimer's Disease. 2010; 19(4): 1263–1272. pmid:20061613
- 22. Querbes O, Aubry F, Pariente J, Lotterie J, Demonet J, Duret V, et al. Early diagnosis of Alzheimer's disease using cortical thickness: impact of cognitive reserve. Brain. 2009; 132(8): 2036–2047.
- 23. Cui Y, Wen W, Lipnicki DM, Beg MF, Jin JS, Luo S, et al. Automated detection of amnestic mild cognitive impairment in community-dwelling elderly adults: a combined spatial atrophy and white matter alteration approach. Neuroimage. 2012; 59(2): 1209–1217. pmid:21864688
- 24. Farhan S, Fahiem M A, Tauseef H. An ensemble-of-classifiers based approach for early diagnosis of Alzheimer’s disease: classification using structural features of brain images. Computational and mathematical methods in medicine. 2014, 862307–862307. pmid:25276224
- 25. Gerardin E, Chételat G, Chupin M, Cuingnet R, Desgranges B, Kim H, et al. Multidimensional classification of hippocampal shape features discriminates Alzheimer's disease and mild cognitive impairment from normal aging. Neuroimage. 2009; 47(4): 1476–1486. pmid:19463957
- 26. West M J, Kawas CH, Stewart WF, Rudow GL, Troncoso JC. Hippocampal neurons in pre-clinical Alzheimer’s disease. Neurobiology of aging. 2004; 25(9): 1205–1212. pmid:15312966
- 27. Zhu X, Suk HI, Shen D. A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis. NeuroImage. 2014; 100: 91–105. pmid:24911377
- 28.
Liu F, Suk HI, Wee CY, Chen H, Shen D. High-order graph matching based feature selection for Alzheimer’s disease identification. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer Berlin Heidelberg. 2013; 311–318.
- 29. Thung KH, Wee CY, Yap PT, Shen D. Identification of progressive mild cognitive impairment patients using incomplete longitudinal MRI scans. Brain Structure and Function. 2015; 1–17.
- 30. Liu F, Wee CY, Chen H, Shen D. Inter-modality relationship constrained multi-modality multi-task feature selection for Alzheimer's Disease and mild cognitive impairment identification. NeuroImage. 2014; 84: 466–475. pmid:24045077
- 31. Zhu X, Suk HI, Lee SW, Shen D. Subspace Regularized Sparse Multitask Learning for Multiclass Neurodegenerative Disease Identification. IEEE Transactions on Biomedical Engineering. 2016; 63(3): 607–618. pmid:26276982
- 32. Zhang D, Shen D, Alzheimer's Disease Neuroimaging Initiative. Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer's disease. Neuroimage. 2012; 59(2): 895–907. pmid:21992749
- 33. Duchesne S, Caroli A, Geroldi C, Barillot C, Frisoni GB, Collins DL. MRI-based automated computer classification of probable AD versus normal controls. IEEE transactions on medical imaging. 2008; 27(4): 509–520. pmid:18390347
- 34. Hinrichs C, Singh V, Xu G, Johnson SC, The Alzheimers Disease Neuroimaging Initiative. Predictive markers for AD in a multi-modality framework: an analysis of MCI progression in the ADNI population. Neuroimage. 2011; 55(2): 574–589. pmid:21146621
- 35. Ahmed OB, Benois-Pineau J, Allard M, Amar CB, Catheline G, The Alzheimers Disease Neuroimaging Initiative. Classification of Alzheimer’s disease subjects from MRI using hippocampal visual features. Multimedia Tools and Applications. 2015; 74(4): 1249–1266.
- 36. Simões R, van Walsum AMC, Slump CH. Classification and localization of early-stage Alzheimer’s disease in magnetic resonance images using a patch-based classifier ensemble. Neuroradiology. 2014; 56(9): 709–721. pmid:24948425
- 37. Toews M, Wells W, Collins DL, Arbel T. Feature-based morphometry: Discovering group-related anatomical patterns. NeuroImage. 2010; 49(3): 2318–2327. pmid:19853047
- 38. Maintz JBA, Viergever MA. A survey of medical image registration. Medical image analysis. 1998; 2(1): 1–36. pmid:10638851
- 39. Sotiras A, Davatzikos C, Paragios N. Deformable medical image registration: A survey. IEEE transactions on medical imaging. 2013; 32(7): 1153–1190. pmid:23739795
- 40. Chen Y, Storrs J, Tan L, Mazlack LJ, Lee JH, Lu LJ. Detecting brain structural changes as biomarker from magnetic resonance images using a local feature based SVM approach. Journal of neuroscience methods. 2014; 221: 22–31. pmid:24041480
- 41.
Rueda A, Arevalo J, Cruz A, Romero E, Gonzalez FA. Bag of features for automatic classification of Alzheimer’s disease in magnetic resonance images. Iberoamerican Congress on Pattern Recognition. Springer Berlin Heidelberg. 2012: 559–566.
- 42. Daliri MR. Automated diagnosis of Alzheimer disease using the scale-invariant feature transforms in magnetic resonance images. Journal of medical systems. 2012; 36(2): 995–1000. pmid:21584770
- 43.
Mizotin M, Benois-Pineau J, Allard M, Catheline G. Feature-based brain MRI retrieval for Alzheimer disease diagnosis. 19th IEEE International Conference on Image Processing. IEEE. 2012; 1241–1244.
- 44. Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience. 2007; 19(9): 1498–1507. pmid:17714011
- 45. Marek K, Jennings D, Lasch S, Siderowf A, Tanner C, Simuni T, et al. The parkinson progression marker initiative (PPMI). Progress in neurobiology. 2011; 95(4): 629–635. pmid:21930184
- 46.
Mikolajczyk K, Schmid C. An affine invariant interest point detector. European conference on computer vision. Springer Berlin Heidelberg. 2002: 128–142.
- 47.
Vedaldi A, Fulkerson B. Vlfeat: an open and portable library of computer vision algorithms. Intenational Conference on Multimedea. 2010:1469–1472.
- 48. Lowe DG. Distinctive image features from scale-invariant keypoints. International journal of computer vision. 2004; 60(2): 91–110.
- 49. Escoufier Y. On vector random variables(French: Le traitement des variables vectorielles). Biometrics. 1973; 29: 751–760.
- 50. Zhang H, Zhang X, Sun Y, Liu J, Li W, Tian J. A weighted-RV method to detect fine-scale functional connectivity during resting state. NeuroImage. 2011; 54(4): 2885–2898. pmid:21029783
- 51. Chang CC, Lin CJ. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology. 2011; 2(3): 27.
- 52. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research. 2002; 16: 321–357.
- 53. Brun A, Englund E. Regional pattern of degeneration in Alzheimer's disease: neuronal loss and histopathological grading. Histopathology. 1981; 5(5): 549–564. pmid:7286917
- 54. Chan D, Fox NC, Scahill RI, Crum WR, Whirwell JL, Keschziner G, et al. Patterns of temporal lobe atrophy in semantic dementia and Alzheimer's disease. Annals of neurology. 2001 49(4): 433–442. pmid:11310620
- 55. De Jong LW, Van der Hiele K, Veer IM, Houwing JJ, Westendorp RGJ, Bollen ELEM, et al. Strongly reduced volumes of putamen and thalamus in Alzheimer's disease: an MRI study. Brain. 2008; 131(12): 3277–3285.
- 56. Zhang Y, Schuff N, Jahng GH, Bayne W, Mori S, Schad L, et al. Diffusion tensor imaging of cingulum fibers in mild cognitive impairment and Alzheimer disease. Neurology. 2007; 68(1): 13–19. pmid:17200485
- 57.
Sherman SM, Guillery RW. Exploring the Thalamus. Elsevier. 2001.
- 58. Burns A, Iliffe S. Alzheimer’s disease. BMJ, 2009; 338:b157.
- 59.
Smith EE, Kosslyn SM. Cognitive psychology: mind and brain. Pearson/Prentice Hall. Pearson Education International. 2009.
- 60. Donoho DL, Tanner J. Precise Undersampling Theorems. Proceedings of the IEEE. 2010; 98(6):913–924.
- 61. Pedrosa DJ, Reck C, Florin E, Pauls KAM, Maarouf M, Wojtecki L, et al. Essential tremor and tremor in Parkinson's disease are associated with distinct ‘tremor clusters’ in the ventral thalamus. Experimental neurology. 2012; 237(2): 435–443. pmid:22809566
- 62. Petzinger GM, Fisher BE, McEwen S, Beeler JA, Walsh JP, Jakowec MW. Exercise-enhanced neuroplasticity targeting motor and cognitive circuitry in Parkinson's disease. The Lancet Neurology. 2013; 12(7): 716–726. pmid:23769598
- 63. Duncan GW, Firbank MJ, O'Brien JT, Burn DJ. Magnetic resonance imaging: a biomarker for cognitive impairment in Parkinson's disease?. Movement Disorders. 2013; 28(4): 425–438. pmid:23450518
- 64. Sveinbjornsdottir S. The clinical symptoms of Parkinson's disease. Journal of Neurochemistry. 2016; 139 Suppl 1:318–324.
- 65. Stuss DT, Gow CA, Hetherington CR. "No longer Gage": frontal lobe dysfunction and emotional changes. Journal of Consulting & Clinical Psychology. 1992; 60(3):349–59.
- 66. Wang H, Ren Y, Bai LJ, Zhang W, Tian J. Morphometry based on effective and accurate correspondences of localized patterns (MEACOLP). Plos One. 2012; 7(4).
- 67. Bay H, Tuytelaars T, Gool LV. SURF: Speeded Up Robust Features. Computer Vision & Image Understanding. 2006; 110(3):404–417.
- 68. Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on pattern analysis and machine intelligence. 2002; 24(7): 971–987.
- 69. Liu L, Fieguth P, Zhao G, Matti P, Hu D. Extended local binary patterns for face recognition. Information Sciences An International Journal. 2016; s 358–359(C):56–72.
- 70. Zhou L, Zhou Z, Hu D. Scene classification using multi-resolution low-level feature combination. Neurocomputing. 2013; 122:284–297.
- 71. Yuan L, Chen F, Zhou L, Hu D. Improve scene classification by using feature and kernel combination. Neurocomputing. 2013; 170:213–220.
- 72. Moradi M, Abolmaesoumi P, Mousavi P. Deformable registration using scale space keypoints. Medical Imaging Processing. 2006; 61442G-61442G-8.
- 73.
Scovanner P, Ali S, Shah M. A 3-dimensional sift descriptor and its application to action recognition. Proceedings of the 15th ACM international conference on Multimedia. ACM, 2007; 357–360.