Group Sparse Representation Based on Nonlocal Spatial and Local Spectral Similarity for Hyperspectral Imagery Classification

Spectral-spatial classification has been widely applied for remote sensing applications, especially for hyperspectral imagery. Traditional methods mainly focus on local spatial similarity and neglect nonlocal spatial similarity. Recently, nonlocal self-similarity (NLSS) has gradually gained support since it can be used to support spatial coherence tasks. However, these methods are biased towards the direct use of spatial information as a whole, while discriminative spectral information is not well exploited. In this paper, we propose a novel method to couple both nonlocal spatial and local spectral similarity together in a single framework. In particular, the proposed approach exploits nonlocal spatial similarities by searching non-overlapped patches, whereas spectral similarity is analyzed locally within the locally discovered patches. By fusion of nonlocal and local information, we then apply group sparse representation (GSR) for classification based on a group structured prior. Experimental results on three real hyperspectral data sets demonstrate the efficiency of the proposed approach, and the improvements are significant over the methods that consider either nonlocal or local similarity.


Introduction
Recent advances in remote sensing sensors, especially for hyperspectral imagery (HSI), increase the possibility of more accurate discrimination of materials of interest [1,2]. Given a set of observations (i.e., pixel vectors in a HSI), the purpose of classification is to assign a unique label to each pixel vector, such that it can be presented by a given class [3]. Although HSI is characterized by its high spectral resolution and abundant information, which promotes capturing fine details of spectral features for classification, it has been demonstrated that the original HSI contains high redundancy, and in addition there are high correlations in both the spectral and the spatial domains [4][5][6]. Therefore, the analysis problem is essentially low-rank and can be represented sparsely [7]. In this context, sparse representation (SR) has been widely exploited for HSI. SR-based classifiers (SRC) code a testing pixel over a dictionary which is constructed by sets of labeled samples [8,9]. The spectral features of pixels belonging to the same class are assumed to approximately lie in a lower-dimensional subspace and yield a relatively concentrated response under specific constraints towards the dictionary, such that the result is determined by the class with the minimum residual error of representation [10,11].

1.
Local spatial information is first considered in the proposed framework. Compared with the traditional SR-based methods, our approach represents the testing pixel with its neighborhood simultaneously, which allows for the consideration of local spatial consistency.

2.
Nonlocal spatial information is then emphasized by our proposed method. As an important check of local spatial consistency, NLSS is integrated with local similarity to provide a global spatial constraint under local and nonlocal spatial consistency.

3.
Local spectral information is further exploited through NLSS. Based on the exploration of local spectral similarity, our proposed NSLS-GSRC takes into account both the nonlocal spatial and the local spectral information, and contributes to a more comprehensive representation based on a group structured prior of GSRC. as NSLS-GSRC (nonlocal spatial and local spectral similarity based GSRC), can be summarized as follows: 1. Local spatial information is first considered in the proposed framework. Compared with the traditional SR-based methods, our approach represents the testing pixel with its neighborhood simultaneously, which allows for the consideration of local spatial consistency. 2. Nonlocal spatial information is then emphasized by our proposed method. As an important check of local spatial consistency, NLSS is integrated with local similarity to provide a global spatial constraint under local and nonlocal spatial consistency. 3. Local spectral information is further exploited through NLSS. Based on the exploration of local spectral similarity, our proposed NSLS-GSRC takes into account both the nonlocal spatial and the local spectral information, and contributes to a more comprehensive representation based on a group structured prior of GSRC. The remainder of this paper is organized as follows: Section 2 formulates the representation models adopted in this paper. Section 3 details the proposed NSLS-GSRC approach. Section 4 evaluates the performances of our method compared with those of other related hyperspectral image classifiers, using data sets collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over the Indian Pines site in northwestern Indiana (USA), Salinas Valley in California USA), and the Reflective Optics Spectrographic Imaging System (ROSIS) over the University of Pavia in Italy. Section 5 presents some concluding remarks.

Background
Compared with the traditional processing techniques for HSI such as band fitting and support vector machine (SVM) [38][39][40], representation-based methods have gained great interest due to their no assumption of data density distribution. By representing testing pixels as a linear combination of a small subset of labeled pixels, they have been proven to provide rather plausible results. One of the classic models is SRC, which has been widely exploited for HSI. Later, joint SRC (JSRC) was developed to introduce local spatial correlation to the SRC framework. However, obstacles remain in the development of this method, such as the inherent instability of sparse coefficients [41]. Therefore, GSRC is designed to reconstruct the dictionary based on group structured prior, such that the representation can be processed in a more comprehensive model [28].

Sparse Representation-Based Classifier (SRC)
Given a hyperspectral image denoted as X, which contains a total of B spectral bands, K labeled classes and N pixels, where N = r × c, and r and c are the length of row and column of X. Let xi,j define Figure 1. General framework. (X: a HSI; P: a patch extracted from X; S: the window size of the extraction; X i,j ,X u,v : the neighborhoods centered on pixel x i,j and x u,v ; X f : the fusion matrix of X i,j and X u,v ; D: the dictionary of representation; A: the coefficient matrix.).
The remainder of this paper is organized as follows: Section 2 formulates the representation models adopted in this paper. Section 3 details the proposed NSLS-GSRC approach. Section 4 evaluates the performances of our method compared with those of other related hyperspectral image classifiers, using data sets collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over the Indian Pines site in northwestern Indiana (USA), Salinas Valley in California USA), and the Reflective Optics Spectrographic Imaging System (ROSIS) over the University of Pavia in Italy. Section 5 presents some concluding remarks.

Background
Compared with the traditional processing techniques for HSI such as band fitting and support vector machine (SVM) [38][39][40], representation-based methods have gained great interest due to their no assumption of data density distribution. By representing testing pixels as a linear combination of a small subset of labeled pixels, they have been proven to provide rather plausible results. One of the classic models is SRC, which has been widely exploited for HSI. Later, joint SRC (JSRC) was developed to introduce local spatial correlation to the SRC framework. However, obstacles remain in the development of this method, such as the inherent instability of sparse coefficients [41]. Therefore, GSRC is designed to reconstruct the dictionary based on group structured prior, such that the representation can be processed in a more comprehensive model [28].

Sparse Representation-Based Classifier (SRC)
Given a hyperspectral image denoted as X, which contains a total of B spectral bands, K labeled classes and N pixels, where N = r × c, and r and c are the length of row and column of X. Let x i,j define a testing pixel in X with the location (i,j). Traditional SRC represents the testing pixel x i,j by a sparse linear combination of labeled samples under a l 1 -norm constraints of coefficients as follows: where D is defined as a dictionary composed of random selected labeled samples from each class, α is a weight vector corresponding to D towards the x i,j during the representation, and λ is a regularization parameter. g 1 and g 2 denote the l 1 -norm and l 2 -norm constraints. The class label is determined by the minimum residual error between x i,j and its approximation as follows: where k ∈ {1, . . . ,K} is the class index, and δ k (g) is an indicator operation which can zero out all the elements in α that does not belong to class k.

Joint SRC (JRSC)
SRC focuses on the spectral characteristics, and it largely neglects the spatial correlation of contextual information in a remote sensing scene. In HSI, pixels in a neighborhood usually consist of similar materials. This spatial prior can be incorporated with a representation process, so that the local spatial coherence can be exploited to avoid some misclassified errors caused by the phenomenon of the same material with different spectra. Therefore, the main principle of JSRC is to represent pixels in close proximity with a common sparsity support. Considering X i,j is a B × (S) 2 sized matrix which is transformed by a S × S sized neighborhood center on x i,j from the original scene, the objective function of the JSRC is represented as follows: where A is a coefficient matrix composed of the weight vectors corresponding to pixels in X i,j , and · F is the Frobenius norm. Note that A 2,1 = n ∑ i=1 a i is a l 2,1 -norm constraint, n is the number of labeled samples selected in D, and a i is defined as the i th row of A, which is different from α (α is the column vector of A). The class label of the centered pixel x i,j is determined by the minimum residual error with the following function: where δ k (·) is set to zero out all the elements in A that does not belong to class k.

Group SRC (GRSC)
In JSRC and SRC, the testing pixels are represented by individual atoms of a dictionary. The dictionary of representation-based classifiers has an inherent group structured property, which means that the atoms from the same class can be grouped together and the pixels can be represented by groups of atoms. By encouraging coefficients of only certain groups to be active and the remaining groups inactive, the representation can obtain a more centralized optimization response. Therefore, GSRC reconstructs the dictionary as D = (D 1 , . . . , D K ) with each column vector of D k representing a labeled sample randomly selected from class k. Group Lasso optimization can be exploited to sum up the l 2 -norm of group coefficients based on a sparsity prior [42], such that the optimization function of GSRC can be represented as follows: where g ⊂ {G 1 , . . . , G K } defines the K groups and A g represents the coefficient matrix of each group corresponding to the subdictionary D k in D, and ω g is a regularization parameter which is adopted to compensate for different sizes of groups. ∑ g∈G A g 2 can be regarded as a group-level l 2,1 -norm constraint, and the GSRC can degenerate to a JSRC if the group size reduces to one.

Proposed Approach
As introduced in Section 2.3, GSRC exploits the prior structured group of a dictionary, and integrates the spatial information from the local area for better optimization. In order to satisfy the homogeneity assumption and the local spatial consistency during the representation, GSRC usually considers a small neighborhood around the testing pixel. Assuming that this small neighborhood contains some spectral abnormal pixels (caused by the presence of same material with different spectra), GSRC probably misclassifies the testing pixel. However, in this project, when we consider a larger patch which contains the current testing neighborhood, its local spectral similar area in a nonlocal spatial similar patch can be found in the original scene, such that this new area can strengthen the constraints towards the problem testing neighborhood during the representation, and obtain an improved response. Therefore, it is reasonable to exploit the nonlocal spatial and local spectral similarities of image for adequate supplement to the discrimination of materials of interest in spectral-spatial frameworks.
As illustrated in Figure 1, let P i,j represent a S 1 × S 1 sized patch with the center pixel of x i,j , the proposed NSLS-GSRC firstly searches the most spatially similar non-overlapped patch from P a,b towards P i,j . Then, the most similar spectral pixel from x u,v towards x i,j is located in the found nonlocal patch P a,b . After that, two S 2 × S 2 sized neighborhoods X i,j and X u,v centered on x i,i and x u,v can be extracted from the original scene X. With a fusion process, the new testing matrix can now be sparsely represented by GSRC. The final class label of x i,j is determined by the minimum representation error with group structured dictionary D and coefficient matrix A. The details of this process are presented in the following subsections.

Nonlocal Spatial and Local Spectral Similarity (NSLS)
In order to implement the search of a nonlocal patch towards each pixel in the whole image, X needs to be expanded to a (r + S 1 ) × (c + S 1 ) sized image. Then, we start to extract S 1 × S 1 sized patches from the upper left corner of the expanded image with step of one pixel, such that patches centered on each pixel from the original scene can be obtained. For example, P denotes the set of all the patches, where P i,j ∈ P represents the patch centered on x i,j . The first step of determination of our proposed NSLS is to find the most similar nonlocal spatial patch P a,b ∈ P towards P i,j . This process adopts two principles: 1. P a,b should be the most similar patch-based one relative to P i,j in the search area. 2. P a,b and P i,j cannot overlap, i.e., either the distance between abscissa or ordinate of their centered pixels should be larger than the side length of the extracted patch. Therefore, the determination of P a,b can be represented as follows: where x m and x n are the m th and n th pixel vector in P i,j and P a,b , and d(x m , x n ) calculates the Euclidian distance between them. (a, b) is the coordinate of centered pixel x a,b of P a,b , and the constraint condition is corresponding to the second principle listed above. After P a,b is obtained, the most similar spectral pixel x u,v compared with x i,j can be located in P a,b as follows: where x b i,j and x b u,v are the values of x i,j and x u,v in b th band, and d(x b i,j , x b u,v ) calculates their Euclidian distance. Since the measurement between P i,j and P a,b is on a patch-based level, x u,v obtained here might not be located at the center of P a,b (as illustrated in Figure 1), i.e., x u,v and x a,b are not the same pixel. Therefore, compared with other related NLSS-based methods, the innovation of the proposed NSLS-GSRC approach is the further exploration of local spectral similarity through nonlocal spatial similarity [32,33].

NSLS-GSRC
Following the formulation described in previous sections, x u,v is found and can be considered the most similar local spectral pixel in nonlocal spatial similar patch P a,b towards the current testing pixel x i,j in its neighboring patch P i,j . Then, a S 2 × S 2 sized neighborhood centered on x u,v can be obtained and transformed to a B × (S 2 ) 2 sized matrix denoted as X u,v . After that, X u,v is fused with X i,j by a pixel-by-pixel average process. The fusion result can be then processed in the representation-based framework by Group Lasso optimization. Finally, the objective function of the proposed NSLS-GSRC method can be represented as follows: where class label of testing pixel x i,j is determined by the minimum total residual error as follows: where δ k (A) represents the operation to zero our all the elements in A that do not belong to class k. The pseudo code for the proposed NSLS-GSRC method is shown in Algorithm 1.

Algorithm 1. The proposed NSLS-GSRC method
Input: A HSI image X, dictionary D is constructed by class orders and randomly selected from the labeled samples, a testing pixel x i,j and the patch-size parameters S 1 and S 2 Step 1: Extract a S 1 × S 1 sized patch denoted as P i,j with centered pixel of x i,j from X; Step 2: Search the nonlocal spatial similar patch P a,b towards P i,j according to Equation (6); Step 3: Search the local spectral similar pixel x u,v in P a,b towards the current testing pixel x i,j according to Equation (7); Step 4: Extract two S 2 × S 2 sized patches with the center pixels of x i,j and x u,v , and transform them in to two-dimensional formed matrix denoted as X i,j and X u,v ; Step 5: Obtain the fusion matrix X f of X i,j and X u,v , and use them using GSRC to obtain the coefficient matrix A according to Equation (8); Step 6: Compute the minimun total residual error and identify the class label of the testing pxiel x i,j according to Equation (9); Output: class (x i,j ).

Experimental Results
In this section, the proposed NSLS-GSRC method is evaluated using three widely used hyperspectral data sets. The first one is the Indian Pines scene collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) (NASA Jet Propulsion Laboratory, Los Angeles, CA, USA), with spectral coverage ranging from 0.25 to 2.4 µm and geometric resolution of 20 m per pixel. The scene contains 145 × 145 pixels, with 220 spectral bands. The ground reference contains sixteen classes, which are associated with different kinds of crops. The second scene is the Salinas scene recorded by the AVIRIS sensor. The spatial resolution of this image is 3.7 m per pixel. The scene contains 512 × 217 pixels, with 224 spectral bands ranging from 0.9 to 1.4 µm. The ground reference contains sixteen ground-truth classes. The last scene is University of Pavia scene captured by the Reflective Optics Spectrographic Imaging System (ROSIS) (DLR Institute of Optoelectronics, Berlin, Germany), with spectral coverage from 0.43 to 0.86 µm and geometric resolution of 1.3 m per pixel. The scene consists of 610 × 340 pixels, with 103 spectral bands and nine ground-truth classes. For comparative purposes, several competing spectral and spectral-spatial classifiers are considered in experiments, such as SVM [43], SVM based Markov Random Field (SVM-MRF) [44], SRC [16], CRC [17], JSRC [20], GSRC [29]. In addition, NL-means based SRC (NL-SRC) is also carried out for comparison similar to the literature [35] to evaluate the proposed NSLS-GSRC method. We conduct 20 Monte Carlo runs while varying the randomly selected labeled samples, and report the corresponding results.

Parameter Settings
In the experiments of this paper, the regularization parameter λ for the representation-based methods ranges from 10 −3 to 10 −1 , ω g for the group sparse representation-based methods is set to 1 due to the same number of labeled samples randomly selected per class in experiments. The parameters of the other methods have been optimized by means of a fivefold cross-validation according to the procedure provided in the literature [16,17,20,29,43,44]. In particular, the values of S 1 and S 2 are tested extensively and illustrated in Figure 2.

Experimental Results
In this section, the proposed NSLS-GSRC method is evaluated using three widely used hyperspectral data sets. The first one is the Indian Pines scene collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) (NASA Jet Propulsion Laboratory, Los Angeles, CA, USA), with spectral coverage ranging from 0.25 to 2.4 μm and geometric resolution of 20 m per pixel. The scene contains × 145 145 pixels, with 220 spectral bands. The ground reference contains sixteen classes, which are associated with different kinds of crops. The second scene is the Salinas scene recorded by the AVIRIS sensor. The spatial resolution of this image is 3.7 m per pixel. The scene contains × 512 217 pixels, with 224 spectral bands ranging from 0.9 to 1.4 μm. The ground reference contains sixteen ground-truth classes. The last scene is University of Pavia scene captured by the Reflective Optics Spectrographic Imaging System (ROSIS) (DLR Institute of Optoelectronics, Berlin, Germany), with spectral coverage from 0.43 to 0.86 μm and geometric resolution of 1.3 m per pixel. The scene consists of × 610 340 pixels, with 103 spectral bands and nine ground-truth classes. For comparative purposes, several competing spectral and spectral-spatial classifiers are considered in experiments, such as SVM [43], SVM based Markov Random Field (SVM-MRF) [44], SRC [16], CRC [17], JSRC [20], GSRC [29]. In addition, NL-means based SRC (NL-SRC) is also carried out for comparison similar to the literature [35] to evaluate the proposed NSLS-GSRC method. We conduct 20 Monte Carlo runs while varying the randomly selected labeled samples, and report the corresponding results.

Parameter Settings
In the experiments of this paper, the regularization parameter λ for the representation-based methods ranges from 10 −3 to 10 −1 , ωg for the group sparse representation-based methods is set to 1 due to the same number of labeled samples randomly selected per class in experiments. The parameters of the other methods have been optimized by means of a fivefold cross-validation according to the procedure provided in the literature [16,17,20,29,43,44]. In particular, the values of S1 and S2 are tested extensively and illustrated in Figure 2.  As shown in Figure 2, with a fixed number of labeled samples, the overall classification accuracy is relatively stable with respect to S 1 , but obviously decreases with respect to S 2 . This is because S 2 defines the size of neighborhood in group sparse representation, and it needs to be constrained in a relatively small range such that to satisfy the homogeneity assumption and the local spatial consistency. Instead, S 1 decides on the searching patch-size which determines the nonlocal spatial and local spectral similarity, and it is allowed to range in a relatively larger range compared with S 2 due to the global structure priors. According to the results of Figure 2, we adopt S 1 = 7 and S 2 = 3 for the Indian Pines scene, as well as S 1 = 11 and S 2 = 7 for the Salinas scene. Finally, for the University of Pavia scene, S 1 and S 2 are chosen to be 5 and 3.

Experiments with the AVIRIS Indian Pines Scene
In the experiment with the Indian Pines scene, eight mutually exclusive classes with a total of 8624 labeled samples are adopted from the reference data to avoid some classes with very small training samples, and also to satisfy the sparsity requirement in the process of representation. Figure 3 shows the false-color composite of the image and the reference map. Our first test randomly selects 50 labeled samples per class with a total of 400 samples (which represents approximately 4.6% of the labeled samples) for training and dictionary construction, where the remaining samples are used for validation. Table 1 shows the overall and individual classification results of different testing methods. Figure 4 shows the classification maps obtained by the different testing methods. Several conclusions can be drawn.

1.
Compared with SVM, both SRC and CRC provide considerable individual classification accuracies and a slightly better overall classification result. Firstly, it indicates that the representation-based methods can indeed provide plausible results towards traditional models for HSI classification. Also, it is a basis support for the framework of sparse representation exploited in our proposed method.

2.
Compared with SVM and SRC, SVM-MRF and JSRC achieve higher overall classification accuracies and provide more homogeneous classification maps, demonstrating that the incorporation of local spatial information can bring improvement to the classifiers in the spectral domain. The improvement supports the homogeneity assumption and the local spatial consistency in the spectral-spatial framework for the classification of HSI.

3.
Compared with SVM-MRF and JSRC, GSRC achieves better classification results which proves that group structured priors contribute to a more comprehensive integration of spectral and local spatial information. The improvements of GSRC over JSRC indicate the superiority of group sparsity framework and the Group Lasso optimization.

4.
Compared with SRC, NL-SRC brings improvement in classification result that is similar to GSRC. On one hand, it proves that the incorporation of nonlocal spatial information is effective for the classifiers in the spectral domain. On the other hand, it also indicates that both the nonlocal spatial similarity and the local spatial consistency improve the sparse representation framework for the classification of HSI. 5.
The proposed NSLS-GSRC outperforms GSRC and NL-SRC, which firstly demonstrates that the integration of both nonlocal and local spatial information contributes to a more comprehensive consideration of structured priors compared with either of them. In addition, it also indicates that the exploration of local spectral similarity through nonlocal spatial similarity provides more effective means for the discrimination of materials in spectral-spatial frameworks. Furthermore, it proves that the combination of global structured priors and group structured priors, i.e., NSLS and GSRC in our proposed method, can bring significant improvement for the classification of HSI.
In general for this case, the proposed NSLS-GSRC obtains an overall accuracy of 90.54%, which is 5.5% and 3.7% higher than NL-SRC and GSRC, and also 12.36% higher than SRC, respectively. For individual class accuracy, it also provides considerable results, especially for classes 1 and 5. The classification maps in Figure 4 confirm the improvement achieved by the proposed method. individual class accuracy, it also provides considerable results, especially for classes 1 and 5. The classification maps in Figure 4 confirm the improvement achieved by the proposed method.
(a) (b)    individual class accuracy, it also provides considerable results, especially for classes 1 and 5. The classification maps in Figure 4 confirm the improvement achieved by the proposed method.
(a) (b)     In the second test with the Indian Pines scene, the proposed NSLS-GSRC is compared with other test methods using a different number of labeled samples (from 10 to 50 samples per class). Figure 5 and Table 2 show the overall classification accuracies obtained by the different testing methods, as a function of the number of labeled samples used. As shown by Figure 5 and Table 2, several conclusions can be drawn:

1.
Overall classification accuracies are generally positively correlated with the number of labeled samples selected for training and dictionary construction. The improvement is relatively obvious with the situation where the method has a limited number of labeled samples, which can be seen from the variation trend of overall classification accuracies in Figure 5.

2.
The integration of spatial information contributes to more accurate discrimination of materials.
In particular, both local spatial information and nonlocal spatial information both contribute to better characterizing the image in the spectral-spatial domain, which can be seen from the improvement of overall classification accuracies of SVM-MRF, JSRC, GSRC and NL-SRC when compared with the original counterparts, i.e., SVM and SRC. 3.
The combination of local and nonlocal spatial information contributes to more comprehensive consideration of global structured priors. This is especially true for the proposed method, because with a further exploration of local spectral similarity through nonlocal spatial similarity, NSLS-GSRC brings reliable and stable improvement of classification in comparison with other methods either using local or nonlocal spatial information only. In the second test with the Indian Pines scene, the proposed NSLS-GSRC is compared with other test methods using a different number of labeled samples (from 10 to 50 samples per class). Figure 5 and Table 2 show the overall classification accuracies obtained by the different testing methods, as a function of the number of labeled samples used. As shown by Figure 5 and Table 2, several conclusions can be drawn: 1. Overall classification accuracies are generally positively correlated with the number of labeled samples selected for training and dictionary construction. The improvement is relatively obvious with the situation where the method has a limited number of labeled samples, which can be seen from the variation trend of overall classification accuracies in Figure 5. 2. The integration of spatial information contributes to more accurate discrimination of materials.
In particular, both local spatial information and nonlocal spatial information both contribute to better characterizing the image in the spectral-spatial domain, which can be seen from the improvement of overall classification accuracies of SVM-MRF, JSRC, GSRC and NL-SRC when compared with the original counterparts, i.e., SVM and SRC. 3. The combination of local and nonlocal spatial information contributes to more comprehensive consideration of global structured priors. This is especially true for the proposed method, because with a further exploration of local spectral similarity through nonlocal spatial similarity, NSLS-GSRC brings reliable and stable improvement of classification in comparison with other methods either using local or nonlocal spatial information only.

Experiments with the AVIRIS Salinas Scene
The Salinas scene used in our second experiment was recorded by the AVIRIS sensor over the Salinas Valley, California. Figure 6 shows the false-color composite of the image and the reference

Experiments with the AVIRIS Salinas Scene
The Salinas scene used in our second experiment was recorded by the AVIRIS sensor over the Salinas Valley, California. Figure 6 shows the false-color composite of the image and the reference map which contains a total of 54,129 labeled samples. We first randomly select 20 labeled samples per class with a total of 320 samples (which represents approximately 0.6% of the labeled samples) for training, where the remaining samples are used for testing. The classification results and maps obtained by different comparison methods are provided in Table 3 and Figure 7.  map which contains a total of 54,129 labeled samples. We first randomly select 20 labeled samples per class with a total of 320 samples (which represents approximately 0.6% of the labeled samples) for training, where the remaining samples are used for testing. The classification results and maps obtained by different comparison methods are provided in Table 3 and Figure 7.   As shown in Figure 7 and Table 3, SRC and CRC provide comparable results with SVM in the spectral domain. In the spatial domain, GSRC, JSRC and SVM-MRF bring significant improvement relative to SRC and SVM by integrating local spatial consistency. On the other hand, NL-SRC also achieves an obvious increase of classification accuracy relative to SRC with consideration of nonlocal spatial self-similarity. Last but not least, the proposed NSLS-GSRC obtains the best classification result with an overall accuracy of 91.06%, which is 2.44% and 2.95% higher than NL-SRC and GSRC, also 9.57% higher than SRC, respectively. In addition, it brings considerable improvements for individual class accuracy, especially for class 14 and 15, which can be observed from the classification map illustrated in Figure 7.
Our second test of the Salinas scene evaluates the proposed NSLS-GSRC method with a varying size of labeled samples (from 10 to 50 samples per class). Figure 8 and Table 4 show the overall classification accuracies obtained by different testing methods, as a function of the number of labeled samples adopted for training and dictionary construction. As shown in Table 4 and Figure 8, the improvement of JSRC and GSRC relative to SRC proves the effectiveness of integration of local spatial information with consideration of spatial coherence. The effectiveness of nonlocal spatial information is demonstrated by the increase of overall accuracies acquired by NL-SRC in comparison with SRC. The superiority of local spectral similarity through NSLS is confirmed by the best overall classification accuracies obtained by the proposed NSLS-GSRC method in all cases, which allows for an overall consideration of local and nonlocal spatial information.

Experiments with the ROSIS University of Pavia Scene
In the experiment with the ROISIS University of Pavia scene, a × 180 180 pixel-size patch with a total of 7398 labeled samples from nine classes is extracted from the original scene with consideration of time efficiency. Figure 9 shows the false-color composite image of the extracted region and the corresponding reference map.
Our first test randomly selected 30 labeled samples per class with a total of 270 samples for training and dictionary construction (which represents approximately 3.6% of the labeled samples), while the remaining samples are used for validation. Table 5 reports the overall and class-specific accuracies of different testing methods, where Figure 10 shows the corresponding classification maps for this case.

Experiments with the ROSIS University of Pavia Scene
In the experiment with the ROISIS University of Pavia scene, a 180 × 180 pixel-size patch with a total of 7398 labeled samples from nine classes is extracted from the original scene with consideration of time efficiency. Figure 9 shows the false-color composite image of the extracted region and the corresponding reference map.
Our first test randomly selected 30 labeled samples per class with a total of 270 samples for training and dictionary construction (which represents approximately 3.6% of the labeled samples), while the remaining samples are used for validation. Table 5 reports the overall and class-specific accuracies of different testing methods, where Figure 10 shows the corresponding classification maps for this case.      As shown in Figure 10 and Table 5, SVM provides considerable results in similar with SRC and CRC in the spectral domain. The methods based on local spatial consistency and structured priors generally bring improvement to their original counterparts, for instance, GSRC outperforms SRC. The methods based on nonlocal spatial similarity offer support for the notion of better consideration of global structure priors, for instance, NL-SRC outperforms SRC. Furthermore, the proposed method brings better characterization of spatial and spectral information based on nonlocal spatial and local spectral similarity, and achieves a more comprehensive discrimination of materials. In general for this case, the proposed NSLS-GSRC obtains an overall accuracy of 93.27%, which is 2.96% and 2.56% higher than NL-SRC and GSRC, also 6.69% higher than SRC, respectively. In addition, the proposed NSLS-GSRC provides reliable individual classification accuracy for each class, especially for classes 2, 7 and 8. It can also be seen from the more homogenous details of the classification map obtained by NSLS-GSRC in Figure 10, which confirms the improvement.
Our second test of the University of Pavia scene evaluates the proposed NSLS-GSRC with a varying size of labeled samples (from 10 to 50 samples per class). Figure 11 and Table 6 show the overall classification accuracies obtained by different methods tested, as a function of the number of labeled samples used for training and dictionary construction. Though CRC obtains relatively poor results in this case, SRC provides competitive overall classification accuracies towards SVM with the increase of the number of training samples in the spectral domain. In the spatial domain, stable increases are obtained with consideration of local spatial consistency, which can be concluded from the comparisons and their trend of SVM-MRF with SVM, and JSRC/GSRC with SRC. On the other hand, improvement are also brought by the combination of nonlocal spatial self-similarity and spectral-domain SR, which is indicated by the performance of NL-SRC versus GSRC, especially for the situation of the limited training samples. In the spectral-spatial domain for all cases, the proposed NSLS-GSRC method achieves the best results in comparison with the other related methods, which demonstrates that the integration of nonlocal spatial and local spectral information achieves more comprehensive discrimination of materials.
hand, improvement are also brought by the combination of nonlocal spatial self-similarity and spectral-domain SR, which is indicated by the performance of NL-SRC versus GSRC, especially for the situation of the limited training samples. In the spectral-spatial domain for all cases, the proposed NSLS-GSRC method achieves the best results in comparison with the other related methods, which demonstrates that the integration of nonlocal spatial and local spectral information achieves more comprehensive discrimination of materials.    Synthesizing the results and analysis in above three experiments, the proposed NSLS-GSRC method obtains comprehensive and considerable overall and individual classification accuracies, and provides more homogenous details in classification maps compared with other related methods. The classification results with varying number of training samples further verifiy the stability of the proposed method. In general, we would like to emphasize that the proposed NSLS-GSRC based on the integration of nonlocal spatial and local spectral similarity is reliable and stable for the classification of HSI.

Conclusions
Hyperspectral images are characterized by their abundant spectral and spatial information. Considering the high redundancy and correlation among spectral bands, it has been demonstrated that the inherent sparse property can be exploited for more accurately discriminating materials under examination. In this context, SR-based methods have shown their effectiveness by representing pixel with a linear combination of labeled samples, and obtained a certain success in spectral domain. In order to better characterize the image for classification, researches have been focused on two major aspects to incorporate the spatial information of image. In the local spatial domain, JSRC and GSRC have been designed to simultaneously represent the pixel in neighborhood based on the local spatial coherence. In the nonlocal spatial domain, NLSS is presented to measure the similarity of pixels based on the nonlocal structured priors. With a further investigation of more comprehensive classification, a new framework should allow the exploration of spectral similarity through NLSS and combination of nonlocal and local spatial information in the spectral-spatial domain.
In this paper, we proposed a new classification framework to exploit nonlocal spatial and local spectral similarity based on group sparse representation for hyperspectral image. The main contribution of the proposed method, abbreviated as NSLS-GSRC, includes the further exploitation of spectral similarity through nonlocal spatial self-similarity, and its incorporation with group structure-based sparse representation based on local spatial consistency. Experiments based on three real hyperspectral data sets demonstrate that the proposed NSLS-GSRC outperforms other related methods for the classification performance.