Single Image Super-Resolution Algorithm based on Fixed-Point Multi-Model Gaussian Process Regression

To solve the problem of large amount of computation in matrix inversion of Gaussian process regression model, a super-resolution algorithm based on local Gaussian process regression of fixed point multi-model is proposed. Firstly, the training samples are classified by Gaussian mixture model, and image patches are randomly selected as fixed points in each type of training samples, and its K nearest neighbor patch are searched. Secondly, local Gaussian process regression model by using its low-mid-frequency components and the corresponding high frequency components. Again, low resolution test image patch is classified, only the K nearest neighbor test image patch is searched during reconstruction. And then, find the nearest fixed point in each kind of image patch, and use the local Gaussian process regression model based on the fixed point to predict its corresponding high-frequency component. Finally, corresponding high frequency information is predicted by using this trained model, which improves the reconstruction efficiency. Experimental results have demonstrated that the proposed algorithm is superior in both quantitative and qualitative aspects against other algorithms.


Introduction
Super-resolution (SR) reconstruction is a valuable research direction of image processing，which can enhance the image effect by software algorithm under existing hardware conditions. Image superresolution reconstruction can obtain clearer, higher quality image or complete higher-level tasks, which is widely used in public safety, aerospace, medical imaging and other fields. A variety of solving methods are proposed to solve this problem [1]. At present, the study focuses on the learningbased reconstruction algorithm, also known as image hallucination or example learning-based superresolution. By learning the training samples of high resolution (HR) and low resolution (LR) patches, a corresponding relationship is established, more high frequency detail is reconstructed and added to LR patch to improve the effect of image reconstruction.
Among the example learning-based reconstruction algorithms, there are two classical methods to realize super-resolution reconstruction, which named dictionary learning and regression learning. Yang et al. [2] utilized sparse coding model to jointly learn high and low resolution overcomplete dictionary pairs with the same sparse coefficients. Zeyle et al. [3] further reduced the dimension of training features according to principal component analysis, and carried out sparse coding through K-SVD and Orthogonal Matching, so that the efficiency of dictionary training is further improved. Nevertheless, these methods are faced with the bottleneck of calculation. Chang et al [4] presented a neighbor embedding (NE) algorithm, but the mapping function is not unique. Timofte et al. [5] put 2 forward a fast anchored neighborhood regression algorithm, however, the key problem of this method is that it requires artificial definition of dictionary size and neighborhood size, which will directly affect the quality and time of the reconstructed image.
Gaussian process regression method has a strict mathematical theoretical foundation, and it is a classical machine learning algorithm. Compared with other regression algorithms, Gaussian process regression has stronger adaptability in dealing with complex problems such as high dimension, small sample and nonlinear, so it has been widely concerned by researchers [6][7][8]. He et al. [9] first introduced Gaussian process regression (GPR) to the problem of SR reconstruction, and utilized similar image patches to establish a Gaussian regression model in the two stages of up-sampling and deblurring respectively. However, because the data information used in the reconstruction process was very limited, the reconstructed image had obvious artifacts and the blurred visualization effect, at the same time, its time complexity was very high. In view of the shortcomings of this method, a series of improved algorithms [10][11] are proposed, these algorithms adjust the number of samples in the training set according to the self-similarity of the image, but do not essentially deal with the problem of high time complexity in the algorithm. To solve the above problems, this work presents a SR algorithm based on fixed-point multi-model local Gaussian process regression (FPMMLGPR), which further improves the performance of Gaussian process regression reconstruction algorithm and improves the reconstructed image effect.

Super-resolution algorithm based on fixed-point multi-model Gaussian process regression
If local Gaussian process regression (LGPR) model can be trained offline in advance, the speed of image reconstruction will be greatly improved. From this point of view, this paper first classifies the training image patches by using the GMM, randomly selects multiple fixed points in each classification, and then trains the LGPR model of each fixed point offline. For image reconstruction, according to the category of test image patches, look for the nearest fixed point within each class. According to the LGPR model of the fixed point, the high frequency information of the corresponding HR pixel is predicted.

Image patch clustering based on Gaussian mixture Model
Gaussian mixture model is a soft clustering algorithm based on sample probability density distribution. For the LR and HR training sample sets based on the Gaussian process regression model constructed in the literature [12], they are marked as {} X and {y}. The LR training sample set {} X is clustered by GMM to obtain M classes, that is, M Gaussian distributions. The weighted sum of M Gaussian density functions is used to express the sample distribution as follows where d is the dimension of sampled image patches, k  and 2 k  represent the mean and covariance of the Gaussian distribution, respectively. According to the formula (2), each Gaussian distribution of GMM is described by the mean and covariance, therefore we must solve the model parameters. In the paper, we iteratively estimate the model parameters through EM algorithm, which includes two steps: M-step: Calculate the model parameters for the new iteration: Repeat steps E and M until convergence, and the likelihood distribution parameters of each class can be obtained.

Establishment of local Gaussian process regression model
Construct the training sample set of the GPR model, which can be realized by the following steps: Step 1: Each original HR image in the training set is down-sampled and reduced to 1/s of the original image by bicubic interpolation, then the LR image is up-sampled s times by bicubic interpolation to produce an enlarged image losing high-frequency details, it is regarded as the middle and low frequency component of the original HR image, or as a degraded HR image. The corresponding high-frequency component is get by subtracting the degraded HR image from the original HR image.
Step 2: Randomly extract image patches from each degraded HR image as the input of the training sample, and at the same time extract the pixels of the central position for each patch in the corresponding high-frequency image as the output of the training sample. Each set of image patches and pixels constructed in this way is a LR and HR training sample set based on GPR model. To make the data processed by regression have more statistical significance, each image patch extracted needs to take the de-mean operation. Each set of image patches and pixels constructed in this way is a LR and HR training sample set based on Gaussian process regression model. It should be noted that in order to make the data processed by regression more statistical, each image patch extracted needs to subtract the mean.
After constructing the training sample set for each LR test image, the local Gaussian process regression model is used to restore the corresponding HR image.
Step 1: The input LR image is magnified to the target magnification by bicubic interpolation to generate a degraded HR image, and each image patch in the image is extracted according to the order of raster scanning.
Step 2: For any degraded HR image patch, firstly, the K-D tree algorithm is used to find its K nearest neighbor image patches and their high-frequency components in the training set. The K-group nearest neighbor image patch sample set is taken as the training set, and its local Gaussian process regression is obtained.

Image super-resolution reconstruction
In order to more clearly describe the image super-resolution algorithm based on fixed-point multimodel local Gaussian process regression, the implementation process of the algorithm is described as follows.
Step 1: Sample and segment for LR test image. Magnify the LR test image to the target magnification to generate a degraded HR image, which is divided into t * kk size image patches 12 { , ,..., } t x x x according to the order of raster scanning.
Step 2: Classify the image patches. Adopt the Gaussian mixture model to classify each degraded HR image patch, and find the corresponding category for each image patch. Step 4: Predict the initial HR image. Rearranged all the generated high-frequency components to obtain an image with only high-frequency components, and then add the high-frequency detail information to the degraded HR image to generate a HR prediction image.
Step 5: Generate the target HR image. The iterative back-projection algorithm is used for postprocessing of the predicted HR image to further improve the quality of the target HR image. The implementation process is shown in Figure 1.

Experimental Results
The training data set used in the algorithm is the natural image training set in reference [2]. For testing, we adopt different types of images in two standard datasets Set 5 and Set 14 for super-resolution quality evaluation. The LR image is divided into patches with the size of 33, the LR test image is reconstructed by 3 times super-resolution, and the size of corresponding HR patch is 99. The LR image is up-sampled 3 times by Bi-cubic interpolation to produce an enlarged image losing highfrequency details, it is regarded as a LR image. Number of neighborhood patches in K nearest neighbor search is set to 60. In the super-resolution reconstruction based on fixed-point multi-model, the number of classifications M is equal to 12, the number of fixed point C is equal to 50.

Quantitative comparison results
In the paper, Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) are adopted as quantitative assessment to evaluate the performance of the comparison methods. In the experiment, the Bicubic interpolation is used as the benchmark algorithm, we compare our algorithm (FPMMLGPR) with Bicubic interpolation, NE+LLE [4], Yang's [2], Zeyde's [3], and GPR [9]. In the experiment, the PSNR and SSIM of these algorithms for 3 times super-resolution reconstruction are calculated, the comparison results are shown in Table 1. As can be seen from Tables, the PSNR and SSIM values using this algorithm are obviously better than those of other algorithms.

Qualitative comparison results
To further prove the superiority of this method, the visual effects of different super-resolution algorithms are compared in the experiment. Figure 2~Figure 3 respectively enumerate the visual effect comparison results of representative images Head and ppt3 for 3 times super-resolution reconstruction. The Head image in Figure 2 contains a lot of detail information. From the local reconstruction results of the bridge of nose region in Head images, we can see that the reconstructed images obtained by Bi-cubic algorithm, NE+LLE algorithm, Yang's algorithm and Zeyde's algorithm all have different blurring degrees. The visual effect of facial spots reconstructed by GPR method is improved compared with the above methods. The blurring phenomenon of the restored image by FPMMGPR algorithm is obviously improved, and the facial spots in Head images can be seen clearly. The ppt3 image in Figure  3 contains a lot of text character information. There is obvious ambiguity between the text characters reconstructed by Bicubic algorithm, while the improvement of the reconstructed characters obtained by NE+LLE algorithm, Yang's algorithm and Zeyde's algorithm is not obvious, but there is still ambiguity. There are obvious artifacts between the text characters reconstructed by GPR method, and the lack of detailed information is serious, even affecting the content of the image. Compared with other methods, the FPMMGPR super-resolution algorithm proposed by this paper basically eliminates the ambiguity between characters. Thus our work is suitable for the reconstruction of images with text information and shows better performance.

Conclusion
In this paper, considering that the mapping of image patches to pixels is nonlinear, a Gaussian regression model is introduced to establish the nonlinear relationship between them. we adopt GMM to cluster patches with the similar local structure, and build the mapping relationship through fixed point local Gaussian process regression model. Furthermore, our work can adaptively choice the optimal regression function for LR test patch. Experimental results have demonstrated that our proposed algorithm is superior in both quantitative and qualitative aspects against other competing methods.