Parameter Analysis on Backprojection for Super-Resolution (SR) Reconstruction Algorithm

In this paper, an image SR reconstruction scheme by using k-Singular Value Decomposition (k-SVD) with Orthogonal Matching Pursuit (OMP) as sparse coding method was proposed to obtain the High-Resolution (HR) image. The system conducted in this paper consists of two parts: image SR reconstruction algorithm and also the backprojection process. Since this paper is focused on analysing the effect of parameters in backprojection on the performance of final images produced at the end of the process, therefore, the flow of backprojection is discussed. Generally, the backprojection algorithm is added to the approach to improve the image quality by sharpening the edges of the HR image. However, the parameters used in backprojection could be the factor that caused decrease in the image quality. Thus, the main idea of this paper is to analyse these parameters throughout the flow of backprojection algorithm. The parameters include the predefined 2D filter and interpolation method. Based on the results, it can be concluded that the use of averaging filter (hsize = 3), Gaussian filter (hsize = 5, 7 and 9; σ = 1) and bicubic interpolation method or also known as the cubic kernel can be adopted to the backprojection algorithm since these parameters achieved the highest RMSE, PSNR and SSIM values of 13.87, 25.29dB and 0.83 respectively. The analyses done in this paper has brought a clear understanding on the backprojection algorithm and further implementation and analysis in backprojection such as by using cascaded filters for averaging and Gaussian filters can be considered in the future.


Introduction
In recent years, the implementation of Super-Resolution (SR) technology to produce a High-Resolution (HR) image from its input Low-Resolution (LR) image has gathered significant attention from the researchers in the field of computer vision and image processing [1]. Noted that the LR problems in imaging system might cause by several factors include: hardware limitation, motion blur, illuminations and also motion blur when capturing the images [2]. In this paper, a learning-based image SR reconstruction scheme which consists of training and testing phase is proposed to obtain the HR image. According to [3], the k-Singular Value Decomposition (k-SVD) is a useful algorithm in producing the dictionary in training phase. Theoretically, the update of dictionary atoms in k-SVD is linked together with an update of sparse representation coefficient using sparse coding algorithm such as the greedy algorithms [4]. In this case, the greedy algorithm which was frequently used by the researcher is known  [5][6][7][8]. Hence, the k-SVD algorithm with OMP as the sparse coding method is used in this paper to produce the final dictionaries in training phase.
A high-quality HR image is important in many applications nowadays in order to obtain an accurate result especially in medical imaging, video surveillance or biometric identifications. However, it is still a challenging task in obtaining the desired HR image due to physical limitation and also high cost of imaging applications. Although the image SR reconstruction scheme is able to obtain an HR image after the testing phase by inferring all the missing high frequency contents from the LR image, but the produced HR image was still having limitation especially on the edges of the images. So, a backprojection algorithm is required to improve the image quality by sharpen the edges of the HR image. Backprojection method is known as a useful process in imaging process especially in the tomographic field as described in [9][10][11].
In this paper, the backprojection method is added to the SR approach in order to improve the image quality by sharpen the edges of the image. Nevertheless, the parameters chosen in the process can be the factor that effect the performance of the HR image. Thus, the main idea of this paper is to analyse the parameters which will affect the performance of the HR images produced throughout the flow of backprojection algorithm. The evaluation of the HR image produced after the backprojection is done in terms of both qualitative and quantitative ways. The parameters which could produce the best results will be chosen to be incorporated in the backprojection algorithm and also will be taken into account for further implementation purpose for instance the use of cascaded filters in the future. Figure 1 shows the system for the works conducted in this paper. It consists of two parts: image SR reconstruction algorithm and also the backprojection algorithm. Since this paper focuses on analysing the effect of the parameters in backprojection based on the final images produced, hence, this paper will highlight the flow of backprojection algorithm. The first part of the work (Figure 1(a)) is known as the image SR reconstruction scheme and has two phases: training and testing. For training phase, the dictionaries were trained using k-SVD algorithm with OMP as sparse representation algorithm. Then, the dictionaries were used in the testing phase to reconstruct the LR image, L to produce the HR image, H. For the second part ( Figure 1(b)), the process of backprojection algorithm starts with creating a predefined two-dimensional (2D) filter, as shown by the highlighted part. This is because the predefined 2D filter, used is one of the parameters in backprojection algorithm that affect the performance of the output image. In this case, different type of predefined 2D filters, were tested in order to choose the suitable filters to be used. The details are discussed in Section 2.1. Then, the predefined 2D filter, was normalised using Equation (1).

Methodology
After the normalization step, the HR image, H obtained from the Figure 1(a) was down-sampled to have the same dimension as the LR image, L. In this step, the interpolation methods used will also affect the image quality of the final image produced. Therefore, different type of interpolation methods was substituted in this step for analyzation purposes. The details about the interpolation methods tested are discussed in Section 2.2. Next, image A was computed by calculating the difference of the corresponding pixel values between the down-sampled HR image, H with the LR image, L. The image A produced was then up-sampled again to have the same dimension as the original HR image, H.
After that, the convolution of image A, with the 2D filter was obtained followed by updating the HR image, H.

Predefined 2D filter
As mentioned before, the use of different predefined 2D filters is known as one of the parameters which affect the image quality of the HR image. In this case, there are many predefined 2D filters were available in the field of image processing. The filters which were tested in this paper include: averaging filter, circular averaging filter, Gaussian filter, Laplacian filter and also motion filter. Table 1 describes the details and input arguments that need to be considered when using these filters. In this paper, the 2D filters listed were tested with different input arguments in order to obtain the output image with best performance.   Table 1. Description of the predefined 2D filters tested [12].

Predefined 2D filters Description Input Arguments
Averaging filter The operation of averaging filter creates a filter of dimension, = ℎ × ℎ with pixels values of 1 ⁄ where ℎ is denoted as size of the filter used. ℎ

Circular averaging filter
The operation of circular averaging filter creates a diskshaped filter of dimension, = 2 × + 1 where is the radius of the filter.

Gaussian filter
The operation of Gaussian filter creates a rotationally symmetric Gaussian lowpass filter of dimension, = ℎ × ℎ with standard deviation, where ℎ is the size of the filter used.

Laplacian filter
The operation of this predefined 2D filter produces a filter with dimension of 3 × 3 by approximating the shape of the 2D Laplacian operator, ∇ 2 which is controlled by the value as described by Equation (2).
Motion filter The operation of motion filter creates a filter which approximate the linear motion of a camera once convolved with an image in which the values of ℓ and denoted the length and angle of the motion respectively. ℓ,

Interpolation methods
Interpolation is also an important parameter to be taken into account during the process of designing the backprojection operation. The interpolation methods tested in this paper include: bicubic interpolation or cubic kernel, bilinear interpolation or triangle kernel, nearest neighbour interpolation, Lanczos-2 kernel and Lanczos-3 kernel. Table 2 provides the description for each interpolation method analysed in this paper. Table 2. Description of the interpolation methods.

Interpolation methods Description Nearest neighbour
This method estimates the output pixel by simply assigned the pixel value that the nearest point that falls within it [12]. Bilinear/triangle kernel This method estimates the new pixel value by using the distance weighted average of the four nearest pixel values (2×2) [12]. Bicubic/cubic kernel This method estimates the new pixel value by considering the distance weighted average of sixteen nearest pixels values (4 × 4) [12].

Box kernel
This method interpolated the image by using a box-shaped kernel [12]. Lanczos-2/3 kernel Lanczos filter is basically used to interpolate the values of digital signals between its samples where the positive integers (2 or 3) indicate the size of the kernel [13].

Results and Discussion
The performance of the output image produced was evaluated by calculating the values of PSNR, SSIM and RMSE. The analysis on the effect of the parameters in backprojection include: predefined 2D filters and interpolation method as denoted in Figures 3 and 4 respectively.   The highlighted parts specify the overall highest results obtained by considering all the parameters in backprojection algorithm with different input arguments. The highest RMSE, PSNR and SSIM values obtained are 13.87, 25.29dB and 0.83 respectively (Figure 3 and 4). For the first parameter (predefined 2D filters), the averaging filter (ℎ = 3) and Gaussian filter (ℎ = 5, 7 and 9; = 1) obtain the highest values; while for the second parameter (interpolation methods), the highest value produces by bicubic interpolation method or also known as the cubic kernel. Therefore, these parameters can be substituted into the backprojection algorithm to obtain the desired HR image, H to produce a good quality image. It can also be observed from Figure 2 that the widowing size, ℎ is an important factor which affects the overall results of the final image produced. For instance, in the case for averaging filter, when the windowing size,ℎ is greater than 3, it is not able to obtain good results in terms of RMSE, PSNR and SSIM values. Hence, it can be concluded that the windowing size of a filter provides a degree of effect on removing noise in an image, small windowing size is able to remove noise and produce an output image with better performance. Figure 4 shows the maximum iterations, tested in the backprojection algorithm by considering both parameters: Gaussian filter (ℎ = 5, 7 and 9; = 1) and also bicubic interpolation method. Since the values of RMSE, PSNR and SSIM remain the same after 20 iterations, hence, maximum iteration is set as 20 for the backprojection algorithm.  Figure 5 shows the resulting image with = 20 as stated above using Baboon image [14] which includes the LR image used and also different images produced after the process of SR reconstruction scheme and also backprojection algorithm. In this case, the backprojection algorithm not only enhance image quality in terms of RMSE, PSNR and SSIM values, but also sharpen the edges as shown in Figure C