Image Enlargement Based on Proportional Salient Feature

—This paper proposes an image enlargement method that produces proportional salient content of image magnification. To obtain the proportional salient image content: first, we enlarge the source image to the high size of the target image using uniform enlarging. Second, we slice the image into sections from top to bottom following the minimum energy and detect the salient feature of the image. Third, we enlarge the slice of the image region that does not containthe salient feature of the image to the full size of the target image. The proposed method has been tested in several images, such as akiyo, butterfly, cameraman, canoe, dolphin, and parrot. The experimental results show that the proposed method results in a proportional content for image enlargement in the different ratios compared with the comparison method.


I. INTRODUCTION
urrently, the various sizes and ratios of displays have been presented.The ratios of display size rang from 3:2, 4:3, 5:4, 16:9, 16:10 and 17:9 respectively for the width and height.It requires a good image enlargement method.
Image enlargement is chang the smaller image size to a larger image size.In other words, image enlargement produces a high-resolution (HR) [1,2] image.We distinguish the enlargement of an image into two, namely uniform and nonuniform image enlargement.Nonuniform image enlargement produces image magnification in different ratios.There are two problems in image magnification; The first is having an enlarged image look rough, jagged edge area and blurry; the second is non-proportional content, especially in image magnification in different ratios.
Many methods [1][2][3][4][5] have been developed for image enlargement to obtain good image quality.The image enlargement method with edge improvement is also done to result in good image quality [3,4].Image enlargement based on regularity on the geometric is proposed by [3].This method uses covariance-based adaptive interpolation, especially in edges and pixels near the edge.They use bilinear interpolation for nonedge pixels.This method is called NEDI.However, NEDI method is only for uniform image enlargement and it is based on two scale factors.The reverse diffusion interpolation (RDI) [4] to recover sharp edges on HR image is capable of reducing highfrequency in the edge image.In this method, reverse diffusion serves as a nonlinear high-pass filter.Partial volume correction with reverse diffusion needs 200 iterations, which causes the RDI method require long computation time.The RDI method is only used to enlarge the image in the same ratios.Image enlargement methods based on the linear weighting technique to produce a better HR image are implemented on weighted sum of linear extrapolations (WLE) [5].The WLE method uses eight pixel samples to determine one pixel interpolation.The quality of the image enlargement result using the WLE method is slightly higher compared with classic methods such as bilinear interpolation and cubic interpolation.
The methods [1][2][3][4][5] are only for image enlargement in uniform size, and scale factor must be an integer value.They are difficult to implement in different ratios and shapes.We solve these problems by using image enlargement based on window kernel.This method is easily used to enlarge the image in different ratios and shapes.
One of the popular resizing methods is seam carving [6].Seam carving is a novel content-aware resizing scheme.This method uses gradient magnitude for measuring the image importance.However, visual distortions often occur when the content of image is complex backgrounds.To overcome this problem, many researchers, especially on image retargeting, start to focus on dealing with visual salience while suppressing nonsalient regions [6][7][8][9][10][11].These methods resize image to smaller size in different ratios.Nevertheless, resizing an image to a bigger size from the source image produces distortion between the left side and the right side of image carving.This is caused by the interpolation of a pixel in the carving of the image.
In this paper, to overcome the problem, we propose an image enlargement method by combining the scaling method based on the window kernel, the image slicing method, and feature detection, which produces the proportional content of image enlargement.

A. Image Enlargement Based on the Window Kernel
The image enlargement using window kernel is illustrated in Fig. 2.This method is used to enlarge the source image in the same ratio or in a different ratio.In this case, the method is used to the uniform and full-size images enlarged.The source image is the selected nonsalient feature, that will be enlarged to full-size image, as shown in Fig. 2.
The steps of image enlargement based on the window kernel are presented as follows: -The first step is finding the corresponding coordinates in the source image for the sample pixel kernel, as shown in 1 on Fig. 2. The corresponding coordinates (c h and c w ) are obtained by dividing the height (n 1 ) and width (n 2 ) coordinates of the target image with the scale factors (S h and S w ), as shown in Eq. (1).
-The second step is taking four pixels for the sample pixel window (ws) as shown in 2 on Fig. 2 and weight window is calculted based on Fig. 3.
Referring to Eq.( 1), there are three rules to fill the sample pixel window.
-If the value of c h is a fraction and the value of c w is an integer, then the element of the sample pixel is as in Eq. (3).
-If the value of c h is an integer and the c w is a fraction, then the element of the sample pixel is as in Eq. ( 4).
-If the value of c h is a fraction and the c w is a fraction, then the element of the sample pixel is as in Eq. ( 5).
Three steps determine the four pixels for window w s that we call by the window kernel.
2) Weight window.We use nonlinear interpolation, which is based on the curve for the image enlargement method as in Fig. 3.In the curve weighting method, the shape of curves will determine the enlargement method.If the curve is the linear shape, then it will generate a linear interpolation and vice versa.Both of coordinates x and y on Fig. 3. have two weight values.There are  x1 and  x2 for the x coordinate and  y1 and  y2 for the y coordinate.The curve weighting on Fig. 3. gives weight value greater than the linear weighting value for the pixel nearest the c h and c w points.Equation ( 6) is used to obtain  x1 value for the downhill curve, whereas Eq. ( 7) is used to obtain the  x2 value for the uphill curve.
 y1 and  y2 are obtained in the same way as  x1 and  x2 ; however, the variable c h in Eqs. ( 6) and ( 7) is replaced with c w .The size of the weight window is 2×2, which has same size with sample pixel window.The weight window is shown in Eq. ( 8).
The matrix element of w w is obtained by multiplying  xi with  yi as in Eq. (9).
wherein, w ij is element matrix of w w with i=j=1,2.
3) Filling pixel.A pixel to fill the corresponding coordinate in target image is obtained by multiplying the sample pixel window with weight window.

B. Image Slicer
Image slicer is used to slice the image into several sections that start at the same distance (L w ).L w is calculated using Eq.(10).
where, L w is the distance between each slice, w is the width of the image, and n s is the number of slices.Slicing the image starts from top to bottom following the minimum energy of the image.The energy of the image (E) is calculated using Eq.(11).
E is two images energy (E h and E v ), in which each point contains the horizontal and vertical absolute derivative approximations; the computations are as in Eq. ( 12) and Eq.( 13) where, f is the source image. is convolution.H h and H v are the absolute horizontal and vertical derivative mask with 3×3 size as in Eq.( 14).

C. Feature Detector
Feature detector is to detect salient features of the image.We use a simple formula to obtain the salient feature in the image as shown in Eq. (15).
where, f D is feature detection as shown in Fig. 4 (c), w d is 3×3 sample pixel window from image u f ˆ, D i are sample pixels that consist of several essential elements of the image sample, and u f ˆ is uniform image enlargement, as shown in Fig. 4 (b).
If D i has a part in common with the image u f ˆ, then it will be given a white mark (255) as shown in Fig. 4. (c).The white mark indicates the salient feature in the image.

D. Full size Image Enlargement
This stage is the final stage of the image enlargement process.Figure 4 (e) is obtained by combining the image output of image slicer and feature detector.The salient feature on Fig. 4. (e) is marked with white.If the part of the slice image contains a salient feature, as in (Fig. 4. (g)), then this section will not be enlarged.However, if the slice image does not contain a salient feature, it will be enlarged to the size of the target image.Figure 4 (f) and (h) show the selected slice region without white markings as in Fig. 4 (e).The full size image enlargement is formulated in Eq.( 16).where, y o is the full size image enlargement, R is the slice region of image, n is the number of slice region, S is the scale factor and  is up-scaling by using window kernel method in section A.

III. EXPERIMENTAL RESULT
In this experiment, the proposed method is tested for uniform and nonuniform image enlargement.For uniform image enlargement, we use four sample standard images, such as lena, peppers, baboon and the clown.On the other hand, for nonuniform image enlargement, we use six samples such as akiyo, butterfly, cameraman, canoe, dolphin and parrot images as shown in Fig. 5.All methods are implemented in Matlab 7 with computer specification 2.9 GHz CPU and 4 GB RAM.

A. Experimental Result
The proposed method in uniform image enlargement is evaluated using quantitative evaluation, while the nonuniform image enlargement is evaluated using qualitative evaluation.
1) Quantitative evaluation: The image enlargement quality is evaluated using objective quality evaluations such as peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) [12].We use the default value constant in the SSIM index formula K = [0.010.03] (where K is small constant with K1, in this paper, we use K value from experiment in [12]) and 11×11 Gaussian window size with =1.5.
We create a scenario to test the image enlargement result using the PSNR and MSSIM evaluation as follows: First, we resize the standard image into the smaller size.The sizes of images which would be enlarged were one-half of the standard image size.We use the nearest-neighborhood (NN) method to resize the standard image size.Second, we enlarge the small image size using all methods by the scale factor equal to two to become the standard image size or reference image size.
Figure 7 shows the image quality evaluation using PSNR by scale factor equal to two.The uniform image enlarged using the proposed method has a bigger PSNR value compared with the comparison method.In addition, the image evaluation using MSSIM in shows the proposed method closest to 1, which represents the proposed methods having close similarity to the image reference.
2) Qualitative evaluation: Figure 5 shows the target image size and source image that will be enlarged to the different ratio, whereas, Fig. 6. shows the image magnification using scaling techniques, carving and the proposed method.
Image enlargement using scaling and carving methods results in a disproportionate salient content compared with the proposed method.For example, Fig. 6 (a) and (b) have an Akiyo that looks fat compared with that in the proposed method.This is the same for the butterfly, cameraman and

IV. CONCLUSION
Proportional salient content image enlargement has been proposed in this paper.This method combines image slices, feature detection and window kernel scaling methods.The proposed method has been evaluated by using PSNR and MSSIM, which result in good image quality, and for images enlarged in the different ratio, the proportional salient content is obtained.
For next study, we have plans to implement our method to improve image sequence and feature detection and to obtain an autoselection nonsalient content in an image slice to be enlarged.

Fig. 1 .
Fig.1.Diagram block of proposed method image slicing and feature detection methods, as shown in Fig.1.There are four main blocks, namely uniform image enlargement, image slicer, feature detector and full size image enlargement.Uniform image enlargement is used to enlarge the source image uniformly to the high size of the target image, as shown in Fig.4.(b).Image slicer is used to slice an image into several parts.Feature detector is used to detect the important features in the image, whereas image enlargement is used to enlarge the slice image region with different scale factors based on the important features of the image.

Fig. 4 .
The step by step result of image enlargement.(a) Source image and target image size, (b) uniform image enlargement, (c) featutre detection, (d) energy of image, (e) image slicing of (c), (f) nonsalient feature slice region selection, (g) slice region selection with fiture content, (h) nonsalient feature slice region selection, (i) image enlargement result.

Fig. 5 .
Target image size and source images size.