Successive Image Interpolation Using Lifting Scheme Approach

: Problem statement: Fast and accurate interpolation and resizing of images and video frames is a much sought after research area in multimedia applications. The more accurate schemes are computationally expensive and require more time for execution. We proposed here Discrete Wavelet Transform (DWT) using lifting scheme as an accurate and computationally inexpensive interpolation technique for image resizing. Approach: In this study, the lifting scheme DWT algorithm was applied for interpolation of the images to the scale 2 -n for reduction in size where n indicated the level of DWT. To magnify the image to the scale 2 n , IDWT was used after the zeroeth level DWT, while DWT was used for subsequent reconstruction. Results: In case of reduction in size, the DWT components were calculated to a level so that the four DWT components were reduced to single pixel size and the reconstruction is again carried out to the original size. The reconstruction results were bench marked using Mean Squared Error (MSE) and Peak Signal to Noise Ratio (PSNR) with other schemes like Bilinear and Bicubic interpolations. The two component Harr mother wavelet was found to be suitable for the fast computations of the interpolated images and their subsequent versions. Conclusion: It has been found that after higher level of DWT computations like 10 or 11th level, the MSE increases beyond acceptable limits and the PSNR drops below 20 dB. But the interpolation or reconstruction is completed in much less time with better MSE and PSNR as compared to the bilinear and bicubic scheme. The present technique provided fast interpolation algorithm for multimedia and video processing applications.


INTRODUCTION
Wavelets provide a very good and accurate enough tool for approximation of the signals, functions and data sets.Mother wavelet acts as basic building block for approximation of complex signals and functions at reasonable computational overhead.In other words, wavelets can approximate complex signals using only a small set of coefficients without much computational complexity.This is of course possible because, most data sets have correlation both in time (or space) and frequency domain.Rather amongst the family of xxxlet (wavelet, curvelet, ridgelet) transforms, wavelet is the first one to preserve the correlation in spatial and frequency domain both.Transformations in most of the frequency domain transforms do not preserve the original form of a signal like image in spatial domain.One cannot make out the visible form of an image from the results of sine, cosine or Discrete Fourier Transforms.However, the LPLP component of DWT of an image is visibly similar to the original image and other components are the edges in different directions.Thus the DWT may serve as an excellent tool for simulating human vision phenomenon like continuous interpolation, because of the time-frequency localization property of wavelets.
Lifting scheme was first introduced by Sweldens (1996a;1996b;1996c).The lifting scheme is a new method for constructing bi-orthogonal wavelets (Sweldens, 1996b).The basic idea behind the lifting scheme is that, it gradually builds a new wavelet, corresponding to resized version of the image with improved properties, by adding new basis functions (Daubechies and Sweldens, 1998).In the first generation wavelets, Fourier transform was a basic tool for wavelet construction.On the other hand, a wavelet transform computation using lifting scheme is entirely in spatial domain and therefore ideally suitable for computing fast second generation wavelets.The lifting scheme consists of three main steps: SPLIT, which sub samples the original data into odd and even sets; PREDICT, which finds the wavelet coefficients and UPDATE, which updates the even set using the wavelet coefficients as discussed further in introduction (Sweldens, 1996a;1996c).
A water marking scheme for digital images based on lifting scheme is proposed in (Kim and Lyu, 2003).Image feature extraction is performed using lifting scheme (Latha et al., 2007).Edge detection based on lifting scheme is discussed in (Ge et al., 2007).Image resizing algorithms using DWT are used to resize the image to any desired scale (Asamwar et al., 2010;2009).In the proposed work the Lifting scheme divides the complete data set into two equal parts that is even and odd based on the spatial sequence.Hence the resizing process based on Lifting Scheme is a comparatively simple task when the total pixels in the image are 2 n ×2 n in number.The problem arises when the image to be resized does not have pixels in the size 2 n ×2 n , thereby making the process cumbersome and complex.This is what contributed to the problem definition of our research work and thus motivated us to think for an appropriate solution for resizing to different scales.Image with dimension not equal to 2 n ×2 n is made equal to 2 n ×2 n size by padding zeros at the required row and column positions and then it is allowed to undergo Lifting Scheme procedure.On similar grounds, magnification of the image can also be achieved with doubling of pixels using zeroeth level DWT transform followed by the IDWT algorithm as proposed in materials and methods.The resizing results can be compared with traditional methods like bilinear and bicubic using MSE and PSNR as presented by many researchers (Tsai et al., 2002;Tsai and Acharya, 2006).The image resizing using DWT is still an unexplored research area.Also the multi resolution property of discrete wavelet transform attributes to efficient methodology for magnifying an image.Comparison between filter based DWT and lifting scheme and measures to evaluate the performance of the proposed technique for resizing are discussed in discussion.The corresponding results of image size reduction and magnification are presented in the results.

Discrete Wavelet Transform (DWT):
When digital images are to be viewed or processed at multiple resolutions, the discrete wavelet transform is the most popular mathematical tool.The Fourier transformation and many other frequency domain transformations reveal only frequency content or attributes of the images.Wavelet transform is the first transform which has been explored a lot in image and signal processing for its unique property of maintaining the spatial domain and frequency domain contents.It also allows you to compromise its resolution in one domain if you require more details in the other domain.This facilitates the most sought multiresolution analysis using discrete wavelet transform.Wavelet series expansion maps a function of continuous variable into a sequence of coefficients.If the function being expanded is a sequence of numbers like samples of a continuous function f(x), the resulting coefficients are called the discrete wavelet transform of f(x).Discrete wavelet transform decomposes the input image (2-D signal) into four different wavelet coefficients at the first level.The process of obtaining these four coefficients (Vetterli and Kovacevic, 1995) is presented in Fig. 1.After dividing the data set into two parts that is even and odd, average and difference is computed.Out of these two coefficients, one is called average and other is called difference.These two coefficients are again processed to compute average-average, average-difference, difference-average and difference-difference terms.Thus after completion of the whole process we have four coefficients.The coefficient average-average is also called as LPLP coefficient (this is exactly visually similar to the original image but 50% in size) and average-difference is called as LPHP coefficient, difference-average is also called as HPLP and difference-difference is also called as HPHP.Now these four coefficients are exactly 50% of the original image.These four coefficients contain the different information of the original image as given below:  Lifting scheme forward transform: Lifting scheme forward transform consists of three steps: Split: In split step the data is divided into ODD and EVEN elements.

Predict step:
The difference between the odd and the even data forms the odd elements of the next step wavelet transformation.The predict step, where the odd value of next iteration is "predicted" from the even value of present step is described by the Eq. 1. Index 'j' represents iteration and 'i' represents element: odd j+1,i = odd j,i − P(even j,i ) (1) Update step: The update step replaces the even elements of the next step with an average of the earlier step.These results in a smoother input (even element) for the next step wavelet transform.The update step follows the predict phase.So in calculating an average the update phase must operate on the differences that are stored in the odd elements: Even j+1,i = even j,j + U(odd j+1,i ) A simple lifting scheme forward transform is diagrammed in Fig. 2.After dividing the complete data set into two parts that is even and odd, the processing is done as follows: For the forward transform iteration j and element i, the new odd element j+1, i would be: odd j+1,i = odd j,j − even j,i (3) Even element of the next step is calculated as: j,i j,i j 1,i even odd even 2 The original value of the odd j,i element has been replaced by the difference between this element and its even predecessor.Simple algebra lets us recover the original value from Eq. 3: Substituting this into the average that is Eq. 4 we get: j,i j,i j 1,i j 1,i even even odd even 2 Fig. 3: Lifting scheme inverse transform The averages (even elements) become the input for the next recursive step of the forward transform.The number of data elements processed by the wavelet transform must be a power of two.If there are 2n data elements, the first step of the forward transform will produce 2n-1 averages and 2n-1 differences (between the prediction and the actual odd element value).These differences are sometimes referred to as wavelet coefficients.
Lifting scheme inverse transform: Inverse lifting scheme transform as the name suggested is the mirror process of forward lifting scheme transform.We recover the original data sequence by going upwards as shown in Fig. 3. Additions are substituted for subtractions and subtractions for additions.The merge step replaces the split step.

MATERIALS AND METHODS
Image resizing using lifting scheme: Image is a data set made up of rows and columns.Monochrome images are two dimensional with one intensity value associated with every pixel.Being two dimensional, they are to be processed first row wise and then column wise assuming that the image signal is orthogonal in both the directions.Here, we present computation of one dimensional wavelet transform using lifting scheme and then extend it to the 2D images to compute the Harr Wavelet transform.Consider the following pixel row intensity data array: 10, 20, 38, 37, 28, 38, 34} Step 1 (split): Initially this data set is divided into two parts.First, third, fifth, seventh pixel are accommodated in the first part called odd part.Second, forth, sixth, eighth pixel are accommodated in the second part called even part that is: Odd = 32,20,37,38 and Even = 10,38,28,34 Now this even and odd data parts are processed in the following stepwise manner to obtain average and difference terms.
Step 2 (predict): The first difference is obtained by subtracting the first even element from first odd element.Similar procedure is adopted for remaining all elements of odd and even vector sequentially, to result in even j+1,I and here DIFF1 vector sequence as given below: even j+1,i = odd j+1,i -even j,i and thus DIFF1= {22,-18, 9, 4} Step 3 (update): The first average term is obtained by adding the first odd and first even element and by dividing this sum by two.This process is continued for the complete data set of odd and even elements sequentially and the resultant vector odd j+1,i and AVG1 here is obtained as shown below: Even j+1,i = (even j,i + odd j,i )/2 thus AVG1 = {21, 29, 32.5, 36} Thus initial eight elements present in a data array are reduced to four average and four difference terms.Same procedure from Step 1-3 is repeated further, considering AVG1 vector as input data array for the next level forward lifting scheme transformation.Thus the successive transformations can be taken till each component reaches single coefficient.
For image processing applications, this procedure of Step 1-3 is applied to all the rows present in the image.The resulting matrix is subjected to the same procedure of 1-3 Step on all columns in the image.This results in Low Pass Low Pass (LPLP) approximation of the image and is the 50% reduced version of the original image.Thus at each level, the LPLP (AVG component after row-wise and column-wise computation of lifting scheme DWT) is considered as the interpolated version of the original image.The difference components obtained at every stage are used for reconstruction of the original image.For the next level lifting scheme forward transformation average component of earlier level acts as an input.At every lifting scheme forward transformation, image size gets reduced to 1/2 of the input image size.Thus finally individual the component (LPLP, LPHP, HPLP and HPHP) of an image gets reduced to single DWT coefficient after successive transformations.Figure 4 and 5 shows resizing of cameraman.tifimage.

Reconstruction of the image using Inverse discrete wavelet transform:
The original data can be reconstructed with the help of inverse lifting scheme.Four pixels obtained in forward lifting scheme transform now acts as an input to reconstruct average vector in inverse lifting scheme.four components (LPLP, LPHP, HPLP and HPHP).
These are now subjected to the Inverse Discrete Wavelet Transformation using Inverse Haar lifting wavelet.The procedure of successive magnification using zeroeth level DWT is presented in following algorithm.
Algorithm 1: (1) Zeroeth level discrete wavelet coefficients are computed for an image (m×n) resulting in the four zeroeth level DWT components (LPLP, LPHP, HPLP and HPHP) each of size (m×n).Here the Harr filters [1 1] and [-1 1]are applied row-wise and column-wise for all i = 1:m and j = 1:n, similar to regular DWT transformation (2) Inverse Discrete Wavelet Transformation algorithm is applied on these four zeroeth level DWT components using inverse discrete Harr filters [1 1] and [1 -1]; resulting in magnification of the original image to size (2m×2n) i.e., i = 1:2*m and j = 1:2*n (3) Resulting image can again be subjected to the zeroeth level DWT and subsequent inverse discrete wavelet Transformation for further magnification as described in ( 1) and (2) (4) Above steps from (1): (3) are repeated till required magnification level is achieved Table1 and 2 presents comparative performance of bilinear, bicubic and DWT methods using MSE and PSNR as evaluation parameters for color and b/w images respectively.MSE and PSNR values pertaining to magnification and subsequent reconstruction of images are presented in Table 3 and the corresponding images are shown in Fig. 11.Variable scale magnification can also be implemented using piecewise application of lifting IDWT-DWT procedure for magnification using fractional level DWT concept and piecewise application of DWT (Asamwar et al., 2009).----------------------------------

Comparisons between filter based dwt and lifting scheme:
The lifting scheme is a new method for constructing biorthogonal wavelets.The main difference with classical constructions is that it does not rely on the Fourier transform.The lifting scheme and filter based DWT can be compared as follows: • The lifting scheme allows a faster implementation of the wavelet transform.Traditionally, the fast wavelet transform is calculated with a two-band sub band transform scheme, Fig. 1.In each step the signal is split into a high pass and low pass band and then sub sampled.Recursion occurs on the low pass band.The lifting scheme makes optimal use of similarities between the high and low pass filters to speed up the calculation.The number of hops can be reduced by a factor of two • The lifting scheme allows a fully in-place calculation of the wavelet transform.In other words, no auxiliary memory is needed and the original signal (image) can be replaced with its wavelet transform • In the classical case, it is not immediately clear that the inverse wavelet transform actually is the inverse of the forward transform because here the filter is used to obtain DWT.Only with the Fourier transform one can convince oneself of the perfect reconstruction property.With the lifting scheme, the inverse wavelet transform can immediately be found by undoing the operations of the forward transform.In practice, this comes down to simply changing each '+' into a '-' and vice versa • The lifting scheme is a very natural way to introduce wavelets in a classroom.Indeed, since it does not rely on the Fourier transform, the properties of the wavelets and the wavelet transform do not appear as somehow "magical" to students without a strong Fourier background Measures to evaluate the performance: The signal in this case is the original image data and the error can be computed by subtracting the reconstructed image data from the original image data.Each error component is squared and then mean is found out as Mean Squared Error (MSE).Noise is represented using the Mean squared error resulting out of the transformation and the subsequent reconstruction.The MSE is defined for two m×n size monochrome images I and K, where one (K) of the image is considered a noisy approximation of the other (I) and is defined in Eq. 8: The parameter peak signal-to-noise ratio often abbreviated as PSNR and is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation.As many signals have a very wide dynamic range, PSNR is usually expressed in terms of the logarithmic decibel scale.The PSNR is most commonly used as a measure of quality of reconstruction of lossy reconstruction.The PSNR is defined as in Eq. 9: Here, MAX I is the maximum possible pixel value of an image.When the pixels are represented using 8 bits per sample, this is 255.For color images with three RGB values per pixel, the definition of PSNR is same except that, MSE is the sum over all squared value differences divided by image size and by three.Typical values for the PSNR in lossy image and video compression are between 30 and 50 dB, where higher PSNR indicates better reconstruction.When the two images are identical the MSE will be equal to zero, resulting in an infinite PSNR.MSE and PSNR values for the work presented here for various gray and color images are tabulated in Table 1 and 2. Mean squared error in case of magnification experimentation is presented in Table 3.

CONCLUSION
During the course of this work it has been observed that the interpolation of images and subsequent reconstruction using lifting DWT scheme requires less computational overhead, as compared to the DWT filter methods.This requires less memory as the calculations are in place.Lifting scheme DWT computation of an image is done till one pixel for each of the four components is obtained and using this single pixel value of each component, the original image is reconstructed using the inverse lifting transform.Thus, resizing of an image to its reduced size and its reconstruction to the original image size is carried out with better MSE and PSNR as compared to the bilinear and bicubic scheme.As already discussed, the lifting scheme is easily applied on the images (monochrome and colored) of size 2 n ×2 n , however zero padding enables application of the scheme on all sizes of images but with additional computational overhead.The MSE and PSNR obtained during reconstruction of the images have been presented and are better as compared to the bilinear and bicubic interpolation schemes.An additional advantage of the DWT based approach for image resizing against the bilinear and bicubic method is that the reconstruction from very small size interpolated images using bilinear and bucubic method does not yield very good MSE and PSNR while the reconstruction using DWT even from a single pixel yields good MSE and PSNR.Magnification of an image to four times its size and the subsequent reconstruction to original size using the proposed approach yields surprisingly less MSE and better PSNR.During the course of magnification it has been found that the MSE and PSNR are not better than bilinear and bicubic performance.However, these results are obtained at much less computational load as compared to the bilinear and bicubic interpolation methods as presented in Table 4.

•
LPLP: This coefficient contains the low frequency image content and is the reduced resolution • LPHP: It contains Horizontal information of the image • HPLP: It contains vertical information of the image • HPHP: It contains diagonal information of the image Implementation of DWT results in 50% reduction in size (row and columns) at the 1st level DWT.Further decomposition is carried out by applying the LPLP output component of the first level to the same algorithm, resulting in the four 2nd level DWT components.The second level DWT components will have 25% size (rows and columns) of the original image size.Thus one can go on computing DWT till each of the four components reduces to a single pixel size.

Fig. 1 :
Fig. 1: QMF implementation of DWT Lifting scheme: The lifting scheme is a technique for designing wavelets and performing the discrete wavelet transform.The forward lifting scheme wavelet transform divides the data set being processed into an even half and an odd half.

Fig. 4 :
Fig. 4: Lifting scheme based reduction in image size of cameraman.tif

Table 3 :
MSE and PSNR for magnification of images BilinearBicubic DWT

Table 4 :
Computational requirement for interpolation of one pixel No. of mathematical processes Bilinear Bicubic Proposed DWT