TWO-STEP METHODS FOR IMAGE ZOOMING USING DUALITY STRATEGIES

. In this paper we propose two two-step methods for image zooming using duality strategies. In the ﬁrst method, instead of smoothing the normal vector directly as did in the ﬁrst step of the classical LOT model, we reconstruct the unit normal vector by means of Chambolle’s dual formulation. Then, we adopt the split Bregman iteration to obtain the zoomed image in the second step. The second method is based on the TV-Stokes model. By smoothing the tangential vector and imposing the divergence free condition, we propose an image zooming method based on the TV-Stokes model using the dual formulation. Furthermore, we give the convergence analysis of the proposed algorithms. Numerical experiments show the eﬃciency of the proposed methods.


1.
Introduction. Image zooming schemes are becoming increasingly popular due to the wide use of digital imaging devices and the increasing sensor-capturing capabilities in spatial resolution, several good zooming techniques have been presented, see [2,3,7,33,37,43] for details. In this paper we address the problem of producing an enlarged picture from a given digital image (zooming).
A large class of image zooming is done by interpolating the discrete source image. The simplest method for resizing image is pixel replication, the methods with higher precision than pixel replication include bilinear interpolation (BLI) [28,32] which use polynomials of degree one, making the resized image visually more pleasing than pixel replication.

TINGTING WU, YUFEI YANG AND HUICHAO JING
However, the above methods cause undesired effects, e.g., low quality edge blurring, or making jagged or synthetic edges. In order to produce an anti-aliasing effect, we need to find some methods which have relatively smooth edges with hardly any jaggies.
The Rudin-Osher-Fatemi (ROF) model [36] is a classical second order model in image restoration which looks for a minimizer u of the total variation (TV) regularization problem: where f represents the observed image, Ω ⊂ R 2 is the domain where the image is defined, α > 0 is a regularization parameter which controls the tradeoff between the contribution of the fidelity term Ω (u − f ) 2 dx and the TV regularization term Ω |∇u|dx. In general, the nondifferentiable regularization term |∇u| is replaced by its smooth approximation like |∇u| ζ = u 2 x + u 2 y + ζ for some ζ > 0. It is wellknown that the ROF model preserves sharp region and edges well. However, the TV regularization suffers from the so-called staircase effects in smooth regions, e.g., see [1,6,9,26,27,34].
In order to overcome the staircase effects appeared in the second order ROF model, a number of fourth order models [11,26,42,45] have been proposed. One of the fourth order models proposed by Lysaker, Lundervold, and Tai (known as LLT model) [26] is as follows: where β > 0, |∇ 2 u| = u 2 xx + u 2 yy + u 2 xy + u 2 yx 1 2 . Unfortunately, the fourth order PDE-based methods damp out high frequency components of images faster than second order PDE-based methods and tend to introduce some blurring in regions of image edges [30]. Moreover, this model fails to be easily implemented due to the existence of the high order derivatives.
Recently, a two-step method has been proposed by Lysaker, Osher, and Tai (called LOT model) in [27], its basic idea is to decouple the fourth order problem into a set of two second order problems. By replacing the TV-norm of u in the ROF model or ∇u in the LLT model by the TV-norm of ∇u |∇u| , we have the following minimization problem: where γ > 0. It is obvious that ∇u |∇u| is the unit normal vector for the level curves of the image u. Then, (3) can be solved by the following two steps: • In the first step, the normal vector is regularized via the TV norm: where n 0 = ∇f |∇f | is the initial normal vector. • In the second step, a desired image is reconstructed by finding a surface to fit the smoothed vector field: On the other hand, based on some geometrical considerations of isophote directions, the two-step method in the LOT model is improved with the divergence free constraint, inducing a nonlinear Stokes equation in the fist step of the modified model (called TV-Stokes model) [39].
The staircase effects and edges blurring are among the most prominent dilemmas of many image zooming methods. From the literature [12,13,15,27,34], we can find that the above two two-step models are good at preserving edges while reducing staircase effects and avoiding the explicit computation of the fourth order model for image denoising. In this paper, based on the above two-step models, we propose two two-step image zooming methods. In the first method, we improve the first step in the LOT model by reconstructing the unit normal vector of the restored image by means of Chambolle's dual formulation. Then, we find a surface to fit the obtained dual variable in the second step. In the second method, we use the dual formulation of TV-Stokes model to zoom images, where the first step involves trying to reconstruct the isophote directions for the missing data, and the second step tries to construct an image fitting the restored directions. One essential idea is that we impose the zero divergence condition on the constructed directions. This guarantees that there exists an image such that its isophote directions are the restored vectors. This is important when the zooming image is relatively large.
The organization of the paper is as follows. In Section 2, we give a detail description of the modified LOT model based image zooming with the split Bregman iteration algorithm. We propose a dual scheme for TV-Stokes model based image zooming in Section 3. Section 4 is devoted to implementation details of numerical experiments, followed by some conclusions in Section 5.   is a subspace of functions u ∈ L 1 (Ω) such that the following quantity is satisfied: where C 1 c (Ω) stands for the set of functions in C 1 (Ω) with compact support in Ω. If u ∈ W 1,1 (Ω) ⊆ BV (Ω), then Ω |Du| = Ω |∇u|dx. According to the above definition, as shown by Chan-Golub-Mulet [10], the minimization problem (1) is equivalent to the min-max problem: The min and max can be swapped according to the minimax theorem [35], which gives the explicit minimizing solution for u: By combining (6) with (7), we obtain the following max problem:

TINGTING WU, YUFEI YANG AND HUICHAO JING
Based on the Legendre-Fenchel transformation, Chambolle [7,8] proposed a semiimplicit gradient descent (or fixed point) algorithm to solve the dual variable for the ROF model. Then, the above formulation (8) can be solved with the following semi-implicit fixed point iteration: where δ > 0 is a fixed time-step. Chambolle's dual algorithm avoids smoothing the nondifferentiable TV regularization term and has been proven to be an efficient algorithm [7,8,14,40].
On the other hand, the Euler-Lagrange equation corresponded with the ROF model (1) can be written as: that is which is just Chambolle's formulation (7) if the dual variable ω is replaced by ∇u |∇u| . Motivated by the connections between the dual variable ω and the unit normal vector n = ∇u |∇u| of the image, the normal vector n can be reconstructed by computing the dual variable ω. Based on this fact, Pang and Yang [31] proposed a two-step model for image denoising.
• In the first step, the normal vector is obtained by Chambolle's dual formulation: • In the second step, by replacing the normal vector n in (5) with the dual variable ω, the following minimization problem is solved: where g ∈ ∂J(v). In general, D g J (u, v) = D g J (v, u) and the triangle inequality does not hold, which mean that the Bregman distance D g J (u, v) is not a distance in the usual sense.
Given a general form of the unconstrained l1-regularized problem: where both |Φ(u)| and H(u) are convex functions, | · | denotes the l1 norm, |Φ(u)| is nonsmooth and nonseparable. To overcome these drawbacks, we first replace the term |Φ(u)| in (12) by a separable term |d| and then add a new constraint d = Φ(u) into (12). Hence, (12) becomes Based on the penalty method, the above problem (13) can be converted into an unconstrained problem as follows: If set E(u, d) := |d| + H(u), we then can apply the Bregman iteration [20] to solve the minimization problem (14) as follows: One natural choice for solving the first subproblem in (15) is the alternative minimization or a block nonlinear Gauss-Seidel algorithm. So, we get the following iteration: whose convergence analysis can be found in [5,24].

Two-
Step Image zooming Method Using Duality Strategy and Split Bregman Iteration. As we know, the original LOT model outperforms the ROF model and some fourth order models. Recently, some improvements of the LOT model have also been studied in [15,34,38]. However, the improved methods mentioned above suffer from computational difficulties, especially in the first step (4). The zooming model we shall propose is based on the two-step model described in Subsection 2.1. Let Ω 1 represent the grid points in the original low resolution image pixels, we want to get a zoomed image in Ω ⊃ Ω 1 . Set Ω = Ω 1 ∪ Ω 2 .
• In the first step, we get the normal vector by • In the second step, the surface fitting step (10) is reformulated as with ∈ Ω 1 where f represents the observed data with low resolution, u denotes the desired zooming image, and γ > 0 is a regularization parameter. Γ Ω1 (u − f ) is the characteristic function of Ω 1 [19] and will be denoted as Γ. Now, we consider how to perform the two-step image zooming scheme described in the above. In fact, it follows from (16) that the dual variable ω can be solved directly with a semi-implicit iteration on an analogy of the Chambolle's dual algorithm. To solve the second step (17), we exploit the split Bregman iteration introduced in Subsection 2.2, which possesses fast computational speed.
Actually, the minimization problem (17) is equivalent to the unconstrained l1regularized problem (12) with Φ(u) = ∇u. Based on the split Bregman iteration, we can solve (17) by the following iteration processing: For the first subproblem in (18), the optimality condition for u is easily derived as follows: For the second subproblem of (18), it is easy to deduce that the optimal value of d can be obtained by shrinkage operator: where shrink(x, ξ) i := sign(x i ) * max(|x i | − ξ, 0), ∀x ∈ R n , ξ > 0. Finally, for the third subproblem of (18), what we need to do is to update the Bregman variable simply: Theorem 2.3. Assume that there exists at least one solution u * of (17) and γ > 0, λ > 0. Then, the following property for the unconstrained split Bregman iteration (18) holds: lim Proof. The first order optimality conditions for (18) are given as follows: Furthermore, following the same manipulations as did in [5], it is not difficult to deduce that where u k e = u k − u * , d k e = d k − d * , and b k e = b k − b * are the notation of errors. Summing the above equation from k = 0 to k = K and by the assumption λ > 0, we get lim k→+∞ ∇H(u k ) − ∇H(u * ), u k − u * = 0, lim k→+∞ p k − p * , d k − d * = 0, and lim k→+∞ ∇u k+1 e − d k+1 e 2 2 = 0. Because the Bregman distance is nonnegativity, we get from (11) that Moreover, d * = ∇u * , therefore lim k→+∞ ∇u k+1 − d k+1 2 2 = 0, which shows lim k→+∞ |∇u k | − |∇u * | − ∇u k − ∇u * , p * = 0.
Summing this and (22) up, we obtain The proof of the following theorem is similar to that of Theorem 3.2 in [5]. Theorem 2.4. As k → +∞, γ > 0, and λ > 0, {u k } generated by (19) converges to the unique solution u * of (17).
Update k := k + 1. end 3. Two-Step Image Zooming Method Based on TV-Stokes Model.
3.1. TV-Stokes Model. In this subsection, we introduce another new variation of the LOT model, TV-Stokes model [17,25,39]. Instead of smoothing the normal vectors in the first step of the LOT model, the first step of the TV-Stokes model smoothes the tangential vector field with the condition that the vector field is divergence free (incompressible).
Given an image u and a constant c ∈ (−∞, ∞), the level curves: has normal vector n and tangential vector τ given respectively by The vector fields n and τ satisfy the following irrotational condition and the incompressible condition in the fluid mechanics: The TV-Stokes model consists of two steps [21]: in the first step, we regularize the tangential vector field to the level curves of the image and in the second step we reconstruct the image whose gradient fits the regularized normal vector obtained in the first step as did in the LOT model.
Given an observed image f , we get the initial vector field τ 0 = ∇ ⊥ f . In the first step of the TV-Stokes model, we obtain a regularized vector field τ = (τ 1 , τ 2 ) T via the energy functional minimization: The TV norm preserves discontinuities well and is good at dealing with sharp ridges or valleys in image processing. The divergence free constraint is reasonable because we regularize vector field τ from τ 0 = ∇ ⊥ f . Once the regularized tangent vector field τ = (τ 1 , τ 2 ) T is obtained in the first step, the corresponding regularized normal vector field n = (τ 2 , −τ 1 ) T can be computed. So, in the second step, we solve the same problem as (5).
When solving the above TV-Stokes model numerically, it is an usual process to find the corresponding Euler-Lagrange equations and then to iterate the nonlinear PDEs with an explicit gradient descent method until the steady-state is reached. However, such an algorithm tends to suffer from computational difficulties due to the time steps choosing. In the following subsection we will introduce the dual formulation of the primal TV-Stokes problem [17], which improves the computation speed drastically, and present another image zooming method based on the dual TV-Stokes model.

Two-Step Image Zooming Method Using Duality Strategies and TV-Stokes Model.
In TV-Stokes model, the dual variable ω is now a vector p composed by the two dual variables for both directions in the tangential vector field. Definition 3.1. Let Ω ⊂ R 2 be an open subset with Lipschitz boundary, the total variation of the tangential vector field satisfies: where p 1 = (p 11 , p 12 ), It follows from analysis in Subsection 2.1 that the primal problem (23) can be rewritten as the following equivalent form: Now, let us introduce a trick to deal with the constraint divτ = 0 [17].
By exchanging the order of max and min in (24), the problem (24) can be reformulated as Choose t 1 > 0. The solution of (25) can be computed by Chambolle's iteration: Once we have the smoothed tangent field τ = (τ 1 , τ 2 ) T from (26) in the first step, the corresponding normal vector field is n = (τ 2 , −τ 1 ) T . In the second step the zoomed image u is reconstructed by fitting the normal vector field n. Consider the following minimization problem: Similar to the derivation in the first step, the primal problem (27) can be reformulated as the min max problem

Numerical Experiments.
In this section we present some numerical results to demonstrate the competitive performance of the two-step model based image zooming algorithms proposed. For both gray scale images and RGB color images, our results will be compared with the BLI, the LLT model-based image zooming method [19]: and the LOT model-based image zooming method with time marching algorithm, where the step-size parameters for two steps in LOT model is denoted as t 1 lot and t 2 lot , respectively. In practice, the time steps of Chambolle's dual strategy for stability and convergence are achieved by choosing δ, t 1 and t 2 is less or equal to 1 4 and not 1 8 [7,16]. The stopping criterion we used in the experiment consists in checking that the maximum variation between u k+1 and u k is less than 10 −3 . The experiments were performed under Windows 7(x64) and MATLAB v7.14 (R2012a) running on a personal computer with an Intel Core i5-2450M CPU at 2.50 GHz and 6.00GB of memory. The standard mean squared error (M SE), peak signal-to-noise ratio (P SN R), and signal to noise ratio (SN R) are used to measure the quality of the restored images. They are defined respectively by Here, M × N is the size of the original high-resolution image, I and J are the original and zoomed images respectively, while  Our first test is the well-known gray scale Lena image. The 128 × 128 down sampled image in Fig.1 (a) is zoomed by factor 4 with the original size of 512 × 512 pixels in Fig.1 (b). In the LLT model-based image zooming method, we choose ξ = 0.31, β = 10, t llt = 0.06, n = 739. For the image zooming with TSSB mehtod, we set α = 0.125, δ = 0.12, λ = 0.001 and γ = 100. The parameters of the TSTVS image zooming method are selected as t 1 = 0.2, t 2 = 0.13, ν = 0.02, = 0.0015 and η = 10. Obviously, there are many jagged edge artifacts in the BLI zoomed image Fig.1 (c) (SN R = 6.0489, M SE = 665.8421, P SN R = 19.8671). As can be seen in Fig.1 (d), the LLT zooming method smooths the region too much so that it cannot preserve the details exactly (SN R = 6.3789, M SE = 600.4897, P SN R = 20.3457). In Fig.1 (e), we can observe that these drawbacks are reduced by using the TSSB image zooming method (SN R = 6.4495, M SE = 577.4758, P SN R = 20.5155). Furthermore, the image zoomed by TSTVS method produces relatively smooth edges with hardly any jaggies in image Fig.1 (f) (SN R = 6.5299, M SE = 525.9683, P SN R = 20.9212). Because the two-step models take full advantage of both the ROF model and the fourth order model, it is easy to see that our proposed image zooming methods yield better performance compared with the BLI and LLT schemes.
The second test is the binary Circle image with size 140 × 140, which is down sampled to a 35 × 35 one in Fig.2 (a) by factor 4. As we know, the LOT model is one of the two-step models. In this example, we make some comparisons with the LOT model-based image zooming method, where we choose µ = 0.008, γ = 20,  The image zooming algorithm described above for gray scale pictures can be easily generalized to the case of RGB color digital images. The last example is a portion of the color image Pepper with 130 × 140 pexels. Firstly, we also resize the original image to a 65 × 70 one in Fig.3 (a) by factor 2. In the LOT model-based image zooming method, we choose µ = 0.01, γ = 10, t 1 lot = 0.12 and t 2 lot = 0.11.
For the image zooming with TSSB mehtod, we set α = 0.3, δ = 0.25, λ = 0.002 and γ = 60. The parameters of the TSTVS image zooming method are selected as t 1 = 0.1, t 2 = 0.15, ν = 0.03, = 0.0018 and η = 8. From the zoomed images Fig.3 (c)-(f), we can see that the zoomed images by our proposed method have less jagged effects than the BLI, LOT zoomed image and make a good compromise between edge preserving and smoothing. This demonstrates the competitive performance of our two-step methods in color image zooming.

5.
Conclusions. In this paper the two image zooming strategies based on two-step models were studied. Motivated by the connections between the dual variable and the unit normal vector, we improved the first step in the LOT model. In the second step, we applied the reliable and effective split Bregman iteration algorithm with shrinkage operator to solve the related subproblem. By imposing the divergence free condition on the tangential vector field, we got a nonlinear TV-Stokes image zooming model. To solve the minimization problem in the TV-Stokes model, the dual strategies were presented with fast computational speed. In particular, the twostep models took full advantages of both the ROF model and the fourth order model so that our proposed image zooming methods can retrieve relatively smooth edges while alleviating the jagged effects. Compared with other schemes, the experimental results of our two-step zooming models exhibited a better visual impression by different scaling factors, both in gray scale and RGB color images.