Convex Image Segmentation Model Based on Local and Global Intensity Fitting Energy and Split Bregman Method

We propose a convex image segmentation model in a variational level set formulation. Both the local information and the global information are taken into consideration to get better segmentation results. We ﬁrst propose a globally convex energy functional to combine the local and global intensity ﬁtting terms. The proposed energy functional is then modiﬁed by adding an edge detector to force the active contour to the boundary more easily. We then apply the split Bregman method to minimize the proposed energy functional e ﬃ ciently. By using a weight function that varies with location of the image, the proposed model can balance the weights between the local and global ﬁtting terms dynamically. We have applied the proposed model to synthetic and real images with desirable results. Comparison with other models also demonstrates the accuracy and superiority of the proposed model.


Introduction
Image segmentation is a fundamental task in image processing and computer vision.Active contour models have become one of the most successful methods for image segmentation 1-5 .Some of active contour models 1, 4-8 are based on the edge information.These models use the image gradient information to stop the evolving contours on the object boundaries.We call them edge-based models.Typical edge-based active contour models 4, 5 have an edge-based stopping term and a balloon force term to control the motion of the contour.There are also some active contour models 2, 9-13 which are based on the region information of the image instead of the edge information.We call them region-based models.Models of this kind use certain region descriptors to segment different regions or identify interested regions of an image.
The split Bregman method 17-19 has also been applied to image segmentation problems recently in 20-22 .In 21 , the authors proposed a convex and fast segmentation method by applying the split Bregman concept to the CV model.Their method is mainly for homogenous images.Thus, Yang et al. 22 applied the split Bregman method to the region-scalable fitting RSF energy model to deal with images with inhomogeneity efficiently.
In this paper, we proposed a new convex region-based image segmentation method based on the LGIF model to consider the local and global information together.We first use the globally convex segmentation idea from Chan et al. 23 to propose a convex energy functional for image segmentation.To minimize the proposed energy functional more efficiently, we use the split Bregman method just as our previous work in 22 .Instead of using a constant weight for the global fitting term, a weight function that varies dynamically with location of the image is applied in this paper.In this way, the proposed model can balance the weights between the local fitting term and the global fitting term by itself.Therefore, the proposed model can segment more general images accurately and efficiently.
The remainder of this paper is organized as follows.We first review some related models and their limitations in Section 2. The proposed model is introduced in Section 3. The experimental results of the proposed model are given in Section 4. This paper is concluded in Section 5.

The Mumford-Shah Model for Image Segmentation
Mumford and Shah proposed a famous image segmentation model in 24 .Let Ω ⊂ 2 be the image domain, and I : Ω → be a given gray level image.Their idea is to find a contour C that segments the given image I into nonoverlapping regions and a piecewise smooth image u that approximates I. u is smooth within each of the connected components in the image domain Ω separated by the contour C. The energy functional they formulated is where μ and ν are positive constants.|C| is the length of the contour C. The Mumford-Shah model has been often used in image segmentation.However, it is difficult to minimize the functional 2.1 in practice due to the unknown contour C of lower dimension and the nonconvexity of the functional.

The CV Model
For a special case of the Mumford-Shah problem when the image u in the functional 2.1 is a piecewise constant function, Chan and Vese 2 formulated a piecewise constant model called the CV model without using the image gradient.For an image I, they proposed to minimize the following energy: where λ 1 , λ 2 , and ν are positive constants.outside C and inside C represent the regions outside and inside the contour C, respectively, and c 1 and c 2 are two constants that approximate the image intensity in outside C and inside C .This energy can be represented by a level set formulation, and then the energy minimization problem can be converted to solve a level set evolution equation 2 .One of the most attractive properties of the CV model is that it is much less sensitive to the initialization.The optimal constants c 1 and c 2 that minimize the above energy are the averages of the intensities in the entire regions outside C and inside C , respectively.Such optimal constants c 1 and c 2 will not be accurate if the intensities within outside C or inside C are not homogeneous.Local intensity information which is crucial for inhomogeneous image segmentation is not considered in this model.This is the reason why the CV model can not handle image inhomogeneity.Similarly, more general piecewise constant models in a multiphase level set framework 9, 13 are still not suitable for images with intensity inhomogeneity.

The LBF Model
Li et al. proposed the LBF model in 14, 15 to segment images with intensity inhomogeneity by using the local intensity information efficiently.The energy functional they proposed is where Ω 1 outside C and Ω 2 inside C .f 1 and f 2 are two local fitting functions that approximate the intensities outside and inside the contour C. K σ is a Gaussian kernel with the standard deviation σ.The localization property of this kernel function plays a key role in segmenting images with intensity inhomogeneity.However, such localization property may also introduce many local minimums of the energy functional.Consequently, the result is more dependent on the initialization of the contour.This has been deeply explained in 16 .
The local intensity fitting LIF energy E LIF φ, f 1 , f 2 is defined as the first two terms of the LBF model 14, 15 : where φ is the level set function, H is the Heaviside function.
The global intensity fitting GIF energy E GIF φ, c 1 , c 2 is defined as the first two terms of the CV model 2 : Then the other two terms P φ 1/2 |∇φ x | − 1 2 dx and L φ |∇H φ x |dx are also needed as in 2, 7, 13 to regularize the level set function φ and the contour C, respectively.In practice, the Heaviside function H is approximated by a smooth function H ε defined by where ε is a positive constant.The derivative of H ε is the smoothed Dirac delta function: 2.8 Thus the energy functional they proposed is where ν and μ are positive constants.

The New Proposed Convex Model
We propose a new and convex region-based image segmentation model to consider both the local and the global information as 16 .The energy functional 2.9 of the LGIF model in Section 2.4 is not convex.Following the idea in Chan et al. 23 , we first propose a convex energy functional based on the LGIF model.According to Wang et al. 16 , the optimal local fitting functions f 1 , f 2 , constants c 1 , c 2 , and level set function φ that minimize the energy functional 2.9 in the LGIF model are updated using the standard gradient descent method as follows:

3.4
F 1 and F 2 are called the LIF force and the GIF force, respectively.
To apply the idea in Chan et al. 23 to the LGIF model, we consider the gradient flow equation 3.3 .The last term is used to regularize the level set function φ to be close to a distance function.Here we drop it first, the new obtained gradient flow equation is and without loss of generality, we take ν 1.
We then apply the globally convex segmentation idea of Chan et al. 23 ; the stationary solution of 3.5 coincides with the stationary solution of We now propose a new energy functional as follows: It can be clearly seen that the simplified flow 3.6 is just the gradient descent flow of the new proposed energy functional 3.7 .Thus the minimization problem we want to solve is To guarantee the global minimum, the solution is restricted to lie in a finite interval.In this paper we use a more general form a 0 ≤ φ ≤ b 0 as follows: min where r x − F 1 x F 2 x .The segmented region can be found by thresholding the level set function for some α ∈ a 0 , b 0 if the optimal φ is found: where the thresholding value α is chosen as α a 0 b 0 /2 in this paper.
The first term in the proposed energy functional 3.7 is in fact the traditional total variation TV norm:

3.11
To incorporate information from an edge detector 25 , we then replace the standard TV norm 3.11 with the weighted TV norm: where I is the given image and g ξ 1/ 1 β|ξ| 2 is the nonnegative edge detector function.β is a parameter that determines the detail level of the segmentation.
Thus the proposed minimization problem becomes min where φ, r φ x r x dx.

Split Bregman Method for Minimization of the Proposed Model
The efficiency of the split Bregman method for image segmentation has been demonstrated in 21, 22 .We now apply the split Bregman method to solve the proposed minimization problem in a more efficient way.We introduce the auxiliary variable, d ← ∇φ.To weakly enforce the resulting equality constraint, we add a quadratic penalty function which results in the following unconstrained problem:

3.14
The Bregman iteration is then applied to strictly enforce the constraint d ∇φ.The optimization problem becomes arg min

3.16
Keeping d fixed, the Euler-Lagrange equation of the optimization problem 3.15 with respect to φ is For 3.17 , a central difference is used for the Laplace operator and a backward difference is used for the divergence operator; the numerical scheme for 3.17 is φ i,j max min β i,j , b 0 , a 0 .

3.18
For a fixed level set function φ, we minimize 3.15 with respect to d and obtain where shrink x, γ is the shrinkage operator 18, 21 defined as

3.20
Thus the split Bregman algorithm for the proposed minimization problem 3.13 can be summarized in Algorithm 1.
In this algorithm we use GS r k , d k , b k , λ to denote one sweep of the Gauss-Seidel formula 3.18 .This algorithm is different from the one in our previous work 22 when updating r.
Note here that we say the proposed energy functional E φ in the minimization problem 3.13 is convex; in fact it means that it is convex with respect to φ for a fixed r.
From Algorithm 1, it can be seen that r is computed before updating φ.Thus each time when we update φ using the Gauss-Seidel formula, the value of r is in fact fixed.

The Choosing for the Parameter ω
The parameter ω controls the influence of the LIF force and GIF force which can be seen clearly from 3.4 .When the intensity inhomogeneity is severe, the accuracy of the segmentation relies on the LIF force.Thus a smaller value of ω should be chosen as the weight of the GIF force.For images with minor inhomogeneity, the GIF force alone is able to attract the contour to a location near the object boundaries.In this case, relatively larger ω should be chosen as the weight of the GIF force.In 16 , ω is chosen as a constant for a given image.Wang et al. need to choose an appropriate value for ω according to the degree of inhomogeneity.
In our paper, we choose ω in a different way as 26 .Instead of a constant value for ω, a weight function that varies dynamically with location of the image is chosen in this paper.The weight function ω is defined as follows where γ is a fixed parameter and C N represents the local contrast ratio of the given image, which is defined as where N denotes the size of the local window, M max and M min are the maximum and minimum of the intensities within this local window, respectively.M g represents the intensity level of the image, for gray level images, it is usually 255.C N x varies between 0 and 1.It reflects how rapidly the intensity changes in a local region.It is smaller in smooth regions and larger in regions close to the object boundaries.

Experimental Results
We have tested the proposed model with synthetic and real images in this section.As in 22 , we simply initialize the level set function φ as a binary step function which takes a constant value b 0 inside a region and another constant value a 0 outside.The advantage of using a binary step function as the initial level set function is that new contours can emerge easily and the curve evolution is significantly faster than the evolution from an initial function as a signed distance map.We use a 0 −2, b 0 2, σ 3.0, ε 1, γ 0.1, and λ 0.001 for all images shown in this paper.We choose β 100 for all gray images and β 1 for all color images.The values chosen for the parameters λ 1 and λ 2 are specified in each figure .Figure 1 shows the results for an inhomogeneous image from 26 with different methods.a and b show the original image and the initial contour, while the final contours using the CV model, the LBF model, the LGIF model, and the proposed model are shown in c -f .From this example, we can see that the CV model fails to get the correct segmentation result.The LBF model traps into the local minimum.The LGIF model also gets an incorrect result by using a constant weight ω.The proposed model gives the right segmentation result.We choose λ 1 1.1e − 6 and λ 2 1e − 6 for this image.This example demonstrates the superiority of our proposed model over other models.Furthermore, our result is even better than the result gotten from 26 ; the upper part of the object was missed The results for an airplane image using different methods are shown in Figure 2. λ 1 1.1e − 6 and λ 2 1e − 6 are also used for this image.We can see that the result obtained by applying the proposed model is the best one among all these four models.This can also be seen clearly from the final level set function φ shown in Figure 3.
Figure 4 shows the results for two X-ray images of vessels with different methods.We use λ 1 λ 2 1e − 5 for these two vessel images.We can see from b and g that the CV  In Figure 6 we give a synthetic inhomogeneous image that the proposed model can handle while the other models fail to segment it correctly.We use λ 1 λ 2 1e − 5 for this image.From b -d we can see that there are unwanted curves in the final segmentation results when using the other three models.e shows that the proposed model can get the correct final contour.In fact by observing the curve evolution processes of these four models, unwanted curves also grow when using the proposed model.However, these unwanted curves will finally disappear with our model because they can move quickly by using the split Brgman method.
Figure 7 shows the corresponding mesh figures of the final level set function φ with different models for the same synthetic image from Figure 6.Comparing these four mesh figures of the final φ, we can observe that the proposed model can obtain the best final level set function φ shown in d of Figure 7.
The proposed model can be easily extended to be applied for color images.Figure 9 shows the curve evolution process from the initial contour to the final contour for a color image of starfish with the proposed model.This image is very inhomogeneous both in the background and foreground and is difficult to segment.We choose λ 1 λ 2 1e − 7 for this image.The proposed model can segment it correctly.
In Figure 10, we show the results of several other color images with the proposed model.We use λ 1 1.1e − 5 and λ 2 1e − 5 for the first banana image, while λ 1 λ 2 1e − 6 for the other three images.a -d show the original images with the initial contours.e -h show the original images with the final contours.Experimental results show that our model can segment color images well.
In our previous work 22 , we have proposed a convex model by applying the split Bregman method to the RSF model.We call it the SBRSF model here.In the SBRSF model we only consider the local information without considering the global information.Thus the SBRSF model may get many local minimums, which can be seen in Figure 11.In Figure 11, we give the results of some color images from Figures 8, 9, and 10.The same initial contours and parameters have been used for the SBRSF model as the proposed model.However, local minimums will occur in the leg part of the back elephant, in the right part of the starfish image, in the middle part of the banana image and in the green pepper.The proposed model considers both the local and the global information and thus can get better results than the SBRSF model.

Conclusion
A new convex region-based image segmentation model is proposed in this paper.We consider the local and global intensity fitting terms together and propose a convex energy functional using the globally convex segmentation method.By applying a weight function that varies dynamically with location of the image, the proposed model can adjust the weight of the global intensity fitting term by itself.The split Bregman method is then used to minimize the proposed energy functional more efficiently.We have compared the proposed model with the CV model, the LBF model, the LGIF model, and our previous SBRSF model with synthetic and real images.Experimental results have shown the advantages of the proposed model in image segmentation.The proposed model is a little sensitive to the parameters λ 1 and λ 2 .In fact, the SBRSF model also has this problem.It may be caused by the application of the split Bregman method.This is what we should study more in the future work.

Figure 1 :
Figure 1: Results of an inhomogeneous image with different models.a The original image.b The initial contour.c The final contour with the CV model.d The final contour with the LBF model.e The final contour with the LGIF model.f The final contour with the proposed model.

Figure 2 :
Figure 2: Results of an airplane image with different models.a The original image.b The initial contour.c The final contour with the CV model.d The final contour with the LBF model.e The final contour with the LGIF model.f The final contour with the proposed model.

Figure 3 :Figure 4 :Figure 5 :Figure 6 :Figure 7 :
Figure 3: Comparison of the final level set function φ with different models for the airplane image from Figure 2. The final φ obtained by the CV model, the LBF model, the LGIF model, and the proposed model is shown in a -d , respectively.

Figure 8 : 2 i 1 Figure 9 :
Figure 8: Results of the proposed model for a color image of elephants.a -d show the curve evolution process from the initial contour to the final contour.e -h , i -l , and m -p show f 1 , f 2 and the fitting image f 2 i 1 M ε i φ f i , respectively, at different iterations.

Figure 10 :Figure 11 :
Figure 10: Results of the proposed model for several other color images.a -d The original images with the initial contours.e -h The original images with the final contours.

2 i 1
Figure 8 shows an application of our model to a color image of elephants.λ 1 1.1e − 6 and λ 2 1e − 6 are used for this image.The first row shows the active contours on the original image from its initial to converged state.The proposed model can segment this image correctly which can be seen from d .The second and third rows show the corresponding two fitting images f 1 and f 2 .The whole fitting images f M ε i φ f i at different iterations are shown in the last row.The final fitting image shown in p can fit the original image well.
Wang et al. proposed the LGIF model in 16 .The LGIF model combines the advantages of the CV model and the LBF model by taking the local and global intensity information into account.There, they define the local and global intensity fitting energy as follows: