Medical image fusion by wavelet transform modulus maxima

(cid:1)(cid:2) Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.


Introduction
In the recent years, the study of multimodality medical image fusion attracts much attention with the increasing of clinic application demanding.Radiotherapy plan, for instance, often benefits from the complementary information in images of different modalities.Dose calculation is based on the computed tomography (CT) data, while tumor outlining is often better performed in the corresponding magnetic resonance (MR) scan.For medical diagnosis, CT provides the best information on denser tissue with less distortion, MRI provides better information on soft tissue with more distortion, and PET provides better information on blood flow and flood activity with low space resolution in general.With more available multimodality medical images in clinical applications, the idea of combining images from different modalities becomes very important and medical image fusion has merged as a new and promising research field.
The general object of image fusion is to combine the complementary information from multimodality images.Some image fusion methods have been introduced in the literatures, including statistical method (Bayesian's decision) [1], Fuzzy set method [2], Neural network method [3], Laplacian pyramid method [4] and wavelet transform method [5][6][7].It should be noted that the fusion methods are application-dependent.In case of edge detection for object recognition, margin information is important.While in application of image restoration, complementary component is necessary.Thus, an effective fusion algorithm should integrate all the relevant information as much as possible.
In this study we present a scheme of image fusion by calculating the wavelet transformation modulus maxima of input images.The wavelet transformation was employed in [5][6][7], but the wavelet transformation coefficients were directly used as the reference for the fusion rules.In those cases, the edge and margin information was not efficiently extracted.By employing the wavelet transformation modulus maxima in this paper, we emphasize the extraction of the edge and margin information.This is the essential difference between this research and the others [5][6][7].
The advantages of the proposed algorithm are: 1.A better preservation of both edge features and component information of the objects from different modalities in new fused image; 2. Fusion can be performed at different levels and bandwidths.
In addition, we also propose a metric based on mutual information for evaluating the fusion performance in this study.The metric can be used to evaluate the statistical dependence between the fused image and the original image.
It also should be noted that, a fundamental task of image fusion is image registration.Before image fusion, the multimodality medical images must be registered perfectly.Image registration is also an interesting task.The bulk of registration algorithms can be found in the literatures [8,9] .In this paper, the images to be fused are assumed having been registered.
The paper is organized as follows.A brief introduction of the wavelet transform modulus maxima is presented in Section 2. The new fusion scheme and mutual information for evaluating the fusion result are described in Section 3. Experiment results are shown in Section 4.

Wavelet transform modulus maxima for images
The goal of image fusion is to integrate complementary information from multimodality images so that the new images are more suitable for the purpose of human visual perception and computer processing.Therefore, the task of image fusion is to make many salient features in the new image such as regions and their boundaries.
To date wavelet transform modulus maxima has been used to extract explicit important features of images, as these features carry the information of sharp signal transitions and singularities.In his book [10], Marr conjectured that images could be reconstructed from multiscale edges.For a Canny edge detector, this is equivalent to recovering images from multiscale edges.Mallat and Zhong [11] have introduced an alternative projection algorithm which can be used to compute an image approximation that visually identical to the original one from their wavelet maxima.In this study we employ wavelet transform maxima as a means for image fusion performance.Here, we give a brief introduction of the calculation of wavelet transform maxima.
Supposed that wavelets¸ and¸ have exactly one vanishing moments and a compact support.Then there exists a function θ of compact support such that ¸∂θ∂x,¸∂θ∂y and } θ (t)dt 0. Thus wavelet transform can be written as a multiscale differential operator The modulus of this gradient vector is proportional to the wavelet transform modulus And its angle is Where Then the local maximum of the wavelet transform modulus M * f(u,v,2 j ) at point (u 0 ,v 0 ) can be calculated by solving ∂Mf(u,v,2 j )=0.

Reconstruction from modulus maxima
Wavelet transform maxima reflect the properties of sharp signal transitions and singularities.If a signal can be reconstructed from those maxima, it is then possible to modify the singularities of the signal by processing the wavelet transform modulus maxima.To study the completeness and stability of wavelet maxima representations, Mallat and Zhong introduced an alternate projection algorithm that recovers signal approximation from their wavelet maxima.The following is a simple and fast reconstruction algorithm presented by Mallet [12] from a frame perspective.At each scale 2 j , a multiscale edge representation provides the positions (u j,p , v j,p ) of the wavelet transform modulus maxima as well as the values of the modulus Mf(u j,p ,v j,p ,2 j ) and the angle Af(u j,p ,v j,p ,2 j ) .The modulus and angle specify the two wavelet transform components It is the orthogonal projection of f in the closed space V generated by the family of wavelets {¸M S 1 , ¸M S 2 }.If {¸M S 1 , ¸M S 2 } is a frame of V, which is true in finite dimensions, then a f is computed by calculating The reconstructed image a f is not equal to the original image but their relative mean-square difference is below 10

2
. Singularities and edges are nearly perfectly recovered and no spurious oscillations are introduced.The images differ slightly in smooth regions, which is not visually noticeable.

The scheme of fusion
The general image fusion scheme of this study involves decomposition of the input images, calculation of the wavelet transform modulus maxima, fusion strategy, and reconstruction for new fusion image as shown in Fig. 1.
First, the input images to be fused are decomposed by forward wavelet transformation.In this case, each image is decomposed into the same levels using a periodic discrete wavelet transform.The wavelet transform decomposes each image into low-and high-frequency subband images.At a given level, a series of parameter sets of the forward wavelet transformation can be obtained.
Wavelet transform modulus maxima at different levels can then be calculated according to the formula mentioned in section 2. The third step is to combine the information from each image by fusion rules.It must be pointed out that the main idea of wavelet-based image fusion is motivated by the factor that the subband images constitute the details and the features of the original images.Low-frequency subbands related to the coarse part of the images, while high-frequency corresponds to the region boundaries or edges.As mentioned above, the task of image fusion is to combine the complementary information carried by multimodality images.Therefore, the general principle of making fusion rules is to keep the salient features in the new images such as regions and edges as much as possible.Thus the fusion parameter selection rule can be obtained

DWT
Eq. ( 9) can be explained that the transform values of details are fluctuating around zero.The larger absolute transform values in these subbands correspond to sharper intensity changes and thus to salient features in the image such as edges, lines, and region boundaries.Therefore a good integration rule is to select the larger absolute value of the two coefficients of the wavelet transform at each point.
Finally, by using the backward wavelet transform incorporating with the selected coefficients, we can get the fusion image.

Evaluation of the fusion result
We propose to use the mutual information or relative entropy to describe the dispersive behavior of the salient features in images in this research.As it is known that MI is a basic concept from information theory, measuring the statistical dependence between two random variables or the amount of information that one variable contains about the other.The mutual information or entropy is widely used in image compression.In those cases, it has been employed for designing image coders [13~15] .The MI fusion criterion presented here states that the MI of the image intensity values of respective pixel pairs is maximal if the images are well fused.Mutual information I (A, B) is given by Where p(x) and p(y) are the probability density functions in the individual images and p(x, y) is the joint probability density function.Estimations for the joint probability density function and marginal density functions p AB (x,y), p A (x), p B (y) can be obtained by simple normalizing the joint and marginal histograms of the overlapping parts of both images.According to Eq. ( 10), we can define the image fusion performance metric as Where

Experiment results
In this section, we report the experiments on CT and MRI images to test the proposed image fusion scheme.Fig. 2 shows the original matched CT and MRI images.Generally speaking, the size of each image for fusion may not be the same.In Fig. 2, each image is resized as 256 pixels by 256 pixels.The value of every pixel in CT and MRI images usually takes two bytes.
Without losing generality, we take 256 gray levels for fusion performance test.To get a good effect, the window level and width must be adjusted well.Each image is then decomposed into four levels using periodic wavelet transform.Fig. 3 shows the distributions of wavelet transform modulus maxima sets of CT and MR images in  To evaluate the fusion performance, the mutual information measurement described in section 3 is employed.According to Eq. ( 11)~( 13), the fusion performance assessing results can be obtained and are shown in table 1.
.6277 In this study, we also compare the performance of the proposed fusion scheme with other two distinct schemes.Both the fusion methods use wavelet transform.In the first approach, the larger values of low and high subbands of two input images are selected for reconstruction of the new image.In the alternative one, the low subbands of two input images are put together and the larger values of high subbands are selected.Table 2 shows the fusion performance assessing results of the two approaches.Comparing Table 1 with Table 2, one could easily fined the advantage of the proposed fusion scheme.

Summary
An image fusion algorithm based on the wavelet transform modulus maxima is described and applied to multimodality medical images.The advantages of the proposed algorithm are: (1) both edge features and component information of the objects from different modalities are preserved in the fused image effectively; (2) features at different levels and bandwidths are extracted for fusion.Besides, it introduced a meaningful fusion performance evaluation metric based on mutual information.It can be used to assess the effectiveness of different image fusion algorithms.
Fig.2The original matched images

Fig. 4
Fig.4 The new fused image

Fig. 5 Fig . 5
Fig .5The marginal distributions of the original images Fig. 6 shows the joint probability density functions of p FA (f,a) and p FB (f,b).

Table 1 .
The fusion performance assessing results

Table 2
fusion performance-assessing results of the two approaches