A Hybrid Retinal Image Registration Using Mutual Information

Image registration has tremendous applications in medical industry both in diagnosis and therapy. But registering the retinal images is very difficult task because of the structural and illumination criteria. Normal registration algorithms are proved to be less effective and more complicated for such applications. This study aims at combining two methods of retinal image registration: Vessel based and non-vessel based registration to overcome the drawbacks of the existing algorithms. This combined method of registration is efficient and less time consuming when compared to the techniques applied individually. This hybrid technique of retinal image registration has fast convergence in diagnosing retinal occlusion which is very common in diabetic patients of temperate countries. The dye injected retinal image to be registered against the normal red free image is first preprocessed and the bifurcations are extracted using image segmentation. Then the matching criterion between the two segmented images is determined using mutual information. Extraction of bifurcations will make the process of selecting the matching criterion easier by reducing the search space. The best angle of rotation that matches the base image and the reference image is optimized using Simulated Annealing (SA) and this is compared with the results obtained by using Genetic Algorithm (GA). The experimental results have shown that the computational time is reduced when this hybrid technique of registration is implemented with SA when compared to GA.


INTRODUCTION
The requirement for high quality and fast image registration algorithm for registering multimodal retinal images is increasing day by day.Flawless registration of intra or inter modality retinal images helps in integrating the useful information that aids the physician in diagnosing the retinal syndrome.Retinal occlusion is a risky disease which is commonly found worldwide.It is presented with symptoms like sudden decrease in central vision, transient obscuration of vision.Opthalmascopy complaints with diluted and tortuous veins, flame shaped hemorrhage Drexler (2007).Another harmful disease of retina is the macular oedema which affects the retina drained by vein or the whole fundus.Deng et al. (2010) have suggested that registering a sequence of retinal image will give a complete view of the retinal fundus region which may lead to early diagnosis and treatment of AIDS-CMV retinopathy.Gendrin et al. (2012) has suggested in their findings that the real-time image registration methods assist ophthalmologists and avoid possible damages during the surgery.
The registration of retinal images is complicated because of its curved structure and it also requires a high perspective calibrated camera.The difficulty also extends to the factor that the extraction of the retinal vessels is a tough process because the retinal regions greatly vary in their texture and the illumination will not be uniform (Li et al., 2011).Besides, the multimodal retinal images may vary in their structure and color.

AJAS
Any medical image registration technique will come under two major categories like non vessel based (area based) or vessel based (feature based) registration (Gavet et al., 2012).Feature based techniques of registration is dependent on the anatomical structure of the medical images or the landmarks that are prominent and more or less similar between the images to be registered.Few landmarks or anatomical structures that can be taken into consideration in retinal images are curves, corners, bifurcations, fovea.In area based methods the intensity or the gray scale information of each image plays a vital role.Any matching criteria between the two images that have to be registered have to be determined and optimizing this similarity metric completes the registration task successfully.Image registration is normally geometrical aligning of a floating image (sensed image) with respect to a base image (Reference image).It is an effective pre-processing step which will make the fusion more efficient and accurate.The hybrid method of combining area based and feature based techniques is done in three stages .The initial stage being the preprocessing of the retinal image.This is followed by image segmentation to extract bifurcations in the second stage and the actual registration of the retinal image is the final stage.The results are validated by optimizing the registered image using Simulated Annealing and Genetic Algorithm.

Hybrid Method of Registration
The registration of retinal images is a toughest process because of the complications in the structure of the retina and the problem of non uniform illumination.Hence a new criterion has to be resolved which will increase the effectiveness of registration with less computational time.This study proposes a hybrid technique of multimodal retinal image registration which will use the advantages of both the feature based and area based techniques.
The input image which is affected by retinal occlusion is considered as the base image.This has to be registered against a reference image, a Red Free image.This is acquired initially from a normal eye without disturbing the eye with any external dye.The fluorescein angiographic images are usually taken after injecting the fluorescein dye into the patient's vein.The timing between imaging the dye injected retina is crucial because the illumination has to be adjusted in the earlier stages because some time is required for the dye to reach the retina which is approximately 3-4 sec.
The bifurcations are extracted from this FA image using image segmentation by fixing a threshold.This image with extracted bifurcations will be free from non uniform illumination and the matching criterion can be easily determined from the decreased search space (Zheng et al., 2009).This image is now transformed geometrically with the Reference image (RF).The similarity metric between the transformed image and the reference image is determined by mutual information.The mutual information is an entropy based technique where the joint histograms of two images are found which in turn is used to compute the probability density function.The probability of occurrence of one image metric over the other image and the probability density function are determined and the resulting best angle of rotation and the computational time are validated using simulated annealing and genetic algorithm.

Image Preprocessing
Pre processing is the primary step in all image analysis which either suppresses the unwanted distortions or enhances the required fine data in any image.Image preprocessing can be based on brightness of the individual pixel, geometrical transformation, neighborhood pixel or restoration.Any one of the above techniques can be chosen depending on the nature of the available image which in turn is highly dependent on the mode of acquisition and the level or uniformity of illumination (Akram, 2012).
Since the retinal image acquisition itself is a complex process and it has non uniform illumination with fine bifurcations and vascular structures a combination of neighborhood and geometrical processing can be implemented which will enhance the extraction of the bifurcations from the raw retinal image.
Let the input retinal image is represented by Equation ( 1): This is transformed to Equation (2): ( )( ) ( ) where, f is the kernel of size 3×3.This discrete convolution with its kernel f = ℒ (δ) will reduce the error caused in the image acquisition and enhance the brightness of the image which will also accounts for the non uniform illumination.
The noise in the retinal image is removed by using a 2D Gaussian smoothing kernel as given in Equation ( 3

Image Segmentation
Segmentation is a technique used in medical images for extracting the region of interest from a raw image for diagnosis and therapy.Segmentation can be done using many algorithms like thresholding, region growing.Thresholding is a simplest technique which can be done by assigning a threshold value and converting any gray image into a binary image.But such technique may lose the useful information available in retinal images which has non uniform illumination.Hence it is essential that the bifurcations and vascular structures have to be extracted from the retinal image.A stochastic algorithm is required in this process of extracting the bifurcations (Akram, 2012).The major hurdles in this phenomenon are the noise and blurring of the image due to transition in the imaging modalities and the motion during the imaging also causes equivalent de linearization in the retinal image.This complicates the extraction of the bifurcations.Hence a region based segmentation algorithm is also not preferred in such cases because it could not meet the gap between the bifurcations that are caused mainly in retinal occlusion.
A hybrid technique of combining the advantages of the region based segmentation and edge based segmentation can be used for effective segmentation of the bifurcations from the retinal image.The region joining can be done based on the combined data from the individual pixel, neighborhood pixel and also the information from the extracted edges and bifurcations.The identified regions for merging by this method should comprise of sufficient strength in the gradient magnitude.Any noise present in this structure may affect the region merging which can be rectified by the holes present in the image due to the noise forms a sufficiently large fraction of the boundary (Gavet et al., 2012).Table 1 shows the extracted bifurcations of three different retinal images after preprocessing and segmenting.

Image Registration
Image registration can be done in three basic steps.First the base image has to be geometrically transformed with respect to the reference image.The geometrical transformation can be linear, affine or projective transformation depending upon the geometrical features of the reference image.Then the matching criterion is found between the geometrically transformed image and the reference image.Finally the matching criterion is optimized for an effective registered output image using any global optimization techniques.
In this study a hybrid technique of combining gray scale information and geometric transformation is been implemented.Here the original image is degraded by the image acquisition and also illumination by an error factor assumed to be e (x, y).The expected image is g (x, y) whereas the degraded image is assumed to be f (x, y) as shown in Equation (4 and 5): ( ) ( ) ( ) f x, y e x, y * g x, y = (4) But the above method has a limitation of calculating the brightness and illumination over time to time and if it cannot be followed then the error function may overflow: e x, y = (5) But gray scale information is independent of the position of the pixel.Hence the geometric transformation can be carried out on the expected image f(x, y).
Geometrically transformed image will also help in detecting the difference in the required image and the acquired image.It is the mapping or geometrical alignment of a pixel f(x, y) in image A to a new position f (x', y') in image B which is a continuous value as the position does not necessarily match the grid or digital image after the required transformation.Geometrical mapping requires two images the reference image 'A' and a floating image 'B'.The overall registration process is shown in the Fig. 1.
Correlation methods cannot be applied for multimodal images because in this method the intensities are assumed to be uniform.Hence mutual information is an appropriate parameter which can be used to register the two multimodal temporal images effectively (Reel et al., 2013).Mutual information is an entropy based technique given by information theory where the statistical dependencies of one random variable over the other are measured.Here the image registration process involves in finding the similarity metric between the input image and the reference image.The input image is assumed to be the floating image and the reference is the base image.
The dependency or occurrence of the reference image over the base image is calculated by using the Shannon's entropy formula for finding the mutual information.The similarity criterion is selected after extracting the bifurcations using image segmentation as explained in the previous section.
The registration by mutual information can be mathematically formulated by the following objective function given in Equation ( 6):

Experimental Results
This proposed method of hybrid retinal image registration for detecting retinal occlusion at earlier stages by combining the vessel based and non vessel based technique is tested by selecting ten test images of the affected retina.The images are obtained from the data base and are listed in Table 2.The image size is also mentioned along with the respective images.The matching criterion is optimized using simulated annealing and it is also validated using genetic algorithm.The required time of convergence and the best matching coordinates in both the methods are tabulated.The results show that there is a tradeoff between the computational time and the best matching coordinates.

Image Registration by Simulated Annealing
Simulated Annealing is a search heuristic that can be applied for solving optimizing problems globally.The major advantage of this technique is that it skips the local minima in its search which makes this technique to be used for wide range of applications including image registration.
This study uses this SA algorithm to evaluate the suitable run time parameters like different cooling rates, Markov chain length and adaptable stopping criteria for effective image registration (Raiha, 2010).
Starting the search with a suitable initial temperature is tedious and by selecting a temperature with a probability p 0 = 0.8 is universally accepted.This is determined by finding the average increase of δ f in the objective function where all the increase in temperature is accepted and T 0 is given in Equation ( 7 This generated neighborhood should decrease as the search progresses hence this has to be made dependent with the temperature.This is given in Equation ( 9 The acceptance ratio P acc neither cannot be chosen to a lower value in order to avoid the efforts of searching the rejected values and nor it should not be a higher value which will slow down the convergence and hence an optimum value of 0.5 is chosen.

Image Registration by Genetic Algorithm
This hybrid retinal image registration is also tested with genetic algorithm which is an evolutionary optimization technique that contributes for the survival of the fittest.The population size should be carefully chosen because of the fact that the cross over is done pair wise and the exclusive individual is pre-selected and this makes the implementation more comfortable.A randomly selected bit divides the genes and the portions on the least significant side are exchanged between the genes.The x & y offset s are incremented or decremented with a random number between -2 and 2. This is also tried with various combinations of binary encoding with two points cross over and also with value encoding mutation.
Binary encoding is chosen in which each parameter gets a number of bits and all the selected bits are added to form a full chromosome representing the present state.Crossover and mutation rates are tested in the range [0.0, 1.0] respectively.The mutation rate are determined by using the below formula: P mutation x = 1- where, σx0, σy0, σθ0, σs0 are the standard deviation of the initial population for each parameter respectively.The computational time and matching coordinates are shown in Table 3.The registered retinal images after rotation and matching are listed in Table 4.

CONCLUSION
The registration of retinal images is critical because of its vascular structure and non-uniform illumination.These factors have complicated the process when this registration technique is purely vessel based or non vessel based.This study aims at combining the vessel based and non vessel based techniques of retinal image registration which has reduced the computational time.This is achieved by initially obtaining the retinal images and then preprocessing it for uniform illumination and then the bifurcations are extracted using image segmentation.This makes the process of selecting the matching criteria between the base and the reference images in which the mutual information is calculated.The resulting best angle of rotation, the computation time using the standard and normalized methods are validated using the simulated annealing and genetic algorithm.The results show that the computational time is less when the registration is implemented with simulated annealing as the local minima is been excluded in this optimization technique whereas the search time exceeds in genetic algorithm because of the population size and false convergence at the local minima.The study can be further extended by including additional parameters for registration besides the mutual information.

Fig. 1 .
Fig. 1.Flow of image registration (X;Y) denotes the registered image, p(x,y) is the conditional probability and p(x) and p(y) are the individual probability function of the base and the reference image respectively.The search is a time consuming process if the similarity criterion is not known.Mutual information based registration techniques have hardly considered the shape information where the search space is comparatively large and the iterations that required for convergence are voluminous.This complexity is overcome by the extraction of bifurcations initially and then finding the matching of the similarity criterion between the base and the reference image.
temperature schedule the neighborhood function generation is an important task which accounts for the success of SA.The generated neighbors should be such that they are reachable from any point in the search space through the Markov walk given in Equation (8): vector to regulate the step size g T , h * g T , maxrot * g T , maxscale * g T (P mutation x + P mutation y + P mutation θ + P mutation s )/4

Table 1 .
Extracted bifurcations after pre processing

Table 2 .
Various retinal images affected by retinal occlusion and their

Table 4 .
The registered retinal images of three pairs