Noise Removal and Detail Enhancement of Passive Infrared Image Pretreatment Method for Robot Vision

Abstract: In order to explore the usefulness of thermal infrared imaging as a mobile robot sensing modality and to make feature extraction more effective and accurate, an algorithm for noise removal and detail enhancement of the blurred infrared image based on Guided filter has been raised in this paper. We first used a guided filter to smooth the input image and separate it into a base layer and a detail layer. Then constraining the gradient of the detail will get a reliable halo free detail layer and using gain mask to enhance it later. Meanwhile, the base layer controls the gray scale contrast which also needs to be processed with adaptive histogram equalization. Finally, the two parts of the image combined with weighted coefficients will be exported into the second guided filter. This method has the advantages of computational simplicity and a great performance. Many experiments and illustrations have been made to prove its effectiveness in improving the perception ability of vision system for a mobile robot.


Introduction
The properties of infrared thermal imaging technology make it not only visually display the object surface temperature field in the form of pictures, but also enable it to show the environment and identify targets highly robustly to changing lighting conditions and other environmental effects.It is a relatively new field to exploit the thermal infrared imaging as a mobile robot sensing modality, such as being used for mobile robot object Identification [1], Simultaneous Localization And Mapping (SLAM) [2].
However, compared with a visible light camera, the context of infrared images in the video-based SLAM system has some most serious challenges: -Limited resolution -High noise -Low contrast -Poor texture distribution The four listed disadvantages of infrared thermal imaging have a great influence in achieving a large number of reliably tracked features between frames in a thermalinfrared video sequence [3].Therefore, infrared image pretreatment method is meaningful and necessary for robot vision.
Most researchers' work about passive infrared image pretreatment is for the purpose of human vision, which is different from robot vision.Moreover, we need to not only deal with the higher temperature object of the infrared image, but also focus on the blurred infrared image with no significant difference in temperature or the background part of the heat source object.In this paper, we introduce a pretreatment method that aims at enhancing the details, preventing noise amplification and removal unwanted halo artifacts effectively.

The principle of the proposed algorithm
The basic strategy of our proposed algorithm is that a guided filter could smooth the input image and separate it into the base layer and the detail layer.Then high frequency information is enhanced by our modified certain strategy in the detail layer.Meanwhile, the base layer controls the gray scale contrast which also needs to be processed with histogram equalization.Finally, the two parts of the image combined with weighted coefficients, the assembled image will more smooth and less noisy after exported into the second guided filter.The completed processing pipeline is schematically illustrated in Figure 1.Since the GIF is the key to our algorithm, we first give a fast review on how the GIF works.

Guided Image Filtering [4]
The guided Image filter smooths an input image p under the guided image I.A local linear model assumes that filtered output q is expressed by a linear function of I in a window ω k centered at pixel k: where (a k , b k ) are some linear coefficients assumed to be constant in window ω k .A square window is used in the original formula of 2D guided filtering.We can determine the linear coefficients by minimizing the difference between q and p.
The least squares solution of ( 2) is given as: where and are the mean and variance of I in w k , |w| is the number of pixels in, w k and is the mean of p in w k .Because each pixel lies in the overlapping windows, then a pixel's smooth result is the average of multiple estimates, we rewrite the final Equation: where and are the average coefficients of all windows overlapping i.

Histogram Redistributionfor base component
A projection histogram is very suitable in image dynamic range compression [5].Because the basic level of detail is less, and the dynamic range is large, so it can be compressed directly without taking into account the loss of details.In this paper, we use the improved projection histogram technique.
Firstly, with statistical image histogram information H (x), the histogram of the base layer is binarized by a threshold T. The number of pixels in the histogram is greater than the gray value of T marked as 1, otherwise marked as 0. Then the cumulative histogram distribution is obtained by the histogram D (x): The maximum range of output image R as: where n valid denotes the total number of the valid gray levels and D is the dynamic range of a monitor.This limit ensures that the output image gray level range does not exceed the display effective display range.Taking into account it that the image background is relatively simple and effective gray level is relatively small, the histogram could be distorted and full of noise.Here we focus on how to raise the output value in a better and more effective way.The histogram projection output is modified as follows: Pare is a controllable parameter for fine tuning the output brightness.To ensure that the output gray level is less, then the output image will not be compressed to a very low brightness.I BP is the basic layer after histogram redistribution.

Removal of halos Artifacts
Though the guided filter is better than bilateral filter about avoiding the gradient reversal artifacts may appear in detail enhancement detail enhancement for highdynamic-range infrared images which have been run through a Guided Filter produce halos near some edges [6].The problem is more acute when enhance the Larger multiples of details, so requires a reliable halo free detail layer.
In [7] prevent gradient reversal by directly constraining the gradient of the detail.We also force the detail derivatives and the input derivatives to have the same sign.However, For the Guided filter's better edge preserving performance avoid gradient reversal, we modify the detail's gradient by an amplification parameter: in which is the x directions' gradient of original details, is the x direction's gradient ofinput image.
is the modified x directions' gradient.According to next process, the gain mask enhancement, δ = 1.5 can give out a good result.The y component is defined similarly.After modifying the gradient of detail layer, the corrected detail layer is obtained by solving the reconstruction Poisson equation: Here, I is the reconstructed image, v is a 2D field of 2D vectors. is the gradient of I.

Gain mask enhancement for detail component
The edge-preserving filtering property of guided filter explained intuitively help us to design the gain mask [4].Consider the case where I = p.
We simplify the gain mask as follows: (12) The parameter KH(the max), KL: (the min) can modify the masking function, which makes the detail enhancement more controllable.With this function we are able to enhance the detail region as much as we need, and directly using save a number of computation.3 Experimental results

Test images and the process of the method
The workflow of the method displays in Figure 2. In Figure 2a is the original IR image.Figure 2b and 2d are the base layer and detail layer after filtering original image by GF.Then, we first project histogram the base layer's gray levels distribution and get Figure 2c, and it can be noticed that the brightness of Figure 2c is brighter than Figure 2b.Meanwhile, we process the detail layer.As shown in the detail layer Figure 2d, there is very dark and blurry of the details because the temperature of objects in original IR image is very small.It is necessary to make sure that the detail derivatives and the input derivatives have the same sign.After applying the technique removal of halos artifacts, the correction result in Figure 2f.Then the gain mask enhancement of Figure 2f is shown in Figure 2g.After dealing with the base layer and detail layer, Figure 2e is the combination of them.Finally, Figure 2h is the output of the second GF of Figure 2e.The bright of the enhancement showsthe details clearly.The contrast is better than original images, and the local details are perceived well.The parameters setting are as follow: the twice of GF is same as: r = 4; eps = 0.5^2.The Gain Mask is KL = 1; KH = 3;

Comparison with two Guided filter output image
Why use a guided filter twice?This is because after the first filtering of original image, the result of the flow of process until the combination of the base layer and the detail layer is often used for human observation.However, it is not necessarily suitable for robot vision.In the process of detail enhancement will inevitably enhance some noise or interference light intensity difference the details are not needed, and micro texture are strengthened.Then the second filter not only effectively erases the increased noise and the unnecessarily details, but also highlights the key edge.We use the "canny" edge extraction algorithm for verification.The results of comparison are the output edges images of three images.It is very obvious that the noise of the original image is terrible, especially the less the temperature difference is in an indoor environment.Then, the first GF has good noise removal, but the edges are cluttered and there are too many.The last edge image has a good effect to use for robot vision, and to a number of works such as SLAM.

Conclusion
In this paper, a novel HDR IR image enhancement approach is presented.Firstly, the guided filter is utilized to separate the input image into the base layer and the detail layer.Then the projection histogram to make the distribution of the base layer more evenly.Then we constrain the gradient of the detail to get a reliable halo free detail layer and using gain mask to enhance it.Finally, the two parts of the image combined can be used for human observation.In order to get less noise and only main details of image, the arranged image will be to exported into the second guided filter.The performance of HDR IR image detail enhancement of the proposed method has been demonstrated can improve the contrast, enhance the scene details effectively with less artifacts.This approach can be useful in many applications for robot vision.

Figure 2 .
Figure 2. The proposed method flow chart on test image.(a) Original IR image.(b) The base layer by GF.(d) The detail layer by GF.(c) The projection histogram of base layer.(f) The halo removal of detail layer.(g) The Gain mask enhancement of (f).(e) The combination of (c) and (g).(h) The second GF of (e).

Figure 3 .
Figure 3.The "canny" algorithm on test image inside the hall.(a) Original IR image.(b) the edge of Original Image.(c) The edge of the first GF process.(d) The edge of the second GF.

Figure 4 .
Figure 4.The "canny" algorithm on test image the square.(a) Original IR image.(b) The edge of Original Image.(c) The edge of the first GF process.(d) The edge of the second GF.