Real-time reflection removal algorithm using stereo camera only*

In this paper, we implemented a light reflection elimination algorithm which is one of the main obstacles of various object recognition systems using vision systems. In other words, this algorithm can be used to remove robotic recognition obstacles from the factory and perform tasks such as locating robots, autonomous vehicles, or unmanned aerial vehicles. This algorithm has been studied with versatility in mind and does not require sensors other than stereo cameras. Algorithm that intuitively interprets pixels in HIS(Hue ,Saturation, Intensity) space and processes pixels corresponding to light. For intuitive analysis, take an angle, intensity, or other light source (sunlight, fluorescent light, candle) from the camera, then record each pixel value and separate it from a white object (whiteboard, paper, etc.). After pixel operation, object recognition is implemented using the CNN(Convolutional Neural Network) method using the YOLO(You Only Look Once)_V2 algorithm[5]


Introduction
One of the big problems in the field of image recognition is the detection problem due to light. In order to solve the problem of light recognition, researches on shadow removal technique [3], removal of light reflected in water during rain [2], and removal of afterimage using time difference motion in edge channel are being studied. However, it is difficult to apply these studies in an industry where real -time image recognition is required. For example, removing shadow, removing afterimages using time-lag motion in the edge channel results in elimination of objects (such as a person or a vehicle that should actually be collected) that should not be removed, On the contrary, whe0n the light is removed in the rain, the object is considered more. However, considering the object, the light reflection component itself is not removed much. In this study, considering the marketability, we came to the study with general purpose in mind. This algorithm is divided into two major stages. First, white objects and light reflection components are distinguished in the HSI channel through pixel analysis, And then processes the distinct Light Pixel components.

Pixel analysis
First, the first step is to analyze the pixel values in the HSI channel [1]. In the HSI channel, the RGB values are different depending on the original color, saturation, and brightness. Assuming that the y-axis reference is brightness and the degree of brightness is expressed as 0 ~ 128, RGB color is expressed as shown in Fig.1.
In this paper, to derive the supernatural components of light, several angles and various light sources are photographed in the color space as shown in Fig. 1, and the pixel values corresponding to the light are derived.. Then, layers are constructed based on the collected data, and the pixel values extracted by the layer information are processed by 0 to discriminate. Fig. 2 show the result of light separation in a situation where various objects are placed. In the case of Fig 2, I took a picture with a ZED stereo camera when the sunlight penetrated strongly through the window at 7 o'clock in the morning, and I analyzed various situations in a single photograph by adding a fluorescent light source. In Figure 2, light separates into red boxes and white objects into green boxes..  The meaning of each point is P1, P2 is the light source (fluorescent light, sunlight) P3 Weak sunlight P4 is the weak sunlight on the white wall P5 is the sunlight passing through the plastic water bottle and G1 and G2 are white paper and white glass bottles. Fig. 3 shows the HSI Pixel values for the example Point in Fig. 2. In the light sources P1 and P2, R channel value is 255, G and B channel value are 0 It is possible to confirm that the R channel value of the light material transmitted through the remaining object is relatively high. On the other hand, G1 and G2, which are white objects, have a relatively low R channel value and relatively high G and B channel values. [4] The method proposed in this paper distinguishes between bright objects and white objects in the same way as the analysis in Fig. 3, and shoots bright objects and white objects with various angles and brightness to construct a large number of Data tables and layers about light

Experiment
In order to test the performance of the algorithm, we experimented with the recognition rate by placing an object named H in front of the window in an environment where the window is very sunny at 2:00 pm. For object detection, YOLO_v2 algorithm was used and total 800 pieces of learning data and 500,000 pieces of learning data were used, and in an additional situation, the environment is configured so that the light noise can enter more intensely in the presence of a fluorescent lamp in a relatively bright corridor.
The camera used in the experiment was ZED stereo Cam, Increase the distance to H in the red arrow direction as shown in Fig 4. We collected data about when applying the algorithm presented in this paper and when not applying it.   The y axis in Fig. 5 is the detection rate for the object, and the x axis is the detection distance. Fig. 5 shows the difference in detection rate and detection distance when applying the algorithm (green line) and when not applying it (red line).

Restore
However, if you treat the Pixel of light as '0', the object may be processed as '0' when there is light on the object. In this case, we present a method for restoring objects using surrounding pixel information and color correction. [1] The basic perimeter Pixel information usage method follows eq. (1).  Fig. 6 below shows a restored object using surrounding pixel information.

Conclusion
Although this algorithm has been studied in consideration of versatility, it is impossible to restore the shape of objects such as small letters due to light. Therefore, robots that perceive that the range of objects is simply limited, or peripheral pixels may be applicable in simple fields. These problems will require more research or results from some media that can block light.