A Localization Method for Manta Robots Using an Optical Flow Technique

—In order for an underwater robot to investigate an investigation environment everywhere, it needs to grasp an own position. However, since underwater is the environment where an electric wave does not arrive, it cannot acquire position information from GPS (Global Positioning System) etc. Moreover, small AUVs like the Manta robot are difficult to use a large-sized sensor for ships. Therefore, in this research, we build a localization technique suitable for small AUVs like the Manta robot. It discusses about the optical flow technique suitable for the Manta robot, and verifies about the influence of change of the height from a camera to an object. Furthermore, after pointing out that the pitch movement of the Manta robot influences the detection of an optical flow greatly, in this paper, an oscillatory motion that is similar to the pitch movement of the Manta robot is artificially given to an optical flow, and we discuss about its influence.


I. INTRODUCTION
In recent years, inspection of an underwater structure, ecological observation of aquatic animals, and investigation of seabed resources are conducted actively. The result of the observation and the investigation is recorded by a memory of a diver's viewing, or the taken picture with camera. However, the underwater long-term work in investigation is accompanied with danger. Therefore, development of the underwater research robot which investigates instead of a diver is expected [1] [2].
A propeller type propulsion mechanism is used for the position or attitude control of a conventional underwater robot. The propeller type has a possibility that it is entangled in an underwater plant and is stopped. Furthermore, the underwater radiation noise made by a cavitation may affect action of an aquatic animal [3] [4].
On the other hand, since the propulsion mechanism imitating a living thing does not have a rotor blade, there are the features that underwater radiation noise is smaller than the propeller, and that there are few possibilities of entangling in an underwater animal and plant [5]. Therefore, we have developed a Manta robot which has the propulsion mechanism with pectoral fins that mimicked the pectoral fin of Manta [6]. The Manta robot is a small autonomous underwater robot, and the movements of forward, turning, and diving have been realized by the generated propulsive forces with the wavy motions on the propulsion mechanism with the pectoral fins. Therefore, it can swim silently like an actual aquatic animal, and does not have a bad influence to investigation environments. Moreover, there is the feature that it can swim freely even in the environment where an aquatic plant grows thick.
In order for an underwater robot to investigate the target environment everywhere, it is important to know the own position. However, since underwater is environment which an electric wave tends to decrease, the robot cannot get position information from GPS (Global Positioning System) etc. Moreover, small AUVs such as the Manta robot is difficult to use the large-sized sensor for ship use. Therefore, in this paper, we build a localization method suitable for small AUVs like the Manta robot. It discusses about the optical flow technique suitable for the Manta robot, and verifies about the influence of change of the height from a camera to an object. Furthermore, after pointing out that the pitch movement of the Manta robot influences the detection of an optical flow greatly, in this paper, an oscillatory motion that is similar to the pitch movement of the Manta robot is artificially given to an optical flow, and we discuss about its influence.

II. MANTA ROBOT
The Manta robot developed at our laboratory is shown in Fig. 1. The Manta robot has a pair of the "propulsion mechanism with pectoral fin" on the right and left side as a kind of propulsion mechanism. On each left and right side, the six "fin-rays" are mounted on the propulsion mechanism with pectoral fin, and it has a "fin" between the fin-rays to push out water around the robot. Each fin-ray is driven independently by a RC servo motor, and the traveling wave formed by the upand-down motion of the fin-rays generates a propulsive force.
Moreover, the Manta robot is an autonomous underwater robot (AUV). The inertia sensor which obtains the position and attitude of the body, the micro controller which controls the body, and the lithium ion battery used as the source of power are mounted, and the robot itself can perform an underwater exploration autonomously.

A. Localization Method of the Manta Robot
Since underwater is a special environment, the robot without a telecommunication cable such as AUV cannot obtain a self-localization easily. This is caused by the factors shown below.
 Since the decrease of an electric wave is underwater extremely, the GPS or wireless communications cannot be used.  It needs three-dimensional position information for underwater movement.  There is a lack in the environment information underwater for obtaining a self-position. Therefore, in this research, we propose an underwater selflocalization method suitable for a small AUV, such as the Manta robot shown in Fig. 2.
In the proposed localization method, a suitable sensor mounted on the robot is switched and used corresponding to an environmental condition. 1. Since the electric wave communication can be used when the robot moves on the water surface, the robot obtains the self-position information from the GPS or a ground base station. 2. When the robot moves underwater where a camera cannot obtain the visual information, the robot estimates a self-position using the inertia and a water pressure. Since the inertial measurement unit (IMU) used for a small robot may accumulate an error value in proportion to the amount of movements, the moving distance estimated by the method of this part should be made as small as possible. 3. When moving in the seabed or the neighborhood of a structure, the robot estimates a self-position by using visual information. From the viewpoint of calculation loads, the robot uses the optical flow method which uses only two pictures. More exact localization is realized by fusing the estimated result from the optical flow, and the estimated result from the IMU.
The objective of this paper is to mount a localization method, which uses the optical flow given in section 3, on the Manta robot.

B. Estimation of the Amount of Movements by Optical Flow
An optical flow expresses the amount of movements of an object using a flow vector, which consists of the amount of movements and the direction of movements for every pixel, corresponding to two successive pictures in a video.
Although some methods are proposed to calculate an optical flow, among them, we use the Lucas-Kanade method involved in image-processing library Open CV 2.4.9. The Lucas-Kanade method is a method that calculates the solution of the least squares of a flow vector, using the constraint condition of an optical flow such as the invariant brightness for each pixel on the small portion of a picture, when the optical flows are assumed to be uniform within such a small portion, with respect to several features obtained in advance. In addition, the feature point is a pixel with the feature suitable for the pursuit on a picture.
The amount of movements is estimated by detecting and accumulating the flow vector to the external world, using the camera fixed to the bottom of the Manta robot.

C. Conversion from the Flow Vector to the Real Distance
The flow vector obtained by the optical flow is based on the number of pixels of a camera. In order to estimate the amount of movements of the Manta robot, the following equation is used to convert the number of pixels into the actual moving distance.
where D denotes a coefficient for unit conversion from the number of pixels to millimeter, p [pixel] means the horizontal or perpendicular number of pixels of the camera, L [mm] is the distance from the camera to an objective plane , and  [deg] shows the horizontal or perpendicular angle of view of the camera.
In addition, although the L is considered to be a constant value on the experiments in this paper, it needs to be measured in real time under real environment, because the value always changes with robot movements.

III. EXPERIMENTS TO MEASURE THE AMOUNT OF MOVEMENTS BY OPTICAL FLOW
In order to verify the effectiveness of a measurement method with the amount of movements by optical flow, the

A. Experimental Method
The camera attached at the tip of a robot manipulator (RV-M2) is moved, and the amount of movements of a camera is measured. A laptop PC with a CPU of the Core i5 2.5 Hz and with a RAM of 8 GB is used for image processing. And the camera (BSW13K08H) which has the horizontal angle of view of 32.13 deg and the perpendicular angle of view of 24.37 deg obtains the pictures that have 320  240 pixels with 30 fps. The camera attached to the robot arm is moved 500 mm to the horizontal direction of the camera, keeping a height L = 100 mm, L = 300 mm, L = 500 mm from the objective plane on the picture of a seabed, as shown in Fig. 4. About each height L, the movement speeds of the camera are changed in the range from 10 mm/s to 300 mm/s, and the amount of movements is measured.

B. Experimental Results
The experimental result is shown in Fig. 5. About 500 mm in height, although the amount of movements is smaller than the true value when it is measured with a slowest movement speed 10 mm/s, the value close to each true value is measured to the case of a greater speed. On the other hand, about 300 mm in height, the error is increased corresponding to the increase of the speed in the speed greater than 150 mm/s. However, since the maximum error is 3 % at the speed 300 mm/s, the measurement of the amount of movements has an enough accuracy in all the speed at this height. Moreover, about 100 mm in height, the amount of movements is smaller than the true value when the speed changes according to the increase of the speed, so that it can be considered that the measurement of the amount of movements has failed except for the speed of 10 mm/s. From the result, when the robot moves at a high speed in the height near the seabed, the correct amount of movements may be unmeasurable. However, since the Manta robot is affected by the reaction force which the fins generate at the height near the seabed, depending on the characteristic that the robot moves the fins up and down, it is difficult to swim stably at a high speed at the height. Therefore, in the height near the seabed, the robot needs to swim at a low speed, and there exists few possibilities that a problem occurs in the measurement of the amount of movements.

IV. INFLUENCE OF PITCH MOVEMENT IN OPTICAL FLOW
An attitude of the Manta robot is changed periodically by a movement to swim. Especially, the rotational motion (or the pitch motion), which makes the head move up and down by setting the center of robot as a rotation center, may affect the camera image to detect an optical flow. Therefore, the periodic motion that mimicked a pitch-motion of the Manta robot is given to the camera for the optical flow using a device to generate a pitch motion, and the influence is verified for the measurement of the amount of movements.

A. Device for Generating a Pitch Motion
The device for generating a pitch motion, mounting a camera, is shown in Fig. 6. The device can rotate a camera at an arbitrary angle and cycle by a servomotor. The distance between the rotation center and the camera is set to 30 mm, if considered a case when it is mounted on the real robot.

B. Influence of Pitch Motion Without any Translational Motion in Optical Flow
To verify an influence of the pitch motion to the optical flow, first, the camera is only rotated without any translational motion, and the amount of movements is measured with the optical flow.
The experimental environment is the same as the environment shown in section 3, and the device for generating a pitch motion is set at the tip of the robot arm. The amount of movements is measured in the situation that oscillates the camera by the device for generating a pitch motion, but does  199 not move the robot arm in a translational direction. An oscillatory motion to be applied to the camera in this experiment is a sinusoidal wave which is referred to from the pitch motion of the Manta robot, where the wave has 6 deg in the amplitude and 1.0 s in the period when the plane perpendicular to the objective plane is set to 0 deg. Furthermore, in order to verify the stability of the measurement, the amount of movements is measured at three different points (A, B, C) as in Fig. 4.

C. Experimental Result
Since the amount of movements measured is always changed due to the pitch motion, the minimum value measured within one cycle is set to the reference value for comparison. The minimum value should be constant in an ideal situation, because the camera is not moved by the robot arm. Thus, Table  1 shows the amount of changes of the minimum value per minute as the cumulative error. Table 1(a) is the cumulative error when not applying any noise rejection filter described later, whereas Table 1(b) is the cumulative error when applying a noise rejection filter. For Table 1(a), any accumulated error is observed in all positions A, B, and C. The maximum error is -190 mm/min at the point A, and the minimum error is 25 mm/min at the point C. The sign and magnitude of the cumulative error are found to be completely different, depending on the measured position.

D. Denoising Filter
The size of the optical flow in the x direction which is detected in a certain one frame during the experiment is shown in Fig. 7. At this frame, many flows show the vicinity of -2 pixels, but some of the flows indicate an entirely different magnitude. There is no correlation to the flows of different sizes, so that their values are considered to be noises due to the measurement errors. Therefore, a filter is applied to reduce the noises.
There is a rejection test as a method to remove the unreliable data, such as noises. In the rejection test based on using a standard normal distribution, when the distribution of the data set is assumed to follow to a normal distribution, the upper and lower bounds of the 95% confidence interval in the standard normal distribution are given by the following equation.
The x denotes an average value of the data set, and  means the standard deviation. The upper and lower bounds are set as the thresholds and a value greater than these thresholds is ignored as the noise data. However, the average value x and the standard deviation  are affected easily to one noise data having a large difference from a normal data. Therefore, the trimmed mean x  which is less likely to be affected by the noise and the normalized median absolute deviation madn are used, instead of using x and  .
Here, the n is the number of data,  denotes the ratio to discard some data, and med(x) means the median. In the experiment, we set  = 0.25. The trimmed mean x  is a median value that does not contain the top and bottom data at the ratio of  , and it is not affected by the extremely large and smaller data. The normalized median absolute deviation madn is an unbiased estimator of standard deviation based on median, and it is hard to be affected by the extremely large and smaller data. The threshold value using the standard normal distribution, and the threshold value using the x and madn are shown in Fig.  7.

E. Applying the Noise Rejection Filter and Considerations
When the noise rejection filter is applied to the flow in Fig.  7, the threshold value using the standard normal distribution does not reject the flow at 4 pixel and -4 pixel, but the threshold using x  and madn can reject these data. Therefore, when measuring the amount of movements, the noise is removed by using a threshold value with the x  and madn .
As a result, in Table 1(b), the accumulated error is reduced at the position C where an accumulated error was less relatively, so that the filter is considered to be effective. Note, however, that the cumulative error is increased at the positions A and B.
For the reason why the accumulated error is increased at the positions A and B, it is attributed to the fact that the accumulated errors generated at these positions are not due to noises and may have failed to detect the correct flows. In the flow detected in a certain frame, when the number of flows that fails to detect is greater than the number of flows correctly measured, the correctly detected flow is rejected as a noise by  the noise rejection filter. Consequently, the accumulated error is increased.
To prevent the detection failure of flow, there are some techniques such as an image stabilization by image processing, a camera stabilizer for mechanically stabilizing the attitude of the camera, etc.

V. CONCLUSION AND FUTURE WORKS
In this paper, in order to build a self-position estimation method suitable for a small AUV such as the Manta robot, the amount of movements estimated by the optical flow has been described, and the effect to the optical flow by velocity changing or pitch motion has been also evaluated with considering the actual environment. Furthermore, a noise rejection filter was applied for the cumulative error of amount of movements generated by the pitch motion, and its effectiveness was verified.
As future works, a camera stabilizer will be implemented to reduce the effect by pitch motion for optical flow, and a function will be installed to measure the distance from the camera to an imaging object as required information to estimate the amount of movements.