Vision-based markerless measurement system for relative vessel positioning

The measurement of relative vessel positioning is important to prevent the risk of vessel collision or the deterioration of working efficiency. This study presents a vision-based system for measuring the relative position and heading between a vessel and a target object without the installation of transponders, reflectors, or markers. Such a markerless measurement system, which is not restricted by the presence or the absence of transponders, is a method of expanding feasible conditions for relative vessel positioning. The proposed system consists of multiple camera units that can measure the distance and direction between an initially designated point on a target object while keep track of the designated point. To evaluate the measurement performance for the proposed system, the authors conducted measurements by using a prototype system in water. The results demonstrated that the system can keep track of the designated points and accurately measure the position and heading of a vessel relative to a target object, despite the vessel's roll motion.


Introduction
A vessel operator is forced to concentrate on the position and heading of a vessel continuously while manoeuvring it during long time relative positioning, such as loading and unloading. The operator's fatigue caused by this continuous manoeuvring leads to a reduction in working efficiency and safety. In particular, the risk of accidents, such as vessel collision, greatly increases in close proximity to a target. To reduce such risks, the relative position and heading of a vessel must be measured for safer and efficient positioning by triggering an alarm or automatic manoeuvring.
GPS-based techniques have been used as an effective standard approach for measuring a vessel's position at sea. Ueno [1] proposed a GPS-based system with multiple receivers that provides sufficiently accurate position and attitude of a vessel for berthing operations. Relative positioning will be achieved by measuring both positions of the target and vessel. However, the usable range of centimetre-level GPS-based positioning is limited, because it needs a shore-based reference station for carrier-phase observations. In addition, in close proximity to a target, GPS-based systems risk signal interruptions from tall structures, such as cranes or derricks.
The use of a radar [2] is another dominant technique. This system consists of an interrogator on a moving vessel and several transponders on a target object. The relative position and heading of a vessel are obtained from measured distances and bearings between the sensor and each transponder. Similarly, a laser-based system consisting of a laser sensor on a vessel and reflector on a target also has been used [3]. Although these systems achieve robust and reliable relative position and heading measurements, the usage scenarios of these systems are limited to where transponders or reflectors are placed on a target.
A solution to the problem of measuring the relative position without any installation on a target is markerless measurement using vision sensors. Generally, in vision-based systems, the accuracy of distance measurement relies on the baseline between two cameras. As a result, the proposed system will be inaccurate for the more distant targets. However, the risk of vessel collision decreases with distance from the target. On the other hand, at close proximity, high frequency measurement is necessary for precise positioning of the vessel in order to avoid collision. Hence, vision-based measurement is considered as an optimal solution for vessel positioning at close proximity to the target. A few vision-based systems have been proposed for autonomous navigation [4,5]. Although these systems efficiently detect and avoid obstacles, they are not designed to measure the position and heading of a vessel relative to a designated target. The authors have been proposed vision-based system for relative position and heading measurement [6]. However, this system also needs the installation of specific landmarks for measuring the distance between a vessel and a target [7][8][9].
In this paper, we propose a vision-based measurement system without any installation on a target object, to expand the applicable range of relative vessel positioning. The proposed vision-based system consists of two camera units with multiple cameras and a motorised camera mount. Each camera unit is capable of keeping track of the designated point on a target object and measuring the distance and bearing to the tracked point. Results of an experiment performed using a prototype system will demonstrate that the proposal is feasible for relative position measurement even during the vessel's rolling in rough water.
The remaining of this paper is organised as follows. In Section 2, the structure of the proposed system and the principles of position and heading measurements are described in detail. In Section 3, we show the results from our evaluation of the proposed system. Our conclusions are presented in Section 4.
2 Vision-based measurement system for position and heading 2.1 System configuration Fig. 1 shows the configuration of the proposed position measurement system. This system consists of two camera units placed on a vessel as shown in Fig. 1a. We designate a point on the target object for each camera unit at the start of the measurement. Each camera unit keeps track of the designated point. The vessel's position (x, y) and heading θ relative to the midpoint of two designated points are given by the distances d L , d R and bearings j L , j R between each camera unit and its designated point.  Fig. 1b shows the configuration of the camera unit. The camera unit consists of a tracking camera, two measurement cameras, and a motorised camera mount, which is a called a pan-tiler. The tracking camera has a wide field of view used to track the designated point without losing sight of it. In addition, the tracking camera is used to obtain the bearing to the designated point. The measurement cameras obtain the distance from the camera unit to the designated point. These cameras have a longer focal length than the tracking camera to accurately measure the distance. The pan-tilter rotates the cameras on its pan and tilt axes to keep track of the designated point.
Typically, two types of configurations can be considered for stereo-vision-based measurement with a moving target. One configuration is where actuators rotate each camera, as shown in Fig. 2a. This configuration requires the cross point of the optical axes of the cameras to be on the target for distance measurement. However, the complexity of the system increases for accurate distance measurement because the positional relation between each camera changes continuously. Another configuration is where actuators rotate an entire stereo camera, as shown in Fig. 2b. Considering the need for simple implementation and accurate measurement, we selected the configuration that has a fixed positional relation of cameras.
In practical use, cameras are wobbled continuously by waves or by the motion of the vessel. The positions of the designated point in each camera are temporally shifted by differences in photographic timings. To eliminate positional shift, the photographic timings of all cameras are synchronised by the input of a trigger signal. In addition, images are blurred owing to the wobbling of the cameras. Although the motion of a vessel is not fast, the estimated distances might be inaccurate because of the blur of the designated points in measurement cameras. To reduce blur in images, we employed progressive global shutter cameras.

Tracking of the designated point
To keep track of the designated point, the positional shift between a reference image and the current input image is calculated successively. The reference image is memorised by the tracking camera at the start of the measurement. We assume that the centre point of the reference image is the designated point.
The positional shift between the reference image and the current input image is calculated based on image feature points. In recent years, excellent feature detectors and descriptors have been proposed [10][11][12][13][14]. Although some algorithms [10][11][12] yield good results in a short time, they have lower accuracy than SIFT (scale invariant feature transform) [13] and SURF (speeded-up robust features) [14]. In this study, we use a GPU-based SURF algorithm, which has sufficient processing speed, scale invariance, and rotation invariance. Fig. 3a shows detected feature points using a GPU-based SURF algorithm from the reference image. Fig. 3b shows detected feature points from the image captured at a slightly different position. From these detected feature points in each image, point correspondences can be obtained as shown in Fig. 3c. In this figure, each line with a circle represents the displacement between matched feature points. However, these point correspondences include many mismatch pairs, particularly, when the target object has repetitive patterns, such as the images shown in Fig. 3. We eliminate visibly wrong feature matching points by using a voting algorithm based on the Euclidean distances between the matched feature points, as shown in Fig. 3d. Then, the homography between two images is estimated from obtained point correspondences. Here, the mismatch pairs are rejected using the RANSAC algorithm [15]. The arrow in Fig. 3d shows the positional shift of the designated point from the reference image.
The designated point is projected into an input image with a pixel position (u o , v o ) relative to the optical axis, as shown in Fig. 4a. However, this pixel position is shifted in practical use by the distortion of the lens. The corrected pixel position (u, v) is rescaled according to the following equations where k 1 , k 2 , and k 3 denote the coefficients of radial distortion, and p 1 and p 2 denote the coefficients of tangential distortion. Although the distortion factors can be estimated by camera calibration (e.g. [16]), in the proposed system, we employed a low-distortion lens and assumed that all distortion factors are equal to 0 for simplification. As shown in Fig. 4b, the horizontal angle c cx between the optical axis and designated point is calculated from the horizontal coordinate u by using the following equation where f denotes the focal length of the lens and s u denotes the pixel

Distance and bearing measurement using a camera unit
Fig . 5 illustrates the distance measurement from the camera unit to the designated point on the target. In Fig. 5a, the x-mark denotes the intersection of the target object and another measurement camera's optical axis. The homography between the two measurement cameras is given in the same manner as the tracking of the designated point. The projected point of the intersection in a camera image given by the homography. The angle c x between the optical axis and intersection is calculated from (4), by using the intersection instead of the designated point. The distance δ is calculated from the angle c x using the following equation where L C denotes the baseline length between two measurement cameras in Fig. 5a. As shown in Fig. 5b, the distance d between the camera unit and designated point is obtained using the following equation where δ L and δ R denote the distances of each measurement camera obtained from (5).
The bearing j to the designated point is determined from the angle c cx obtained by the tracking camera and the pan angle j p of the pan-tilter using the following equation With regard to the vertical angle in an image, the tilt of the camera unit is calculated in a manner similar to the calculation of the horizontal angle j.

Position and heading estimation
We assume that the length between the camera units L U is defined in advance, during the installation of camera units. The heading angle θ for the vessel is also given as  The reference point (x 0 , y 0 ) is determined as follows In the proposed system, the midpoint between the two designated points is considered the reference point for the vessel's position. The installed positions of the camera units (x CL , y CL ) and (x CR , y CR ) are determined as follows Then, the vessel's position (x, y) is determined with the following equation 3 Performance evaluation of the proposed system To evaluate the measurement performance of the proposed system, we measured the positions and headings of a model vessel by using a prototype system in water. Fig. 7 depicts the experimental setup. The model vessel was 2.0 m long and 1.0 mm wide. The draft (distance between base of the model vessel to water surface) was ∼150 mm. Table 1 shows the configuration parameters for the prototype system constructed for the experiment. The camera units were mounted 850 mm above the base of the model vessel. The baseline length between the camera units was 1316 mm. The camera units each had a 300-mm baseline length. The measurement cameras and tracking camera had 16-mm and 8-mm focal lengths, respectively. All cameras had a 7.5-μm pixel size and 640 × 480 image resolution. At the start of the measurement, we designated the points A and B on the wall to each tracking camera as shown in Fig. 7. The distance between the vessel and the designated points was ∼7 m. To obtain the actual motion of the vessel, we attached a chessboard pattern and IMU (inertial measurement unit, Xsens MTi-10) on the vessel. Here, we obtained the 6 degree-of-freedom motion of the chess board by using a technique for the extrinsic camera parameter estimation [16] with a camera installed on the ceiling above the pool. We assume that the motion of the chessboard is actual motion of the model vessel.
In the experiment, we measured the positions and headings of the vessel under three conditions, wherein the vessel was given x-translation (surging), y-translation (swaying), and yawing, respectively. Furthermore, we tested its performance both in the presence and absence of the vessel's rolling, to evaluate the ability of the proposed system to provide stable measurements in rough water. The amplitude for the roll, which was measured by the IMU, was approximately ±2.7°when the vessel surged, approximately ±2.5°when the vessel swayed, and approximately ±2.2°when the vessel yawed. The period of roll motion was 1.     Table 2 shows the average error, absolute maximum error, and standard deviation of the relative positions and headings for each condition. The average errors and standard deviation of the x and y positions are less than 0.1 m, which can be assumed sufficient accuracy for relative poisoning, despite the roll motion of the vessel. Similarly, the average errors and standard deviations under each condition are almost the same, irrespective of the presence of roll motion. Although the headings should be measured with 0.1°a ccuracy for relative vessel positioning, note that the distance between the prototype's camera units was only 1.3 m. Indeed, this distance would be several magnitudes longer on an actual vessel. This means that the accuracy of headings θ would improve drastically on an actual vessel. In other words, we can assume that the proposed system would satisfy the sufficient-accuracy requirements of heading measurements on an actual vessel. These results will be inaccurate for more distant targets. However, the measurement accuracy can be satisfied for more distant targets by using a longer baseline between measurement cameras. Thus, the experimental results demonstrate that the proposed system is sufficiently accurate for vessel positioning at close distances, even in waves.

Conclusion
In this paper, we proposed a vision-based measurement system for relative vessel positioning without the installation of transponders, reflectors, or markers on a target. The position and heading of a vessel are measured using distances and directional angles between the designated points on the target and the camera units installed on the side of the vessel. The camera unit has the ability to track the designated point using a GPS-based feature point detection and matching algorithm. The distance to the designated point is estimated based on the homography obtained from the feature point correspondences between two measurement cameras. We evaluated the measurement performance of the proposed system given positional displacement and rotation under the conditions of the presence and the absence of roll motion of the vessel in water.
The results of this experiment demonstrated that the proposed system can keep track of the designated points and continuously measure the relative positions and headings of a vessel. The