Image-based Fuzzy Parking Control of a Car-like Mobile Robot

—This paper develops a novel automatic parking system using an image-based fuzzy controller, where in the reasoning the slope and intercept of the desired target line are used for the inputs, and the steering angle of the robot is generated for the output. The objective of this study is that a robot equipped with a camera detects a rectangular parking frame, which is drawn on the floor, based on image processing. The desired target line to be followed by the robot is generated by using Hough transform from a captured image. The fuzzy controller is designed according to experiments of skilled driver, and the fuzzy rules are tuned and the fuzzy membership functions are optimized by experimentally for output. The effectiveness of the proposed method is demonstrated through some experimental results with an actual mobile robot.


I. INTRODUCTION
In recent years, more and more intelligent technologies are being applied to automobiles in the automobile industry. One of them is the automatic parking technology. Although some automobile companies have recently developed an automatic parking system, issues on safety and cost still remains for commercialization. Most of these research have been concentrated on path planning and posture stabilization methods. The path planning approach plans a feasible geometry path in advance, taking into account the environmental model as well as the vehicle dynamics and constraints, and then control commands are generated to follow the reference path [1]. The posture stabilization method is to stabilize the robot to a desired final posture from any initial posture.
On the other hand, a skill-based approach that uses the fuzzy logic [2] mimics an experienced driver's parking skill. There is no reference path to be followed by a robot and the control command is generated by considering the position and orientation of the robot relative to the parking space. This approach requires an exact vehicle pose relative to the parking space. Daxwanger [3] bypassed the pose estimation by using a neural network that can directly map the video sensor's image of the environment to a corresponding steering angle. A sensor guided autonomous parallel parking approach is discussed in [4], where they used the ultrasonic sensors for parking. Ultrasonic sensors normally have the distance error to the angle of reflection as well as cannot obtain the distance in a parking lot without obstacles. However, all these research on parking control systems used the odometry data from the wheel encoders, which seems to be not really practical. For automatic parking, it is considered that image based localization will be a more practical choice.
Moreover, vision sensor and digital image processing technology become easily available with the development of computer technology in recent years. There are two major approaches for visual servoing in which the robot is controlled using a camera [5]. One is called "Position-based visual servoing (PBVS)," which estimates the posture (posture means both the position and the orientation of the robot) of the robot by matching previous knowledge of 3D environmental models with a captured image and controls it directly. The other is called "Image-based visual servoing (IBVS)," which does not need the knowledge of the 3D model of the environment and can operate a robot by controlling only image information which is acquired from a camera image without using the robot position. Compared to PBVS, IBVS can work with faster processing speed and higher robustness against camera calibration errors due to the exclusion of the position estimation process [6]. In this study, an automatic parking system of a car-like mobile robot is developed using an image-based control method. By image processing and computing, a rectangular parking space is detected from a captured image and the target line for automatic parking system is generated. An image-based fuzzy controller is evaluated through experiments for an actual mobile robot. This paper is organized as follows. In Section II, a problem setting is described. By an image processing technique with a Hough transformation, a rectangular parking frame is detected and a target line for an automatic parking system is generated in Section III. A fuzzy controller is designed to follow such a target line in Section IV. In Section V, the experimental results with three different initial positions are conducted to demonstrate the effectiveness of the proposed method. Section VI concludes this paper.

II. PROBLEM SETTING
The objective of this study is to build an automatic parking system of a car-like mobile robot using an image-based fuzzy controller, in which a robot equipped with a camera detects a rectangular parking frame drawn on the floor. In this study, the rectangular parking frame with two colors is used to easily separate the vertical and horizontal lines of the parking frame. coordinate be represented by x-y coordinate and the image coordinate for the camera be described by u-v coordinate. The following two equations represent the vertical lines of the parking frame on the image coordinates, A four-wheeled steered mobile robot is used as the controlled object as shown in Fig. 2. The robot state is denoted by the position and the orientation of the vehicle with respect to the x-axis such as (x, y, θ) and the control input is composed of the forward speed and the steering angle, (s, ϕ). In addition, the distance between the front and rear wheels is denoted by L and the camera is directed to the forward direction of running.  Figure 3 shows the block diagram of the proposed method, where the controller determines the control inputs s and ϕ from two states t  and t  , which are extracted from a captured image by using Hough transform.

III. GENERATION OF A TARGET LINE USING IMAGE PROCESSING
Nowadays image processing is widely used for detection in intelligent automobiles because it is one of the quick and accurate methods for detection of parking place. To detect the rectangular parking frame and generate the target line of an automatic parking system, some steps of image processing on a captured image are discussed in this section. The summarization of image processing which is used in this research is illustrated in Fig. 3.

A. Thresholding of the Parking Space
At first, some threshold functions are used to extract red color and remove a noise. A canny edge detector is used to obtain the edge of the binary image. The original image and the preprocessed image of the rectangular parking frame are shown in Fig. 5(a) and (b). The edged image of the parking frame is shown in Fig. 6(a).

B. Extraction of Parking Lines Using Hough Transform
Hough transform is one of the most popular methods for the detection of lines in the binary image. In our research, the parking lines are extracted and the slopes and intercepts of these lines are calculated by applying Hough transform as shown in Fig. 6(b). By averaging the slopes and intercepts of those lines, the vertical center line of the parking frame is estimated. Next step is to find the mid-point of such a line and construct the target line for our parking system depending on the point, where the resultant images are shown in Fig. 7 In this study, a fuzzy controller is designed by using so called min-max centroid method for navigating a car to a parking lot. According to some image features from a captured image, the proper steering angle has been determined to locate the robot in appropriate parking position. The fuzzy controller determines the control inputs through mainly three steps: fuzzification of state variables, calculation of grade of each rule and defuzzification of input values.
The state variables to be used as inputs for reasoning are the slope and intercept of the target line. Figure 8 shows the membership function of each state variable. The value of each state variable is fuzzified to linguistic representations (N: negative, ZO: zero, P: positive) with grades which are given these membership functions. Table 1 shows the fuzzy control rules for the steering angle ϕ. For the control rules, the desired control inputs are set in the form of if-then rule manually depending on the use of fuzzified state variables, where s is assigned to be a constant. For example, the interpretation of these rules is such that the rule no.  The control inputs on the fuzzy sets are transformed into real numbers through a defuzzification. The defuzzification is the calculation of a weighted mean by a simplified fuzzy reasoning method with singletons as follows: where , N h  denotes the grade of ϕ = N.
V. REAL-TIME FUZZY PARKING CONTROL EXPERIMENTS A real system experiment was conducted to evaluate the image-based fuzzy parking control approach here. The size of the parking space is 390 mm in length and 210 mm in width. The experimental setup of the parking system shown in Fig. 8 is composed of a laptop PC, a microcontroller, a car-like mobile robot and a USB camera. The robot was made by  In this experiment, the designed controller was tested with three different initial positions as shown in Fig. 10. The experimental results for our automatic parking system are shown in Figs. 11, 12 and 13, respectively.
According to the experimental results, the present controller was able to be park the vehicle properly only for case 2, because the fuzzy rules described in section IV are very simple. From this point of view, it may need to use more refined fuzzy rules for cases 1 and 3, if a much better control result is desirable for such cases.

VI. CONCLUSIONS
In this paper, a novel automated parking system of car-like mobile robots utilizing an image-based fuzzy controller has been presented. The experimental results show the usability of the designed controller. To obtain a better result than the current case, a more refined fuzzy rule will be introduced as future work.