Abstract
This study implements Fuzzy logic-based obstacle avoidance and human tracking on an omnidirectional mobile system for service robots. The mobile system could be separated and combined with the robot which can be controlled remotely and switched to go forward and avoid obstacles in an indoor environment automatically. The system is able to track and go to the user according to the user’s position. The omnidirectional wheel was adapted in the power system to perform translating and spinning movements. The translating movement enables the robot to avoid obstacles faster and flexibly in paths. With the spinning movement, the robot can quickly find the direction of the object. Finally, the experiments show that the proposed system has good performance in service environments.
1. Introduction
The mechanical construction and artificial intelligence of robots have developed rapidly with the advancement of science and technology in the past decades [1]. Robots are being increasingly applied, not only in the fields of traditional industry, agriculture, and aquaculture but also in many environments such as medicine [2], medical equipment [3], education [4], home [5], and entertainment [6] where they had been widely used; robots replace manpower to save labor costs for enterprises. Factory automation robots such as handling robots and assembling robots can accelerate productivity and improve product yield. Demolition robots and exploration robots can replace people in performing tasks in hazardous environments, reducing risks such as monitoring gas poisoning or gas leakage. Additionally, people have rising expectations for quality of life. The development of service robots decreases the workload of service personnel, allowing service personnel to do more valuable things, thereby improving the service quality. Therefore, intelligent service robots have become an important development trend.
This study extends the previous research by Chen et al. [7]. The proposed system is equipped with a multifunctional human-machine interactive multimedia interface so that users can interact with the robot, and the robot can also guide users to purchase goods. The robot can adopt Kinect to track the user and follow or come to the user through the fuzzy logic control of the motor. The robot body moves the base and is separable, enabling remote control of the mobile system and avoiding obstacles along paths. The system can be controlled when it is not detached remotely. Finally, some experiments to test the design are designed. The success rate is computed in each case after 30 test runs per experiment. The outline of this study is as follows. Section 2 reviews the recent relevant work and limits the scope of this paper. Section 3 describes the system overview. Section 4 introduces the proposed composite solution. Section 5 presents the simulation of this solution. Section 6 draws conclusions and describes future work.
2. Related Work
In recent years, more and more scholars have researched mobile robots and have shown many achievements in the field, such as path planning [8, 9], obstacle avoidance system [10], navigation system [11], and tracking. As research into mobile robots is extensive, this study mainly focuses on exploring and implementing target tracking, obstacle avoidance, and fuzzy control for the proposed mobile robot.
2.1. Robot Movement
Robots can be classified by movement mode into wheel robots and foot robots. Foot robots move at slower speeds than wheel robots but can handle more complex terrains. The most common foot robots are hexapod [12], quadruped, and biped robots [5]. Wheeled robots change direction and move faster, suitable for moving in flat terrain. Chen et al. [13] and Huang and Chiang [14] adopted wheeled robots as research projects. This study adopts a wheeled robot as the base of the proposed robot system.
2.2. Target Tracking
This section reviews the relevant research of mobile platforms and mobile robots that are capable of target tracking. Li et al. [15] applied 3D cameras to estimate the target distance and establish the relationship between the control of input and the depth image, thus constructing a human tracking algorithm on a mobile robot. Zhang et al. [16] used utilized matching to track the target. The system locks the target according to the clothing color. Benli et al. [17] examined target tracking for intelligent perception in unmanned systems and focused on perspective visual and infrared bands, as well as O-D visual band tracking to approach target tracking. Feng et al. [18] proposed a target person tracking model based on ultra-wideband (UWB) technology for human tracking robots and overcome the challenge of the measurement errors with a modified hyperbolic positioning algorithm. Williams and Sukhatme [19] demonstrated a metric of observability to improve system localization. Sun et al. [20] presented a 3-DOF pedestrian trajectory prediction approach for autonomous mobile robots.
2.3. Obstacle Avoidance
The main aim of obstacle avoidance is to measure the distance from obstacles, determine the location of obstacles, and identify the shape of different obstacles. Owing to the rise in computing speed in recent years, many visual sensing methods for obstacle avoidance have been presented. Al Arabi et al. [21] proposed a method for navigation and path planning to avoid obstacles. The robot generates a 2D map using range measurement sensors and obstacle detection through radar. Mvemba et al. [22] utilized a sensor mounted on the front of a mobile robot. The servo scans and detects obstacles within the interval by estimating the distance of detected obstacles. Ziaei et al. [23] developed a global path planning method using a single overhead camera based on the principle of artificial potential fields. Chen et al. [13] presented a path tracking method for wheeled mobile robots. Their method includes path planning and controller design. The path planning employs a B-spline to generate an obstacle avoidance path, thus decreasing statistically the possibility of collision.
3. System Overview
This section introduces the design concept of robots and the important components.
3.1. Robot Mechanism Architecture
The robot is composed of DC motors, servo motors, aluminum members, plastic parts, and a control board and is controlled by a Servo Commander 16 control board. The movement method adopts four DC motors with omnidirectional wheels, which can be rotated in the same direction or moved in all directions. The joint utilizes five 25 kg servo motors and six 10.8 kg servo motors, with an Intel mini host and 10-inch IPS screen. The power supply applies a 12 V 7.2 Ah lead-acid battery for the motor and module, while the screen, Intel mini host, Kinect, and speaker power supply are powered with a UPS uninterruptible power system. The head, hands, and waist have 11 degrees of freedom. The height is about 112 cm. The system can be moved by remote control using a handheld device, and the robot can be set to move autonomously as shown in Figure 1.
3.2. System Architecture
The proposed system comprises a basic commander, RGB-D camera (Kinect), DC motors, ultrasonic sensors, and a mini PC as shown in Figure 2. The RGB-D camera transmits target tracking information while other modules such as the motor controller and sensors are connected to the basic commander.
3.3. Mobile Platform
The omnidirectional mobile platform is integrated with four omnidirectional wheels. Generally, the turning process causes cumulative deviation, causing severe body displacement after several turnings. Omnidirectional wheels enable the platform to move horizontally along the path without constant turning, thus solving the problem of error accumulation. The platform’s size is 480 mm × 480 mm and the four omnidirectional wheels are 102 mm diameter and width 35 mm width. The rollers are shaped such that the overall side profile of the wheel is circular, so as to ensure the consistency and stability of the wheel in contact with the ground. The partial force of the wheel can be transformed into the perpendicular direction by rollers. The torque in an arbitrary direction is synthesized with the cooperation of speed and direction of the wheels. Figure 3 shows the omnidirectional mobile platform.
3.4. Depth Camera
Depth data extraction and color image data acquisition are similar operations, both requiring an understanding of depth data which is still necessary. Infrared light is emitted, and with the detection of infrared light reflection, the depth value of every pixel in the visual view can be calculated. In the image frame provided by the depth data stream, each pixel represents the closest distance from the object to the plane of the camera. The maximum value of the depth data is 1220 mm to 3810 mm. The skeleton tracking scheme builds the joints of the human body, which can then identify and locate different body parts, such as the hand, head, and body. When the object walks into Kinect’s visual field, Kinect v1 as shown in Figure 4 can find the locations of the 20 joint points and represent the positions in coordinates. Unlike the depth image spatial coordinates, these coordinate units are meters. In this way, when objects perform many complex actions in front of Kinect, the displacement between the locations of these joints is very obvious, helping the computer to understand what action objects are performing.
3.5. Ultrasonic Sensor
The system adopts a Sonar A module to receive user instructions and measures distance based on user demands under proper environmental conditions. The detection distance is 5 meters. Eight ultrasonic waves were set up on the mobile platform, at positions as shown in Figure 5.
4. Proposed Methods
4.1. Fuzzy-Based Obstacle Avoidance
This study proposed a new anomaly detection method of ultrasonic obstacle avoidance, which can solve the problem of the ultrasonic sensor failing to detect the distance between the obstacles due to its shape and material. Therefore, the method can effectively improve the anomaly detection ability of the ultrasonic sensor when facing extreme conditions. The method provides stable ultrasonic input values to the fuzzy controller, and with the stable input range, the method simplifies the design of the initial member function and fuzzy logic rules and therefore reduces the difficulty and the time spent in subsequent optimization.
In addition, the method has low computational complexity, enabling the entire system to be operated in real time and to integrate more flexibly with other sensing systems. In the application of the actual target tracking, the method makes the translate obstacle avoidance of mobile platforms safer and thus more practical. Therefore, the mobile platform can avoid obstacles to target tracking without turning, decreasing the risk of losing the target when avoiding obstacles. The method can enhance the ability of anomaly sensing and thus avoid the obstacle safely even if it is applied in a steering-oriented obstacle avoidance strategy. Because the ultrasonic distance measurement is triggered and calculated with the reflected wave, whether the reflected wave can truly show the presence and the distance from the obstacle is a critical issue. The distribution of the ultrasonic emission can be simulated as fan-shaped, while the ultrasonic receiver is triggered to derive the distance only if it receives enough energy. Therefore, if the obstacle cannot reflect the ultrasonic wave due to its shape, placement, or material, then the obstacle detection system will measure the distance inaccurately or even fail to detect the obstacle. The new method adopts the ground as a flat plane to reflect the ultrasonic wave stably, with a fixed difference between the actual distance and the measurement value. Several distance ranges of stable reflection zones, called the critical boundary, were chosen and tested through different types of obstacles, to see which critical boundary can determine more abnormal conditions with clear rules. The final critical boundary was chosen after multiple experiments as a fixed benchmark of anomaly detection. Figure 6 shows the side view of the critical boundary formed by the ultrasonic wave.
If the obstacle touches the critical zone, then the value of the measurement from the reflected wave of the critical boundary changes, leading to an abnormal state. This abnormal state indicates that the obstacle is too close to the mobile platform in a specific direction, and therefore, the mobile platform must move accordingly in order to turn the abnormal state back to normal, maintaining a clear critical boundary to ensure safety. In general, if the invaded obstacle is a plane with little or no incline, then the distance measured by the ultrasonic sensor falls, such that even the conventional obstacle detection method knows that an obstacle is coming close. However, if the obstacle material absorbs ultrasound, or if its shape is cylindrical or with an inclined plane, then the value of the ultrasonic sensor measurement will be unstable or simply shows a far distance, since the reflected wave is received at different times with different energy levels or is absorbed by the obstacle. In these cases, the conventional obstacle detection method and the avoidance rules are ineffective. In the new method, even if the obstacle is cylindrical, with an inclined plane or made of a material that absorbs ultrasonic waves, it still blocks the ultrasonic reflection of the critical boundary from the ground, making the value no longer limited to a stable range. Therefore, these cases can be identified as the obstacle entering the critical boundary in a particular direction and enabling the mobile platform to take the corresponding avoidance strategy. Figure 7 illustrates the top view of the critical boundary formed by multiple ultrasonic sensors. The proposed method has much better security of a translate obstacle avoidance strategy for target tracking.
The traditional translate obstacle avoidance strategy can avoid target loss by keeping the target inside the sight view of Kinect, when the target is not too large. However, the traditional strategy can only rely on the left or right front of the ultrasonic sensor to determine whether it can move forward safely. Unfortunately, experimental results indicate that the left front or the right front of the ultrasonic sensor, when encountering an inclined plane or cylindrical obstacle, often fails to find the obstacle or measures the distance from the obstacle inaccurately, causing the sensor to hit the obstacle on the side when moving forward. Using the new detection method, the ultrasonic sensors can detect anomalies earlier and try to clean the critical boundary by translating in the opposite direction. The new method can improve the ability to sense an obstacle, whether it is applied to the translate avoidance or steering-oriented avoidance strategy. Figure 8 illustrates the comparison results of the conventional and new methods when using a translate obstacle avoidance strategy.
The inference needs to find the mass center by integrating continuously changing functions, making the calculation process inefficient. To achieve real-time operation, this study adopts the Sugeno-style inference method.
The Sugeno-style inference scheme uses only one spike of the single state function as the latter membership function of the fuzzy rule, which takes the following form:
All latter terms of membership functions are represented by a single state spike. The final step is to adopt the center of gravity method to calculate a clear output. The fuzzy rule adopted herein is the zero-order Sugeno fuzzy model, which is also commonly used in most systems and has this fuzzy rule: where is a constant. The output of each fuzzy rule is a constant, and all latter terms of membership functions are represented by a single state spike. The final step is to adopt the center of gravity method to compute a clear output:
Figures 9 and 10 show the corresponding input and output membership functions of our fuzzy logic control system: (A)The relative distance from the target to the Kinect is adopted to decide whether to stop, spin, or follow the target:(1)Stop. 0-150 cm(2)Spin to Find the Target. 150-250 cm(3)Follow the Target. 250-450 cm (B)The relative absolute angle deviation from the target to the Kinect: A1: 0-13 degrees (small-angle deviation) A2: 10-20 degrees (medium-angle deviation) A3: 17-30 degrees (huge-angle deviation)(C)Four-motor rotation output: B1: 0% power (stop following or spinning) B2: 50% power (slow following or spinning) B3: 80% power (fast following or spinning)(D)Fuzzy rules: Rule 1: . If the target angle deviation is small, stop turning or spinning. Rule 2: . If the target angle deviation is medium, slow turning or spinning. Rule 3: . If the target angle deviation is huge, fast turning or spinning.
4.2. Target User Tracking
The accurate position of a specific target user, whether from the front or the back of the mobile system, is computed by adopting the skeleton extraction function to calculate the distance and angle between the target users. The extracted skeleton data contain the following information: (1)The user tracking state(2)The user tracking ID
The fuzzy logic control system needs to receive several commands from the tracking subsystem, including the start commands of each of two functions and a cancel command for both. Generally, the user can give commands to the robot by speech, but voice recognition may also fail. Additionally, if the user forgets the commands, it causes annoying user experiences. In order to solve the problem, this study adapted gesture recognition. This is particularly suitable for our system. Figures 11 and 12 show the three chosen specific gestures.
4.3. Path Planning
The robot path planning problem is one of the most important research issues at this time. The requirements of the robot’s path include avoiding the obstacles and shrinking the moving distance. The global map which contains all obstacle information is needed in some approaches, such as [24]. However, it is not practical in some situations in which obstacles may move or change directions. In this paper, we applied the elastic net-based optimizing policy on the path which originated from obstacle avoidance. The elastic net is firstly proposed for the traveling salesman problem in [25]. It is an effective method for optimizing the path. The basic idea of the elastic net is an elastic ring with several points. Each point can be attracted by the neighbors and the nearest city (in the traveling salesman problem). In our approach, the elastic net is simplified to an elastic line from the start point to the stop point, like in Figure 13. Each point can be attracted by the neighbors and repulsed by the nearest obstacle. Figure 14 shows the forces on a point of the elastic net. The point is attracted by N1 and N2, and the attractive force is proportional to the distance. The attractive forces have a tendency to shrink the length of the robot’s path. The point is repulsed by the nearest obstacle O1. The repulsive force is inversely proportional to the square of the distance. The repulsive force makes the path to be distant from the obstacle.
5. Experiment Results
5.1. Experiment Environment
Several experiments were designed to test the proposed system. These experiments comprise obstacle avoidance in three environments, when the mobile platform was separated from the robot, following the user, and coming to the user. The success rate of each case was calculated after each experiment 10 times.
5.1.1. Obstacle Avoidance Environment
Figures 15–17 show the three spaces in the simulated home environment. The experiments were performed in a real home environment with a living room, kitchen, and bedroom. The obstacles in the living room were a sofa, coffee table, and TV cabinet. The obstacles in the kitchen were a refrigerator and stove cabinet. The bedroom obstacles were a bed and bedside cabinet.
5.1.2. Obstacle Types
In order to verify the effectiveness of robot obstacle avoidance, the experiments were designed with a variety of obstacles. Irregular shaped obstacles included the coffee table, sofa, and electric fan in the living room. Some obstacles were shaped like cylinders. Figures 18–21 show the different obstacles in each space.
5.2. Experimental Robot Mechanism
The robot was constructed according to the design. Figure 22 shows the main body, which is an aluminum alloy frame including a turntable with a height of 110–120 cm. The total weight is about 40 kg. The robot can carry an additional 5 kg and can be moved by the remote control of the handheld device. The arm part utilizes a connecting rod structure to increase the load-carrying capacity of the motor. The movement method adopts four DC motors with omnidirectional wheels of size and a 6 mm hub adapter shaft, which can be rotated in situ or moved in all directions. The joint adopts five 25 kg servo motors and six 10.8 kg servo motors.
5.3. User Tracking Results
The experiments were first performed in the hall. The robot began to follow the user after it recognizes the gesture. In this experiment, the user turned at a corner to test the tracking performance of the robot. The “come to me” test was performed 30 times at different distances and failed 1 time at 10 meters, 5 times at 15 meters, and 10 times at 20 meters. The sensor could detect the user within 10 meters but failed increasingly beyond this distance. Table 1 lists the experimental results.
In the “follow me” experiment, the ability of the robot to follow has a good performance and can follow the distance of the subject by more than 5 meters. Table 2 lists the experimental results.
In the “follow me” experiment, the user gestured to the robot, and the robot began to follow the user, followed the user to turn, and continued to follow the user after turning. The experiment scenes are shown in Figures 23–26.
5.4. Obstacle Avoidance Results
The obstacle avoidance experiment was run with the living room, kitchen, and bedroom as the experimental areas. Table 3 lists the experimental results. The main test criterion is whether the robot can effectively avoid irregular shaped objects. Experimental results indicate that the robot avoided 100% of all irregular obstacles. The experimental record was made into a show film, in addition to the actual test video in Demo Link.
6. Conclusions
This study develops a separable omnidirectional mobile system for a service robot. This system performs better than other tracking methods. The gesture recognition enables users to make a robot track themselves. Similar to the “follow” function of a tour-guide robot, the “come to me” function and gesture recognition allow users to call the robot, a feature that is especially useful in helping someone who is seriously injured. The tracking function can lock the object and lock the target if it returns. The relock ability can be strengthened in the future, by locking the same target using depth images or based on the clothing color and body sizes. In terms of remote control and obstacle avoidance, the proposed robot can patrol manually or automatically. Future work would be to strengthen the patrolling ability by making the robot start from one place to another destination and return to the origin. Additionally, traction wheels cause difficulty for a mobile platform to synchronize its steering direction with the 30 kg robot, causing real-time steering of the whole robot to fail, even with accurate motor control and calibration. Replacing the traction wheels with omnidirectional wheels in the future may improve the steering ability and speed of the robot. Conversely, to reduce the friction caused by the robot, the traction wheels could be constructed with low-friction materials such as plastic in the future. Although the mobile platform needs more friction to drag the load, high-friction materials such as rubber can be replaced to reduce the friction on the current omnidirectional wheels.
Data Availability
All used data has been included in this article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.