Development and Application of Gesture Recognition System for Intelligent Robot

Based on the current technology of gesture recognition is difficult, high cost and applicability is not strong, this paper designed a kind of intelligent robot based on gesture recognition system, the system possesses the advantages of convenient operation, simple and effective. The gesture recognition robot consists of three parts: gesture recognition system, intelligent robot and Android APP. The gesture recognition system has three independent recognition modules, and the algorithm based on feature extraction and template matching can be used to achieve different gesture recognition control robot motion effect. The intelligent robot is adjusted by PWM duty cycle and has functions such as Bluetooth communication and WIFI communication. Android APP can switch between gesture recognition mode and automatic mode, and the relatively beautiful interface is conducive to user operation. The experimental results show that the average recognition rate of the gesture recognition system is 94.6% and the accuracy is high.


Introduction
With the rapid development of science and technology, computers and other devices have entered thousands of households in various forms, and people are attaching more and more importance to human-computer interaction experience. Gesture recognition as a new interactive technology enables the interaction between humans and machines to become more natural, convenient and efficient. In the aspect of smart home, you can directly control the light switch, curtain push-pull, switch air conditioning fan, etc. At the same time, it can be controlled by hand gesture in the bathroom with high humidity to avoid electric shock hazard of wet hand touch electrical switch. Gesture recognition system has strong user demand and good market prospect.
Gesture detection collection is the key to gesture recognition, and more representative gesture detection methods include: Kinect gesture detection, feature extraction and skin color detection [1]. Kinect detection includes gesture recognition and tracking function, with good performance. However, Kinect device is large in size, and its application in smart home and other devices is limited by space [2][3]. Skin color detection is greatly affected by light intensity, and intelligent robot requires that hardware equipment is not sensitive to light intensity [4][5]. Feature extraction includes static and dynamic features, and static features such as extracting gray image binarization of gestures are  [6]. It is relatively difficult to extract dynamic features during the simultaneous movement of hand and robots [7][8].
To sum up, the existing gesture recognition equipment on the market is large in size, high in cost price, complex in extracting gesture algorithm, and weak in application. (1)The gesture module studied in this paper is small in size, costs within 100 yuan, easy to carry and embedded in various devices, and can be used for development and application in various practical fields such as smart home and archaeological exploration. (2) In this paper, a effective gesture recognition system is proposed, and dynamic gesture recognition is realized by using the template matching algorithm of static gesture flexibly. By establishing the optimal feature extraction algorithm of space rectangular coordinate system, the complex problem can be simplified to reduce the interference of environment on the collected data. The failure of the gesture system will not affect the overall system, and the hardware and software platform can be transplanted easily according to the need.

System design
The gesture recognition robot in this paper is divided into three parts: gesture recognition system, Android APP and intelligent robot. Figure 1 shows the system structure.

Gesture recognition
The gesture system is divided into three independent modules. The three all can control the robot independently and realize the recognition of different gestures effectively. Modules include "wave" mode, "virtual button" mode and "glove" mode. With different processing modules as the main body, the detailed information of the gesture is obtained through the recognition algorithm, which can be judged according to the characteristics, and finally the type of the input gesture is obtained Transmission part: the type of gesture is conveyed to STM8 homemade circuit board. According to the different gesture types, the corresponding control signal is sent to the intelligent robot via Bluetooth to realize the control.
2.1.1. Feature extraction algorithm. The extraction of gesture features is closely related to the gesture model. Different gesture models will have different gesture features. For example, the model-based gesture model has the state characteristics of each joint of the hand [9].
The "glove" module in this paper adopts the algorithm based on feature extraction to extract the coordinate values of gestures in the reference coordinate system, and takes the glove acquisition module as O point to establish the space rectangular coordinate system as shown in the figure. After the data extracted by the acquisition module is collected, four points are selected for coordinate processing. The plane formed by four points is compared with the x-y plane of the reference 3 coordinate system to calculate the size and direction of the Angle between two planes. When the Angle meets a certain trigger range, the gesture module sends instructions to control the intelligent robot.

Template matching algorithm.
Template matching is a common method of digital image processing. The imaging of the same sensor under different conditions is compared and corresponding patterns are found according to the acquired images [10]. In this paper, the "wave" module USES the template matching algorithm to match the matrix level signal characteristics with the template. Acquisition part of the image as a matrix of a certain number of points, each point including infrared emission and receiving tube, tube infrared emitting a certain frequency, when detected gestures, infrared reflection is receiving tube, after the comparator circuit processing, the green light is lit up, at the same time signal output digital signal (low level). The limited infrared tubes are combined together, and the characteristics of gestures can be judged by the level signal of the matrix. Time 2: high level of the matrix on the right and low level of the matrix on the left; Time 3: matrix restore high level. The collected gesture features are then matched with the template to perform corresponding operations.  After a certain amount of test research, the method of adjusting the slant accuracy was simplified, and the parameters were modified directly in the source program, and the glove scheme modified the four data sets. The initial data group defaults to: array 1: X-axis -100~100, Y-axis -50~50, and Z axis 150~230; Array 2: -200~-100, Y-axis -50~50, z-axis 150~230... Users can adjust the Angle of tilt and trigger distance according to their own will.
2.1.4. "Virtual button" accuracy. The "virtual button" module: the ultrasonic transmitter emits the ultrasonic wave. When it senses that there is an object in front of it blocking, the sound wave will rebound onto the ultrasonic receiver. The ultrasonic receiver means that it receives the signal and 4 transmits the signal to STM8. Therefore, this mode trigger only needs to adjust the ultrasonic trigger distance, and it is more convenient to adjust, only one distance trigger parameter can be changed! GPOIinit(); distance8 = pulseIn(EchoPin8, HIGH) / 58.0; val=digitalRead(Sensor);//read the data if(distance1<20) {Flag_forward = ON; gpio_set(forward,1); delay(50);} Figure 4. "Virtual button" mode distance parameter.

Intelligent robot
The intelligent robot is built through the STM8 hardware platform, and the motor speed is adjusted by PWM duty ratio to realize the movement of the robot in multiple directions, and the receiving terminal is realized through a Bluetooth module. Intelligent robot has basic motor drive module and Bluetooth module. Motor drive module is the key of robot motion. Input control signal CT: 12V square wave PWM signal, 25Hz (10% error), the duty ratio range is 0-100%, and multiple directions of robot motion can be realized through the duty ratio of PWM. The Bluetooth module is a bridge for communication with the gesture recognition system. By using SSCOM software for serial debugging, the Bluetooth of the gesture system will be automatically matched and connected with the Bluetooth of intelligent robot when the power is connected, so as to facilitate the control of signal transmission. At the same time, the intelligent robot has four-way infrared obstacle avoidance module, tracking module, infrared remote control module, temperature and humidity detection module, ultrasonic distance measurement module and WIFI module, which can realize multiple functions such as obstacle avoidance, tracking, distance measurement and wireless remote control [11].

Hardware structure
The hardware of the gesture recognition system mainly includes: PAJ7620U2 sensor, ultrasonic sensor, three-axis gyroscope, self-made circuit board STM8S207 and Bluetooth hc-05 module; the intelligent robot hardware mainly includes: motor drive module, STM8AF6223 homemade circuit board and Bluetooth module.
Firstly, after the user gives the gesture, the recognition module gets the hand information. Secondly, the hand information is analyzed by the recognition algorithm, and corresponding commands are sent to the intelligent robot according to the analytical results. Finally, after receiving the instruction, the robot executes the corresponding action.

Android APP
The software is built through the Android platform, mainly including the interface: start page, login page, mode selection (main interface) and intelligent connection, and the left-right sliding switchable interface. The main role of the APP is to switch the functions of intelligent robots, so as to minimize the impact on the whole system after the failure of some functions of the robots [12].

Gesture recognition effect
"Glove" mode: module can identify the palm tilt, the characteristics of the palm Angle reaches a certain scope, sending control signals, according to the tilt direction of the palm determine the movement direction of the robot.
"Virtual button" mode: the hand is suspended above an ultrasonic sensor, eight sensors correspond to eight motion directions, and the robot motion direction is determined by the "button" selected by the hand. The infrared obstacle avoidance sensor is the stop key. 5 "Wave" mode is sensed in a three-dimensional position, while the hand moves the air above the sensor, the robot moves in the direction of the hand sliding direction. Figure 5. "Glove" module,"virtual button" module,"wave" module.

APP interface display
The main purpose of the APP is to switch the function of intelligent robot, optimize the management of gesture recognition system, and also realize the APP to control the robot independently. In the intelligent connection interface, the mobile phone is paired with the robot via Bluetooth, and the robot function is switched in the main interface. A simple APP enables users to see how to operate at a glance. The main interface is shown below.

Compare with others
Algorithm: Depending on the complexity of the system, Algorithm of artificial neural network need to process information by adjusting the interconnection relationship between a large number of nodes. Depth image processing continuously samples N data from the image, averages the data, and obtains 6 the filtering results. The algorithm not only sampled more data, but also complex and hard to understand [13].
Price: Visual gesture recognition costs about 200 yuan for just one camera module. The Kinect device sells for about 200 yuan. In this paper, the "glove" module price about 95 yuan, the "virtual button" about 136 yuan, and the "wave" module about 123 yuan.
Volume: The Kinect is too large to get rid of the space limitation of the hardware device [2][3]. The "wave" module and "glove" module in this paper are small in size, easy to carry and can be worn directly on the hand.

Experimental result
In order to test the recognition accuracy of the gesture system, this paper conducts experiments on three kinds of gestures. The robot moves forward, backward, left, right, and stops five motion states to determine whether the gesture is successfully identified, and each gesture state operates 40 times. The experimental results show that the wave mode accuracy is 92%, the virtual button mode is up to 98%, the "glove" mode is about 94%, and the average accuracy of gesture recognition is 94.6%. The accuracy of APP Bluetooth control is about 96.5%, statistical results are as follows. The experimental data show that the wave pattern recognition error is even more, and the research shows that the infrared sensor inside the PAJ7620U2 sensor is more sensitive to the light intensity. The brightness is very important to the recognition effect. Experiments in areas where light is not strong can improve the recognition rate. Some of the gestures are misjudged and can be used "if" in the program to make a second gesture judgment to reduce misjudgment.

Conclusion
In this paper, a new type of gesture recognition system is proposed. There are two difficulties in the study of dynamic gesture recognition: one is the spatial and temporal difference of gestures; the other is the automatic segmentation of continuous gestures. Through the establishment of the spatial 7 rectangular coordinate system optimization feature extraction algorithm, the complex problem is simplified, has a certain reference significance. The system absorbs the advantages of convenient operation of the gesture recognition system on the market. It not only has the basic function of intelligent robot, but also increases the function system of APP switching, which complies with the scientific research development direction of non-contact control equipment. It has the following market advantages: low hardware cost; excellent user experience; Wide application prospects. From the perspective of economic efficiency, the gesture recognition system in this paper increases the lower cost, but improves the larger market and profits. From the perspective of market demand, the gesture recognition system designed in this paper is more in line with the use demand of the public, without being limited to the touch button on the electronic screen, and the economic benefit is expected to be optimistic. From a long-term perspective, this design can be widely used in a variety of electronic devices to provide a solution to the current problem of easy control.