Human Motion Gesture Recognition Based on Computer Vision

,


Introduction
Human body gesture recognition is an important research direction of computer vision [1,2]. Its ultimate purpose is to output the structural parameters of the person's overall or partial limbs, such as the outline of the human body, the position and orientation of the head, and the position of the human joint points or the category of parts. It should be said that the research methods of gesture recognition cover almost all theories and technologies in the field of computer vision, such as pattern recognition, machine learning, artificial intelligence, image graphics, and statistics [3,4].
So far, many identification methods have been proposed. Dong et al. [5] proposed to learn the corresponding target contour model from the segmented image and then used the boost classifier to find the contour of the target in the image so as to obtain the position information of each part of the human body. e literature [6,7] uses the HOG method to extract the information of each part of the human body in the image and then uses the classical algorithm support vector machine and random forest to identify and classify. Cui et al. [8] found the global optimal features from many features such as Fourier descriptors, shape context, edges, and gradients to quickly and accurately complete the backprojection process from features to three-dimensional poses. Zhang and Lu [9] used the histogram of gradient directions to restore the human pose and trained multiple local linear regressions to restore the human pose in a single frame of image. Li et al. [10] used the canny operator to extract edge features from the image in combination with pixel depth information and determined the head position of the person in the image through distance transformation and model matching and positioned the human body according to the prior human body proportion. Zhang et al. [11] used the iterated closest point (ICP) method to track the initialized human skeleton to achieve the purpose of human pose estimation. e literature [12] uses Markov random field (MRF) to segment the point cloud data containing the human body into four parts: head, torso, limbs, and background and recognizes the body's posture by part detection.
Nadeem et al. [13] first constructed a 3D grid model of the human body to find points of interest related to the measurement extreme value from the grid and then used these points to detect the head, hands, and feet of the human body.
In the past few years, research on human behaviour recognition based on depth images has received widespread attention [14][15][16][17]. Compared with ordinary optical images, because depth image pixels' record distance information and are color-independent, this overcomes the abovementioned problems encountered by ordinary optical images to a certain extent. With the development of optoelectronic technology, many researchers have combined the nature of depth images and applied many classic algorithms to such images. Alshawabkeh [18] combined pixel depth information and used canny operator to extract edge features from the image, through the distance transformation and model matching method, to determine the head position of the person in the image and locate the human body based on the prior human body proportion. Luo et al. [19] used computer graphics technology to construct a depth image database of human pose and used a classifier model to detect human body parts on a common PC. Jaffar et al. [20] used contour feature parameters combined with moments, directional gradient histogram, and human bone feature angles to perform multifeature fusion characterization of the depth information, color information, and bone information of the human body. Alzahrani et al. [21] comprehensively utilized the depth data and skeleton data provided by Kinect and effectively improved the real-time and robustness of gesture recognition through anthropometric knowledge and back propagation neural network. Ghazal et al. [22] used the Kinect camera to collect human bone information and 3D data corresponding to joint points for action recognition. Franco et al. [23] also used Kinect equipment to obtain depth information and designed a human posture recognition system specifically for sitting and standing postures. is paper proposes a generative adversarial network to solve the problem of large deformations of parts of the body, while also considering the complexity of different levels of body parts. e internal structure of the generator and discriminator proposed in the article has been optimized, and it can simulate the hierarchical relationship between body parts. Hierarchical perception terms are also introduced in the objective function to standardize the relationship between parents and children. Hierarchical adversarial networks help to accurately estimate the positions of various parts of the body, especially body parts that are deformed or highly occluded.

Human Motion Gesture Recognition
According to the Oxford Dictionary, the posture of the human body is a special posture of the body and the way a person maintains his physical state [24]. Human posture recognition is the extraction, classification, and identification of human posture features and natural language description. It is a research hotspot that has received wide attention in recent years. It is a research on the application of human physiology, digital image processing, pattern recognition, and other disciplines field [25].

Technical Classification.
From the perspective of the collection of human posture information, human posture recognition technology can be divided into two categories: contact recognition and noncontact recognition technology [26][27][28][29]. Contact recognition technology refers to a person's body wearing a sensor, through which the sensor collects parameter information such as acceleration and position of each part of the limb to realize the analysis and recognition of the posture. Noncontact recognition technology is usually based on vision-based human gesture recognition, that is, extracting and recognizing human features through video surveillance, which is a hot topic in computer vision.

Contact Recognition Technology.
Contact human body gesture recognition technology collects motion parameters through sensors worn on the human body and analyses the parameter information to realize gesture recognition. Contact recognition technology can accurately capture the changes in posture by allowing users to wear data collection equipment, and the posture recognition rate after computer analysis is quite high. However, in practical applications, it will cause inconvenience to users and does not meet the requirements of human-computer interaction. At the same time, such equipment is expensive and difficult to be popularized in real life.

Noncontact Recognition Technology.
Vision-based noncontact human posture recognition technology is the hotspot in the field of human-computer interaction. It acquires image information through video capture devices such as cameras, and after computer processing and analysis, specific representative features are extracted and classified to achieve posture recognition.

Algorithm Classification.
According to different classification standards, it can be divided into a variety of human body gesture recognition algorithms. From the perspective of implementation methods, it is usually divided into three types: (1) ree-dimensional model reconstruction method, which extracts three-dimensional features from valid samples to construct a three-dimensional model (2) e human body appearance model method, which establishes a two-dimensional model by acquiring the shape characteristics of the human body and uses the model matching method to complete the recognition (3) e motion model method is classified according to motion characteristics From the perspective of pattern recognition, gesture recognition is a problem of classifying time-varying features, that is, the process of matching the test sequence with the 2 Complexity present sequence according to the obtained feature information. From the perspective of matching methods, there are usually two types.

Template Matching Method.
e template matching method regards the human body motion posture as a combination of a series of discrete static images in a certain period of time. By extracting the static features of each discrete image from them and matching them with the trained posture model, the highest matching degree of motion posture is the best result.

State Space Method.
e state space method is to set each key static posture to a specific state. And, all dynamic postures are a traversal process that connects these specific states with the best probability relationship, and the best probability at this time is set as the classification standard of this posture. However, the state space method finds the global optimal solution through complex iterative operations, so it is difficult to apply it in practical work.

Improved Generative Adversarial
Network Algorithm Figure 1, the main steps of human posture recognition based on depth information are divided into three parts: first, we preprocess the image data collected by the depth information sensing device and then extract the corresponding human posture image features, which is the ROI area, according to the different targets and finally use appropriate the classification algorithm which performs posture classification and recognition. After the training and classification process, you can view the system's classification and recognition results of a new input image, select a picture from the test sample set, and input it into the network. According to the results of the softmax classifier, the label with the highest probability is the current recognition result. e final recognition result will appear in the following two situations: when the system predicted classification does not match the expected label, the classification error is displayed; when the test result matches the expected pose label, the classification is displayed correctly.

Deep Convolutional Hourglass
Network. e basic unit of the deep convolutional network is the residual network, as shown in Figure 2. It mainly includes three convolutional layers. e size of their convolution kernels is different. e size of the first layer is 1 × 1, and the size of the second layer is 3 × 3, and the size of the third layer is 1 × 1. Before passing through each convolutional layer, a Batch Normalization layer and a ReLU activation layer will be passed. e network extracts high-level features through the convolutional layer of the main path. e branch is a jump layer, which is composed of a convolutional layer with a core size of 1 × 1. e main function is to retain the features of the original layer, increase the nonlinearity of the model, and reduce the amount of calculation. In the residual network, only the number of channels of the image is changed, and the step size of all convolutional layers is 1. If the channel of the input image is M and the channel of the output image is N, the number of the first and second convolutional layer kernels of the main path is N/2 and the number of the third convolutional layer kernel is N.
Based on the residual network, a first-order hourglass network is constructed following its structure. Both the main loop and the branch include several residual networks. Among them, the branch is extracted at the original size through the jump layer, which can well retain the spatial information of each joint on the picture; the main road first uses the maximum pooling layer to change the size to half of the original size and then performs feature extraction.
rough this method, the network extracts the joint point features at different resolutions and finally restores the size to the original size through nearest neighbour interpolation and adds it to the branch output. Defining the feature of the image output through the first-order hourglass network as F1 (I (x)), the following equation is obtained: (1) Among them, C (x) is the output feature of the residual network.
By replacing the residual network inside the dotted line in the first-order hourglass network with the first-order hourglass network, a second-order hourglass network can be obtained. In the same way, by replacing it in the third-order hourglass network, you can get a fourth-order hourglass network. In the node detection network, a stacked four-stage hourglass network is used as the detection network. Among them, each branch is branched before passing the maximum pooling layer to preserve the spatial information between the joints on the picture. ree residual modules are used for feature extraction after each down-sampling. erefore, the 4-order hourglass network can extract the joint point features on the original size, 1/2, 1/4, and 1/8 size. After each feature extraction, the image is restored to its original size through up-sampling. After adding the original size features, a residual network is used for feature extraction. erefore, the network does not change the size of the image, only the number of feature channels. Defining the output feature of the image through the fourth-order hourglass network as F4 (I (x)), it can be briefly described as follows: 3.3. Improved Model. Since the detection of the original size of the image requires a lot of time, the pixels are first reduced to 64 × 64 through the convolutional layer and the pooling layer, and the residual network is used for feature extraction in the middle. e structure is as follows: (1) Turn the 256 × 256 pixel RGB image through 64 7 × 7 convolution kernels with a step size of 2 to 128 × 128 pixels, and the number of I channels becomes 64. Complexity (2) rough the residual network, the input channel is 64 and the output channel is 128.
(3) Carry out maximum pooling and further reduce the data pixel 128 × 128 to 64 × 64 without changing the number of channels.
(4) In the same way, three consecutive residual networks are used after down-sampling. e input and output channels of the first two networks are both 128, and the last output channel is 256.    4 Complexity e model proposed in this paper is DCGAN, which includes two parts: generator and discriminator. As shown in Figure 3, Stack 1, Stack 2, and Stack 3, respectively, represent different motion postures. In addition, the basic structure of the generator and discriminator is based on the hourglass network, which is an encoder and decoder network with jump connections and up-sampling layers, which can adapt to images of different sizes while capturing the diversity of human body poses. e input and output of the hourglass network are preprocessed and postprocessed using residuals and linear blocks to extract features and estimate the position of body parts to obtain a valuable confidence score map. e position of the joints of the limbs of the human body belongs to the articulated structure, which leads to obvious deformation of the limbs when performing movements, and in most cases, the degree of deformation of the joints on the trunk is relatively small. erefore, it is more challenging to estimate the posture of the parts of the human body with large deformation, and a large number of different training samples are required as support. In order to solve these problems, this research optimizes the structure of generator and discriminator networks. e new design uses stacked hourglass networks, which can be connected to build the spatial hierarchy of human body parts. In addition, this study also introduced a new loss function term to regulate the relationship between parents and children. e hierarchical adversarial network design and new hierarchical perceptual loss help to accurately estimate the position of various parts of the body, especially for those body parts that are highly deformed or highly occluded.

Hierarchical Generation Network.
e generator maps the input image to the confidence maps P and C of the parent and child, respectively. e mapping can be performed through the learning function as follows: en, connect the confidence map of the parent part and the child part to the output of each stack generator. e predicted pose can be inferred using the following equation: Estimate the predicted pose Z by summing the confidence maps of all sequential connection stacks and using softmax on the confidence scores of the connection maps. is allows the entire process to be trained end-toend: the generator network itself has no adversarial branch and can only be trained by minimizing the following loss function:

Hierarchical Discrimination Network.
Training the discriminator is a very important step because it is more inclined to reconstruct the discriminator from the generated pose than from the generated pose, which makes the discriminator unable to distinguish whether the generated pose is true or false. In order to solve this problem, this paper uses a balance strategy between reconstructing the real pose and generating the pose. Use the following loss function to train the discriminator: Loss D � min(α * HD + β * HR).
Among them, α and β are balance terms, and HR and HD are the loss terms of the real pose and the generated pose, respectively.

Experimental Data and Experimental Environment.
In this chapter, in order to verify the performance of the proposed network, three challenging human pose estimation data sets will be used to conduct experiments and get the corresponding results.
(1) e LSP data set and its extended version here contain 12,000 images, of which 11,000 images will be used for training and the other 1,000 will be used for testing. Under laboratory conditions, the hardware equipment needed for this paper are as follows: (1) One computer with win10 system and Linux system is needed. Due to the special requirements of the Kinect SDK, the computer needs to be a 64-bit system, the computer CPU is Intel (R) Core (TM) i7-2640M, the CPU frequency is 2.80 GHz, and the installed memory is 8.00 GB. e quality of the computer configuration will affect the subsequent classifier training and classification efficiency. e experiments in this paper are all carried out on this computer.
(2) Kinect for the windows camera is needed. e camera's parameters are shown in Table 1.

Network Training.
First, initialize the network and input each set of pose samples in the training set into the previously constructed CNN system and train one pose in one round. As shown in Figure 4, with the increase in training samples, the loss function value of the network continues to decrease, and the accuracy of verification continues to increase. And, when the training sample reaches 800, it starts to converge and tends to stabilize so as to determine the parameter value of the network. It can be seen that 1000 training samples can fully meet the training needs of the network.

LSP Data Set.
e result is shown in Figure 5. e percentage of correct key points for the model proposed in this paper can reach 94.2%. is result is better than the results in [21,23]. ese two methods are also considered to the method proposed in this paper and is the closest comparable method. Although the literature [22] and the literature [23] can estimate the structure information of the human body more accurately, the model proposed in this paper can estimate the pose on challenging body parts such as elbows, wrists, knees, and ankles.
is shows that the  Figure 3: DCGAN framework. proposed network can not only achieve the effectiveness of estimating human posture but also consider the hierarchical structure of various body parts. Figure 6 shows the PCKh results of this method and compares them with the results of the previous method. PCKh is the same as ordinary PCK, and the difference is that the tolerance is calculated based on the head instead of the torso. e model proposed in this paper is better than other results that use training these data sets for pose estimation. Similarly, the model in this paper focuses on the hierarchical relationship of the human body structure, which can be better detected in key parts that are prone to large deformations. e accuracy of the comparison between the wrist part and the method in [22] can be explained this point. Hybrid pose machine [30] and pyramid stream network [31] are generally considered to be classic methods for training on LIP data sets. is paper compares these two methods, and the results show that the method in this paper is better. is also confirms the effectiveness of the proposed method and the importance of encoding the hierarchical structure into the model.

MPII Data Set.
e MPII data set represents the most challenging benchmark data set in human pose estimation because it contains many changes in various poses of people performing different activities. In Figure 7, the PCKh results are shown in the test set using the MPII data set, and all results have a tolerance of 0.5. Compared with the methods in the literature [22,23]   Since the MPII data set is the most challenging in human pose estimation, this article will conduct comparative experiments with different tolerances for this data set. Figure 8 shows the average PCKh value on the MPII data set. It can be seen from the curve that the method in this paper is better than existing methods under low tolerance.

Self-Created Data Set.
rough the training and recognition of the neural network, the experimental results obtained are shown in Figure 9. It can be seen from Figure 9 that the recognition accuracy rate has reached more than 90%, and the expected accuracy rate goal has been achieved.
Because different users have individual differences, such as tall, short, fat, and thin, there will be user independence problems in the human body gesture recognition process, and the human body gesture features extracted by CNN in this paper are invariant to the user's posture changes. erefore, in order to verify the versatility of the recognition method in this paper, 450 pictures of two fatter and smaller users that are not included in the human body pose database are selected to verify the recognition effect of this paper, and each type of sample is 150.
e experimental results are shown in Table 2.
When the user's posture changes, the classification accuracy of the recognition method in this paper is almost unaffected. is shows that the human posture feature extraction method in this paper can well characterize the characteristics of human posture and to a certain extent solves the user independence in the process of human posture recognition.
In the past, human body gesture recognition is often limited by lighting conditions, and the recognition effect is often limited in an environment with too dark or too bright light. In order to verify the recognition effect of the recognition algorithm in this paper in a universal environment, this paper collected 360 sample images of the human body pose of the same person under different lighting and background conditions. Among them, there are 120 different kinds of human body postures. It was identified, and the results are shown in Table 3.
e experimental results show that when environmental factors such as light and background change, the classification accuracy of the recognition method in this paper is almost unaffected, and the overall average accuracy rate reaches 0.965, which shows that the recognition algorithm in this paper has universal applicability to the environment.
Collect 300 images containing subjects under joint occlusion and joint nonocclusion mainly to calculate the deviation distance between the actual coordinates of the shoulder, elbow, wrist, and knee joints and their theoretical coordinates and the maximum error of the human body gesture recognition in this article 3 cm for comparison. After calculation, when the joint is not occluded, the joint recognition accuracy of this paper is 0.82, as shown in Figure 10. When the joints (wrists and knees) are blocked, the joint recognition accuracy can also reach 0.98. Literature [21] Literature [22] Literature [23] This paper

Conclusion
is paper proposes a generated confrontation network based on stacked hourglasses, which can implicitly infer the structure and hierarchy of various body parts. e network proposed in this article shows the ability to learn to estimate the spatial relationship of body parts. It is trained end-to-end and tested on three different benchmark data sets. e network has shown the ability to estimate body parts, which are largely deformed and highly occluded. e method proposed in this paper can overcome these obstacles and achieve better performance. It also obtains the latest experimental results on a variety of data sets and compares them with the results on other related data sets. In addition, the network proposed in this paper can also be extended to encode other associated features between adjacent body parts. Moreover, the network is also suitable for processing multiple hierarchical levels between body parts. Currently our algorithm is more sensitive to light, which is also the limitation of our algorithm. Our next work is to solve the problem that the algorithm is sensitive to light.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no known conflicts of interest or personal relationships that could have appeared to influence the work reported in this paper.