A research on foot size measurement algorithm based on image

Foot size measurement has a wide range of applications. It can not only be used in medical foot health, evaluate the health index of the foot, improve the health of the human foot, but also provide a large amount of foot data for the study of kinematics. Due to the limitations of measurement equipment and related algorithms, 3D foot measurement cannot be widely used. We proposed a fast and convenient foot measurement method based on deep learning and image segmentation algorithms. The HED model is used to detect the edges of A4 paper; the traditional image segmentation algorithm GrabCut is used for segmentation of the foot area. The results show that the algorithm proposed in this paper has a faster measurement speed and higher accuracy for foot size measurement.


Introduction
The foot is a very important part of the human body. There are many bones in the foot. The foot is the only part in contact with the ground when the human body is walking or standing, which plays a vital role in the load of the human body. Shoe products support and protect the foot when the human body is standing, walking and jumping. Medical research shows that a suitable pair of shoes is closely related to the distribution of plantar pressure. Too much plantar pressure can cause foot pain, abnormal gait and other problems. Wearing shoes that do not fit for a long time may not only cause a variety of feet diseases, and it is very easy to cause sports injuries and affect human health. Therefore, designing, producing and helping consumers to buy footwear products that fit their foot size has important value and social significance.
The traditional contact measurement method has low work efficiency and large measurement equipment, which cannot meet the needs of rapid foot size measurement. On the other hand, the contact measurement method will deform the surface of the foot, and the foot parameters cannot be accurately obtained.
The measurement method based on 3D reconstruction uses structured light to reconstruct the surface of the foot, and then uses a plantar scanner to obtain the foot information. The complete foot shape is obtained by splicing the two information, and the common parameters of the foot are measured. Although the measurement method of 3D reconstruction has high accuracy, it is not suitable for largescale promotion and use due to the expensive equipment and long time consumption.
Measurement based on image does not need to reconstruct the 3D shape of the foot. It is more convenient and faster to use advanced image processing technology to measure the corresponding object. Measurement based on image technology is more convenient and quicker than 3D measurement, and measurement can be performed only by taking pictures of the measured object. At present, there are not many researches on this methods in foot measurement. With the rapid development of deep learning, it has shown good results in many research fields. Some scholars combine deep learning and image measurement to improve measurement speed and accuracy. Based on deep learning, this paper used deep learning to detect edges, propose a more accurate and efficient foot measurement method. This paper used deep learning to process the edge information of the image, and then used the traditional opencv algorithm to segment the foot to complete the measurement. This method is more convenient, faster and more efficient than other methods.

Algorithm
Combining deep learning and traditional image processing technology, this paper proposes a foot measurement method based on the edge of the image, using relevant edge information to complete the foot measurement.
With standard A4 paper as the background, it is easy to obtain foot images. When the algorithm measures the foot, there is no need to go through 3D reconstruction process, and the foot size can be measured quickly and accurately. The algorithm can be divided into three main parts: A4 paper edge detection; foot segmentation; foot measurement. The foot measurement algorithm is shown in Table 1.

A4 paper edge detection
In the process of detecting the edge of A4 paper, due to the complexity of illumination and background, traditional image processing technology cannot detect the edge of A4 paper well. HED (Holisticallynested edge detection) is a deep learning model proposed by Saining Xie et al. for edge detection. The HED network model is based on the improvement of the VGG16 network model. It is a multi-scale fusion network model.
The algorithm in this paper uses the HED deep learning model to detect the edge of A4 paper, and the HED network model is shown in Figure 1. The HED model removes the pooling layer of the fifth convolutional layer, the last fully connected layer and the softmax layer, and takes the output of the last convolutional layer of the 5-stage convolution for transposed convolution and deconvolution operations. The HED model obtains five images of the same size for multi-scale fusion. The HED method is a popular image edge detection method, which has a good edge detection effect.

Input
Output  Figure 1. Network Structure

Foot segmentation
After the edge detection is completed, the correlation function in opencv is used to find the contour of the A4 paper and determine the corner coordinates, and the picture is corrected by affine transformation. After obtaining the foot image based on the A4 paper as the background, due to the simple background, the foot area can be segmented using the traditional GrabCut segmentation algorithm.
Grabcut algorithm is an improvement of GraphCut algorithm, which is an image segmentation method based on artificial interaction. The idea of the algorithm is to map the entire image to an s-t network graph with non-negative weights, where s and t represent the foreground and background endpoints, respectively. The edge set contains the connecting edges between the source point and sink point and all other nodes, and the edges between adjacent nodes in the graph. At the same time, the similarity between pixels and foreground, background and the difference between neighboring pixels are represented by edge weights. In this method, the foreground and background are established by manual labeling, and the image target and background model are respectively established with a multidimensional Gaussian Mixture Model (GMM).
The GrabCut algorithm has fast segmentation speed and good effect. The GrabCut algorithm uses the color information and boundary contrast information in the image. It requires the user to enter a bounding box as the segmentation target position to complete the optimal segmentation of the foreground and the background, thereby achieving the separation/segmentation of the target and the background. The research method in this article In the A4 paper, the entire area where the A4 paper is located is used as the bounding box segmentation target position to achieve the segmentation of the foot and the A4 paper background. Figure

Foot measurement
After GrabCut segmentation, the corresponding mask image is generated. The mask image has large pixel differences and can directly use the threshold segmentation to complete the foot measurement, which is faster and more accurate. The specific calculation formula is as follows: = * (1) = * (2) L and W respectively represent the length and width of A4 paper in the real world; l and w respectively represent the pixel length of A4 paper in the foot image; F_L and F_W respectively represent the length and width of the foot in the real world; F_l and F_w respectively represent the pixel length of the foot in the image.

Computer Configuration
The experimental environment is based on the Tensonflow open source framework to achieve deep learning. The GPU model is NVIDIA GeForce GTX 2080Ti, and the CPU is i9-9900K.

Edge Detection
The HED model uses the BSDS500 data set as the training set and the test set for detecting the edge of the image. BSDS500 is a data set provided by the University of Berkeley Computer Vision Group, which can be used for image segmentation and object edge detection. The data set contains 200 training images, 100 verification images and 200 test images.
In the training phase, the number of iterations is set to 100 000, and the learning rate is set to 0.001. As the number of training increases, the accuracy rate gradually increases, and the loss value gradually decreases. Table 2 shows the training results.

Measurement results and analysis
In order to verify the effectiveness of the algorithm, we selected 25 volunteers to collect foot data for testing.. In order to meet the algorithm requirements, the following foot data collection methods are defined: in any scene, the heel of the foot is stepped on the edge of the narrow edge of the A4 paper, the vertical direction of the foot is kept level with the long edge of the A4 paper, and the camera is held horizontally from the top of the foot. At the same time, manually measure the volunteer's foot size, record the foot length and foot width. Through this method, We collected four photos of each of 25 volunteers, a total of 100 test images. Part of the foot data is shown in Figure 3.   5 We measured 100 foot images of 10 volunteers and calculated the measurement error. Figure 4 shows the results of part of the image edge detection. Table 3 lists the average error between the algorithm measurement results and the manual measurement (unit: mm). The last column lists the satisfaction of 10 volunteers with the foot measurement algorithm. Satisfaction degree (1 to 10 points, 1 point represents Dissatisfied, 10 points means very satisfied. The above experimental results show that using deep learning and image processing algorithms to measure foot size can obtain more accurate measurement data, and the algorithm has better accuracy and robustness.

Conclusion
This paper proposes a foot size measurement method based on deep learning. Compared with the traditional manual measurement, the algorithm proposed in this paper has fast measurement speed and high efficiency. Compared with the measurement based on three-dimensional reconstruction, the measurement process is simple, and it can be measured conveniently and quickly.
After testing, the algorithm proposed in this paper can measure foot size conveniently and quickly, and has certain popularization value. However, the algorithm in this paper still has some shortcomings. For example, in the process of foot segmentation based on threshold, the traditional grabcut segmentation method is very effective. The shadow part of the image cannot be effectively processed, which affects the accuracy of foot size measurement.
Foot measurement based on two-dimensional images can only obtain the length and width parameters of the foot. In the future, it will focus on the study of three-dimensional parameters such as the circumference of the metatarsal toe, the circumference and height of the anterior tarsal bone.