Fall Direction Detection in Motion State Based on the FMCW Radar

Accurately detecting falls and providing clear directions for the fall can greatly assist medical staff in promptly developing rescue plans and reducing secondary injuries during transportation to the hospital. In order to facilitate portability and protect people’s privacy, this paper presents a novel method for detecting fall direction during motion using the FMCW radar. We analyze the fall direction in motion based on the correlation between different motion states. The range–time (RT) features and Doppler–time (DT) features of the person from the motion state to the fallen state were obtained by using the FMCW radar. We analyzed the different features of the two states and used a two-branch convolutional neural network (CNN) to detect the falling direction of the person. In order to improve the reliability of the model, this paper presents a pattern feature extraction (PFE) algorithm that effectively eliminates noise and outliers in RT maps and DT maps. The experimental results show that the method proposed in this paper has an identification accuracy of 96.27% for different falling directions, which can accurately identify the falling direction and improve the efficiency of rescue.


Introduction
Fall accidents pose a major threat to the health and life of the elderly. When a fall accident occurs, the elderly person may be seriously injured or lose consciousness. It is necessary to detect falls in time and provide detailed fall information to help medical staff quickly formulate a treatment plan [1][2][3].
The directions of falls in motion can be categorized into four types: forward, backward, left, and right. Different fall directions can result in injuries to different parts of the body [4,5]. In reality, elderly individuals often lose consciousness and roll unconsciously after a fall. Upon arriving at the scene, medical staff are often faced with the daunting task of identifying the precise location of the injury, which can prove to be a challenging and time-consuming process, ultimately delaying the administration of prompt and effective treatment. Therefore, it is essential to conduct a specialized study on this subject. By detecting the precise direction of the fall, paramedics can receive valuable information to promptly formulate an effective treatment plan [6][7][8].
Researchers have explored different sensors for the detection and warning of fall accidents, such as wearable sensors and vision sensors [9][10][11][12][13][14][15]. In [16,17], the researchers used wearable sensors to detect the fall direction and determine the location of the fracture. In [18], the researchers proposed a relationship between the direction and severity of a fall during fall detection. They classified falls by using an accelerometer-based multi-classifier and trained five different models. In [19], the researchers investigated the correlation between fall direction and injury severity. They used the SisFall dataset and a support vector machine classifier to determine the direction and impact of falls. In [20], a further study of fall and non-fall actions was conducted, and a classification method was developed We combine range-time features and Doppler-time features and use a dual-branch CNN to achieve fall direction detection and recognition. To improve the model's reliability and reduce the system's computational cost, this paper proposes a pattern feature extraction (PEF) algorithm to eliminate noise and outliers in the environment.
The rest of this paper is structured as follows. Section 2 introduces the Materials and Methods. Section 3 introduces the Experimental Platform. Section 4 provides the Experimental Results. Section 5 provides the Conclusions, summarizing the research conducted and their implications for future research in the field.

Fall Direction Detection System
The fall direction detection system we designed is depicted in Figure 1. We collected human motion information using the FMCW radar and used a signal processing algorithm to process the original radar information to obtain the range change and Doppler change from the motion state to the falling state. We use the pattern feature extraction (PEF) algorithm to remove noise and outliers in the environment and to feed the processed data into the built dual-branch CNN to identify the direction of the fall.
Sensors 2023, 23, x FOR PEER REVIEW 3 of 17 and identifying the direction of falls in motion. Based on the aforementioned analysis, this study utilizes the FMCW radar to detect the direction of falls in motion among elderly individuals. We combine range-time features and Doppler-time features and use a dualbranch CNN to achieve fall direction detection and recognition. To improve the model's reliability and reduce the system's computational cost, this paper proposes a pattern feature extraction (PEF) algorithm to eliminate noise and outliers in the environment. The rest of this paper is structured as follows. Section 2 introduces the Materials and Methods. Section 3 introduces the Experimental Platform. Section 4 provides the Experimental Results. Section 5 provides the Conclusions, summarizing the research conducted and their implications for future research in the field.

Fall Direction Detection System
The fall direction detection system we designed is depicted in Figure 1. We collected human motion information using the FMCW radar and used a signal processing algorithm to process the original radar information to obtain the range change and Doppler change from the motion state to the falling state. We use the pattern feature extraction (PEF) algorithm to remove noise and outliers in the environment and to feed the processed data into the built dual-branch CNN to identify the direction of the fall.

Radar Raw Data Processing
When the FMCW radar is working, the transmit and receive antennas are switched on synchronously. The frequencies of the transmitted and received signals are depicted in Figure 2. The frequencies of the signal waveform. The transmitted signal can be written as

Radar Raw Data Processing
When the FMCW radar is working, the transmit and receive antennas are switched on synchronously. The frequencies of the transmitted and received signals are depicted in Figure 2. The frequencies of the signal waveform. The transmitted signal can be written as where A T is the transmitting power, ϕ(t) is the phase noise of the transmitter, f c is the initial frequency of the chirp; B is the bandwidth of the chirp and T c is the duration of the chirp.

023, 23, x FOR PEER REVIEW
where is the transmitting power, ( ) is the phase noise initial frequency of the chirp; is the bandwidth of the chirp the chirp. The point target within the effective range of the radar wil which the radar receives after a brief delay. The delay can be w where is the distance of the measured target. The signal th written as: where is the factor of influence amplitude change caused b vironment and the scattering of the object. The specific form signal can be written as: where is the slope of the chirp signal. We can obtain base obtain complex baseband data ( ) through the A/D convert When all chirps are ordered by time, a comprehensive m generated, accurately capturing critical information. The matr and vertical axes, the former representing the slow time dime senting the fast time dimension. As shown in Figure 3, we u transform the slow time dimension of ( ) to obtain the DT formula for STFT can be expressed as The point target within the effective range of the radar will generate a reflected signal which the radar receives after a brief delay. The delay can be written as where d is the distance of the measured target. The signal that the radar receives can be written as: where α is the factor of influence amplitude change caused by the attenuation of the environment and the scattering of the object. The specific formula for the beat frequency signal f b can be written as: where β is the slope of the chirp signal. We can obtain baseband I/Q signals and then obtain complex baseband data Z(n) through the A/D converter. When all chirps are ordered by time, a comprehensive matrix of radar raw data is generated, accurately capturing critical information. The matrix is crafted with horizontal and vertical axes, the former representing the slow time dimension and the latter representing the fast time dimension. As shown in Figure 3, we used the STFT algorithm to transform the slow time dimension of Z(n) to obtain the DT characteristics. The specific formula for STFT can be expressed as where z(t) is the radar echo signal, ω(t) is the sliding window function, and f d represents the Doppler frequency. The distance equation of the measured object can be written as:  To improve the reliability of the experiment, we merged the four falling directions when close to the radar and the four falling directions when away from the radar into four falling directions in motion. Figure 4 shows the range-time map of different fall states. In addition, Figure 5 shows the Doppler-time map of different fall states. We used the DFT algorithm to transform the fast time dimension of Z(n) to obtain the RT characteristics. The specific formula for DFT can be written as: where x[n] is the original signal.
To improve the reliability of the experiment, we merged the four falling directions when close to the radar and the four falling directions when away from the radar into four falling directions in motion. Figure 4 shows the range-time map of different fall states. In addition, Figure 5 shows the Doppler-time map of different fall states.
To improve the reliability of the experiment, we merged the four falling directions when close to the radar and the four falling directions when away from the radar into four falling directions in motion. Figure 4 shows the range-time map of different fall states. In addition, Figure 5 shows the Doppler-time map of different fall states. From the range-time diagram in Figure 4, we can observe that when a person falls forward or to the left during motion, the distance from the motion state to the fallen state follows a similar trend. When approaching the radar, the distance between the person and the radar increases. In addition, when moving away from the radar, the distance between the person and the radar decreases. However, the change in distance for a forward fall is greater than that for a left fall. On the other hand, when a person falls backwards or to the right during motion, the distance from the motion state to the fall state changes in the right during motion, the distance from the motion state to the fall state changes in the opposite direction. When approaching the radar, the distance between the person and the radar first decreases and then increases. In addition, when moving away from the radar, the distance between the person and the radar first increases and then decreases. However, the change in distance for falling backwards is greater than for falling to the right. Therefore, we can identify different fall directions based on the range-time features. From the Doppler-time graph in Figure 5, we can observe that when a person falls forward or to the left during motion, the Doppler change trend from the moving state to the falling state is the same. When approaching the radar, the Doppler information is negative. In addition, when far away from the radar, the Doppler information is positive. However, the Doppler change for a forward fall is greater than that for a left fall. On the other hand, when a person falls backward or to the right during motion, the Doppler change from the motion state to the fallen state is the opposite. When approaching the radar, the Doppler information is first negative and then positive. In addition, when far away from the radar, the Doppler information is first positive and then negative. However, the Doppler change for a backward fall is greater than that for a rightward fall. Therefore, we can identify different falling directions based on the Doppler-time features.
In summary, both range-time features and distance-time features can differentiate the direction of falls. However, to enhance the reliability of the model, we adopted a fusion of the two features for the recognition of fall direction.

Pattern Feature Extraction
We can observe that there is a lot of noise and many outliers in the RT map and the DT map, which cause great interference in identifying the falling direction. Therefore, we From the range-time diagram in Figure 4, we can observe that when a person falls forward or to the left during motion, the distance from the motion state to the fallen state follows a similar trend. When approaching the radar, the distance between the person and the radar increases. In addition, when moving away from the radar, the distance between the person and the radar decreases. However, the change in distance for a forward fall is greater than that for a left fall. On the other hand, when a person falls backwards or to the right during motion, the distance from the motion state to the fall state changes in the opposite direction. When approaching the radar, the distance between the person and the radar first decreases and then increases. In addition, when moving away from the radar, the distance between the person and the radar first increases and then decreases. However, the change in distance for falling backwards is greater than for falling to the right. Therefore, we can identify different fall directions based on the range-time features.
From the Doppler-time graph in Figure 5, we can observe that when a person falls forward or to the left during motion, the Doppler change trend from the moving state to the falling state is the same. When approaching the radar, the Doppler information is negative. In addition, when far away from the radar, the Doppler information is positive. However, the Doppler change for a forward fall is greater than that for a left fall. On the other hand, when a person falls backward or to the right during motion, the Doppler change from the motion state to the fallen state is the opposite. When approaching the radar, the Doppler information is first negative and then positive. In addition, when far away from the radar, the Doppler information is first positive and then negative. However, the Doppler change for a backward fall is greater than that for a rightward fall. Therefore, we can identify different falling directions based on the Doppler-time features.
In summary, both range-time features and distance-time features can differentiate the direction of falls. However, to enhance the reliability of the model, we adopted a fusion of the two features for the recognition of fall direction.

Pattern Feature Extraction
We can observe that there is a lot of noise and many outliers in the RT map and the DT map, which cause great interference in identifying the falling direction. Therefore, we propose a pattern feature extraction (PEF) algorithm to lessen computing costs and enhance the reliability of the model. Figure 6 shows the algorithm flowchart.  After choosing an appropriate threshold, there are still some discrete feature map. Then, by removing the outliers by using the Hampel filter, t formula of the Hampel filter can be written as: The median {•} is the median of the sought signal, and ℎ is the fina Figure 7a,d is the unprocessed RT map and the DT map. The area mar circle is the noise from the surrounding environment. Figure 7b,e shows th where a ∈ (0, 1) is an adaptive parameter. The highest power value, lowest power value, and average power value in the feature diagram are represented by max{P}, min{P}, and P, respectively. When creating a dataset, the power values in each feature map vary.
Choosing the appropriate threshold for each feature image can reduce the ambient noise more effectively.
After choosing an appropriate threshold, there are still some discrete outliers in the feature map. Then, by removing the outliers by using the Hampel filter, the calculation formula of the Hampel filter can be written as: The median {·} is the median of the sought signal, and n th is the final threshold. Figure 7a,d is the unprocessed RT map and the DT map. The area marked by the red circle is the noise from the surrounding environment. Figure 7b,e shows the RT map and the DT map with less noise and stronger features after processing with the power-threshold algorithm. The red circles in Figure 7b,e represent the remaining outliers. Figure 7c,f shows the results processed by the Hampel filter, which are more suitable as input for the CNN to improve the reliability of the model. After choosing an appropriate threshold, there are still some discrete outliers in the feature map. Then, by removing the outliers by using the Hampel filter, the calculation formula of the Hampel filter can be written as: The median • is the median of the sought signal, and is the final threshold. Figure 7a,d is the unprocessed RT map and the DT map. The area marked by the red circle is the noise from the surrounding environment. Figure 7b,e shows the RT map and the DT map with less noise and stronger features after processing with the power-threshold algorithm. The red circles in Figure 7b,e represent the remaining outliers. Figure 7c,f shows the results processed by the Hampel filter, which are more suitable as input for the CNN to improve the reliability of the model.

Dual-Branch CNN
The convolutional neural network (CNN) is a cutting-edge technology that utilizes internal convolution to extract a wide range of input features, including target color, edges, corners, and other critical information. These features serve as the foundation for CNN classification, allowing for unparalleled accuracy and precision in a variety of applications. By reducing the computational cost of preprocessing, CNN has become a go-to solution for classification tasks across a broad range of industries and disciplines. To identify different falling directions, it is necessary to detect changes in distance and Doppler before and after the person falls. Therefore, this experiment uses the RT map and the DT map as the input for the dual-branch CNN. The dual-branch CNN used in the experiments is traditional and basic. We are committed to using the simplest network structure to realize the recognition of the direction of the fall.

Dual-Branch CNN
The convolutional neural network (CNN) is a cutting-edge technology that utilizes internal convolution to extract a wide range of input features, including target color, edges, corners, and other critical information. These features serve as the foundation for CNN classification, allowing for unparalleled accuracy and precision in a variety of applications. By reducing the computational cost of preprocessing, CNN has become a go-to solution for classification tasks across a broad range of industries and disciplines. To identify different falling directions, it is necessary to detect changes in distance and Doppler before and after the person falls. Therefore, this experiment uses the RT map and the DT map as the input for the dual-branch CNN. The dual-branch CNN used in the experiments is traditional and basic. We are committed to using the simplest network structure to realize the recognition of the direction of the fall.
The network can fuse motion information from both the RT map and the DT map. In Figure 8, the network model of the dual-branch CNN is shown. Table 1 shows the parameter settings of each convolutional layer, pooling layer, and fully connected layer.
The convolutional neural network (CNN) is a cutting-edge technology that utilizes internal convolution to extract a wide range of input features, including target color, edges, corners, and other critical information. These features serve as the foundation for CNN classification, allowing for unparalleled accuracy and precision in a variety of applications. By reducing the computational cost of preprocessing, CNN has become a go-to solution for classification tasks across a broad range of industries and disciplines. To identify different falling directions, it is necessary to detect changes in distance and Doppler before and after the person falls. Therefore, this experiment uses the RT map and the DT map as the input for the dual-branch CNN. The dual-branch CNN used in the experiments is traditional and basic. We are committed to using the simplest network structure to realize the recognition of the direction of the fall.
The network can fuse motion information from both the RT map and the DT map. In Figure 8, the network model of the dual-branch CNN is shown. Table 1 shows the parameter settings of each convolutional layer, pooling layer, and fully connected layer.

Experiment Platform
The FMCW radar in the experiment is an IWR6843 produced by Texas Instruments (Dallas, TX, USA) [38]. The FMCW radar operates from 60 to 64 GHz. In our experiments, the radar is started at 60 GHz with a chirp slope of 33 MHz/µs. The number of sampling points selected on each chirp is 256. The other radar parameters are shown in Table 2.

Experimental Environment
The experimental environment setup is shown in Figure 9. The radar sensor is installed approximately 6 m in front left of the volunteer, with a height of about 1.5 m above the ground and inclined downward at 15 • to ensure that the radar can collect the overall information. The experiment considers five fall states, including forward fall in motion, backward fall in motion, left fall in motion, right fall in motion, and fall in non-motion.  Figure 10 shows the data collection scene for some actions in the experiment. The blue arrow in the figure represents the motion direction of the volunteers, while the yellow arrow represents the fall direction of the volunteers.

Data Collection
Six volunteers participated in the experiment. The FMCW radar collected each action for 10 s. Each volunteer performed five actions. In addition, each movement was performed 80-100 times. A total of 573 pieces of data were collected. The experiment used the data sample generation method [39], and the radar mode was set to one transmission and four receptions. Through this method, we expanded the dataset to 7500 samples. Figure 11 is the proportion of data collected. Each color in the figure corresponds to a fall direction category. Each share is 1500 datasets.

Data Collection
Six volunteers participated in the experiment. The FMCW radar collected each action for 10 s. Each volunteer performed five actions. In addition, each movement was performed 80-100 times. A total of 573 pieces of data were collected. The experiment used the data sample generation method [39], and the radar mode was set to one transmission and four receptions. Through this method, we expanded the dataset to 7500 samples. Figure  11 is the proportion of data collected. Each color in the figure corresponds to a fall direction category. Each share is 1500 datasets.

Results
This experiment uses the Pytorch framework. The learning rate of the CNN is 0.0005, and the Step LR learning rate update strategy was used. The number of iterations is 50.

Results
This experiment uses the Pytorch framework. The learning rate of the CNN is 0.0005, and the Step LR learning rate update strategy was used. The number of iterations is 50. Figure 12 shows the changes in accuracy and the loss of the dual-branch CNN during the training process. It can be observed that after 10 rounds of training, the network model tended towards a stable state. In addition, after 20 rounds, the model's accuracy reaches its peak. Figure 11. Data samples corresponding to each action.

Results
This experiment uses the Pytorch framework. The learning rate of the CNN i and the Step LR learning rate update strategy was used. The number of iterations Figure 12 shows the changes in accuracy and the loss of the dual-branch CNN du training process. It can be observed that after 10 rounds of training, the network tended towards a stable state. In addition, after 20 rounds, the model's accuracy its peak.  In order to comprehensively evaluate the efficacy and accuracy of our proposed PEF algorithm, we utilized both the unaltered original dataset and the refined PEF dataset to detect the fall direction. Figure 13a shows the confusion matrix using the original dataset as input in the dual-branch CNN. In addition, Figure 13b shows the confusion matrix using the PEF dataset as input in the dual-branch CNN.
The results of our experimentation reveal that when using the original dataset, the network's recognition accuracy rates for forward fall in motion, backward fall in motion, left fall in motion, right fall in motion, and fall in non-motion were 92.40%, 91.73%, 89.73%, 89.74%, and 95.80%, respectively. When using the PEF dataset, the network's recognition accuracy rates for forward fall in motion, backward fall in motion, left fall in motion, right fall in motion, and fall in non-motion were 96.02%, 95.49%, 94.44%, 94.84%, and 98.20%, respectively. The dual-channel CNN built in this experiment has a recognition rate of 96.27% for the direction of the fall. Our proposed PEF algorithm improves the accuracy of the model by 3-4%.
Forward fall and left fall have the same change trend in the distance, but the change in the distance is different. In order to analyze the influence of the height of the tested person on the experimental results, we conducted experiments using the respective datasets of six volunteers. Table 3 is the height of the tested person and the accuracy of the corresponding model.
In order to comprehensively evaluate the efficacy and accuracy of our proposed PEF algorithm, we utilized both the unaltered original dataset and the refined PEF dataset to detect the fall direction. Figure 13a shows the confusion matrix using the original dataset as input in the dual-branch CNN. In addition, Figure 13b shows the confusion matrix using the PEF dataset as input in the dual-branch CNN. The results of our experimentation reveal that when using the original dataset, the network's recognition accuracy rates for forward fall in motion, backward fall in motion, left fall in motion, right fall in motion, and fall in non-motion were 92.40%, 91.73%, 89.73%, 89.74%, and 95.80%, respectively. When using the PEF dataset, the network's recognition accuracy rates for forward fall in motion, backward fall in motion, left fall in motion, right fall in motion, and fall in non-motion were 96.02%, 95.49%, 94.44%, 94.84%, and 98.20%, respectively. The dual-channel CNN built in this experiment has a recognition rate of  The experimental results show that the taller the subject is, the easier it is to recognize the direction of the fall. This may be because the height of the person affects the distance change during the fall. The larger the distance variation, the more distinct features we acquire.
In order to further evaluate the superiority of the PEF algorithm, we chose CNNs with different structures for fall direction detection on the PEF dataset and the original dataset, including the two-branch CNN, LeNet [40], AlexNet [41], and VGG [42], and the DT map or the PEF-DT map as the input to the single-branch network. Tables 4 and 5 are the falling direction detection results of different CNNs. The experimental results show that the recognition of the fall direction by dual-branch CNN detection is higher than that of the other three single-branch networks. This is because its input is composed of distance characteristics and Doppler characteristics, which contain more human motion information. When using the PEF dataset, LeNet, AlexNet, and VGG show better results in terms of precision and recall. The results show that the PEF algorithm also performs excellently in CNNs with different structures.

Discussion
In this section, we review related works on fall orientation detection and discuss their methods and results. Table 6 shows the results of using different sensors and models in the field of identifying different fall directions. In [16], Farhad Hossain et al. utilized a single 3D accelerometer to detect forward fall, backward fall, left fall, right fall, and ADL. They employed several machine learning algorithms for analysis and demonstration. The experimental results showed that the classification results using the SVM algorithm were the most accurate. The accuracy of the model was 94.54%. In [17], Abbas Shah Syed et al. used 3D accelerometers and gyroscopes to detect forward fall, backward fall, and lateral fall. In addition, they divided the fall action in each direction into two categories: hard fall and soft fall. They proposed the XGB-CNN network. The accuracy of the model was 90.02%.
In [18], Ryan M. Gibson et al. employed accelerometers to identify seven types of fall events, including forward fall, backward fall, left fall, right fall, hard fall, soft fall, and fall in any situation. They used the CVM algorithm to conduct experiments, and the experimental results showed that the accuracy of the model was 98.3%. In [19], Abbas Shah Syed et al. utilized the IMU-sensor-collected SisFall dataset to determine the direction of falls. They used the inertial measurement sensor and SVM algorithm to identify falls in three directions: forward, backward, and lateral. The model accuracy for fall detection was 90.4%.
In [24], Chunmiao Yuan et al. introduced a novel video-based method for detecting the direction of falls. They calculated human body joint points using body posture and identified four types of falls (forward, backward, left, and right) using the SVM algorithm. The model achieved an accuracy of 97.52% on the Le2i dataset.
However, there are some limitations to this paper. Specifically, we only employed a basic two-branch convolutional neural network to detect the direction of falls. In future research, we aim to develop a better performing network architecture that improves the overall performance of the system while improving accuracy.

Conclusions
This paper presents a fall direction detection method in motion based on the FMCW radar. In the study, we extract the range-time features and Doppler-time features of people in motion and fall states through the FMCW radar to distinguish different fall directions. We propose a PEF algorithm to reduce noise and outliers in feature maps. We built a dual-branch CNN to detect different fall directions and analyzed the performance of the PEF algorithm in different network structures. This method achieves high accuracy in identifying different fall directions.
In future work, we will collect fall direction data from volunteers of different ages to make our dataset more diverse. Additionally, we plan to prompt the injured body part and the corresponding treatment method after detecting the direction of the fall. This will enable elderly individuals to receive quick and effective treatment, reducing the risk of secondary injuries.