Elderly care: activities of daily living classification with an S band radar

: Falls in the elderly represent a serious challenge for the global population. To address it, monitoring of daily living has been suggested, with radar emerging to be a useful platform for it due to its various benefits with acceptance and privacy. Here, we show results from the use of an S band radar for activity detection and the importance of selecting specific frequency bins to improve its suitability for human movement classification. The use of feature selection to improve detection of key activities such as falls has been presented. Initial results of 65% are improved to 85% and further to 90% with the aforementioned methods.


Introduction
Falls are challenges both for public health and society worldwide, and especially in Europe, due to an ageing population. Health dependency rate in France will skyrocket based on the demographic estimates for the next 40 years [1]. The number of people over 60 will increase by 10.4 million (80% extra) between 2007 and 2060 reaching the number of 23.6 million. The number of people over 75 will increase from 5.2 million in 2007 to 11.9 million in 2060, and 1.3 million to 5.4 million for people over 85. The transformation of society and budget cuts for health services are straining the healthcare sustainability with lifestyle-related diseases, increasing urbanisation, and demand for elderly care. Caring for dependent or at high risk is a major economic and societal issue. France has over 1.1 million dependent and at-risk individuals, with an estimated growth of 2%/year by 2040. Thus, 2 million people will need support in 2020 [2]. Health technologies are an important, dynamic, and high added-value market, especially for risk prevention estimated in France at 6.7 billion euros in 2005 [3]. It includes medical devices, technical assistance, medical benefits, and domestic help [4].
This favourable economic situation, combined with recent advances in the field of ICT, sensor networks, and device miniaturisation, enables the development of new technological solutions for prevention (identification of potential risks) and for situational detection (falls), particularly in the context of specialised institutions. Note the use of these technologies in medical monitoring does not yet have legal and ethical frameworks. Many questions remain unanswered, including their use and reliability, the confidentiality level of the data exchanged, and the social acceptability [5].
According to the World Health Organisation (WHO), falls are a major public health problem, over 650,000 fatal falls are recorded each year [6,7]. They are the second leading cause of unintentional injury death in the world after deaths from road traffic injuries. WHO [6] reports that 28-35% of people aged over 65 fall every year, and increases to 32-42% for people over 70. The frequency of falls increases with age and the level of frailty. Seniors living in retirement homes fall more often than those in community do. 30-50% of people living in long-term care facilities die each year, and 40% of them have experienced recurrent falls. Worldwide, people over 60 are the most affected with a high death rate. More than 37.3 million falls each year are serious enough to require medical attention. Although it does not systematically result in death, fall victims most often have a physical disability requiring long-term care or having to be placed in a specialised institutions [8].
In France, falls are responsible for >12,000 deaths a year [9]. If they do not lead to death, they are disabling with a certain loss of autonomy. 5% of falls result in fractures of the humerus, wrist or pelvis. 2% of falls result in a broken hip (55,000 in France/year). In the elderly, a 'benign' fall may be complicated and life threatening, but also functional. Other serious injuries (e.g. head, internal injuries, and lacerations) occur in ∼10% of cases. Some injuries related to a fall are fatal. About 5% of seniors with hip fractures die during hospitalisation. The overall mortality in the 12 months following a hip fracture varies from 18 to 33%. Totally, 50% of the elderly who fall cannot get up without help. Staying on the ground over 2 h after a fall increases the risk of dehydration, pressure ulcers, rhabdomyolysis, hypothermia, and pneumonia [10]. Even without direct injury, 50% of people who spent a long time on the ground [6] (over 1 h) following a fall died within 6 months of the incident [11].
Inequalities in healthcare in urban and rural areas are multiplying with medical deserts in France, and the closures of small hospitals and of certain services in rural areas. Health infrastructures are not ready to handle the wave of elderly people by 2050. Critical events (falls, stroke etc.) are both serious and costly. Physical/cognitive decline if detected early allows better management of the symptoms and consequences of the disease, which lightens the weight of someone's institutionalisation on the infrastructure and keep people autonomous longer in their own homes [8].
The cost of trauma from falls is important [6]. For persons over 65, the average cost per injury due to a fall for the health system of Finland and Australia is $3611 and $1049, respectively. According to Canadian data, the implementation of effective prevention strategies, lowering the incidence of fall by 20%, could result in a net saving of over $120 million per year. Tree factors directly affect the cost of a fall: its consequences, the pathologies responsible, and the recurrence.
Early detection of these factors will reduce the potential number of falls [7]. The issue of monitoring indoor activities was addressed by several research projects, with the aim of reliably discriminating falls against other activities, and more generally analysing the daily activities of the subjects [12].

Sensing modalities
The automatic detection of postures has given rise in recent years to intense research activity and major economic spin-offs. Kinect 3D sensors have made it possible to change the paradigm in the field by offering the possibility of taking into account depth information (3D), and thus to discriminate more effectively different types of human movements. Today, most of these measurements and motion analysis systems are found in virtual animation (walking simulation) as well as in the biomechanical and medical fields [13]. They enable studying pathology detection problems, analysing human motion, or understanding the kinematic mechanisms of walking to be reproduced in humanoid robots. The academic competitions -Challenge -recently organised on this theme show the limitations of current sensors, especially for the recognition of posture in various situations and the falls in deficient people.
Current systems can be grouped into three categories; wearables, remote, and hybrid. Numerous wearables have been proposed for the surveillance of people and specifically for fall detection [12,14,15] over the last 30 years. These include portable devices such as pedometers [16], accelerometers [17], gyroscopes and panic pushbuttons [18], inertial sensors such as smartphones, magnetic sensors [19] and infrared, vibratory, acoustic [20]. Although these devices give good results [21] on fall identification (98%), most of these wearables solutions suffer from limiting factors [8]: i. Must be worn (depends on user compliance or thinking about it if you wake up at night to go to the bathroom). ii. Easily broken if they fall, get a shock or if someone sits on it.
iii. Need to be recharged (difficult for patients with dementia). iv. Stigmatising for people. v. Respect for privacy.
Monitoring systems are integrated in the user's place of life: video cameras [13,22], RGBI-D sensors [22][23][24][25][26] and radars, or a combination of these systems. In [27,28], a complete apartment has been equipped with PIR motion sensors, gas stove sensors, sensors in the bed, floor sensors etc. … and gives good activity diagrams of everyday life. However, they are not able to give finer information on gait analysis for change detection and require many changes for an installation or readjustment of sensors in their home. In [13], a thorough review of video cameras and radar technologies for assisted living is proposed. For radar and RGBI-D systems, open challenges remain in deploying and using these systems in practical scenarios at home or in specialised institutions: • In the case of cameras [13], the challenges remain occlusions (dead zones), night-time operation, 3D dead zones, accuracy, camera resolution, and respect for privacy. Monitoring people in their daily lives poses a real problem of confidentiality. Depending on the sensor, the perception of intrusion and respect for privacy are different. • For radar systems [11], the challenges are multipath, strong scatterers, emission regulations, and mutual interference.
Although there are more technological challenges with radar, the fact that there is no legal problem regarding image rights and no image of anyone is taken, thus respecting privacy facilitates the acceptance of end users and investors. For the reasons mentioned above, the radar mode is an interesting research field still untapped in specialised environment.

Radar and machine learning for activities of daily living
Radar is considered an important technology for health monitoring and fall detection in assisted living due to a number of attributes not shared by other sensing modalities [11]. Generally, activity classification is based on extracting characteristics from micro-Doppler (mD) signatures. The relative motion of the components of a body generates unique patterns in the time-frequency domain of radar returns. As a result, different activities generate unique features used for classification. A mD overview is provided in [29]. Mainstream techniques for classification consist of measuring and extracting different spectrogram characteristics (centroid, Doppler bandwidth, repetition period…), followed by different classifiers [21,[30][31][32]. Common classification techniques include Fisher discriminant analysis [33,34], K-nearest neighbours (KNN) [35], Naïve Bayes [36], Ensembles (e.g. [37]), support vector machine (SVM) [38].
In [11], an overview of classifiers is given for fall detection. They recommend using multiple sensors to raise the precision of fall detection by covering the target movement from multiple directions and to combat occlusions. Data fusion is performed by feature combination or selection. Although more complex, the combination method outperforms the selection method for different motion classifications. A variety of classifiers are used for fall detection [39,40] with SVM being the most popular. However, the choosing relevant features has been determined to have a greater impact on classification accuracy than the classifier applied [41]. Numerous contributions try to extract features and classify actions from the radar mD signatures [13,42,43].
This paper builds on our previous work on experimental radar design [44][45][46][47][48] for the experimental setup and procedure. As for the activities of daily living, this is the continuation of our work in gait analysis [49], activity classification with radar [19,21,32,50] taking into account similar actions challenging the classification process and including a Parkinsonian gait and a gerontology test for independence assessment. Fig. 1 shows the experimental setup with the radar and the Kinect V2 for ground truth. Ten different actions were performed (Table 1) with five different subjects ( Table 2) obtaining 424 radar signatures.

Methodology
The actions were purposely designed to include confusers specifically for falls as it is important to maximise fall detection and minimise the false alarm rate. A Parkisonian gait was mimicked to see if it was distinguishable from regular walk. In gerontology, the test time up and go [51] is assessing the ability of an elderly person to live independently. This would allow automated tests to be carried out on a regular basis if a radar system was fitted in retirement/private homes. For the experiments, a custom-built S-band radar from Beijing Institute of Technology (BIT) with carrier frequency 3.3 GHz and instantaneous bandwidth 320 MHz was used with two standardgain horn antennas (15 dB). The signal is a phase-coded stepped frequency (PCSF) [52] with a step of 5 MHz and a pulse repetition period of 64 μs. The ADC used to capture the data is TI ADC12D1000 with 12-bit resolution with a sampling frequency of 100 MS/s generating 200 MB/s of data. The measurements took place in BIT.

Signal processing
The code processing the raw data is organised as follows. First, the raw data is digital down-converted to baseband, then the I and Q data are reconstructed using a Hilbert filter before decimation by a factor of five. Then, the range compression is done for every pulse repetition period of 64 μs [52] resulting in range-profile vector length of 4800.
A first-order moving target indicator filter is applied to remove static clutter before generating range-Doppler images.
A total of 2 13 range profiles (0.5243 s) are used to generate one image with a fast Fourier transform (FFT) of length 2 14 (zeropadding) applied per range bin across the slow time dimension yielding: an unambiguous range of 9600 m, range resolution 1.5 m, Doppler ambiguity of 15.625 MHz, and a Doppler resolution of 1.9 Hz without windowing and 2.62 Hz with a Hamming window (selected for this study). The resolutions in range and Doppler are sufficiently fine to observe the mD modulations. The range-Doppler image is summed over the range dimension to obtain one spectrogram slice only retaining the Doppler information. A 95% overlap is used on the generation of range-Doppler images to obtain smoother spectrograms. The use of FFT for spectrograms imposes a trade-off between time and Doppler resolution. Other solutions for time-frequency transforms such as wavelet or bilinear transforms (Wigner-Ville, Cohen's class) exist with their own advantages and disadvantages [53], and this will be the subject of further research but are beyond the scope of this paper.

Feature extraction and selection
The features used were selected from previous activity recognition applications with radar [19,21,32] with new features which are expected to improve classification accuracy.
The principle features are the same: Doppler centroid and bandwidth. The former indicates the signature centre mass with modulations from limb movement contributing to this metric, the latter is the average spread of the mD signature which is dependent on the movements generated by the targets.
To estimate the overall signature information, we use entropy which indicates the randomness within spectrogram images. For interpreting the shift, which occurs when a limb is moved as certain Doppler bins shift values, we use skewness of the monochrome spectrogram. Singular value decomposition has been used here again as it provides a reduced information set while maintaining valuable spectral and temporal information.
Cadence velocity diagram (CVD)-based features in [32] have been reapplied to exploit instantaneous frequency. CVD upper and lower envelopes have been utilised along with periodic movement features (step repetition frequency).
In addition, we have also used the energy curve feature. It takes the energy within the time-frequency plot and uses the accumulated intensity in the frequency bands to indicate overall movement. Finally, to identify salient features, we perform an iterative selection method described in [32,41]. Here, we test feature combinations to find the best performing set by training and testing on a feature-by-feature basis.

Importance of selecting correct Doppler frequency bands
In preliminary tests, instead of selecting specific frequency band of the spectrogram which encompasses the range for human movement we used it in its entirety shown in Fig. 2 (left).
Using SVM with a cubic kernel and feature selection, the results (Fig. 3) were generated. The training to testing ratio was 7:3 with stratified subsampling, meaning each of the classes had the same training to testing ratio, to prevent class imbalance. This process was repeated 20 times and the average is presented in this paper. Here, we see an average classification of 65% with low   classification rates for activities 2 and 9. There is a non-uniform pattern of confusion, notably for activity 1, where confusion in the range of 15-25% occurs. Four cases of severe misclassification, highlighted in dark red, occurred meaning while there is enough information within the spectrogram with the extended frequency range; it is not enough for the features to correctly be attributed to the correct class. In short, it makes the feature space noisy and inaccurate and this has direct effect on some classes. Looking at activity 1 (TUG), it is mostly confused with two other activities, activity 3 (fall) and activity 9 (walk) towards the radar. Considering TUG is composed of standing up, a forward movement, and walking towards the radar, it is understandable that it is confused as these two activities. Radial movement of both sitting up and falling is expected to be similar as both involve forward movement of the torso. For a healthy individual, the TUG and walk should have the same signature for the walk, which would have the same radial component making it easily confusable. Activity 2 (check under a chair) is often confused with activity 8 (tie shoelaces). The central component of the movement is bending forwards, going to a low elevation and remaining there for a certain duration. In this instance, the hand movements, which separate the classes, seem to be ignored due to the noise in the feature space.
Misclassification occurs with activity 10 (walk carrying a bar) and activity 9 (walk). The increased noise hinders the detection of the arm movements, which separate the activities.
The general similarity of the various movements increases the difficulty for classifying between them, when the spectrogram and the derived features have redundancies or do not carry useful information. Less severe misclassification events across the activities also occur but the motions present variations between participants making the task arduous.
After selecting frequency bands for feature extraction, we used feature selection to investigate which features were contributing to the accuracy and if any of them were redundant. Fig. 4 shows the results from the feature selection process and when all features are used, the classification accuracy is 85%. The confusion matrix resulting from selecting all the features is shown in Fig. 5 and when compared to Fig. 3, we see the effect of selecting the correct frequency band as all the severe misclassification events, which are in squares in Figs. 3 and 5 and 6; either decrease or disappear. Using the specific bands, appears to increase the accuracy of detecting A3 (falls in blue), which is a key activity in assisted living. Through this selection, it rises from 80 to 98% meaning most falls are detected.

Feature selection and overall results
In Fig. 4, we see the optimal features are around the 13-feature mark. It is notable that with seven features, it approaches the maximum accuracy. From the pool of features, the centroid and spectral SVD-based features appear to be the most salient with entropy, energy curve, and step repetition frequency. Fig. 6 displays the result from the post-feature-selection classification. Looking at the previous cases where activities were misclassified severely, the boxed items in Figs. 3, 5 and 6; we see that the misclassification events are further attenuated after feature selection. The remarkable result, however, is again, in fall detection, highlighted in blue, where it increased to 100%. This simply emphasises the critical nature of having salient features, since the presence of redundancies in the feature space appears to reduce the detection of fall events, which is a necessary characteristic of any assisted elderly care system.

Conclusion
For an automated elderly care system, the various benefits of radar make it an attractive sensing modality to use. In this paper, we demonstrated the use of an S band radar to classify a set of activities representative of movements made by the elderly on a daily basis. We demonstrated the importance of selecting frequency bands close to the human movement when extracting features along with the use of feature selection and its importance in classifying specific critical activities such as falls to a high accuracy.
Despite the feasibility of using radar for this set of similar activities is apparent, it would be desirable for the detection rates of other activities to be as high as possible. For this, new features, which exploit the arm movements, would be required. Other future work will involve having a varied sample of participants and evaluating the importance of sensor location.

Acknowledgments
We are very grateful to the Beijing Institute of Technology for lending us an operational radar to carry out this experiment. Special thanks go to Prof. Long Teng, Dr. Liu Quanhua and their students during the experiments that were essential for carrying them out in a timely fashion. The collaboration between University of Glasgow, University of Electronic Science and Technology of China and Université Cergy-Pontoise was partly funded by Campus France with PHC Xu Guangqi -38715QJ. A. Shrestha is supported for his PhD by the UK Engineering and Physical Sciences Research Council (EPSRC) Doctoral Training Award to the School of Engineering. The authors acknowledge support from the UK EPSRC through grant EP/R041679/1 INSHEP.