Fall Risk Reduction for the Elderly Using Mobile Robots Based on the Deep Reinforcement Learning

Slip-induced fall is one of the main factors causing serious fracture among the elderly. This paper proposes a deep learning based fall risk reduction measures by mobile assistant robots for the elderly. We use a deep convolutional neural network to analyze fall risks. We apply a deep reinforcement learning to control robots and reduce slip-induced fall risks of the elderly. The results suggest that the applicability of our method to other cases of the fall and other cases of accidents.


Introduction
2] In order to reduce the increasing fall accidents among the elderly, we take an approach to use mobile assistant robot.We consider safety in the environments where humans and assist robots coexist in the fields of medical welfare and assist living.
This paper proposes a deep learning based fall risk reduction measures by mobile assistant robots for the elderly.First, we collect preparatory data regarding past incidents and accidents data as input data to analyze fall risks and to learn examples of risk reduction measures.Second, we use a deep convolutional neural network 3 to analyze fall risks of the elderly.Third, we apply a deep * Panasonic Advanced Technology Development Co., Ltd, 600 Saedo-cho, Tsuzuki-ku, Yokohama City, 464-8603, Japan.namba.takaaki@jp.panasonic.comreinforcement learning [4][5] to control mobile robots and reduce slip-induced fall risks of elderly.Moreover, we evaluate the effect of risk reduction.

Risk analysis
The risk analysis is carried out through methods such as FTA (Fault Tree Analysis), 6 HAZOP (Hazard And Operability Study), [7][8] and FMECA (Failure Modes Effects and Criticality Analysis). 9Conventional risk analyses conducted manually or partially use a probability calculation tool when employed in applications such as system safety requirement analysis or design.This type of analysis has the following limitations:  Failures to identify hazards may occur depending on the proficiency of the safety management officers, that is, their experience and capability,  Risk analysis procedures are complex and require a specific number of man-hours depending on the scale of the assessment.
Most analyses and assessments are carried out offline based on prior information and require time to be conducted.The results are not immediately available.Therefore, the measures of risk reduction are often delayed, and it becomes difficult to respond promptly and flexibly in continually changing situations.
In this paper, we use online real-time risk analysis based on the deep convolutional neural network.The input is environmental sensing data aimed at detecting HEs (Hazard Elements), IMs (Initiating Mechanisms, triggers of the accident), and T/T (Target and Threat) in the.Ericson's Hazard Theory. 10According to the Ericson's Hazard Theory, the accident (mishap) occurs when HE, IMs and T/T are appeared at the same time in specific timing.The sensing data is mainly time-series images of the environment and distance information to the objects and parts of human body.The main output is the result of risk analysis and the level of risk as quantitative information.The risk is presented with following formula: (1) (2)   .Eq. (1) shows that the risk is multiplication of the severity of the harm (S) and the probability of the harm (Ph).Eq. ( 2) displays the probability of the harm (Ph) is the combination of following: frequency of exposure to hazards (F), the probability of occurrence of hazardous event (Ps), the possibility of avoiding or limiting the harm (A).Furthermore, according to the result of risk analysis, we generate a painting of the future by blending a representative hazardous image per each risk category with a current image.

Risk reduction
2][13] In addition to the above, risk reduction learns from the previous incidents and accidents, 14 which improves the environment, for example, removes hazard elements, supports avoiding hazardous situation and stops the trigger of incidents and accidents.
This paper proposes online real-time risk reduction based on the deep reinforcement learning.Figure 1 illustrates that inputs are following environmental data: the elderly's outer factors and inner factors.Outer factors are some sensing data, for example, images from stereo camera and laser range finder.Inner factors are elderly's physical condition, for example, breathing.The other input is the risk level that is the result of risk analysis as a negative reward.
This deep neural network is trained with the Q-learning algorithm, [4][5] with Adam 15 for optimization of parameters.Moreover, this network uses batch normalization for improving the learning rate and robustness of initial value of weight. 16he output is the action as the risk reduction measures.

Experiment
We performed an experiment on a simulation.Figure 2 shows a visualization of an environment and an agent.A scenario is that an elderly person who is assisted by mobile robot goes to the toilet at a certain interval in the hospital.Sometimes the floor is partially wet.The mobile robot (agent) assists preventing from slip-induced fall for the elderly.
Figure 3 presents a process how the reward changes during training on the simulation.The increase of the reward indicates the reduction of the risk.We evaluate the effects of our approach through these change of graphs.We obtained the result of simulation that the

Conclusion
In this paper we proposed a fall risk reduction measures for the elderly using mobile robots based on the deep reinforcement learning, and have presented its usability.The results of experiments suggest that the applicability of our method to other cases of the fall and other cases of accidents.

Fig. 1 .
Fig. 1.Reinforcement learning for fall risk reduction of the elderly.

Fig. 4 .
Fig. 4. A deep neural network model generating a risk reduction measures.

Fig. 3 .
Fig. 3. Change of reward during training on the simulation.

Fig. 2 .
Fig.2.A simulation environment of experiments.ER is an elderly person assisted by a robot as an agent.E1 is another elderly person.From a hospital room, ER goes to the toilet at a certain interval by walking on the corridor between Wall_1 and Wall_2.E1 also goes to the same toilet.Sometimes it is wet on the floor and ER detects wet area.The fan-shaped area shows the sensing rage of ER and E1.