Vision-integrated Physiotherapy Service Robot Using Cooperating Two Arms

This study present the mechanical architecture, control system and other modules of a physiotherapy service robot which can treat degenerative disease and chronic disease of middle-aged and aged people by Chinese massage skill. The main body of the robot includes a massage adjustable bed, two 4-DOF robot arms and two massage hands that can accomplish various massage manipulations. Two arms cooperate to improve the massage efficiency and provide sufficient strength and enough reachable workspace for massage. The main control system is based on embedded module and the manipulators are controlled by a TRIO multi-axes motion controller. Physiological signal and massage pressure is detecting in real time in massage process to ensure a scientific and safe therapy. Vision System sends the recognized acupoint position to the master system to track the patient's body and the acupoint being massaged is displayed in real time by the 3D virtual model. The robot can execute ten massage manipulations, which make traditional Chinese massage can have a robot instead. The effectiveness for degenerative lumbago in middle-aged and aged is demonstrated by laboratory examination and clinical trial.


INTRODUCTION
Physiotherapy service robot is one of the service robots, which is developed for the clinical requirement of physical therapy and health care of degenerative disease and chronic disease of middle-aged and aged people. Physical therapy, e.g. massage, has a long history in China, but massage clinical treatment, a physical labor with strong intensity, is still made by experienced masseuse in hospital nowadays. There are some instruments simulating massage manipulations in the market right now, such as massager and massage chair, which can make a role of relaxing, care and eliminating fatigue, rather than treating diseases. Sanyo Electric Co. realized the massage manipulations of pinching first in 1996 [1], and P.
Minyong (Toyohashi University of Technology) et al., presented a four-fingers massage hand with pressure sensors in 2006 [2][3][4][5], which was able to create the movement and force of robot to behave as similar as the human's massage.
Tatsuya Teramae (Tottori University) et al., proposed the control strategy of the similar professional masseur's process. The appropriately impressed force is decided depending on the estimated elasticity, and the decided impressed force is realized by the impedance control [6].
Waseda University, also in Japan, has succeeded in developing the Waseda-Asahi oralrehabilitation robot in 2007, which is composed by two six-degrees of freedom arms with plungers attached at their end-effectors [7][8][9]. Konkuk University has developed a massage robot tapping human backs in 2007, which is composed of a chair, a 1-DOF (degree-of-freedom) torso, monitor face, and two 3-DOF arms and hands [10]. All of above researches have realized some massage action, but making treatment plan according to acupoint is not mentioned.
R. C. Luo, C. C. Chang et al., in Taiwan, developed a relative new and feasible application of multi-fingered robot hands except for the use as prosthesis and grasping applications. And the Surface electromyographic signals (EMS) measured from the trapezius muscles before and after the massage therapy are analyzed [11][12][13]. But, they only used EMS rather than more representative electrocardiograph (ECG) etc. In China, Lei Hu, et al. (Beihang University), presented a finger-kneading manipulation model based on pain threshold and a hybrid force position control strategy, and they used visual servo to track massage position [14][15]. Xiaoqin Yin and Yonggen Xu of Jiangsu University put forward one parallel mechanism of 2-DOF realizing 1 translation-1 rotation with single-opened-chain as a unit [16]. Jingguo Wang and Yangmin Li proposed a new application of 7-DOF redundant manipulator to do the massaging work for human feet with the tactile sensor equipped to the end-effector [17]. Above results are all theory research or laboratory products, and therapeutic effects have not verified through clinical test.
In this paper, we propose a physiotherapy service robot system, developed by the technology of Chinese massage, modern machinery, computer, control and mechanical-electrical integration, which can execute ten massage manipulations of finger-/palm-kneading, finger-/palm-rolling, pressing, pushing, tapping on lumbar and pinching, vibrating, patting on lower limb. It can calculate the coordinates of the acupoints and make treatment plan based on an expert system. It allows robot to replace human to do Chinese massage health care treatment, an ancient and meaningful work, so that the situation that health care resources are limited and an aging population structure society becomes more and more serious is effectively improved.

II.
MECHANICAL ARCHITECTURE III.

a. Main body
We have developed two models of physiotherapy service robot, whose names are JZMR-I and JZMR-II. The photos of them are shown in Figure 1. The main body, also called massage platform, of JZMR-I robot is 2300mm long, 1000mm wide and the height of the massage bed can adjust in the range of 715-870mm. Both right and left robot arm can move respectively, on the Y axis direction of movement distance to reach 900 mm, Y axis direction to reach 1900 mm, and Z axis to reach 300 mm. Rotating arm can rotate around the Z axis with the rotation range ±60°. The main body of JZMR-I robot is shown in Figure 3. In order to enhance the adaptability of space, JZMR-II robot reduced its size with 1900mm long, 550mm wide and the height of the massage bed is 500mm.

b. Robot arm
The robot arm architecture of JZMR-I robot combines rectangular type and joint type. Two arms working together are designed taking advantage of series structure and parallel structure because massage is a work that needs enough stiffness and accuracy. The robot arms mainly include three parts of horizontal movement mechanism, vertical movement mechanism and the deep movement mechanism, and the horizontal movement mechanism and vertical movement mechanism both are rectangular type. The greatest motion range of horizontal movement mechanism is 900mm and its highest speed can be up to 300mm/s. The greatest motion range of vertical movement mechanism is 300mm and its highest speed can be up to 80mm/s. While what the deep movement mechanism used is joint type mechanical structure with two axes named R1 and R2. The greatest motion angle of R1 is 150°and its highest speed can be up to 50°/s. The greatest motion angle of R2 is 270°and its highest speed can be up to 50°/s. The motion range of robot arm is shown in  The architecture of JZMR-II robot changed greatly. A gantry structure driven by X-axis robot arm is adopted. Another joint type robot arm that can rotate in a horizontal plane within the range of 270° is equipped on the gantry.  Figure 3, and Figure 5 shows the structure of robot hand and the distribution of motors in it. Positive and negative limit switches are installed for all axes, and software limiting is provided to ensure the movement of the robot arm in a safe range.
At the same time, the motion control system can send real-time pressure data to main control system by the 10 sensors equipped on the thumb, palm, roller, finger array, and wrist of the robot hand, so the main control system can adjust the massage force less than anyone being massaged ought to endure , then the security is enhanced. The diagram of motion control system is shown in Figure 6. The physiotherapy service robot needs the real-time physiological signals of the massaged man, including blood pressure, sphygmus, ECG, etc. The physiological signals can help the robot to judge whether the massage force is suitable and whether the process is appropriate for going on.
As the massage in process, the physiological signals are also captured and displayed on the screen of the main control console as some curves, so the change rule can be shown conveniently.
And the expert system will analyze these data to get the accurate characteristic, and guide the massage of the robot thereby.
Conceptual architecture of the module for ECG signal acquisition is shown in Figure 7: analog signal acquisition with electrodes, signal amplification, signal processing with a band-pass filter and analog to digital signal conversion. There are three types of electrodes, called wet, dry and insulated or capacitive electrodes, which are also used to capture other biosignals, such as EMG, EEG, etc. Wet disposable electrodes are nowadays prevailing in use. These are also used in our service robot. The signal from electrodes enters the amplifier system. The main issue with the ECG signal which reaches the amplifier is noise, which may have much greater amplitudes than the observed signal. Choosing the right amplifier largely depends on its noise characteristics. For example, common mode interference can be partially abolished by the instrumentation amplifier. To further eliminate interference, we use filters, which are set according to our understanding of the useful signal and interference. Since the frequency of most valid signal is between 0.05 Hz and 100 Hz, the cutoff frequencies of band-pass filter are set accordingly. Both, passive and active filters can be used in this case. Amplifying and filtering stage is followed by sampling and A/D conversion, to prepare the signal for digital processing. Theoretically the sampling frequency of 200 Hz is sufficient. However, in the event that we apply the analysis of ventricular late potentials and heart blocks, we must sample with a minimum of 1000 Hz, since the spectrum of these disorders may extend all the way to 500 Hz. Sampling frequency is therefore chosen depending on the complexity of the ECG signal analysis.
For signal monitoring the module must be complemented with system memory, microcontroller and transceiver unit by using wireless communication of Bluetooth. The rectangular region that the robot hand can reach is 1900mm×900mm, while the actual massage region is less than 1500mm×500mm because the massaged positions are all on back, waist, and legs. We hope the tracking error is less than 2mm, but we must make the error less than 1mm in theory to allow for random errors caused by many interference factors. Comparing the massage region with the tracking accuracy, a conclusion can make that the corresponding actual object for one pixel of the image captured by the camera should be less than 1mm. After investigation, we select the uEye series industrial camera made by IDS Company in German, which is equipped with a very sophisticated high-speed CMOS sensor (resolution ratio of 1600×1200). The corresponding object for one pixel is 0.218mm by using the uEye camera with a 5mm lens. The camera can continuously shoot at the speed of 18 frames per second, that is, each shooting interval time is about 55ms, which meets the project's requirements.

b. Acupoint detection
In order to facilitate matching, we choose the circular mark. The color of marks can be any, then the first work after acupoint detection program starting is to get the RGB value of the marks.
After the marks fixed on the body, take an image and find the marks, and right click on any position of the marks, then the coordinate and RGB value of the pixel are recorded. But if we take a number of pictures even for the same color, each of the RGB value will be slightly different, especially in the case of the light is not strong enough. So a certain threshold of the RGB value is set to increase the probability of captured the marks.
Some image preprocessing, such as filtering noise and image enhancement, must be make beforehand. The commonly spatial adaptive noise filter can reduce the noise effectively, but at  The coordinate frame is robot motion coordinate frame, defined as A coordinate frame, while coordinate frame is image frame, defined as B coordinate frame. Because we ignore the change of z coordinates, B coordinate frame can be described as the result of a series of transformation from A: rotation  about z axis, then translation a units about x axis, and translation b units about y axis [20].
So we can get the homogeneous coordinate transformation matrix of coordinate frame A to coordinate frame B shown as Formula (1).
In Formula (1)  it will cause the fatal accident. Connection-oriented socket is adopted to ensure the reliability.
The multi-axis controller made by TRIO is equipped with a TCP Server allowing multithreading. When powered on, the TCP Server will start listening "502" port to wait for connection requests from client application. The main control system will create a TCP Client trying to connect to the TCP Server during startup. A socket communication stream is built as the TCP Server accepted the connection request.
The internal storage space of TRIO controller is divided into two kinds named  When apply WM_COPYDATA message, window handle is specified and sent by the first message parameter, and the second message parameter is the indicator of data structure COPYDATASTRUCT which is related to same data. Among them, it only needs to give lpData to the first address of ready-to-send data, and data block length is designated by cbData. After message was sent, the receiver code in response function of WM_COPYDATA message receives the data block through the second parameter which was transmitted with message.
Visual positioning subsystem can be started and stopped by main control system, "SendMessage" Click "Teaching" button to enter into teaching interface (shown in Figure 10). Teaching is a process of confirming the coordinate of massage acupoint. Remote control mechanical arm to reach massage acupoint and records the coordinate of this acupoint.   Figure 11. The design process of physiotherapy service robot prototype is introduced in this paper. A massage hand which can integrate many massage skills is designed for this service robot and high safety new massage robot body mechanism is designed to achieve 10 typical TCM massage manipulations. Massage process can be adjusted according to real-time physiological signal and feedback from acupoint identification system, but the cost of this prototype is quite high, in order to promote the industrialization of massage robot, how to reduce cost of mechanical structure and control system needs to be considered. In addition, freedom degree of mechanical arm could be increased to achieve massage manipulation flexibly. Clinical course also found that teaching process is very long, so it could be improved to visual system teaching method, which would greatly improve efficiency.