Mouse Cursor-like Control System in Consideration of the DC-EOG Signals using EOG-sEMG Human Interface

Patients who suffered with the limb disorders cannot take care of themselves by communication barrier. However, it is possible to improve the communication abilities of the patients by using biological signal such as ocular potential and muscle potential which is caused by moving eyes or facial muscle. Therefore, we have developed the human interface using the electrooculogram (EOG) and the facial surface electromyogram (sEMG) signals which can control PC. However, this system could not control the mouse cursor in accordance with the direction and magnitude of the movement of eyes to control PC smoothly and intuitively. Thus, we proposed a new mouse cursor-like control system in consideration of the DC elements of EOG signals using the EOG-sEMG human interface. Our proposed method has both drift and blink countermeasures which had better performance in mouse cursor control, and all subjects could control the mouse cursor for both moving and clicking flexibility.


Introduction
There are a large number of persons with disabilities around the world currently.However, it is possible to improve the communication abilities of patients who suffer from diseases by using a biological signal such as P -520 ocular potential and muscle potential caused by movement of the eyes or facial muscle 1 .The aim of this paper is to present a system using biological signals that can efficiently be used as a human computer interface.The authors conducted joint research with multiple companies to develop an interface device called "FARG" that can detect biological signals such as ocular potential and muscle potential on the face as shown in Fig. 1.A conventional method can control a PC as a switch with 7 patterns: 4 patterns of eyes moving in 4 directions (up, down, left, and right), and 3 patterns of facial muscle operation (left blink, right blink, and bite) 2 .However, the purpose of this paper is to control the mouse cursor in accordance with the direction and magnitude of movement of the eyes to control the PC more smoothly and intuitively by using the proposed methods.

Proposed Method
As we introduced in section 1, our method could recognize 7 patterns of biological signals.However, the patients do not have such great m to operate the PC under their mind except by using support software 3 .In this paper, we propose a modified method to move the mouse cursor to the vertical direction and horizontal direction freely by using the EOG signals and the click by the EMG signals.We set the mouse cursor to click by a right blink and to double-click by biting.The mouse cursor was set to move 50 pixels to the right when the eyes moved to the right direction.In the case of the operation in the sequence of "move to right", "move to right", "bite", the operation performs double-click after the mouse cursor has moved 100 pixels to the right.In the modified method proposed, we defined the thresholds used in the basic method for recognizing the vertical direction and horizontal direction (up, down, left, and right) as the thresholds that show the EOG signals changes.When the patients gazed on the up, down, left, or right side of the screen.The formula used for computing the pixel of the mouse cursor is shown in Eq. ( 1), ( 2) and the update interval is set as 0.1 [sec].W is the width of the screen; the benchmark is changed into the pixel of the mouse cursor at 0.1 [sec] before (Xpre, Ypre).Further, we imported the blink counter-measure.

Experimental Method
In this study, we tried four experiments in order to evaluate the performance of the mouse cursor control system which we built by the methods we proposed.All four experiments are designed step by step in the following sequence: A) horizontal movements only, B) horizontal movements and clicking, C) both horizontal and vertical movements, D) practical application.

Experimental Method 1
In experiment 1, we set the desktop as shown in Fig. 2. We asked the subjects, who are 5 healthy males (subject A, B, C, D, E) in their 20s and 1 female (subject F) in her  20s, to focus on the center of the screen.Next, they moved their sight to the first icon 1, on the left side from the center and stopped on the icon for 1 sec, then moved to the next icon and stopped for 1 sec one by one from icons 1 to 8.After that, they continued moving to the icon on the right side from the center, from icons 9 to 16 on the right edge of the screen.We measured the time from first moving the mouse cursor until moving to the final icon 16.Each subject performed the same experiment 3 times.

Experimental Method 2
In experiment 2, we used the same desktop as in experiment 1.We asked the subjects, who are 2 healthy males (subjects G, H) in their and 1 female (F), to focus on the center of the screen.Then they moved their sight to the next icon on the left side from the center and clicked the icons one by one from 1 to 8.After that, they continued moving to the icon on the right side from the center and clicking the icons from 9 to 16 on the right edge of the screen.We measured the time from starting the moving mouse cursor until clicking the final icon, 16.Each subject performed the same experiment 3 times.

Experimental Method 3
In experiment 3, we set the desktop wallpaper as shown in Fig. 3 and cleared all icons.Next, we asked subjects G and H to focus on the 'start' point, then to move their sight to follow the arrows and do their best to finish one circle for 3 tasks at 5 min rest intervals.The EOG signals changed and the times used for one task were recorded as the evaluation standards.The size of the screen was 22 inches with full HD (1920*1080)

Experimental Method 4
In experiment 4, we tried to control Google Earth 4 as verification experiments using the mouse cursor control system that we built.We opened Google Earth and put the map of Japan on the center of the screen.Then we asked the subject to rotate the globe and put the map of Brazil on the center of the screen by the click and drop action of the mouse.Finally, subjects double left clicked the map of Brazil to zoom in the camera.The same subjects were asked to try the operations for 3 times in 5 min intervals.When the mouse cursor moved into the left or right 1/10 area to the sides of the screen, the mouse cursor control function paused until a left blink, right blink or bite were recognized by the EMG signals.

Results of Experiment 1
In experiment 1, subjects move their sight to the 16 icons one by one.The time includes 1 [sec] stopped on each icon.Everyone finished the experiments at one time, so it means the correct rate is 100%.The average time of 6 subjects for one task is 87.5 [sec].

Results of Experiment 2
Table 1 shows the time taken by 3 subjects in experiment 2. The average time of 3 subjects for one task is 111 [sec].As in Experiment 1, the influence of a blink caused the results of subject I to be the worst because the muscle potential caused unstable ocular potential.In addition, we consider one of the reasons is that the threshold is different due to individual differences.P -522

Results of Experiment 3
The experiments results of subjects G are shown in Figure 4.The left-top chart shows the ideal value, and the next three ones show the actual data of three tasks.From the results, we can see the influence of a blink was decreased, and with subject G, although there were mistakes that recognized a blink up in task 3, all of the tasks could finish matching the arrows roughly.Table 2 shows the pixels and time of 2 subjects with the modified method we proposed.

Results of Experiment 4
In experiment 4, we asked the subjects to control Google Earth by using the mouse cursor control system and following the steps in the verification experiments.The time that a subject took and the suspend count results are shown in Table 3.With subject G, the experiments could be finished smoothly without any pauses, and with subject H, the experiments were suspended once in the case that the mouse cursor moved from the left 1/10 area to the sides of screen.But in task 3, subject H could finish the experiments in the same time as did subject G.

Conclusion
In this paper, we introduced a PC control method using a biological signal measuring device that could be used for ALS patients.With the proposed method and the experiments results, it is possible to control a mouse cursor by moving the eyes and clicking by moving the facial muscle when using the mouse cursor control system we built.The patients also can control the application as they wish.We validated that the influence of drift and blink cannot be ignored simply, and those influences are also the difficult points in our research.The modified method having both drift and blink countermeasures had better performance of mouse cursor control, and all subjects could control the mouse cursor for horizontal moving and clicking flexibility.
y=0, and if y>H, y=H

Table 1 .
Time of 3 Subjects in Experiment 2.

Table 2 .
Pixels and T of 2 subjects in experiment 3.