Brandi House
ABSTRACT
We present a system whereby the human voice may specify continuous control signals to manipulate a simulated 2D robotic arm and a real 3D robotic arm. Our goal is to move towards making accessible the manipulation of everyday objects to individuals with motor impairments. Using our system, we performed several studies using control style variants for both the 2D and 3D arms. Results show that it is indeed possible for a user to learn to effectively manipulate real-world objects with a robotic arm using only non-verbal voice as a control mechanism. Our results provide strong evidence that the further development of non-verbal voice controlled robotics and prosthetic limbs will be successful.
- BILMES, J., ET AL. The Vocal Joystick: A voice-based human-computer interface for individuals with motor impairments. In Proc. Human Language Tech./Epirical Methods in Natural Language Processing (Oct. 2005), pp. 995--1002. Google Scholar
Digital Library
- CHOI, Y. S., ANDERSON, C. D., GLASS, J. D., AND KEMP, C. C. Laser pointers and a touch screen: intuitive interfaces for autonomous mobile manipulation for the motor impaired. In ASSETS (Oct. 2008), pp. 225--232. Google ScholarDigital Library
- HARADA, S., LANDAY, J., MALKIN, J., LI, X., AND BILMES, J. The Vocal Joystick: Evaluaton of voice-based cursor control techniques. In ASSETS (Oct. 2006), pp. 27--34. Google ScholarDigital Library
- HARADA, S., WOBBROCK, J., AND LANDAY, J. VoiceDraw: A hands-free voice-driven drawing application for people with motor impairments. In ASSETS (Oct. 2007), pp. 27--34. Google ScholarDigital Library
- HARADA, S., WOBBROCK, J. O., MALKIN, J., BILMES, J., AND LANDAY, J. A. Longitudinal study of people learning to use continuous voice-based cursor control. In CHI (Apr. 2009). Google ScholarDigital Library
- IGARASHI, T., AND HUGHES, J. Voice as sound: using non-verbal voice input for interactive control. In UIST (2001), pp. 155--156. Google ScholarDigital Library
- KIM, S.-P., SIMERAL, J. D., HOCHBERG, L. R., DONOGHUE, J. P., AND BLACK, M. J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. Journal of Neural Engineering 5, 4 (2008), 455--476.Google ScholarCross Ref
- LI, X., MALKIN, J., BILMES, J., HARADA, S., AND LANDAY, J. An online adaptive filtering algorithm for the Vocal Joystick. In Interspeech (Pittsburgh, PA, Sept. 2006).Google Scholar
- LI, Y., WANG, C., ZHANG, H., AND GUAN, C. An eeg-based bci system for 2d cursor control. IEEE Int'l Joint Conf. on Neural Networks (June 2008), 2214--2219.Google Scholar
- MALKIN, J., LI, X., AND BILMES, J. Energy and loudness for speed control in the Vocal Joystick. In Automatic Speech Recognition and Understanding (Dec. 2005), pp. 409--414.Google ScholarCross Ref
- MARTENS, C., RUCHEL, N., LANG, O., AND GRASER, A. A FRIEND for assisting handicapped people. IEEE Robotics and Automation Magazine 8, 1 (2001), 57--65.Google ScholarCross Ref
- MIHARA, Y., SHIBAYAMA, E., AND TAKAHASHI, S. The Migratory Cursor: Accurate speech-based cursor movement by moving multiple ghost-cursors using non-verbal vocalization. In ASSETS (Oct. 2005). Google ScholarDigital Library
- REICHENSPURNER, H., ET AL. Use of the voice-controlled and computer-assisted surgical system ZEUS for endoscopic coronary artery bypass grafting. J. Thoracic and Cardiovascular Surgery 118 (1999), 11--16.Google ScholarCross Ref
- SHENOY, P., KRAULEDAT, M., BLANKHERTZ, B., RAO, R., AND MUELLER, K.-R. Towards adaptive classification for BCI. J. Neural Eng. 3 (2006), R13--R23.Google ScholarCross Ref
- SPORKA, A. J., AND SLAV´IK, P. Vocal control of a radio-controlled car. ACM SIGACCESS Accessibility and Computing, 91 (2008), 3--8. Google ScholarDigital Library
- STEVENS, S. S., VOLKMANN, J., AND NEWMAN, E. B. A scale for the measurement of the psychological magnitude pitch. J. of Acoustical Society of America 8, 3 (1937), 185--190.Google ScholarCross Ref
- WANG, L.-C., AND CHEN, C. A combined optimization method for solving the inverse kinematics problem of mechanical manipulators. IEEE Trans. on Robotics and Automation 7, 4 (1991), 489--499.Google ScholarCross Ref
- WELMAN, C. Inverse kinematics and geometric constraints for articulated figure manipulation. Master's thesis, Simon Fraser University, Burnaby, BC, 1993.Google Scholar
- WON, S. Y., LEE, D.-I., AND SMITH, J. Humming control interface for hand-held devices. In ASSETS (Oct. 2007), pp. 259--260. Google ScholarDigital Library
Index Terms
- The VoiceBot: a voice controlled robot arm
Recommendations
The vocal joystick:: evaluation of voice-based cursor control techniques
Assets '06: Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibilityMouse control has become a crucial aspect of many modern day computer interactions. This poses a challenge for individuals with motor impairments or those whose use of hands are restricted due to situational constraints. We present a system called the ...
Demo of VJ-Voicebot: control of robotic arm with the Vocal Joystick
Assets '07: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibilityWe explore the use of the Vocal Joystick (VJ) for robotic limb control using a small-scale robotic arm. The purpose of this research is to allow individuals with mobile disabilities to obtain greater independence by continuously controlling a robotic ...
Longitudinal study of people learning to use continuous voice-based cursor control
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe conducted a 2.5 week longitudinal study with five motor impaired (MI) and four non-impaired (NMI) participants, in which they learned to use the Vocal Joystick, a voice-based user interface control system. We found that the participants were able to ...
Comments