Amber Wagner
Jeff Gray
ABSTRACT
This paper introduces a voice-driven tool applied to an Initial Programming Environment (IPE), which gives motorically challenged individuals the opportunity to learn programming skills; in particular, our project allows programming by voice within Scratch. Although the native Scratch environment allows users to create a program by arranging graphical blocks logically, such visual languages are completely dependent on the use of a mouse and keyboard. This modality of interaction limits users based on physical abilities. Our solution is a tool, called Myna, which is a voice-driven Java application executed parallel to Scratch. Myna processes voice commands from the user, interprets those commands according to a pre-defined grammar, and simulates synonymous actions of a mouse and keyboard within Scratch. The resulting environment assists those with a motor disability (particularly young children) in learning the joy of programming. This extended abstract describes the motivation behind the project, a technical description of Myna, and defines the current work in progress.
- http://info.scratch.mit.edu/About_ScratchGoogle Scholar
- Gray, J., Lin, Y., and Zhang, J. Automating change evolution in model-driven engineering. In IEEE Computer, vol. 39, issue 2, (2006). Google ScholarDigital Library
- Harada, S., Wobbrock, J., Malkin, J., Bilmes, J., and Landay, J. Longitudinal study of people learning to use continuous voice-based cursor control. In Proc. Intl. Conf. on Human Factors in Computing Systems, Boston 2009, (2009), 347--356. Google ScholarDigital Library
- Java Robot Class: http://forum.codecall.net/java-tutorials/25923robot-class.htmlGoogle Scholar
- Leff, A. and Rayfield, J.T. Web-application development using the model/view/controller design pattern. EDOC 2001: 118--127. Google ScholarDigital Library
- National Institute of Neurological Disorders and Stroke. National Institutes of Health. http://www.ninds.nih.gov/disorders/disorder_index. htm.Google Scholar
- Scratch. http://scratch.mit.edu.Google Scholar
- WIMP Interfaces, Ashley George Taylor, Winter '97 http://www.cc.gatech.edu/classes/cs6751_97_winter/Topics/dialog-wimp/Google Scholar
- Wobbrock, J., Kane, S., Gajos, K., Harada, S., and Froehlich, J. Ability-based design: Concept, principles, and examples. In ACM Trans. On Accessible Computing, vol. 3, no. 3, article 9, (2011). Google ScholarDigital Library
Index Terms
- Programming by voice: a hands-free approach for motorically challenged children
Recommendations
The Appropriateness of Scratch and App Inventor as Educational Environments for Teaching Introductory Programming in Primary and Secondary Education
Teaching programming is a complex task. The task is even more challenging for introductory modules. There is an ongoing debate in the teaching community over the best approach to teaching introductory programming. Visual block-based programming ...
Programming by Voice: Exploring User Preferences and Speaking Styles
CUI '23: Proceedings of the 5th International Conference on Conversational User InterfacesProgramming by voice is a potentially useful method for individuals with motor impairments. Spoken programs can be challenging for a standard speech recognizer with a language model trained on written text mined from sources such as web pages. Having an ...
Strategies for concatenating recordings in a voice user interface: what we can learn from prosody
CHI EA '01: CHI '01 Extended Abstracts on Human Factors in Computing SystemsFindings from the present study show that different strategies for concatenating voice recordings significantly affect subjective preferences and memory of aurally presented information. Specifically, two different automatic telephone number ...
Comments