Jakob Karolus
Thomas Kosch
Albrecht Schmidt
Paweł W. Wozniak
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Novel input methods for game design often excite users, especially if they extend the way one interacts with the system. Electromyography (EMG) has the inherent potential to provide an intuitive - yet challenging - input channel for interactive systems. While this difficulty in control often limits the scope of applications for EMG in most systems, we argue that these qualities are especially relevant for games and playful interaction. The inherently challenging qualities of EMG input make the modality a prime candidate for designing body-centric playful experiences. Yet, we still need to understand its limitations to create engaging rather than frustrating experiences for users. In this work, we investigate EMG's potential to support playful interaction through exploratory studies, deriving feasible game interactions based on EMG's technical constraints, and study their application in game design. Based on our findings, we highlight design implications and pitfalls to avoid when creating EMG-based entertainment systems.
Supplemental Material
Available for Download
zip
v6mhci190aux.zip (29 MB)
- emg_mounting_manual: guide explaining how to mount the electrodes and connect the EMG device - pilot_questionnaire_results: detailed results for the questionnaire in the pilot study (same questions as used in Study I, see paper) - playful_emg: video and corresponding subtitles of the submission
- Ashu Adhikari, Abraham M. Hashemian, Thinh Nguyen-Vo, Ernst Kruijff, Markus von der Heyde, and Bernhard E. Riecke. 2021. Lean to Fly : Leaning-Based Embodied Flying Can Improve Performance and User Experience in 3D Navigation. Frontiers in Virtual Reality , Vol. 2 (2021). https://www.frontiersin.org/article/10.3389/frvir.2021.730334Google Scholar
- Christoph Amma, Thomas Krings, Jonas Böer, and Tanja Schultz. 2015. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 929--938. https://doi.org/10.1145/2702123.2702501Google ScholarDigital Library
- D. T. Barry, K. E. Gordon, and G. G. Hinton. 1990. Acoustic and Surface EMG Diagnosis of Pediatric Muscle Disease. Muscle Nerve, Vol. 13, 4 (April 1990), 286--290. https://doi.org/10.1002/mus.880130403Google ScholarCross Ref
- Enrico Costanza, Samuel A. Inverso, and Rebecca Allen. 2005. Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, New York, NY, USA, 481--489. https://doi.org/10.1145/1054972.1055039Google ScholarDigital Library
- Enrico Costanza, Samuel A. Inverso, Rebecca Allen, and Pattie Maes. 2007. Intimate Interfaces in Action : Assessing the Usability and Subtlety of Emg-based Motionless Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 819--828. https://doi.org/10.1145/1240624.1240747Google ScholarDigital Library
- Kate C. Ewing, Stephen H. Fairclough, and Kiel Gilleade. 2016. Evaluation of an Adaptive Game That Uses EEG Measures Validated during the Design Process as Inputs to a Biocybernetic Loop. Front Hum Neurosci , Vol. 10 (May 2016), 223. https://doi.org/10.3389/fnhum.2016.00223Google ScholarCross Ref
- Florian 'Floyd' Mueller, Rakesh Patibanda, Richard Byrne, Zhuying Li, Yan Wang, Josh Andres, Xiang Li, Jonathan Marquez, Stefan Greuter, Jonathan Duckworth, and Joe Marshall. 2021. Limited Control Over the Body as Intriguing Play Design Resource. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1--16. https://doi.org/10.1145/3411764.3445744Google ScholarDigital Library
- Yue Gao and Regan L Mandryk. 2011. GrabApple: The design of a casual exergame. In International Conference on Entertainment Computing. Springer, 35--46.Google ScholarDigital Library
- Nadia Garcia-Hernandez, Karen Garza-Martinez, Vicente Parra-Vega, Armando Alvarez-Sanchez, and Laura Conchas-Arteaga. 2019. Development of an EMG-based exergaming system for isometric muscle training and its effectiveness to enhance motivation, performance and muscle strength. International Journal of Human-Computer Studies , Vol. 124 (2019), 44--55.Google ScholarCross Ref
- Mohammad Ghassemi, Kristen Triandafilou, Alex Barry, Mary Ellen Stoykov, Elliot Roth, Ferdinando A. Mussa-Ivaldi, Derek G. Kamper, and Rajiv Ranganathan. 2019. Development of an EMG-Controlled Serious Game for Rehabilitation. IEEE Trans Neural Syst Rehabil Eng , Vol. 27, 2 (Feb. 2019), 283--292. https://doi.org/10.1109/TNSRE.2019.2894102Google ScholarCross Ref
- R. a. R. C. Gopura and Kazuo Kiguchi. 2012. Application of Surface Electromyographic Signals to Control Exoskeleton Robots. IntechOpen. https://doi.org/10.5772/26389Google Scholar
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index ): Results of Empirical and Theoretical Research. In Human Mental Workload. North-Holland, Oxford, England, 139--183. https://doi.org/10.1016/S0166--4115(08)62386--9Google Scholar
- A. Hernandez Arieta, R. Katoh, H. Yokoi, and Y. Wenwei. 2006. Development of a Multi-DOF Electromyography Prosthetic System Using the Adaptive Joint Mechanism. https://doi.org/10.1533/abbi.2005.0060Google ScholarDigital Library
- Cedric Honnet, Hannah Perner-Wilson, Marc Teyssier, Bruno Fruchard, Jürgen Steimle, Ana C. Baptista, and Paul Strohmeier. 2020. PolySense : Augmenting Textiles with Electrical Functionality Using In-Situ Polymerization. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1--13. https://doi.org/10.1145/3313831.3376841Google ScholarDigital Library
- Donny Huang, Xiaoyi Zhang, T. Scott Saponas, James Fogarty, and Shyamnath Gollakota. 2015. Leveraging Dual-Observable Input for Fine-Grained Thumb Interaction Using Forearm EMG. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). Association for Computing Machinery, New York, NY, USA, 523--528. https://doi.org/10.1145/2807442.2807506Google ScholarDigital Library
- Alayna Hughes and Sergi Jorda. 2021. Applications of Biological and Physiological Signals in Commercial Video Gaming and Game Research : A Review. Frontiers in Computer Science , Vol. 3 (2021). https://www.frontiersin.org/article/10.3389/fcomp.2021.557608Google ScholarCross Ref
- Richard J. Jagacinski, Daniel W. Repperger, Martin S. Moran, Sharon L. Ward, and Betty Glass. 1980. Fitts' Law and the Microstructure of Rapid Discrete Movements. Journal of Experimental Psychology: Human Perception and Performance, Vol. 6, 2 (1980), 309--320. https://doi.org/10.1037/0096--1523.6.2.309Google ScholarCross Ref
- Jakob Karolus, Francisco Kiss, Caroline Eckerth, Nicolas Viot, Felix Bachmann, Albrecht Schmidt, and Paweł W. Wo'zniak. 2021. EMBody : A Data-Centric Toolkit for EMG-Based Interface Prototyping and Experimentation. Proc. ACM Hum.-Comput. Interac. , Vol. 5, EICS (2021), 195:1--195:29. https://doi.org/10.1145/3457142Google ScholarDigital Library
- Jakob Karolus, Hendrik Schuff, Thomas Kosch, Paweł W. Wozniak, and Albrecht Schmidt. 2018. EMGuitar : Assisting Guitar Playing with Electromyography. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). Association for Computing Machinery, Hong Kong, China, 651--655. https://doi.org/10.1145/3196709.3196803Google ScholarDigital Library
- Frederic Kerber, Michael Puhl, and Antonio Krüger. 2017. User-Independent Real-Time Hand Gesture Recognition Based on Surface Electromyography. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3098279.3098553Google ScholarDigital Library
- J.F. Knight, C. Baber, A. Schwirtz, and H.W. Bristow. 2002. The Comfort Assessment of Wearable Computers. In Proceedings. Sixth International Symposium on Wearable Computers,. 65--72. https://doi.org/10.1109/ISWC.2002.1167220Google ScholarCross Ref
- John W. Krakauer and Reza Shadmehr. 2006. Consolidation of Motor Memory. Trends Neurosci. , Vol. 29, 1 (Jan. 2006), 58--64. https://doi.org/10.1016/j.tins.2005.10.003Google Scholar
- Dennis Krupke, Paul Lubos, Lena Demski, Jonas Brinkhoff, Gregor Weber, Fabian Willke, and Frank Steinicke. 2016. Control Methods in a Supernatural Flight Simulator. In 2016 IEEE Virtual Reality (VR ). 329--329. https://doi.org/10.1109/VR.2016.7504788Google Scholar
- G.M. Lyons, P. Sharma, M. Baker, S. O'Malley, and A. Shanahan. 2003. A Computer Game-Based EMG Biofeedback System for Muscle Rehabilitation. In Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No. 03CH37439 ), Vol. 2. 1625--1628 Vol.2. https://doi.org/10.1109/IEMBS.2003.1279682Google ScholarCross Ref
- Regan L. Mandryk, M. Stella Atkins, and Kori M. Inkpen. 2006. A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1027--1036. https://doi.org/10.1145/1124772.1124926Google ScholarDigital Library
- Joe Marshall, Alexandru Dancu, and Florian "Floyd" Mueller. 2016. Interaction in Motion: Designing Truly Mobile Interaction. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS '16). Association for Computing Machinery, New York, NY, USA, 215--228. https://doi.org/10.1145/2901790.2901844Google ScholarDigital Library
- Soroosh Mashal, Gerold Hoelzl, and Matthias Kranz. 2019. Valkyrie Project : Flying Immersion in Virtual Reality. In 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games ). 1--4. https://doi.org/10.1109/VS-Games.2019.8864545Google ScholarCross Ref
- Daniel Medeiros, Mauricio Sousa, Alberto Raposo, and Joaquim Jorge. 2020. Magic Carpet : Interaction Fidelity for Flying in VR. IEEE Transactions on Visualization and Computer Graphics, Vol. 26, 9 (Sept. 2020), 2793--2804. https://doi.org/10.1109/TVCG.2019.2905200Google ScholarCross Ref
- Roberto Merletti and Dario Farina. 2016. Surface Electromyography : Physiology, Engineering and Applications. John Wiley & Sons.Google Scholar
- Florian Mueller, Stefan Agamanolis, and Rosalind Picard. 2003. Exertion Interfaces: Sports over a Distance for Social Bonding and Fun. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). Association for Computing Machinery, New York, NY, USA, 561--568. https://doi.org/10.1145/642611.642709Google ScholarDigital Library
- Lennart Erik Nacke, Michael Kalyn, Calvin Lough, and Regan Lee Mandryk. 2011. Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, New York, NY, USA, 103--112. https://doi.org/10.1145/1978942.1978958Google ScholarDigital Library
- Jakob Nielsen. 1993. Usability Engineering. Morgan Kaufmann Publishers Inc. , San Francisco, CA, USA.Google ScholarDigital Library
- Anton Nijholt, Danny Plass-Oude Bos, and Boris Reuderink. 2009. Turning Shortcomings into Challenges: Brain textendash Computer Interfaces for Games. Entertainment Computing , Vol. 1, 2 (April 2009), 85--94. https://doi.org/10.1016/j.entcom.2009.09.007Google ScholarCross Ref
- Aditya Shekhar Nittala, Arshad Khan, Klaus Kruttwig, Tobias Kraus, and Jürgen Steimle. 2020a. PhysioSkin : Rapid Fabrication of Skin-Conformal Physiological Interfaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/3313831.3376366Google ScholarDigital Library
- Aditya Shekhar Nittala, Arshad Khan, Klaus Kruttwig, Tobias Kraus, and Jürgen Steimle. 2020b. PhysioSkin : Rapid Fabrication of Skin-Conformal Physiological Interfaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1--10. https://doi.org/10.1145/3313831.3376366Google ScholarDigital Library
- D. Peleg, E. Braiman, E. Yom-Tov, and G. F. Inbar. 2002. Classification of Finger Activation for Use in a Robotic Prosthesis Arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 10, 4 (Dec. 2002), 290--293. https://doi.org/10.1109/TNSRE.2002.806831Google ScholarCross Ref
- Monica Perusquía-Hernández, Saho Ayabe-Kanamura, Kenji Suzuki, and Shiro Kumano. 2019. The Invisible Potential of Facial Electromyography : A Comparison of EMG and Computer Vision When Distinguishing Posed from Spontaneous Smiles. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3290605.3300379Google ScholarDigital Library
- M. Perusquía-Hernández, T. Enomoto, T. Martins, M. Otsuki, H. Iwata, and K. Suzuki. 2017. Embodied Interface for Levitation and Navigation in a 3D Large Space. In Proceedings of the 8th Augmented Human International Conference (AH '17). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3041164.3041173Google ScholarDigital Library
- Angkoon Phinyomark, Pornchai Phukpattaranont, and Chusak Limsakul. 2012. Feature Reduction and Selection for EMG Signal Classification. Expert Systems with Applications , Vol. 39, 8 (June 2012), 7420--7431. https://doi.org/10.1016/j.eswa.2012.01.102Google ScholarDigital Library
- Angkoon Phinyomark and Erik Scheme. 2018. EMG Pattern Recognition in the Era of Big Data and Deep Learning. BDCC, Vol. 2, 3 (Aug. 2018), 21. https://doi.org/10.3390/bdcc2030021Google ScholarCross Ref
- Alejandro Lopez Rincon, Hiroshi Yamasaki, and Shingo Shimoda. 2016. Design of a Video Game for Rehabilitation Using Motion Capture, EMG Analysis and Virtual Reality. In 2016 International Conference on Electronics, Communications and Computers (CONIELECOMP ). 198--204. https://doi.org/10.1109/CONIELECOMP.2016.7438575Google ScholarCross Ref
- Yvonne Rogers. 2006. Moving on from Weiser 's Vision of Calm Computing : Engaging UbiComp Experiences. In UbiComp 2006: Ubiquitous Computing (Lecture Notes in Computer Science ), , Paul Dourish and Adrian Friday (Eds.). Springer, Berlin, Heidelberg, 404--421. https://doi.org/10.1007/11853565_24Google ScholarDigital Library
- Wataru Sakoda, Ramin Tadayon, Yusuke Kishishita, Masataka Yamamoto, and Yuichi Kurita. 2020. Ski exergame for squat training to change load based on predicted Locomotive risk level. In 2020 IEEE/SICE International Symposium on System Integration (SII). IEEE, 289--294.Google ScholarCross Ref
- T Scott Saponas, Desney S. Tan, Dan Morris, and Ravin Balakrishnan. 2008. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 515--524. https://doi.org/10.1145/1357054.1357138Google ScholarDigital Library
- T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making Muscle-computer Interfaces More Practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 851--854. https://doi.org/10.1145/1753326.1753451Google ScholarDigital Library
- E. Scheme and K. Englehart. 2014. On the Robustness of EMG Features for Pattern Recognition Based Myoelectric Control; A Multi-Dataset Comparison. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Chicago, IL, 650--653. https://doi.org/10.1109/EMBC.2014.6943675Google Scholar
- Mel Slater, Martin Usoh, and Anthony Steed. 1995. Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Trans. Comput.-Hum. Interact. , Vol. 2, 3 (Sept. 1995), 201--219. https://doi.org/10.1145/210079.210084Google ScholarDigital Library
- Ludger van Dijk, Corry K. van der Sluis, Hylke W. van Dijk, and Raoul M. Bongers. 2016. Learning an EMG Controlled Game : Task-Specific Adaptations and Transfer. PLoS One, Vol. 11, 8 (Aug. 2016), e0160817. https://doi.org/10.1371/journal.pone.0160817Google ScholarCross Ref
- Scott R. Vrana. 1993. The Psychophysiology of Disgust: Differentiating Negative Emotional Contexts with Facial EMG. Psychophysiology, Vol. 30, 3 (May 1993), 279--286. https://doi.org/10.1111/j.1469--8986.1993.tb03354.xGoogle Scholar
- Pawel W. Wozniak, Jakob Karolus, Florian Lang, Caroline Eckerth, Johannes Schöning, Yvonne Rogers, and Jasmin Niess. 2021. Creepy Technology: What Is It and How Do You Measure It?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, May 08--13, 2021. ACM, 13. https://doi.org/10.1145/3411764.3445299Google ScholarDigital Library
- Yaying Zhang, Bernhard E. Riecke, Thecla Schiphorst, and Carman Neustaedter. 2019. Perch to Fly : Embodied Virtual Reality Flying Locomotion with a Flexible Perching Stance. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). Association for Computing Machinery, New York, NY, USA, 253--264. https://doi.org/10.1145/3322276.3322357Google ScholarDigital Library
Index Terms
- Imprecise but Fun: Playful Interaction Using Electromyography
Recommendations
Biomechanics and Kinematic Responses of the Upper Extremity during Isotonic and Isokinetic Wrench-Turning Tasks
The purpose of this study was to investigate the impact of using a wrench under isotonic constant torque and isokinetic constant speed task modes TM at three work surface inclinations WSI 0ï ź, 45ï ź, and 90ï ź on the biomechanical muscle activity and ...
NeuroRehab + the "Fun" factor
Sandbox '10: Proceedings of the 5th ACM SIGGRAPH Symposium on Video GamesRehabilitation processes for neurological traumas have traditionally relied on physical therapy that involves repetitive exercises. While proven to be effective, these processes can become monotonous for patients and labor-intensive for therapists. ...
Optimal Placement of Multi-Channels sEMG Electrod for Finger Movement Classification
ICBBE '17: Proceedings of the 2017 4th International Conference on Biomedical and Bioinformatics EngineeringThis research aims to propose the optimal electrode positions for surface EMG by using a modern gesture control device called MYO armband. Seven healthy volunteers participated in this research. The sEMG signal was collected form three different ...
Comments