Abstract
In recent years, as smartphones have become a part of life of young generation, opportunities to enter text using keyboards have decreased, and the speed of keyboard text entry has declined. However, it is still necessary for young children to use keyboards. In typing using a keyboard, the correspondence between keys and fingers is defined; however, there is no practical system for learning fingering to press keys with the correct fingers. A general typing learning software can only judge whether the correct input has been given or not and does not consider the correct fingering of the keys. If the system can evaluate fingering when learning typing, the user can concentrate on typing without having to judge the correctness of their own fingering. In previous research, there have been methods to acquire fingertip coordinates by attaching color stickers to fingers as markers or by using a distance camera, but these methods lack versatility. In this study, we used a monocular camera to acquire fingertip images on a keyboard. After obtaining images, we used MediaPipe to perform hand tracking and obtain the position of the fingertip. For each frame, the system calculates the distance between each key coordinate and each fingertip coordinate, determines that the closest finger hits the key, and feeds back the fingering information.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Miyoshi, Y., Kosakai, M.: A study of computer literacy education and keyboard input. Saitama Women’s Junior Colle. Res. Bull. 39, 85–92 (2019)
Damien, M., Goguey, A., Malacria, S., Casiez, G.: WhichFingers: identifying fingers on touch surfaces and keyboards using vibration sensors. In: Proceedings of the 30th annual ACM symposium on User Interface Software and Technology, pp. 41–48 (2017). https://doi.org/10.1145/3126594.3126619
Soga, S., Tamura, T., Taki, H.: Development of a typing skill learning environment with diagnosis and advice on fingering errors. Procedia Comput. Sci. 22, 737–744 (2013). https://doi.org/10.1016/j.procs.2013.09.155
Kishimoto, T., Imura, M.: Construction of a learning support system for proper fingering in typing practice. In: Proceedings of the 83rd National Convention of Information Processing Society of Japan, No. 1, pp. 713–714 (2021)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)
MediaPipe (2020). https://google.github.io/mediapipe/
MediaPipeHands (2020). https://google.github.io/mediapipe/solutions/hands.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kishimoto, T., Imura, M. (2022). Development of Fingering Learning Support System Using Fingertip Tracking from Monocular Camera. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2022 Posters. HCII 2022. Communications in Computer and Information Science, vol 1582. Springer, Cham. https://doi.org/10.1007/978-3-031-06391-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-06391-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06390-9
Online ISBN: 978-3-031-06391-6
eBook Packages: Computer ScienceComputer Science (R0)