Skip to main content
SpringerLink
Log in

Non-verbal Sound Detection by Humanoid Educational Robots in Game-Based Learning. Multiple-Talker Tracking in a Buzzer Quiz Game with the Pepper Robot

  • Conference paper
  • First Online:
Robotics in Education (RiE 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 747))

Included in the following conference series:

  • 199 Accesses

Abstract

While most research and development in social human–robot interaction, including interactions of learners with educational robots, has focused on verbal communication through speech (synthesis and recognition), research on non-verbal signals has just started to gain attention. Non-verbal sounds such as clapping, laughter, and mechanical sounds are often overlooked elements in design and research in social human–robot interaction. This paper presents the design of an educational buzzer quiz game for groups of learners, which is facilitated by the humanoid educational robot Pepper, detecting non-verbal sounds as part of the game-play. The paper describes the scenario and a prototype of a robot-supported buzzer quiz game in classroom settings, outlines the challenges in the real-time detection of a mixture of sounds, and presents the approach to real-time auditory and visual multiple-talker tracking, which has been applied in the buzzer quiz game with the Pepper robot.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pelikan, H., Robinson, F.A., Keevallik, L., Velonaki, M., Broth, M., Bown, O.: Sound in human-robot interaction. In: HRI ‘21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, pp. 706–708 (2021)

    Google Scholar 

  2. Trouvain, J., Truong, K.P.: Comparing non-verbal vocalisations in conversational speech corpora. In: Devillers, L., Schuller, B., Batliner, A., Rosso, P., Douglas-Cowie, E., Cowie, R., Pelachaud, C. (eds.) Proceedings of the 4th International Workshop on Corpora for Research on Emotion Sentiment and Social Signals (ES3 2012), pp. 36–39. European Language Resources Association (ELRA) (2012)

    Google Scholar 

  3. Yilmazyildiz, S., Read, R., Belpaeme, T., Verhelst, W.: Review of semantic-free utterances in social human-robot interaction. Int. J. Hum.-Comput. Interact. 32, 63–85 (2016)

    Article  Google Scholar 

  4. Aylett, M.P., Vazquez-Alvarez, Y., Butkute, S.: Creating robot personality: effects of mixing speech and semantic free utterances. In: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (2020)

    Google Scholar 

  5. Lascio, E.D., Gashi, S., Santini, S.: Laughter recognition using non-invasive wearable devices. In: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare (2019)

    Google Scholar 

  6. Rychlowska, M., McKeown, G., Sneddon, I., Curran, W.: Laughter during cooperative and competitive games. In: SMILA (2022)

    Google Scholar 

  7. Gillick, J., Deng, W.H., Ryokai, K., Bamman, D.: Robust laughter detection in noisy environments. In: Interspeech (2021)

    Google Scholar 

  8. Tennent, H., Moore, D.J., Jung, M.F., Ju, W.: Good vibrations: how consequential sounds affect perception of robotic arms. In: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (ROMAN), pp. 928–935. IEEE, Lisbon (2017)

    Google Scholar 

  9. Moore, D.J., Tennent, H., Martelaro, N., Ju, W.: Making noise intentional: a study of servo sound perception. In: 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 12–21 (2017)

    Google Scholar 

  10. Khan, M.M., Tasneem, N., Marzan, Y.: ‘Fastest finger first—educational quiz buzzer’ using Arduino and seven segment display for easier detection of participants. In: 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC), pp. 1093–1098 (2021)

    Google Scholar 

  11. Dengri, C., Gill, A., Chopra, J., Dengri, C., Koritala, T., Khedr, A., Korsapati, A.R., Adhikari, R., Jain, S., Zec, S., Chand, M., Kashyap, R., Pattan, V., Khan, S.A., Jain, N.K.: A review of the quiz, as a new dimension in medical education. Cureus 13 (2021)

    Google Scholar 

  12. Andrés, B., Sanchis, R., Poler, R.: Quiz game applications to review the concepts learnt in class: an application at the University context (2015)

    Google Scholar 

  13. Kenney, K., Bailey, H.: Low-stakes quizzes improve learning and reduce overconfidence in college students. J. Sch. Teach. Learn. 21(2), 79–92 (2021)

    Google Scholar 

  14. Yokomoto, C.F., Ware, R.: Variations of the group quiz that promote collaborative learning. In: Proceedings Frontiers in Education 1997 27th Annual Conference. Teaching and Learning in an Era of Change, vol. 1, pp. 552–557 (1997)

    Google Scholar 

  15. Deterding, S., Khaled, R., Nacke, L.E., Dixon, D.: Gamification: toward a definition. In: Proceedings of CHI 2011. Workshop on Gamification, Vancouver, BC, 7–12 May 2011, pp. 12–15

    Google Scholar 

  16. Chou, Y.: Actionable Gamification: Beyond Points, Badges, and Leaderboards. Leanpub (2016)

    Google Scholar 

  17. Pawelka, F., Wollmann, T., Stöber, J., Lam, T.: Successful learning through gamified E-learning (Erfolgreiches Lernen durch gamifiziertes E-Learning). In: Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI), pp. 2353–2364 (2014)

    Google Scholar 

  18. Dennis, J., Tran, H., Li, H.: Spectrogram image feature for sound event classification in mismatched conditions. IEEE Signal Process. Lett. 18(2), 130–133 (2011)

    Article  Google Scholar 

  19. Gerosa, L., Valenzise, G., Tagliasacchi, M., Antonacci, F., Sarti, A.: Scream and gunshot detection in noisy environments. In: Proceeding of the European Signal Process Conference, pp. 1216–1220 (2007)

    Google Scholar 

  20. Chu, S., Narayanan, S., Kuo, C.: Environmental sound recognition with time–frequency audio features. IEEE Trans. Audio Speech Lang. Process. 17(6), 1142–1158 (2009)

    Article  Google Scholar 

  21. Lyon, R.: Machine hearing: an emerging field [exploratory DSP]. IEEE Signal Process. Mag. 27(5), 131–139 (2010)

    Article  Google Scholar 

  22. Muhammad, G., Alghathbar, K.: Environment recognition from audio using MPEG-7 features. In: Proceedings of the International Conference on Embedded and Multimedia Computing (EM-Com), pp. 1–6 (2009)

    Google Scholar 

  23. Tran, H., Li, H.: Sound event recognition with probabilistic distance SVMs. IEEE Trans. Audio Speech Lang. Process. 19(6), 1556–1568 (2011)

    Article  Google Scholar 

  24. Costa, Y.M., Oliveira, L., Koerich, A.L., Gouyon, F., Martins, J.: Music genre classification using LBP textural features. Signal Process. 92(11), 2723–2737 (2012)

    Article  Google Scholar 

  25. Buchem, I., Mc Elroy, A., Tutul, R.: Designing and programming game-based learning with humanoid robots: a case study of the multimodal “Make or Do” English grammar game with the Pepper robot. In: ICERI2022 Proceedings, 15th Annual International Conference of Education, Research and Innovation (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Berlin University of Applied Sciences, Luxemburger Str. 10, 13353, Berlin, Germany

    Ilona Buchem, Rezaul Tutul & André Jakob

  2. Humboldt University, Unter den Linden 6, 10099, Berlin, Germany

    Niels Pinkwart

Authors
  1. Ilona Buchem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rezaul Tutul
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. André Jakob
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Niels Pinkwart
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilona Buchem .

Editor information

Editors and Affiliations

  1. Slovak University of Technology (STU), Bratislava, Slovakia

    Richard Balogh

  2. Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

    David Obdržálek

  3. University of Cyprus, Nicosia, Cyprus

    Eftychios Christoforou

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Buchem, I., Tutul, R., Jakob, A., Pinkwart, N. (2023). Non-verbal Sound Detection by Humanoid Educational Robots in Game-Based Learning. Multiple-Talker Tracking in a Buzzer Quiz Game with the Pepper Robot. In: Balogh, R., Obdržálek, D., Christoforou, E. (eds) Robotics in Education. RiE 2023. Lecture Notes in Networks and Systems, vol 747. Springer, Cham. https://doi.org/10.1007/978-3-031-38454-7_16

Download citation

Publish with us

Policies and ethics

Search

Navigation