Skip to main content
SpringerLink
Log in

Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13916))

Included in the following conference series:

Abstract

In embodied team learning activities, students are expected to learn to collaborate with others while freely moving in a physical learning space to complete a shared goal. Students can thus interact in various team configurations, resulting in increased complexity in their communication dynamics since unrelated dialogue segments can concurrently happen at different locations of the learning space. This can make it difficult to analyse students’ team dialogue solely using audio data. To address this problem, we present a study in a highly dynamic healthcare simulation setting to illustrate how spatial data can be combined with audio data to model embodied team communication. We used ordered network analysis (ONA) to model the co-occurrence and the order of coded co-located dialogue instances and identify key differences in the communication dynamics of high and low performing teams.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abitino, A., Pugh, S.L., Peacock, C.E., D’Mello, S.K.: Eye to eye: gaze patterns predict remote collaborative problem solving behaviors in triads. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds.) AIED 2022. LNCS, vol. 13355, pp. 378–389. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-11644-5_31

    Chapter  Google Scholar 

  2. Adrot, A., Bia Figueiredo, M.: “Lost in digitization’’: a spatial journey in emergency response and pragmatic legitimacy. In: de Vaujany, F.-X., Adrot, A., Boxenbaum, E., Leca, B. (eds.) Materiality in Institutions. TWG, pp. 151–181. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-97472-9_6

    Chapter  Google Scholar 

  3. Bowman, D., et al.: The mathematical foundations of epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 91–105. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_7

    Chapter  Google Scholar 

  4. Härgestam, M., Lindkvist, M., Brulin, C., Jacobsson, M., Hultin, M.: Communication in interdisciplinary teams: exploring closed-loop communication during in situ trauma team training. BMJ Open 3(10) (2013)

    Google Scholar 

  5. Ioannou, M., Georgiou, Y., Ioannou, A., Johnson, M.: On the understanding of students’ learning and perceptions of technology integration in low-and high-embodied group learning. 1, 304–311 (2019)

    Google Scholar 

  6. Kelly, S., Olney, A.M., Donnelly, P., Nystrand, M., D’Mello, S.K.: Automatically measuring question authenticity in real-world classrooms. Educ. Res. 47(7), 451–464 (2018)

    Article  Google Scholar 

  7. Liaw, S.Y., Rethans, J.J., Scherpbier, A., Piyanee, K.Y.: Rescuing a patient in deteriorating situations (rapids): a simulation-based educational program on recognizing, responding and reporting of physiological signs of deterioration. Resuscitation 82(9), 1224–1230 (2011)

    Article  Google Scholar 

  8. Lin, J., et al.: Is it a good move? Mining effective tutoring strategies from human-human tutorial dialogues. Futur. Gener. Comput. Syst. 127, 194–207 (2022)

    Article  Google Scholar 

  9. Ludlow, K., Churruca, K., Ellis, L.A., Mumford, V., Braithwaite, J.: Decisions and dilemmas: the context of prioritization dilemmas and influences on staff members’ prioritization decisions in residential aged care. Qual. Health Res. 31(7), 1306–1318 (2021)

    Article  Google Scholar 

  10. Marquart, C., Tan, Y., Cai, Z., Shaffer, D.W.: Ordered network analysis (2022). https://epistemic-analytics.gitlab.io/qe-packages/ona/cran/

  11. Martinez-Maldonado, R., Echeverria, V., Schulte, J., Shibani, A., Mangaroska, K., Buckingham Shum, S.: Moodoo: indoor positioning analytics for characterising classroom teaching. In: Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS (LNAI), vol. 12163, pp. 360–373. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52237-7_29

    Chapter  Google Scholar 

  12. McHugh, M.L.: Interrater reliability: the kappa statistic. Biochemia medica 22(3), 276–282 (2012)

    Article  MathSciNet  Google Scholar 

  13. Miller, K., Riley, W., Davis, S.: Identifying key nursing and team behaviours to achieve high reliability. J. Nurs. Manag. 17(2), 247–255 (2009)

    Article  Google Scholar 

  14. Mitri, D.D., et al. (eds.): Proceedings of the First International Workshop on Multimodal Artificial Intelligence in Education. CEUR Workshop Proceedings (2021)

    Google Scholar 

  15. O’Daniel, M., Rosenstein, A.H.: Professional communication and team collaboration. In: Patient Safety and Quality: An Evidence-Based Handbook for Nurses (2008)

    Google Scholar 

  16. Praharaj, S., Scheffel, M., Schmitz, M., Specht, M., Drachsler, H.: Towards collaborative convergence: quantifying collaboration quality with automated co-located collaboration analytics. In: 12th International Learning Analytics and Knowledge Conference, pp. 358–369 (2022)

    Google Scholar 

  17. Pugh, S.L., Subburaj, S.K., Rao, A.R., Stewart, A.E., Andrews-Todd, J., D’Mello, S.K.: Say what? Automatic modeling of collaborative problem solving skills from student speech in the wild. In: 14th International Conference on Educational Data Mining, pp. 55–67 (2021)

    Google Scholar 

  18. Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision. OpenAI Blog (2022)

    Google Scholar 

  19. Riley, W., Hansen, H., Gürses, A.P., Davis, S., Miller, K., Priester, R.: The nature, characteristics and patterns of perinatal critical events teams. In: Advances in Patient Safety: New Directions and Alternative Approaches (2008)

    Google Scholar 

  20. Salas, E., Sims, D.E., Burke, C.S.: Is there a “big five” in teamwork? Small Group Res. 36(5), 555–599 (2005)

    Google Scholar 

  21. Salas, E., Stevens, R., Gorman, J., Cooke, N.J., Guastello, S., von Davier, A.A.: What will quantitative measures of teamwork look like in 10 years? In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 59, no. 1, pp. 235–239 (2015)

    Google Scholar 

  22. Schlotterbeck, D., Uribe, P., Jiménez, A., Araya, R., van der Molen Moris, J., Caballero, D.: TARTA: teacher activity recognizer from transcriptions and audio. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds.) AIED 2021. LNCS (LNAI), vol. 12748, pp. 369–380. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78292-4_30

    Chapter  Google Scholar 

  23. Setti, F., Lanz, O., Ferrario, R., Murino, V., Cristani, M.: Multi-scale F-formation discovery for group detection. In: 2013 IEEE International Conference on Image Processing, pp. 3547–3551. IEEE (2013)

    Google Scholar 

  24. Shaffer, D.W., Collier, W., Ruis, A.R.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Anal. 3(3), 9–45 (2016). https://doi.org/10.18608/jla.2016.33.3

    Article  Google Scholar 

  25. Shaffer, D.W., et al.: Epistemic network analysis: a prototype for 21st-century assessment of learning. Int. J. Learn. Media 1(2) (2009)

    Google Scholar 

  26. Siebert-Evenstone, A.L., Irgens, G.A., Collier, W., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: In search of conversational grain size: modeling semantic structure using moving stanza windows. J. Learn. Anal. 4(3), 123–139 (2017)

    Google Scholar 

  27. Sorokowska, A., et al.: Preferred interpersonal distances: a global comparison. J. Cross Cult. Psychol. 48(4), 577–592 (2017)

    Article  Google Scholar 

  28. Southwell, R., et al.: Challenges and feasibility of automatic speech recognition for modeling student collaborative discourse in classrooms. In: 15th International Conference on Educational Data Mining, pp. 302–315 (2022)

    Google Scholar 

  29. Takizawa, P.A., Honan, L., Brissette, D., Wu, B.J., Wilkins, K.M.: Teamwork in the time of Covid-19. FASEB BioAdvances 3(3), 175–181 (2021)

    Article  Google Scholar 

  30. Tan, Y., Ruis, A.R., Marquart, C., Cai, Z., Knowles, M.A., Shaffer, D.W.: Ordered network analysis. In: Damşa, C., Barany, A. (eds.) ICQE 2022. CCIS, vol. 1785, pp. 101–116. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-31726-2_8

    Chapter  Google Scholar 

  31. Wang, Y., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: Simplification of epistemic networks using parsimonious removal with interpretive alignment. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 137–151. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_10

    Chapter  Google Scholar 

  32. Westli, H.K., Johnsen, B.H., Eid, J., Rasten, I., Brattebø, G.: Teamwork skills, shared mental models, and performance in simulated trauma teams: an independent group design. Scand. J. Trauma Resuscitation Emerg. Med. 18(1), 1–8 (2010)

    Article  Google Scholar 

  33. Yan, L., Zhao, L., Gasevic, D., Martinez-Maldonado, R.: Scalability, sustainability, and ethicality of multimodal learning analytics. In: 12th International Learning Analytics and Knowledge Conference, New York, NY, USA, pp. 13–23 (2022)

    Google Scholar 

  34. Zhao, L., et al.: METS: multimodal learning analytics of embodied teamwork learning. In: 13th International Learning Analytics and Knowledge Conference (2023, in press)

    Google Scholar 

  35. Zhao, L., et al.: Modelling co-located team communication from voice detection and positioning data in healthcare simulation. In: 12th International Learning Analytics and Knowledge Conference, New York, NY, USA, pp. 370–380 (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Monash University, Clayton, VIC, 3108, Australia

    Linxuan Zhao, Dragan Gašević, Lixiang Yan, Riordan Alfredo, Xinyu Li & Roberto Martinez-Maldonado

  2. University of Wisconsin-Madison, Madison, WI, USA

    Yuanru Tan & David Williamson Shaffer

Authors
  1. Linxuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanru Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dragan Gašević
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Williamson Shaffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lixiang Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Riordan Alfredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xinyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Roberto Martinez-Maldonado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linxuan Zhao .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  4. Despacho 3.01, UNED-Grupo de Investigación aDeNu, Madrid, Spain

    Olga C. Santos

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, L. et al. (2023). Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36272-9_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36271-2

  • Online ISBN: 978-3-031-36272-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation