skip to main content
10.1145/3290688.3290747acmotherconferencesArticle/Chapter ViewAbstractPublication PagesacswConference Proceedingsconference-collections
research-article

A Systematic Review of the Definition and Measurement of Engagement in Serious Games

Authors Info & Claims
Published:29 January 2019Publication History

ABSTRACT

A systematic review was designed to address the question of "What is engagement and how has the term been used, defined and measured in the context of serious games?". The goal of the review was to collect, evaluate, and analyse literature related to the definition and measurement of engagement in serious games published between in 1970 to 2015 across a broad range of disciplines. A total of 1390 papers were initially identified from the search criteria. These were reduced to 107 papers that directly assessed engagement in a serious game. These selected papers were then analysed to examine the use of the term 'engagement' in the serious games, the genres of the games studied, the various definitions of engagement and the methods used to measure different aspects of engagement. Three distinct types of engagement, related to behaviour, affect and cognition are found in the studies, along with a broad range of evaluation methods including interviews, questionnaires, physiological approaches, in game metrics, and time and performance on task.

References

  1. Koster, R. A theory of fun for game design. Scottsdale, AZ: Paraglyph Press, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bearman, M., Smith, C. D., Carbone, A., Slade, S., Baik, C., Hughes-Warrington, M. and Neumann, D. L. Systematic review methodology in higher education. Higher Education Research & Development, 31, 5 (2012), 625--640.Google ScholarGoogle ScholarCross RefCross Ref
  3. Sánchez González, L., García Rubio, F., Ruiz González, F. and Piattini Velthuis, M. Measurement in business processes: a systematic review. Business Process Management Journal, 16, 1 (2010), 114--134.Google ScholarGoogle ScholarCross RefCross Ref
  4. Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J. and Linkman, S. Systematic literature reviews in software engineering--a systematic literature review. Information and software technology, 51, 1 (2009), 7--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Šmite, D., Wohlin, C., Gorschek, T. and Feldt, R. Empirical evidence in global software engineering: a systematic review. Empirical software engineering, 15, 1 (2010), 91--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Smith, S. P., Blackmore, K. and Nesbitt, K. A meta-analysis of data collection in serious games research. Springer, City, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  7. Abt, C. C. Serious games. University press of America, 1987.Google ScholarGoogle Scholar
  8. Annetta, L. A., Cheng, M. T. and Holmes, S. Assessing twenty-first century skills through a teacher created video game for high school biology students. Research in Science and Technological Education, 28, 2 (2010), 101--114.Google ScholarGoogle ScholarCross RefCross Ref
  9. Ferko, A., Černeková, Z., Dadová, J., Major, V., Onačilová, D., Šikudová, E., Švarba, R., Valíková, M., Varhaníková, I., Vataha, M., Vesel, M. and Dušková, E. Schola ludus, serious games, and measurement of interestingness. City, 2011.Google ScholarGoogle Scholar
  10. Lomas, D., Stamper, J., Muller, R., Patel, K. and Koedinger, K. R. The effects of adaptive sequencing algorithms on player engagement within an online game. City, 2012.Google ScholarGoogle Scholar
  11. Codish, D. and Ravid, G. Detecting playfulness in educational gamification through behavior patterns. IBM Journal of Research and Development, 59, 6 (2015), 6: 1-6: 14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Zichermann, G. and Cunningham, C. Gamification by design: Implementing game mechanics in web and mobile apps. " O'Reilly Media, Inc. ", 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Filsecker, M. and Hickey, D. T. Incentives in educational games: A multilevel analysis of their impact on elementary students ' engagement and learning. City, 2013.Google ScholarGoogle Scholar
  14. Filsecker, M. and Hickey, D. T. A multilevel analysis of the effects of external rewards on elementary students' motivation, engagement and learning in an educational game. Computers & Education, 75 (2014), 136--148.Google ScholarGoogle ScholarCross RefCross Ref
  15. Lim, C. P. Global citizenship education, school curriculum and games: Learning Mathematics, English and Science as a global citizen. Computers and Education, 51, 3 (2008), 1073--1093. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Adams, S. S. and Holden, J. Games, ethics and engagement: Potential consequences of civic-minded game design and gameplay. City, 2010.Google ScholarGoogle Scholar
  17. Schuurink, E. L., Houtkamp, J. and Toet, A. Engagement and EMG in serious gaming: Experimenting with sound and dynamics in the levee patroller training game. City, 2008.Google ScholarGoogle Scholar
  18. Xhembulla, J., Rubino, I., Barberis, C. and Malnati, G. Intrigue at the Museum: Facilitating Engagement and Learning through a Location-Based Mobile Game. International Association for Development of the Information Society (2014).Google ScholarGoogle Scholar
  19. Pettit, R. K., McCoy, L., Kinney, M. and Schwartz, F. N. Student perceptions of gamified audience response system interactions in large group lectures and via lecture capture technology. BMC medical education, 15, 1 (2015), 92.Google ScholarGoogle Scholar
  20. Brom, C., Buchtová, M., Šisler, V., Děchtěrenko, F., Palme, R. and Glenk, L. M. Flow, social interaction anxiety and salivary cortisol responses in serious games: A quasi-experimental study. Computers & Education, 79 (2014), 69--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hsieh, Y. H., Lin, Y. C. and Hou, H. T. Exploring the role of flow experience, learning performance and potential behavior clusters in elementary students' game-based learning. Interactive Learning Environments (2013).Google ScholarGoogle Scholar
  22. Ke, F. and Abras, T. Games for engaged learning of middle school children with special learning needs. British Journal of Educational Technology, 44, 2 (2013), 225--242.Google ScholarGoogle ScholarCross RefCross Ref
  23. Kiili, K. and Lainema, T. Foundation for measuring engagement in educational games. Journal of Interactive Learning Research, 19, 3 (2008), 469--488.Google ScholarGoogle Scholar
  24. Csikszentmihalyi, M. Toward a psychology of optimal experience. Springer, City, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  25. Gerling, K., Fuchslocher, A., Schmidt, R., Krämer, N. and Masuch, M. Designing and evaluating casual health games for children and teenagers with cancer. Springer, City, 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Oksanen, K. and Hamalainen, R. Assessing Game Experiences Caused by Educational Collaborative Game, 2011.Google ScholarGoogle Scholar
  27. Gowases, T., Bednarik, R. and Tukiainen, M. Gaze vs. mouse in games: The effects on user experience. City, 2008.Google ScholarGoogle Scholar
  28. Parsons, T. D., Rizzo, A. A., Courtney, C. G. and Dawson, M. E. Psychophysiology to assess impact of varying levels of simulation fidelity in a threat environment. Advances in Human-Computer Interaction, 2012 (2012). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Procci, K. and Bowers, C. An examination of flow and immersion in games. City, 2011.Google ScholarGoogle Scholar
  30. Witmer, B. G. and Singer, M. J. Measuring presence in virtual environments: A presence questionnaire. Presence, 7, 3 (1998), 225--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Natkin, S., Soriano, D., Erjavec, G. and Durand, M. Could the player's engagement in a video game increase his/her interest in science?, City, 2013.Google ScholarGoogle Scholar
  32. De Leo, G., Diggs, L. A., Radici, E. and Mastaglio, T. W. Measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment. Simulation in Healthcare, 9, 1 (2014), 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  33. Frommel, J., Rogers, K., Brich, J., Besserer, D., Bradatsch, L., Ortinau, I., Schabenberger, R., Riemer, V., Schrader, C. and Weber, M. Integrated questionnaires: Maintaining presence in game environments for self-reported data acquisition. ACM, City, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Mills, C., D'Mello, S., Lehman, B., Bosch, N., Strain, A. and Graesser, A. What makes learning fun? exploring the influence of choice and difficulty on mind wandering and engagement during learning. City, 2013.Google ScholarGoogle Scholar
  35. Fredricks, J. A., Blumenfeld, P. C. and Paris, A. H. School engagement: Potential of the concept, state of the evidence. Review of educational research, 74, 1 (2004), 59--109.Google ScholarGoogle Scholar
  36. Kiili, K., Lainema, T., de Freitas, S. and Arnab, S. Flow framework for analyzing the quality of educational games. Entertainment computing, 5, 4 (2014), 367--377.Google ScholarGoogle Scholar
  37. Skinner, E. A. and Belmont, M. J. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of educational psychology, 85, 4 (1993), 571.Google ScholarGoogle Scholar
  38. Deloitte Media Consumer Survey 2017. City, 2018.Google ScholarGoogle Scholar
  39. Rogers, Y., Sharp, H. and Preece, J. Interaction design: beyond human-computer interaction. John Wiley & Sons, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Coller, B. D. and Shernoff, D. J. Video game-based education in mechanical engineering: A look at student engagement. International Journal of Engineering Education, 25, 2 (2009), 308--317.Google ScholarGoogle Scholar
  41. Hong, J.-C., Hwang, M.-Y., Tai, K.-H. and Lin, P.-C. Self-efficacy relevant to competitive anxiety and gameplay interest in the one-on-one competition setting. Educational Technology Research and Development, 63, 5 (2015), 791--807.Google ScholarGoogle ScholarCross RefCross Ref
  42. De Byl, P. Factors at play in tertiary curriculum gamification. International Journal of Game-Based Learning, 3, 2 (2013), 1--21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Sernani, P., Dalpiaz, F., Dragoni, A. F. and Brinkkemper, S. Smart Tales: An Awareness Game for Ambient Assisted Living. Springer, City, 2015.Google ScholarGoogle Scholar
  44. Kanthan, R. and Senger, J.-L. The impact of specially designed digital games-based learning in undergraduate pathology and medical education. Archives of pathology & laboratory medicine, 135, 1 (2011), 135--142.Google ScholarGoogle Scholar
  45. Cheng, M. T., She, H. C. and Annetta, L. A. Game immersion experience: its hierarchical structure and impact on game-based science learning. Journal of Computer Assisted Learning, 31, 3 (2015), 232--253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Wang, J., Baranowski, T., Lau, P. W., Pitkethly, A. J. and Buday, R. Acceptability and applicability of an American health videogame with story for childhood obesity prevention among Hong Kong Chinese children. Games for health journal, 4, 6 (2015), 513--519.Google ScholarGoogle Scholar
  47. Watson, D., Mandryk, R. L. and Stanley, K. G. The design and evaluation of a classroom exergame. City, 2013.Google ScholarGoogle Scholar
  48. Petridis, P., Liarokapis, F., Lameras, P., Hendrix, M., Dunwell, I., Babaoukas, A., Smithson, E. and Gaterell, M. Green@ CU: An environmental game for residential accommodation. IEEE, City, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  49. Schrader, C. and Bastiaens, T. J. The influence of virtual presence: Effects on experienced cognitive load and learning outcomes in educational computer games. Computers in Human Behavior, 28, 2 (Mar 2012), 648--658. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Hung, H.-C. and Young, S. S.-C. An investigation of game-embedded handheld devices to enhance English learning. Journal of Educational Computing Research, 52, 4 (2015), 548--567.Google ScholarGoogle ScholarCross RefCross Ref
  51. Koivisto, M. Player enjoyment in game-based learning. Int. J. Assess. Eval., 20, 2 (2014), 57--62.Google ScholarGoogle Scholar
  52. Vahldick, A., Mendes, A. J. and Marcelino, M. J. Analysing the Enjoyment of a Serious Game for Programming Learning With two Unrelated Higher Education Audiences. Academic Conferences International Limited, City, 2015.Google ScholarGoogle Scholar
  53. Hou, H.-T., Wang, S.-M., Yu, H.-H. and Liu, S.-H. The development and evaluation of an educational Game-Shimmer? with computer visualization for optics learning. City, 2014.Google ScholarGoogle Scholar
  54. Yin, L., Ring, L. and Bickmore, T. Using an interactive visual novel to promote patient empowerment through engagement. City, 2012.Google ScholarGoogle Scholar
  55. Cargnin, D. J., d'Ornellas, M. C. and Prado, A. L. C. A Serious Game for Upper Limb Stroke Rehabilitation Using Biofeedback and Mirror-Neurons Based Training. City, 2015.Google ScholarGoogle Scholar
  56. De Lima, L. G. R., de Lima Salgado, A. and Freire, A. P. Evaluation of the user experience and intrinsic motivation with educational and mainstream digital games. ACM, City, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Dezentje, P., Cidota, M. A., Clifford, R. M., Lukosch, S. G., Bank, P. J. and Lukosch, H. K. Designing for engagement in augmented reality games to assess upper extremity motor dysfunctions. IEEE, City, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  58. IJsselsteijn, W., De Kort, Y., Poels, K., Jurgelionis, A. and Bellotti, F. Characterising and measuring user experiences in digital games. City, 2007.Google ScholarGoogle Scholar
  59. Ke, F. A case study of computer gaming for math: Engaged learning from gameplay? Computers & education, 51, 4 (2008), 1609--1620. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Diehl, L. A., de Souza, R. M., Gordan, P. A., Esteves, R. Z. and Coelho, I. C. M. User assessment of "InsuOnLine," a game to fight clinical inertia in diabetes: A pilot study. Games for health journal, 4, 5 (2015), 335--343.Google ScholarGoogle Scholar
  61. Lorenzini, C., Faita, C., Barsotti, M., Carrozzino, M., Tecchia, F. and Bergamasco, M. ADITHO---A Serious Game for Training and Evaluating Medical Ethics Skills. Springer, City, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Farrell, D. and Moffat, D. Applying the Self Determination Theory of Motivation in Games Based Learning. Academic Conferences International Limited, City, 2014.Google ScholarGoogle Scholar
  63. Janssen, A., Shaw, T., Goodyear, P., Kerfoot, B. P. and Bryce, D. A little healthy competition: using mixed methods to pilot a team-based digital game for boosting medical student engagement with anatomy and histology content. BMC medical education, 15, 1 (2015), 173.Google ScholarGoogle Scholar
  64. Lin, D. T., Park, J., Liebert, C. A. and Lau, J. N. Validity evidence for Surgical Improvement of Clinical Knowledge Ops: a novel gaming platform to assess surgical decision making. The American Journal of Surgery, 209, 1 (2015), 79--85.Google ScholarGoogle ScholarCross RefCross Ref
  65. Schlegel, E. F. and Selfridge, N. J. Fun, collaboration and formative assessment: Skinquizition, a class wide gaming competition in a medical school with a large class. Medical teacher, 36, 5 (2014), 447--449.Google ScholarGoogle Scholar
  66. Di Palma, S., Tonacci, A., Narzisi, A., Domenici, C., Pioggia, G., Muratori, F., Billeci, L. and group, M. s. Monitoring of autonomic response to sociocognitive tasks during treatment in children with autism spectrum disorders by wearable technologies: A feasibility study. Computers in biology and medicine, 85 (2017), 143--152.Google ScholarGoogle Scholar
  67. Dagan, N., Beskin, D., Brezis, M. and Reis, B. Y. Effects of social network exposure on nutritional learning: development of an online educational platform. JMIR serious games, 3, 2 (2015).Google ScholarGoogle Scholar
  68. Hsu, M. and Cheng, M. Bio detective: Student science learning, immersion experience, and problem-solving patterns. City, 2014.Google ScholarGoogle Scholar
  69. Andre, T. S., Fidopiastis, C. M., Ripley, T. R., Oskorus, A. L., Meyer, R. E. and Snyder, R. A. Augmented cognition methods for evaluating serious game based insider cyber threat detection training. City, 2011.Google ScholarGoogle Scholar
  70. Chittaro, L. and Buttussi, F. Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety. IEEE transactions on visualization and computer graphics, 21, 4 (2015), 529--538.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Ghali, R., Chaouachi, M., Derbali, L. and Frasson, C. Motivational strategies to support engagement of learners in serious games, 1 (2014), 518--525. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Ghergulescu, I. and Muntean, C. H. A novel sensor-based methodology for learner's motivation analysis in game-based learning. Interacting with Computers, 26, 4 (2014), 305--320.Google ScholarGoogle ScholarCross RefCross Ref
  73. Kiili, K., Perttula, A., Lindstedt, A., Arnab, S. and Suominen, M. Flow experience as a quality measure in evaluating physically activating collaborative serious games. Internationaljournal of Serious Games, 1, 3 (2014), 35--49.Google ScholarGoogle Scholar
  74. Adamo-Villani, N., Haley-Hermiz, T. and Cutler, R. Using a serious game approach to teach 'operator precedence' to introductory programming students. City, 2013.Google ScholarGoogle Scholar
  75. Nevin, C. R., Westfall, A. O., Rodriguez, J. M., Dempsey, D. M., Cherrmgton, A., Roy, B., Patel, M. and Willig, J. H. Gamification as a tool for enhancing graduate medical education. Postgraduate medical journal (2014), postgradmedj-2013-132486.Google ScholarGoogle Scholar
  76. Angelia, S., Ohta, N. and Sugiura, K. Design and evaluation of educational kinesthetic game to encourage collaboration for kindergarten children. ACM, City, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Fitz-Walter, Z., Tjondronegoro, D. and Wyeth, P. A gamified mobile application for engaging new students at university orientation. City, 2012.Google ScholarGoogle Scholar
  78. Nicolaidou, I., Antoniades, A., Constantinou, R., Marangos, C., Kyriacou, E., Bamidis, P., Dafli, E. and Pattichis, C. S. A virtual emergency telemedicine serious game in medical training: a quantitative, professional feedback-informed evaluation study. Journal of medical Internet research, 17, 6 (2015).Google ScholarGoogle Scholar
  79. Annetta, L., Mangrum, J., Holmes, S., Collazo, K. and Cheng, M. T. Bridging realty to virtual reality: Investigating gender effect and student engagement on learning through video game play in an elementary school classroom. International Journal of Science Education, 31, 8 (2009), 1091--1113.Google ScholarGoogle ScholarCross RefCross Ref
  80. Ford, M., Wyeth, P. and Johnson, D. Self-determination theory as applied to the design of a software learning system using whole-body controls. City, 2012.Google ScholarGoogle Scholar
  81. Pannese, L. and Carlesi, M. Games and learning come together to maximise effectiveness: The challenge of bridging the gap. British Journal of Educational Technology, 38, 3 (2007), 438--454.Google ScholarGoogle ScholarCross RefCross Ref
  82. Annetta, L. A., Minogue, J., Holmes, S. Y. and Cheng, M. T. Investigating the impact of video games on high school students' engagement and learning about genetics. Computers and Education, 53, 1 (2009), 74--85. Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Fu, F. L. and Yu, S. C. Three layered thinking model for designing web-based educational games. City, 2008.Google ScholarGoogle Scholar
  84. Antle, A. N., Tanenbaum, J., Bevans, A., Seaborn, K. and Wang, S. Balancing act: Enabling public engagement with sustainability issues through a multi-touch tabletop collaborative game. City, 2011.Google ScholarGoogle Scholar
  85. Gauthier, A., Corrin, M. and Jenkinson, J. Exploring the influence of game design on learning and voluntary use in an online vascular anatomy study aid. Computers & Education, 87 (2015), 24--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. Pereira, G., Prada, R. and Paiva, A. Disaster prevention social awareness: The stop disasters! case study. IEEE, City, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  87. Arnab, S., Brown, K., Clarke, S., Dunwell, I., Lim, T., Suttie, N., Louchart, S., Hendrix, M. and De Freitas, S. The development approach of a pedagogically-driven serious game to support Relationship and Sex Education (RSE) within a classroom setting. Computers & Education, 69 (2013), 15--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. Bedek, M., Seitlinger, P., Kopeinik, S. and Albert, D. Multivariate Assessment of Motivation and Emotion in Digital Educational Games, 2011.Google ScholarGoogle Scholar
  89. Bedek, M., Seitlinger, P., Kopeinik, S. and Albert, D. Inferring a learner's cognitive, motivational and emotional state in a digital educational game. Electronic Journal of e-Learning, 10, 2 (2012), 172--184.Google ScholarGoogle Scholar
  90. Bindoff, I., Ling, T., Bereznicki, L., Westbury, J., Chalmers, L., Peterson, G. and Ollington, R. A computer simulation of community pharmacy practice for educational use. American Journal of Pharmaceutical Education, 78, 9 (2014), 168.Google ScholarGoogle ScholarCross RefCross Ref
  91. Rizvic, S. and Prazina, I. Taslihan virtual reconstruction-Interactive digital story or a serious game. IEEE, City, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  92. Ruggiero, D., De Hurtado, B. G. and Watson, W. R. Juvenile offenders: Developing motivation, engagement, and meaning-making through video game creation. International Journal of Game-Based Learning, 3, 2 (2013), 112--129. Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Brown, D., Shopland, N., Battersby, S., Lewis, J. and Evett, L. Can serious games engage the disengaged?, City, 2007.Google ScholarGoogle Scholar
  94. Buttussi, F., Pellis, T., Vidani, A. C., Pausler, D., Carchietti, E. and Chittaro, L. Evaluation of a 3D serious game for advanced life support retraining. International Journal of Medical Informatics, 82, 9 (Sep 2013), 798--809.Google ScholarGoogle ScholarCross RefCross Ref
  95. Schwebel, D. C., McClure, L. A. and Severson, J. Usability and feasibility of an internet-based virtual pedestrian environment to teach children to cross streets safely. Virtual reality, 18, 1 (2014), 5--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. Janssen, A., Shaw, T. and Goodyear, P. Using video games to enhance motivation states in online education: protocol for a team-based digital game. JMIR research protocols, 4, 3 (2015).Google ScholarGoogle Scholar
  97. Shah, N., Amirabdollahian, F. and Basteris, A. Designing motivational games for stroke rehabilitation. IEEE, City, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  98. Shute, V. J., D'Mello, S., Baker, R., Cho, K., Bosch, N., Ocumpaugh, J., Ventura, M. and Almeda, V. Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game. Computers & Education, 86 (2015), 224--235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  99. Cook, N. F., McAloon, T., O'Neill, P. and Beggs, R. Impact of a web based interactive simulation game (PULSE) on nursing students' experience and performance in life support training---a pilot study. Nurse education today, 32, 6 (2012), 714--720.Google ScholarGoogle Scholar
  100. Simkins, D., Egert, C. and Decker, A. Evaluating Martha Madison: Developing analytical tools for gauging the breadth of learning facilitated by STEM games. City, 2012.Google ScholarGoogle Scholar
  101. Cretu, C. M., Rogoz, N. and Chihaia, D. How to Design and Implement a Validation Methodology for Virtual Educational Games. Proceedings of the 11th European Conference on E-Learning (2012), 97--105.Google ScholarGoogle Scholar
  102. Taylor, N., Jenson, J. and De Castell, S. Epidemic: E-Learning goes viral. City, 2010.Google ScholarGoogle Scholar
  103. Creutzfeldt, J., Hedman, L., Heinrichs, L., Youngblood, P. and Felländer-Tsai, L. Cardiopulmonary resuscitation training in high school using avatars in virtual worlds: an international feasibility study. Journal of medical Internet research, 15, 1 (2013).Google ScholarGoogle Scholar
  104. Tegos, S., Kioumousidou, M., Vavami, M., Kioutsiouki, D., Gkoumas, D. and Tsiatsos, T. HistoryLand: A Serious Game for History Learning in Primary Education. Academic Conferences International Limited, City, 2014.Google ScholarGoogle Scholar
  105. Dankbaar, M. E., Alsma, J., Jansen, E. E., van Merrienboer, J. J., van Saase, J. L. and Schuit, S. C. An experimental study on the effects of a simulation game on students' clinical cognitive skills and motivation. Advances in Health Sciences Education, 21, 3 (2016), 505--521.Google ScholarGoogle ScholarCross RefCross Ref
  106. Van Oostendorp, H., Van Der Spek, E. and Linssen, J. Adapting the complexity level of a serious game to the proficiency of players. City, 2013.Google ScholarGoogle Scholar
  107. D'Aprile, G., Di Bitonto, P., De Asmundis, R. and Severino, A. U. Social, constructivist and informal learning processes: together on the Edge for designing digital game-based learning environments. Journal of e-Learning and Knowledge Society, 11, 3 (2015).Google ScholarGoogle Scholar
  108. Vieira, L. d. S., Coutinho, C. P., Graça, J. I. and Graça, J. The implementation of mobile location based-games and QR-codes: the case of MobiGeo. Iated, City, 2014.Google ScholarGoogle Scholar
  109. Lameras, P., Pisithpunth, C. and Petridis, P. A quiz-based game for addressing growing population issues: Linking learning mechanics to THE GROWTH serious game. IEEE, City, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  110. Von Wangenheim, C. G., Thiry, M. and Kochanski, D. Empirical evaluation of an educational game on software measurement. Empirical Software Engineering, 14, 4 (2009), 418--452. Google ScholarGoogle ScholarDigital LibraryDigital Library
  111. Lanyi, C. S., Brown, D. J., Standen, P., Lewis, J. and Butkute, V. Results of User Interface Evaluation of Serious Games for Students with Intellectual Disability. Acta Polytechnica Hungarica, 9, 1 (2012), 225--245.Google ScholarGoogle Scholar
  112. Lee, M. J., Ko, A. J. and Kwan, I. In-game assessments increase novice programmers' engagement and level completion speed. ACM, City, 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. Li, M. C.,Holf, H. T., Wu, Y. S., Chen, H. and Chou, Y. S. Exploring learners' problem-solving behaviors in an educational game for computer assembly instruction: A preliminary study. City, 2012.Google ScholarGoogle Scholar
  114. Wernbacher, T., Pfeiffer, A., Wagner, M. and Hofstätter, J. Learning by playing: Can serious games be fun?, City, 2012.Google ScholarGoogle Scholar
  115. Whitehouse, J., Waller, B. M., Chanvin, M., Wallace, E. K., Schel, A. M., Peirce, K., Mitchell, H., Macri, A. and Slocombe, K. Evaluation of public engagement activities to promote science in a zoo environment. PloS one, 9, 11 (2014), e113395.Google ScholarGoogle Scholar
  116. Dib, H. and Adamo-Villani, N. Serious sustainability challenge game to promote teaching and learning of building sustainability. Journal of Computing in Civil Engineering, 28, 5 (2013), A4014007.Google ScholarGoogle Scholar
  117. Wouters, P., van Oostendorp, H., Boonekamp, R. and van der Spek, E. The role of Game Discourse Analysis and curiosity in creating engaging and effective serious games by implementing a back story and foreshadowing. Interacting with Computers, 23, 4 (Jul 2011), 329--336.Google ScholarGoogle ScholarCross RefCross Ref
  118. Dunwell, I., Lameras, P., de Freitas, S., Petridis, P., Star, K., Hendrix, M. and Arnab, S. MeTycoon: A game-based approach to career guidance. IEEE, City, 2013.Google ScholarGoogle ScholarCross RefCross Ref
  119. Looi, Q. and See, S. Effectively engaging students in educational games by deploying HCI evaluation methods. Citeseer, City, 2010.Google ScholarGoogle Scholar
  120. Ekanayake, H., Backlund, P., Ziemke, T., Ramberg, R. and Hewagamage, K. Game interaction state graphs for evaluation of user engagement in explorative and experience-based training games. City, 2010.Google ScholarGoogle Scholar
  121. Looi, Q. E. and See, S. L. Assessing the suitability of MMORPGs as educational games using HCI evaluation methods. City, 2011.Google ScholarGoogle Scholar
  122. Youngblood, P., Hedman, L., Creutzfeld, J., Fellander-Tsai, L., Stengard, K., Hansen, K., Dev, P., Srivastava, S., Kusumoto, L., Hendrick, A. and Heinrichs, W. L. Virtual worlds for teaching the new CPR to high school students. City, 2007.Google ScholarGoogle Scholar

Index Terms

  1. A Systematic Review of the Definition and Measurement of Engagement in Serious Games

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      ACSW '19: Proceedings of the Australasian Computer Science Week Multiconference
      January 2019
      486 pages
      ISBN:9781450366038
      DOI:10.1145/3290688

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 29 January 2019

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed limited

      Acceptance Rates

      ACSW '19 Paper Acceptance Rate61of141submissions,43%Overall Acceptance Rate61of141submissions,43%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader