Skip to main content

Advertisement

SpringerLink
Log in

Integrating adaptivity in educational games: a combined bibliometric analysis and meta-analysis review

  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

In this synthesis, we systematically review research on educational games with adaptivity. Although an adaptive gaming experience provides personalization to learning, the complexity of design makes it difficult to evaluate its effectiveness. In this systematic review, we adopt three analytic approaches: (1) bibliometric analysis, (2) qualitative thematic analysis, and (3) meta-analysis. We identified 62 relevant publications and used bibliometric analysis to visualize the hidden conceptual structure among the articles. We then used thematic analysis to investigate the emerging themes in the research and inform the coding for meta-analysis. Twelve articles that used experimental designs were further screened to model the effect of adding adaptivity to educational games. We found that an adaptive learning condition does not produce a substantial overall effect compared to a non-adaptive condition (g = .11, p = .26). Furthermore, moderator analysis reveals that the design variability of adaptivity does not contribute as much to heterogeneity as hypothesized. However, the effect size is positive when the target outcome focuses on learning (g = .39, p < .001) and engagement (g = .41, p = .13). The effect size is negative when the target outcome focuses on game performance (g = − .27, p = .04). In addition, we find evidence for a potential publication bias based on the distribution of effect sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007.

    Article  Google Scholar 

  • Barra, E., Gordillo, A., Gallego, D., & Quemada, J. (2013). Integration of SCORM packages into web games. In 2013 IEEE frontiers in education conference (FIE) (pp. 685–690). https://doi.org/10.1109/FIE.2013.6684913.

  • Bedek, M., Seitlinger, P., Kopeinik, S., & Albert, D. (2011). Multivariate assessment of motivation and emotion in digital educational games. In M. Gouscos, D and Meimaris (Ed.), Proceedings of The 5th European conference on games based learning (pp. 18–25).

  • Bontchev, B., & Georgieva, O. (2018). Playing style recognition through an adaptive video game. Computers in Human Behavior, 82, 136–147. https://doi.org/10.1016/j.chb.2017.12.040.

    Article  Google Scholar 

  • Bontchev, B., Vassileva, D., Aleksieva-Petrova, A., & Petrov, M. (2018). Playing styles based on experiential learning theory. Computers in Human Behavior, 85, 319–328. https://doi.org/10.1016/j.chb.2018.04.009.

    Article  Google Scholar 

  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley. https://doi.org/10.1002/9780470743386.

    Book  Google Scholar 

  • Broadus, R. N. (1987). Toward a definition of “bibliometrics”. Scientometrics, 12(5–6), 373–379. https://doi.org/10.1007/BF02016680.

    Article  Google Scholar 

  • Chee, Y. S. (2007). Embodiment, embeddedness, and experience: Game-based learning and the construction of identity. Research and Practice in Technology Enhanced Learning, 02(01), 3–30. https://doi.org/10.1142/S1793206807000282.

    Article  Google Scholar 

  • Clark, D. B., Martinez-Garza, M. M., Biswas, G., Luecht, R. M., & Sengupta, P. (2012). Driving assessment of students’ explanations in game dialog using computer-adaptive testing and hidden markov modeling. In Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 173–199). New York: Springer. https://doi.org/10.1007/978-1-4614-3546-4_10.

  • Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016a). Digital games, design, and learning. Review of Educational Research, 86(1), 79–122. https://doi.org/10.3102/0034654315582065.

    Article  Google Scholar 

  • Clark, D. B., Virk, S. S., Barnes, J., & Adams, D. M. (2016b). Self-explanation and digital games: Adaptively increasing abstraction. Computers & Education, 103, 28–43. https://doi.org/10.1016/j.compedu.2016.09.010.

    Article  Google Scholar 

  • Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459. https://doi.org/10.3102/00346543053004445.

    Article  Google Scholar 

  • Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology, 62(7), 1382–1402. https://doi.org/10.1002/asi.21525.

    Article  Google Scholar 

  • Conati, C., & Manske, M. (2009). Adaptive feedback in an educational game for number factorization. In V. Dimitrova, R. Mizoguchi, B. DuBoulay, & A. C. Graesser (Eds.), Artificial intelligence in education: Building learning systems that care: From knowledge representation to affective modelling (Vol. 200, pp. 581–583). Amsterdam: IOS Press. https://doi.org/10.3233/978-1-60750-028-5-581.

    Chapter  Google Scholar 

  • Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with everyday life. New York: HarperCollins. https://doi.org/10.5860/choice.35-1828.

    Book  Google Scholar 

  • Dick, W., Carey, L., & Carey, J. O. (2009). The systematic design of instruction (7th ed.). Upper Saddle River, NJ: Merrill.

    Google Scholar 

  • Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x.

    Article  Google Scholar 

  • Ferguson, C. J., & Moritz, H. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7(6), 555–561. https://doi.org/10.1177/1745691612459059.

    Article  Google Scholar 

  • Gavriushenko, M., Karilainen, L., & Kankaanranta, M. (2015). Adaptive systems as enablers of feedback in English language learning game-based environments. In 2015 IEEE Frontiers in Education Conference (FIE) (pp. 1–8). https://doi.org/10.1109/FIE.2015.7344107.

  • Gros, B. (2007). Digital games in education: The design of games-based learning environments. Journal of Research on Technology in Education, 40(1), 23–38.

    Article  Google Scholar 

  • Hallinger, P., & Kovačević, J. (2019). A bibliometric review of research on educational administration: Science mapping the literature, 1960 to 2018. Review of Educational Research, 89(3), 335–369. https://doi.org/10.3102/0034654319830380.

    Article  Google Scholar 

  • Hedges, L. V. (1981). Distribution theory for glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107. https://doi.org/10.2307/1164588.

    Article  Google Scholar 

  • Jagušt, T., Botički, I., & So, H.-J. (2018). Examining competitive, collaborative and adaptive gamification in young learners’ math learning. Computers & Education, 125, 444–457.

    Article  Google Scholar 

  • Kai, S., Paquette, L., Baker, R. S. J. D., Bosch, N., D’Mello, S. K., Ocumpaugh, J., … Ventura, M. (2015). A comparison of video-based and interaction-based affect detectors in physics playground. In Proceedings of the 8th international conference on educational data mining (pp. 77–84). Madrid, Spain.

  • Katsionis, G., & Virvou, M. (2004). A cognitive theory for affective user modelling in a virtual reality educational game. In 2004 IEEE international conference on systems, man and cybernetics (IEEE Cat. No.04CH37583) (Vol. 2, pp. 1209–1213). https://doi.org/10.1109/ICSMC.2004.1399789.

  • Ke, F. (2016). Designing and integrating purposeful learning in game play: A systematic review. Educational Technology Research and Development, 64(2), 219–244. https://doi.org/10.1007/s11423-015-9418-1.

    Article  Google Scholar 

  • Kickmeier-Rust, M., Mattheiss, E., Steiner, C., & Albert, D. (2011). A psycho-pedagogical framework for multi-adaptive educational games. International Journal of Game-Based Learning, 1(1), 45–58. https://doi.org/10.4018/ijgbl.2011010104.

    Article  Google Scholar 

  • Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1.

    Article  Google Scholar 

  • Kujala, J. V., Richardson, U., & Lyytinen, H. (2010). A Bayesian-optimal principle for learner-friendly adaptation in learning games. Journal of Mathematical Psychology, 54(2), 247–255. https://doi.org/10.1016/j.jmp.2009.10.001.

    Article  Google Scholar 

  • Kuk, K. V., Milentijevic, I. Z., Randelovic, D. M., Popovic, B. M., & Cisar, P. (2017). The design of the personal enemy—MIMLeBot as an intelligent agent in a game-based learning environment. Acta Polytechnica Hungarica, 14(4), 121–139. https://doi.org/10.12700/APH.14.4.2017.4.7.

    Article  Google Scholar 

  • Lamb, R. L., Vallett, D. B., Akmal, T., & Baldwin, K. (2014). A computational modeling of student cognitive processes in science education. Computers & Education, 79, 116–125.

    Article  Google Scholar 

  • Law, E. L.-C., & Sun, X. (2012). Evaluating user experience of adaptive digital educational games with activity theory. International Journal of Human-Computer Studies, 70(7, SI), 478–497. https://doi.org/10.1016/j.ijhcs.2012.01.007.

    Article  Google Scholar 

  • Leemkuil, H., & de Jong, T. (2012). Adaptive advice in learning with a computer-based knowledge management simulation game. Academy of Management Learning and Education, 11(4), 653–665. https://doi.org/10.5465/amle.2010.0141.

    Article  Google Scholar 

  • Leutner, D. (1993). Guided discovery learning with computer-based simulation games: Effects of adaptive and non-adaptive instructional support. Learning and Instruction, 3(2), 113–132. https://doi.org/10.1016/0959-4752(93)90011-N.

    Article  Google Scholar 

  • Leydesdorff, L. (2008). On the normalization and visualization of author co-citation data: Salton’s Cosine versus the Jaccard index. Journal of the American Society for Information Science and Technology, 59(1), 77–85. https://doi.org/10.1002/asi.20732.

    Article  Google Scholar 

  • Lindberg, R. S. N., & Laine, T. H. (2017). Approaches to detecting and utilizing play and learning styles in adaptive educational games. In G. Costagliola, J. Uhomoibhi, S. Zvacek, & B. McLaren (Eds.), Computers supported education (Vol. 739, pp. 336–358). https://doi.org/10.1007/978-3-319-63184-4_18.

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Long, Y., & Aleven, V. (2014). Gamification of joint student/system control over problem selection in a linear equation tutor. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panouriga (Eds.), Intelligent tutoring systems. ITS 2014 (pp. 378–387)., Lecture Notes in Computer Science Honolulu, HI: Springer. https://doi.org/10.1007/978-3-319-07221-0_47.

    Chapter  Google Scholar 

  • Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. American Psychologist, 59(1), 14–19. https://doi.org/10.1037/0003-066X.59.1.14.

    Article  Google Scholar 

  • Millán, E., Loboda, T., & Pérez-de-la-Cruz, J. L. (2010). Bayesian networks for student model engineering. Computers & Education, 55(4), 1663–1683. https://doi.org/10.1016/j.compedu.2010.07.010.

    Article  Google Scholar 

  • Moreno-Ger, P., Burgos, D., Sierra, J. L., & Manjón, B. F. (2007). A game-based adaptive unit of learning with IMS learning design and < e-Adventure>. In E. Duval, R. Klamma, & M. Wolpers (Eds.), Creating new learning experiences on a global scale. EC-TEL 2007 (Vol. 4753, pp. 247–261)., Lecture Notes in Computer Science Berlin: Springer. https://doi.org/10.1007/978-3-540-75195-3_18.

    Chapter  Google Scholar 

  • Morrison, G. R., Ross, S. M., & Kemp, J. E. (2007). Designing effective instruction (5th ed.). Hoboken, NJ: Wiley.

    Google Scholar 

  • Murphy, N., & Messer, D. (2000). Differential benefits from scaffolding and children working alone. Educational Psychology, 20(1), 17–31. https://doi.org/10.1080/014434100110353.

    Article  Google Scholar 

  • Orvis, K. A., Horn, D. B., & Belanich, J. (2008). The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames. Computers in Human Behavior, 24(5), 2415–2433. https://doi.org/10.1016/j.chb.2008.02.016.

    Article  Google Scholar 

  • Padilla-Zea, N., Medina Medina, N., Gutierrez Vela, F. L., Paderewski, P., & Collazos, C. A. (2018). PLAGER-VG: Platform for managing educational multiplayer video games. Multimedia Tools and Applications, 77(2), 2115–2152. https://doi.org/10.1007/s11042-017-4376-8.

    Article  Google Scholar 

  • Peirce, N., & Wade, V. (2010). Personalised learning for casual games: The ‘language trap’ online language learning game. In B. Meyer (Ed.), Proceedings of the 4th European conference on games based learning (pp. 306–315).

  • Plass, J. L., Homer, B. D., Pawar, S., Brenner, C., & MacNamara, A. P. (2019). The effect of adaptive difficulty adjustment on the effectiveness of a game to develop executive function skills for learners of different ages. Cognitive Development, 49, 56–67. https://doi.org/10.1016/j.cogdev.2018.11.006.

    Article  Google Scholar 

  • Pons, P., & Latapy, M. (2006). Computing communities in large networks using random walks. Journal of Graph Algorithms and Applications, 10(2), 191–218.

    Article  Google Scholar 

  • Raybourn, E. M. (2007). Applying simulation experience design methods to creating serious game-based adaptive training systems. Interacting with Computers, 19(2), 206–214. https://doi.org/10.1016/j.intcom.2006.08.001.

    Article  Google Scholar 

  • Sampayo-Vargas, S., Cope, C. J., He, Z., & Byrne, G. J. (2013). The effectiveness of adaptive difficulty adjustments on students’ motivation and learning in an educational computer game. Computers & Education, 69, 452–462. https://doi.org/10.1016/j.compedu.2013.07.004.

    Article  Google Scholar 

  • Sandberg, J., Maris, M., & Hoogendoorn, P. (2014). The added value of a gaming context and intelligent adaptation for a mobile learning application for vocabulary learning. Computers & Education, 76, 119–130. https://doi.org/10.1016/j.compedu.2014.03.006.

    Article  Google Scholar 

  • Shabihi, N., Taghiyareh, F., & Abdoli, M. H. (2016). Analyzing the effect of game-elements in e-learning environments through MBTI-based personalization. In 2016 8th international symposium on telecommunications (IST) (pp. 612–618). https://doi.org/10.1109/ISTEL.2016.7881895.

  • Shabihi, N., & Taghiyareh, F. (2017). Toward a personalized game-based learning environment using personality type indicators. In A. Mesquita, & P. Peres (Eds.), Proceedings of the 16th European conference on e-learning (ECEL 2017) (pp. 476–483). Porto, Portugal.

  • Shute, V. J., Ke, F., Almond, R. G., Rahimi, S., Smith, G., & Lu, X. (2019). How to increase learning while not decreasing the fun in educational games. In R. Feldman (Ed.), Learning science: Theory, research, and practice (pp. 327–357). New York: McGraw Hill.

    Google Scholar 

  • Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106–117. https://doi.org/10.1016/j.chb.2016.05.047.

    Article  Google Scholar 

  • Shute, V. J., & Zapata-Rivera, D. (2008). Adaptive Technologies. In J. M. Spector, D. Merrill, J. van Merriënboer, & M. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 277–294). New York: Lawrence Erlbaum Associates, Taylor & Francis Group.

    Google Scholar 

  • Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. Durlach (Ed.), Adaptive technologies for training and education (pp. 7–27). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Soflano, M., Connolly, T. M., & Hainey, T. (2015). An application of adaptive games-based learning based on learning style to teach SQL. Computers & Education, 86, 192–211. https://doi.org/10.1016/j.compedu.2015.03.015.

    Article  Google Scholar 

  • Tijssen, R., & Van Raan, A. (1989). Mapping co-word structures: A comparison of multidimensional scaling and LEXIMAPPE. Scientometrics, 15(3-4), 283–295. https://doi.org/10.1007/bf02017203.

    Article  Google Scholar 

  • Tobias, S. (1994). Interest, prior knowledge, and learning. Review of Educational Research, 64(1), 37–54. https://doi.org/10.3102/00346543064001037.

    Article  Google Scholar 

  • Torgerson, C. J. (2006). Publication bias: The Achilles’ heel of systematic reviews? British Journal of Educational Studies, 54(1), 89–102. https://doi.org/10.1111/j.1467-8527.2006.00332.x.

    Article  Google Scholar 

  • Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K. M., & Deci, E. L. (2004). Motivating learning, performance, and persistence: The synergistic effects of intrinsic goal contents and autonomy-supportive contexts. Journal of Personality and Social Psychology, 87(2), 246–260. https://doi.org/10.1037/0022-3514.87.2.246.

    Article  Google Scholar 

  • Vlachopoulos, D., & Makri, A. (2017). The effect of games and simulations on higher education: A systematic literature review. International Journal of Educational Technology in Higher Education, 14(1), 22. https://doi.org/10.1186/s41239-017-0062-1.

    Article  Google Scholar 

  • Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229–243. https://doi.org/10.2190/FLHV-K4WA-WPVQ-H0YM.

    Article  Google Scholar 

  • Vygotsky, L. S. (1978). Interaction between learning and development. In M. Gauvain & M. Cole (Eds.), Readings on the development of children (2nd ed., pp. 33–40). New York: Scientific American Books. https://doi.org/10.2307/j.ctvjf9vz4.11.

    Chapter  Google Scholar 

  • Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412–425. https://doi.org/10.1016/j.compedu.2012.07.018.

    Article  Google Scholar 

Download references

Funding

The authors received no funding for this project.

Author information

Authors and Affiliations

  1. Department of Educational Psychology and Learning Systems, College of Education, Florida State University, 1114 Call Street, Tallahassee, FL, 32303, USA

    Zhichun Liu, Jewoong Moon, Byungjoo Kim & Chih-Pu Dai

Authors
  1. Zhichun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jewoong Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Byungjoo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chih-Pu Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhichun Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Moon, J., Kim, B. et al. Integrating adaptivity in educational games: a combined bibliometric analysis and meta-analysis review. Education Tech Research Dev 68, 1931–1959 (2020). https://doi.org/10.1007/s11423-020-09791-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-020-09791-4

Keywords

Search

Navigation