Karen D. Wang
ABSTRACT
Digital games offer promising platforms for assessing student higher-order competencies such as problem-solving. However, processing and analyzing the large volume of interaction log data generated in these platforms to uncover meaningful behavioral patterns remain a complex research challenge. In this study, we employ sequence mining and clustering techniques to examine students’ log data in an interactive puzzle game that requires player to change rules to win the game. Our goal is to identify behavioral characteristics associated with the problem-solving practices adopted by individual students. The findings indicate that the most effective problem solvers made fewer rule changes and took longer time to make those changes across both an introductory and a more advanced level of the game. Conversely, rapid rule change actions were linked to ineffective problem-solving. This research underscores the potential of sequence mining and cluster analysis as generalizable methods for understanding student higher-order competencies through log data in digital gaming and learning environments. It also suggests future directions on how to provide just-in-time, in-game feedback to enhance student problem-solving competences.
- Andrew Abbott and Angela Tsay. 2000. Sequence analysis and optimal matching methods in sociology: Review and prospect. Sociological methods & research 29, 1 (2000), 3–33.Google Scholar
Cross Ref
- Saleema Amershi, Cristina Conati, 2009. Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of educational data mining 1, 1 (2009), 18–71.Google Scholar
- Dylan A Arena and Daniel L Schwartz. 2014. Experience and explanation: Using videogames to prepare students for formal instruction in statistics. Journal of Science Education and Technology 23 (2014), 538–548.Google ScholarCross Ref
- Ryan Shaun Baker, Albert T Corbett, Kenneth R Koedinger, and Angela Z Wagner. 2004. Off-task behavior in the cognitive tutor classroom: When students" game the system". In Proceedings of the SIGCHI conference on Human factors in computing systems. 383–390.Google ScholarDigital Library
- Elizabeth A Boyle, Thomas M Connolly, Thomas Hainey, and James M Boyle. 2012. Engagement in digital entertainment games: A systematic review. Computers in human behavior 28, 3 (2012), 771–780.Google Scholar
- Elizabeth A Boyle, Thomas Hainey, Thomas M Connolly, Grant Gray, Jeffrey Earp, Michela Ott, Theodore Lim, Manuel Ninaus, Claudia Ribeiro, and João Pereira. 2016. An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education 94 (2016), 178–192.Google ScholarDigital Library
- Engin Bumbacher, Shima Salehi, Carl Wieman, and Paulo Blikstein. 2018. Tools for science inquiry learning: Tool affordances, experimentation strategies, and conceptual understanding. Journal of Science Education and Technology 27 (2018), 215–235.Google ScholarCross Ref
- EW Burkholder, JK Miles, TJ Layden, KD Wang, AV Fritz, and CE Wieman. 2020. Template for teaching and assessment of problem solving in introductory physics. Physical Review Physics Education Research 16, 1 (2020), 010123.Google ScholarCross Ref
- William G Chase and Herbert A Simon. 1973. Perception in chess. Cognitive psychology 4, 1 (1973), 55–81.Google Scholar
- Mihaly Csikszentmihalyi and Isabella Selega Csikszentmihalyi. 1992. Optimal experience: Psychological studies of flow in consciousness. Cambridge university press.Google Scholar
- David DeLiema, Megan Goeke, Basel Hussein, Jesslyn Valerie, Craig Anderson, Keisha Varma, Bodong Chen, Shima Salehi, and Matthew Bernacki. 2022. Playful Learning Following Deviations: A Mixture of Tinkering, Causal Explanations, and Revision Rationales. In Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, pp. 1421-1424. International Society of the Learning Sciences.Google Scholar
- Susan Dumais, Robin Jeffries, Daniel M Russell, Diane Tang, and Jaime Teevan. 2014. Understanding user behavior through log data and analysis. Ways of Knowing in HCI (2014), 349–372.Google Scholar
- Beate Eichmann, Samuel Greiff, Johannes Naumann, Liene Brandhuber, and Frank Goldhammer. 2020. Exploring behavioural patterns during complex problem-solving. Journal of Computer Assisted Learning 36, 6 (2020), 933–956.Google ScholarCross Ref
- Philippe Fournier-Viger, Jerry Chun-Wei Lin, Rage Uday Kiran, Yun Sing Koh, and Rincy Thomas. 2017. A survey of sequential pattern mining. Data Science and Pattern Recognition 1, 1 (2017), 54–77.Google Scholar
- Peter A Frensch and Joachim Funke. 2014. Definitions, traditions, and a general framework for understanding complex problem solving. In Complex problem solving. Psychology Press, 3–25.Google Scholar
- Alexis Gabadinho, Gilbert Ritschard, Nicolas S Müller, and Matthias Studer. 2011. Analyzing and visualizing state sequences in R with TraMineR. Journal of statistical software 40 (2011), 1–37.Google ScholarCross Ref
- Janice D Gobert, Michael Sao Pedro, Juelaila Raziuddin, and Ryan S Baker. 2013. From log files to assessment metrics: Measuring students’ science inquiry skills using educational data mining. Journal of the Learning Sciences 22, 4 (2013), 521–563.Google ScholarCross Ref
- A-M Hoskinson, Marcos D Caballero, and Jennifer K Knight. 2013. How can we improve problem solving in undergraduate biology? Applying lessons from 30 years of physics education research. CBE—Life Sciences Education 12, 2 (2013), 153–161.Google Scholar
- David H Jonassen. 1997. Instructional design models for well-structured and III-structured problem-solving learning outcomes. Educational technology research and development 45, 1 (1997), 65–94.Google Scholar
- Alboukadel Kassambara. 2017. Practical guide to cluster analysis in R: Unsupervised machine learning. Vol. 1. Sthda.Google Scholar
- Deirdre Kerr and Gregory KWK Chung. 2012. Identifying key features of student performance in educational video games and simulations through cluster analysis.Journal of Educational Data Mining 4, 1 (2012), 144–182.Google Scholar
- John S Kinnebrew, Kirk M Loretz, and Gautam Biswas. 2013. A contextualized, differential sequence mining method to derive students’ learning behavior patterns.Journal of Educational Data Mining 5, 1 (2013), 190–219.Google Scholar
- Tongxi Liu and Maya Israel. 2022. Uncovering students’ problem-solving processes in game-based learning environments. Computers & Education 182 (2022), 104462.Google ScholarDigital Library
- Richard E Mayer. 1992. Thinking, problem solving, cognition. WH Freeman/Times Books/Henry Holt & Co.Google Scholar
- Richard E Mayer and Merlin C Wittrock. 1996. Problem-solving transfer. Handbook of educational psychology (1996), 47–62.Google Scholar
- Fionn Murtagh and Pierre Legendre. 2014. Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?Journal of classification 31 (2014), 274–295.Google Scholar
- Allen Newell, Herbert Alexander Simon, 1972. Human problem solving. Vol. 104. Prentice-hall Englewood Cliffs, NJ.Google Scholar
- Frank Nielsen and Frank Nielsen. 2016. Hierarchical clustering. Introduction to HPC with MPI for Data Science (2016), 195–211.Google ScholarDigital Library
- Dilhan Perera, Judy Kay, Irena Koprinska, Kalina Yacef, and Osmar R Zaïane. 2008. Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on knowledge and Data Engineering 21, 6 (2008), 759–772.Google ScholarDigital Library
- Sarah Perez, Jonathan Massey-Allard, Deborah Butler, Joss Ives, Doug Bonn, Nikki Yee, and Ido Roll. 2017. Identifying productive inquiry in virtual labs using sequence mining. In Artificial Intelligence in Education: 18th International Conference, AIED 2017, Wuhan, China, June 28–July 1, 2017, Proceedings 18. Springer, 287–298.Google ScholarCross Ref
- Argenta M Price, Candice J Kim, Eric W Burkholder, Amy V Fritz, and Carl E Wieman. 2021. A detailed characterization of the expert problem-solving process in science and engineering: Guidance for teaching and assessment. CBE—Life Sciences Education 20, 3 (2021), ar43.Google Scholar
- Frederick Reif and Joan I Heller. 1982. Knowledge structure and problem solving in physics. Educational psychologist 17, 2 (1982), 102–127.Google Scholar
- S Ian Robertson. 2016. Problem solving: Perspectives from cognition and neuroscience. Psychology Press.Google Scholar
- Shima Salehi. 2018. Improving problem-solving through reflection. Stanford University.Google Scholar
- Ketan Rajshekhar Shahapure and Charles Nicholas. 2020. Cluster quality analysis using silhouette score. In 2020 IEEE 7th international conference on data science and advanced analytics (DSAA). IEEE, 747–748.Google ScholarCross Ref
- Valerie J Shute, Lubin Wang, Samuel Greiff, Weinan Zhao, and Gregory Moore. 2016. Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior 63 (2016), 106–117.Google ScholarDigital Library
- Michelle Taub, Roger Azevedo, Amanda E Bradbury, Garrett C Millar, and James Lester. 2018. Using sequence mining to reveal the efficiency in scientific reasoning during STEM learning with a game-based learning environment. Learning and instruction 54 (2018), 93–103.Google ScholarCross Ref
- Karen D Wang, Jade Maï Cock, Tanja Käser, and Engin Bumbacher. 2023. A systematic review of empirical studies using log data from open-ended learning environments to measure science and engineering practices. British Journal of Educational Technology 54, 1 (2023), 192–221.Google ScholarCross Ref
- Karen D Wang and Carl Wieman. 2022. Applying Sequence Mining to Explore Students’ Problem-Solving Practices Using an Interactive Simulation-Based Task. In Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, pp. 433-439. International Society of the Learning Sciences.Google Scholar
Index Terms
- Discovering Players’ Problem-Solving Behavioral Characteristics in a Puzzle Game through Sequence Mining
Recommendations
Evaluating multiple aspects of a digital educational problem-solving-based adventure game
This study aims to evaluate multiple aspects of a problem-solving-based educational adventure game, Boom Room(C). The learning effectiveness, game acceptance, and flow experience of the game were empirically investigated. The game was designed and ...
The impact of learner metacognition and goal orientation on problem-solving in a serious game environment
AbstractTo understand the impact of learner metacognition and goal orientation on problem-solving, this study investigated 159 undergraduate students’ metacognition, goal orientations, and their problem-solving performances and processes in a ...
Highlights- Computer log data was used to analyze leaner problem-solving process.
- ...
The differences in motivations of online game players and offline game players: A combined analysis of three studies at higher education level
Computer games have become a highly popular form of entertainment and have had a large impact on how University students spend their leisure time. Due to their highly motivating properties computer games have come to the attention of educationalists who ...
Comments