Erik Harpstead
Brad A. Myers
Vincent Aleven
ABSTRACT
The field of Educational Games has seen many calls for added rigor. One avenue for improving the rigor of the field is developing more generalizable methods for measuring student learning within games. Throughout the process of development, what is relevant to measure and assess may change as a game evolves into a finished product. The field needs an approach for game developers and researchers to be able to prototype and experiment with different measures that can stand up to rigorous scrutiny, as well as provide insight into possible new directions for development. We demonstrate a toolkit and analysis tools that capture and analyze students' performance within open educational games. The system records relevant events during play, which can be used for analysis of player learning by designers. The tools support replaying student sessions within the original game's environment, which allows researchers and developers to explore possible explanations for student behavior. Using this system, we were able to facilitate a number of analyses of student learning in an open educational game developed by a team of our collaborators as well as gain greater insight into student learning with the game and where to focus as we iterate.
- Ambinder, M. Valve's Approach to Playtesting: The Application of Empiricism. Presented at GDC 2009, (2009).Google Scholar
- Andersen, E., Rourke, E.O., Liu, Y., et al. The Impact of Tutorials on Games of Varying Complexity. Proc. CHI 2012, (2012), 59--68. Google ScholarDigital Library
- Baker, R.S.J. d, Habgood, J.M.P., Ainsworth, S.E., and Corbett, A.T. Modeling the Acquisition of Fluent Skill in Educational Action Games. In User Modeling 2007. 2007, 17--26. Google ScholarDigital Library
- Barab, S., Thomas, M., Dodge, T., Carteaux, R., and Tuzun, H. Making Learning Fun: Quest Atlantis, a Game without Guns. Educational Technology Research and Development 53, 1 (2005), 86--107.Google ScholarCross Ref
- Christel, M.G., Stevens, S.M., Maher, B.S., et al. RumbleBlocks: Teaching Science Concepts to Young Children through a Unity Game. Proc. CGames 2012, (2012), 162--166. Google ScholarDigital Library
- Corbett, A.T. and Anderson, J.R. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modelling and User-Adapted Interaction 4, 4 (1995), 253--278.Google ScholarCross Ref
- Council, N.R. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. The National Academies Press, 2012.Google Scholar
- Delacruz, G.C., Chung, G.K.W.K., and Baker, E.L. Validity Evidence for Games as Assessment Environments (CRESST Report 773). Los Angeles, CA, 2010.Google Scholar
- Gee, J.P. What video games have to teach us about learning and literacy. Palgrave Macmillan, New York, 2003. Google ScholarDigital Library
- Honey, M.A. and Hilton, M. Learning Science Through Computer Games and Simulations. National Academic Press, Washington, D.C., 2011.Google Scholar
- Ketelhut, D.J. The Impact of Student Self-efficacy on Scientific Inquiry Skills: An Exploratory Investigation in River City, a Multi-user Virtual Environment. Journal of Science Education and Technology 16, 1 (2006), 99--111.Google Scholar
- Kim, J.H., Gunn, D. V, Schuh, E., Phillips, B.C., Pagulayan, R.J., and Wixon, D. Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. Proc. CHI 2008, (2008), 443--451. Google ScholarDigital Library
- Koedinger, K.R., Baker, R.S.J. d, Cunningham, K., Skogsholm, A., Leber, B., and Stamper, J. A Data Repository for the EDM community: The PSLC DataShop. In C. Romero, S. Ventura, M. Pechenizkiy and R.S.J. d. Baker, eds., Handbook of Educational Data Mining. 2010, 43--55.Google Scholar
- Manske, M. and Conati, C. Modelling Learning in an Educational Game. In C.-K. Looi, M. Gord, B. Bredeweg and J. Breuker, eds., Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology. IOS Press, Amsterdam, Netherlands, 2005, 411--418. Google ScholarDigital Library
- Martin, B., Koedinger, K.R., Mitrovic, A., and Mathan, S. On Using Learning Curves to Evaluate ITS. Proc. AIED 2005, (2005), 419--426. Google ScholarDigital Library
- Mislevy, R.J., Steinberg, L.S., and Almond, R.G. On the Structure of Educational Assessments. Measurement: Interdisciplinary Research and Perspectives 1, 1 (2003), 3--62.Google ScholarCross Ref
- Newell, A. and Rosenbloom, P.S. Mechanisms of skill acquisition and the law of practice. In J.R. Anderson, ed., Cognitive skills and their acquisition. Lawrence Erlbaum Associates, Hillsdale, NJ, 1981, 1--56.Google Scholar
- Shute, V.J. Stealth Assessment in Computer-Based Games to Support Learning. In S. Tobias and J.D. Fletcher, eds., Computer Games and Instruction. Information Age Publishers, Charlotte, NC, 2011, 503-- 524.Google Scholar
- Spring, F. and Pellegrino, J.W. The Challenge of Assessing Learning in Open Games/: HORTUS as a Case Study. Proc. GLS 2011, (2011), 200--208. Google ScholarDigital Library
- Stamper, J.C., Lomas, D., Ching, D., Ritter, S., Koedinger, K.R., and Steinhart, J. The Rise of the Super Experiment. Proc. EDM 2012, (2012).Google Scholar
- Young, M.F., Slota, S., Cutter, A.B., et al. Our Princess Is in Another Castle: A Review of Trends in Serious Gaming for Education. Review of Educational Research 82, 1 (2012), 61--89.Google Scholar
Index Terms
- In search of learning: facilitating data analysis in educational games
Recommendations
Clashroom: A Game to Enhance the Classroom Experience (Abstract Only)
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science EducationClashroom is a web-based fantasy game that was created to enhance the motivation and learning of a traditional classroom. It was created to address several drawbacks of other educational games by being broadly applicable to any course, requiring no ...
A Case Study of a Non-gamer Student's Learning Process in VISOLE
DIGITEL '10: Proceedings of the 2010 Third IEEE International Conference on Digital Game and Intelligent Toy Enhanced LearningVISOLE (Virtual Interactive Student-Oriented Learning Environment) is a constructivist pedagogical approach to empower game-based learning. With near real-life simulation gaming context and teacher facilitation (scaffolding and debriefing), VISOLE aims ...
A Case Study of an Academic Achievement-oriented Student in Game-based Learning
ICALT '11: Proceedings of the 2011 IEEE 11th International Conference on Advanced Learning TechnologiesVISOLE (Virtual Interactive Student-Oriented Learning Environment) is a teacher-facilitated constructivist pedagogical approach to empower game-based learning. In combination with scaffolding, situated cognition, reflection, and debriefing, VISOLE aims ...
Comments