Abstract
In this chapter, we describe the processes of designing and validating game-based learning assessment and/or support in two different games—Portal 2 (by Valve Corporation) and Earthquake Rebuild. The games represent cases of possible game-based learning (i.e., domain-generic and domain-specific) and provide good vehicles for testing the design decisions underlying stealth assessment and learning support. The chapter starts with a critical review of prior research on game-based assessment of competencies and learning support mechanisms in games, and then focuses on the particular design processes and findings from our two game cases. The review and findings suggested that process-oriented data mining and learning analytics methods help to capture the complex and open-ended learning trajectories in a game setting. They also illustrated how the evidence-centered assessment design and the learning context/task design should and can be interwoven in the early phase of game development. We conclude with a discussion relevant to developing and integrating the assessment and support of learning into other learning-game platforms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baker, R. S. (2007, April). Modeling and understanding students’ off-task behavior in intelligent tutoring systems. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1059–1068). San Jose, CA: ACM.
Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004, January). Detecting student misuse of intelligent tutoring systems. In Intelligent tutoring systems (pp. 531–540). Berlin: Springer.
Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.
Beck, J. E., & Mostow, J. (2008, January). How who should practice: Using learning decomposition to evaluate the efficacy of different types of practice for different types of students. In Intelligent Tutoring Systems (pp. 353–362). Berlin: Springer.
Clark, D. B., Nelson, B., Chang, H., D’Angelo, C. M., Slack, K., & Martinez-Garza, M. (2011). Exploring Newtonian mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students in Taiwan and the United States. Computers and Education, 57(3), 2178–2195.
Common Core State Standards Initiative. (2011). Common core state standards for mathematics. Retrieved May, 2011, from http://www.corestandards.org/assets/CCSSI_Math%20Standards.pdf.
Dede, C. (2012, May). Interweaving assessments into immersive authentic simulations: Design strategies for diagnostic and instructional insights. In Invitational Research Symposium on Technology Enhanced Assessments. Retrieved from http://www.k12center.org/rsc/pdf/session4-dede-paper-tea2012.pdf.
Dorval, M., & Pepin, M. (1986). Effect of playing a video game on a measure of spatial visualization. Perceptual and Motor Skills, 62, 159–162.
Feng, J., Spence, I., & Pratt, J. (2007). Playing an action video game reduces gender differences in spatial cognition. Psychological Science, 18, 850–855.
Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423(6939), 534–537.
Green, C. S., & Bavlier, D. (2007). Action-video-game experience alters the spatial resolution of vision. Psychological Science, 18(1), 88–94.
Hegarty, M., & Waller, D. (2004). A dissociation between mental rotation and perspective- taking spatial abilities. Intelligence, 32, 175–191.
Hwang, G., Sung, H., Hung, C., Huang, I., & Tsai, C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60(4, Special Issue on Personalized Learning), 623–638.
Ke, F., Shute, V., Erlebacher, G., Clark, K., & Ventura, M. (2014, June 9–10). Earthquake rebuild: Math learning through modeling and design. Poster presented at Cyberlearning Summit 2014, Madison, WI.
Kickmeier-Rust, M. D., & Albert, D. (2010). Micro‐adaptivity: Protecting immersion in didactically adaptive digital educational games. Journal of Computer Assisted Learning, 26(2), 95–105.
Leemkuil, H., & de Jong, T. (2012). Adaptive advice in learning with a computer-based knowledge management simulation game. Academy of Management Learning & Education, 11(4), 653–665.
Levy, R. (2014). Dynamic Bayesian network modeling of game based diagnostic assessments (CRESST Report 837). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
MacCallum-Stewart, E. (2011). Stealth learning in online games. In S. de Freitas & P. Maharg (Eds.), Digital games and learning (pp. 107–128). London: Continuum.
Miller, H. J., & Han, J. (Eds.). (2009). Geographic data mining and knowledge discovery. London: CRC Press.
Mislevy, R. J., Steinberg, L. S., Almond, R. G., & Lukas, J. F. (2006). Concepts, terminology, and basic models of evidence-centered design. In D. M. Williamson, R. J. Mislevy, & I. I. Bejar (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 15–47). Hillsdale, MI: Lawrence Elbaum.
Montello, D. R. (1993). Scale and multiple psychologies of space. In A. U. Frank & I. Campari (Eds.), Spatial information theory: A theoretical basis for GIS (Proceedings of COSIT’93. Lecture notes in Computer Science, Vol. 716, pp. 312–321). Berlin: Springer-Verlag.
Montello, D. R., & Golledge, R. G. (1999). Scale and detail in the cognition of geographic information. Report of the Specialist Meeting of Project Varenius, Santa Barbara, CA, May 14–16, 1998. Santa Barbara: University of California.
O’Rourke, E., Haimovitz, K., Ballweber, C., Dweck, C., & Popović, Z. (2014, April). Brain points: A growth mindset incentive structure boosts persistence in an educational game. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3339–3348). ACM.
Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.
Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.). (2011). Handbook of educational data mining. London: CRC Press.
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., et al. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53.
Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte, NC: Information Age Publishers.
Shute, V. J., & Ventura, M. (2013). Measuring and supporting learning in games: Stealth assessment. Cambridge, MA: MIT Press.
Shute, V. J., Masduki, I., & Donmez, O. (2010). Conceptual framework for modeling, assessing, and supporting competencies within game environments. Technology, Instruction, Cognition, and Learning, 8(2), 137–161.
Shute, V. J., Ventura, M., & Ke, F. (2015). The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills. Computers & Education, 80, 58–67.
Shute, V. J., Ventura, M., & Kim, Y. J. (2013). Assessment and learning of qualitative physics in Newton’s playground. Journal of Educational Research, 106(6), 423–430.
Siemens, G., & de Baker, R. S. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.
SoLAR, (February 27–March 1, 2011), 1st International Conference on Learning Analytics and Knowledge, Banff, Alberta; as cited in George Siemens and Phil Long, “Penetrating the Fog: Analytics in Learning and Education,” EDUCAUSE Review, vol. 46, no. 5 (September/October 2011).
Spence, I., Yu, J. J. J., Feng, J., & Marshman, J. (2009). Women match men when learning a spatial skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 1097–1103.
Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., & Warren, C., et al. (2012). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin. Advance online publication. doi:10.1037/a0028446
Vandenberg, S. G., & Kuse, A. R. (1978). Mental rotation, a group test of three-dimensional spatial visualization. Perceptual and Motor Skills, 69, 915–921.
Ventura, M., Shute, V. J., Wright, T., & Zhao, W. (2013). An investigation of the validity of the virtual spatial navigation assessment. Frontiers in Psychology, 4, 1–7.
Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques. San Francisco, CA: Morgan Kaufmann.
Wouters, P., & Van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412–425.
Yairi, T., Kawahara, Y., Fujimaki, R., Sato, Y., & Machida, K. (2006, July). Telemetry-mining: a machine learning approach to anomaly detection and fault diagnosis for space systems. In Space Mission Challenges for Information Technology, 2006. SMC-IT 2006. Second IEEE International Conference on (8-pp). Pasadena, CA: IEEE.
Zapata-Rivera, D., VanWinkle, W., Doyle, B., Buteux, A., & Bauer, M. (2009). Combining learning and assessment in assessment-based gaming environments: A case study from a New York City school. Interactive Technology and Smart Education, 6(3), 173–188.
Acknowledgments
The research was supported by the National Science Foundation, Award Number #1318784 and the James T. and Catherine D. MacArthur Foundation #11-99517. We also would like to thank Matthew Ventura for his work on the Portal 2 project—particularly relative to spatial skill assessment.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Ke, F., Shute, V. (2015). Design of Game-Based Stealth Assessment and Learning Support. In: Loh, C., Sheng, Y., Ifenthaler, D. (eds) Serious Games Analytics. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-05834-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-05834-4_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05833-7
Online ISBN: 978-3-319-05834-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)