Chek Tien Tan
Yusuf Pisan
ABSTRACT
Understanding player experiences is central to game design. Video captures of players is a common practice for obtaining rich reviewable data for analysing these experiences. However, not enough has been done in investigating ways of preprocessing the video for a more efficient analysis process. This paper consolidates and extends prior work on validating the feasibility of using automated facial expressions analysis as a natural quantitative method for evaluating player experiences. A study was performed on participants playing a first-person puzzle shooter game (Portal 2) and a social drawing trivia game (Draw My Thing), and results were shown to exhibit rich details for inferring player experiences from facial expressions. Significant correlations were also observed between facial expression intensities and self reports from the Game Experience Questionnaire. In particular, the challenge dimension consistently showed positive correlations with anger and joy. This paper eventually presents a case for increasing the application of computer vision in video analyses of gameplay.
- Baltrusaitis, T., McDuff, D., Banda, N., Mahmoud, M., Kaliouby, R. E., Robinson, P., and Picard, R. Realtime inference of mental states from facial expressions and upper body gestures. Face and Gesture 2011 (Mar. 2011), 909--914.Google Scholar
- Branco, P. Computer-based facial expression analysis for assessing user experience. PhD thesis, 2006.Google Scholar
- Buenaposada, J. M., Muñoz, E., and Baumela, L. Recognising facial expressions in video sequences. Pattern Analysis and Applications 11, 1 (Oct. 2007), 101--116. Google ScholarDigital Library
- Cohn, J. F., and Sayette, M. A. Automated Facial Coding. Behavioral Research Methods (2010), 1--33.Google Scholar
- Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience, vol. 54 of P. S. Series. Harper & Row, 1990.Google Scholar
- Drachen, A., Nacke, L., Yannakakis, G., and Pedersen, A. Correlation between heart rate, electrodermal activity and player experience in First-Person Shooter games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games, ACM (2010), 49--54. Google ScholarDigital Library
- Ekman, P., and Friesen, W. V. The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1, 1 (1969), 49--98.Google ScholarCross Ref
- Ekman, P., Friesen, W. V., and Hager, J. C. Facial Action Coding System, vol. 48. A Human Face, 2002.Google Scholar
- Gagne, A., Seif El-Nasr, M., and Shaw, C. A Deeper Look at the use of Telemetry for Analysis of Player Behavior in RTS Games. In International Conference on Entertainment Computing ICEC (2011). Google ScholarDigital Library
- Ghijsen, M. Facial Expression Analysis for Human Computer Interaction. Affective Computing, IEEE Transactions on 2, 3 (2004), 147--161.Google Scholar
- Isbister, K., and Schaffer, N. Game Usability: Advancing the Player Experience, 1 ed. CRC Press, 2008.Google ScholarCross Ref
- Mandryk, R. L., Atkins, M. S., and Inkpen, K. M. A continuous and objective evaluation of emotional experience with interactive play environments. Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI '06 (2006), 1027. Google ScholarDigital Library
- McDuff, D., el Kaliouby, R., Kassam, K., and Picard, R. Acume: A new visualization tool for understanding facial expression and gesture data. In Face and Gesture 2011, IEEE (Mar. 2011), 591--596.Google Scholar
- Medler, B., John, M., and Lane, J. Data cracker: developing a visual game analytic tool for analyzing online gameplay. In Proceedings of the 2011 annual conference on Human factors in computing systems, ACM (2011), 2365--2374. Google ScholarDigital Library
- Moura, D., Seif El-Nasr, M., and Shaw, C. D. Visualizing and Understanding Players âĂŹ Behavior in Video Games: Discovering Patterns and Supporting Aggregation and Comparison. In ACM SIGGRAPH Symposium on Video Games (2011), 2--7. Google ScholarDigital Library
- Nacke, L. E. Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment. PhD thesis, 2009.Google Scholar
- Nacke, L. E., Grimshaw, M. N., and Lindley, C. a. More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers 22, 5 (Sept. 2010), 336--343. Google ScholarDigital Library
- Nacke, L. E., and Lindley, C. Flow and immersion in first-person shooters: measuring the player's gameplay experience. Proceedings of the 2008 Conference on Future (2008), 81--88. Google ScholarDigital Library
- Örtqvist, D., and Liljedahl, M. Immersion and Game-play Experience: A Contingency Framework. International Journal of Computer Games Technology 2010 (2010), 1--11. Google ScholarDigital Library
- Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., and Fuller, T. User-centered design in games. In Design, vol. 28. 2003, 883--906. Google ScholarDigital Library
- Paris Mavromoustakos Blom, Bakkes, S., Tan, C. T., Whiteson, S., Roijers, D., Valenti, R., and Gevers, T. Towards Personalised Gaming via Facial Expression Recognition. In Proceedings of Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AAAI Press (Palo, 2014).Google Scholar
- Russell, J. A. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6 (1980), 1161--1178.Google ScholarCross Ref
- Saragih, J. M., Lucey, S., and Cohn, J. F. Deformable Model Fitting by Regularized Landmark Mean-Shift. International Journal of Computer Vision 91, 2 (Sept. 2010), 200--215. Google ScholarDigital Library
- Tan, C., and Johnston, A. Towards a Nondisruptive, Practical, and Objective Automated Playtesting Process. In Workshops at the Seventh Artificial Intelligence and Interactive Digital Entertainment Conference (2011), 25--28.Google Scholar
- Tan, C. T., Rosser, D., Bakkes, S., and Pisan, Y. A feasibility study in using facial expressions analysis to evaluate player experiences. In Proceedings of The 8th Australasian Conference on Interactive Entertainment, ACM Press (2012), 5. Google ScholarDigital Library
- Zammitto, V., Seif El-Nasr, M., and Newton, P. Exploring Quantitative Methods for Evaluating Sports Games. Signals (2010), 8--11.Google Scholar
Index Terms
- Inferring Player Experiences Using Facial Expressions Analysis
Recommendations
Correlating Facial Expressions and Subjective Player Experiences in Competitive Hearthstone
FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital GamesIn this study, we used recordings of players’ facial expressions that are captured during competitive Hearthstone games to analyse the correlation between in-game player affective responses and subjective post-game self-reports. With this, we aimed to ...
The interplay between immersion and appeal in video games
A 2í 2 study on immersion and appeal of video games is presented.Immersion and appeal are found to be highly related.Immersion is found to be unaffected by playing experience.Appeal is found to be affected both by game played and playing experience. ...
The Fall of the Fourth Wall: Designing and Evaluating Interactive Spectator Experiences
Over the past several years, the live-streaming of digital games has experienced a vast increase in popularity, coinciding with the rise of eSports as an entertainment medium. For a rapidly growing audience, streamed content provides material from an ...
Comments