Automated and unobtrusive measurement of physical activity in an interactive playground

Highlights

• Introduction of an automated, unobtrusive measure for player exertion based on top-down depth camera video.

• Experimental validation of the measure and comparison with other, unobtrusive and sensor-based, exertion measures.

• Demonstration that player exertion levels can be manipulated by tuning a single parameter in an interactive playground.

Abstract

Promoting physical activity is one of the main goals of interactive playgrounds. To validate whether this goal is met, we need to measure the amount of physical player activity. Traditional methods of measuring activity, such as observations or annotations of game sessions, require time and personnel. Others, such as heart rate monitors and accelerometers, need to be worn by the player. In this paper, we investigate whether physical activity can be measured unobtrusively by tracking players using depth cameras and applying computer vision algorithms. In a user study with 32 players, we measure the players’ speed while playing a game of tag, and demonstrate that our measures correlate well with exertion measured using heart rate sensors. This makes the method an attractive alternative to either manual coding or the use of worn devices. We also compare our approach to other exertion measurement methods. Finally, we demonstrate and discuss its potential for automated, unobtrusive measurements and real-time game adaptation.

Introduction

Technology has become embedded into many aspects of children’s lives, including children’s play, and studies have suggested that it can limit its users to screen-based solitary interactions (Radesky and Christakis, 2016). A clear example of this is that children currently spend a significant amount of time consuming online digital media, and a considerable part is dedicated to digital gaming (Blumberg et al., 2013). Most young people play video games at least occasionally and many of them play daily (Desai, Krishnan-Sarin, Cavallo, Potenza, 2010, Ferguson, Olson, 2013). In doing so, the opportunities available for children to engage in full-body physical activity and in social interactions, both essential for their development, can be drastically reduced (Aggio, Ogunleye, Voss, Sandercock, 2012, Carson, Hunter, Kuzik, Gray, Poitras, Chaput, Saunders, Katzmarzyk, Okely, Connor Gorber, Kho, Sampson, Lee, Tremblay, 2016, Plötner, Over, Carpenter, Tomasello, 2015). Nonetheless, digital games can also be used to encourage positive aspects of play (Calvert et al., 2013). Exertion games or active video games (AVGs) provide the entertainment value of digital games while encouraging players to engage in physical activity (Müller, Khot, Gerling, Mandryk, 2016, Peng, Crouse, Lin, 2013). Interactive playgrounds are instrumented spaces where exertion games can be played, usually with small groups of players (Poppe et al., 2014). These playgrounds combine elements of traditional playgrounds with digital elements to promote key aspects of play, including physical activity (Moreno et al., 2013). In general, these approaches are designed to put body movement as a core part of the gameplay in order to motivate players to exert themselves (e.g., Landry, Parés, 2014, Müller, Toprak, Graether, Walmink, Bongers, van den Hoven, 2012).

This does not necessarily mean that players engage in appropriate levels of exertion (Peng et al., 2013). Players could move very little, or players might move too much and burn out quickly. Knowing beforehand how to stimulate players appropriately is difficult, and is likely to differ between individuals. One promising alternative to control the level of exertion is to adapt the stimulation of the players in real-time, based on measurements of the players’ levels of physical activity (Altimira et al., 2017).

Traditional methods of measuring physical activity in play include the annotation of game sessions, interviews and self-reports (Hands, Larkin, 2006, Loprinzi, Cardinal, 2011). These validated methods provide varied information, but the outcome only becomes available after, not during, the game session. Annotation requires observers to categorize specific actions using annotation schemes, and it is typically performed on recorded game sessions (Bakker, Markopoulos, de Kort, 2008, Moreno, van Delden, Reidsma, Poppe, Heylen, 2012). Questionnaires to evaluate physical activity are filled in after the game sessions since they ask players about their experiences. A different approach to measuring physical activity in games is to use sensors such as accelerometers, pedometers or heart rate monitors. These sensors are worn or carried and provide continuous in-game measurements so the data can be accessed directly. This presents an attractive alternative to manual annotation and opens up the possibility of in-game adaptation of gameplay based on sensor measurements. While these methods are suitable for the study of play in a laboratory setting, the requirement of fitting sensors and registering them to the game session hinders their use in everyday play. In the current paper, we present an approach that overcomes this limitation by measuring exertion in real-time and unobtrusively, using overhead cameras.

Our contributions are two-fold. First, we present a method to obtain in-game measurements of physical activity using a completely unobtrusive method. We track the players using cameras and computer vision algorithms and determine their level of activity by measuring the average movement speed. We compare this approach to a number of alternative sensor-based approaches and questionnaires. Second, we demonstrate that the level of physical activity can be influenced in real-time by adapting a single gameplay element in an interactive playground. To evaluate our approach, we conduct a user study with eight groups of four players. Together, these contributions demonstrate the potential of automatically and unobtrusively measuring and modulating physical activity in AVGs.

This paper is structured as follows. Section 2 presents an overview of how physical activity is currently measured and evaluated in active video games. In Sections 3 and  4 we describe the physical setup and the design of our user studies respectively. We then present and discuss the results of this study in Section 5, and conclude in Section 6 by discussing avenues for future work.