Skip to main content
SpringerLink
Log in

Monitoring children’s developmental progress using augmented toys and activity recognition

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Previous research has established the connection between the way in which children interact with objects and the potential early identification of children with autism. Those findings motivate our own work to develop "smart toys," objects embedded with wireless sensors that are safe and enjoyable for very small children, that allow detailed interaction data to be easily recorded. These sensor-enabled toys provide opportunities for autism research by reducing the effort required to collect and analyze a child’s interactions with objects. In the future, such toys may be a useful part of clinical and in-home assessment tools. In this paper, we discuss the design of a collection of smart toys that can be used to automatically characterize the way in which a child is playing. We use statistical models to provide objective, quantitative measures of object play interactions. We also developed a tool to view rich forms of annotated play data for later analysis. We report the results of recognition experiments on more than fifty play sessions conducted with adults and children as well as discuss the opportunities for using this approach to support video annotation and other applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. At the time of writing this article only thirty-five of the forty sessions has been completely labeled.

  2. Negative play behaviors do not necessarily correspond to the overall activity column in which they appear.

  3. These visualizations may have also helped increase inter-rater agreement.

References

  1. Adamson L, Bakeman R, Deckner D (2004) The development of symbol-infused joint engagement. Child Dev 75(4):1171–1187

    Article  Google Scholar 

  2. Agata Rozga P (2009) Personal Communication

  3. Aylward R, Lovell SD, Paradiso JA (2006) A compact, wireless, wearable sensor network for interactive dance ensembles. In: International workshop on wearable and implantable body sensor networks (BSN 2006), 3–5 Apr 2006, Cambridge, Massachusetts, USA, IEEE Computer Society, pp 65–70

  4. Baranek GT, Barnett C, Adams E, Wolcott N, Watson L, Crais E (2005) Object play in infants with autism: methodological issues in retrospective video analysis. Am J Occup Ther 59(1):20–30

    Article  Google Scholar 

  5. Baranek GT, David FJ, Poe MD, Stone WL, Watson LR (2005) Sensory experiences questionnaire: discriminating sensory features in young children with autism, developmental delays, and typical development. J Child Psychol Psychiatry 47(6):591–601

    Article  Google Scholar 

  6. Blasco PA (1991) Pitfalls in developmental diagnosis. Pediatr Clin North Am 38:1425–1438

    Google Scholar 

  7. Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  8. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46

    Article  Google Scholar 

  9. First L, Palfrey J (1994) The infant or young child with developmental delay. New Engl J Med 330:478–483

    Article  Google Scholar 

  10. Ganesan M, Russell NW, Rajan R, Welch N, Westeyn TL, Abowd GD (2010) Grip sensing in smart toys: a formative design method for user categorization. In: CHI EA ’10: Proceedings of the 28th of the international conference extended abstracts on human factors in computing systems. ACM, pp 3745–3750

  11. Gorbet MG, Orth M, Ishii H (1998) Triangles: tangible interface for manipulation and exploration of digital information topography. In: Proceedings of CHI ’98. ACM, pp 49–56, http://citeseer.ist.psu.edu/gorbet98triangles.html

  12. Jurafsky D, Martin JH (2000) Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. Prentice Hall PTR, Upper Saddle River

  13. Kehoe C, Cassell J, Goldman S, Dai J, Gouldstone I, MacLeod S, O’Day T, Pandolfo A, Ryokai K, Wang A (2004) Sam goes to school: story listening systems in the classroom. In: ICLS ’04: Proceedings of the 6th international conference on learning sciences, International Society of the Learning Sciences, pp 613–613

  14. Kernberg PF, Chazan SE, Normandin L (1998) The children’s play therapy instrument (cpti): description, development, and reliability studies. J Psychother Pract Res 7:196–207

    Google Scholar 

  15. Kientz JA (2008) Decision support for caregivers through embedded capture and access. PhD thesis, College of Computing, School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, USA

  16. Kientz JA, Arriaga RI, Chetty M, Hayes GR, Richardson J, Patel SN, Abowd GD (2007) Grow and know: understanding record-keeping needs for tracking the development of young children. In: CHI ’07: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Press, pp 1351–1360

  17. Kitamura Y, Itoh Y, Kishino F (2001) Real-time 3d interaction with active cube. In: CHI ’01: CHI ’01 extended abstracts on human factors in computing systems. ACM Press, New York, pp 355–356

  18. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  MathSciNet  MATH  Google Scholar 

  19. Lester J, Hannaford B, Borriello G (2004) ‘Are you with me?’—using accelerometers to determine if two devices are carried by the same person. In: Proceedings of the second international conference on pervasive computing, pp 33–50

  20. Mayrhofer R, Gellersen H (2007) Shake well before use: Authentication based on accelerometer data. In: Proceedings of 5th international conference of pervasive computing. Lecture notes in computer science, vol 4480. Springer, pp 144–161

  21. Minnen D, Westeyn T, Starner T, Ward J, Lukowicz P (2006) Performance metrics and evaluation issues for continuous activity recognition. In: Performance metrics for intelligent systems. NIST, Gaithersburg, pp 141–148

  22. Ozonoff S, Macari S, Young GS, Goldring S, Thompson M, Rogers SJ (2008) Atypical object exploration at 12 months of age is associated with autism in a prospective sample. Autism 12(5):457–472

    Article  Google Scholar 

  23. Presti P (2006) Bluesense—a wireless interface prototyping system. Master’s thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA

  24. Rijsbergen CJV (1979) Information retrieval, 2nd edn. Butterworth Scientific Ltd, London

    Google Scholar 

  25. Rosa Arriaga P (2007) Personal Communication

  26. Shannon CE (1949) Communication in the presence of noise. Proc IRE 37(1):10–21

    Article  MathSciNet  Google Scholar 

  27. Sharlin E, Itoh Y, Watson B, Kitamura Y, Sutphen S, Liu L (2002) Cognitive cubes: a tangible user interface for cognitive assessment. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Press, pp 347–354

  28. Shevell M, Ashwal S, Donley D, Flint J, Gingold M, Hirtz D, Majnemer A, Noetzel M, Sheth R (2003) Practice parameter: evaluation of the child with global developmental delay: report of the quality standards subcommittee of the American Academy of Neurology and the Practice Committee of the Child Neurology Society. Neurology 60:367–380

    Google Scholar 

  29. Technologies C (2010) Caring technologies website: http://www.caringtechnologies.com/. Retrieved 20 Aug 2010. World Wide Web electronic publication

  30. Thelen E (2000) Motor development as foundation and future of development psychology. J Behav Dev 24:385–397

    Article  Google Scholar 

  31. Wang P, Abowd GD, Rehg JM (2009) Quasi-periodic event analysis for social game retrieval. In: Proceedings of IEEE international conference on computer vision, IEEE

  32. Westeyn TL (2010) Child’s play: activity recognition for monitoring children’s developmental progress with augmented toys. PhD thesis, Georgia Institute of Technology

Download references

Author information

Authors and Affiliations

  1. School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Tracy L. Westeyn, Gregory D. Abowd, Thad E. Starner & Kimberly A. Weaver

  2. Interactive Media Technology Center, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Jeremy M. Johnson & Peter W. Presti

Authors
  1. Tracy L. Westeyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregory D. Abowd
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thad E. Starner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeremy M. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter W. Presti
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kimberly A. Weaver
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tracy L. Westeyn.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Westeyn, T.L., Abowd, G.D., Starner, T.E. et al. Monitoring children’s developmental progress using augmented toys and activity recognition. Pers Ubiquit Comput 16, 169–191 (2012). https://doi.org/10.1007/s00779-011-0386-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-011-0386-0

Keywords

Search

Navigation