Abstract
Interdisciplinary exchange of ideas and tools can accelerate scientific progress. For example, findings from developmental and vision science have spurred recent advances in artificial intelligence and computer vision. However, relatively little attention has been paid to how artificial intelligence and computer vision can facilitate research in developmental science. The current study presents AutoViDev—an automatic video-analysis tool that uses machine learning and computer vision to support video-based developmental research. AutoViDev identifies full body position estimations in real-time video streams using convolutional pose machine-learning algorithms. AutoViDev provides valuable information about a variety of behaviors, including gaze direction, facial expressions, posture, locomotion, manual actions, and interactions with objects. We present a high-level architecture of the framework and describe two projects that demonstrate its usability. We discuss the benefits of applying AutoViDev to large-scale, shared video datasets and highlight how machine learning and computer vision can enhance and accelerate research in developmental science.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Vosniadou, S., Brewer, W.F.: Theories of knowledge restructuring in development. Rev. Educ. Res. 57, 51–67 (1987)
Meltzoff, A.N., Kuhl, P.K., Movellan, J., Sejnowski, T.J.: Foundations for a new science of learning. Science 325, 284–288 (2009)
Mitchell, T.M.: The discipline of machine learning. Carnegie Mellon University, School of Computer Science, Machine Learning Department Pittsburgh, PA (2006)
Gilmore, R.O., Raudies, F., Jayaraman, S.: What accounts for developmental shifts in optic flow sensitivity? In: 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pp. 19–25 (2015)
Raudies, F., Gilmore, R.O.: Visual motion priors differ for infants and mothers. Neural Comput. 26, 2652–2668 (2014)
Raudies, F., Gilmore, R.O., Kretch, K.S., Franchak, J.M., Adolph, K.E.: Understanding the development of motion processing by characterizing optic flow experienced by infants and their mothers. In: Proceedings of the IEEE Conference on Development and Learning (2012)
Smith, B., Yu, C., Yoshida, H., Fausey, C.M.: Contributions of head-mounted cameras to studying the visual environments of infants and young children. J. Cogn. Dev. 16, 407–419 (2015)
Bambach, S., Lee, S., Crandall, D.J., Yu, C.: Lending a hand: detecting hands and recognizing activities in complex egocentric interactions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1949–1957 (2015)
Schlesinger, M., Amso, D., Johnson, S.P.: The neural basis for visual selective attention in young infants: a computational account. Adapt. Behav. 15, 135–148 (2007)
Ossmy, O., Hoch, J.E., MacAlpine, P., Hasan, S., Stone, P., Adolph, K.E.: Variety wins: Soccer-playing robots and infant walking. Front Neurorobot 12, 19 (2018)
Adolph, K., Gilmore, R.O., Kennedy, J.L.: Video data and documentation will improve psychological science. Psychol. Sci. Agenda (2017)
Adolph, K.: Video data and documentation will improve psychological science. APS Observer 29, 23–25 (2016)
Gilmore, R.O., Adolph, K.E.: Video can make behavioral science more reproducible. Nat. Hum. Behav. 1 (2017). s41562-41017
Gilmore, R.O., Adolph, K.E., Millman, D.S.: Curating identifiable data for sharing: The databrary project. In: Scientific Data Summit (NYSDS), New York, pp. 1–6 (2016)
Gilmore, R.O., Adolph, K.E., Millman, D.S., Gordon, A.S.: Transforming education research through open video data sharing. Adv. Eng. Educ. 5, 1–17 (2016)
Gilmore, R.O., Kennedy, J.L., Adolph, K.E.: Practical solutions for sharing data and materials from psychological research. Adv. Methods Pract. Psychol. Sci. 1, 121–130 (2018)
Adolph, K., Tamis-LeMonda, C.S., Gilmore, R.O., Soska, K.C.: Play & Learning Across a Year (PLAY) Project Summit, 29 June 2018, Philadelphia. Databrary (2018). http://doi.org/2010.17910/B17917.17724. Accessed 30 Aug 2018
Ramakrishna, V., Munoz, D., Hebert, M., Bagnell, J.A., Sheikh, Y.: Pose machines: articulated pose estimation via inference machines. In: European Conference on Computer Vision, pp. 33–47. Springer (2014)
Simon, T., Joo, H., Matthews, I.A., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 2 (2017)
Wei, S.-E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4724–4732. (2016)
Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint arXiv:1611.08050 (2016)
Hoch, J., O’Grady, S., Adolph, K.: It’s the journey, not the destination: Locomotor exploration in infants. Dev. Sci. (2018)
Lee, D.K., Cole, W.G., Golenia, L., Adolph, K.E.: The cost of simplifying complex developmental phenomena: a new perspective on learning to walk. Dev. Sci. 21, e12615 (2018)
Thurman, S.L., Corbetta, D.: Spatial exploration and changes in infant-mother dyads around transitions in infant locomotion. Dev. Psychol. 53, 1207–1221 (2017)
Adolph, K.: It’s the journey not the destination: Locomotor exploration in infants (2015). Databrary. http://doi.org/10.17910/B17917.17140
Scott, K., Chu, J., Schulz, L.: Lookit (Part 2): assessing the viability of online developmental research, results from three case studies. Open Mind 1, 15–29 (2017)
Scott, K., Schulz, L.: Lookit (part 1): a new online platform for developmental research. Open Mind 1, 4–14 (2017)
Messinger, D.S.: Facial expressions in 6-month old infants and their parents in the still face paradigm and attachment at 15 months in the strange situation (2014). Databrary. http://doi.org/10.17910/B17059D
McNamara, Q., De La Vega, A., Yarkoni, T.: Developing a comprehensive framework for multimodal feature extraction. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1567–1574 (2017)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ossmy, O., Gilmore, R.O., Adolph, K.E. (2020). AutoViDev: A Computer-Vision Framework to Enhance and Accelerate Research in Human Development. In: Arai, K., Kapoor, S. (eds) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, vol 944. Springer, Cham. https://doi.org/10.1007/978-3-030-17798-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-17798-0_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17797-3
Online ISBN: 978-3-030-17798-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)