Abstract
Our research goal is to design an agent that can begin with low-level sensors and effectors and autonomously learn high-level representations and actions through interaction with the environment. This chapter focuses on the problem of learning representations. We present four principles for autonomous learning of representations in a developing agent, and we demonstrate how these principles can be embodied in an algorithm. In a simulated environment with realistic physics, we show that an agent can use these principles to autonomously learn useful representations and effective hierarchical actions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Of course, for any fixed number of phenomena and any fixed number of statements about the phenomena, this is equivalent to a state-based representation. The differences are that the number of phenomena will change over time, and phenomena are considered largely independent of other phenomena.
- 2.
A video describing QLAP can be seen at http://www.youtube.com/watch?v=xJ0g-NoerZ0.
References
Arbib, M. (1992). Schema theory. Encyclopedia of Artificial Intelligence, 2, 1427–1443.
Cohen, P. R., Oates, T., Beal, C. R., Adams, N. (2002). Contentful mental states for robot baby. In Proceedings of the 18th national conference on artificial intelligence (AAAI-2002). San Francisco/Cambridge: AAAI/MIT.
Dean, T., & Kanazawa, K. (1989). A model for reasoning about persistence and causation. Computational Intelligence, 5(2), 142–150.
DeCasper, A. J., & Carstens, A. (1981). Contingencies of stimulation: effects of learning and emotions in neonates. Infant Behavior and Development, 4, 19–35.
Drescher, G. L. (1991). Made-up minds: a constructivist approach to artificial intelligence. Cambridge: MIT.
Fayyad, U., & Irani, K. (1992). On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8(1), 87–102.
Friedman, N., & Goldszmidt, M. (1996). Discretizing continuous attributes while learning bayesian networks. In Proceedings of the thirteenth international conference on machine learning (ICML’96) (pp. 157–165). Los Altos: Morgan Kaufmann.
Gergely, G., & Watson, J. (1999). Early socio-emotional development: contingency perception and the social-biofeedback model. In P. Rochat (Ed.), Early social cognition: understanding others in the first months of life (pp. 101–136). Mahwah: Lawrence Erlbaum Associates.
Goodman, N., Mansinghka, V., Tenenbaum, J. (2007). Learning grounded causal models. In Proceedings of the twenty-ninth annual conference of the cognitive science society (pp. 305–310). Hillsdale: Erlbaum.
Gunderson, J., & Gunderson, L. (2008). Robots, reasoning, and reification. New York: Springer.
Hester, T., & Stone, P. (2009). Generalized model learning for reinforcement learning in factored domains. In Proceedings of the 8th international conference on autonomous agents and multiagent systems (vol. 2, pp. 717–724). Richland: International Foundation for Autonomous Agents and Multiagent Systems.
Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press.
Klein, J. (2003). Breve: a 3d environment for the simulation of decentralized systems and artificial life. In Proceedings of the eighth international conference on artificial life (pp. 329–334). Cambridge: MIT Press.
Kuipers, B. (1994). Qualitative reasoning. Cambridge: MIT.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.
Mandler, J. (2004a). The foundations of mind, origins of conceptual thought. New York: Oxford University Press.
Mandler, J. (2004b). A synopsis of the foundations of mind: origins of conceptual thought. Developmental Science, 7(5), 499–505.
Modayil, J., & Kuipers, B. (2007). Autonomous development of a grounded object ontology by a learning robot. In Proceedings of the national conference on artificial intelligence (AAAI 2007) (vol. 22, p. 1095).
Mugan, J. (2010). Autonomous qualitative learning of distinctions and actions in a developing agent. PhD thesis, University of Texas at Austin.
Mugan, J., & Kuipers, B. (2007). Learning to predict the effects of actions: synergy between rules and landmarks. In IEEE 6th international conference on development and learning, ICDL 2007 (pp. 253–258). New York: Institute of Electrical and Electronics Engineers.
Mugan, J., & Kuipers, B. (2012). Autonomous learning of high-level states and actions in continuous environments. IEEE Transactions in Autonomous Mental Development, 4(1), 70–86.
Needham, A., Barrett, T., Peterman, K. (2002). A pick-me-up for infants’ exploratory skills: early simulated experiences reaching for objects using ‘sticky mittens’ enhances young infants’ object exploration skills. Infant Behavior and Development, 25(3), 279–295.
Oudeyer, P., Kaplan, F., Hafner, V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11(2), 265–286.
Payne, V. G., & Isaacs, L. D. (2007). Human motor development: a lifespan approach. New York: McGraw-Hill Humanities/Social Sciences/Languages.
Piaget, J. (1952). The origins of intelligence in children. New York: Norton.
Pierce, D., & Kuipers, B. (1997). Map learning with uninterpreted sensors and effectors. Artificial Intelligence, 92(1–2), 169–227.
Puterman, M. (1994). Markov decision problems. New York: Wiley.
Rosenberg, K., & Trevathan, W. (2002). Birth, obstetrics and human evolution. BJOG: An International Journal of Obstetrics & Gynaecology, 109(11), 1199–1206.
Schmidhuber, J. (1991). Curious model-building control systems. In IEEE International Joint Conference on Neural Networks (vol. 2, pp. 1458–1463). New York: Institute of Electrical and Electronics Engineers.
Skinner, B. (1961). Cumulative record. New York: Appleton-Century-Crofts.
Smith, R. (2004). Open dynamics engine v 0.5 user guide. http://www.ode.org/ode-latest-userguide.html. Accessed 15 April 2012.
Stober, J., & Kuipers, B. (2008). From pixels to policies: a bootstrapping agent. In IEEE 7th international conference on development and learning, ICDL 2008 (pp. 103–108). New York: Institute of Electrical and Electronics Engineers.
Strehl, A., Diuk, C., Littman, M. (2007). Efficient structure learning in factored-state MDPs. In Proceedings of the national conference on artificial intelligence (AAAI 2007) (vol. 22, p. 645). Menlo Park, CA; Cambridge, MA; London: AAAI; MIT; 1999.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge: MIT.
Sutton, R. S., Precup, D., Singh, S. (1999). Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1–2), 181–211.
Vigorito, C. M., & Barto, A. G. (2010). Intrinsically motivated hierarchical skill learning in structured environments. IEEE Transactions on Autonomous Mental Development (TAMD), 2(2), 132–143.
Winston, P. (1992). Artificial intelligence, 3rd edn. Reading: Addison-Wesley.
Acknowledgements
This work has taken place in the Intelligent Robotics Lab at the Artificial Intelligence Laboratory, The University of Texas at Austin. Research of the Intelligent Robotics lab is supported in part by grants from the National Science Foundation (IIS-0713150).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Mugan, J., Kuipers, B. (2013). Autonomous Representation Learning in a Developing Agent. In: Baldassarre, G., Mirolli, M. (eds) Computational and Robotic Models of the Hierarchical Organization of Behavior. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39875-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-39875-9_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39874-2
Online ISBN: 978-3-642-39875-9
eBook Packages: Computer ScienceComputer Science (R0)