Abstract
The traditional machine learning paradigm is commonly used for intelligent robot design, which causes problems of low learning initiative, lack of adaptability with uncertainty and bad expansibility of knowledge and ability. According to the new research direction called cognitive development learning, an incremental and autonomous visual learning algorithm based on internally motivated Q learning is proposed. The visual novelty is calculated by online PCA. The active learning and accumulation of knowledge is implemented in the form of updating PCA subspace, which is guided by internally motivated Q learning. By equipped with the proposed algorithm, robot makes next learning decision by judging the novelty between learned knowledge and what is seen now. Experimental results show that the algorithm has the ability of autonomous exploring and learning, actively guiding robot to learning new knowledge, acquiring knowledge and developing intelligence in an online and incremental manner.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vernon D, Metta G, Sandini G (2007) A survey of artificial cognitive systems: implications for the autonomous development of mental capabilities in computational agents. IEEE Trans Evol Comput 11(2):151–180
Kazuhiko K, Stephen G (2006) From intelligent control to cognitive control. In: ISORA 2006 11th international symposium on robotics and applications (ISORA), pp 1–9
Weng J, McClelland J, Pentland A, Sporns O, Stockman I, Sur M, Thelen E (2001) Autonomous mental development by robots and animals. Science 291(5504):599–600
Asada M, Hosoda K, Kuniyoshi Y, Ishiguro H, Inui T, Yoshikawa Y, Ogino M, Yoshida C (2009) Cognitive developmental robotics: a survey. IEEE Trans Auton Mental Dev 1(1):12–34
Jochen JS, Heiko W (2006) Recent trends in online learning for cognitive robot. In: ESANN’2006 Proceedings—European symposium on artificial neural networks, pp 77–87
Rachid A, Raja C, Aurelie C, Sara F, Matthieu H, Vincent M, Emrah AS (2006) Towards human-aware cognitive robots. Am Assoc Artif Intell 19(6):91–95
Brooks R, Breazeal C, Irie R, Kemp C, Marjanovic M, Scassellati B, Williamson M (1998) Alternative essences of intelligence. In: Proceedings of the American association of artificial intelligence (AAAI) 42(3–4):146–163
Asada M, MacDorman K, Ishiguro H, Kuniyoshi Y (2001) Cognitive developmental robotics as a new paradigm for the design of humanoid robots. Robot Auton Sys 37:185–193
Kazuhiko K, Stephen MG, Palis R, Erdem E, Joseph F (2008) Hall implementation of cognitive control for a humanoid robot
Gordon SM, Kawamura K, Wilkes DM (2010) Neuromorphically inspired appraisal-based decision making in a cognitive robot. IEEE Trans Auton Mental Dev 2(1):17–39
Weng J (2004) Developmental robotics theory and experiments. Int J Humanoid Robot 1(2):199–235
Weng J (2002) A theory for mentally developing robot. In: Proceedings of the IEEE second international conference on development and learning (ICDL 2002), June 12–15, MIT, Cambridge, pp 131–140
Pfeifer R, Bongard JC (2006) How the body shapes the way we think: a new view of intelligence. MIT Press, Cambridge
Oudeyer P-Y, Kaplan F, Hafner V (2007) Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput, 11
Gao Y, Dong-yue C, Li-ming Z (2005) An exploration of autonomous developing robot with real time vision learning. J Fudan Univ 44(6):964–970
Qu X, Yao M (2009) Cognitive robotics: a survey. chinese intelligence and automation conference, Nanjing
Hua-long Y, Chang-ming Z, Hai-bo L, Guo-chang G, Jing S (2007) A survey on developmental robotics. Caai Trans Intell Sys 2(4):34–39
Deci E, Ryan R (1985) Intrinsic motivation and self-determination in human behavior. Plenum, New York
Schultz W, Dayan P, Montague P (1997) A neural substrate of prediction and reward. Science 275:1593–1599
Skocaj D, Leonardis A (2003) Weighted and robust incremental method for subspace learning. In: Proceedings of the IEEE international conference on computer vision. Nice, France, pp 1494–1501
Mitchell TM (2004) Machine learning. Machine Press, Beijing, pp 263–280
Xiao NF, Nahavandi S (2002) A reinforcement learning approach for robot control in an unknown environment. In: Proceedings of the IEEE international conference on industrial technology. Bangkok II, pp 1096–1099
Yichi W, Usher JM (2005) Application of reinforcement learning for agent-based production scheduling. Eng Appl Artif Intell 18:73–82
Gao Y (2006) Progress of reinforcement learning research machine learning and its application. Tsinghua University Press, Beijing, pp 116–134
Watkins C, Dayan P (1992) Q-Learning. Mach Learn 8:279–292
Acknowledgments
We thank the support of China National Science Foundation: No.61070113 and the Zhejiang Science Project Foundation for University students: No. 2010R403071.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media B.V.
About this paper
Cite this paper
Xinyu, Q., Minghai, Y., Qinlong, G. (2012). An Incremental and Autonomous Visual Learning Algorithm Based on Internally Motivated Q Learning. In: He, X., Hua, E., Lin, Y., Liu, X. (eds) Computer, Informatics, Cybernetics and Applications. Lecture Notes in Electrical Engineering, vol 107. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-1839-5_76
Download citation
DOI: https://doi.org/10.1007/978-94-007-1839-5_76
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-1838-8
Online ISBN: 978-94-007-1839-5
eBook Packages: EngineeringEngineering (R0)