Abstract
Multi-modality is a fundamental feature that characterizes biological systems and lets them achieve high robustness in understanding skills while coping with uncertainty. Relatively recent studies showed that multi-modal learning is a potentially effective add-on to artificial systems, allowing the transfer of information from one modality to another. In this paper we propose a general architecture for jointly learning visual and motion patterns: by means of regression theory we model a mapping between the two sensorial modalities improving the performance of artificial perceptive systems. We present promising results on a case study of grasp classification in a controlled setting and discuss future developments.
Chapter PDF
Similar content being viewed by others
Keywords
References
Rizzolatti, G., Craighero, L.: The Mirror-Neuron System. Annual Review of Neuroscience 27, 169–192 (2004)
Harris, C., Stephens, M.: A Combined Corner and Edge Detector. In: Proceedings of The Fourth Alvey Vision Conference, pp. 147–151 (1988)
Mikolajczyk, K., Schmid, C.: Scale and Affine Invariant Interest Point Detectors. IJCV 60(1), 63–86 (2004)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. Trans on PAMI 27(10) (2005)
Csurka, G., Dance, C.R., Fan, L., Bray, C.: Visual Categorization with Bag of Keypoints. In: ECCV (2004)
Ferrari, V., Tuytelaars, T., Van Gool, L.: Simultaneous Object Recognition and Segmentation from Single or Multiple Model Views. IJVC 67(2) (2006)
Lo Gerfo, L., Rosasco, L., Odone, F., De Vito, E., Verri, A.: Spectral Algorithms for Supervised Learning. Neural Computation 20(7) (2008)
Yao, Y., Rosasco, L., Caponnetto, A.: On Early Stopping in Gradient Descent Learning. Constructive Approximation 26(2) (2007)
Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Computation 17 (2005)
De Vito, E., Rosasco, L., Caponnetto, A., Piana, M., Verri, A.: Some Properties of Regularized Kernel Methods. Journal of Machine Learning Research 5 (2004)
Baldassarre, L., Barla, A., Rosasco, L., Verri, A.: Learning vector valued functions with spectral regularization (preprint)
Gallese, V., Fadiga, L., Fogassi, L., Rizzolatti, G.: Action Recognition in the Premotor Cortex. Brain 119, 593–609 (1996)
Metta, G., Sandini, G., Natale, L., Craighero, L., Fadiga, L.: Understanding Mirror Neurons: A Bio-Robotic Approach. Interaction Studies 7, 197–232 (2006)
Hartigan, J.A., Wong, M.A.: A K-Means Clustering Algorithm. Applied Statistics 28(1) (1979)
Cutkosky, M.: On grasp choice, grasp models and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation (1989)
Castellini, C., Orabona, F., Metta, G., Sandini, G.: Internal Models of Reaching and Grasping. Advanced Robotics 21(13) (2007)
Buhlmann, P.: Boosting for High-Dimensional Linear Models. Annals of Statistics 34(2) (2006)
Micchelli, C.A., Pontil, M.: Kernels for Multi-task Learning. In: NIPS (2004)
Rifkin, R., Yeo, G., Poggio, T.: Regularized Least-Squares Classification. In: Advances in Learning Theory: Methods, Models and Applications (2003)
Argyriou, A., Maurer, A., Pontil, M.: An Algorithm for Transfer Learning in a Heterogeneous Environment. In: ECML/PKDD (1), pp. 71–85 (2008)
Jacob, L., Bach, F., Vert, J.P.: Clustered Multi-Task Learning: a Convex Formulation. In: NIPS (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Noceti, N. et al. (2009). Towards a Theoretical Framework for Learning Multi-modal Patterns for Embodied Agents. In: Foggia, P., Sansone, C., Vento, M. (eds) Image Analysis and Processing – ICIAP 2009. ICIAP 2009. Lecture Notes in Computer Science, vol 5716. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04146-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-04146-4_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04145-7
Online ISBN: 978-3-642-04146-4
eBook Packages: Computer ScienceComputer Science (R0)