Skip to main content
SpringerLink
Log in

Tactile-based active object discrimination and target object search in an unknown workspace

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

The tasks of exploring unknown workspaces and recognizing objects based on their physical properties are challenging for autonomous robots. In this paper, we present strategies solely based on tactile information to enable robots to accomplish such tasks. (1) An active exploration approach for the robot to explore unknown workspaces; (2) an active touch objects learning method that enables the robot to learn efficiently about unknown objects via their physical properties (stiffness, surface texture, and center of mass); and (3) an active object recognition strategy, based on the knowledge the robot has acquired. Furthermore, we propose a tactile-based approach for estimating the center of mass of rigid objects. Following the active touch for workspace exploration, the robotic system with the sense of touch in fingertips reduces the uncertainty of the workspace up to 65 and 70% compared respectively to uniform and random strategies, for a fixed number of samples. By means of the active touch learning method, the robot achieved 20 and 15% higher learning accuracy for the same number of training samples compared to uniform strategy and random strategy, respectively. Taking advantage of the prior knowledge obtained during the active touch learning, the robot took up to 15% fewer decision steps compared to the random method to achieve the same discrimination accuracy in active object discrimination task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. \({\mathbb {R}}\) is the set of real numbers, \({\mathbb {N}}\) is the set of natural numbers.

  2. Due to the constraints of the workspace, it is difficult for the UR10 robot to explore more than five objects. Therefore, five out of 20 objects were selected randomly at uniform for the evaluation.

References

  • Atkeson, C. G., An C. H., & Hollerbach, J. M. (1985) Rigid body load identification for manipulators. In IEEE conference on decision and control (pp. 996–1002).

  • Bhattacharya, A., & Mahajan, R. (2003). Temperature dependence of thermal conductivity of biological tissues. Physiological Measurement, 24(3), 769.

    Article  Google Scholar 

  • Chathuranga , D. S., Wang, Z., Ho, V. A., Mitani, A., & Hirai, S. (2013). A biomimetic soft fingertip applicable to haptic feedback systems for texture identification. In IEEE international symposium on haptic audio visual environments and games (pp. 29–33).

  • Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799.

    Article  Google Scholar 

  • Chu, V., McMahon, I., Riano, L., McDonald, C. G., He, Q., et al. (2013). Using robotic exploratory procedures to learn the meaning of haptic adjectives. In IEEE international conference on robotics and automation (pp. 3048–3055).

  • Csato, L., & Opper, M. (2002). Sparse on-line gaussian processes. Neural Computation, 14(3), 641–668.

    Article  MATH  Google Scholar 

  • Dahiya, R. S., Metta, G., Valle, M., Adami, A., & Lorenzelli, L. (2009). Piezoelectric oxide semiconductor field effect transistor touch sensing devices. Applied Physics Letters, 95(034), 105.

    Google Scholar 

  • Dahiya, R. S., Metta, G., Valle, M., & Sandini, G. (2010). Tactile sensing—From humans to humanoids. IEEE Transactions on Robotics, 26(1), 1–20.

    Article  Google Scholar 

  • Dallaire, P., Giguere, P., Edmond, D., & Chaib-draa, B. (2014). Autonomous tactile perception: A combined improved sensing and bayesian nonparametric approach. Robotics and Autonomous Systems, 6(4), 422–435.

    Article  Google Scholar 

  • Denei, S., Maiolino, P., Baglini, E., & Cannata, G. (2015). On the development of a tactile sensor for fabric manipulation and classification for industrial applications. In IEEE international conference on intelligent robots and systems (pp. 5081–5086).

  • Fishel, J. A., & Loeb, G. E. (2012). Bayesian exploration for intelligent identification of textures. Frontiers in Neurorobotics, 6, 4.

    Article  Google Scholar 

  • Friedl, K. E., Voelker, A. R., Peer, A., & Eliasmith, C. (2016). Human-inspired neurorobotic system for classifying surface textures by touch. IEEE Robotics and Automation Letters, 1, 516–523.

    Article  Google Scholar 

  • Giguere, P., & Dudek, G. (2011). A simple tactile probe for surface identification by mobile robots. IEEE Transactions on Robotics, 27(3), 534–544.

    Article  Google Scholar 

  • Hughes, D., & Correll, N. (2015). Texture ecognition and localization in amorphous robotic skin. Bioinspiration and Biomimetics, 10(055), 002.

    Google Scholar 

  • Hu, H., Han, Y., Song, A., Chen, S., Wang, C., & Wang, Z. (2014). A finger-shaped tactile sensor for fabric surfaces evaluation by 2-dimensional active sliding touch. Sensors, 14, 4899–4913.

    Article  Google Scholar 

  • Jamali, N., Ciliberto, C., Rosasco, L., & Natale, L. (2016). Active perception: Building objects models using tactile exploration. In IEEE international conference on humanoid robots.

  • Jamali, N., & Sammut, C. (2011). Majority voting: Material classification by tactile sensing using surface texture. IEEE Transactions on Robotics, 27(3), 508–521.

    Article  Google Scholar 

  • Jia, Y., & Tian, J. (2010). Surface patch reconstruction from ‘one-dimensional’ tactile data. IEEE Transactions on Automation Science and Engineering, 7, 400–407.

    Article  Google Scholar 

  • Kaboli, M., & Cheng, G. (2018). Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin. IEEE Transaction on Robotics, 34(2), 1–19.

    Article  Google Scholar 

  • Kaboli, M., Feng, D., & Cheng, G. (2017a). Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin. International Journal of Humanoid Robotics, 15(1), 18–51.

    Google Scholar 

  • Kaboli, M., Feng, D., Yao, K., Lanillos, P., & Cheng, G. (2017b). A tactile-based framework for active object learning and discrimination using multimodal robotic skin. IEEE Robotics and Automation Letters, 2(4), 2143–2150.

    Article  Google Scholar 

  • Kaboli, M., Long, A., & Cheng, G. (2015a). Humanoids learn touch modalities identification via multi-modal robotic skin and robust tactile descriptors. Advanced Robotics, 29(21), 1411–1425.

    Article  Google Scholar 

  • Kaboli, M., Mittendorfer, P., Hugel, V., & Cheng, G. (2014). Humanoids learn object properties from robust tactile feature descriptors via multi-modal artificial skin. In IEEE international conference on humanoid robots (pp. 187–192).

  • Kaboli, M., Rosaand, A. D. L., Walker, R., & Cheng, G. (2016a). Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties. In IEEE international conference on robotics and automation (pp. 2242–2247).

  • Kaboli, M., Walker, R., & Cheng, G. (2015b). In-hand object recognition via texture properties with robotic hands, artificial skin, and novel tactile descriptors. In IEEE international conference on humanoid robots (pp. 2242–2247).

  • Kaboli, M., Yao, K., & Cheng, G. (2016b). Tactile-based manipulation of deformable objects with dynamic center of mass. In IEEE international conference on humanoid robots.

  • Kaltenbrunner, M., Sekitani, T., Reeder, J., Yokota, T., Kuribara, K., et al. (2013). An ultra-lightweight design for imperceptible plastic electronics. Nature, 499, 455–463.

    Article  Google Scholar 

  • Lederman, S. J. (1981). The perception of surface roughness by active and passive touch. Bulletin of the Psychonomic Society, 18, 253–255.

    Article  Google Scholar 

  • Lee, S., Reuveny, A., Reeder, J., Lee, S., Jin, H., et al. (2016). A transparent bending-insensitive pressure sensor. Nature Nanotechnology, 11, 472–478.

    Article  Google Scholar 

  • Lepora, N. F., Evans, M., Fox, C. W., Diamond, M. E., Gurney, K., & Prescott, T. J . (2010). Naive bayes texture classification applied to whisker data from a moving robot. In The international joint conference on neural networks (pp. 1–8).

  • Lepora, N. F., Martinez-Hernandez, U., & Prescott, T. J. (2013). Active touch for robust perception under position uncertainty. In IEEE International Conference on Robotics and Automation (pp. 3020–3025).

  • Liarokapis, M. V., Calli, B., Spiers, A., & Dollar, A. M. (2015). Unplanned, model-free, single grasp object classification with underactuated hands and force sensors. In IEEE international conference on intelligent robots and systems (pp. 5073–5080).

  • Liu, H., Greco, J., Song, X., Bimbo, J., & Althoefer, K. (2013). Tactile image based contact shape recognition using neural network. In IEEE international conference on multisensor fusion and integration for intelligent systems (MFI) (pp. 138–143).

  • Liu, H., Song, X., Nanayakkara, T., Seneviratne, L. D., & Althoefer, K. (2012). A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor. In IEEE international conference on robotics and automation (pp. 1410–1415).

  • Liu, H., Nguyen, K. C., Perdereau, V., Bimbo, J., Back, J., Godden, M., et al. (2015). Finger contact sensing and the application in dexterous hand manipulation. Autonomous Robots, 39(1), 25–41.

    Article  Google Scholar 

  • Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. A. (2007). Active policy learning for robot planning and exploration under uncertainty. Robotics: Science and Systems, 3, 321–328.

    MATH  Google Scholar 

  • Martins, R., Ferreira, J. F., & Dias, J. (2014). Touch attention bayesian models for robotic active haptic exploration of heterogeneous surfaces. In International conference on intelligent robots and systems (pp. 1208–1215).

  • Mayol-Cuevas, W., Juarez-Guerrero, J., & Munoz-Gutierrez, S. (1998). A first approach to tactile texture recognition. In IEEE international conference on systems, man, and cybernetics (Vol. 5, pp. 4246–4250).

  • Mohamad Hanif, N. H. H., Chappell, P. H., Cranny, A., & White, N. M . (2015). Surface texture detection with artificial fingers. In 37th annual international conference of the IEEE engineering in medicine and biology society (pp. 8018–8021).

  • Nawrocki, A. R., Matsuhisa, N., Yokota, T., & Someya, T. (2016). 300-nm imperceptible, ultraflexible, and biocompatible e-skin fit with tactile sensors and organic transistors. Advanced Electronic Materials. https://doi.org/10.1002/aelm.201500452.

  • Nguyen, K. C., & Perdereau, V. (2013). Fingertip force control based on max torque adjustment for dexterous manipulation of an anthropomorphic hand. In 2013 IEEE/RSJ international conference on intelligent robots and systems (pp. 3557–3563).

  • Ohmura, Y., Kuniyosh, Y., & Nagakubo, A . (2006). Conformable and scalable tactile sensor skin for curved surfaces. In IEEE international conference on robotics and automation (pp. 1348–1353).

  • Papakostas, T. V., Lima, J., & Lowe, M. (2002). A large area force sensor for smart skin applications. IEEE Sensors, 5, 1620–1624.

    Article  Google Scholar 

  • Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  • Rebguns, A., Ford, D., & Fasel, I. R. (2011). Infomax control for acoustic exploration of objects by a mobile robot. In Lifelong learning.

  • Robles-De-La-Torre, G. (2006). The importance of the sense of touch in virtual and real environments. IEEE MultiMedia, 13, 24–30.

    Article  Google Scholar 

  • Saal, H. P., Ting, J. A., & Vijayakumar, S. (2010). Active sequential learning with tactile feedback. In AISTATS (pp. 677–684).

  • Schmitz, A., Maiolinoand, P., Maggiali, M., Natale, L., Cannata, G., & Metta, G. (2011). Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Transactions on Robotics, 27(3), 389–400.

    Article  Google Scholar 

  • Schneider, A., Sturm, J., Stachniss, C., Reisert, M., Burkhardt, H., & Burgard, W. (2009). Object identification with tactile sensors using bag-of-features. In IEEE international conference on intelligent robots and systems (pp. 243–248).

  • Sinapov, J., Sukhoy, V., Sahai, R., & Stoytchev, A. (2011). Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Transactions on Robotics, 27(3), 488–497.

    Article  Google Scholar 

  • Song, A., Han, Y., Hu, H., & Li, J. (2014). A novel texture sensor for fabric texture measurement and classification. IEEE Transactions on Instrumentation and Measurement, 63(7), 1739–1747.

    Article  Google Scholar 

  • Strehl, A., & Ghosh, J. (2002). Cluster ensembles—A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3, 583–617.

    MathSciNet  MATH  Google Scholar 

  • Strohmayr, M. W., Worn, H., & Hirzinger, G . (2013). The DLR artificial skin part i: Uniting sensitivity and robustness. In IEEE international conference on robotics and automation (pp. 1012–1018).

  • Tanaka, D., Matsubara, T., Ichien, K., & Sugimoto, K. (2014). Object manifold learning with action features for active tactile object recognition. In IEEE international conference on intelligent robots and systems (pp. 608–614).

  • Ulmen, J, & Cutkosky, M. (2010). A robust low-cost and low-noise artificial skin for human-friendly robots. In IEEE international conference on robotics and automation (pp. 4836–4841).

  • Watanabe, K., Sohgawa, M., Kanashima, T., Okuyama, M., & Norna, H. (2013). Identification of various kinds of papers using multi-axial tactile sensor with micro-cantilevers. In World haptics conference (pp. 139–144).

  • Xu, D., Loeb, G. E., & Fishel, J. A. (2013). Tactile identification of objects using bayesian exploration. In IEEE international conference on robotics and automation (pp. 3056–3061).

  • Yao, K., Kaboli, M., & Cheng, G. (2017). Tactile-based object center of mass exploration and discrimination. In IEEE international conference on humanoid robotics (pp. 876–881).

  • Yi, Z., Calandra, R., Veiga, F., van Hoof, H., Hermans, T., Zhang, Y., et al. (2016). Active tactile object exploration with gaussian processes. In IEEE international conference on intelligent robots and systems (pp. 4925–4930).

  • Yogeswaran, N., Dang, W., Navaraj, W. T., Shakthivel, D., et al. (2015). New materials and advances in making electronic skin for interactive robots. Advanced Robotics, 29(21), 1359–1373.

    Article  Google Scholar 

  • Yu, Y., Arima, T., & Tsujio, S. (2005). Estimation of object inertia parameters on robot pushing operation. In IEEE international conference on robotics and automation (pp. 1657–1662).

  • Yu, Y., Kiyokawa, T., & Tsujio, S. (2004). Estimation of mass and center of mass of unknown and graspless cylinder-like object. International Journal of Information Acquisition, 1(01), 47–55.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the European Commission under Grant Agreements PITN-GA-2012-317488-CONTEST.

Author information

Authors and Affiliations

  1. Institute for Cognitive Systems, Department of Electrical and Computer Engineering, Technische Universität München, Karlstrasse 45, 80333, Munich, Germany

    Mohsen Kaboli, Kunpeng Yao, Di Feng & Gordon Cheng

Authors
  1. Mohsen Kaboli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kunpeng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gordon Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohsen Kaboli.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 215115 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaboli, M., Yao, K., Feng, D. et al. Tactile-based active object discrimination and target object search in an unknown workspace. Auton Robot 43, 123–152 (2019). https://doi.org/10.1007/s10514-018-9707-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9707-8

Keywords

Search

Navigation