Skip to main content
SpringerLink
Log in

Implementation of a Stereo Vision System for a Mixed Reality Robot Teleoperation Simulator

  • Conference paper
  • First Online:
Towards Autonomous Robotic Systems (TAROS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14136))

Included in the following conference series:

  • 592 Accesses

Abstract

This paper presents the preliminary work on a stereo vision system designed for a mixed reality-based simulator dedicated to robotic telemanipulation. The simulator encompasses a 3D visual display, stereo cameras, a desktop haptic interface, and a virtual model of a remote robotic manipulator. The integration of the stereo vision system enables accurate distance measurement in the remote environment and precise visual alignment between the cameras’ captured scene and the graphical representation of the virtual robot model. This paper delves into the technical aspects of the developed stereo system and shares the outcomes of its preliminary evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.3d-pluraview.com.

  2. 2.

    www.3dsystems.com/haptics-devices/touch.

  3. 3.

    https://library.vuforia.com/getting-started/vuforia-engine-package-unity.

  4. 4.

    https://library.vuforia.com/objects/model-targets.

References

  1. Adami, P., et al.: Effectiveness of VR-based training on improving construction workers’ knowledge, skills, and safety behavior in robotic teleoperation. Adv. Eng. Inform. 50, 101431 (2021)

    Google Scholar 

  2. Adil, E., Mikou, M., Mouhsen, A.: A novel algorithm for distance measurement using stereo camera. CAAI Trans. Intell. Technol. 7(2), 177–186 (2022)

    Article  Google Scholar 

  3. Dodgson, N.: Autostereoscopic 3D displays. Computer 38(8), 31–36 (2005)

    Article  Google Scholar 

  4. Farkhatdinov, I., Ryu, J.H.: Development of educational system for automotive engineering based on augmented reality. In: International Conference on Engineering Education and Research (2009)

    Google Scholar 

  5. Giudici, G., Omarali, B., Bonzini, A.A., Althoefer, K., Farkhatdinov, I., Jamone, L.: Feeling good: Validation of bilateral tactile telemanipulation for a dexterous robot. In: Iida, F., et al. (eds.) TAROS 2023, LNAI, vol. 14136, pp. 443–454. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43360-3_36

  6. Kaehler, A., Bradski, G.: Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library. Reilly Media, Inc. (2016)

    Google Scholar 

  7. Lin, Q., Kuo, C.: On applying virtual reality to underwater robot tele-operation and pilot training. Int. J. Virtual Reality 5(1), 71–91 (2001)

    Article  Google Scholar 

  8. Livatino, S., et al.: Intuitive robot teleoperation through multi-sensor informed mixed reality visual aids. IEEE Access 9, 25795–25808 (2021)

    Article  Google Scholar 

  9. Murugesan, Y.P., Alsadoon, A., Manoranjan, P., Prasad, P.: A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries. Int. J. Med. Robot. Comput. Assist. Surg. 14(3), e1889 (2018)

    Article  Google Scholar 

  10. Omarali, B., Denoun, B., Althoefer, K., Jamone, L., Valle, M., Farkhatdinov, I.: Virtual reality based telerobotics framework with depth cameras. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1217–1222. IEEE (2020)

    Google Scholar 

  11. Omarali, B., Javaid, S., Valle, M., Farkhatdinov, I.: Workspace scaling in virtual reality based robot teleoperation. In: Proceedings of the Augmented Humans International Conference 2023, pp. 98–104 (2023)

    Google Scholar 

  12. Omarali, B., Palermo, F., Althoefer, K., Valle, M., Farkhatdinov, I.: Tactile classification of object materials for virtual reality based robot teleoperation. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 9288–9294. IEEE (2022)

    Google Scholar 

  13. Sadekar, K.: Depth estimation using stereo camera and OpenCV (Python/C++) (2021)

    Google Scholar 

  14. Saliba, C., Bugeja, M.K., Fabri, S.G., Di Castro, M., Mosca, A., Ferre, M.: A training simulator for teleoperated robots deployed at CERN. In: ICINCO (2), pp. 293–300 (2018)

    Google Scholar 

  15. Shibata, T., Kim, J., Hoffman, D.M., Banks, M.S.: The zone of comfort: predicting visual discomfort with stereo displays. J. Vis. 11(8), 11 (2011)

    Article  Google Scholar 

  16. Szczurek, K.A., Prades, R.M., Matheson, E., Rodriguez-Nogueira, J., Castro, M.D.: Mixed reality human-robot interface with adaptive communications congestion control for the teleoperation of mobile redundant manipulators in hazardous environments. IEEE Access 10, 87182–87216 (2022)

    Article  Google Scholar 

  17. Vitanov, I., et al.: A suite of robotic solutions for nuclear waste decommissioning. Robotics 10(4), 112 (2021)

    Article  Google Scholar 

Download references

Acknowledements

Aaron Smiles was funded by the UKRI EPSRC EngD Data-Centric Engineering CDT at Queen Mary University of London (reference 2601988). The work was partially co-funded by the UKRI EPSRC Q-Arena grant EP/V035304/1. We thank Kaustubh Sadekar, Yik Lung Pang, and the National Oceanography Centre for their feedback.

Author information

Authors and Affiliations

  1. Queen Mary University of London, London, UK

    Aaron Lee Smiles, Kitti Dimitri Chavanakunakorn, Bukeikhan Omarali, Changjae Oh & Ildar Farkhatdinov

Authors
  1. Aaron Lee Smiles
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kitti Dimitri Chavanakunakorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bukeikhan Omarali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changjae Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ildar Farkhatdinov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron Lee Smiles .

Editor information

Editors and Affiliations

  1. University of Cambridge, Cambridge, UK

    Fumiya Iida

  2. University of Oxford, Oxford, UK

    Perla Maiolino

  3. University of Cambridge, Cambridge, UK

    Arsen Abdulali

  4. Brunel University London, Uxbridge, UK

    Mingfeng Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Smiles, A.L., Chavanakunakorn, K.D., Omarali, B., Oh, C., Farkhatdinov, I. (2023). Implementation of a Stereo Vision System for a Mixed Reality Robot Teleoperation Simulator. In: Iida, F., Maiolino, P., Abdulali, A., Wang, M. (eds) Towards Autonomous Robotic Systems. TAROS 2023. Lecture Notes in Computer Science(), vol 14136. Springer, Cham. https://doi.org/10.1007/978-3-031-43360-3_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43360-3_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43359-7

  • Online ISBN: 978-3-031-43360-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation