Skip to main content
SpringerLink
Log in

Integration of Augmented Reality and Digital Twins in a Teleoperated Disassembly System

  • Conference paper
  • First Online:
Advances in Remanufacturing (IWAR 2023)

Part of the book series: Lecture Notes in Mechanical Engineering ((LNME))

Included in the following conference series:

  • 48 Accesses

Abstract

Disassembly, a key step in remanufacturing, is almost always performed manually due to uncertainties associated with end-of-life products. In some cases, such as EV battery disassembly, the operator could be exposed to safety hazards. Teleoperation is an effective solution to this safety issue. This paper proposes a teleoperated disassembly system that integrates augmented reality (AR) and digital twin technology. The system establishes bidirectional human-in-the-loop communication between the physical system (the teleoperated robot) and the virtual model. Using AR, the system can provide real-time robot status information to the operator by superimposing a virtual robot on the physical robot and displaying relevant data either in the form of text or a line chart. Furthermore, the proposed approach allows operators to enter the control loop when anomalies are detected, enabling them to receive real-time feedback while remotely controlling the robot. Experiments have demonstrated the effectiveness of the system in monitoring robot status during complex disassembly and offering a more intuitive approach for remotely controlling the robot, thereby improving the safety and efficiency of disassembly operations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ramírez FJ, Aledo JA, Gamez JA, Pham DT (2020) Economic modelling of robotic disassembly in end-of-life product recovery for remanufacturing. Comput Indus Eng 142

    Google Scholar 

  2. Liu J, Zhou Z, Pham DT, Xu W, Cui J, Yang C (2020) Service platform for robotic disassembly planning in remanufacturing. J Manuf Syst 57:338–356

    Article  Google Scholar 

  3. Huang J, Pham DT, Wang Y, Qu M, Ji C, Su S, Xu W, Liu Q, Zhou Z (2020) A case study in human–robot collaboration in the disassembly of press-fitted components. Proc Inst Mech Eng, Part B: J Eng Manuf 654–664

    Google Scholar 

  4. Liu Q, Liu Z, Xu W, Tang Q, Zhou Z, Pham DT (2019) Human-robot collaboration in disassembly for sustainable manufacturing. Int J Prod Res 57(12):4027–4044

    Article  Google Scholar 

  5. Gerbers R, Wegener K, Dietrich F, Dröder K (2018) Safe, flexible and productive human-robot-collaboration for disassembly of lithium-ion batteries. In: Recycling of lithium-ion batteries: the lithorec way, pp 99–126

    Google Scholar 

  6. Huang J, Pham DT, Li R, Qu M, Wang Y, Kerin M, Su S, Ji C, Mahomed O, Khalil R, Stockton D (2021) An experimental human-robot collaborative disassembly cell. Comput Indus Eng 155

    Google Scholar 

  7. Pan Y, Chen C, Li D, Zhao Z, Hong J (2021) Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device. Robot Comput-Integr Manuf 71

    Google Scholar 

  8. Solanes JE, Munoz A, Gracia L, Martí A, Girbés-Juan V, Tornero J (2020) Teleoperation of industrial robot manipulators based on augmented reality. Int J Adv Manuf Technol 111:1077–1097

    Article  Google Scholar 

  9. Chen H, Sun G, Chen C, Wan B (2019) Augmented reality based assembly guidance for spacecraft conductive network. In: IOP conference series: earth and environmental science, vol 252, no 4, p 042076

    Google Scholar 

  10. Deshpande A, Kim I (2018) The effects of augmented reality on improving spatial problem solving for object assembly. Adv Eng Inform 38:760–775

    Google Scholar 

  11. Li C, Zheng P, Li S, Pang Y, Lee CK (2022) AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robot Comput-Integr Manuf 76

    Google Scholar 

  12. Lin Y, Song S, Meng MQH (2016) The implementation of augmented reality in a robotic teleoperation system. In: 2016 IEEE international conference on real-time computing and robotics (RCAR), pp 134–139

    Google Scholar 

  13. Gattullo M, Evangelista A, Uva AE, Fiorentino M, Gabbard JL (2020) What, how, and why are visual assets used in industrial augmented reality? A systematic review and classification in maintenance, assembly, and training (from 1997 to 2019). IEEE Trans Visual Comput Graph 28(2):1443–1456

    Article  Google Scholar 

  14. Liu YK, Ong SK, Nee AYC (2022) State-of-the-art survey on digital twin implementations. Adv Manuf 10(1):1–23

    Article  Google Scholar 

  15. Shi Y, Shen W, Wang L, Longo F, Nicoletti L, Padovano A (2022) A cognitive digital twins framework for human-robot collaboration. Procedia Comput Sci 200:1867–1874

    Article  Google Scholar 

  16. MoveIt!. http://moveit.ros. Accessed 9 July 2023

  17. KDL: Kinematics and Dynamics Library. http://www.orocos.org/kdl. Accessed 9 July 2023

  18. Sucan IA, Moll M, Kavraki LE (2012) The open motion planning library. IEEE Robot Automat Mag 19(4):72–82

    Article  Google Scholar 

  19. LBR iiwa. https://www.kuka.com/en-gb/products/robotics-systems/industrial-robots/lbr-iiwa. Accessed 14 Aug 2023

  20. HoloLens 2. https://www.microsoft.com/en-gb/hololens/buy. Accessed 14 Aug 2023

  21. Robot Operating System. https://www.ros.org/. Accessed 14 Aug 2023

Download references

Acknowledgements

This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/N018524/1, United Kingdom. Special thanks are due to Dr Mark Cooke for the loan of his personal WiFi router to help the project while a replacement unit was being ordered.

Author information

Authors and Affiliations

  1. University of Birmingham, Birmingham, B15 2TT, UK

    Feifan Zhao & Duc Truong Pham

Authors
  1. Feifan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duc Truong Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feifan Zhao .

Editor information

Editors and Affiliations

  1. Department of Engineering, University of Campania Luigi Vanvitelli, Aversa, Italy

    Marcello Fera

  2. Department of Industrial Engineering, University of Salerno, Fisciano, Italy

    Mario Caterino

  3. Department of Engineering, University of Campania Luigi Vanvitelli, Aversa, Italy

    Roberto Macchiaroli

  4. Department of Mechanical Engineering, The University of Birmingham, Birmingham, UK

    Duc Truong Pham

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, F., Pham, D.T. (2024). Integration of Augmented Reality and Digital Twins in a Teleoperated Disassembly System. In: Fera, M., Caterino, M., Macchiaroli, R., Pham, D.T. (eds) Advances in Remanufacturing. IWAR 2023. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-52649-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-52649-7_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-52648-0

  • Online ISBN: 978-3-031-52649-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation