Skip to main content

Advertisement

SpringerLink
Log in

Digital Twin in smart manufacturing: remote control and virtual machining using VR and AR technologies

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Smart manufacturing becomes a major trend of manufacturing industry in the context of Industry 4.0. The integration of physical manufacturing machines and digitized virtual counterparts is promoted by emerging concepts and technologies of Digital Twin. Aiming to seamlessly integrate the cyberspace and physical world by constructing a bi-directional mapping system, Digital Twin can highly improve the user experience and production efficiency in smart manufacturing. So far little attention has been paid to the mapping from the cyberspace to the physical world in Digital Twin. Without this mapping, operations on the digitized virtual machines are incapable of working on the physical ones, which actually limits the applicability of Digital Twin to more manufacturing processes. In addition, the traditional 2D interactive interface in the cyberspace is limited in visualizing the large amount of digital data and providing concise information to improve the operation efficiency. To optimize the conventional Digital Twin mapping system, this paper proposes a modular-based Digital Twin system for smart manufacturing, where the bi-directional real-time mapping of the cyber–physical space is established through socket communication. Moreover, the proposed Digital Twin system aggregates the functions of remote control and virtual machining using virtual reality and augmented reality. These two essential functions are designed to provide an immersive and friendly operating environment as well as a vivid preview of machining outcomes to improve production efficiency, minimize machining cost, and avoid potential risks. The feasibility and effectiveness of the proposed Digital Twin system are demonstrated by implementing the system on a CNC milling machine where the control latency and virtual machining accuracy are verified. The proposed Digital Twin system can be utilized as an essential part of smart manufacturing, having high potential to be applied to various industrial machines and smart systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Akpan IJ, Shanker M (2019) A comparative evaluation of the effectiveness of virtual reality, 3D visualization and 2D visual interactive simulation: an exploratory meta-analysis. Simulation 95(2):145–170

    Google Scholar 

  • Andrew K (2018) Smart manufacturing. Int J Prod Res 56(1–2):508–517

    Google Scholar 

  • Azuma RT (1997) A survey of augmented reality. Presence Teleoper Virtual Environ 6(4):355–385

    Article  Google Scholar 

  • Bazilevs Y, Deng X, Korobenko A, di Scalea LF, Todd MD, Taylor SG (2015) Isogeometric fatigue damage prediction in large-scale composite structures driven by dynamic sensor data. J Appl Mech. https://doi.org/10.1115/1.4030795

    Article  Google Scholar 

  • Bergs T, Gierlings S, Auerbach T, Klink A, Schraknepper D, Augspurger T (2021) The concept of Digital Twin and digital shadow in manufacturing. Procedia CIRP 101:81–84

    Article  Google Scholar 

  • Botkina D, Hedlind M, Olsson B, Henser J, Lundholm T (2018) Digital Twin of a cutting tool. Procedia CIRP 72:215–218

    Article  Google Scholar 

  • Burdea GC, Coiffet P (2003) Virtual reality technology. Wiley, New York

    Book  Google Scholar 

  • Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M (2011) Augmented reality technologies, systems and applications. Multimed Tools Appl 51(1):341–377

    Article  Google Scholar 

  • Cerrone A, Hochhalter J, Heber G, Ingraffea A (2014) On the effects of modeling as-manufactured geometry: toward Digital Twin. Int J Aerosp Eng. https://doi.org/10.1155/2014/439278

    Article  Google Scholar 

  • Chen EY, Itoh M (2010) A whitelist approach to protect SIP servers from flooding attacks. In: 2010 IEEE International Workshop Technical Committee on Communications Quality and Reliability (CQR 2010), 2010. IEEE, pp 1–6

  • Cimino C, Negri E, Fumagalli L (2019) Review of Digital Twin applications in manufacturing. Comput Ind 113:103130

    Article  Google Scholar 

  • Coronado PDU, Lynn R, Louhichi W, Parto M, Wescoat E, Kurfess T (2018) Part data integration in the shop floor Digital Twin: mobile and cloud technologies to enable a manufacturing execution system. J Manuf Syst 48:25–33

    Article  Google Scholar 

  • Debroy T, Zhang W, Turner J, Babu SS (2017) Building Digital Twins of 3D printing machines. Scr Mater 135:119–124

    Article  Google Scholar 

  • Erwig M (2000) The graph Voronoi diagram with applications. Netw Int J 36(3):156–163

    MathSciNet  MATH  Google Scholar 

  • Gao Y, Lv H, Hou Y, Liu J, Xu W (2019) Real-time modeling and simulation method of Digital Twin production line. In: 2019 IEEE 8th joint international information technology and artificial intelligence conference (ITAIC), 2019. IEEE, pp 1639–1642

  • Grieves M (2014) Digital Twin: manufacturing excellence through virtual factory replication. White Paper 1:1–7

  • Grieves M (2017) Digital Twin: manufacturing excellence through virtual factory replication, 2014. White Paper

  • Haider S, Abbas A, Zaidi AK (2000) A multi-technique approach for user identification through keystroke dynamics. In: 2000 IEEE international conference on systems, man and cybernetics, 2000, vol 2. IEEE, pp 1336–1341

  • Havard V, Jeanne B, Lacomblez M, Baudry D (2019) Digital Twin and virtual reality: a co-simulation environment for design and assessment of industrial workstations. Prod Manuf Res 7(1):472–489

    Google Scholar 

  • Hu Z, He D, Gui W, Yang C (2008) Remote monitoring system of rectifier based on improved heartbeat mechanism. J Comput Appl 28(2):363–366

    Google Scholar 

  • Hu L, Nguyen N-T, Tao W, Leu MC, Liu XF, Shahriar MR, Al Sunny SMN (2018) Modeling of cloud-based Digital Twins for smart manufacturing with MT connect. Procedia Manuf 26:1193–1203

    Article  Google Scholar 

  • Ibanez AS, Figueras JP (2013) Vuforia v1. 5 SDK: analysis and evaluation of capabilities. Master’s Thesis, Universitat Politècnica de Catalunya

  • Kaarlela T, Pieskä S, Pitkäaho T (2020) Digital Twin and virtual reality for safety training. In: 2020 11th IEEE international conference on cognitive infocommunications (CogInfoCom), 2020. IEEE, pp 000115–000120

  • Kadir AA, Xu X, Hämmerle E (2011) Virtual machine tools and virtual machining–a technological review. Robot Comput Integr Manuf 27(3):494–508

    Article  Google Scholar 

  • Kritzinger W, Karner M, Traar G, Henjes J, Sihn W (2018) Digital Twin in manufacturing: a categorical literature review and classification. IFAC-PapersOnLine 51(11):1016–1022

    Article  Google Scholar 

  • Kritzler M, Funk M, Michahelles F, Rohde W (2017) The virtual twin: controlling smart factories using a spatially-correct augmented reality representation. In: Proceedings of 7th international conference on the Internet of Things, 2017, pp 1–2

  • Laaki H, Miche Y, Tammi K (2019) Prototyping a Digital Twin for real time remote control over mobile networks: application of remote surgery. IEEE Access 7:20325–20336

    Article  Google Scholar 

  • Lin W, Fu J (2006) Modeling and application of virtual machine tool. In: 16th International conference on artificial reality and tele-existence—workshops (ICAT’06), 2006. IEEE, pp 16–19

  • Liu J, Zhou H, Tian G, Liu X, Jing X (2019) Digital Twin-based process reuse and evaluation approach for smart process planning. Int J Adv Manuf Technol 100(5):1619–1634

    Article  Google Scholar 

  • Liu S, Lu S, Li J, Sun X, Lu Y, Bao J (2021) Machining process-oriented monitoring method based on Digital Twin via augmented reality. Int J Adv Manuf Technol 113(11):3491–3508

    Article  Google Scholar 

  • Lu Y, Liu C, Kevin I, Wang K, Huiyue H, Xu X (2020) Digital Twin-driven smart manufacturing: connotation, reference model, applications and research issues. Robot Comput Integr Manuf 61:101837

    Article  Google Scholar 

  • Luo LF, Sun S, Meng QG, Li QQ (2010) The process planning simulation of multi-axis numerical control based on virtual reality. Adv Mater Res 97:3146–3150

    Article  Google Scholar 

  • Martínez GS, Sierla S, Karhela T, Vyatkin V (2018) Automatic generation of a simulation-based Digital Twin of an industrial process plant. In: IECON 2018—44th annual conference of the IEEE Industrial Electronics Society. IEEE, pp 3084–3089

  • Negri E, Fumagalli L, Cimino C, Macchi M (2019) FMU-supported simulation for CPS Digital Twin. Procedia Manuf 28:201–206

    Article  Google Scholar 

  • Prakash GL, Prateek M, Singh I (2014) Data encryption and decryption algorithms using key rotations for data security in cloud system. In: 2014 International conference on signal propagation and computer technology (ICSPCT 2014), 2014. IEEE, pp 624–629

  • Pérez L, Rodríguez-Jiménez S, Rodríguez N, Usamentiaga R, García DF (2020) Digital Twin and virtual reality based methodology for multi-robot manufacturing cell commissioning. Appl Sci 10(10):3633

    Article  Google Scholar 

  • Qi Q, Tao F (2018) Digital Twin and big data towards smart manufacturing and Industry 4.0: 360 degree comparison. IEEE Access 6:3585–3593

    Article  Google Scholar 

  • Radharamanan R (2002) Virtual manufacturing: an emerging technology. In: 2002 Annual conference, 2002, pp 7–1297

  • Schroeder G, Steinmetz C, Pereira CE, Muller I, Garcia N, Espindola D, Rodrigues R (2016) Visualising the Digital Twin using web services and augmented reality. In: 2016 IEEE 14th international conference on industrial informatics (INDIN), 2016. IEEE, pp 522–527

  • Slater M, Sanchez-Vives MV (2016) Enhancing our lives with immersive virtual reality. Front Robot AI 3:74

    Article  Google Scholar 

  • Soori M, Arezoo B, Habibi M (2013) Dimensional and geometrical errors of three-axis CNC milling machines in a virtual machining system. Comput Aided Des 45(11):1306–1313

    Article  Google Scholar 

  • Tao F, Zhang M (2017) Digital Twin shop-floor: a new shop-floor paradigm towards smart manufacturing. IEEE Access 5:20418–20427

    Article  Google Scholar 

  • Tuegel EJ, Ingraffea AR, Eason TG, Spottswood SM (2011) Reengineering aircraft structural life prediction using a Digital Twin. Int J Aerosp Eng 1687–5966:2011

    Google Scholar 

  • Uhlemann TH-J, Schock C, Lehmann C, Freiberger S, Steinhilper R (2017) The Digital Twin: demonstrating the potential of real time data acquisition in production systems. Procedia Manuf 9:113–120

    Article  Google Scholar 

  • Wanasinghe TR, Wroblewski L, Petersen BK, Gosine RG, James LA, De Silva O, Mann GKI, Warrian PJ (2020) Digital Twin for the oil and gas industry: overview, research trends, opportunities, and challenges. IEEE Access 8:104175–104197

    Article  Google Scholar 

  • Wang S, Mao Z, Zeng C, Gong H, Li S, Chen B (2010) A new method of virtual reality based on Unity3D. In: 2010 18th International conference on geoinformatics, 2010. IEEE, pp 1–5

  • Weyer S, Meyer T, Ohmer M, Gorecky D, Zühlke D (2016) Future modeling and simulation of CPS-based factories: an example from the automotive industry. IFAC-PapersOnLine 49(31):97–102

    Article  Google Scholar 

  • Xie J (2012) Research on key technologies base Unity3D game engine. In: 2012 7th International conference on computer science and education (ICCSE), 2012. IEEE, pp 695–699

  • Yildiz E, Møller C, Bilberg A (2020) Virtual factory: Digital Twin based integrated factory simulations. Procedia CIRP 93:216–221

    Article  Google Scholar 

  • Zhu Z, Liu C, Xu X (2019) Visualisation of the Digital Twin data in manufacturing by using augmented reality. Procedia CIRP 81:898–903

    Article  Google Scholar 

  • Zhu L, Wang J, Chen E, Yang J, Wang W (2008) Applications of virtual reality in turn-milling centre. In: 2008 IEEE international conference on automation and logistics, 2008. IEEE, pp 2302–2305

Download references

Acknowledgements

This work was supported in part by the Science and Technology Commission of Shanghai Municipality under Grant No. 19511103503.

Author information

Authors and Affiliations

  1. UM-SJTU Joint Institute, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, China

    Ruoxin Geng, Mian Li, Zongyang Hu, Zixi Han & Ruixiang Zheng

  2. Department of Automation, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, China

    Mian Li

  3. Global Institute of Future Technology, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, China

    Mian Li

Authors
  1. Ruoxin Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zongyang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zixi Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruixiang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mian Li.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Replication of results

The code of the case study in Sect. 4 is available online (https://github.com/viviGeng/DT_CNC_Milling_Machine).

Additional information

Responsible Editor: Chao Hu

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Topical Collection: Advanced Optimization Enabling Digital Twin Technology.

Guest Editors: C Hu, VA González, T Kim, O San, Z Hu.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Geng, R., Li, M., Hu, Z. et al. Digital Twin in smart manufacturing: remote control and virtual machining using VR and AR technologies. Struct Multidisc Optim 65, 321 (2022). https://doi.org/10.1007/s00158-022-03426-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00158-022-03426-3

Keywords

Search

Navigation