A cyber training framework for orthopedic surgery

Abstract Purpose: This paper focuses on the development of a cyber training framework for an orthopedic surgical process termed Less Invasive Stabilization System (LISS) plating surgery. Research methodology: The overall methodology involves the design and use of the Virtual Reality based simulators to train surgical medical students and residents. Expert surgeons played an important role in the design and development of this network based training simulator. Hypothesis: The hypothesis was that the Virtual Reality based simulations can be used to educate and train surgical residents in target surgical processes. Results: An assessment of the impact on the residents’ learning confirmed the hypothesis using such simulators did improve the residents’ understanding of the LISS plating surgical process. Conclusion: This paper demonstrated the impact of using such network based simulation frameworks for medical education and training.

ABOUT THE AUTHORS J. Cecil, PhD, is an Associate Professor with the Computer Science Department and the Co-Director of the Center for Cyber Physical Systems at Oklahoma State University (OSU). His expertise includes the design of IoT based cyber physical frameworks and Virtual Reality (VR) based simulators for distributed collaboration in medicine, manufacturing and space systems design.
Avinash Gupta is a senior doctoral research assistant at the Center for Cyber Physical Systems in the Computer Science department at OSU; his areas of interest include IoT, Cyber Physical Systems, VR based simulators and information modeling.
Miguel Pirela-Cruz, MD, is a Professor of Orthopedic Surgery at the Paul L. Foster School of Medicine affliated to the Texas Tech Health Sciences Center (TTHSC) in El Paso, Texas; he is also a senior practising surgeon at TTHSC.
Parmesh Ramanathan is a Professor in the department of Electrical and Computer Engineering at the University of Wisconsin, Madison.

PUBLIC INTEREST STATEMENT
The use of Virtual Reality based simulation environments for training of medical residents is gaining popularity. In this paper, we have described an Internet based training framework for an orthopedic surgical process that can be used to treat fractures of the femur bone. Use of such training simulators can help improve training practices for medical students. We have discussed the main components of our Internet based system which was built to demonstrate the feasibility of our simulation based training approach. The results from our study indicated that this approach helped residents improve their understanding of the specific orthopaedic surgical process for which the simulator was developed.

Introduction
The use of Virtual Reality (VR) based environments for training surgeons and residents in medical universities has increased in recent years. Traditional methods of surgical training including residents training using cadavers, animals, and synthetic mockups (Youngblood et al., 2008) have some major drawbacks. There is a possible risk of infection while training on cadavers. Medical training on animals has been criticized by animal right groups. Synthetic bones are expensive and are not patient specific (Cosman, Cregan, Martin, & Cartmill, 2002;Kunkler, 2006). Other approaches involve residents observing the surgery performed by an expert surgeon and then slowly progressing to assisting in surgeries. VR based simulation environments can serve as a platform for addressing these issues with the current training approaches. There has been an increase in interest in VR based simulator as an alternative method for medical training (Peters et al., 2008;Qin, Pang, Chui, Wong, & Heng, 2010). One of the benefits of using VR training simulators is its long term low cost. Network based simulators can be accessed from multiple locations. The American Board of Orthopedic Surgery (ABOS) has also mandated the use of simulation based training in order to improve surgical skills (https://www.abos.org/abos-surgical-skills-modules-for-pgy-1-residents.aspx). The developed network based cyber training environment, which is the focus of discussion in this paper, deals with training medical residents in an orthopedic surgical process called Less Invasive Stabilization System (LISS) plating surgery which is a medical procedure to treat fractures of the femur bone.
The emergence of cyber physical frameworks and approaches hold significant potential in designing innovative collaborative approaches which support real time collaborations while supporting cyber physical interactions for a range of applications including manufacturing, energy, safety, agriculture, big data analysis and information system including training simulators for the field of medical surgery (Istepanian, Hu, Philip, & Sungoor, 2011;Xu et al., 2014).
In this paper, the emphasis is on the residents interacting with doctors using network based haptic interfaces. These cyber training frameworks can also be viewed as falling within the realm of Internet of Medical Things (IoMT); in this paper, we discuss the design and development of such a complex network based cyber physical framework. These learning interactions can be viewed as a collaborative training enterprise of the future where cyber and physical components are used as an "On Demand" basis. The role of emerging next generation networks in supporting such cyber training activities assumes significance.
A review of relevant literature is provided related to the context of the design and development task of the cyber training framework.

Collaborative virtual environments
Collaborative virtual environments enable distributed users to interact with each other through the Internet (Oliveira & Georganas, 2003;Qin, Choi, Poon, & Heng, 2009;Sales, Machado, & Moraes, 2011;Youngblood et al., 2008). Qin et al. (2009) proposed a framework for CVEs using hybrid network architecture which was cluster-based. However, the approach was implemented using a private intranet (not a public Internet as in our approach) and has not explored cloud computing or Software Defined Networking (SDN) which has been adopted in our approach. SDN helps reduce the complexity present in current networks. SDN also helps in hosting millions of virtual network without using common separation isolation methods (Caceres & Friday, 2012).

Other cyber technologies
There has been a growing interest in exploring Cyber Physical and Internet of Things (IoT) based approaches in healthcare; some of these technologies and approaches can be used in services such remote medical supervision of chronic patients, which in turn can lead to improved healthcare for patients in rural areas (Cecil, Xavier-Cecil, & Gupta, 2017;Jia, Wang, Guo, Gu, & Xiang, 2017). IoT is an emerging area of importance which can be described as a network based approach that supports interaction and data exchange among sensors (called things) embedded in physical devices linked through the Internet. These "things" are capable of collaborating with other cyber and physical entities using cyber infrastructure (Cecil et al., 2017;Istepanian et al., 2011;Jia et al., 2017;Long & Hoang, 2017;Santamaria, Serianni, Raimondo, De Rango, & Froio, 2016;Seymour et al., 2002;Xu et al., 2014). IoT concepts for medical and healthcare domains can be termed as Internet of Medical Things (IoMT) (http://internetofthingsagenda.techtarget.com/definition/IoMT-Internet-of-Medical-Things, 2017). Qin et al. (2009) proposed a novel architecture for automatic monitoring and tracking of patients, personnel, and biomedical devices within hospitals and nursing institutes. IoT applications in health care range in the scope of applications including designing a robotic device using IoT technology to provide gait rehabilitation for the elderly (Long & Hoang, 2017) to developing an IoT based system to collect, integrate and present patient data to support medical emergencies (Xu et al., 2014).
Based on the literature review of the state-of-the-art, the following observations are relevant: (1) Prior research efforts have not emphasized the role of expert surgeons as knowledge sources for understanding target surgical processes; in the approach discussed in this paper, the simulator was developed after interacting with two expert surgeons who served as knowledge sources in understanding the complex LISS plating process. Through discussions with the surgeons, important attributes and relationships in the design and development phases of the simulator framework such as information inputs, constraints and resources needed for completion of each phase and decision outcomes from each phase were identified. Additional information about the process of designing the simulator can be found here (Cecil et al., 2016).
(2) Prior research in virtual surgical simulators and environments has not explored Next Generation Internet technologies including cloud technologies principles. The simulator framework discussed in this paper explores Cloud Computing, Software Defined Networking (SDN), IoMT as well as emerging next generation networking technologies such as those involving the Global Environment for Network Innovation (GENI) initiative.
In Section 2, a discussion of the methodology for developing the simulation based collaborative framework is described which includes description of the architecture of the training environments, haptic modeling and the training environments. In Section 3, the results of the learning interactions and discussions are provided.

Methodology to develop the IoMT based simulator framework
The simulator framework discussed has been "network" implemented so it can be accessible from remote locations using Next Generation Internet technologies. In this implementation, networking principles that are part of the Global Environment for Network Innovations (GENI) initiative (Berman, 2014; www.geni.net) have been adopted with a view towards achieving low latency and high-gigabit bandwidth using SDN and cloud based technologies. In this approach, distributed users can interact from different locations to be trained collaboratively or in standalone training sessions; the expert surgeon can interact with the residents using these next generation networking technologies. In Figure 1, the general principle underlying this network implementation is shown.
The number of redundant Simulation Application Servers (SASs) in this architecture are "r". The architecture can seamlessly tolerate failure to connect to up to r-1 SASs. To achieve such tolerance, the Simulation Application Clients (SACs) do not directly connect to a SAS. Instead, proxies implemented by SDN (OpenFlow) switches are used by each SAC to connect to the SASs; OpenFlow (which is a SDN standard) allows network controllers to decide the network path packets across the network of switches. The SACs are partitioned into m groups if there are "m" number of Open Flow Protocols (OFPs), and each group connects to the SASs through one of the OFPs.
The network based simulation environments allow a user to remotely practice the various surgery steps using a haptic device with the assistance of a training avatar.
The simulation environments were built using C#, Java scripts and the Unity game engine which runs on Windows platforms. The haptic interface is provided using the Geomagic Touch™ device (shown in Figure 2); it allows users to touch, grasp and interact with various surgical tools during the simulation activities. The haptic interface's primarily functions is to provide an intuitive "feel" for various physical tasks inside the simulator (such as picking up various plates or tools, placing them accurately in a certain location, etc.). During the development of the simulator, the design team categorized the risks into various categories: • Cost constraints: completing the project on time and meeting budget requirements.
• Simulation correctness and completeness: Being able to replicate the details of the training procedures within the simulation environment.
• Availability of technology (network related) during training.
To mitigate these risks, monthly meetings were conducted between surgeons and the design team. Surgeons provided timely feedback regarding the correctness of the simulation environment. The creation of a planning framework identifying the major bottlenecks, constraints and availability of resources/information were modeled and used to address the development constraints. As each training environment (see Section 2.3) was completed, detailed feedback and discussion from the surgical experts was conducted to ensure the simulation details were correct and reflected the real world training aspects; when necessary, modifications to a specific training scenario were undertaken immediately before developing the next training environments.
After the development of the simulator was complete, a major risk that the team had to tackle during run time was availability of internet resources during training at the hospital. To mitigate this risk, the internet based experiments were conducted first at the laboratory environment of the project team; subsequently, several trips were undertaken to ensure the network connectivity using the GENI next generation technology was functioning at the hospital site.

The simulator manager and various training environments
The environments built helped to train the residents in several activities including assembling of the LISS plate, inserting the LISS plate, position training, reducing the fracture, screw insertion and guide removal processes. A training Avatar guided the users and students through this sequence of training activities.

The simulation manager
The interactions between the various environments and the user is coordinated by a software component termed the simulation manager. The simulation manager interacts with the user interface manager (UIM) which is responsible for reading user inputs from haptic device, mouse and keyboard. As shown in Figure 3, user training interactions with each of the training environments are guided by the simulation manager, which also interacts with a software based manager to initiate and complete the training activities within each module. The avatar based feature provides a more user friendly interaction during training.
Communication diagrams are design diagrams used to model basic relationships between various software components in a program (Communication Diagrams, 2017). The various training environments in the simulator are coordinated by software entities named "managers" (as shown in Figure  3); for example, the Avatar Manager coordinates interactions between users and simulator with the help of avatars (which are human-like models or representations which can interact with the users). These environments and manager components were implemented using C# and JavaScript.
A brief discussion of some of the simulator architectural components follows including interface functions, haptic control and avatar interactions (among others).

Interface functions
The various interface functions of the simulator (including keyboard/mouse control, haptic control) were modeled as software classes as shown in Figure 4. In general, class diagrams can be used to provide an overview of a software system by describing the classes and objects inside the system and the relationships among them (Class Diagrams, 2017)

Haptic control
The Haptic control class (Figure 4) uses the custom Unity plugin to relay information between the Unity visualization engine and the (Geomagic Phantom Omni) haptic device. The plugin utilizes the libraries in the OpenHaptic Toolkit and Geomagic's Phantom driver library. The plugin also allows to change the mass, static and dynamic friction, stiffness, and other properties by changing variables in a C# script attached to the objects.
Various lower level scripting based functions were used to support the creation of the simulation based training scenarios; some of these functions are listed in Figure 5.

Avatar based interactions
In the simulator, the Avatar movement is accomplished by two components. The first component is an Animation Controller that is constructed in the Unity Engine that controls the individual movements of the Avatar (as it interacts with the user). The second component activates the transitions in the Animation Controller between the different animations. Figure 6 shows a view of the Avatar used in a training module to practice assembling the LISS plate.

The LISS plating training environments
There are six training environments in the cyber training framework: LISS Plate Assembly Training Environment, LISS Insertion Training Environment, Position Training Environment, Fracture Reduction Training Environment, Screw Insertion Training Environment and Guide Removal Training Environment. For the sake of brevity, only two of the six training environments are described below.

Position training environment
An important step for fracture reduction is the positioning of the plate between the distal and proximal. A red/green color indication scheme (shown in Figure 7(a) and (b)) is used to support the proper placement of the LISS plate. The residents use the haptic interface to practice placing the LISS plate in the proper position and orientation. As the haptic device allows a more natural 3D movement, it enables practising various placement and positioning tasks more intuitive.
It needs to be understood that it is difficult to place the LISS plate between the vastus lateralis muscle and the periosteum muscle. The Position training environment can be viewed as a visual/ haptic training tool for the residents and novice surgeons to understand the complications, difficulties, and the significance of positioning and orientation issues in the LISS plating surgery. The major focus of this module is to help medical students become familiar and skilled with the positioning and orienting of the surgical implants. The color indicator turns green when the plate is in the correct position and orientation as shown in Figure 7(b). When the medical residents position the plate relatively close to the recommended area or when they are moving away from the correct position, the indicator turns yellow. A red light appears in the color indicator when it is completely in the wrong position/orientation (see Figure 7(a)). A detailed flowchart of the underlying approach is shown in Figure 8.

Screw insertion training environment
This training activity helps residents practice the screw insertion and tightening tasks on the femur. Appropriate screw diameter and lengths are selected and inserted into the LISS plate through insertion sleeves and tightened using power tools and torque limited screw drivers. Visual cues are provided guiding the residents to perform various tasks as shown in Figure 9.

HoloLens based training environments
Besides the haptic based simulator, a mixed reality simulator has also been designed and developed using the HoloLens ™ system. HoloLens ™ is a portable mixed reality based device and system which allows users to interact with the virtual world without losing the sense of the real world. In general, useful and information rich experiences can be supported by using HoloLens ™ , mixed reality system where people, places, and objects from physical and virtual worlds merge together providing a more natural interaction, and exploration in the three dimensions (https://www.microsoft.com/microsofthololens/en-us).
Windows 10 SDK, which is a programming toolkit to create the simulation environment, was used for the development of the environments. In Figure 10, a user is seen interacting with the HoloLens ™ using hand based gestures. One of the training views a user sees when wearing the HoloLens headset is shown in Figure 11.
For the HoloLens ™ based simulator, the residents can practice various surgical steps such as assembling the plate and guide before inserting inside patient's leg. As it is a mixed (cyber and physical) reality simulator, the resident can see and interact with the virtual (cyber) components in the simulation scene even while being able to view the physical components on a real surgical table. As seen in Figure 11, various LISS components such as guide, plate, etc. can be seen on a virtual table (which can be termed Virtual Object VO) along with the real table (termed as Real Object RO).

Assessment activities and discussion
Learning assessment was conducted in several phases to study the simulator's impact as an educational and training resource through interactions with surgeons, residents and medical students at a leading medical university in Texas.
Using the feedback from surgeons and residents in each phase of the assessment, continous improvements were implemented to the content and interfaces of the simulator environments.
A pre-test was first conducted to test the knowledge/skills of the participants relating to LISS surgery for all the participants. The participants were allowed several interactions as necessary to complete the training; there was flexibility in the training time; this range from 60 to 90 min; this was because some residents had prior skills in using the haptic device; others had not interacted with such a device. To help in this process, separate practice sessions to become familiar with using the haptic interface was introduced. Subsequently, the participants were evaluated through a post-test. Two types of network related studies were conducted; the first (type 1) involved a medical resident interacting directly with the simulation environments without any interaction with anyone else; the second (type 2) involved an expert surgeon guiding a medical resident. In Figure 12, the pre-test and post-test results for type 1 is shown. On the Y axis, scores are indicated on a scale of zero to 100 for each participant. The result show that 15 out of the 20 participants demonstrated improvements in their understanding of the LISS plating surgical process.
For type 2 interactions, eight participants interacted with the IoMT based simulator framework collaboratively. The participants consisted of 4 medical students and 4 nurses. In these experiments, a lead surgeon took on the role of the master/expert surgeon and was able to use the simulator interactively with the participants through the GENI network. In Figure 13, the results of the pre and  post-test are shown. These eight participants showed improvements in their understanding of the LISS plating surgical process. Future work will involve conducting a transfer validation in a physical setup where physical models of the surgical implants/tools are used in the training.
The latency related performance of the network was satisfactory (latency was stable around 47 milliseconds). These network experiments indicate the feasibility of the overall GENI based networking approach to support distributed interactions during the medical surgical training. The scope of the IoMT based simulator surgical training capabilities is currently being expanded to include other surgical processes such as Condylar plating.

Conclusion
In this paper, the design and development of network based simulators for orthopedic surgery has been discussed. Next Generation Internet technologies including cloud principles and Software Defined Networking (SDN) were explored to support interaction from distributed participants. In addition, a discussion of a stand-alone simulation environment built using the HoloLens was presented. The simulator's potential in training residents was validated through interactions with surgical residents at a medical university in Texas; the majority of participants showed significant improvements in their understanding of the LISS plating surgical process after interacting and learning using the simulator.