The forensic holodeck – Recommendations after 8 years of experience for additional equipment to document VR applications

: The forensic holodeck was first introduced in 2013, using the first upcoming commercially available virtual reality gaming headsets to visualize forensic 3D reconstructions. Following the publication of this development virtual reality was introduced in case work in a variety of different ways. After 8 years of using virtual reality in a professional forensic capacity this professional practice report will show, which equipment is necessary in addition to a virtual reality setup. This mostly includes audio-visual and broad-casting technology for complete documentation of the application of virtual reality, but also some other IT equipment, which should be available for as low as 20฀000 US .Guidelines, hintsandtipsregardingequipmentacquisition, setupandusew DOI: https://doi.org/10.1016/j.forsciint.2021.111092


Introduction
In the documentation of accidents and crime scenes, 3D scanning methods are becoming a standard procedure [1][2][3][4][5][6][7]. The availability of 3D data -whether laser scans of incident scenes, 3D surface scans of objects or postmortem computed tomography scans (PMCTs) of deceased -allows detailed 3D reconstructions and visualizations [8][9][10]. Due to the 3D data, the use of virtual reality (VR) as a display method was suggested, fueled by the technological advances providing affordable head-mounted displays (HMDs) for the consumer market. Since then, VR applications have steadily risen in forensic investigations and legal procedures. VR has been used in a few cases around the world, including court cases with witness testimonies in China, [11], scene walks with 360° panoramas in Sweden [12] and investigations about the Holocaust [13]. Others analysed the use of VR in future forensic applications [14][15][16]. Furthermore, VR is also used for education in forensics, spreading from university to humanitarian applications [17,18]. HMDs are the most common setup for VR in forensics, even though other VR techniques exist. There are several advantages of HMDs toward alternative methods. Compared to the Cave Automatic Virtual Environment (CAVE), current generation HMDs are very cost effective, mainly because the main intended use for VR HMDs is gaming [19]. Unlike CAVE systems, HMDs are mobile and are easy to set up, even with limited space, such as police stations, state attorney offices or courtrooms. Ebert et al., (2014) [20] developed the first prototype for a forensic VR system in 2013 based on the first developer kit of the Oculus Rift [20]. Since then, technology has continued to evolve, increasing the resolution of the display and adding possibilities such as tracked controllers, wireless HMDs and room-scale VR. In the past 7 years, three main applications for VR were established and applied to a variety of cases in the east of Switzerland, with the developments being proposed by state attorneys and judges. To date, we use VR for the following applications:

Experience in Switzerland
-For visualizing and presenting 3D crime scenes to the authorities (visualization). -As a tool for forensic experts to perform 3D reconstructions (reconstruction). -For recording expert or witness statements in the 3D documented scene (crime scene visit). After the initial development in 2013, VR was introduced at the Institute of Forensic Medicine, University of Zurich, as a casework tool in 2017 [21][22][23]. Since 2017, a total of fifteen case applications have been completed, with the latest two in 2021 [24]. Most of the cases evolved around homicide or attempted homicide, with a minority of cases dealing with assault. In addition to the real cases, a mock-up case was created for educational purposes, which was used at a dedicated seminar. In all cases that contained witness statements, we had additional legal requirements that had to be met, such as the obligation to record virtual scene walks (German "Augenschein") for both the court and subsequent appeals. In addition to the necessity to record VR usage, the rights of all parties involved in the hearing of evidence and the duty of protection of those parties were necessary. The most recent aspect is the right of the public or any certified member of the press to participate in a trial. Furthermore, the latest case required a multi-user set up, so that witness, court, defense and prosecution were all able to visit the scene in virtual reality at the same time. Besides the multi-user application with the authorities, we also found that discussing case details with multiple experts can benefit from multi user VR, especially if the experts are not in the same location. Sorted by frequency, the requirements were as follows: -Visualization of HMD content (14 cases) -Video documentation of the HMD content (10 cases) -Video and audio documentation of the HMD user (10 cases) -Video and audio documentation of the VR operator, operator screen and manipulations (9 cases) -Video and audio documentation of the involved parties (9 cases) -Video and audio transfer of the HMD user's view to other locations (5 case) -Live video and audio transfer to the press (court case) (1 case) -Multiuser virtual reality (1 case) Most cases were (expert) witness statements, where the aim was complete documentation of the procedure in the pretrial period. However, the most advanced setup was necessary for court cases, as those involved the press and public, which are allowed to participate in court trials.

Aim and Scope
This publication describes only the documentation equipment and setup that we have developed over the recent years to allow for use as evidence in pre-court trails, court trails and subsequent appeals. It is not focused on VR equipment, VR scene setup or VR software, but supposed to be a starting point for acquiring additional equipment besides a VR setup. This paper does not address the 3D documentation of crime scenes and related objects, as these topics have been described in several publications and books [25][26][27][28][29][30][31][32][33][34].

Global considerations
The documentation VR display and recording system has a series of basic requirements: -Quality of the data recording/transfer. -Compatibility with current and next generation standards. These basic requirements are considered essential for all equipment.
Another important step is to test the setup before usage in a case. In this way, it is possible to ensure that all equipment is available and functional. This helps to prevent errors or even missing parts during the case application. In cases of errors during an application, error search can be a tedious and even embarrassing moment. The most common source of errors is usually cables that are not correctly plugged in. However, there are a variety of errors that can occur during application and setup (Table 1).

VR equipment
The VR equipment consists of HTC Vive Pro Eye Tracking with a wireless setup, as it allows room-scale tracking without the danger of entangling cables. To render the stereoscopic images with a sufficient frame rate, we custom made a desktop computer with an AMD Ryzen 9 3900X processor, Nvidia GeForce RTX 2080Ti graphic card, 32 GB DDR4 RAM, 1 TB SSD, and 4 TB HDD. The software to run the VR HMD is in our case Unity (Unity Technologies, San Francisco, USA) with SteamVR (Valve Corporation, Bellevue, WA, USA).
The data presented in the VR scene was prepared as described in [3,22,25,35] and the scene set up following internal quality management based on best practice guidelines from VR experts [36][37][38][39]. For more detailed information we refer to publications of VR experts [40][41][42]. Table 1 List of errors that can occur, identification of the problem and on-site solution recommendation. This list is actually used in our procedures to prevent errors during high stress situations in court. More advanced solutions can usually not performed on-site and need equipment repair.

Cables, adapters, batteries and data mediums
Three types of cables used in the setup: -Video/audio cables such as HDMI cables or 3.5 mm audio jack cables, -Data cables such as RJ-45 or USB cables and -Power cables.
All cables have to be available in multiple lengths. Power extension cords and power multiplugs are required because there are often a limited number of power sockets. A sufficient number of replacement cables should be available. Duct tape allows cables to be secured and reduces the risk of tripping. Dedicated batteries and standard form factor batteries should be charged, and replacement batteries must be available. Depending on the requirements, data are recorded on internal or external disks or SD cards. After a VR recording, all data have to be backed up immediately to prevent data loss.

Visualization and live view of HMD content
The first and most essential request is the visualization of the HMD content to the audience. This requirement can currently be achieved with most computers, as they all have a built-in graphics card that allows for multiple display outputs. These outputs can usually be mirrored, making it possible to visualize the HMD content on other displays. In this scenario, there are two major causes of errors. The first is a faulty display device, which can be compensated by replacing the display device. These are usually provided by the contractor; however, it is advisable to bring a replacement such as a projector. The second source of error can be a computer failure controlling the VR setup. Solving this latter issue can be done by restarting or replacing the computer.

Recording of HMD content
The recording of the content visualized in the HMD can only be performed indirectly by capturing the content on the screen. The first method is "game capture" on the computer running the VR setup. This can be done using dedicated software such as the Microsoft Gaming Overlay (Microsoft Corp., Redmond, USA). The primary weakness of this solution is that it requires additional resources on the operating computer, negatively influencing the performance of the VR setup. Recordings can be lost due to system failure or lack of resources, corrupting the complete recording.
A second method of recording can be through a game capture system that records the video signal. We use two different versions: the first is a capture system that captures an integrated hard disk, and the second captures a USB connection to a computer. Both systems are connected between the computer and the display so that the displayed content visible to all parties is recorded. The main advantage of capturing software is that both systems are independent of the VR computer, and a system crash on the VR computer does not corrupt the recording. The disadvantage of recording via USB is the requirement of an additional computer independent of the VR computer.
Currently, we always use two methods simultaneously: software screen capture and a hardware capture system to allow for fail-safe recordings.

Video and audio documentation
The video and audio documentation of the HMD user, operator and involved parties are performed by a collection of video and audio recording equipment. The video recordings should consider that the moving VR user might turn and face the camera with their back, hiding the hands from the camera. It should be ensured that the VR user can always be seen from the side or the front to allow for unobstructed documentation. This can be achieved with at least two cameras recorded from two viewpoints. If those viewpoints are appropriately chosen, they can also record the operator and part of the audience. In our setup, we use three video cameras for video recording.
It is sensible to choose a high-resolution camera allowing for redefining the frame in postproduction if necessary. Furthermore, the frame rate should be in the common 30 frames per second or faster to avoid stutter in the recordings. Another requirement is that the camera should be able to run on socket power as well as battery, provides an audio line-in for audio recordings and a video-out to display the content on other devices if necessary. In addition, the quality of the recordings should allow for subsequent cutting and adjustment so that possible setup errors can be compensated in postprocessing. The form factor should be reasonably small to allow for fast and easy transport and setup. We found that the Blackmagic Pocket 6 K Camera (Blackmagic Design, Port Melbourne, Australia) fulfills all the requirements and even surpasses what is necessary, making this camera a good choice for future applications. As these cameras come without lenses, additional lenses with the correct lens mount are required. We decided to use a collection of different lenses to add flexibility to the setup. This includes wideangle zoom lenses as well as some normal-angle zoom lenses to allow for a large variety of locations with small or larger spaces. The cameras are equipped with 2TB Samsung T5 SSDs (Samsung, Seoul, South Korea) to provide sufficiently large and fast storage space for the recordings. In addition, we used three Sony HDR-CX675 camcorders (Sony Corporation, Tokyo, Japan) as backups.
For audio recording, there are different requirements. First, the need to record the voice of the VR user. As they are moving around, the recording should be wireless and therefore battery powered. This also prevents tripping hazards for the user. In our setup, audio recording can be easily synchronized with the camera, or audio recording can be performed directly on the cameras. The VR user is also accompanied by an operator who makes sure that the VR user is not accidentally crashing into physical objects and to help whenever necessary. This operator standing next to the VR user should also be equipped with audio recording equipment, similar to the VR user. For both, we decided to purchase Rode wireless microphones (Rode Microphones, Silverwater, Australia), which are connected to the cameras as they do not provide recording capabilities. A third microphone is available as backup, as well as an additional Zoom H1 (Zoom Corporation, Tokyo, Japan) as local recording devices that can be placed on a table. The camera audio is set up so that one audio channel records the audio received by the camera in mono, while the second audio channel is used to record the wireless microphone in mono. This is another fail-safe by providing an additional audio recording of the room, which in the worst-case scenario can be used to make all involved parties audible, even though a quality penalty will occur. It needs to be considered that this setup requires separating both channels postproduction.
While the recording of the VR user (and nearby operator) is required to be flexible, the parties questioning the VR user are usually sitting at a table. To record them, a stationary recording device with a microphone can be used. In addition, it is good to provide an easy method for synchronizing audio and video. Due to the limited number of audio lanes and connections on the cameras, this is not possible, so we decided to purchase an additional recording device. As the parties are often separated and the operator of the VR computer is often sitting in distance to the parties, three Zoom audio recording devices were purchased (Zoom Corporation, Tokyo, Japan). In our case, we decided on two Zoom H5 and one Zoom H6 recording device with two H5 recordings for the parties and H6 recordings for the VR computer operator. All devices allow for ondevice microphone heads as well as XLR microphones, which we have not used yet, but allow us to prove scalability for larger party involvement in the future.
From our experience, there is one essential aspect that is often neglected: lighting. Because the locations can vary, lighting for video recording can be poor, resulting in underexposed or grainy video recording. To compensate for this, we added three Godox LED 1000 C Light Panels (Godox, Shenzhen, China) to the system.
Lamps and cameras also need tripods. For both, we chose light tripods because they have a small footprint, and both cameras and lights are fixed. Only one camera tripod was added to allow for one flexible camera that can be operated and, for example, follow the VR user.
This equipment together allows for an all-around fail-safe documentation procedure of VR usage.

Video and audio transfer
Other requests were regarding the transmission of the video and audio data to other on-site nearby locations, which we solved using wired equipment. The use of wired equipment ensures data safety and reduces the risk of connection issues.
The transfer of video and audio is made possible via an HDMI splitter, which is used in the signal toward the recording device. The split HDMI signal is limited to a few meters, as HDMI cannot cover longer distances due to signal degradation. For this reason, we use an HDMI over the Ethernet Transmitter/Receiver set with a 50 m Ethernet cable to allow for data transfer between locations. At the receiver, we can use the HDMI signal with a screen including speakers or with an HDMI Audio Splitter that can separate the audio from the HDMI signal. The extracted audio signal can be connected to external speakers. As external speakers, we use a JBL Flip 4 (JBL, Los Angeles, US), which provides a 3.5 mm input interface and the ability to work with or without an external power supply. The power is supplied via micro-USB. As a display device, we can use a desktop computer screen or projectors provided by the customer or by us (Fig. 1).
For press participation or other reasons, there is also the possibility to transmit additional camera signals. For this, we use the Blackmagic ATEM Mini, which has four HDMI input signals and two 3.5 mm microphone connectors. With the ATEM, it is possible to switch between the different input signals or even do picture in picture to provide a single HDMI output signal. In this way, the recordings of the scene can be combined with the VR signal to provide a sufficient overview of an external display device. The HDMI output signal of the ATEM can also be transmitted via Ethernet to other onsite locations. This device is not essential, as the data transfer in addition to the VR signal is usually part of the contractor. Therefore, there is no backup for the ATEM controller.

Multiuser virtual reality
The most recent request was to provide multiple VR setups that can visualize and interact in the same virtual scene. For this, it is necessary to provide a network connection between all setups [43]. In our setup, based on Photon PUN (Exit Games Inc., Portland, OR, US), we provide a local area network (LAN) with a router to which all VR computers are connected using standard Ethernet cables. We do not use a wireless network to assure data security. For a multiuser VR setup, it also has to be considered that multiple computers and VR HMD setups are necessary. Furthermore, it also needs to be considered whether all setups can be set up in the same location to allow all setups to be recorded by the same equipment or if each VR HMD setup requires its own dedicated video and audio recording. Up to now we were always able to set up all VR HMD setups in the same location which allowed us to record them with the same recording equipment. Fig. 2 is a simplified connection sketch of the latest setup using all available devices.

Software
To process the recordings there are several programs and software packages that could be used. In our case we first used Adobe Products (Adobe System Incorporate, San Jose, CA, USA) however, soon after switched over to Blackmagic Design's DaVinci Resolve.
The software provides a variety of possibilities including multicamera video editing and sound editing, which are the most important functionality to handle the recordings. This includes compressing or reducing the 6 K video files, synchronizing the audio and video files, providing time stamps, picture in picture functionality to show external filming and internal VR view, and cutting to the best point of view between the different cameras, allowing the court to easily follow the questioning.
As all our devices record in standard file formats and use standardized connection plugs, which means that no propriety cables or software is required to handle the recordings.

Discussion
Virtual reality in forensics has a variety of applications, from pure 3D visualization of 3D reconstructions to scene walks to forensic medical examination in VR [21,22]. Target groups can be from forensic experts, lawyers, judges as well as people of the public that make up a jury [14,23]. As these techniques are applied in forensic investigations, adequate documentation is necessary to allow for subsequent use either in court or in subsequent court instances and appeals. However, up to date no publication on forensic VR application described the documentation procedure of the application of VR. While Sieberth et al., 2019 [22] already mentions a documentation procedure, it only provides limited description of the equipment. The setup described in this paper allows for a variety of applications and its flexible set-up allows for use in various locations, whether it is an office of the state attorney, a police station or a court room. It even allows for inclusion of the public body, which in many countries has the legal right to attend hearings, see the evidence, and hear the statements. By our experience the equipment can be set up by three persons in around 30 min, including 3 VR sets. During the VR usage these three persons can run the computer for the VR set, stand close by the VR HMD user to ensure their safety, while the third person can perform live video switching for the court at the Blackmagic ATEM Mini.
Furthermore, it can be transported easily and set up quickly in varying locations. In our case, the one-time purchase costs were approximately $20,000. At this cost, it was possible to cover a variety of forensic VR applications and is future prove, as it can be easily upgraded, extended or replaced whenever necessary. We still strive to professionalize the setup even more by providing everything in separate rack cases to protect the equipment and allow for faster and easier setup without extensive on-site wiring. This could reduce the staff to two, prevent tedious errors during setup, and probably half the time of setup.
Nontheless, the proposed equipment and setup might not cover every imaginable requirement for a specific case application, requiring additional equipment or documentation procedures. This could for example be requirements to immediately store data in manipulation safe storage, or prohibiting video equipment in court rooms. This should be considered before purchasing any equipment. Also, the use of on-site equipment, such as projectors and screens, might be possible, reducing the set-up time.

Summary
With our experience of eight years in forensic VR usage and a variety of case applications in the past four years, we gradually developed and extended our VR setup. With the increasing interest in VR applications in forensic investigations, the equipment was professionalized, and clear setup procedures were developed. The current VR setup provides a good overall and fail-safe solution to document VR sessions. Most of the equipment we listed is part of any standard videography and live TV amateur set and therefore affordable and easy to maintain.

Ethics and competing interest statement
There are no ethical concerns due to the technical nature of this paper. Furthermore, no authors has competing interests relevant to this publication.

Competing interest statement
No author has competing interests relevant to this publication.

Acknowledgements
We want to thank Emma-Louise Kessler for her generous donation to the Institute of Forensic Medicine.