Exploring underwater archaeology findings with a diving simulator in virtual reality

1 Introduction

The global onset of the COVID-19 pandemic caused drastic changes in the hitherto customary human way of life. Health regulations and safety concerns brought on by the disease have led to paradigm shifts in many fields. Of particular interest within the purview of this article are the changes caused to institutional education and individual recreational and cultural habits, specifically pertaining to the marked increase in the gaming industry compared to the relative decline of cultural and educational institutes (e.g., museums closing entirely, online education).

Virtual Reality (VR) could fill the gap left in the absence of more traditional institutes. Online presence is facilitating virtual visits to museums otherwise physically unavailable and is providing a fluent, active and immersive communication platform allowing for a potentially better method of remote-education over the audio-visual conference tools currently in use.

This potential becomes even more compelling when VR is combined with techniques known from Serious Games (SG), which would tremendously increase user immersion while partaking in said activities. This could also positively affect individual well being as an interpersonal communication platform with a high degree of presence (Canale et al., 2020). An additional benefit of this enhanced presence lies in its ability to make available to the user experiences that would be all but impossible for them under normal circumstances.

The combination of 3D images, the large and encompassing Field of View (FOV), and the natural and intuitive interaction awarded by VR, has the potential to create immersive virtual experiences that replicate real world environments and activities realistically anywhere in the world (Bowman and McMahan, 2007). VR hardware is becoming more widely adopted and with multiple large companies investing in the technology, the barrier of entry is lowered even further through cheaper headsets.

Museums and institutions of cultural heritage in particular could profit from VR by using its immersive nature to make otherwise only locally accessible exhibits and environments available to more people. Lowering the effort required to access these resources potentially raises interest in these aforementioned cultural properties and sites and, once combined with aspects of Serious Games (SGs), provide playful education to the general public.

One field in particular impacted by the lockdowns was diving tourism and Underwater Cultural Heritage (UCH). Museums and public places dedicated to UCH were closed and many diving destinations could no longer be reached by recreational or professional divers due to international travel restrictions. However, even without travel restrictions, recreational diving is not an activity available to everyone, as it requires training, equipment, and the ability to travel to the diving site. Additionally, some diving spots are located in restricted waters or in dangerous locations, requiring further permissions or training. Due to the majority of the Earth’s surface being covered in water, many beautiful environments and remnants of human history are located underwater, and by extension unavailable to a majority of the population. These sites hold great educational value, but are hard to investigate due to their difficulty of access.

Considering almost all aspects related to UCH in general and diving specifically as an example, the combined technology of a VR SG, especially one available to the general consumer in the comfort of their own home, could allow any individual otherwise barred from such experiences (be it due to physical difficulties, financial concerns, time and geographical constraints or all of the above) to enjoy them in a believable manner with little to no effort. To this effect—and as a general best-practice design principle—a high degree of fidelity and authenticity is required from such a UCH VR SG.

SGs are games targeted at delivering real-world knowledge to the player (Abt, 1987). Examples of SGs are true-to-life simulators (e.g., flight simulators used to train pilots) or any other game, according to the definition of Deterding, which is used in a non-gaming context (Deterding et al., 2011). SGs should contain educational material which uses the game as a transmission vector. However, how this transmission occurs depends entirely on the design strategy the developers choose to implement while developing the game.

In this paper we are combining the advantages of VR and mechanics of SG as immersive technologies to present the data collected by archaeologists of the wrecksite of Veliki Piruzi (Croatia).

2 Related work

In recent years, there have been a wide variety of projects dealing with cultural heritage and Mixed Reality (MR) or SG. The latter have been used to convey knowledge about Japanese culture (Froschauer et al., 2010) and language (Plecher et al., 2018) or even ancient Egyptian hieroglyphs (Plecher et al., 2020a) to the player. MR offers different ways to interact with cultural objects (Plecher et al., 2019). For example, VR is used to view exhibits regardless of location, to visit remote cultural sites, or to travel back in time to places of historical significance (Guidi et al., 2007). Augmented Reality (AR) is mostly used to interact with the exhibits on site (Plecher et al., 2020b), to guide the visitors through the museum (Miyashita et al., 2008) or to help archeologist to visualise archaeological finds or excavation data from any location (Benko et al., 2004). There is also one example in terms of UCH and AR. The ancient Roman city of Baiae in present-day Italy is partially submerged in the sea and can hence only be explored by diving. With the help of AR, the Roman buildings and their furnishings can be virtually reconstructed and inspected by divers (Čejka et al., 2021).

In the following, we will now take a look at projects using VR for either diving or visualizing and interacting CH, or a combination of both:

2.1 Cultural heritage in virtual reality

VR is already used several times in combination with tangible cultural heritage. The possibility to virtually travel to historical or cultural places independent of the user’s location and time is one of the great advantages of VR. One example is the application Nefertari: Journey to Eternity¹ in which the user can virtually explore the scanned tomb of Nefertari, the wife of Ramesses the Great. This way, also access is given to parts of the tomb that are closed to tourists.

The Dawn of Art² is a short VR experience released in 2020 by Atlas V in cooperation with The Public Union for the Management of the Chauvet Cave that allows players to explore a virtual recreation of the Chauvet Cave in Ardèche, France. The game’s content is largely focused on education and was created to deliver information on the World Heritage site in an interesting and engaging way, which classifies it as a SG despite not being explicitly advertised as such (de Lope and Medina-Medina, 2017).

Players have the option to choose between two experiences, an immersive 360° film and a virtual tour of the cave as it is today. The story of a tribe arriving at the cave during the beginning is visualized through a 3D immersive backdrop of an early human encampment in the surrounding valley. It is possible to elaborate on the highlighted cave paintings when unveiled by light through the player. There are no direct locomotion methods available and the film changes location automatically.

The second experience is a short guided tour of a virtual 3D scanned recreation of the real Chauvet Cave in which several cave paintings can be seen in great detail. Players are again led by a voiceover that explains the different motives and points out their location on the walls before being given an option to teleport to the next location. Remarkably the virtual environment depicts the real Chauvet-Pont-d’Arc Cave and not its visitor accessible real world replica Grotte Chauvet 2.

2.2 Diving and swimming in virtual reality

Early research focused on reaching the boarders of technology by combining arts and sports in the form of simulating swimming in mid-flight via an harness and weights (Fels et al., 2005). Friction was used to mimic the water resistance combined with an authentic water model. The swimmer is placed on the water’s surface, hence providing the simulation of the transition between air and water. The system dynamically reacts towards the actions of the swimmer and wave interactions are simulated as well. The simulation by Fels et al. (2005) does not feature the option to dive, as the person has a fixed position on the water’s surface.

In SCUBA VR (Hatsushika et al., 2019) a realistic diving experience is created by submerging a person in a swimming pool with a waterproof VR HMD. An underwater motion capture system helps to transfer the movement of the person into the virtual world. The system is wearable and the user has freedom to move around. To achieve a realistic experience the head pose of the player is being tracked.

Jain et al. (2016a) propose an immersive VR SCUBA diving simulator Amphibian with the goal to create a sense of presence without the person being submerged into water. The user lies on a motion platform positioned on the torso. The arms are stretched out, while the legs are inside of a suspended harness. Additionally, visual and aural feedback is created through the HMD. Buoyancy, drag, and even temperature changes are simulated through the help of sensors and the emphasize on the tracking of the hand movement is being placed.

DIVR³ is a VR snorkeling system developed by Ballast Technologies for the use in resorts and waterparks. Their waterproof HMD technology enables visitors to immerse themselves in a virtual environment while physically located within real water. Five different experiences are available, ranging from different underwater locations to space and skydiving, though all related in some way to the feeling of weightlessness awarded by swimming in water. As the HMD blocks a user’s entire FOV, they are anchored to the ground with a tether and kept afloat with a buoyant, while breathing through a snorkel integrated into the setup. For further sensory stimulation the VR system can be extended with a haptic motor system, emitting variable thrusts of water towards the users, in the in DIVR + configuration. An alternative product by the same company implements a waterproof HMD based VR System for the use in waterslides, allowing riders to experience similar virtual environments while descending.

The DIVR system highlights the creative application of VR technology for the immersive simulation of diving by situating the entire experience in physical water. Despite not classifying as a SG due to its explicit focus on entertainment, the technology holds ample value for the use in education or training. The alignment of the virtual and physical environments when undergoing a virtual dive while floating in real water could increase both immersion and presence immensely and allows unique interaction that would be impossible in other VR systems. Additionally DIVR seems to show positive results in the reduction of Visually Induced Motion Sickness (VIMS) (Fauville et al., 2021), which further reinforces its potential for the use in immersive VR diving simulation.

One major downside of the system are the visual fidelity and tracking capabilities in comparison to other VR HMDs. Due to its waterproof nature, the DIVR HMD has to be self contained and runs entirely on mobile hardware, which has less computational power than desktop tethered systems. Furthermore there is no support for hand tracking or input devices of any kind, which limits the possible experiences to non-interactive tours and makes the technology unfit for use in realistic training or practical education.

2.3 Virtual reality underwater archaeology

The Virtual and augmented exploitation of Submerged Archaeological Sites Project (VISAS Project)⁴ is one of the most important related works to this paper. The project includes multiple papers directly relating to VR in use with underwater cultural heritage and diving. The goals of the VISAS project are to improve the responsible and sustainable exploitation of underwater archaeological sites.

The project aims on the one hand to improve the diving experience in VR and on the other hand to enable a virtual tour for exploring archaeological sites underwater. The Project shares its findings via multiple papers and articles (Bruno et al., 2016, 2017, 2018).

To achieve the authentic recreation of diving and underwater cultural heritage sites as 3D models, a multi-step process was enacted to acquire the needed data for accurate reconstruction in 3D. As a first step an initial inspection of the possible diving site is carried out and points of special interest are identified for more detailed further surveys. The entire area that is to be scanned, is then swept with Multibeam echosounders, to generate bathymetric maps in archaeological contexts. This survey is conducted by placing the high frequency multibeam equipment on a boat, from where data can be gathered up to a depth of one hundred meters, resulting in a low resolution bathymetric map representing the seabed of the site to increase the level of detail for previously selected areas of interest, a secondary survey is conducted on only these spots. This secondary optical survey is carried out by divers with underwater cameras taking a large number of overlapping photographs of each individual spot. The images are enhanced using multiple methods to increase their chromatic quality for further processing. A 3D model is then created for each spot from the enhanced images using photogrammetry technology finally both optical and acoustic models need to be merged to form one complete 3D reconstruction of the entire diving site. The location, scaling, and orientation of both objects in relation of each other are performed using multiple techniques, most notably geolocation systems and direct measurements done by divers to increase accuracy. Textures are applied to the high-resolution areas of the mesh by blending and merging the previously taken images of it on the 3D mesh. The remaining low-resolution mesh is textured using standard tile-based texture mapping. The result is a highly accurate and realistic 3D model that can be used in further work. The collected data and reconstructed models are then saved to a database and made available to other services in the project through a web service software. (Bruno et al., 2017).

The Virtual Dive Experience created by the VISAS Project is a VR System designed to transfer instructions and information to the user in a playful way. It simulates a dive in one of two regions in South Italy, Capo Colonna and Cala Minnola, to allow non-divers to experience the underwater archaeological findings present there in a safe and playful, but realistic way. This is intended to promote diving tourism and increase the appeal of the area to tourists (Bruno et al., 2018).

In the project by Liarokapis et al. (2017) the goal is to combine an immersive VR experience of the Mazotos shipwreck site in Cyprus with realistic models as well as mapping of the area to teach visitors archaeological knowledge and strengthen their cultural awareness. Modelling and mapping was done by utilizing calibrated underwater cameras to record optical data. The simulation of swimming and diving was not a main goal, but underwater exploration at an archaeological site. A 3D mesh of the shipwreck is created through a point cloud and the combination of highly accurate photogrammetry and computer vision algorithms. Effort has been placed on a realistic environment with light rays being calculated when passing in the water, plants are placed through an algorithm and groups of fish are simulated. Interactions are being held simple, users can interact with wood and rocks may pick some of them up to get information about artefacts.

Another project by Liarokapis et al. (2020) targets search techniques to discover artefacts in a playful environment (educational games). In their Serious Game it is possible to experience maritime archaeological search including a variety of artefacts. Focus is placed on looking for artefacts with search techniques for maritime archaeology such as circular and compass search. Players can pick up and interact with objects as well as learn more details about their archaeological significance. Emphasize is placed on a realistic environment combined with game features such as scoring points.

In the VENUS project by Chapman et al. (2006) scientific methodologies and technological tools for virtual exploration of deep underwater archaeology sites have been developed. Thereby the unique strengths of VR and AR technology are used for visualisation of immersive and interactive digital models of underwater sites. Merging optic and acoustic sensor data of underwater sites is done to create a realistic experience. In the end an archaeological database engine which automatizes data input to generate Mixed Reality experiences is proposed.

Haydar et al. (2011) are targeting 3D visualization and 3D interaction modalities in VR/AR to study and preserve cultural heritage. Cultural Computing (CC) is used for cultural translation in creative ways. VR allows for immersion within multimodal interactions (audio, video and haptics), if correctly used resulting in presence. Their project targets scientific methodologies and technological tools to improve virtual exploration of deep underwater archaeological sites, e.g., photogrammetric tool which combines point clouds of artefacts from several angels to reconstruct their shapes. With that foundation a realistic virtual environment (VR and AR use cases) with configurable levels of immersion and interaction allows archaeologists to study underwater wrecks.

Summary: In recent years, a lot of research has been done in the field of diving, the presentation of cultural heritage or the combination of both in the form of underwater archaeology in VR. Our project follows the basic idea of presenting and experiencing underwater archaeology and its finds in VR. It combines elements that have already been addressed in the listed projects, such as diving simulation, presentation of archaeological finds and mechanics from serious games for knowledge transfer, but it differs in essential points:

• Clear structuring of the implementation makes it very easy to transfer the concept to other excavation sites.

• Through different modes, the user can influence his VR experience himself by placing more or less emphasis on a more realistic diving simulation (e.g., air consumption).

• Diving simulation incl. motion detection can already be carried out with commercially available hardware and enables the interested layman to experience diving and also exploring the excavation site from home.

3 Theoretical background of a virtual reality diving simulator

As most UCH work takes place underwater, a key aspect of a SG simulating the subject would be a faithful and meaningful recreation of the experience of diving. In order to produce such an experience the theoretical background involved in diving should be taken into consideration. This information is traditionally transmitted via an involved course that combines frontal teaching with practical implementation in the form of lessons and dives implementing the lessons respectively. The disparity between a simulation and the real world poses however several constraints in various aspects of the experience in order to maintain fidelity and thus create a meaningful experience with real-world applicable content.

3.1 Visuals

One of the most important aspects in creating any realistic simulation of an activity in VR is the visual component. As humans rely heavily on their eyesight to experience the real world, a faithful and accurate virtual image is the foundation of any attempt to simulate a virtual reality. Surprisingly some studies have shown, that visual fidelity and realism do not have a strong influence on the feeling of presence and immersion, once a bare minimum visual representation is available (Cummings and Bailenson, 2016). However other studies (Laha et al., 2012; Ragan et al., 2015) conclude that visual realism and display fidelity could have an impact on certain visual tasks.

Visual quality and realism might be of increased importance for immersion if the simulation places an emphasis on the location the dive takes place in. In particular when a real world dive spot is intended to be replicated in detail, be it for entertainment or archaeological use, visual realism and parity to the original is paramount. For a use-case that focuses more on the mechanics than the environment of the dive, visual fidelity might be of lesser importance.

Realistic visuals mostly depend on the quality of the assets used in the creation of the simulation. While high quality models and textures could be generated by artists, as is usual in the games industry, technologies like photogrammetry can be used to create highly realistic assets from pictures of real world objects and locations. For diving simulation specifically, bathymetric survey data could be employed in combination with photogrammetry and traditional asset creation for high visual realism and fidelity underwater environment (Bruno et al., 2018). Additionally, shaders and post-processing effects can be employed to further increase the visual fidelity. Realistic water refraction and other visual effects present in underwater environments can be simulated using physics based shaders and rendering systems.

3.2 Interaction

The importance of interaction to immersive dive simulation in VR is difficult to quantify. On one hand, interaction with the virtual environment is shown to increase immersion in VR, which can lead to a better simulation experience (Hudson et al., 2019). The degree of realism of the interaction on the other hand does not seem to have an influence on the level of immersion (Bowman and McMahan, 2007). Furthermore, interaction based purely on realistic simulation might even cause confusion in players and lower their sense of embodiment in certain situations.

To create realistic interactions with the world, the original interactions have to be studied in detail to be recreated authentically. As interactions can quickly grow in number, especially when combined, it is likely more efficient to have interaction behavior emerge from a general system of rules that governs everything instead of implementing each interaction by itself. This approach is time proven and can be found e.g. in the use of physics engines. For dive simulation specifically, it is important to recreate the function of the dive equipment realistically. A lot of the information needed for that can be found in the PADI Open water diver manual (PADI, 2015).

3.3 Locomotion

One of the most fundamental types of interactions in VR is the movement through the virtual environment, often referred to as locomotion. Diving is largely comprised of movement that is impossible to recreate outside of water, which poses a unique challenge when trying to simulate the activity.

While the more traditional purely thumb stick or button based inputs could be used, motion tracking technology available in many modern VR systems (Angelov et al., 2020) enables virtual locomotion that requires physical movement in the real world. This has been linked to high levels of immersion and flow (Boletsis and Cedergren, 2019).

Swimming could be simulated by tracking players’ hand/arm movements in relation to their body to register their arm strokes and convert them into virtual locomotion. Breaststrokes can easily be performed in a sitting or standing position and are ideal for diving as they work both underwater and at the surface. However, problems arise when trying to simulate leg kicks - the primary method of locomotion underwater - which cannot be performed properly when standing or sitting and require additional motion tracking.

This could be solved by physically situating the entire VR system inside real water, using technology similar to DIVR⁵, allowing players to swim in real life in order to swim in the virtual environment. There are no conclusive studies on the immersion benefit of this method yet, but it does seem to have a positive effect on vection, the illusion of self motion, and can reduce Visually Induced Motion Sickness (VIMS) (Fauville et al., 2021). However, the downside is that the system requires a large body of water and a hardware system that can be used in and under water, which is impractical for most use-cases.

Another option to free players’ legs for proper movement is by suspending them in the air using a harness-support-system. This awards full range of motion in all extremities and players can perform breaststrokes as well as leg kicks, that can be translated to virtual movement using motion tracking. However, a user-study conducted on such a system has shown the satisfaction to be quite low, with the supports negatively impacting immersion and realism (Jain et al., 2016b).

Finally, the influence of breathing on the buoyancy of divers has to be considered when attempting to simulate diving authentically in VR. Divers can increase their body volume and buoyancy by breathing in or decrease it by breathing out. Though seemingly small, this form of vertical control seems to have a large impact in perceived realism of the simulation (Jain et al., 2016b). Breathing can be reliably measured using a gas flow sensor, which could be implemented using a modified second stage mouthpiece (Jain et al., 2016b). Microphones could also be used to estimate the volume of air currently in the player’s lungs.

We conclude that simulating the locomotion of diving is difficult to recreate authentically in VR using hardware available for the user at home. While arm based swimming might be feasibly implemented using motion tracking, the inclusion of real world leg movement into the simulator seems unfavorable and might have to be emulated using traditional thumbstick based locomotion. Movement based locomotion in general can tire the players and reduce immersion in certain cases (Boletsis and Cedergren, 2019), whereas the inclusion of traditional controls as a selectable alternative should be considered. Additionally, buoyancy control through breathing could be used to boost immersion and realism.

3.4 Haptics

Haptics refers to the sensory information delivered by touch or the techniques used to emulate such sensory information. While there is little conclusive evidence on the impact of haptics on immersion or realism, the technology is still developing and could improve the fidelity of VR in the future (Cummings and Bailenson, 2016; Kreimeier et al., 2019).

The feelings of wetness and increased resistance when submerged in water are some of the first haptic sensations that come to mind in relation to diving, and could provide an increase in realism when properly applied to a VR simulator. The previously mentioned DIVR system uses real water to create the most realistic sense of submergence possible, which however comes with the aforementioned downsides. Another way to simulate the feeling of wetness is through thermal and vibrotactile means. Cold temperature, especially in combination with matching visuals, has been shown to improve player experience when used to simulate water. Vibrotactile actuators also show promising results when used in conjunction with temperature and can be used to simulate splashes (Peiris et al., 2018). Thermal exchange pads could be placed on regions of the body sensitive to temperature change, like wrist and face, for increased effect (Jain et al., 2016b).

Water resistance might be replicable using gyroscopes attached to the player’s limbs, utilising the gyroscopic effect to create variable resistance (Gugenheimer et al., 2016). Alternatively, resistance bands, harnesses, or exoskeleton actuator systems could in theory achieve a similar resistance effect.

Objects and surfaces in the real world have a distinct texture and feel when held, that is not present in most VR systems today. Some simulators might incorporate replicas of real world objects in their design to increase the immersion or realism, but this approach is difficult when the virtual environment changes significantly or features a large number of directly interactible objects. An immersive VR diving simulator could incorporate realistic replicas of diving equipment like the mouthpiece, or Buoyancy Control Device (BCD) vest. This might increase immersion and realism by letting players interact with authentic objects, but also poses significant technical challenges, as interactions with the virtual environment have to be taken into account. In particular the perceived weight change of the cylinder when entering or exiting the water would require a complicated system to be dynamically adjustable during the simulation. Haptic gloves might offer a dynamic solution in the future, but a short survey of currently available models showed that they are likely too limited in their ability to create tactile feedback. Some models also require extensive external hardware, that is likely to reduce immersion more than it could improve it in a VR diving simulator.

3.5 Kinesthesia

Kinesthesia is the sense of the position and orientation of the body derived from receptors in muscles and joints. In VR it is relevant to the feeling of embodiment in the virtual environment, especially if no haptic feedback is present.

During a dive, a diver’s body is usually in a horizontal position, except when ascending or descending, while most VR systems require a user to sit or stand. This mismatch might have a negative effect on the level of immersion and could feel unrealistic. Two systems discussed earlier in this section solve this problem by allowing the player to experience the simulation in a authentic horizontal position by having them float in water or suspending them with a support system (Jain et al., 2016b). This could increase the feeling of embodiment by lining up the virtual body with the real player. However, both systems showed significant limitations with one even suffering from a decrease in immersion due to a unsatisfactory implementation (Jain et al., 2016b).

Despite this, matching the virtual body correctly with the player’s might not be required for an immersive and authentic experience. Playtests have shown that players tend to forget the position of limbs in the real world when they are not in their VR FOV and can misjudge the extent of their body in the virtual world. Furthermore it is possible for players to accept virtual bodies as their own, despite a significant visuomotor mismatch (Caola et al., 2018).

Inverse Kinematics (IK) could be used to create a realistic simulation of a player’s arms from small amounts of motion tracking data, potentially resulting in increased feelings of embodiment in situations where the virtual and real body match. Adversly, the contrast between realistic virtual arms and incorrectly displayed legs could cause a break in immersion (Parger et al., 2018). Kinesthesia is ultimately an important sense that can greatly influence a player’s sense of embodiment in VR. However, under certain conditions, likely when levels of immersion are generally high due to other factors, it might be more beneficial to help players forget the discrepancy between their real and virtual body than it is to align their real limbs with the simulation.

3.6 Acoustics

The sense of hearing is perhaps the second most important when assessing our surroundings and helps us perceive things outside our FOV. In immersive VR environments high fidelity audio can increase perceived realism and lead to a more immersive experience (Bowman and McMahan, 2007; Lindquist et al., 2020).

When examining the role of audio in the simulation of diving, the drastically different behaviour of sound in water compared to air has to be taken into account. Due to water having a similar density to human eardrums, humans can not perceive the direction of sound underwater. Audio perception is further disrupted by increased sound propagation and unfamiliar acoustic effects.

Despite this, realistic audio was shown to have positive effects in VR diving simulators and can have a significant impact on immersion (Jain et al., 2016b). Furthermore, specialised underwater sound propagation models could be employed to modify audio dynamically and increase audio fidelity and simulation realism (Ding and Liu, 2021).

3.7 Summary

The complexity of diving demonstrated above poses many difficulties while attempting to create a realistic simulation of the sport. Diving provides a multitude of different stimuli of which many are hard to replicate reliably and without immense effort. Many of the technologies intended to simulate complicated sensory inputs, while increasing simulation realism, tend to cause a decrease in levels of immersion due to factors related to the technical implementation. Some of the most effective methods also require extensive and specialised hardware setups and cannot be easily combined with others or used outside of specific locations. Altogether, this results in a majority of the technologies being unfit for use in a VR simulation system, that aims to provide a realistic and immersive diving experience.

As a result, it was decided against the use of a large amount of technologies to simulate as many sensations present in diving as possible, instead placing the focus on creating a highly immersive experience using traditional VR technologies (e.g. HMDs, motion tracked controllers, etc.), increasing realism through authentic visuals, sounds, and interactions within those limitations.

If built in a modular and extensible way, select additional technologies can be added for a more realistic sensory experience but not at the cost of lowered immersion. Remaining discrepancies with the real world are then likely to be far less noticeable. The result is a highly immersive VR system that can realistically simulate diving to a sufficient degree without dependence on certain locations or circumstances.

4 Developed implementation of the 200Bar prototype

To test the validity of certain design choices and as a first step in the execution of the full design into a functional immersive VR diving simulation system, development on a prototype of the previously proposed experience was started. The research and findings showcased throughout this paper were taken into account and the prototype was built according to many best practices to maximise immersion and flow while providing a sufficiently realistic diving experience.

The Unity⁶ game engine was chosen as the development environment for the prototype, as it offers support for a large variety of platforms and is known to be user friendly, in particular for smaller teams with limited team size and strength. VR development is supported out of the box and multiple frameworks can be chosen from, eliminating the need for low-level modification to enable certain features.

Water and underwater environments make up a majority of the environments encountered in the game, requiring support for performant and realistic looking bodies of water, with great variety in looks and behaviour. After initial experiments with custom shaders failed to produce sufficient results, the Crest Ocean System⁷ has been elected as the primary water rendering system for the prototype. Its performance centered design is ideal for the use in VR and its in-depth customisation options fit our design principles of variability. Untested in VR applications, the system initially suffered from visual bugs when used in our prototype, but they were fixed over time in cooperation with the Crest developers.

4.1 Player controller

The player avatar is the centerpiece of the simulation, allowing players to view and interact with the virtual world through their eyes and body. The player controller is the entity that controls the player avatar in most aspects and allows the players to immerse themselves in the virtual world.

As building natural interactive systems in VR requires increased play testing and development time compared to non-VR systems, the VR Interaction Framework (VRIF)⁸ was chosen as a foundation to build our own systems upon. The VRIF provides a number of base features present in various modern VR games and allowed us to focus on the features specific to dive simulation. This decision was made in response to our observations in other VR diving SGs, where many simple interactions were less than satisfying resulting in an overall drop of immersion and realism.

The Advanced VR Rig added by VRIF serves as the basis for our own player controller module, the Advanced Diving Rig, providing hand and head tracking, basic item interaction capabilities and simple ground movement systems. With player movement deeply reliant on physics, as can be seen in Figure 1, the Advanced Diving Rig had to be heavily modified to function on rigid body basis.

FIGURE 1. Simplified overview of the different subsystems withing the player controller.

The player controller dynamically switches between locomotion methods based on the circumstances the Advanced Diving Rig is in. When on ground it defaults to either controller based continuous locomotion or controller based instant teleportation, depending on the settings provided by the Player Manager. In water swimming/diving locomotion is activated and once off the ground and sufficiently submerged, ground locomotion is disabled. Buoyancy is applied to the player in two stages, basic buoyancy once the water is entered, and additional BCD buoyancy once the rig is sufficiently submerged (75% by default).

4.2 Physics and buoyancy

The custom physics behaviour created for the player is one of the main features on this prototype and has been a major focus in development. The Advanced Diving Rig can in theory be influenced by the physics engine like any other Rigid body, but the interaction is intentionally limited by allowing only certain interactions to reduce the likelihood of a physics error affecting the player. Early tests using regular imported physics behaviour often produced insufficient or hard to control results, that lead to uncomfortable player movement. It was therefore decided to create custom physics scripts for some of the most important interactions of the player with the world.

There are two central forces that influence the player avatar on a simulated dive, gravity and buoyancy. Gravity is a constant downward force applied to most objects by default. To have full control over the player physics behaviour, a modified calculation is used instead. Buoyancy on the other hand is not present in the engine by default and was implemented on the basis of real-world principles. The big difference between buoyancy and gravity for the player in comparison to other simulated objects in the scene, is that the players physics behaviour is overall dynamic and influenced by many different components, whereas objects use a simplified model.

As shown in Figure 2, the rig’s buoyancy and gravity are influenced by three main components, the diver itself, the BCD, and the additional essential diving gear (e.g., lead, air cylinder, etc.). Diver and gear are similar in that they have two static variables assigned to them at the beginning of the dive, resulting in static forces that can be calculated once in the beginning and then reused. The BCD is a special case, because its volume can be adjusted during the dive to increase or decrease the buoyant force of the whole rig. Its gravity can not be influenced as the weight remains unchanged during the dive.

FIGURE 2. The different forces influencing the overall buoyant force applied to the player avatar.

When the dive is started, variables relayed from outside systems (i.e water density) and player attributes are used to calculate all static forces once, summing them up into two variables to decrease the necessary operations each time forces are applied. Additionally a static bcd buoyant force is calculated using the maximum possible inflation of the BCD but not yet added to the other forces, so it can be dynamically modified by the inflation during runtime.

The gravitational force $F_{g_{base}}$ is calculated in the usual way using

where m in kg is the mass of a component, a in m/s² the acceleration applied to the component, which in this case is the gravitational acceleration g set in the physics settings ( − 9.81 m/s² by default).

Static buoyancy $F_{b_{\mathit{static}}}$ is calculated with the following formula at startup

where V is the volume in m³, ρ the fluid density in kg/m³ (1000 kg/m³ for water), and g in m/s² (—9.81 m/s² by default). The dynamic buoyancy of the BCD is calculated by multiplying the maximum possible buoyant force

with the percentage of inflation every physics step to calculate an accurate buoyant force at any given time.

Finally, any additional buoyant or gravitational forces added by items the player is currently holding are also added, disabling any force calculations the items would otherwise make to float on their own. The composite buoyant force is multiplied with the percentage of submersion to ensure only parts actually submerged create lifting forces. This results in a final set of 2 dynamic forces, the final buoyant force $F_{b_{\mathit{final}}}$ and the final gravitational force $F_{g_{\mathit{final}}}$, which are updated every physics frame for accurate simulation:

4.3 Water movement

In contrast to the proposed design, the swimming and diving locomotion systems have been implemented together in the prototype to allow for faster prototyping and development of the similar systems. The implemented locomotion is closest to the immersive hybrid diving method, using a mixture of motion based natural swimming and controller based pointing.

Motion based natural swimming tracks the players hands using the handheld controllers and calculates a propulsion vector based on the difference of hand position from one timestep to the next, which can be further amplified using a speed modifier. Players can swim or dive in the game by performing a horizontal breaststroke while standing or sitting in real life, as visualised in Figure 3.

FIGURE 3. Conversion of real life breaststrokes to virtual propulsion vectors for immersive swimming and diving.

Initially, we experimented with tracking and converting all significantly large hand movements into locomotion, excluding only a small region in front of the player’s body to allow for the return of hands into the initial breaststroke position without backwards movement. However, this resulted in erratic, unexpected movement that distracted more than it increased realism. The inclusion of hand orientation was also attempted, only applying movement to the player, when the palm of the hand points in the same direction the controller is moving, similar to real swimming where water is scooped with the hands. The results were unsatisfactory, and further research into commercial releases using arm movement prompted us to switch to a trigger based approach. Hand movement is only tracked and converted into virtual propulsion when the trigger buttons on both controllers are pressed down. This allows players to precisely control their swimming direction and speed without any unintentional inputs causing confusion or motion sickness.

In addition to the motion tracked approach, it is possible at any time to move the left or right hand to a position in front of the lower chest area to activate the controller based pointing locomotion method. In this mode, the Advanced Diving Rig receives propulsion in the direction the controller is pointing when its trigger is pressed down. The speed can be modified by depressing the trigger further, if an analogue trigger is present (which it is in the Index “Knuckles”), or set in a variable in the options.

Both locomotion types work well together and lead to immersive experiences during testing and the reliance of both methods on the trigger could lead to a more clear separation from other inputs, possibly reducing confusion in players.

Finally, players can pick up the LPI from an inventory slot on their virtual waist, to control the amount of air present in the BCD. With LPI in hand the A and B buttons on the top face of the controller can be used to inflate or deflate the BCD volume, which, due to the changes it causes in the buoyancy calculations, causes the player to slowly rise or sink. In our game this is the primary way of controlling the player avatar’s depth as breathing and lung volume control are not present. While technically the two previous locomotion types are capable of vertical movement as well, a variable dampening in the up and down direction can be applied to them, in turn encouraging the use of the more realistic BCD locomotion.

4.4 Frameworks and tools

The following frameworks and tools were used in the development of the prototype (see Table 1):

TABLE 1. List of the main frameworks and tools used in the creation of the prototype.

5 Archaeological content: The wreck of Veliki Piruzi

The wreck site of Veliki Piruzi consists of a late roman merchant ship (see Figure 4), which was damaged off the coast of Istria in the first half of the 5th century AD while coming from the modern tunisian coast. It was researched and excavated by a bavarian croatian cooperation team from 2014 until 2017.

FIGURE 4. Reconstruction of the Dramont E wreck, a comparable late roman merchant ship of the same period as the Veliki Piruzi wreck, also carrying a mixed cargo from North Africa (Fiederling, 2019)

The excavation site can be divided into two main areas. On the one hand the submerged pits in the southeast of the island of Veliki Piruzi where the ship hit the rocks and was damaged and on the other hand the bay up north where the material and parts of the cargo (see Figure 5) were washed shortly after the accident. The spot itself is representative for many others, which are in shall waters and were heavily robbed and lost also finds over the decades by storms and waves. Therefore it is a exemplary approach to try to make also these heavily disturbed underwater archaeological sites again visible also to an interested public by using VR technology. To transfer all these informations and results in a diving simulation can lay ground for future all round simulations, which might shape the understanding of real underwater field work for non insiders.

FIGURE 5. Cargo—in situ (left) and reconstructions (right) (Fiederling, 2019)

At that point it is also important to mention that the slight milky visibility is a reacreation attempt and chosen an additional challenge for simulation users as they will get to experience not another clear water sport dive but a surrounding which is closer to the real underwater archaeological fieldwork atmosphere.

The archaeological dig focused on the cargo, as the ship itself has not been preserved except for a few parts like broken ship nails. The majority of finds are late roman amphorae, most of which were produced in Africa and were used to transport olive oil and wine. Another part of the amphorae comes from the Aegean as well as from the Levant (Fiederling, 2019). To implement informations about the fragments from the cargo, where they were produced and for what purpose in combination with a dive simulation in this manner is until now without comparison as many skip one of both parts.

For now we are aiming to combine the during the field work created SfM models (see Figure 6) of the wrecksite with all scientific information about the wreckage and produce an VR based underwater archaeological dive and excavation simulator, in which the digital visitor will be able to experience diving on an underwater archaeological findspot. As an addition in the future the user will than also try to learn how to use the equipment of such an project like underwater dredges and lift bags as well as handling measuring and photographic equipment, which will lift the experience for the user even to a next level and might also provide a basic training simulation for future underwater archaelogists.The wreck site of Veliki Piruzi was a late roman merchant ship (see Figure 4), damaged off the coast of Istria in the first half of the 5th century AD. The excavation site can be divided into two main areas. On the one hand the pits in the southeast of the area where the ship hit the rocks and was damaged and on the other hand the bay where the material and parts of the cargo (see Figure 5) was washed shortly after the accident. The archaeological site focuses on the cargo, as the ship itself has not been preserved except for a few parts. The majority of finds are late roman amphorae, most of which were produced in the North of modern Tunisia, and were used to transport olive oil and wine. Another part of the amphorae comes from the Aegean as well as from the Levant. (Fiederling, 2019)

FIGURE 6. SfM model of the wrecksite.

We are now aiming to combine the created SfM models (see Figure 6) of the wreck site with all scientific information about the wreckage and create an VR based underwater archaeological dive and excavation simulator, in which the digital visitor will be able to experience diving on an underwater archaeological find spot and also try to learn how to use the equipment of such an project like underwater dredges and lift bags as well as handling measuring and photographic equipment.

6 Knowledge transfer via “game mechanics” and interaction methods

Several aspects of UCH related work were taken into consideration when planning and designing the prototype. Multiple gameplay elements coalesced as primary game mechanics present in the prototype. These game mechanics are: a simple gamified “research” mechanic, a mechanical emulation of diving techniques and an item collection mechanic that enables the player to collect coins or fragments of amphorae and refuse present in the prototype.

6.1 Diving and air consumption

An integral part of all UCH work and thus entirely indispensable as a Game Mechanic (GM) is the experience of diving. Equally relevant while considering the experience of diving, and closer to a pure GM rather than a control scheme, is the consumption of air while underwater. The depletion of air in a diver’s tanks is a very strong impetus for the diver to terminate his current actions and surface as fast as possible. This impulse was translated into the game via air consumption calculations and termination of the current dive upon air depletion (see Figure 7). Several degrees of complexity in this GM were implemented, specifically in the complexity of the calculations being performed to determine the rate at which air is consumed. The highest complexity level takes into account the diver’s depth and the pressure that is exerted on them as a result. Which means that air consumption increases the deeper the dive is made. “Preliminary” data is also meant to represent the diver’s calculated air consumption of a previous dive. An important reference point is also the diver’s speed, which results from the swimming movements of the player. Fast movements increase the speed, but also the air consumption.

FIGURE 7. Displaying air consumption.

The air consumption is then dynamically calculated based on all of the data mentioned above. This represents an authentic recreation of real-world air consumption. The least complex implementation on the other hand performs absolutely no air calculations and is intended to provide the user with a stress-free, tour-like visit to an underwater environment akin to a museum visit. The middle-ground is a simplified air calculation based on fixed values.

6.2 Research mode

Another critical facet of UCH work, the research and examination of UCH assets and archaeological finds stands at the core of the desired experience for the prototype and therefore is another example of a GM. As was mentioned above the magnitude to which the work done by UCH researchers can be explored is astounding. In order to provide a sensible simplification for purposes of producing a Minimum Viable Product (MVP) the process was stripped down to the scientific method behind it. However, if the player wants, it is also possible to use technical archaeological equipment in the simulation.

This research mode begins with the player swimming close to physical evidence of historical significance present in the scene and moving close enough so that the Point of Interest (POI) is visible. This is verified via a trigger-zone around the object that is set in accordance with the draw-distance setting of the water. Upon entering the trigger-zone the player is prompted by a diving tablet that new information is available.

Inspecting the tablet (see Figures 8, 9) will reveal to the player an image of the POI alongside a short explanation of what is visible and two drop-down menus labeled as “theory” and “reasoning”. The player may select a theory they deem acceptable based on the physical evidence present in the scene as to the nature of the POI presented and a corresponding reasoning for their theoretical assumption.

FIGURE 8. Information about the excavation is shown on the diver’s tablet.

FIGURE 9. Research during gameplay.

For example, one of the tasks is to decide whether it is a warship, ferry or merchant ship. An inspection of the excavation site reveals a large number of amphorae, which may have filled most of the ship. From this, the find can be classified as a merchant ship.

When the player is satisfied with their selection they may hit a button to submit their solution for evaluation. If they picked the correct theory and reasoning the player is presented with a more detailed explanation of the POI and awarded points. It is planned that due to combinatorial explosion discovering the solution via “brute force” will not be feasible. The score value of the “research” attempt will be decremented with each incorrect guess to further discourage this.

At the core of this GM is a basic simplification of the scientific method - the player observes their surroundings, postulates based on findings, examines evidence to confirm or deny the theory and reexamines the hypotheses if need be. The GM should provide a certain level of challenge, as the correct solution is based on the meta-data of the construction of the POI only. To this effect the small degree of prompting—the presence of the answer as an option in a drop-down menu providing the player with the correct answer before they can analyze the problem—present in the implementation of drop-down menus could be beneficial, acting as an organic hint that would still require active thinking from the player.

6.3 Item collection

This GM was implemented initially as a temporary test-case for interactions with the VR control system, but eventually gained permanence for several reasons. Firstly, while contemplating a believable manner by which to guide the player from POI to POI an emergent solution presented itself—making use of coins present in the system as a “breadcrumb trail” would enable guiding the player to the different locations by means of motivational reward in a way that is not too unrealistic. Secondly, later developmental iterations on the prototype saw the concept of refuse collection as well as coin collection as a means of incorporating the important subject of environmental preservation to the game. The coins are chosen thematically to match the excavation.

At the wreck site it is possible to collect fragments of amphorae (see Figure 10) or other small findings. In future iterations we will add the possibility to salvage bigger findings by using special equipment.

FIGURE 10. Amphora at the wrecksite.

6.4 Voice-over narration and information mode

Throughout the game the player is guided by voice-over narration emulating a controller assisting the player in their work. The narration was designed to react to player input based on the interaction performed by them, such as collecting coins, trash or local fauna, as well as performing research.

If a user prefers to experience a “classical” guided tour, the game mechanics can be disabled. Instead, they will hear the appropriate explanations at the corresponding POIs by voice-over narration. The POIs can be reached by swimming or clicking.

7 Expert’s feedback

This project was implemented in times of the pandemic. Therefore, it was not possible to do a study with a sufficient number of participants. Instead we presented our application to an underwater archaeologist to get feedback from an expert. In the following we summaries the insights we have gained from this:

From the perspective of a fully professional underwater archaeologist, it was always a wish to have a diving simulation that guides from the surface to the submerged cultural features, accompanied by information and further tasks. The present application has made up for this in some respects. It presents an original find site of the coast of Istria of a late Roman wreck from North Africa, which was captured using structure from motion. Up to now, find material has been presented to the interested layman either in a classical scientific form of technical drawings and/or photographs, or partly in the form of 3D models, mostly detached from the find. The step taken in this application of showing the material exactly where it was found during the work is therefore still a novelty in an overall comparison. The advantages of this approach are obvious: the exact circumstances are made even clearer to the viewer and user, through an experience that is as true to nature as possible. From the point of view of an underwater archaeologist, who in this particular case accompanied the excavations at the presented site from beginning to end, this means that the user of the application is given the opportunity, to a certain extent, to experience some of the impressions on site even without training and diving experience and, in addition, to gain cultural knowledge about the wreck find. The disadvantages of the implementation immediately present the possibilities for future extensions. A major aspect of underwater archaeological excavations is missing, which has never been made accessible to the user in digital form so far. The handling and preparation of the necessary equipment such as dredges (sediment suction devices), photographic and video equipment as well as tools for surveying, salvage and excavation would be an ambitious but quite conceivable addition to future approaches of this kind. This could provide another aspect that would be good to channel especially in digital technology and whose importance cannot be overstated. Implementing this next step would give the user the opportunity to comprehend such research work even more without having to put on the diving equipment himself.

8 Conclusion and future work

In this paper we presented a diving simulator in VR which enables the virtual visit of a real excavation site underwater. This was possible by the cooperation of the faculty of computer science at TUM and the faculty for the study of culture at LMU.

The potential for further research in the field of SGs is vast. Even in the relatively specialized field of UCH VR SGs there are multiple available directions to explore and in which to advance the state-of-the-art.

Furthermore, the implementation of different methodologies used in UCH work could offer valuable insight into the challenges and benefits inherent to them. This would include an expansion of the existing implementation scope to include various tools and techniques currently employed by underwater archaeologists. A notable example of such a tool that should be present in future implementations of the prototype is a water dredge - a vacuum-like tool used to remove sediment and silt from sites of archaeological activity to improve visibility and potentially locate artefacts among the debris. The aforementioned tool is in common use in UCH work nowadays and is conspicuous in its absence from UCH related SGs.

The use of “Vive” trackers could theoretically enable the use of the user’s legs to control swimming - presenting a much closer experience to real-world diving. This would require a redesign of the diving controls present in the current version of the game into an even more realistic recreation of diving but would also improve overall immersion in the player.

In the future we will conduct studies to evaluate the player experience and the knowledge transfer in the different modes of our simulation.

Data availability statement

The original contributions presented in this article, will be made available by the authors. Further inquiries can be directed to the corresponding author.

Author contributions

DP had the initial idea und was supervising the project together with GKl. LK and GK implemented the diving simulator. MF was responsable for the archeological content. All authors were involved in the writing process of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹https://store.steampowered.com/app/861400/Nefertari_Journey_to_Eternity/.

²https://atlasv.io/projects/dawn-of-art/last access(03/18/2022).

³https://www.ballastvr.com/divr last access 03/18/2022.

⁴http://www.visas-project.eu/en/(last access 03/18/2022).

⁵https://www.ballastvr.com/divr (last access 03/18/2022).

⁶https://haptx.com/∼(lastaccess03/18/2022).

⁷https://github.com/wave-harmonic/crest (last access 03/18/2022).

⁸https://assetstore.unity.com/packages/templates/systems/vr-interaction-framework-161066 (last access 03/18/2022).

References