The Augmented Laboratory – a mixed reality setup for physics education

We discuss an educational project, The Augmented Laboratory, aimed to bring innovation in educational physics laboratories in formal and informal contexts. It is based on the capability of obtaining 3D reconstructions of the physical world which can be visualized both live as the data is acquired and later on as interactive playbacks. The setup allows to obtain 3D tracking of multiple objects, to project data of interest directly on and around the objects used in a real experiment, and to introduce the new concept of “augmented experiments”. The setup was tested with a group of university students and as a scientific exhibit for hands-on activities.


Introduction
We present an educational project, The Augmented Laboratory, aimed to bring innovation both into the physics laboratory and into physics teaching in informal settings (exhibits, museums). It is based on the capability of obtaining and visualizing a three-dimensional reconstruction of the physical world in real time, using a depth camera and a custom software developed by one the authors [1,2]. The "OpenKinect" [3] and "Processing" [4] open source libraries were used to code the software. In figure  1 two 3D reconstructions of different scenes can be seen. These reconstructions can be seen in real time and also recorded to be seen and analyzed later as interactive playbacks. Adding a projector to the setup, mixed/augmented reality is achieved through the projection on the objects and the environment. This opens to new frontiers of exploration of physical phenomena. It is also possible to make simulations interact with the physical world in various ways, introducing the concept of augmented simulations. Some videos available at [2] show in more detail how The Augmented Laboratory works. This setup allows for three main categories of activities here presented: 1) multiple object, threedimensional motion tracking, 2) augmented experiments and 3) augmented simulations.

Multiple object, three-dimensional motion tracking
At first we discuss the capability of acquiring three-dimensional motion data of multiple objects. There are various techniques that allow us to distinguish particular objects from the background [1] [5], allowing us to do multiple objects motion tracking, meaning to acquire data on the motion of the IOP Conf. Series: Journal of Physics: Conf. Series 1287 (2019) 012059 IOP Publishing doi:10.1088/1742-6596/1287/1/012059 2 objects of interest. The acquired data can be visualized and analyzed in a 3D virtual environment to help students understanding the physics behind the performed experiments. Some experiments regarding some fundamental features of Newtonian mechanics were performed to test the tracking capabilities of our setup, paying particular attention to check and visualize constants of motion [1]. We present here two simple experiments. In the first experiment a collision between two spheres is recorded. We then look at the total linear momentum of the system before and after the collision obtaining, respectively, P0 = (32 ± 1)×10 -3 kg m s -1 and Pf = (31 ± 2)×10 -3 kg m s -1 , concluding that such measurements are consistent with the expected conservation of momentum. We then compute the total (kinetic) energy of the system: the measured loss of energy is below the measurement uncertainty, with the initial and final values being E0 = (0.63 ±0.01) J and Ef = (0.64 ±0.02) J, respectively. Of course this implies a good elasticity of the observed collision.   For the second experiment we created a bolas (a throwing weapon made of several balls connected by strings) using a nylon line and 2 spheres (figure 3, left). We threw the bolas making it spin and tracked its motion. The resulting motion is the superposition of the motion of the centre of mass and the rotation of the spheres around it ( figure 3, right). The measurements are consistent with the expected conservation of angular momentum, showing relative fluctuations of about 10% around the mean value: L = (1.9 ± 0.2) × 10 -3 kg m 2 s -1 . Similarly, measurements are consistent with the conservation of energy, with relative fluctuations of about 3% around the mean value E = (16.5 ± 0.5) J.

Augmented Experiments
As already mentioned in the introduction, using a projector we can achieve mixed reality projecting information of interest directly on and around the objects used in the experiments. For this reason, we will refer here to augmented experiments. This capability translates into more user-friendly activities in the classroom.

Augmented Simulations
A new door opened by the mixed reality setup is that of exploring new interactions between computer simulations and the real world. We will refer to this new kind of interactions as "augmented simulations". An example of augmented simulation is that of projecting simulated objects on the real world whose movement is influenced by the physical world itself. This way it is possible to directly see the behavior of the objects following the same physical laws that are taught, and it also allows us to compare the results of measurements and simulations ( figure 5, top).
A second example of augmented simulation is that of the ripple tank. It is possible to perform any experiment regarding wave propagation by moving physical objects in the chosen area. The objects represent wave sources, obstacles, lenses, and areas of different wave propagation speed, and the software is capable of recognizing and distinguish them. This way, the discussion of topics such as

Conclusions
We presented the tracking and mixed reality capabilities of our software/hardware setup, "The Augmented Laboratory". We also introduced two new concepts, augmented experiments and augmented simulations. This setup opens to new ways of presenting and discussing various physics topics, ranging from motion to wave propagation and much more. We believe that these features may have great value for educational purposes in the near future. In order to be able to bring this setup in schools and museums as soon as possible a new, open source version of the software has to be designed and distributed: for this reason, a simple website of the project has been created [2], to present the project and to try to get as many collaborators around the world as possible.