ACCURACY ASSESSMENT OF UNDERWATER PHOTOGRAMMETRIC THREE DIMENSIONAL MODELLING FOR CORAL REEFS

Recent advances in automation of photogrammetric 3D modelling software packages have stimulated interest in reconstructing highly accurate 3D object geometry in unconventional environments such as underwater utilizing simple and low-cost camera systems. The accuracy of underwater 3D modelling is affected by more parameters than in single media cases. This study is part of a larger project on 3D measurements of temporal change of coral cover in tropical waters. It compares the accuracies of 3D point clouds generated by using images acquired from a system camera mounted in an underwater housing and the popular GoPro cameras respectively. A precisely measured calibration frame was placed in the target scene in order to provide accurate control information and also quantify the errors of the modelling procedure. In addition, several objects (cinder blocks) with various shapes were arranged in the air and underwater and 3D point clouds were generated by automated image matching. These were further used to examine the relative accuracy of the point cloud generation by comparing the point clouds of the individual objects with the objects measured by the system camera in air (the best possible values). Given a working distance of about 1.5 m, the GoPro camera can achieve a relative accuracy of 1.3 mm in air and 2.0 mm in water. The system camera achieved an accuracy of 1.8 mm in water, which meets our requirements for coral measurement in this system. * Corresponding author


INTRODUCTION
The Moorea Island Digital Ecosystem Avatar (IDEA) project was recently developed as a joint initiative by an interdisciplinary group of experts to create a digital Avatar of the Pacific island of Moorea, French Polynesia (Cressey, 2015, Davies 2016).The major goal is to develop a unifying framework where ecological data and social information are assembled into a comprehensive time-evolving model of the island -an Avatar that would allow scientists to generate and test hypotheses about the impact of natural and man-made activities (http://mooreaidea.org/).Moorea has attracted the interests of biologists, ecologists and environmentalists due to its strategic location in the middle of the South Pacific Ocean, where its surrounding coral reefs, offering some of the most complex ecosystems on earth, can be studied to reveal the effects of natural and anthropogenic.The precise 3D measurement of coral and its change over time will be an essential component of future investigation of impacts of climate change (e.g.ocean warming, increases in cyclones, ocean acidification).Changes in amount of corals on reefs are typically measured through two-dimensional assessments of coral cover.However, this metric fails to capture changes occurring in architectural complexity (Alvare-Flilip et al., 2011), and the importance of assessing this aspect has been increasingly recognized.Risk (1972) was among the first to report a strong positive correlation between fish species diversity and substrate topographic complexity, an observation that has been repeated for both abundance and species richness in a variety of coral reef systems including Moorea (Holbrook et al. 2002, 2003, Schmitt and Holbrook 2007).This is an important (albeit not the only) reason why the topography of coral habitats is so much of interest.After testing different, quite primitive methods of measuring coral topography (McCormick, 1994) the "chain-and-tape" method became a standard approach.This method uses "rugosity" as a measure of coral topographic complexity (small scale variation of height differences).Rugosity is defined as fr = Ar/Ag, whereby Ar is the real (true) surface and Ag is called the "geometric" surface area.The chain-and-tape method measures only profiles in vertical planes.It uses a fine-link chain which follows the height variations and its length is compared with the length of its vertical projection onto a horizontal reference plane.A decisive step forward in methodology was introduced by Du Preez and Tunnicliffe (2012), who used a laser point profiler together with a single video camera to reconstruct height profiles from moving platforms like ROVs, scuba divers, AUVs, or mounted on a trolley platform.This microtopographic laser scanning (MiLS) method follows the standard photogrammetric structured light principle.Friedman et al. (2012) further refined the technique, using a geo-referenced stereo camera system mounted on an ROV, AUV, manned submersible or diver-held platform and photogrammetric stereo processing principles to generate Digital Surface Models.
Traditional aerial and terrestrial photogrammetry has very successfully employed 3D modelling and the number of its applications in various disciplines has grown considerably in recent decades.In the field of underwater photogrammetry great efforts have been made in the areas of underwater archaeological surveys, marine biological investigations and offshore and other industrial inspection applications (Capra, 1992).In particular, recent advances in both underwater photographic equipment and automation of photogrammetric 3D modelling software have triggered interest in building simple and low-cost solutions for reconstructing highly accurate 3D object geometry in underwater environments.
The first underwater camera systems were developed by the French marine biologist Louis Boutan at the end of the nineteenth century (Nocerino et al., 2016).Underwater photographic equipment remained complicated and costly, and thus limited to professional usage by experts until recently when consumer-grade compact digital cameras with waterproof housings became widely available.Among a large number of off-the-shelf underwater cameras, the GoPro cameras with their waterproof dive cases are very popular because they are very small, compact, robust, and low-cost, with a high resolution still camera that has high definition video capability.Currently GoPro cameras are increasingly used for underwater photogrammetry, including bathymetry measurement (Schmidt et al., 2012), underwater archaeological surveys (Capra et al., 2015.Advances in software for automated processing and production of 3D models from image sequences greatly expand the usability of low-cost camera systems in photogrammetric applications, including underwater measurement and modelling.Traditional photogrammetric methods require manually controlled processing such as precise measurements of control/tie points for image orientation, camera calibration and geo-referencing processes, which normally leads to considerable effort and expense to process large blocks of images.Automated processing techniques have seen considerable progress recently.Improvements in feature extraction and image matching have helped automation performance (Remondino et al., 2008) speeding up processing even of very large blocks of images considerably (Agarwal et al., 2011).Moreover, many low-cost commercial and open-source software packages for automated 3D modelling from images have been developed (reviewed by Remondino et al., 2012).
The accuracy of the high-resolution 3D model produced by such solutions is comparable to more expensive photogrammetric and Lidar techniques (Thoeni et al., 2014;Javernick et al., 2014;Remondino et al., 2012).Therefore it is very straightforward to adapt such techniques to underwater 3D modelling.
There have been numerous recent publications reporting underwater 3D modelling in archaeological surveys, biological investigations and industrial inspection, and most of the generated 3D models are visually very impressive.For the applications of visualization, object-based navigation and other similar purposes, these 3D models are sufficient, but if accuracy and reliability matter, it is essential to precisely quantify 3D geometry.However, to date the accuracy issue of underwater 3D modelling has not been adequately addressed.For example, in the Moorea IDEA project (Cressey 2015, Davies 2016) an important task is to detect the changes of corals in 3D over time in order to monitor the growth of coral colonies and their responses to environmental changes..According to a previous study (Bessat et al., 2001) the average annual growth rate of coral in Moorea is about 11-14 mm.This means that our 3D coral model accuracy should be on mm level.
Assessment of accuracy of underwater 3D modelling is challenging, because the precise measurement of Ground Control Points (GCPs) and check points underwater is difficult.Both the properties of water and camera influence image formation, by altering the path of optical rays (Nocerino et al., 2016).Specifically, the camera model is quite different underwater compared to air.Mathematical models have been presented that address the refraction problems (Maas 2015; Agrafiotis et al. 2015). .
In order to conduct a successful underwater photogrammetric project, minimization of the effects from the potential error sources at every stage is crucial.It is important to understand what factors influence the underwater photography and the processing functions of 3D modelling.First of all, the properties of water need to be considered.The density of water is nearly 800 times that of air, and is also not constant, being a function of temperature, salinity and pressure, all of which can be correlated (Nocerino et al., 2016).This results in strong variability of the refractive index for seawater.Illumination conditions underwater are also very complicated.A great amount of the sunlight radiation is reflected on the surface and absorbed, and the different components of light in different wavelengths are absorbed differently so that it makes the underwater environment appear bluish and greenish.Moreover, turbidity, scattering, and backscattering of the external light sources lead to poor illumination conditions underwater.This may lead to blurred images that could affect feature identification and automatic extraction, as well as produce poor quality texture at the data processing stage.Last but not least, the characteristics of underwater photographic equipment are also critical.The geometric and optic characteristics of the lens port such as the shape (flat and dome), the volume of the port space, and even the alignment of the port with the lens affect the geometric quality of the images (Nocerino et al., 2016).
Good calibration of the underwater camera system is fundamental to ensure the accurate and reliable measurement of 3D objects.The conventional approach is to carry out the calibration in a suitable environment beforehand, such as in a swimming pool.However, the difference between the calibration environment and the actual working environment in the sea can be substantial.Instead of such off-line calibration we prefer in-situ calibration, which involves placing the calibration devices in the actual environment, and simultaneously acquiring the objects and calibration information embedded in the same images in order to minimize influences from the environment and to improve the accuracy of 3D models as much as possible.
The objective of this study was to develop a method for assessing the accuracy of a low-cost underwater photogrammetric 3D modelling solution, which consists of an off-the-shelf underwater consumer-grade camera system and automated 3D modelling software tools.Furthermore, we show an in-situ calibration method to ensure the accurate and reliable measurements of corals in 3D.For this we use a precisely surveyed 3D reference frame, which we put into the scene (both in air and underwater) and applied self-calibration.In addition, we also put cinder blocks, which have a similar surface roughness as some corals, into the scene.We do not have reference data on the cinder blocks.Therefore we determined their 3D models with the Lumix camera in air (which gave us the best possible values) and used these models as reference for all other investigations.The accuracies, which are in the mmrange, open the possibility of monitoring the growth of coral reefs in detail and documenting temporal change in these ecosystems with simple underwater photogrammetric solutions.

Underwater photography equipment
The underwater photography equipment used in this study comprise the cameras with lenses, waterproof housings with lens ports, and a calibration frame (Figure 1).To investigate the performance of different types of consumer-grade camera systems, two cameras mounted with waterproof lens ports as listed in The GoPro Hero4 is a popular, low cost and high resolution camera, and comes with a waterproof dive case with flat port that is capable of withstanding depths up to 40 meters.The Lumix GH4 is a mirrorless system camera used with an M.Zuiko ED 12-40mm f2.8 PRO lens in a waterproof Nauticam NA-GH4 housing with a Nauticam 180mm optical glass wide angle dome port , which makes the light rays enter the spherical dome port almost perpendicularly to the dome port and go to the perspective centre without refraction.
The calibration frame is composed of PVC bars which form approximately the shape of a parallelepiped of the following size: 0.9 x 0.2 x 0.15 m (Figure 1).Its weight is about 3 kg.Calibrated points are marked with square targets, 30 mm wide, with alternate black and white crosses.All targets on the frame have been numbered and measured in a reference system fixed on the frame.The measurement was performed by scanning the frame with a triangulation-based laser scanner (Konika Minolta RANGE 7) and by identifying the centre of each target on the three-dimensional model with an accuracy of better than 0.1 mm.The PVC thermal dilatation coefficient is about 7 ppm per °C.In our case, the variation of temperature from the sea surface to the working area, where the depth is up to about 15m is within a range of 3-5°C, which produces a potential (maximum) length variation of about 35 micron for the 1 m length of the bar.This geometric variation is far less than the accuracy requirements at some mm level for coral reefs measurement, and therefore the calibration frame is capable of providing very reliable control information in underwater environments.
Figure 1.Calibration frame with camera systems Lumix GH4 and the GoPro Hero4 stereo camera bar

Camera systems calibration in air and underwater
We utilized the Agisoft PhotoScan package in this study.
PhotoScan provides automatic image orientation, camera calibration, dense 3D point cloud generation, 3D mesh construction, and texture mapping from large datasets of overlapping image sequences.Control and reference information can be manually annotated, and then used to optimize image orientation, camera calibration and coordinate system referencing.PhotoScan uses the Brown approach for self-calibration (Brown, 1971).
The reliability and accuracy of the camera calibration is mainly affected by the image quality and the geometry and redundancy of the calibration image network.To ensure the calibration accuracy, the following criteria have been suggested: (1) the camera and target arrays are three dimensional in nature; (2) the different convergent camera views approach a 90 degree intersection at the centre of the target array; (3) the calibration fixture or range fills the field of view of the camera(s) to ensure that image measurements are captured across the entire format; and (4) the camera(s) are rolled around the optical axis for different exposures so that 0, 90, 180 and 270 degree orthogonal rotations are spread throughout the calibration network (Shortis, 2015).
In practice, it is difficult to meet all of these criteria.The compact calibration fixture is very easy to transport and used in various environments.In particular, for in-situ calibration underwater, a calibration fixture has to be mobile in order to be set up easily at different working sites.In this study, it was not always possible to fill the full format of images while measuring coral reefs by "flying" over at 1-1.5 meter working distance.In some cases, the dimension of our 3D calibration frame could not fill entire image format.On request of our project partners we placed several objects --five cinder blocks with various shapes in the calibration scenes.One of the purposes was to measure these blocks with a good accuracy with the photogrammetry approach and to compare with the measurement using a sonar device employed by one of the coauthors in another project.We also utilize the different arrangement of the these cinder blocks both in air and underwater to compare the geometric variations of the 3D models generated in different cases, and to investigate the accuracy consistency in the area next to the calibration frame (Figure 2). Figure 2 shows both calibration sites in air and underwater.The cameras were used to take images at a working distance of about 1.5 meter in air and underwater.Figure 2(d) shows that it was not easy for a diver to shoot images evenly distributed covering the objects in the underwater environment, thus there are gaps in the 3D point clouds (Figure 2c).The measurements of the markers of the calibration frame were done manually in single image mode, while the cinder blocks were measured fully automatically, producing very dense point clouds.The results of self-calibration are shown in Table 2.
Table 2. Calibration results of camera systems in air and underwater.f is the focal length; cx and cy are the principal point offsets; k1, k2, and k3 are symmetric radial distortion coefficients; b1 and b2 are affinity and non-orthogonality (skew) coefficients; p1 and p2 are decentering distortion coefficients (Agisoft PhotoScan User Manual).
The calibration results show a large variation with the GoPro camera system both in air and underwater.The change is expected as the flat waterproof case has less geometric accuracy and more refractive influence than does the dome housing.In particular, the symmetric radial distortion increases at the borders of the flat lens case.Figure 3 shows the distortion maps of each camera in air and underwater.The Lumix system has an offset to the principle point, but less radial distortion, than the GoPro, which has a small fish eye type lens, resulting in significant radial distortion towards image borders, especially underwater.

Accuracy assessment of 3D point clouds
There are 45 highly accurate reference points on our 3D calibration frame.Of these 12 evenly distributed points were selected for the self-calibration procedure and the other points were treated as check points for accuracy assessment.The RMSE between the computed coordinates and the known coordinates of check points were calculated (Table 3).The results were quite encouraging.Even though the accuracy of the GoPro was degraded about 0.25 mm compared to the Lumix, the GoPro was still able to achieve an accuracy of about 0.5 mm underwater, which meets our requirements for coral measurement.In an additional effort we investigated the accuracy consistency of objects in the space next to 3D calibration frame.

Accuracy Lumix GoPro Air
Underwater Air Underwater RMSE (mm) 0.12 0.15 0.36 0.40 Table 3. Accuracy (RMSE) of camera systems in air and underwater computed from 33 check points at the reference frame.
Firstly, for the calibration site in air, by using the open source software CloudCompare (www.danielgm.net),we were able to compare the 3D point clouds of the full scene generated using images from Lumix and GoPro camera systems (Figure 4 and Table 4).The difference between the two 3D point clouds is quite small.Larger variations are observed on the blue plastic sheet at the background (see the red portions of the error surface), because the surface shape of the plastic sheet was changing during image acquisition due to wind.The same comparison was also conducted for the calibration site underwater as shown in Figure 5 and Table 4.The results are quite similar.The variation comes mainly from different coverage of the background (in air we have a blue plastic sheet, under water we have sand).Table 4. Difference between 3D point clouds from Lumix and GoPro camera systems of calibration scenes in air and underwater.
In order to mitigate the influence of the background in the comparison of the 3D point clouds, a comparison of the 3D point clouds of each object (cinder block) was also conducted.
Considering the good geometric and optical characteristics and the accuracy as shown in Figure 3 and Table 3, 3D point clouds generated by using Lumix in air were relatively precise.Therefore, the 3D point clouds of five cinder blocks and the calibration frame were extracted from the 3D scene generated by using Lumix in air (Figure 6).Since the intensive editing has been done in the extraction processing, they were matched back to the original 3D point cloud of the calibration scene to check the accuracy/correctness to their original shapes.This resulted in a relative RMSE of 0.13 mm for 3D geometric variation.
These 3D point clouds of objects were further utilized as the reference in the following object comparison.
Figure 6.3D point clouds of objects (a-e are cinder blocks) extracted from the 3D scene generated by using the Lumix camera system in air.The 3D point cloud of each reference object was compared to its counterpart in other 3D scenes respectively.The comparison results are as shown in Figure 7 and the RMSEs are shown in Table 5.Table 5. Accuracy comparison of 3D point clouds generated using Lumix and GoPro camera systems in air and underwater.

CONCLUSIONS
As a part of the Moorea Island Digital Ecosystem Avatar (IDEA) project (http://mooreaidea.org/),this study aimed to develop a low-cost and high-efficiency approach for assessing the growth of coral reefs over time in detail in 3D space, which provides valuable data and metrics to quantify changes and trends in reef ecosystems.Because the average annual growth rate of coral in Moorea is about 11-14 mm, it requires that the solution is able to provide 3D measurement at mm level accuracy underwater.
The combination of off-the-shelf underwater consumer-grade camera systems and automated 3D modelling software tools have made it possible to apply photogrammetric 3D modelling methods in underwater environments in an economical and efficient way.However, the accuracy assessment issue of underwater 3D modelling is quite challenging, because the precise measurement of GCPs and check points in the unconventional environment of underwater is difficult, and water body--lens port--air in waterproof lens housings form a complicated multiple media environment that greatly alters the path of optical rays.Moreover, physical factors in the Calibration of underwater camera systems is fundamental to ensuring the accurate and reliable measurements of 3D objects.Instead of the conventional way, which carries out the calibration beforehand, we applied an in-situ calibration method by placing a calibration frame in the actual working environment to minimize the influence from environmental factors and improve the accuracy of 3D models as much as possible.We compared two underwater camera systems -a GoPro camera with a flat waterproof case, and a Lumix GH4 system camera with a hemispherical dome port.The selfcalibration results suffer due to the fact that the software we used (Agisoft PhotoScan) did not deliver a statistical analysis of the self-calibration parameters (e.g. the covariance matrix).Therefore, the stability and significance of individual parameters could not be checked.But it was found that, due to the dome port of the Lumix camera, the correction parameters are much smaller than for the GoPro camera with flat port housing.
The 3D measurement accuracies of both underwater camera systems are quite encouraging.Though the accuracy was degraded about 0.25 mm comparing to Lumix, GoPro was still able to achieve the accuracy of about 0.5 mm underwater, which meets our requirements for coral measurement in this study.We further examined the relative accuracy of the point cloud generation by comparing the point clouds of the individual objects with the objects measured by the system camera in air (the best possible values).Given a working distance of about 1.5 m, the relative accuracy of 3D point clouds generated by the Lumix camera system underwater was about 1.82 mm, while GoPro in air was about 1.34 mm, and underwater about 1.97 mm.The results show both underwater camera systems are able to meet the 3D modelling accuracy requirements for coral measurements.It must be emphasized that these accuracy values have been computed by comparing the point clouds of the individual objects with the corresponding ones generated with the Lumix camera in air.They are not based on comparisons with independent reference values.
Figure 8 shows the 3D point cloud of a 5m x 5m coral reef patch generated using images from a GoPro underwater system.The calibration frame is clearly visible, as is a band that marks the border of the patch and serves as a visual reference for the diver for spatial orientation during image acquisition.The accuracy of the camera calibration and point cloud generation is strongly affected by both the image quality and the geometry and redundancy of calibration image network.Therefore, it is essential to plan and ensure an optimal image network.We also note that the flat case of lens caused the different refraction shifts to different wavelengths towards the image border as shown in Figure 9, which caused image blur and would affect the quality of 3D point clouds.In such cases efforts should be made to correct for this blue shift during the calibration phase.Future studies will include a complete accuracy assessment to the 3D point clouds of coral patches, and detection of changes in corals in 3D and further linkages to ecosystem studies such as responses of coral reefs to environmental disturbances and their rates of re-growth of coral. .

Figure 2 .
Figure 2. Camera systems calibration settings in air and underwater.(a) 3D point cloud of calibration site in air.(b) Image network in air.(c) 3D point cloud of calibration site underwater.(d) Image network underwater.

Figure 3 .
Figure 3. Distortion maps of camera systems.The red circle is the initial principle point, and the blue one is the calibrated principle point.(a) Lumix in air, (b) Lumix underwater, (c) GoPro in air, (d) GoPro underwater.

Figure 4 .
Figure 4. Comparison of 3D point clouds from Lumix and GoPro camera systems of calibration scene in air.

Figure 5 .
Figure 5.Comparison of 3D point clouds from Lumix and GoPro camera systems of calibration scene underwater.

Figure 7 .
Figure 7.Comparison of 3D objects.(a) GoPro in air to Lumix in air (b) Lumix underwater to Lumix in air (c) GoPro underwater to Lumix in air 3D point clouds Air Underwater GoPro Lumix GoPro RMSE(mm) 1.34 1.82 1.97

3D
can vary strongly with depth and rapidly over time.

Figure 9 .
Figure 9. Blue band shift towards image border

Table 1 .
Table 1 are compared.Underwater cameras used in this study.