TREE STEM RECONSTRUCTION USING VERTICAL FISHEYE IMAGES : A PRELIMINARY STUDY

A preliminary study was conducted to assess a tree stem reconstruction technique with panoramic images taken with fisheye lenses. The concept is similar to the Structure from Motion (SfM) technique, but the acquisition and data preparation rely on fisheye cameras to generate a vertical image sequence with height variations of the camera station. Each vertical image is rectified to four vertical planes, producing horizontal lateral views. The stems in the lateral view are rectified to the same scale in the image sequence to facilitate image matching. Using bundle adjustment, the stems are reconstructed, enabling later measurement and extraction of several attributes. The 3D reconstruction was performed with the proposed technique and compared with SfM. The preliminary results showed that the stems were correctly reconstructed by using the lateral virtual images generated from the vertical fisheye images and with the advantage of using fewer images and taken from one single station. * Corresponding author


INTRODUCTION
Tree stem reconstruction from optical images is a topic of growing interest, but usually dozen of images from different viewpoints are used, mainly when using techniques like Structure from M otion (SfM ).The computation of SfM is based on image correspondences over images acquired with large overlap, being nowadays one of the most popular image-based 3D modelling algorithms.A collection of SfM applications in geosciences was reported by Westoby et al. (2012).
Although stereo-photogrammetry has a long history, the automatic 3D reconstruction using point clouds extracted from multiple images was limited by the computational load and problems with image matching.In recent years, the significant advance of digital cameras and dense matching techniques has made image-based point cloud an important data source for tree measurements (Liang et al. 2015).Thus, several studies have been proposed using terrestrial optical images in forest applications.Typically, ground surveys are conducted in forest sample plots to collect variables as diameter at breast height (DBH) and tree positions, as well as to identify types of species and environmental conditions.Hapca et al. (2007) used two digital images taken from two stations with convergence defining an intersection angle of 90° to reconstruct the 3D shape of standing trees.Herrera et al. (2011) presented a fisheye stereovision method using image segmentation and image matching to separate textures of interest in forest environments and generating disparity maps of tree trunks with their approach.Liang et al. (2014) presented a work assessing the potentiality of point clouds generated using an uncalibrated hand-held camera at a forest plot.Individual trees stems were detected and modelled from the point cloud.Forsman et al. (2016) introduced a prototype of a multi-camera rig to collect images from a single station at the plot centre.Tree stem attributes were estimated by photogrammetric techniques.However, this technique used a complex system with five cameras that had to be accurately calibrated and synchronize to obtain reliable measures.
The previously reported approaches used several images to extract tree data or reconstruct stems, which make them timeconsuming techniques.In this paper, an alternative is proposed using fisheye cameras with vertical displacement, which reduces significantly the field work because fishey e lenses increase the possibility of collecting images with a large ground coverage area.
Terrestrial laser scanning (TLS) is a technology that has been widely used for forest measurements.However, according to Forsman et al. (2016), when TLS is compared with photogrammetric techniques, camera-based methods offer potentially less expensive hardware, increase mobility and reduce time on-site.
The main objective of this preliminary study is to investigate the feasibility of the technique proposed for stem reconstruction, which enables later measurement and extraction of several attributes from the reconstructed stem model.In this technique, the original vertical panoramic image acquired with fisheye camera is transformed to horizontal lateral images following conventional perspective geometry.
Thus, photogrammetric procedures can be performed with ordinary software based on collinearity equations.

Image data and study area
A camera with fisheye lens is used to collect panoramic terrestrial images of tree stems inside forests.This type of image enables to acquire a large field of view, which reduces the number of images needed to generate the 3D reconstruction.The image acquisition system and the processing technique for forest areas were originally presented by Tommaselli and Berveglieri (2014) and Berveglieri et al. (2014) and it was adapted to be applied with this reconstruction technique.A telescopic pole attached to the fisheye camera in nadir viewing is used to collect vertical images among trees, as shown in Figures 1(a,b).2015)) is used with addition the Conrady-Brown model (Fryer and Brown 1986).A bundle adjustment is performed using constraints imposed to the ground coordinates, observations and Exterior Orientation Parameters (EOPs) as described by Tommaselli and Berveglieri (2014) and Berveglieri et al. (2014).
The study area is located in M asala (60.15°N, 24.53°E), southern Finland, where Scots pines (Pinus sylvestris L.) in mature stage are found.Such pines, common in Scandinavian regions, have features that are suitable for applying the proposed technique.

Tree stem reconstruction technique
Firstly, the fisheye camera is positioned and leveled is suitable place inside the forest trying to minimize occlusions from that view point.A vertical image sequence is acquired using small variations in heights ranging from 4 m to 5 m above the ground.
The images are collected in the same planimetric position (X, Y) but at different heights (Z).Small vertical displacements are used to avoid abrupt viewpoint changes among sequential images.During the image acquisition, each camera height must be recorded to be later used as constraint in the image orientation.
Each fisheye image is rectified to vertical planes, generating lateral views, as exemplified in Figure 2(a).This procedure produces images of the vertical trunks at similar scales in the image sequence, facilitating image matching which is later performed only with corresponding points belonging to stems regions.As depicted in Figure 2(b), the virtual horizontal images simulate a strip sequence in a local reference system.After rectification, the images follow the perspective geometry and then the collinearity equations can be used for the image orientation with bundle adjustment.Area-or feature-based techniques can be used to generate tie points, with the advantage that points will be restricted to areas close to the trunks.As ground control points are not used in this technique, a relative image orientation is performed in a local reference system.Next, a dense matching technique is applied to generate the point cloud, which allows the stem reconstruction.

EXPERIMENTS AND RES ULTS
The tree stem reconstruction was performed with the proposed technique and compared with SfM , based on the generated point clouds.

Image acquisition and data processing
A set of nine images was collected with the fisheye camera raised to heights ranging from 4.15 m to 4.55 m with increments in Z of 5 cm to perform the experiments in the study area, as shown in Figure 2.
After rectification to vertical planes, a photogrammetric project in Leica Photogrammetric Suite (LPS-ERDAS) was configured to perform a relative image orientation in a local reference system as follows:  IOPs were fixed.


EOPs of the first image (the lowest height) were fixed with absolute constraints.The coordinates (X, Y) were fixed as the origin (0, 0) planimetric coordintates and Z used the height measured during the image acquisition;  EOPs of the following images were defined with weighted constraints, considering standard deviations of σ = 5 cm for XY, due to moviments when lifting the camera, and σ =1 mm for Z because this coordinate was accurately measured in field.The initial values for XY were (0, 0) and Z for each image was defined with its respective height collect in field;  Tie points were automatically extracted from the segmented stems using high point density.
As the aim is to use few images in the stem reconstruction, only four images were inserted into the LPS project to generate a point cloud.In the tests, two lateral views (left and right in Figure 2(a)) were sufficient to cover all trees appearing in the scene.Then, two image strips from opposite sides were used in the bundle adjustment.After image orientation, a Digital Surface M odel (DSM ) was automatically generated with grid spacing of 2 cm and using the Automatic Terrain Extraction (ATE) module from LPS. Figures 5(a,b) show the point clouds generated with dense matching via ATE-LPS.
Typically, many images are required to apply the SfM technique.Thus, the nine rectified and segmented images of each side (left and right) were used to generate point clouds for comparison with the results from LPS.The Agisoft Photoscan software, which implements a SfM strategy, was used with default configuration.As can be seen in the comparison between Figures 5 and 6, four imagens using LPS were sufficient to produce point clouds with more density than those with SfM , which used sequences with nine images.Only four images in the SfM were not enough to generate point clouds.Later, the produced point clouds can be used to extract measures of the stems.
From the point clouds produced, a surface or mesh can be built and texturized to generate models.Figures 7(a,b) present models in tiles generated by Photoscan using a default configuration.In this case, the purpose was only to show a realistic model that can be produced with the virtual horizontal images.

Accuracy assessment with distances
An accuracy assessment with distances was performed over the reconstructed stems.During the image acquisition, distances at approximately 1.30 m of height were measured from the camera station for each tree using an electronic distance measurement device.Considering that the measurements were performed without targets over the stems, there may be an error around ~10 cm associated with the distances due to uncertainties in the field measurements, irregularities in the stem surfaces and in the point location over the point cloud.Even that, the distances were used to verify if the results were close to the observed distances.
From each stem within the 3D point clouds generat ed with ATE-LPS, the centre point closest to 1.30 m height was selected to check the distance between the camera station and the tree.As can be seen, the differences had values in a centimetre-level.The largest and smallest values were respectively 21.3 cm and 5.7 cm, being the largest error generated with the largest distance from the tree to the camera station.The differences were 12.8 cm on average.It is important to note that the discrepancies showed in Table 3 evidenced that the point clouds have acceptable 3D geometry provided by the fisheye technique.

CONCLUS IONS
This paper presented a feasibility study for stem reconstruction using few images taken with fisheye lenses.The purpose was to develop a straightforward technique for image acquisition in forest plots without requiring significant time demand.Furthermore, the technique was designed to use ordinary photogrammetric software for data processing with perspective geometry, enabling the accurate data extraction of tree trunks.
The experiments showed that it was possible to reconstruct stems using horizontal virtual images created from the vertical fisheye images, with the additional advantage of using fewer images and taken from one single station.When comparing point clouds generated by ATE-LPS and SfM , the bundle adjustment via LPS produced more dense 3D point clouds.The preliminary results were only checked with distances measured between the trees and the camera station.However, the achieved accuracy (12.8 cm) make feasible to conclude about the technical feasibility, which can be improved with some further refinements.
For future studies, the results achieved with the fisheye technique should be assessed to verify the accuracy level of tree attributes, for example, errors of DBH measures and tree locations for forest inventory.In addition, point clouds generated with fisheye images should be compared with point clouds collected with TLS, verifying whether or not the data provided by photogrammetry can present similar accuracy.
Figure 1.(a) Fisheye system positioned among trees.(b) Acquisition of multiples vertical images with height variation.
Figure 2. (a) Vertical fisheye image rectified to four horizontal lateral views.(b) Lateral image sequences simulating photogrammetric strips.

Figure 3 .
Figure 3. Sequence of image processing to extract tree stems: (a) Rectified and smoothed image in gray levels; (b) detection of straight structures; (c) Stem labelling windows.
The fisheye camera was firstly calibrated in a 3D terrestrial calibration field composed of coded targets with Aruco format(Garrido-Jurado et al. 2014), which were automatically located in the images using a software adapted bySilva et al. (2014) to identify the targets in the field.Figure4shows an example of fisheye image with Aruco target corners extracted.IOPs were estimated using the Calibration M ulti-Camera (CM C) software(Ruy et al. 2009;M arcato Junior et al., 2015) via bundle adjustment with equidistant model for fisheye lens.Table2displays the estimated IOPs.

Figure 4 .
Figure 4. Example of a fisheye image collected in the terrestrial calibration field showing Aruco target corners that were automatically extracted.

Figure 5 :Figure 6 :Figure 7 :
Figure 5: Point clouds generated by bundle adjustment and ATE in LPS using images from the (a) right and (b) left .

Table 1 .
Features of the fisheye camera.

Table 3 .
Table3shows the differences obtained by subtracting the estimated distances from the directly measured distances.Error assessment with distances.