3 D OCTREE BASED WATERTIGHT MESH GENERATION FROM UBIQUITOUS DATA

Despite of the popularity of Delauney structure for mesh generation, octree based approaches remain an interesting solution for a first step surface reconstruction. In this paper, we propose a generic framework for a octree cell based mesh generation. Its input is a set of Lidar-based 3D measurements or other inputs which are formulated as a set of mass functions that characterize the level of confidence on the occupancy of each octree’s leaf. The output is a binary segmentation of the space between occupied and empty areas by taking into account the uncertainty of data. To this end, the problem is then reduced to a global energy optimization framework efficiently optimized with a min-cut approach. We use the approach for producing a large scale surface reconstruction algorithm by merging data from ubiquitous sources like airborne, terrestrial Lidar data, occupancy map and extra cues. Once the surface is computed, a solution is proposed for texturing the mesh.


INTRODUCTION
Surface reconstruction from point cloud is an important topic that has already been greatly studied.One of the reasons is the constantly increasing number of applications that use surface reconstruction: digital elevation model computation for flood simulation, 3D modelling, robot path planning, etc., and technical acquisitions: Lidar, images, etc.Despite of the important number of approaches proposed in the state of the art, the ability to handle the acquisition ubiquity of the data remain a challenging task.It is even more true in a large-scale context.It is within this context that (Nexa et al., 2015) has recently provided a benchmark that covers different relevant tasks as camera calibration and dense reconstruction on challenging data.This benchmark is decomposed in 3 scenes that contains thousands of images on both airborne and terrestrials camera coupled with millions of 3D points from Laser.In the context of data merging, (Fuhrmann and Goesele, 2011) proposes a method for merging depth maps with multiple scales, this method takes various depth maps at different scales and produces a piecewise surface.
Thanks to efficient optimization methods such as graph-cuts, global optimization formulation became very popular for producing watertight surfaces.The problem can be seen as a volumetric segmentation of a discretization of the space where the resulting surface is represented by the interface between inside and outside areas.It has been first introduced in (Snow et al., 2000) with a photometric approach.
In this paper, we propose an approach based on the Dempster Shafer theory that allows, with an arbitrary number and type of inputs, a large scale reconstruction of the space.We apply this model to the lidar point cloud reconstruction case.We also propose a formulation for texturing the segmentation based on extra information like laser reflectance.
Figure 2 shows the proposed scheme of the surface reconstruction in 2D.The input is a set of occupancy function that are merged with the Demptser Shafer theory.The final function is then segmented into a binary indoor/outdoor partition.In this case, each pixel is a leaf of the octree.Figure 1 shows the final 3D result of the algorithm.

SURFACE RECONSTRUCTION FORMULATION
The aim of the proposed method is to find, for a set of inputs I, a segmentation of the space as occupied or empty following a suitable model.Inputs are given by a set of mass functions mi defined by (see section 2.3) : where e, o and u are the occupancy masses for a 3D point P ∈ R 3 to be empty, occupied or unknown.The mass functions are merged to have a global mass function m(P ) is the final mass that gives, for a point P ∈ R 3 , the confidence to be empty, occupied or unknown for the set of mass function I.
The problem is then defined as a binary segmentation of the space regarding to m where the label set is L = {0, 1} where l = 1 represents the state occupied and l = 0 the state empty.The space is discretized with an octree, where V is the set of octree's leaves and F ⊂ V 2 the set of interface between adjacent leaves.We note lv the label of the leaf v ∈ V and lV = (lv)v∈V the labelling lv of all leaf of the octree V .
Finally, the goal is to find, for each leaf, a label lv that encodes the fact in being either occupied or empty for a given set of input mass functions mi.To this end, the problem is formulated as an energy minimization framework composed of two terms: the data term E data (lV ), which represents how a label is close to the global mass function m and the smoothness term Eprior(lV ), which pe- nalizes solutions with large surfaces.The final energy is : where λs is the weight balancing the prior and data terms.Now the two terms will be described in details.

Data term
The data term models how a label l fits to the global mass function m defined in equation 2. The confidence of a label l to be occupied is decomposed into 2 functions pe(l), po(l) defined as: Finally, the data term is the integral of the difference between the function pi(l) and the mass function mi for each label i ∈ (e, o) over the volume of the leaf, which gives:

Prior term
The prior term generally forces solutions with a small surface.This leads to the following term: where area(t1 ∩ t2) returns the area of the interface t1 ∩ t2.When lt 1 is equal to lt 2 , the score is equal to 0. Inversely, it is equal to 1 when lt 1 is different to lt 2 .

Mass function definition
All that remains is to define the function me, mo and mu for an input set of data.With lidar data, when the center of the sensor is known, the visibility prior tells that cells crossed by the beam formed by the sensor center and the 3D point have great confidence to be empty and those right after the 3D point have great confidence to be occupied.This idea is encoded here in the independent definition of a mass function for each input measurement, which will be merged into a single over function using DST. is the signed distance of the projected point P to Q.When r < 0, P is between O and Q, this is empty.Consequently, when |r| is important, the mass er is equal to 1 and the occupied mass or is equal to 0. Inversely, when r > 0, P is behind Q, this is occupied, then the mass er is equal to 0 and the occupied mass or is equal to 1.
where σn is the scale of the uncertainties related to the sensor and σt the scale of the scene thickness prior.
The mass of occupancy is important on the ray while decreasing as it moves away from the ray.Following this assumption, the occupancy function is represented as where σ θ is the scale of the angle uncertainty.The mass function of a 3D point for a ray is then defined by: If the normal − → nq at Q is known, it can be added as an extra parameter.r is then redefined as the orthogonal distance from P to the plane defined by the point Q and the normal − → nq.The figure 3 shows the mass function for a beam in 2D.
Figure 4: Example of fusion.First, four rays with an important uncertainty.The confidence at the intersection is much higher.Then, a fusion with both a strong confidence to be empty and occupied.The confidence at the intersection is equal to 0.5 for both input, the unknown label remains small.

Global mass : Dempster-Shafer theory
The DST is used to combine all inputs and gives the final mass m.Let m1 m2 two mass functions, the fusion can be computed with the DST as following: where K = o1 • e2 + e1 • o2 represents the conflict term.It means that when K is equal to 0, there is no conflict between inputs.
The combination rule is commutative and associative, enabling an order-independent aggregation of the measurements.At a location P for a set of mass functions mi ∈ I, the overall mass is computed following the equation 2. Figure 4 shows two examples of mass merging.

REFLECTANCE ESTIMATION FORMULATION
Extra information as reflectance can be used for texturing the resulting mesh.Contrary to the image intensity, there is no straightforward way to texture the mesh with the lidar reflectance.To tackle this problem, we propose to labelize each leaf of the octree with the reflectance of the data and use a global formulation approach in order to regularize the reflectance where no information is provided.Finally, the reflectance of a facet is the reflectance of leaves on the surface that are labelized as inside.
The problem is defined as a multi-label segmentation of the space regarding to the reflectance r of the data where the label set is L = {0, .., n}.The actual reflectance of our data is between −20 and 40.We thus do a linear mapping between the reflectance intensity and the label set.
We note lrv the reflectance label of the leaf v ∈ V and lrV = (lv)v∈V the labelling lrv of all leaves of the otree V .
Two terms are used: the data term E data ref l (lrV ), which represents how a label is close to the reflectance of a leaf r(v) and the smoothness term Eprior(lrV ), which penalizes solutions with a large reflectance difference in the neightbourhood.The energy is :

Data term
The data term is the difference between the reflectance label lr and the reflectance information in each leaf r.For each leaf v, the reflectance average r(v) of all points that remain in the same cell is computed.The data term is then : Where V * is the set of leaves where the reflectance r is defined.

Prior term
The prior term generally forces solutions with closes neighbors labels.This leads to the following term: Thank to the metric function on the quadratic term, this energy remains sub-modular and thus, can be optimized in polynomial time.
The algorithm is decomposed into four steps : First, the scene is segmented with the octree.Then for each leaf of the octree, the equation 5 is computed.The energy 3 is minimized with graphcut approach to compute the indoor/outdoor labeling.Finally, the reflectance of each leaf is computed and the solution regularized.

Octree
Thanks to the previous formulation, any 3-space partitioning could by used to discretize the aggregated mass function in order to yield a tractable combinatorial optimization problem.Ideally, this tessellation should be driven directly by the behaviour of the mass function itself, with lower densities where it is homogeneous and facets orthogonal to its gradients.For that, the space is discretized with an octree.For each 3D point, the octree is sub-divised until the maximum depht at the 3D point location is reached.The octree is also subdivided in the 8 neighbors of the point leaf in order to allows a good local estimation of the surface.

Score computation
Equation 5 requires to compute the integration of the mass function on each octree's leaf.The integral t |pi(lt) − m f (s)|ds can be approximated using MC integration as P ∈S where S is a set of |S| uniformly sampled points in the leaf v. Volume(v) is the volume of the leaf v.The mass function of each point P is computed with the equation 2.

Surface optimization
As mentioned in (Boros and Hammer, 2002), in our case where the quadratic function on the two labels is a metric, the equation 3 can always be solved globally in polynomial time in the binary case with a min-cut reduction.We use the graph-cut code introduced in (Boykov and Kolmogorov, 2004) to solve the binary problem.

Reflectance optimization
According to (Boykov et al., 2001), the equation 13 with a metric function on the prior term 15 is sub-modular and thus, the global minimum can be reached in polynomial time.One of the most efficient ways to solve this problem is the α-expansion algorithm first introduced in (Boykov et al., 2001).It is fast and converges in practice on a good local minimum which is guaranteed to have an energy at most twice the globally optimal energy.

RESULT
We tested the proposed method with lidar data where 3D positions and reflectance are available.Figure 5 shows the result on different scene.Each scene has a 128 × 128 × 128 meter bounding box.The deepest leaf has a size of 0.03m which leads to a 12 levels of depth octree.λr = λs = 0.02.We can see that where 3D data are avialable, the surface is composed of small facets and are well estimated.Areas that have not been scanned are regularized with bigger leaf because of the lack of data.Challenging objects like gate and wire are also well estimated thanks to the higher level of detail.The regularized reflectance allows a good visualisation of the current mesh.The volumetric estimation of the reflectance allows modification like carving without extra computation.

CONCLUSION
A octree based framework is proposed for the watertight surface reconstruction of ubiquitous data in the same scene.The problem is formalized as a global optimization problem that is efficiently solved with a graph-cut approach.Results on real data shows the efficiency of the proposed method on challenging scenes.

Figure 1 :
Figure 1: Result of the proposed methods : first the resulting surface, then textured with the regularized reflectance of the data.

Figure 2 :
Figure 2: 2D example of the main scheme.The captured scene (a).The function related to the space occupancy : from a database (b), from ground and airborne laser information (c) (Red is outside, green is inside).Merging of source (d).inside/outside segmentation (e).

Figure 3 :
Figure 3: Mass function in R 2 for a beam defined by the sensor center O and the 3D point Q.The parameters are: the incertitude on the angle σ θ , on the data σn and the thickness of the scene σt.
When P is close to O, or and er tends to 0.5.When r < −| − − → OQ|, r is behind O, this is unknown, then er = or = 0. Finally, behind Q, or is represented by decreasing function from 1 to 0 that models the thickness of the scene.The resulting mass functions when r > −| − − → OQ| are : r > 0 : er = 0.5 e

Figure 5 :
Figure 5: Result of the proposed method.Left column : the resulting mesh.Right column : the mesh textured with the regularized reflectance of the data.