POLYGONAL SHAPES DETECTION IN 3 D MODELS OF COMPLEX ARCHITECTURES

A sequential application of two global models defined on a variational framework is proposed for the detection of polygonal shapes in 3D models of complex architectures. As a first step, the procedure involves the use of the Mumford and Shah (1989) 1st-order variational model in dimension two (gridded height data are processed). In the Mumford-Shah model an auxiliary function detects the sharp changes, i.e., the discontinuities, of a piecewise smooth approximation of the data. The Mumford-Shah model requires the global minimization of a specific functional to simultaneously produce both the smooth approximation and its discontinuities. In the proposed procedure, the edges of the smooth approximation derived by a specific processing of the auxiliary function are then processed using the Blake and Zisserman (1987) 2nd-order variational model in dimension one (edges are processed in the plane). This second step permits to describe the edges of an object by means of piecewise almost-linear approximation of the input edges themselves and to detects sharp changes of the first-derivative of the edges so to detect corners. The Mumford-Shah variational model is used in two dimensions accepting the original data as primary input. The Blake-Zisserman variational model is used in one dimension for the refinement of the description of the edges. The selection among all the boundaries detected by the Mumford-Shah model of those that present a shape close to a polygon is performed by considering only those boundaries for which the Blake-Zisserman model identified discontinuities in their first derivative. The output of the procedure are hence shapes, coming from 3D geometric data, that can be considered as polygons. The application of the procedure is suitable for, but not limited to, the detection of objects such as foot-print of polygonal buildings, building facade boundaries or windows contours. The procedure is applied to a height model of the building of the Engineering Faculty of the University of Trento and to other data. The obtained results show the effectiveness of the variational methods in handling geometric data describing complex objects. The most original and innovative contributions of this work are: the consecutive and coordinated use of the proper variational models (of first and second order respectively) at first in two dimension and subsequently on the one dimensional sets arising in the first segmentation, the application to geometrical data that represent a quite complex architecture.


INTRODUCTION
The detection of polygonal shapes as, for example, building footprints, facade boundaries and window contours, can serve for the construction of vector models of architectures, in particular for essential polyhedral volumes with polygonal faces.Polygonal shapes can be selected from a set of shapes of various type detected by the processing of geometric data.A set of shapes can be produced by finding edges in the geometric data.
In this work a procedure for the detection of edges and the subsequent identification of polygonal shapes is presented.Two global models are exploited to accomplish the main goals of the procedure.
The procedure starts with the application of the Mumford-Shah variational model (Mumford and Shah, 1989) to height data for the detection of sharp changes (edges, discontinuities) of a piecewise smooth approximation of the data.The edges of such approximation of height data are then processed using the Blake-Zisserman variational model (Blake and Zisserman, 1987).The output of the Blake-Zisserman model is finally treated to identify those edges that can be approximates as piecewise almost-linear curves connected by corners, i.e., polygonal shapes.
The key idea behind the proposed procedure is deeply related to the characteristics of the same variational models involved.A concise description of the main characteristics of the models is given hereafter so to outline the basic elements that driven the definition of the procedure.Details about the models and their application will be given in the next two sections.
The Mumford-Shah model is an energy functional built up of three terms depending on two unknown quantities of very different kind.One unknown is a piecewise continuous function that must smoothly approximate the data almost everywhere in the data domain.The second unknown is the set of the curves of the data domain where the requirements on the continuity and on the smoothness of the approximating functions are not imposed, i.e. a set of discontinuities.The smoothness of the approximating function is controlled my a specific term of the first order involving the gradient of such function.The other two terms measure the proximity of the unknown approximation to the data and the length of the discontinuity set.The length of the discontinuity set needs to be controlled to avoid over-segmented approximations.The relative strength of the three terms is governed by specific weight parameters.To find the two unknowns one has to minimize the energy functional, globally over the entire data domain.
The Mumford-Shah model is applied to height data to obtain a smooth approximation of the data and a set of homogeneous regions, those bordered by the curves of the discontinuity set repre-senting jumps in the height data.Since buildings have, in general, a polygonal footprint, the idea is to select those regions that can be described as polygonal shapes, i.e., a set of linear segments jointed by corners.
The Mumford-Shah model can not be directly used for the recovery of corners or junctions in the curves of the discontinuity set because only the length of the discontinuity set is controlled and because the presence of corners and junctions does not affect the measure of the length of the set.A further process of the Mumford-Shah output is hence needed to explicitly detect corners and edges that can be approximated by a piecewise almostlinear curve, i.e,. a polygonal shapes.A variational model of the second order proposed by Blake and Zisserman (1987) has been involved to accomplish this second step of the procedure.The Blake-Zisserman model is an energy functional depending on a piecewise continuous function that smoothly approximates the data and on two discontinuity sets.In the model not only the approximating function but also its first derivative is requested to be piecewise continuous.Moreover, the smoothness of the function is controlled by a term of the second order involving the second derivative of the approximating function.These two facts leave as unknown the two discontinuity set, one for the discontinuity of the approximating function and one for the discontinuities of its first derivative.Also for the Blake-Zisserman model, the unknowns can be found minimizing the energy functional, globally over the entire data domain.
The Blake-Zisserman model is applied to the boundaries of the homogeneous regions detected by the application of the Mumford-Shah model to height data.The presence of a corner in a boundary will be detectable by observing the discontinuity set of the first derivative of the function that smoothly approximates the boundary.
For technical reasons, the computation of the solution of both variational models requires the discontinuity sets to be described by means of some specific auxiliary functions.The functions that represents the discontinuity sets have been mainly intended as down-weighting functions of the smoothing terms, but in fact in many applications they are a quite useful outcome of the models.
The described procedure has been implemented into a series of original computer programs that have been extensively tested on a variety of both simulated and real data, including optical images and geometric data (Vitti, 2012a,b;Zanetti and Vitti, 2013).After many application with land images and laboratory images this work extends the application range of the Mumford-Shah model and of the Blake-Zisserman model to data of a complex architecture.
The metric quality of the result is obviously related to the resolution and to the overall accuracy of the data.The method is anyway quite claver in filtering out the unavoidable noise.

TWO GLOBAL MODELS
The Mumford-Shah model and the Blake-Zisserman model will be described to highlights their main characteristics.Deeper treatments on the subject are available in the literature and will be pointed out at the need.

The Mumford-Shah model
The Mumford-Shah model in dimension two is: where g : Ω → R is the data, Ω ⊂ R 2 is the data domain, u ∈ C 1 (Ω \ K) is the piecewise smooth approximating function of the data, K is the discontinuity set and H 1 (K) is the 1-dimensional Hausdorff measure of the set.The first term in ( 1) is a global measure of the proximity of the solution u to the data, the smoothness of the function u is induced by the second term, the last term is the so called geometric term measuring the length of the unknown set K. The parameters λ and α controls the relative strength of the three terms.The problem here is to find the pair (u, K) that minimizes the Mumford-Shah energy functional.
To solve the problem in practice it is necessary to work on a functional different from (1).Ambrosio and Tortorelli (1992) proposed to solve the Mumford-Shah variational problem by means of the following functional: (2) The explanation of the need for a new functional and of its definition requires mathematical tools and developments that are well outside the scope of this work.We just highlight here the main characteristics of (2) that make it treatable in practice and that distinguish it from (1).All the terms of the functional F (u, s) are defined over the entire domain Ω, the smooth term is now scaled by a new auxiliary function s and the third term of (1) has been replaced by a functional of s.This last quadratic functional approximates the H 1 measure of the discontinuity set K exploiting the auxiliary function s which is defined on the bi-dimensional domain Ω.The function s has another specific role: to downweight the gradient of the approximating function u by going to 0 right were the gradient of u takes big values and remaining close to 1 almost everywhere on Ω.The parameter that appears in (2) is a convergence parameter.In fact it is a sequence of functionals like F (u, s) that is proved to approximate, according to a particular type of convergence criterion (Braides, 2002), the Mumford-Shah functional (when goes to zero, the minimizers of (2) converge to the minimizers of (1)).
To better understand the output of the Mumford-Shah model we highlight that the auxiliary function s has values close to 0 either where the approximating function u has very big gradient, i.e., where it is discontinuous, or where the approximating function u is not homogeneous with respect to the smoothness measure expressed by the second term of the functional (scaled by the parameter λ).In this work the function s will be used to identify parts of Ω with homogeneous values of u whose boundaries will be further processed to select those having a polygonal structure.

The Blake-Zisserman model
The Blake-Zisserman functional in dimension one is: where g : I → R is the data, I ⊂ R is the data domain, u ∈ C 2 (I \K1) and piecewise continuous in I \K0 is the approximating function of the data, K0 and K1 are the discontinuity sets and #(•) is the counting measure of a set.The first term in (3) is a global measures of the proximity of the solution u to the data, the smoothness of the function u is induced by the second term, the last two terms measure the size of the unknown sets K0 and K1.In dimension one the elements of the sets K0 and K1 are points.The parameters λ, α and β control the relative strength of the four terms.The problem here is to find the triplet (u, K0, K1) that minimizes the Blake-Zisserman energy functional.
For reasons similar to those requiring a sequence of approximating functionals to find the minimizers of the Mumford-Shah functional, also the Blake-Zisserman functional needs to be properly approximated in order to be used in practice.Ambrosio et al. (2001) proposed to solve the Blake-Zisserman variational problem by means of the following functional: In ( 4) the new auxiliary function z is introduced to approximate the discontinuity set K1 following the reasoning that lead the introduction of the approximating function s in (2).The function z is close to 0 on the discontinuities of the first derivative of the approximating function u and close to 1 almost everywhere on I.
In this work the function z will be used to identify sharp changes in the coordinates of the points lying on the boundary of the homogeneous regions detected using the Mumford-Shah model and to identify where the variation of point coordinates is almostlinear.Sharp changes in the point coordinates, where z values shrink to 0, are expected to correspond to corners.The function z remains close to 1 on the parts of I corresponding to straight segments.

APPLICATIONS
The applications presented hereafter have been realized using original software implementing the Mumford-Shah model in dimension two and the Blake-Zisserman model in dimension one.The source code is available at www.ing.unitn.it/~vittia/sw and licensed as Free Software.

Example 1
A first example illustrates the use of the Blake-Zisserman model for the detection of corners and straight segments on a line.A wall plan of small Church is processed.The plan has been obtained by slicing the 3D point clouds of the Church acquired using a terrestrial laser-scanner.
To apply the 1D BZ model to a plane curve (the boundary of a 2D flat object) we make use of the curvilinear coordinate on the curve.When dealing with discrete data, we work with the set of Cartesian coordinates of points on the plane (assumed to lie on a unknown continuous curve) and with a discrete implementation of the Blake-Zisserman model.The curvilinear coordinate of the points is approximated by the chordal coordinate c computed as the summation of the euclidean distances between successive points.The discrete Blake-Zisserman model is the applied twice: once to the coordinate pair (c, x) and once the the coordinate pair (c, y).

Example 3
In this example the Mumford-Shah model is applied to a highresolution (0.5mx0.5m)Digital Surface Model (DSM) of an area including a polygonal building of complex architecture.Again the boundaries of the homogeneous parts of the DSM detected using the plot of the Mumford-Shah auxiliary function s have been processed using the Blake-Zisserman model to identify the polygonal elements of the building.with a dead vertex.Those lines describes discontinuities of the function u that gently disappear and that can not be identified as boundaries.This fact is deeply related to the characteristics of the Mumford-Shah model that, right for this reason, is also known as a model for the detection of free discontinuity rather than of free boundaries.The word free refers to the fact that the set K is one of the unknowns involved in the model (De Giorgi, 1991;Ambrosio et al., 2000).Figure (11) shows in color the polygonal shapes identified using the Blake-Zisserman auxiliary function z obtained by the processing of the boundaries of the homoge-

DISCUSSIONS AND OUTLOOKS
The Mumford-Shah and Blake-Zisserman models produce an optimal segmentation of the data by means of the minimization of specific energy functionals.In practice, the main difference between the two models is in the feature of the smoothing term that is of the first order in (1) and of the second order in (3).Beside this difference the Blake-Zisserman model also permits the detection of discontinuities of the first derivative of the function that approximates the data.In treating geometric data the Mumford-Shah model can manifest an important limitation related to the feature of the smoothing term.In presence of strong variation of the data the smoothing therm, being of the first order, may lead to a over-segmented approximation of the data, i.e., to the appearance of spurious discontinuities.A proper choice of the  2) can however allow to process geometric data without the appearance of this kind of defects.The use of the Blake-Zisserman model with bi-dimensional geometric data is however feasible and may be more appropriate than the use of the Mumford-Shah model (Zanetti and Vitti, 2013).
The variational models used in this work were originally developed in the computer vision field and devoted to the processing of classical images.Nevertheless, the models have been proved to be very well suited for the treatment of geometric data.The 1D Blake-Zisserman the model has also been successfully applied to geodetic data of different kinds (A.Borghi, n.d.;B. Benciolini, 2009).In this work the Mumford-Shah model has been applied to gridded data.Changing the numerical scheme would make possible the application of the model to unstructured data (Bourdin and Chambolle, 2000;Zanetti and Bruzzone, n.d.).
An interesting extension of the procedure presented in this work may exploit a variational model proposed by Braides and March (2006) that extends the features of the Mumford-Shah to permit a direct detection of corners along the discontinuity set of the smoothing approximating function.

Figures ( 1
Figures (1) and (2) show a view of the point cloud and the wall plan of the Church respectively.The plot of the auxiliary function z produced by applying the Blake-Zisserman model to the red element of the wall plan of Figure (2) is given in Figure (3).The cyan dots of figure (2) have been placed on the wall plan

Figure 1 :
Figure 1: View of the point cloud of a small Church

Figure ( 4 -Figure 3 :
Figure (4)  shows the geometric data processed using the shows a detail of the same area of the geometric data (top) and of its smooth approximation (bottom).It is possible to note how the approximating function u is smoother that the data everywhere but along the borders of the window openings.The plot of the auxiliary function s of the facade produced by the Mumford-Shah model is given inFigure (6).Figure(7)  shows in green and orange the polygonal shapes identified using the auxiliary function z obtained by the application of Blake-Zisserman model to the boundaries of the homogeneous regions detected after the application of the Mumford-Shah model.

Figure ( 8
Figure (8) shows an aerial view of the building.Figures (9) and (10) give the plot of the Mumford-Shah auxiliary function u for the entire study area and a detail of the close surrounding of the building.It is possible to observe how the auxiliary function describes not only the discontinuities of the approximating function u but also the domain parts where the function u is not homogeneous (in light gray).In figure (10) there are some lines

Figure 6 :
Figure 6: Plot of the auxiliary function s produced by the Mumford-Shah model applied to the facade of Figure (4)

Figure 8 :
Figure 8: Aerial view of a complex building

Figure 10 :
Figure 10: Detail of the plot of the auxiliary function s in the close surrounding of the building of Figure (8)