State of research in automatic as-built modelling

Building Information Models (BIMs) are becoming the ofﬁcial standard in the construction industry for encoding, reusing, and exchanging information about structural assets. Automatically generating such representations for existing assets stirs up the interest of various industrial, academic, and governmental parties, as it is expected to have a high economic impact. The purpose of this paper is to provide a general overview of the as-built modelling process, with focus on the geometric modelling side. Relevant works from the Computer Vision, Geometry Processing, and Civil Engineering communities are presented and compared in terms of their potential to lead to automatic as-built modelling. Crown Copyright (cid:2) 2015 Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).


Introduction
Building Information Models (BIMs) are digital representations of facilities that encode all the relevant information about their life cycle from construction to demolition, including, but not limited to, 3D design drawings, schedule, material characteristics, costs, and safety specifications.Conceived using open standards 1 to facilitate knowledge sharing and interoperability between different stakeholders, BIMs gained a general acceptance within the construction industry, as they are expected to provide significant cost savings and improved productivity in construction projects.Whilst creating as-designed BIMs (i.e.BIMs generated in the design stage of a facility) is a straightforward process becoming increasingly common, generating as-built BIMs (i.e.BIMs that reflect a facility in its as-built conditions) is a challenging, but necessary process for facilities not equipped with an as-designed BIM and for facilities where the asbuilt conditions differ from the as-designed BIM.The focus of this paper is set on the as-built BIM (AB BIM) generation, considering the cases when an as-designed BIM (AD BIM) is present or not.
Creating an AB BIM requires two major steps: data collection, to capture the as-built conditions, and data modelling, to generate compact, but rich representations readily understandable by other processes.Although historically, (AD) BIMs were introduced in the 70's, gaps in technology (in data collection and storage) and knowledge (in data modelling) prevented the construction industry from equipping facilities with AB BIMs; to this date very few AB BIMs exist.Auspiciously, the 3D reconstruction field, related to Computer Vision and/or 3D laser scanning techniques, filled the technology gap, offering off-the-shelf tools for generating 3D models of scenes.Consisting, basically, of a set of 3D points endowed with 3D Cartesian coordinates and possibly colour information, these models that will be denoted from now on as point clouds, are useful for visualisation or augmented reality purposes [90,91,1,94].However, the data modelling side continues to be deficient, and the problem of converting the raw point cloud into a semantically rich BIM model is far from being settled.The commercial and academic tools available at this point to perform this conversion require extensive human intervention, making them expensive [23] and error-prone [68].Given the expected economic impact, automatically generating AB BIMs from point clouds is a key objective for the industrial, academic, and governmental parties involved in the Architectural/Engineering/Construction and Facility Management industry (AEC/FM).
This paper provides a general overview of the as-built modelling process, with focus on the data modelling side, and presents relevant works from different research communities, discussing their potential of being used to automatically generate AB BIMs.

Relation to other fields
As-built modelling has a large overlap with the Geometry Processing field, but the input is restricted to point clouds of infrastructure assets, and not any generic object, simplifying the problem to rigid shape analysis [33,70,107].However, the huge amount of data to be processed (more than one hundred million points in some cases) and the high number of elements in the scene require efficient methods, and therefore generic Geometry Processing tools can not be straightforwardly applied.Considering the data to be modelled, as-built modelling acts like a zoom in into the urban modelling problem.Naturally, the approaches proposed in the two areas bear many resemblances.However, as-built modelling must operate at a finer level of detail, requiring more complex elements to be detected and recognised.But even so, due to the prior information available on the shape and distribution of the scene elements, the object recognition within as-built modelling is a simpler problem than the generic object recognition encountered in Computer Vision.Still, the common lack of distinct features and the non-discriminant shape of the elements typically encountered in facility scenes, generate ambiguities and often cause Computer Vision tools to fail [120].Being closely related to several research fields, good interdisciplinary communication and joint research efforts are essential for the state-of-the-art in asbuilt modelling to progress, and it is our hope that this paper contributes in this direction.

Organisation of the paper
The paper starts off with an extended statement of the as-built modelling problem (Section 2).The subsequent sections address the key steps of as-built modelling.Section 3 briefly discusses the obtainment of the point cloud representing the input of our problem.Since the point cloud is commonly corrupted by noise and outliers, the cleaning step which usually precedes the actual modelling is described in Section 4. A concise reminder of the different representations involved in as-built modelling is given in Section 5. Depending on the availability of an AD BIM, different approaches exist for as-built modelling without AD BIM, covered in Section 6, or with AD BIM; see Section 7. Whilst these approaches cover generically the as-built modelling process for buildings and infrastructure assets (e.g.bridges), particular methods are used to address mechanical, electrical, plumbing (MEP), and plant systems (see Section 8).Section 9 concludes the paper.

Problem statement
Given as input a raw point cloud, the goal of the as-built modelling process is to generate a semantically rich 3D model of the facility, composed of objects characterised by geometry, relations, and attributes [60].Ideally, the result would be an AB BIM file, encoded using a standard language, e.g.IFC in the case of buildings.
Preliminarily, it is worth noting that the as-built modelling process is limited by several objective factors, and thus even the best possible as-built modelling method cannot be expected to output an AB BIM as rich as an AD BIM.The following aspects, and possibly others, induce objective limitations in the as-built modelling process: AD BIMs contain semantic information pertaining to the designer's high level knowledge, but which cannot be (physically) inferred from a digital model, e.g.specifications, costs [23].The level of detail of the resulting AB BIM is limited by practical aspects related to data collection.Even if presumably the technology used for data collection is able to capture fine structures like outlets, or fasteners, the added-value of modelling these small elements does not justify the time and costs needed to meticulously collect and model the data [58].Non-visible and partially occluded building elements cannot be captured during data collection, hence they will not appear in the final AB BIM; e.g. the reinforcement inside a concrete column [82].Note that devices to capture the non-visible elements exist (e.g.non-destructive testing technologies like ground penetrating radars), but the research on automatically interpreting data coming from such devices is in very early stages, and no successful procedure is available to date, hence these devices will not be considered in this study.
Taking these limitations into account, we define the desired output of the as-built modelling process as a working BIM model, which represents a reduced version of the complete BIM, and encodes visible building and spatial elements, together with their nesting and connectivity relationships.For the sake of example, we can further elaborate this definition by considering the case of IFC for as-built modelling of buildings; equivalent reasonings stand for MEP modelling and bridges.
Using an object-based inheritance hierarchy, IFC defines three abstract concepts: object definitions, relationships, and property sets, whose specialised sub-classes are used to define a BIM model [3].The working AB BIM is expected to contain visible objects encoded in IFC as instances of the IfcBuildingElement class (beam, chimney, column, door, wall etc.), and instances of Ifc-SpatialStructureElement (building, building storey, site), but not IfcElementComponent (fastener, reinforcing mesh), or Ifc-DistributionElement; building elements also include information about material properties.Relation-wise, instances belonging to (at least) IfcRelDecomposes and IfcRelConnects need to be modelled, since the proximity and nesting relations of building elements can be captured from the as-built conditions, whereas higher-level semantic relations like IfcRelAssociatesConstraint which links a certain IfcConstraint object to a building element cannot be inferred.
The key task in generating the AB BIM described above is the recognition of the predefined building elements and their relationships 2 .Depending on the approach, different auxiliary tasks can be used to facilitate this process: Geometric primitive detection: report if and where predefined simple geometric shapes appear in the given point cloud, e.g.detect planar patches.Point cloud clustering: given predefined criteria, e.g.planarity measure, cluster the input points to obtain segments of points with similar descriptors.Shape fitting: given a subset of the original point cloud and a predefined model (e.g. a cylinder), find the parameters of the model.Classification: given the segmented point cloud obtained above, assign to each segment a unique building element label.
Note that when an AD BIM is available, the sought output remains the same, but the tasks undertaken to achieve it shift their focus from detection and recognition to one-to-one matching and verification.
The general as-built modelling process, from data collection to BIM generation, is summarised in Fig. 1, and comprises the following steps: point cloud generation, point cloud pre-processing, and as-built modelling per se.For self-containment, we first discuss briefly the former two steps, and then we address extensively the as-built modelling step.The aim is to give a critical analysis of the state of research, comparing the existing works in terms of the output they provide, and how close or far they are from the desired output.This will result in positioning the existing works in the space of possible outputs, governed by the objective limitations mentioned above and the intrinsic limitations of each method, as depicted in Fig. 2.

Generation of 3D point clouds
The input of the as-built modelling process is a 3D point cloud, usually obtained through a 3D reconstruction method, which takes its data either from imaging (camera) systems [7,45,52,51], or from (time-of-flight, phase-shift) laser scanners [21,89].Regardless of the data source, the core of the 3D reconstruction process consists of a registration procedure aiming to bring different sets of data into the same coordinate system.The differences, however, reside in the deformations induced by each system, and the features available to recover them.Both cameras and laser-scanners need to be moved around the scene to ensure a good coverage, hence in both cases, rigid deformations (translation, rotation) must be considered.In addition, losing the third dimension during the imaging process, the image-based systems need to first infer the depth by triangulating corresponding points belonging to images taken from different viewpoints, under a certain assumption for the transformation induced by the internal camera model; most commonly the perspective transformation is used, in tandem with the pinhole camera model [59].This process is guided by appearance cues (e.g.local interest points [83,111]).By comparison, registering laser scans is less involved and less error-prone.However, in this case appearance cues might not be available, and the registration procedure relies mostly on geometric information.More details on the registration problem will be given in Section 7, in the context of as-built modelling with as-designed BIM.An in-depth survey of 3D reconstruction techniques can be found in [85].
The two techniques have been extensively discussed and compared in the Civil Engineering community, contrasting the low cost and time-efficiency of the imaging systems, with the pricey higher accuracy of the laser-scanning systems [25].With the advent of RGBD cameras, the depth and appearance sources become two registered modalities, used in complementarity to obtain accurate 3D point clouds [101,56,63].

3D point cloud preprocessing
The measurement accuracy of the point cloud data has a crucial influence on the accuracy of the as-built modelling process.Preprocessing the input data to remove outliers, reduce noise, and compensate for missing data, has become a prerequisite in as-built modelling.In the case of very large point clouds, an initial segmentation into smaller parts can be performed, which are processed in parallel [120,92].The alternative is to downsample the point cloud, e.g. through voxelisation: partition the point cloud according to a 3D grid, and replace all the points within a cell with the centroid of the cell.Note that voxelisation can also be used to gain adjacency information between the points [112,128], useful for density estimation [116], or later during modelling [47].For memory efficiency, tree structures are used to store the voxelised point cloud [127,92,128].

Outlier removal
Most of the approaches define some local outlier factor based on local statistics, e.g.local density [99], distance to nearest neighbours [71], or eigenvalues of the local covariance matrix [114]; see [88] for a more complete list.Outliers are identified either by directly applying a hard threshold [71], or by defining a global distribution -usually a normal distribution is assumed -and by identifying points diverging from this distribution [99].The method proposed in [88] compares the local sampling density of a point to the average local sampling density of its neighbours, and identifies outliers in a parameterless setup.Some authors [115,69] further distinguish sparse outliers from temporal artifact outliers, e.g.artifacts due to movement in the scene.The latter are pruned in [69] by identifying the apparently coherent clusters that pass the local statistics test, but that appear in only one or few scans, and other scans that cover them see through them.

Noise filtering
Similar to outlier removal, noise filtering techniques study the local neighbourhood of each point and perform a smoothing operation.The main issue is to avoid over-smoothing the sharp local features.The kernel density estimator proposed in [99] performs poorly near edges and sharp features.Similarly, the iterative down-sampling up-sampling strategy proposed in [81] is efficient in filling holes [38,96], but it has difficulties in recovering sharp features.To prevent this behaviour, the key idea is to perform anisotropic filtering.The filtering proposed in [77] uses mean curvature flows to perform anisotropic filtering of point clouds, but the high complexity makes it prohibitive for large point clouds.A more efficient procedure was introduced in [115], where the authors perform an anisotropic filtering on the normals of the points computed using a new technique; eventually, the positions of the points are updated accordingly.Some works focus on designing algorithms robust to noise, bypassing the filtering step in order to avoid the potential loss of information [8,6].
To conclude this section, it is worth noting that point cloud preprocessing is a very active research field, and more importantly from a practical point of view, numerous tools are available from open-source libraries and software, e.g.PCL [98], Meshlab [4].

Representations used in as-built modelling
Designed to encode a holistic view of a facility, BIM is an objectoriented representation, built upon a volumetric or solid representation of the scene (Fig. 3).Constructive Solid Geometry (CSG) [120] and Boundary Representation (BRep) [84] are the most common 3D representations.Conciseness and simplicity are the attractive features of the former, whereas flexibility and high representation power characterise the latter.
These models, in turn, rely on geometric representations.To reason about the geometry of a scene, a number of geometric representations are available, and they differ in terms of completeness, compactness, and uniqueness [26].The choice of a particular representation depends on the complexity of the scene elements and the application's needs [11].
Complete representations, as opposed to discriminatory representations [28], fully describe the geometry of an object, and they can be explicit or implicit; explicit representations can be further subdivided into parametric, and non-parametric.Simple shapes like lines, circles, planes, spheres, cylinders, or general quadrics, admit explicit parametric and implicit representations, which are extensively used in the first steps of as-built modelling due to their compactness; the low number of parameters allows relatively efficient detection and fitting methods.For example, using appropriate parameterisations, Hough transform can be used to detect planar and 3D shapes (planes, spheres, generalised cylinders), or different fitting methods take advantage of the fact that implicit representations use the zeros of a function to define a shape, which can be conveniently integrated in a least squares fitting formulation [125,54].For more complex shapes, explicit parametric representations are still available (e.g.Bézier curves, B-spline, NURBS) but they are used mostly as design tools.Since their control points can neither be easily inferred from given data, nor are they unique, these representations are rarely used in shape analysis.Finally, for highly complex shapes that do not admit parametric models, nonparametric representations are used, e.g.polygonal meshes.Despite their approximation power, the lack of compactness of these representations limits their use especially when dealing with large point clouds.Hence, using a combination of parametric and non-parametric representations is a reasonable tradeoff [75,76] when a complete representation is required, e.g. for visualisation purposes.
For other steps involved in as-built modelling, the discriminative power of a representation can prevail over completeness.This is the case during object recognition, where the representation is expected to encode, as compactly as possible, the distinctiveness of an object.To this end, local or global descriptors can be used.Local representations are robust to occlusions and clutter, compared to global descriptors, but are less compact.Simple parametric shapes (cylinders, planes) can be represented by compact feature vectors encoding moments, and other local statistics [129].For non-parametric shapes, descriptors with more discriminant power are available: spin images, fast point feature histograms, shape contexts, to mention only a few; we refer the reader to [55] for a more in-depth general study, and to [10] for an evaluation of their discriminatory power when applied for geometric primitives classification.

As-built modelling without AD BIM
Generating the full geometric model of a facility without an AD BIM is an intricate problem, mainly because the point cloud to be processed is generally very large, and it contains a high number of infrastructure elements that need to be identified (e.g.walls, floors, windows), possibly in the presence of clutter.Detecting and recognising predefined infrastructure elements is the core of as-built modelling.However, of equal importance is the modelling of the relationships between elements, which can contribute in obtaining a coherent global geometric representation.
Depending on the criterion used to define this global coherency, existing approaches in geometric modelling can be classified into global optimisation approaches and local heuristics.The former are generally model-based, and reason about the scene as a whole, trying to find a global geometric interpretation that is optimal in terms of maximum a posteriori estimation [40,123,76] or energy minimisation [62].The relationships between the elements can be used as an active ingredient of the global model, allowing to inject prior information about the scene layout [40,76], e.g. common relationships are ''adjacent walls are perpendicular to each other'', ''walls and floors are perpendicular'', ''doors are contained within walls'', etc. Opposed to this holistic approach, which can be quite complex model-wise and expensive computation-wise, the local heuristic approaches use bottom-up reasonings, in which the elements of the scene are treated independently and possibly the prior information about the relationships between the elements is used as a rejection criterion for potentially erroneous local configurations [117,93].
In both categories, there exist few works that describe full modelling procedures for point clouds, pertaining to Computer Vision, Geometry Processing, or Civil Engineering communities.But other works that address either particular parts of the as-built modelling, or the geometric modelling problem in general, are also of interest, hence they will be included as well in our discussion.However, we focus on automatic methods or works that have potential in the process of automatically obtaining BIM models; we do not detail works that depend on some form of user interaction [102,80]; see e.g.[85] for a presentation of these works.

Global optimisation approaches
Dick et al. proposed, through a series of papers [39,41,40], an automatic framework for the geometric modelling of point clouds of buildings, whose output is a volumetric representation of the building, with objects labelled as ''wall'', ''door'', ''window'', ''column'' etc.Their approach uses a Bayesian model, whose overall prior distribution is obtained by modelling the prior distribution of each building element represented through simple parametric shapes, together with their relationships.At modelling time, a Markov Chain Monte Carlo (MCMC) algorithm [53] is used to find the parameters of the building elements that maximise the a posteriori likelihood.A similar overall reasoning is used by Lafarge et al. for urban modelling [73].The main strength of these works resides in the fact that they lead directly to a semantic interpretation of the scene objects, together with their relationships.
A natural possibility to exploit the prior information on relationships between elements is to use graphical models trained using labelled data, i.e. a segmented point cloud with elements labelled as ''wall'', ''door'', ''clutter''.Based on this idea and inspired from [9], the authors of [123] train a Conditional Random Field to model the relationships between planar patches extracted from point clouds of buildings.Explicitly modelling the relationships between the elements of the scene improves the accuracy of the overall geometric modelling process.A different formulation based on graphical models is used in [76] for urban modelling.The authors start by semantically segmenting the point cloud into four different classes (''building'', ''vegetation'', ''ground'', and ''clutter'') and minimise an energy functional to find an optimal configuration.The outcome is a combination of parametric shapes (planes, cylinders, spheres or cones) to describe regular roof sections, and mesh patches for irregular roofs.The compactness and the representation power of this approach make it appealing for modelling buildings in the presence of clutter.In a similar reasoning, but targeting indoor scenes, the authors of [101] first extract a coarse scene structure by semantic segmentation of pixels into ''ground'', ''walls&ceiling'', ''furniture'', ''props'', then use integer programming to obtain a refined optimal configuration of the scene, while considering support relationships between the elements of the scene.Again targeting indoor scenes, Bao et al. [13] distinguish between (planar) surfaces that belong to the room layout and those that belong to objects in the room, by using cues from both the point cloud and the images, in a cost function minimisation problem that seeks to estimate the room's layout.
An integrated framework to model a scene as a set of interrelated networks of labels, functionalities, and descriptors, is proposed in [124].Although it allows to reason about a scene at a very high semantic level, the framework has reduced flexibility in modelling the interactions between the networks; hence the contribution of this work is mostly at a theoretical level.
Although not specifically addressing the problem of semantic point cloud modelling, the works on geometric multi-model fitting proposed in [62,119] are relevant for as-built modelling.In both approaches, the authors formulate a global energy minimisation problem, that considers the multi-model geometric fitting as an optimal labelling problem, solved using graph cuts [22].

Local heuristics
Sacrificing the notion of statistical optimality, a large number of works related to as-built modelling consider local heuristics approaches, in order to obtain more efficient algorithms.Generally, these methods take advantage of the fact that the geometric model of a building can be fairly decomposed into simple parametric surfaces.The typical process starts by segmenting or clustering the point cloud using arbitrary criteria, and then fits different parametric shapes on the segments obtained.Strong cues specific to architecture scenes like orthogonality or symmetry, are typically used to guide the heuristic search, and discard abnormal configurations.Prior information about the layout of the buildings are used to attach semantic information to the detected elements.
More specifically, the authors of [37] assume that walls are orthogonal to the ground, and apply space sweep algorithm [36] to detect them.The same algorithm is used in [27] to recognise the planar structures of a room.In [117], the authors first automatically infer a coarse model of the scene, by detecting the planes associated to its principal directions.This coarse model guides the detection of more complex polyhedral objects, representing doors and windows.
In a series of papers, Stamos and Allen [105,104] seek to build textured geometric models from range scans and unregistered images.The two data sources are processed in parallel to extract 3D lines from range scans and 2D lines from images, which are then matched to obtain images-scans registration, needed to texture the 3D model.Using laser scans data, Wang and Cho detect boundaries as line segments and use them to identify roofs, windows, doors, and walls [32,34].
Closely related to the model-based approaches mentioned in the previous class, the method introduced in [113] for urban modelling, uses graphs to encode the connectivity of planar elements, and recognise predefined configurations by subgraph matching.In [93], the authors extract planar patches, which are classified into semantic elements (''wall'', ''door'', ''window'' etc.) using hardcoded prior knowledge.The result is a polyhedral model of the building.A similar approach is used in [5], where parametric primitives are detected using Hough transform, and described through their geometric characteristics (area, relative scale, planarity score etc.), which are used to identify openings' type and distinguish them from clutter by training an SVM classifier.In [122], planar patches previously extracted using a region growing algorithm are classified using a stacked learning approach and contextual features.Similar to [5], openings are distinguished from occluded regions on a wall surface using an SVM classifier.In [129], the authors first detect planar surfaces and quadrics, which are then classified using multi-class AdaBoosted decision trees [132].
Targeting scenes containing only planar surfaces, Xiao and Furukawa [120] generate a textured CSG representation of a point cloud by processing 2D horizontal layers to get room layout hypotheses, which are then merged to obtain the 3D model using regularity constraints about the structure of the building.The authors of [68] adopt a robust approach based on RANSAC to detect walls as planar patches bounded by 3D lines, and then identify openings within walls.
In a more recent work [92], Poullis presents a full bottom-up framework for automatic modelling of urban point clouds, without any prior constraint on the elements types or their relationships.A hierarchical clustering through greedy region growing allows to segment the point cloud into surfaces, whose boundaries are determined using a graph-cut energy minimisation.The results on very large point clouds are impressive, proving that even simple local statistical reasoning can lead to robust techniques for point cloud segmentation and extraction of parametric primitives.

Auxiliary heuristics
This section describes heuristic geometric modelling works that do not provide a semantic interpretation of the scene, but which could be used as building blocks of the as-built modelling process.Some authors addressed the problem of obtaining more compact representations of point clouds, while ignoring the semantic modelling.In this line of works, the authors of [74,75] detect parametric primitives (planes, cylinders, spheres) that replace the regular parts of the mesh, whereas the highly detailed parts of the mesh are kept as is.This strategy reduces the memory requirements, while preserving the distinctive features of the building.In civil infrastructure, primitive detection was used as the basis of different frameworks for tasks involved with operation and maintenance [70,72,107].
In [130], the authors address the planar patch extraction problem, and apply sparse subspace learning to cluster the points into linear subspaces [46], and then robustly fit planes [109] to each detected segment.In the same line of works, many versions of region growing algorithms have been proposed to extract planar patches from point clouds, by first clustering the points using some local measurements, and then fitting planes on the resulted clusters [57,86,61].
To fit higher-order parametric primitives to point cloud data, algebraic and iterative methods exist.Admitting a closed form solution, algebraic methods are efficient [129], but can have strong bias on incomplete data [48].To obtain more accurate estimations, iterative methods are used which minimise a geometric error or an approximation of it [84,125].The method proposed in [42] fits NURBS to pre-segmented point cloud parts, offering shape freedom and accounting for missing data.The point cloud segments can be obtained using a context free segmentation method, as the one described in [50], which employs a region growing algorithm based only on geometric properties.
Targeting the problem of robustly fitting multiple different geometric models to point cloud data, Schnabel et al. propose a RAN-SAC version based on a new sampling strategy together with an early termination scheme that provides a significant speedup [100].Another approach for robustly fitting multiple geometric models, J-linkage, is proposed in [108]: each point is represented by its preference set, defined as the set of models that are satisfied by the point within a tolerance.By clustering points with similar preference sets, the point cloud is segmented and the underlying models retrieved.In the same line of works, Nurunnabi et al. [87] proposed the Diagnostic Robust PCA (DRPCA) to locally fit surfaces, obtaining improved performance compared to other methods based on PCA, MSAC and RANSAC; it accurately fits planes in the presence of outliers and calculates the local surface normal.

Overall analysis
The high diversity of the works presented in this section in terms of application field, methodology, and goals, makes it difficult to conclude with an objective comparative analysis.Considering the problem statement enunciated in Section 2, we will position some of the main works discussed above in the output space of as-built modelling; the result3 is depicted in Fig. 4. It can be readily observed that the majority of works concentrate on modelling planar surfaces along with their relationships.This is due, on the one hand, to the high frequency of planar elements encountered in building models, and on the other hand, to the reduced complexity of the problem; as soon as higher-order primitives are included in the analysis, non-trivial model selection issues occur.It is encouraging that few of the existing works reach volumetric representations and model inner relationships, getting close to the desired target.Among them, [129,5] are computationally more efficient.The missing link appears to be the material modelling, but since separate works on material modelling exist [24,133,43], including this step should be straightforward.

As-built modelling with AD BIM
For facilities equipped with AD BIMs, the expected output of asbuilt modelling is an AB BIM that accurately reflects the as-built conditions, which might diverge from the original design.However, the means to obtain the AB BIM in this case differ from the previous section, because the AD BIM can be actively used to guide the modelling process.The interest in the as-built modelling process with AD BIM is twofold: to check that the as-built conditions correspond to the as-designed conditions (geometry checker), and/ or to update the existing BIM to reflect the as-built conditions.This section describes works that can achieve these tasks in a fully automatic manner; other works that depend on some form of user input exist, but they are not described here in detail [16,17].
The core of the as-built modelling with AD BIM is the registration of the point cloud sampled from the AD BIM, with the point cloud obtained from 3D reconstruction.Once the registration is performed, the recognition of the objects contained in the AD BIM can be carried out to eventually identify missing elements and deviations from the model.The accuracy of the modelling depends on the accuracy of the registration.
Iterative Closest Point (ICP) [15] and its variants [31,121] are among the most common registration methods.ICP searches for the best rigid transformation (rotation, translation) that aligns the point cloud data with the model, by iteratively alternating between (i) a stage that computes point correspondences between model points and data points given a transformation, and (ii) a stage that updates the current transformation given the correspondences.The registered models are not limited to point sets, but also include polylines, parametric curves, implicit curves, triangles, parametric surfaces, and implicit surfaces [15].The convergence to (at least) a local minimum is guaranteed, and accurate alignment is obtained providing that the initialisation is good enough; with poor initialisation, the method gets stuck at a local minimum far from the global minimum.The speed and robustness of the ICP method have been studied extensively [67,97].
To avoid local minima, it is common to perform the registration in two steps [8]: (1) Coarse registration, to obtain a rough initial alignment, and (2) Fine registration, based on ICP optimisation to refine the solution.
The coarse registration is generally performed by detecting and matching local features within geometric hashing schemes [118] or RANSAC-based approaches [30].The local features can be any combination of points, lines, curves, and surfaces [49,64,79,106,126].The approach proposed by Johnson and Hebert [66] computes Spin Image local features in a dense manner, avoiding the error-prone feature detection step.Other approaches rely directly on points, bypassing the feature matching completely; e.g.Aiger et al. proposed a randomised approach based on geometric invariants of sets of points [8].The performance of the methods mentioned here is highly dependent on the noise level, percentage of outliers, and overlap between the two point clouds.Hence the choice of the coarse registration method is data driven.
The fine registration is often completed using ICP variants [15,31].Other methods include free form surface registration [35] using angular invariant features [65], or modified ICP methods such as the dual interpolating point-to-surface method [121].The ICP version proposed by Fitzgibbon [47] does not follow the two traditional steps (i,ii) mentioned above; instead it proposes a formulation that includes the search for correspondences into the optimisation problem.Recently, Zollhöfer et al. [134] used this for-mulation, initialised using the PatchMatch algorithm [14], obtaining state-of-the-art results on real-time registration of depth images.
Once the registration is completed, the two point clouds are compared to determine which points are matching between the two data sets, and thereby determining which objects in the BIM are present in the scan data [20].Developing this approach, Turkan et al. incorporated 4D CAD models, which include schedule, for the purpose of automated progress tracking [110].

Overall analysis
Using the AD BIM to guide the modelling process, the object recognition is less error-prone compared to the previous section.The main difficulty is to perform an accurate registation between the AD BIM and the scan, in a computationally efficient manner.However, if large deviations exist between the as-built conditions and the original design, the problem of as-built modelling with AD BIM encounters similar difficulties as the as-built modelling without AD BIM.

MEP modelling
The 3D reconstruction and as-built modelling of industrial infrastructure to obtain accurate information about the built status of the constituent parts is crucial for maintenance purposes.The key difference between MEP as-built modelling and other civil infrastructure modelling resides in the shapes and connections of the elements composing the MEP infrastructure; specifically, generalised cylinders interconnected by elbows and T-sections, represent the elements commonly encountered.The relative simplicity of MEP related infrastructure opens up various solutions for their as-built modelling.
Similar to the approaches presented in Sections 6 and 7 for generic facilities, the as-built modelling of industrial facilities and MEP components can be carried out taking into account already existing BIMs or without prior BIM.Recent research attempts for MEP reconstruction and as-built modelling are discussed in this section.Most of these approaches use local heuristics.

MEP modelling without as-designed BIM
Pipe (spools) detection in point clouds is based on specific geometric features, e.g.curvature.Son et al. [103] extract pipelines by segmenting the point clouds into local patches using a region growing procedure based on the normal vectors of the points.The curvature at randomly selected points is then extracted, and patches with similar curvatures potentially belonging to the same pipe are merged, and the pipe spool is then fully fitted.Their approach is capable of modelling the elbows and connections.However, occlusions in the point cloud lead to an oversegmentation of the pipe spools.
Another distinctive feature commonly used in pipe modelling is the geometric skeleton [12,29,131].Since the pipe sections are extruded along an axis, skeleton extraction can be employed to fully identify the line of extrusion from laser scanned pipe spools and plants [78].Once the skeleton is extracted, the pipe spools are modelled by calculating the diameter and modelling the T-sections and elbows.The skeleton-based reconstruction for pipelines was found to be sufficiently accurate.However, the skeleton extraction is computationally intensive.
The Hough transform [44] can be employed to detect circular pipe sections.However, the limitations of the 3D Hough transform, discussed thoroughly in [95], make it inefficient in civil infrastructure modelling.Recently, [6] used a 2D-based Hough transform by projecting the points into orthogonal slices (planes) for detecting pipe sections.The 2D circular sections are then consolidated in 3D to obtain the full as-built model of the pipe spools.Although their approach overcomes the time ineffectiveness issue of the 3D Hough transform, it is limited to the objects parallel to the orthogonal axes and therefore incapable of modelling inclined pipe spools and adjunction regions.

MEP modelling with as-designed BIM
Existing BIMs can be employed to leverage the components detection and recognition processes.Bosché et al. [19] showed the possibility of tracking the built status of MEP components using existing BIMs, generated for example with the method proposed in [20].Recently, this framework was also integrated with an automatic method based on Hough transform [6], resulting in a significantly accurate reconstruction that facilitates the assessment of the as-built status of MEP infrastructure [18].Through this work, the authors actually challenge the paradigm that neither the AD BIM nor the 3D scan is the ground truth, but rather, a combination of these data is required for proper modelling of MEP installations [18].

Discussion
This paper gives an overview of the as-built modelling process, by presenting various research works from different research communities (Computer Vision, Geometry Processing, Civil Engineering) that are currently used, or have the potential of being used, for successfully solving the challenging task of automatic as-built BIM generation for infrastructure.Significant progress has been reported in the last years in this direction.While obtaining complete BIMs can engender high (unjustified) costs in practice, we believe that automatically generating working BIMs is achievable.For the future, the focus should be put on consolidating and integrating the existing techniques, along with developing new methods for object recognition.Joint research efforts within interdisciplinary projects can lead to accurate as-built BIM generation, producing a high impact in the construction industry.

Fig. 4 .
Fig.4.Existing works represented in the output space of as-built modelling.