DETECTING AND UPDATING CHANGES IN LIDAR POINT CLOUDS FOR AUTOMATIC 3 D URBAN CARTOGRAPHY

This work presents a method that automatically detects, analyses and then updates changes in LiDAR point clouds for accurate 3D urban cartography. In the proposed method, the 3D point cloud obtained in each passage is first classified into 2 main object classes: Permanent and Temporary. The Temporary objects are then removed from the 3D point cloud to leave behind a perforated 3D point cloud of the urban scene. These perforated 3D point clouds obtained from different passages (in the same place) at different days and times are then matched together to complete the 3D urban landscape by incremental updating. Different natural or man-made changes occurring in the urban landscape over this period of time are detected and analyzed using cognitive functions of similarity and the resulting 3D cartography is progressively modified and updated accordingly. The results, evaluated on real data using different standard evaluation metrics, not only demonstrate the efficacy of the proposed method but also shows that this method is easily applicable and well scalable, making it suitable for handling large urban scenes.


INTRODUCTION
In this paper we present a new method for 3D urban cartography that automatically detects different man-made or natural changes occurring in urban environment and then effectively incorporates them in the resulting 3D point cloud of the cartography by incremental updating.Automatic detection of changes in urban environment for updating 3D urban models, cartography and maps recently become a hot topic in the scientific community as it continues to offer several challenges (Champion and Jurgen, 2009).Most of the proposed techniques detect changes in the urban environment from airborne data using DSMs (Digital Surface Models) i.e. (Vu et al., 2004).Vögtle and Steinle (Vögtle and Steinle, 2004) propose a methodology for detecting changes in urban areas following disastrous events.Instead of solely computing the difference between the laser-based DSMs, a region growing segmentation procedure is used to separate the objects and detect the buildings; only then, an object-based comparison is applied.But this method remains susceptible to misclassifications.Bouziani et al. (Bouziani et al., 2010) presented a knowledge-based change detection method for the detection of demolished and new buildings from very high resolution satellite images.Different object properties, including possible transitions and contextual relationships between object classes, were taken into account.Map data were used to determine processing parameters and to learn object properties.Unlike these methods, in our work we handle the 3D point cloud at 3D grid level for change detection, instead of object level, making it more robust.Most of the work using terrestrial laser scans focuses on deformation analysis for designated objects.Change is detected by subtraction of a re-sampled set of the data (Schäfer et al., 2004), or adjustment to surface models like planes and cylinders (Van Gosliga et al., 2006).In order to detect changes in large scenes, Hsiao et al. (Hsiao et al., 2004) combine terrestrial laser scanning and conventional surveying devices to acquire and register topographic data.The dataset is then transformed into a 2D grid and is compared with information obtained by the digitization of these existing topographic maps.In (Girardeau-Montaut et al., 2005), changes are detected in the 3D Cartesian world, and the possibilities of scan comparison in point-to-point, point-to-model, or model-to-model manners are discussed.The authors then use point-to-point comparison with some adaptations and make use of an octree as a data structure for accessing the 3D point cloud.
Comparison is then carried out by using the Hausdorff distance as a measure for changes.Hyyppä et al. (Hyyppä et al., 2009) also use the method of point-to-point matching for detecting changes in 3D urban scenes but in case of bad point registration, incorrect corresponding point pairs can cause false change detection.Zeibak and Filin (Zeibak and Filin, 2007) extend this method by further characterizing the changes caused by occlusions.Kang et al. (Kang and Lu, 2011) not only detect changes in buildings in the urban environment but also quantify the changed regions using a series of point cloud epochs over time and rebuilt building models.Nevertheless, this work only focuses on disappearing changes.In our work, we not only detect but also analyze both appearing and disappearing changes in the 3D urban scene by using 3D evidence grid and cognitive similarity functions.According to the best of our knowledge no prior work has ever been presented that first effectively detects, then analyzes the changes occurring in the 3D urban scene, in this manner, using terrestrial data and eventually updates the 3D cartography accordingly.The proposed method handles the 3D point cloud at two different levels: 1) point level to characterize the urban environment and accurately complete occluded regions; 2) grid level to analyze and handle different changes occurring in the urban environment.In this method, the 3D urban data obtained from mobile terrestrial LiDARs in each passage is directly geo-referenced with the help of an integrated GPS/IMU system.The use of these directly geo-referenced points along with grid level analysis makes this method independent of the type of Lidar sensors used in different passages.These geo-referenced 3D point clouds are segmented and classified into basic object types and then further characterized into 2 main object classes: Permanent and Temporary.The Temporary objects are then removed from the 3D point clouds, leaving behind a perforated 3D point cloud of the urban scene.These perforated 3D point clouds obtained from different passages (in the same place) on different days and at different times are then matched together to complete the 3D Urban landscape.Different changes occurring in the urban landscape over this period of time are studied using cognitive functions of similarity and the resulting 3D cartography is progressively modified

CLASSIFICATION AND EXTRACTION OF OCCLUDING OBJECTS FOR 3D URBAN CARTOGRAPHY
In urban environments, the quality of data acquired by different mobile terrestrial data acquisition systems is widely hampered by the presence of temporary stationary and dynamic objects (pedestrians, cars, etc) in the scene.As a result there is a problem of occlusion of regions.Moving objects or certain temporary stationed objects (parked cars, traffic, pedestrian etc) present in the area hide certain zones of the urban landscape (buildings, road sides, etc.).So the first step for 3D urban cartography is to obtain the permanent cartography.This is done by removing/extracting the Temporary objects from the scene/point cloud leaving behind only the permanent features.
In order to achieve this, we first classify the urban environment into 2 main categories: Permanent objects and Temporary objects.Although several methods have been proposed for the classification of urban environments, we have used one of the more recent methods (Aijazi et al., 2013) for this task.In this method the 3D point cloud is first segmented into objects, using a voxel based approach, which are then classified into basic object classes using geometrical models and local descriptors.Once classified into these basic classes, they are then grouped under one of the 2 mentioned categories using inference based on basic reasoning.The salient features of this method are data reduction, efficiency and simplicity of approach.Some results of this method are shown in Fig. 1.
Once the objects present in the urban scene are classified into these two main classes, in each passage, the objects classified as Temporary are separated from the scene for each passage to obtain perforated point clouds.This perforation is due to occlusions caused by the temporarily static and mobile objects in the scene.These perforated 3D point clouds of the same place obtained via a single passage on different days and at different times are then combined together to complete the 3D cartography by incremental updating as discussed in § 4. Furthermore, as the nature of the unclassified objects is not known, they are considered temporary by default and removed from the 3D point cloud/image.If by any chance, any of these unclassified objects belonged to the permanent cartography, they are then detected as changes in subsequent passages and are upgraded/added as Permanent objects in the 3D cartography during incremental updating.
Figure 1: Classification of the objects present in the urban scene into 3 main classes.

AUTOMATIC CHANGE DETECTION
The perforated 3D point cloud obtained in subsequent passages of any particular place (on different days and/or at different times) are combined together to fill in the occluded regions and complete the 3D urban cartography.The perforated point cloud is first mapped onto a 3D evidence grid and the corresponding 3D cell scores are calculated.A similarity map is generated in subsequent passages.Based on this similarity map and the associated uncertainty, different changes occurring in the urban environment are analyzed and appropriate actions are taken to cater for these changes.The details are provided below.

3D Evidence Grid Formulation
As the 3D point cloud is directly geo-referenced (Urban Data Challenge, 2011), the use of an occupancy grid for comparison in subsequent passages as compared to the elaborate graph theory is more logical and practical.The perforated 3D point cloud obtained in each passage is mapped onto a 3D evidence grid as shown in Fig. 2. Each 3D cell or voxel of this grid occupies a volume L 3 and is assigned a cell score CS based on certain attributes of the constituting 3D points using (1).These attributes include ratio of occupied volume VOcc, surface normal along X, Y and Z axes NX,Y,Z , mean laser reflectance intensity and mean RGB color values i.e.RI , Rc (c ∈ { R, Ḡ, B}) respectively and the number of the current passage np.The normalized values of these attributes are used to compute the cell score: where j is the number of cell, wOcc = 1, wN = 0.5 are Occupation weight and Normal weight while wRI = 0.25 and wRc = 0.125 are intensity and color weight respectively.wnp = 0.0625 is the passage number weight.The values of these weights are chosen to bias the score more towards occupancy (magnitude and orientation) and less towards representation (intensity and color) as the former is more invariant in the urban environment and also keeps our approach closer to the classical occupancy grid method (Souza and Gonc ¸alves, 2012).These 3D evidence grids are constructed in each passage for both the previous (P (np − 1)) and latest perforated 3D point cloud acquired and are used to formulate and update the similarity map along with the associated uncertainty.They are then deleted at the end of the process in each passage (Alg.1).This makes this approach most suitable for analyzing large mapping areas.

Similarity Map Construction
In order to obtain a similarity map in each passage, the 3D evidence grid in successive passages is compared.Instead of finding the overall graph/grid similarity, we are more interested in measuring the similarity of each cell as this indicates exactly which part of the 3D cartography has changed.Currently, many distances have been developed to compare two objects (in this case cell) according to the type of attributes, such as χ 2 or Mahalanobis.However, when working with real values, the most widely used (and simplest) metric is Minkowski measure dp: In this measure, xi and yi are the values of the i th attribute describing the individuals x and y.Wi is the numerical weight correlated with this attribute.k is the total number of attributes.In order to transform Minkowski distance (2) into a similarity measure sp a value Di is introduced, that corresponds to the difference between the upper and the lower bounds of the range of the i th attribute: This similarity function may provide a measure indicating the amount of changes occurring in a 3D grid cell in subsequent passages but it remains silent on the type of change taking place.Now this information could be useful when deciding how to handle these changes in the 3D cartography.Thus, in order to get more insight into the type of changes we incorporate the notion of distance between individuals or objects studied in Cognitive Sciences.For this purpose, we use the method proposed by Tversky (Tversky, 1977) to evaluate the degree of similarity Sx,y between two individuals x and y respectively, described by a set of attributes A and B, by combining the four terms A ∪ B, A ∩ B, A − B and B − A into the formula: As we want to compare a pair of individuals (in this case, cells in successive passages) described by a set of numerical attributes, we combine the definitions proposed by Tversky and Minkowski.
In these measures, we use Tversky's model to compare the two sets of attributes describing the individuals; the function f of this model is the Minkowski's formula as rewritten in (3).The parameter p of this formula equals 1 since in Tversky's model the function f corresponds to a linear combination of the features.Now, depending upon the way the parameters α and β are instantiated, different kinds of cognitive models of similarity can be expressed.By instantiating α = β = 0 we obtain the symmetric similarity measure Sym while by instantiating α = 0 and β = −1 we obtain the asymmetric similarity measure ASym.Now we use these values to fill up similarity map SMap as shown in Fig. 2. The values of ASym allow to evaluate the degree of inclusion between the first cell (reference) into the second cell (target).Hence, with the attributes (along with their corresponding weights) assigned to these cells (discussed in § 3.1), the value of ASym can be used to assume the type of changes occurring in the 3D grid cell as summarized in Tab. 1 where x and y are the same 3D grid cell in different passages.These values of Sym and ASyms not only help in ascertaining the type of changes occurring but also the most suitable action required to handle that particular 3D grid cell.Condition 1 is automatically handled in the incremental updating phase where as for condition 2 and 3, Automatic Reset function is called into action.This similarity map SMap is updated in each passage and only the different cells (with Sym < Similarity threshold ) along with their associated uncertainty values are kept in the map whereas the remaining cells considered as identical cells (with high level of similarity) are deleted from the map.Hence, this not only reduces the size of the map progressively in subsequent passages, but also avoids possible storage memory issues for large point clouds in case of large mapping areas (see § 5, Fig. 8, for more details).mapped onto an evidence grid of cell size L 3 respectively.In (c), the similarity map obtained from the two evidence grids.

# Condition
Possible assumption 1 ASymx,y < ASymy,x Addition of structure (could be new construction or earlier misclassified objects) 2 ASymx,y > ASymy,x Removal of structure (could be demolition) 3 ASymx,y = ASymy,x Modification of structure (depending on the value of Sym) Table 1: Type of changes

Associated Uncertainty
Let the cell scores of a particular cell j in n number of passages: C j S 1 , C j S 2 , . .., C j S n , be an iid sequence of random variables, then the n th sample variance s j n 2 is given as: then, adding and subtracting Cj Expanding and solving this to get Using the standard mean , the sum-term simplifies to 0: The uncertainty associated with each cell in the map u j is hence estimated and updated in each passage (n > 1) using the following relations: (5) ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey where Cj Sn is the moving or running average given as: There is no need to initialize ( 5) and ( 6) as for n = 2 the first term in (5) equals 0 and in (6) Cj S 1 = C j S 1 .This uncertainty measure, updated using ( 5) and ( 6), for each cell, in each passage, sheds light on the reliability of the state of the mapped cells as high uncertainty value means that the contents of the 3D cell are changing quite frequently: that could suggest high traffic circulation and other movements in the urban area.On the other hand, low uncertainty means that the contents of the 3D cell are fairly stable.This uncertainty measure can also be added as a criterion for selecting the appropriate action required to handle the changing 3D cells.The uncertainty distribution of neighborhood-1 can be seen in Fig. 3.
Figure 3: Uncertainty distribution map of a busy neighborhood-1 presented in the form of color coded heat map (with yellow being the lowest and red the highest level of uncertainty).The map clearly shows that the bottom part of the environment (close to the ground) is highly uncertain whereas the top part is generally stable.

INCREMENTAL UPDATING
The 3D cartography is updated in successive passages.The perforated 3D point clouds obtained for each passage of any particular place are not only combined to complete the occluded regions of the 3D urban cartography but all changes detected in the 3D cartography are analyzed and then progressively incorporated in the resulting cartography.This ensures that the resulting 3D cartography is not only accurate but is also well updated.The different steps involved in this method are discussed below.

3D Point Cloud Matching
Each subsequent 3D point cloud is registered with the former point cloud by using the ICP (Iterative Closest Point) method (Besl and McKay, 1992).This method is most suitable for this task as the 3D point clouds are already geo-referenced and hence lie in close proximity.It is observed that the major part of the 3D urban point clouds is composed of building points which are also found to be most consistent.Thus instead of applying the ICP method to complete 3D point clouds, only the building points are taken into account.First the profile/envelope of the buildings is extracted and then the ICP method is applied, matching these boundaries to obtain the transformation matrix.The outlines/envelopes of the buildings are extracted using a sweep scan method.As the bottom part of the building outline close to the ground is often occluded and hence inconsistent due to the presence of different objects in the scene (Fig. 3), only the boundary of the top half of the building outline is subjected to ICP as presented in (Aijazi et al., 2013).To avoid redundant points, a union of 3D points belonging to the two registered images is performed.Each 3D point of the first point cloud is matched with that of the second if pOa − pOb ≤ 3 √ e tol is satisfied.Here pOa and pOb (3 × 1 vectors) are point positions in two point clouds along X, Y and Z axes.e tol (3 × 1 vector) is equal to the inverse of the maximum number of 3D points per cubic meter that is desired in the 3D cartographic point cloud/image.The matched 3D points are considered as one point.This ensures that only the missing points are added completing the perforated 3D point cloud/image.Now this also ensures that even if there is some new construction or addition of certain 3D points in the scene in subsequent passages, they are automatically added hence catering for the first type of change (Tab.1).

Automatic Reset
The main purpose of this function is to detect and analyze different changes occurring in the urban environment, over a number of passages, before incorporating them in the resulting 3D cartography.The function analyzes the similarity map SMap and compares the Sym, ASyms and the uncertainty measures u of the 3D cells after every nreset number of passages.Those 3D cells C j n which satisfy conditions 2 and 3 (see Table 1) along with a low value of uncertainty measure u (7) are reset with those in the recently acquired image/point cloud (perforated) i.e. their contents are replaced by the contents of the same 3D cells in the recently acquired point cloud/image.This ensures that any changes that occur in the urban environment are automatically incorporated in the resulting 3D cartography in a very smooth manner without affecting the remaining part of the 3D cartography.
where m = {(n − nreset) • • • n}, with n > nreset.The proposed reset method was verified by synthetically modifying and demolishing/changing different parts of the urban environment including parts of buildings, roads, poles and trees in the datasets as shown in Fig. 6.Now (7) ensures that a detected change is only incorporated in the resulting cartography only if we are certain about it.If the change is well established, the uncertainty value of changing cells will progressively decrease and when it falls below the u threshold , these changes are incorporated in the 3D cartography via the Reset function (as shown in Fig. 4(a)).Whereas on the contrary, detected change with increasing or high uncertainty indicates rapid continuing changes occurring which could be either due to ongoing construction/reconstruction or rapid traffic movement in the scene (as shown in Fig. 4(b)).In such situations, the detected changes are not incorporated at once, but in fact we wait for the u to progressively decrease below u threshold as the number of passages np increases.This ensures the reliability and accuracy of our method.

Automatic Checks and Balances
In case of certain 3D points or objects are wrongly added or removed in the 3D cartography due to either misclassification those initially unclassified objects that were removed after being wrongly considered as Temporary, are then detected in subsequent passages as changes and after analysis are progressively added or removed from the 3D cartography by the Automatic Reset function: for example, in case of neighborhood-1 some building walls and for neighborhood-2 some permanently present trash cans were added to the cartography respectively.Hence, this ensures that the resulting 3D cartography contains only the exact, actual and permanent features.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey In order to fully evaluate the proposed method, we first constructed a ROC (Receiver Operating Characteristic) curve.The value of Similarity threshold (used as a discriminatory threshold) was varied from 0 to 1 and corresponding True Positive Rates (TPR) and False Positive Rates (FPR) were calculated for neighborhood-1 (modified).The variation of this ROC curve with respect to u threshold (varied from 0 to 0.5) is presented by a 3D ROC curve (surface in this case) in Fig. 5. From the figure it could be observed that best/optimal results (A on the figure) can be obtained from Similarity threshold = 66% and u threshold = 0.15.This analysis was conducted at 3D cell level.Figure 6 shows some of these detected changes in the urban scene.Using these values of Similarity threshold and u threshold along with L = 2 m, e tol = (0.000125 0.000125 0.000125) T (in m 3 ) and nreset = 3 (maximum number of passages possible in our case is 4), we then, evaluated the performance of our method, for all three neighborhoods, using different standard evaluation metrics as described in (Vihinen, 2012) (see Table 2).Although, all these metrics are commonly used to evaluate such algorithms, MCC (Matthews Correlation Coefficient) is regarded as most balanced measure as it is insensitive to different class sizes (like in our application the number of changed cells (changes) is generally quite less as compared to unchanged cells in the urban environment).The MCC, like the other measures, is calculated based on the count of the true positives (i.e.correct detection of changed 3D cells), false positives, true negatives, and false negatives.A coefficient of +1 represents a perfect prediction, 0 no better than random prediction and −1 indicates total disagreement.The detailed results including overall accuracy ACC and MCC greater than 85% and +0.6 respectively, clearly demonstrate the efficacy of the proposed method.The variation in the size of the similarity map, for the three neighborhoods, was also studied.Figure 8 shows the progressive reduction of the size of the similarity map in subsequent passages.This is due to the fact that in subsequent passages, as occluded regions are completed, the number of low similarities caused by the missing occluded features decreases and also the different changes occurring in the cartography are catered by the Automatic Reset function.In case of neighborhood-1 (modified), Fig. 8, the large size of SMap is due to large parts of the building, poles, trees, etc. being demolished in the neighborhood-1 (see Fig. 6) whereas a sharp decrease in size occur at np = 4 due to Automatic Reset function (nreset = 3).We see that some non-repeating or those cells with high associated uncertainty remain even after the Automatic Reset.This number of cells in the SMap should ideally reduce to zero as number of np increases.This shows that the proposed method is most suitable for handling large point clouds in case of large mapping areas. Neigh

CONCLUSION
In this work, we present a method for automatic change detection and incremental updating for 3D urban cartography.Different man-made or natural changes occurring in the urban landscape are automatically detected and analyzed using cognitive functions of similarity and the resulting 3D cartography is progressively modified accordingly.The proposed method ensures that the resulting 3D image/point cloud of the cartography is well updated and it contains only the exact and actual permanent features.The results evaluated using different standard metrics demonstrate the technical prowess of the method which can be easily integrated in different commercial or non-commercial applications pertaining to urban landscape modeling and cartography that need to frequently update their database.

Figure 2 :
Figure 2: Formulation of 3D evidence grids and similarity map.(a) & (b) show the 3D point clouds (P(np) and P(np − 1))mapped onto an evidence grid of cell size L 3 respectively.In (c), the similarity map obtained from the two evidence grids.

Figure 4 :
Figure 4: shows the uncertainty u variation against number of passages np of a group of three adjacent cells in which change was detected.(a) represents the case of established changes while (b) represents the case of rapid or continuing changes.

Figure 5 :
Figure 5: 3D ROC curve for neighborhood-1 (modified).Point A on the surface represents optimal/best results.

Figure 6 :
Figure 6: The different changes detected in one of the urban scene are presented in red.These changes were due to demolished building wall, larger poles cut in half and trimming/cutting of bushes/trees in the scene.
Figure 7: (a) shows the initial point cloud related to urban cartography full of perforations.In (b), (c) & (d) completion of occluded and missing features in the urban cartography, by incremental updating, as changes of type-1 detected.In (e), Automatic Reset function came into action after changes of type 2 & 3 were detected to update the modifications in the 3D cartography.On (e) are also marked the changes successfully detected and updated after 4 passages.These include: 1-Building wall/ roof added due to initial misclassification; 2-Part of Building demolished; 4, 5 & 6-Trimmed or cut trees/bushes; 3, 7 & 8-Longer poles cut into half.

Figure 8 :
Figure 8: The progressive reduction of the size of similarity map SMap is presented.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey accordingly by incremental updating.An overview of the method is presented in Alg. 1.