POTENTIAL OF MULTI-TEMPORAL OBLIQUE AIRBORNE IMAGERY FOR STRUCTURAL DAMAGE ASSESSMENT

Quick post-disaster actions demand automated, rapid and detailed building damage assessment. Among the available technologies, post-event oblique airborne images have already shown their potential for this task. However, existing methods usually compensate the lack of pre-event information with aprioristic assumptions of building shapes and textures that can lead to uncertainties and misdetections. However, oblique images have been already captured over many cities of the world, and the exploitation of preand post-event data as inputs to damage assessment is readily feasible in urban areas. In this paper, we investigate the potential of multitemporal oblique imagery for detailed damage assessment focusing on two methodologies: the first method aims at detecting severe structural damages related to geometrical deformation by combining the complementary information provided by photogrammetric point clouds and oblique images. The developed method detected 87% of damaged elements. The failed detections are due to varying noise levels within the point cloud which hindered the recognition of some structural elements. We observed, in general that the façade regions are very noisy in point clouds. To address this, we propose our second method which aims to detect damages to building façades using the oriented oblique images. The results show that the proposed methodology can effectively differentiate among the three proposed categories: collapsed/highly damaged, lower levels of damage and undamaged buildings, using a computationally lightweight approach. We describe the implementations of the above mentioned methods in detail and present the promising results achieved using multi-temporal oblique imagery over the city of L’Aquila (Italy).


INTRODUCTION & RELATED WORKS
Structural damage assessment is an imperative process to be carried out immediately after the disaster event for effective planning and execution of response and recovery actions.Assessing building damages over large areas affected by hazard events with ground observations is not efficient.Alternatively, remote sensing-based approaches have been recognized as useful means for assessing synoptic building damage.Detailed information of an affected area can be provided in a short time using a variety of sensors such as optical, SAR and LiDAR (Khoshelham et al., 2013;Miura et al., 2013;Uprety and Yamazaki, 2012).In particular airborne oblique images have been recognized as a valuable data source to assess building damages because, compared to traditional nadir views, they allow the complete inspection of the external outlines of the building, such as roofs and façades (Murtiyoso et al., 2014).Nowadays, airborne images are captured with high overlap, and the generated point clouds can be exploited in the damage assessment process as well (Sui et al., 2014).Geometrical deformations such as partial/complete collapse, pancake collapse, inclination, broken and dislocation of elements can be easily derived by 3D geometric information, while damages such as cracks and spalling can be inferred from the images directly.
Several papers have highlighted the potential of synergistic use of 3D point cloud and images for building damage assessment (Gerke and Kerle, 2011;Vetrivel et al., 2015).However, only few studies have looked at the use of digital oblique aerial imagery for structural damage assessment, and were focused on (mono-temporal) post-event information (Gerke and Kerle, 2011;Vetrivel et al., 2015).The major limitation of this approach is that * Corresponding author.
damage is inferred based on a set of ontological assumptions: i.e. a surface with unusual radiometric or geometric characteristics is assumed to be damaged, while manmade objects are assumed to have a regular shape and uniform radiometric characteristics.These assumptions have limitation in complex environments, leading to a high rate of false alarms, which reduces their reliability and operational utility.In Vetrivel et al. (2015), damages presenting regular and uniform shapes (false negative), or intact regions characterized by cluttered and non-uniform radiometric distributions (false positive), were incorrectly classified due to these assumptions.The above uncertainties can be alleviated if pre-event data are available for reference.Many studies have demonstrated the potential of multi-temporal data for damage assessment, though with most focusing on nadir-view images (Dong and Shan, 2013;Murtiyoso et al., 2014).To our knowledge, no methods have been reported yet for identifying building damages using multitemporal oblique images and/or 3D point clouds.In this paper the first implementation of an automated algorithm for building damage assessment from multi-temporal oblique images is presented.Although geometrically more stable cameras are used nowadays in oblique airborne systems, many data sets are captured with less sophisticated camera systems, and image overlap is often restricted to 2-fold.Hence, for such configurations one has to cope with dense image matching point clouds of minor quality (relatively large random error margin, gaps).In particular façade regions are generally represented by sparse and very noisy 3D points, as they are more cluttered and often occluded (Rupnik et al., 2014).The proposed methods take advantage of both 2D and 3D information and efficiently cope with these problems.
The paper addresses two independent objectives:  The development of a methodology to identify severe building damages related to geometrical deformation, using multi-temporal oblique images and 3D point clouds.To this end three different change detection methods are proposed to identify building elements that are geometrically deformed between the two epochs.Subsequently, a change classification method is proposed to identify the geometric deformation of an element marked as damaged.


The analysis of multi-temporal oblique images for identifying the damages along façades which are often not well modelled in the generated point clouds.
The detailed description of the methodologies and the results achieved on the test area of L'Aquila (Italy) will be presented in detail.

DATA DESCRIPTION
The data used are corresponding to the city of L'Aquila, Italy in which an earthquake occurred on 6 th April 2009.The data consist of two airborne oblique acquisitions (August 2008 and May 2009) covering the city with both oblique (4 cameras) and nadir (1 camera) imagery, captured by small format DSLR cameras.
Images were acquired at a flying height of approximately 1000 m allowing for an average ground sampling distance of 16 cm on oblique views.The flight was conducted considering a forward overlap between 60-70% and side overlap between 35-45%, allowing to derive a 3D point cloud.The registration was achieved computing tie-points from all the imagery, forcing both epochs to share a local coordinate system.Dense image matching was then performed separately on both epochs.

Methodology
Any severe structural damage, such as partial or complete collapse, pancake collapse, dislocated or inclined elements, leads to the absence of the given elements in their actual 3D boundary in the pre-event data.These damaged elements are referred to missing elements.The missing structural elements can be identified by comparing the accurately co-registered multitemporal (pre and post-event) 3D point clouds.However, the absence of pre-event segments in post-event 3D point clouds may be due to many reasons, such as occlusion/a building part being exposed to a lower number of camera views, or poorly textured surfaces, leading to missing 3D points.On the other hand, an element can be missing because it was damaged or cleared deliberately.Therefore, it is important to infer the reasons for the absence of a pre-event element in the second epoch data after detecting them.The missing elements due to damage are detected using the preand post-event images and 3D point clouds derived from them by three pipeline processes (cf. Figure 1).As an initial step, the individual buildings in the area are delineated from the pre-event 3D point cloud.Subsequently, the delineated buildings in the preevent data are compared to the post event 3D point cloud to detect changes.Finally, the changes are classified to isolate the changes caused by damage.The detailed methodology to carry out the aforementioned processes is described below.In remote sensing data, the roof segments are the most visible and least cluttered building elements.Moreover, they can be recognised based on simple geometric constraints.For instance, the roof segments are mostly elevated horizontal or slanted planar surfaces with respect to ground.Hence, in this study the roofbased building delineation approach based on 3D point clouds is adopted.Moreover, this method has previously shown reliable results (Vetrivel et al., 2015).The procedure followed is described below.The 3D point cloud is segmented using planar segmentation as described by Vosselman (2012).The segments with Z component of the plane normal > TZ (0.4) and above height TH (3 m) are labelled as roof segments.The spatially connected roof segments are identified by defining the alpha shapes with an alpha radius of TR (0.3).Finally, the 3D points that are covered by single alpha shapes are delineated as roofs of a single building.Also, all the 3D points that lie within the 2D boundary of the alpha shape are registered as the 3D points of the building, i.e. all 3D points that lie below the roof elements are also registered as the 3D points of the building.

Change detection to identify the missing building elements in post event:
The pre-event building elements that are missing in the post-event point cloud are identified using the following three approaches.To provide a clear understanding of the proposed methods, they are illustrated with examples from the data used and results obtained from our experimental study.Also, the pros and cons, and appropriate scenario that make a particular method suitable, are discussed.

Voxel-based approach (VBA):
A unit of 3D space that is occupied by a specific building element in the pre-event but not the post-event data is straight-forwardly identified and classified as a change.The 3D space defined by the delineated building in the pre-event point cloud is divided into voxels.The edge length of voxels is defined based on image ground resolution (0.5 m ~ three times the image ground resolution).The 3D points of the pre-and post-event point clouds that lie within the defined 3D boundary are added to the corresponding voxels.The voxels that contain pre-event but no post-event 3D points are classified as changed voxels.However, it is challenging to differentiate between the changed and unchanged voxels as there is a high probability that 3D points of the pre-and post-event epochs of the same building element may fall into different (adjacent) voxels, due to the varying noise level between the two point clouds.To overcome this problem the spatial buffers along the horizontal and vertical directions with buffer thresholds of THB (0.5 m) and TVB (1.0 m), respectively, are created for each voxel occupied with the pre-event 3D points.These voxels are classified as unchanged voxels if post-event 3D points lie within the buffer area; otherwise they are classified as changed voxels.
Finally, the pre-event 3D points of the changed voxels are detected as 3D points of missing building elements and considered for the further change classification process.The overall process of this approach is shown in Figure 2.
Pros: In many cases photogrammetric point clouds are very noisy, hindering the recognition of individual building elements.
In such cases, this approach is more suitable as it does not need any prior information or assumptions about the building, unlike the segment-based approaches discussed below.

Cons:
The presence of artefact 3D points, common in photogrammetric point clouds in unfavourable image flight configurations, strongly affect the performance of this approach.Moreover, in specific scenarios, it cannot detect or accurately delineate the missing portion of the element, since the voxels are classified in a binary fashion, i.e. whether or not they contain a certain element.Therefore, even if only a minor portion of a voxel is occupied by an element, it will be classified as occupied voxel.Hence this results in a failure to detect missing elements that characterises the remaining majority portion of the voxel.Since it does not consider the geometry of the elements it may fail to detect the damages in specific scenarios.For example, consider a horizontal roof segment is missing and the below vertical elements are visible.In such a case, this approach may fail as the 3D points of the vertical elements fall into the voxels of the missing elements when the voxel and its buffer sizes are significantly large.Moreover, this approach is computationally intensive.

Figure 2. Workflow of voxel-based approach
Segment-based approach (SBA): In general, most buildings are made of a composition of planar segments.Hence, we anticipate that comparison of pre-and post-event data based on planar segments will help to precisely identify changes on element level.This would also lead to more object-oriented analysis compared the voxel-based approach.However, direct comparison of 3D segments that are obtained from independent segmentation of pre and post-event is not always practically possible.This is because, though the 3D point clouds of the two epochs are segmented using the same segmentation algorithm, in practice it is not always feasible to obtain the same segments even for corresponding areas.This is predominantly due to varying noise level between point clouds.Therefore, we propose a method where the pre-event point cloud alone is segmented and the corresponding segments in the post-event are derived based on the pre-event segments.This is done by fitting a plane to preevent 3D segment and, subsequently, the post-event 3D points that lie within the plane-offset of TD (1.0 m) to be derived as postevent segment.Then the missing (damaged) portions of pre-event segments are identified by comparing them with the corresponding derived post-event segments.Only the segments with an area greater than TA (5 m 2 ) are considered for the change detection process.The overall procedure of this approach is shown in Figure 3.
Pros: Compared to the voxel-based approach it is 1) less sensitive to artefacts, 2) more robust in detection and accurate delineation of missing elements, and 3) computationally less demanding.
Cons: This approach fully depends on the performance of the planar segmentation algorithm.The planar segmentation fails to produce accurate segmentation for portions of point cloud corresponding to very high noise, and also for regions of nonplanar elements.Composite segment-based approach (CSBA): The above segment-based approach is defined particularly for plane-based segmentation.Here, we develop an alternative segment-based approach that is independent of the segmentation methods.
Pertaining to this, a composite segment-based approach is developed where the pre and post-event point clouds are merged and segmented together using a plane-based segmentation algorithm (although any segmentation method can be used).Therefore, the corresponding segments in the two dataset share the same segment label, which facilitates direct comparison between them.Thereby the completely and partially missing elements can be identified in simple and faster way than in the above proposed methods.The proposed change detection strategy is described below.


The pre-event segments that have no corresponding segment in the post-event point cloud represent the completely missing segments.This can be obtained in a single step by a simple set difference: i.e. suppose sets A and B are the segment label lists of pre and post-event epoch, respectively, then the completely missing segments are obtained by A-B (set difference).


To identify the partially missing segments select the corresponding segments in the pre-and post-event data and define the boundary for the post-event 3D segment using the alpha shapes.The pre-event 3D points that lie outside the defined boundary are considered as the missing portion of the pre-event segment in the postevent data.
Pros: It is faster and simple than the other two proposed methods.

Cons:
The choice of segmentation algorithm is critical as it should detect building elements of different geometry (planar and no-planar) and robust to varying noise levels between the point clouds.It is also highly vulnerable to co-registration errors of multi-temporal point clouds.

Change classification (inferring reason for change):
In general, the disappearance of a building element due to damage will lead to two kinds of scenarios: 1) the absence of certain elements will create an opening, leading to a visibility of the element below it.Therefore, there will be a new surface (3D points) in the post-event data, corresponding to this disappeared element (cf. Figure 4), or 2) the disappearance of partial elements may create a hole in the structural element which appear darker due to poor radiometric reflection (cf. Figure 4).This is termed as structural holes and it will be a gap in the 3D point cloud.However, gaps in 3D point clouds may also be caused by mis/no matches in 3D point generation (e.g., poorly textured surfaces) and occlusions.Therefore, it is important to distinguish between these gaps to identify the ones caused by damage.As the image radiometry plays a major role here, we used pre-and post-event images in addition to point clouds for the change classification process.The procedure for inferring the reasons for the absence of pre-event element in post-event is described below.Case 1: Element missing due to occlusion/partial visibility: The pre-event 3D points missing in the post-event data due to occlusion can be identified by analysing the visibility of those points in the post-event camera views.This is done by adding the missing pre-event 3D points to the post-event point cloud.Subsequently, the visibility of 3D points in each post-event camera view is estimated using the Hidden Point Removal operator (HPR) (Katz et al., 2007).Finally, the newly added preevent 3D points that are visible in less than 'N (3)' camera views (post-event cameras) are removed by labelling them as occluded points, and the remaining pre-event 3D points are classified as visible points and considered for further change classification process.
Case 2: Element missing due to damage and mis/no matches in 3D point generation: The change classification is preferred to be performed at segment level, as manmade objects are largely composed of planar/regular elements, thus it helps to reduce false decisions.The segment-based approaches will directly provide the 3D points of missing element in terms of segments.However, for the voxel-based approach, it will provide the collection of independent pre-event 3D points that are absent in the post-event situation.Therefore, the 3D points obtained from the latter are grouped into dis-jointed 3D segments based on their spatial connectivity, using the alpha shapes for further processing.
A rule-based approach is adopted and a set of rules is defined to classify the missing elements into the aforementioned scenarios.This classification approach is independent from the methods used to detect missing elements and is described below.

Element missing due to damage and the surface below it is visible:
The rule for this class is defined below and illustrated in Figure 5.

Rules:
The post-event 3D points should be present within the 2D boundary (change boundary) of the missing pre-event 3D segment.Also the post-event points and pre-event 3D segment should be visible in the same post-event camera.Also the area covered by the post-event 3D points should be of similar size compared to the area of the missing pre-event segment (at least cover 30 %).
Figure 5. Example for missing 3D segment classified as damaged and the surface below it is visible in post-event.
Pre-event element missing in post-event due to damage (structural hole) or mis/no matches in 3D point generation: In contrast to the above scenario, if the change-boundary does not contain a significant number of post-event 3D points then the radiometric characteristics of the change boundaries in the preand post-event data are compared by delineating them in the corresponding images.If they are not similar and if the post-event image surface appears significantly darker compared to the preevent one, then it is classified as damage (structural holes), otherwise it is classified as mis/no matches in 3D point generation.The grey-value (image pixel value) based histogram is used as a feature to compare the radiometric characteristics as it is well proven and widely used region-level image descriptors (Wenjing et al., 2006).

Results of building delineation:
All 48 buildings are detected using the method described in section 3.1.1and they are categorized into three cases: 1) Buildings that were delineated with close approximation to the actual boundary (# 36); 2) buildings with some portions not delineated (# 7); 3) single buildings detected as multiple buildings, particularly the buildings with multi-level roofs.(#5).A sample result for building delineation is shown in Figure 9.

Results of the change detection methods to identify the missing building elements in post event:
In this section, the overall results and the major inferences associated with each change detection method are summarized.The overall results are provided in Table 1.An example result of missing elements that were detected by the composite segment-based approach for a small block is depicted in Figure 10.Table 1.The results of missing pre-event 3D segments detected by all three approaches.

Results of change classification:
For the change classification process, the missing elements identified by the segment-based approach were considered (cf.Table 1 ).
Case 1: Element missing in the post-event data due to occlusion: only one segment that was very small (area < 5m 2 ) was identified as missing in the post-event data due to occlusion (cf.Table 1), and it was classified correctly and removed from the further classification process.
Case 2: Element missing due to damage and mis/no matches in 3D point generation: The results of the change classification process are provided in Table 2.In total all 20 elements that are missing due to damage were classified correctly.Among the three non-damaged missing elements two were correctly classified as mis/no matches in 3D point generation.The remaining one was misclassified as damage related to structural hole as it was affected by shadow in the post-event data.

. Results of classification of missing 3D segments detected by segment-based approach
The final outcome of the overall process is the report for each building that shows the area and boundary map of each missing elements of the building due to damage.For example, Figure 11 depicts the area and outline of each missing elements for a building shown in Figure 3.

Discussion
The pre-and post-event point clouds derived from the oblique images are noisy.However, the noise level varies from place to place as they are susceptible to the radiometric characteristics of the surface.Therefore, the places corresponding to very noisy 3D points were not segmented well, and because of this three (out of 23) damaged elements were not detected by both segment-based approaches (cf.Table 1).The voxel-based approach has not detected six damaged elements.This is due to its limitations that are highlighted in the cons of the voxel-based approach (cf.3.1.2).The change classification (cf.3.1.3)is a straightforward approach which correctly classified the missing elements due to earthquake damages or man-made changes.The classification of mis/no matches failed just in one case, in correspondence of a shadowed building element that was wrongly classified as structural hole.

MULTI-EPOCH FACADE IMAGE CORRELATION FOR DAMAGE ASSESSMENT
The objective of the following method is to automatically detect building façade changes by comparing their radiometric values.It will lay the ground for further developments focusing first on the extraction and rectification of the image patches containing the façades, followed by the comparison itself, and considering three main categories: highly damaged or collapsed façades, lower levels of damage (changes), and undamaged buildings.

Method
To perform the multi-temporal comparison between the façades, these building elements must be, beforehand, extracted from the images.The pre-event 3D point cloud allows the identification and extraction of the points relative to the façades.These can be back-projected into the image, using the correspondent projection matrices, defining the boundaries of the image patches to extract.The 3D points corresponding to the façades will also be used to define the plane containing the façade by fitting a least square plane.Using this 3D plane and the extracted façade patches, these can be rectified using a homography matrix.An interpolation is performed on the gaps produced by the projection of the pixels to the real world façade plane (see Figure 12).

Results
This section presents the results according to the categories defined earlier.As explained before the intra-epoch radiometric comparison will serve as reference value for the inter-epoch comparison.The undamaged case will be assessed first (Figure 13).It will be followed by an example posing the possible intraepoch differences between the extracted image patches from two distinct images (Figure 14), given the problems addressed in the previous section.The inter-epoch comparison will then be addressed considering damage related changes (Figure 15) and other changes (Figure 16).Finally, the collapsed building case will be depicted (Figure 17).
Considering the façade presented in Figure 13, the correlation coefficient was 0.78 intra-epoch and 0.75 inter-epoch.This similarity can categorize this façade as unchanged and consequently not damaged.Not all changes are damage related as can be seen in Figure 16.
Here the intra-and inter-epoch correlation coefficients were 0.66 and 0.33, respectively.Unlike the latter case, the changes which decreased the correlation coefficient are not hazard related and are due to the removal of banners present in the pre-event.However, correlation values are completely different in the case of complete collapses.The façade shown Figure 15 is considered in an inter-epoch comparison: the mean of the correlation coefficient drops drastically (from 0.51 to 0.04) indicating directly the presence of a collapsed building (Figure 17).This can be performed for several façades to confirm the outcome of the categorization, as collapsed, for the whole building.
Figure 17.Total collapse example, rectified images on both epochs and correlation coefficient matrix.

Discussion
The results obtained above allow to confirm that the present methodology can differentiate between the three proposed categories, collapsed/highly damage, presence of lower levels of damage and undamaged buildings, using a computationally light approach.In the collapsed or highly damaged case this can even be confirmed using the available façade elements and also roof elements corresponding to a same building.Considering the undamaged/unmodified case, the correlation coefficient similarity will mostly indicate the presence of the same unchanged element.Although, as seen in Figure 14.Pre-event rectified image patches and corresponding correlation coefficient., the presence of balconies or other overhanging details will decrease the correlation between facades, since these were assumed flat in order to perform the image rectification.Analogously a very low correlation value can immediately indicate a high level of damage.Concerning the intermediate category, where the changes happened at a façade level, the definition of a correlation interval which includes these elements may not be so direct like the previously referred categories; nonetheless, the multi-temporal component can certainly aid in the definition of such interval.

CONCLUSION AND OUTLOOK
In the presented paper, two independent methodologies were developed to identify the structural damages by utilizing both 2D and 3D information derived from multi-temporal, pre-and postevent oblique images.
The first method of the paper focussed on developing methods to identity structurally deformed elements due to damage using the pre-and post-event images and 3D point cloud.The developed methods produced significant results, particularly, the segmentbased approaches detected 87% (20 out of 23) of geometrically deformed elements, and all of them were correctly classified as damaged by the proposed change classification approach.However, while the proposed methods can identify the structurally deformed elements due to damage, they cannot infer the type of structural deformation, such as dislocation, inclination, complete collapse or pancake collapse.Generally, any missing structural element in the pre-event data will emerge as a new element (at least debris) in the post-event epoch.These newly emerged post-event elements can be identified by detecting the missing elements from the post-to pre-event data using any one of the proposed change detection methods.A semantic analysis by mapping of corresponding missing preevent and the newly emerged post-event elements would help to infer the specific reason of structural deformation.This would be the logical extension of this work.
The planar segmentation was adopted to derive segments in the segment-based change detection approaches.It often failed to provide accurate segmentation for very noisy and non-planar regions in the 3D point cloud, which hindered the assessment for those regions.However, numerous point-cloud segmentation methods have been developed which utilize the contextual information, the image-radiometric and points cloud-geometric information in combination, and which provide better segmentation for noisy and non-planar regions (cf.Anh and Bac (2013)).Adopting such segmentation coupled with our proposed composite segment-based approach can yield better assessment.
The second presented methodology aims at comparing building façades at an image level, in order to infer the presence of damages on them.A rough distinction between the three proposed damaged levels using a fast approach has been demonstrated.However, due to the variability of light conditions and different point of views, the correct selection of damages on the façades still remains a challenge.For now the method does not include a visibility analysis to automatically select building façades without occlusions in the images: this will be added in the following implementation.Actually, areas in which large variations of visibility changes occurred or showing non-planar elements like in-/extrusions can already indicate candidates for damaged areas.The ones that are present in just an epoch will have to be carefully assessed.These variations in the occlusion can have its origin in the data acquisition itself, another example can be vegetation changes or the modification of the urban configuration.A segmentation of the building façade (in 2D and 3D) will be performed on the façades in order to detect and remove windows and balconies, restricting the change search on the facade walls.The shadow detection will be addressed as well.This method does not need the computation of point clouds from different epochs but only co-registered images.Already existing 3D city models could be used to define (and rectify) the façade position, strongly reducing the time needed in the damage map generation since there is no need to generate a point cloud.It would also allow not only the integration of the damage results with the city model itself but also to ease an integration with damage maps from other sources.

Figure
Figure 1.Overall workflow 3.1.1Building delineation from 3D point cloud:In remote sensing data, the roof segments are the most visible and least cluttered building elements.Moreover, they can be recognised based on simple geometric constraints.For instance, the roof segments are mostly elevated horizontal or slanted planar surfaces with respect to ground.Hence, in this study the roofbased building delineation approach based on 3D point clouds is adopted.Moreover, this method has previously shown reliable results(Vetrivel et al., 2015).The procedure followed is described below.The 3D point cloud is segmented using planar segmentation as described byVosselman (2012).The segments with Z component of the plane normal > TZ (0.4) and above height TH (3 m) are labelled as roof segments.The spatially connected roof segments are identified by defining the alpha shapes with an alpha radius of TR (0.3).Finally, the 3D points that are covered by single alpha shapes are delineated as roofs of a single building.Also, all the 3D points that lie within the 2D boundary of the alpha shape are registered as the 3D points of the building, i.e. all 3D points that lie below the roof elements are also registered as the 3D points of the building.

Figure 3 .
Figure 3. Overall workflow of segment-based approach

Figure 4 .
Figure 4. Example of element collapses leading to an opening with the surface below it visible (left) and not visible, i.e. structural holes (right) are highlighted in red circles.

Figure 6 .
Figure 6.Example for missing 3D segment classified as (structural hole) caused by damage.

Figure 7 .
Figure 7. Example for missing 3D segment classified as mis/no matches in 3D point generation.
used: three subsets of building blocks from different parts of the city were considered for the damage detection process, containing a total of 48 buildings.Of those 23 building elements are identified as either completely or partially missing in the post event data due to damage.The pre-event 3D point clouds of the considered sub-blocks are shown in Figure8.

Figure 8 .
Figure 8. Pre-event 3D point clouds of the sub-blocks considered for damage assessment.

Figure 9 .
Figure 9. Image subset of airborne image (left) and delineated buildings based on 3D point cloud are projected over the image (right).

Figure 10 .
Figure 10.The detected missing pre-event segments using composite segment-based approach are projected as red points over a pre-event image (left), and outlined in the post-event (right) images with yellow circles.Method Detected damage (not detected) Missing elements due to: Occlusions Mis/no matches VBA 17 (6) 3 8 SBA 20 (3) 1 3 CSBA 20 (3) 1 2

Figure 11 .
Figure 11.Outline of each missing segment of a building and their areas are annotated in m 2 (left) and superimposed on the corresponding building in post-event (right).
Variable resolution and brightness can be detected according to the point of view of each image.These problems affect the results independently of the epoch of the images (same or different epoch).The comparison itself is made by determining the correlation coefficient between the rectified façade patches.This correlation coefficient is computed using a 7 by 7 pixels moving window, determining local (on each window position) and global (mean of the considered façade) values of the computed cross correlation.Nevertheless, only results of the inter-epoch correlation are not sufficient, since they do not have an actual meaning of change/no change, but just provide a correlation value of the pixels of the compared façades.To normalize the correlation values and increase the feasibility in the change detection, the correlation coefficient is first performed using different images of the same epoch as this value serves as reference to judge the multi-epoch comparison.The façades with a difference between intra-and inter-epoch correlation coefficients bigger than an imposed value will be considered as highly damaged or collapsed buildings.Again, this imposed value to limit the difference in the correlation values between epoch is based on the intra-epoch correlation results (see 4.2 Results).The same for undamaged buildings where similar correlation coefficients in both the intra-and inter-epoch indicate the presence of an undamaged element.The correlation values from the intermediate category, lower levels of damage, are still the most critical to be automatically interpreted, and they are classified as changes in the current method implementation.

Figure 12 .
Figure 12.Example of two pre-and post-event subsets of oblique images containing the façade (above) and respective rectified images (below).

Figure 13 .
Figure 13.Same façade extracted from both epochs.a) and b) relative to pre-event and c) post event.

Figure 14
Figure 14 represents another façade element in which the intraepoch correlation coefficient is lower than in the former example.The balconies which are not in the defined plane, are hence consequently wrongly rectified.Different illumination settings are also noticeable on the shadows of the shown rectified patches.The global correlation value on the façade is therefore very low (0.51) and the correlation values are extremely low (darker areas) in correspondent balconies and associated shadows (Figure 14, rightmost part)

Figure 15 .
Figure 15.Hazard-related changes.Same façade extracted from both epochs.a) and b) relative to pre-event and c) post event.

Figure 16 .
Figure 16.Changes not hazard related.Same façade extracted from both epochs.a) and b) relative to pre-event and c) post event.