A NEW OBJECT BASED METHOD FOR AUTOMATED EXTRACTION OF URBAN OBJECTS FROM AIRBORNE SENSORS DATA

The classification of urban objects such as buildings, trees and roads from airborne sensors data is an essential step in numerous mapping and modelling applications. The automation of this step is greatly needed as the manual processing is costly and time consuming. The increasing availability of airborne sensors data such as aerial imagery and LIDAR data offers new opportunities to develop more robust approaches for automatic classification. These approaches should integrate these data sources that have different characteristics to exceed the accuracy achieved using any individual data source. The proposed approach presented in this paper fuses the aerial images data with single return LIDAR data to extract buildings and trees for an urban area. Object based analysis is adopted to segment the entire DSM data into objects based on height variation. These objects are preliminarily classified into buildings, trees, and ground. This primary classification is used to compute the height to ground for each object to help improve the accuracy of the second phase of classification. The overlapping perspective aerial images are used to build an ortho-photo to derive a vegetation index value for each object. The second phase of classification is performed based on the height to ground and the vegetation index of each object. The proposed approach has been tested using three areas in the centre of the city of Vaihingen provided by ISPRS test project on urban classification and 3D building reconstruction. These areas have historic buildings having rather complex shapes, few high-rising residential buildings that are surrounded by trees, and a purely residential area with small detached houses. The results of the proposed approach are presented based on a reference solution for evaluation purposes. The classification evaluation exhibits highly successful classification results of buildings class. The proposed approach follows the exact boundary of trees based on LIDAR data which provide above average classification results for the trees when compared to the assumed ideal circular shaped trees in the reference data.


INTRODUCTION
The need for accurate extraction of urban objects such as buildings, trees and roads is highly growing due to its vital role in different applications such as urban planning, civil engineering, and environment protection.The continuous advancements of the airborne LIDAR and image sensors offered new enhanced data specifications that need evolving processing techniques to exploit the new abilities, resolutions, and accuracies in highly automatic fashion to overcome the slow and costly but yet accurate human processing.The emerging LIDAR technology complements the aerial imagery technology towards more complete and accurate sensing to enhance the automatic extraction process.LIDAR data lack the semantically rich data provided by the different bands of the optical images which is very useful for the detection of many classes such as vegetation.Also, optical images typically offer higher resolution data than LIDAR.However, despite the sophisticated approaches of image processing, feature extraction and matching, automatic DTM generation using optical images is struggling against several problems such as occlusions, shadows, and steep slopes.These problems, on the other hand, can be obviously reduced using LIDAR technology that offers reliable height data regardless of objects textures and illumination conditions.LIDAR also plays an important role in deriving ortho-photos from aerial photos.The effectiveness of LIDAR is very noticeable due to its level of accuracy and its highly automated data acquisition workflow.
A wide range of approaches have been developed to employ LIDAR data in land cover classification tasks.Several iterative methods have been proposed to filter the non-terrain points out such as successive spline interpolation using gradient and surface orientation analysis (Brovelli et al., 2002), fitting an interpolating surface using iterative least squares (Kraus et al., 1998), iterative densification of a triangular irregular network (TIN) (Axelsson, 2000).Clustering algorithms such as k-means (Chehata et al., 2008), and fuzzy c-means (Zulong et al., 2009) have been proposed to cluster the LIDAR points into different classes.Geometric descriptors such as static moments, curvature, and data anisotropy have been used by (Roggero, 2002) for clustering LIDAR data.(Song et al., 2002) assessed the possibility of using LIDAR intensity data for land-cover classification.(Parrish, 2008) have utilized wavelet analysis to detect vertical objects and classify buildings from LIDAR data points.(Filin, 2002) have used connectivity and principal component analysis to cluster LIDAR data in surface categories.
On the other hand, many approaches have been proposed to perform the classification task using aerial images.These approaches exhibit different features, models, and classifiers to accomplish the classification task.Several texture features have been used as an input to the classification stage, such as Gabor filter (Baik et al., 2004), fractal dimension and coefficient of variation (Solka et al., 1998), and Non Subsampled Contourlet Transform NSCT (Wei et al., 2010).The pixel color components have been used directly as the input to the classifier (Mokhtarzade et al., 2007).Both texture and color features have been used together for classification (Haim et al., 2006).A wide variety of classification algorithms have been employed, such as Naive Bayes classifier (Maloof et al., 2003), fuzzy logic (Sheng-hua et al., 2008), Neural Networks (Mokhtarzade et al., 2007), Support Vector Machine SVM algorithm (Corina et al., 2008).
Object Based Image Analysis (OBIA) gained recently a lot of attention among the geographical mapping applications as an alternative analysis framework that can avoid the drawbacks associated with pixel based analysis.In spite of the advantages of pixel based image analysis, it suffers from problems such as sensitivity to variations within objects significantly larger than pixel size (Alpin et al., 2008).The spatial extent of the objects to be classified is of more importance to the classification task than the spatial scale of image pixels (Platt et al., 2008).Object based classification can remarkably improve the classification accuracy by relieving the problem of misclassifying individual pixels (Alpin et al., 1999).
The proposed approach, presented in this paper, uses a single return LIDAR data along with aerial images to extract buildings, and trees of urban areas.Object based analysis is adopted to segment the entire DSM data into objects based on height variation.The classification task is based on two stages where the primary classified objects can help to derive new feature which is the height to ground for the second stage.Among the many features provided by the aerial imagery, a normalized difference vegetation index based on R and IR bands have been used due to its high significance in vegetation extraction.The second classification stage uses the object size, average height to ground, and the vegetation index to fine tune the classification of objects.
The following section demonstrates the steps of the proposed approach.Then, experimental results of the proposed approach for different urban areas are presented.Finally, the conclusions are provided.

METHODOLOGY
The proposed approach, adopts object based analysis where objects are the targets for classification.The first step is to perform image segmentation on DSM height image to divide the whole scene into objects.A region growing algorithm is conducted over the entire DSM height image starting from the upper left corner based on neighbourhood height similarity.The same traverse of data during object's extraction is exploited to calculate the area of each object to be used in the classification step.Based on the neighbourhood height similarity used in the segmentation step, the points of each extracted object tend to belong to the same object plane.Planar objects such as ground and building surfaces will exhibit large patches as they maintain smooth height changes.On the other hand, trees typically exhibit high variation of height due to the frequent LIDAR penetrations of its crowns.Consequently, trees areas exhibit small areas.As a preliminary classification, objects under minimum area threshold are classified as vegetation; this threshold represents the smallest expected area of a building object and was selected as 10 m 2 during our tests.The rest of objects are classified as buildings except for the largest object which is classified as ground.The largest object is used as a height reference, and the height to ground of each pixel of the rest of area is calculated as the difference between the pixel height and the nearest ground pixel height.Due to the interpolation applied to the LIDAR data, some walls of the buildings exhibit misleading high height variation that results in small patches misclassified as vegetation, the same misclassification is encountered for the architectural details of buildings as they also show abrupt height changes over small areas.These misclassifications are revised during the second classification stage.
For finding the corresponding spectral data of the extracted objects, an ortho-photo of the scene is constructed using all the overlapping images over the scene.All the ortho-rectified images that intersect with the scene boundary are merged together to obtain a true ortho-photo of the scene where the occluded or invisible areas in an ortho-photo is complemented by the other ortho-photos from the other images.Figure 1 illustrates sample ortho-photos of an area along with the merged true ortho-photo obtained.
The second stage of classification is conducted to tune the preliminary classification of the first stage according to the following rules:  Objects of high height-to-ground (>0.2) and high NDVI (>0.18) are classified as trees.


Objects of high height-to-ground (>0.2) and low NDVI (<0.18) are classified as buildings.


Objects that do not satisfy the previous two conditions maintain their preliminary classification.

RESULTS
To evaluate the proposed approach, both aerial images and LIDAR data of three urban areas in the centre of the city of Vaihingen are used for testing.These data sets are provided by ISPRS test project on urban classification and 3D building reconstruction.These areas have historic buildings with rather complex shapes, few high-rising residential buildings that are surrounded by trees, and a purely residential area with small detached houses.The digital aerial images are a part of the high-resolution DMC block of the DGPF test (Cramer, 2010).They were acquired using an Intergraph/ZI DMC by the company RWE Power on 24 July and 6 August 2008.In total, the block consisted of five overlapping strips with two additional cross strips at both ends of the block.The images are pan-sharpened colour infrared images with a ground sampling distance of 8 cm and a radiometric resolution of 11 bits with known interior and exterior orientation parameters.
The Vaihingen data set also contains Airborne Laser Scanner (ALS) data.The entire data set consists of 10 ALS strips acquired on 21 August 2008 by Leica Geosystems using a Leica ALS50 system with 45° field of view and a mean flying height above ground of 500 m.The average strip overlap is 30%, and the median point density is 6.7 points/m 2 .Point density varies considerably over the whole block depending on the overlap, but in regions covered by only one strip the mean point density is 4 points/m 2 .In addition to the original ALS point cloud, a digital surface model (DSM) is provided.This DSM was interpolated from the ALS point cloud with a grid width of 25 cm, using only the points corresponding to the last pulse.
To quantitatively evaluate the proposed approach, the obtained classification results are compared to reference data acquired using photogrammetric plotting.The evaluation is based on the technique described in (Rutzinger et al., 2009) that provides completeness, correctness, and quality of the results both on a per-object and on a per-area level.The 2D RMS error of the object outlines of the correct objects are also provided to be compared with those of the reference data.
Figure 3 shows the generated true ortho-photo for the first area, and Figure 4 illustrates the classification results of the buildings class for the first area.Figure 5 depicts the classification results of the trees class for the first area.
Figure 6 shows the generated true ortho-photo for the second area, and Figure 7 illustrates the classification results of the buildings class for the second area.Figure 8 depicts the classification results of the trees class for the second area.Table 1 gives the classification results of the buildings class for the three areas.The buildings misclassifications are noticed for objects under our proposed threshold for the minimum building area (10 m 2 ).Building parts under the ground level was also misclassified as our assumption is that buildings extend only above the ground.One building that is highly surrounded by vegetation was also misclassified.In most of the cases, the RMS of the extracted boundaries and the RMS of the coordinates of center of gravity of extracted buildings are less than or equal to the corresponding RMS of the reference data.

Classification Results
Area  2 gives the classification results of the trees class for the three areas.The reference data assumed trees to be of ideal circular shape.The proposed approach does not take this assumption into consideration and only considers the true boundary of trees based on LIDAR data which affects the classification results.In most of the cases, the RMS of the extracted boundaries and the RMS of the coordinates of center of gravity of extracted objects are less than or equal to the corresponding RMS of the reference data.

CONCLUSIONS
In this research, an object based classification approach has been presented that fuse both aerial imagery and LIDAR data.This object based analysis enabled a rule based classification where the decisions are based on clear and interpretable rules related to the scene parameters such as minimum building height and minimum building area.In the proposed approach, the classification has been performed on two phases where the first classification results help to provide the second phase with derived feature to help improve the classification accuracy.This iterative classification scheme could be further expanded to include more features based on the previous successive classification phases.The used thresholds are interpretable and could be easily changed to match the underlying scene for better classification results.The proposed classification rules are expandable to include more classes without reconstruction of the classifier from scratch.The achieved classification results show the significance of the proposed approach.
Figure 1.True ortho-photo generation Normalized Difference Vegetation Index (NDVI) is computed for all objects in the scene using IR and R bands of the generated true ortho-photo as in (1)

Figure 2 .
Figure 2. The three test areas shown on DSM

Figure 3 .Figure 5 .
Figure 3. Generated true ortho photo for the first area

Figure 8 .
Figure 8. Trees classification (correctly classified as yellow, misclassified as red, missing as blue)

Figure 11 .
Figure 11.Trees classification (correctly classified as yellow, misclassified as red, missing as blue)

Table 1 .
Classification results for the buildings class

Table 2 .
Classification results for the trees classTable