Soft computing strategy for stereo matching of multi spectral urban very high resolution IKONOS images

Abstract

This work aims to define a new strategy for extracting and stereo matching of buildings using very high resolution multi spectral IKONOS images having a ratio base/height about 0.53, we do not have the intrinsic and extrinsic parameters of the images acquisition system. These images contain dense urban scenes including various kinds of roads, cars, vegetation and buildings. We are interested by buildings, some of them have different shapes or colours and others have close colours or shapes, so, they generate a lot of “false matches”. To solve this issue, we propose in this paper an approach based on soft computing field in order to extract regions of interest (buildings) and to match them, it contains two main steps: region segmentation and thresholding step using a specific fuzzy thresholding algorithm and a neural Hopfield matching stage based on new constraints including geometric and photometric regions properties. The presented strategy is nearly all automatic, it is fast and simple and the results of its applied tests on several kinds of stereo dense urban images are satisfactory.

Introduction

In the last two decades, the stereo matching issue has attracted a lot of researches, and several approaches have been proposed [1], [2], [3], [4], [5], [6].

These approaches can be broadly classified into two categories: the area-based matching techniques and the feature-based matching techniques. In the first category, the matching process is applied directly to the intensity profiles of the two images, it allows an overall dense restitution but it manages with difficulties the discontinuities and homogeneous zones, the absence of semantic information harms a good management of coherence and scene structure [7]. In the second category, features are first extracted from the images and the matching process is applied to the features like edge elements [8], [9], the line segments [10], [11], [12] and the homogeneous regions [13], [14], [15], [16].

In this paper, we are interested by “buildings” which constitute a basic part of urban landscapes. They are keys elements of urban morphological analysis, so, buildings stereo matching supposes that the second category of approaches is the most appropriate in our application [17]. We choose “region” as a primitive because many of the shortcomings inherent in approaches based on points or lines can be overcome by taking more developed entities [18], [19], [20], [21], [22], [23]. The higher dimensionality of regions makes them richer of target object's geometric properties such as shape and size, and photometric properties such as colour. The higher dimensional character of regions also makes their matching more stable to small illumination and viewpoint changes across given images [15].

However, “a building” is so variable and it is not the only object in our pairs of high resolution multi spectral Ikonos images which include cars, roads, vegetation, etc. Also, shapes and colours of buildings can be close or different in the same image, from right to left image or from one pair of images to another which complicate more and more their extraction. Buildings extraction must be reliable, especially if we have to do other treatment after it, like stereo matching which depends on the quality of extraction results.

There are a lot of building extraction techniques applied to aerial or satellite images like [24] whom used laser remote sensing data to develop a method based on the standard deviation to distinguish between trees and buildings using the height variation at the periphery of the objects present in the data. Sohn and Dowman extracted buildings tracks automatically from a combination of the Ikonos imagery with pan-sharpened multi spectral bands and lidar data [25]. Lafarge and al presented an automatic buildings extraction method that involved digital elevation models based on an object approach. Using this method, a rough approximation of all relevant building footprints was first calculated from marked point processes. The resulting rectangular footprints were then normalized by improving the connections between neighboring rectangles and detecting any roof height discontinuities [26].

All these techniques cited above require a high computational effort or need other technological resources like digital elevation model, Lidar or Laser data, etc. To overcome these difficulties and in order to realize both buildings extraction and stereo matching, we propose in this paper a soft computing strategy able to exploit the given tolerance of imprecision, partial truth, and to achieve tractability, robustness and low solution cost.

Concerning buildings extraction step, we have to detect these last as only interesting regions in order to match them, so this process can be viewed as “color image thresholding problem”, for that, we propose an algorithm based on fuzzy logic (fuzzy clustering method) having the particularity to realize automatically at the same time both segmentation and thresholding. Compared to other tresholding techniques like: global thresholding (otsu method) and K-means thresholding, a proposed soft computing technique gives best results.

For stereo matching step, we choose a feature-based matching approach. It exists many optimisation techniques which allows to find homologous couples using soft computing, we mention for example relaxation method used by Brockers [27] and by Sidib [28], genetic algorithm used by Goulermas and Liatsis [29] and Hopfield neural network used by Jan Jae Lee to put in correspondence points [30], by Nasrabadi to put in correspondence characteristics points [31], by Nichari who uses as primitive the edge points [32] and by pajares and al which identified edge segments as features [33].

Generally, all these works mentioned above require some constraints to guide stereo matching process such as: the similarity, the continuity, the order and the epipolar constraint, etc. The implementation of these constraints is not always very easy, in particular the epipolar one which requires the knowledge of the intrinsic and extrinsic parameters of the acquisition system [34]. As we do not have these parameters and in order to overcome these restrictions, we are inspired by similarity constraint to propose in this work Hopfiel neural stereo matching technique using new constraints including geometric and photometric regions properties: surface, elongation, perimeter, colour and gravity center coordinates. This network will be initialized by simple method which we called classical matching technique.