Interest point detection through multiobjective genetic programming
Graphical abstract
Highlights
► Multi-objective genetic programming is applied to the problem of interest point detection. ► Evolved estimators are Pareto optimal solutions when compared to state-of-the-art methods. ► Results are validated on a large set of real-world images.
Introduction
Over the past decade, many researchers have developed computer vision systems that base their core functionality on the detection and description of sparse local features. The basic approach was introduced in [1], [2], and consists of the following steps. First, small image regions centered around salient image pixels, better known as interest points, are detected [3], [4]. Then, each region is described using compact numerical vectors that attempt to capture the main characteristics of local shape and appearance; these are called local image descriptors [5], [6], [7]. The set of local regions and their corresponding descriptors are used to construct models of objects or the scene captured within the image. Afterwards, when a new image is analyzed, this process is repeated and the extracted features are compared with the stored models. In this manner, a vision system can search for correspondences of local features; a simple example is depicted in Fig. 1. This approach provides several advantages for high-level systems: (1) traditional image segmentation, a difficult mid-level task, is not required; (2) the approach is robust to partial occlusions and to several types of geometric and photometric transformations; and (3) the total amount of image information is sharply reduced because only a subset of image regions are analyzed and described by a compact representation.
The performance of vision systems that are based on this approach will depend on the quality of the algorithms used to detect and describe the local features. In both cases, many proposals have been developed over the past thirty years, and most of them have been derived using standard techniques from computer vision research [8], [3]. Recently, however, some researchers have developed Genetic Programming (GP) algorithms for the automatic design of image operators that can detect [9], [10], [11], [12], [13], [14], [15] and describe [16], [17] local image features. The underlying hypothesis of these GP-based proposals is that current methods should not be considered to be optimal in any general sense and that the design of interest point detection can be accomplished through a machine learning approach based on genetic programming. Therefore, these works have proposed to use evolutionary algorithms to search for specialized image operators that can outperform the standard techniques.
Following this line of research, this paper presents a multiobjective GP (MO-GP) approach to design interest point operators. The main contribution is that the problem of interest point detection is formulated in terms of MO optimization, which is unique. Moreover, we show that MO-GP can produce a diverse set of detectors that achieve different trade-offs between the various performance criteria that are widely used to evaluate interest point detectors. Considering the problem domain, this paper is closely related to other GP applications in computer vision, of which several examples are reviewed next.
In computer vision, most problems are often ill-defined and considered to be intrinsically very difficult. Therefore, some researchers have attempted to reduce the complexity of designing vision systems by relying on automated methods, such as evolutionary computation [18]. In particular, GP has proven to be a useful paradigm in the development of vision algorithms because of its ability to directly generate specialized computer functions or mathematical operators [19], [20]. For instance, some have used GP to search for operators that can detect low-level features that were predefined by human experts, features that posses a clear semantic interpretation such as corners [9] or edges [21]. Also, GP has achieved promising results in measuring a more formal characteristic image structure, the Hölder regularity [22]. Others have used GP to construct domain-specific features that need not be interpretable by a human expert [19], [23], [24]. On the other hand, GP has been used to directly solve a specific visual task, such as image segmentation [25], object recognition and detection [26], [27], texture analysis [28] and content based retrieval [29]. Other recent examples are [16], [17] where GP is used to optimize a part of the SIFT algorithm [2], the most widely used method for region description.
However, all the above cited works pose the problem of evolving vision algorithms in terms of a single objective function. Therefore, despite their promising results, they are limited by the manner in which other objective criteria could be added to help guide the search, especially if conflicts exist among them. Moreover, the evolutionary process, which is normally computationally expensive, needs to be executed several times in order to possibly obtain more than a single type of solution. One manner in which these difficulties could be lessened, or eliminated, is to use a MO approach based on Pareto dominance relations. However, MO search techniques have not gained a wide acceptance within the vision community and only a small number of examples exist. For instance, in [30] a MO genetic algorithm is used to solve the sensor planning problem for vision metrology. Other examples include [31] where MO-GP is used to automatically learn simple visual concepts, and [32] where MO-GP is used to control program size for a feature extraction problem. More recently, in [15] we present a hybrid computer assisted design approach to develop a feature detection algorithm using genetic programming.
This paper presents a MO approach to synthesize image operators that detect interest points. The MO formulation allows us to integrate several performance criteria in a principled manner, and produce a non- dominated set of near-optimal solutions. The evolved operators provide different trade-offs among the various objectives, that are based on standard evaluation criteria. Moreover, experimental results validate the approach and illustrate the intrinsic MO nature of the interest point detection problem.
This work is a continuation of previous contributions we have developed on this subject. In [11], [14] interest point operators were synthesized by considering a single objective, and in [13], [15] only preliminary results of the MO approach were presented. Therefore, in order to contextualize our work, we list the main contributions made in this paper.
- •
This paper studies the problem of interest point detection from the standpoint of MO optimization, an extension of the single objective approach followed in [11], [14] and a continuation of preliminary results published in [13], [15]. The proposal is unique in the MO treatment of this ubiquitous computer vision task providing novel insights for future research.
- •
The MO-GP algorithm considers Pareto dominance relations and produces several sets of near-optimal operators. Three well-known performance criteria are considered: (1) Stability; (2) Point Dispersion; and (3) Information Content. Moreover, the MOGP can integrate more objectives in a straightforward and principled manner.
- •
Finally, several novel detectors are presented, each exhibiting different trade-offs between each of the considered objectives. These operators provide novel tools for vision applications that outperform, in the Pareto sense, many previously proposed methods. Moreover, the performance of the evolved operators is validated with an extensive database of 500 test images of real world scenes.
The remainder of this paper is organized as follows. Section 2 presents an introduction to interest point detection, a review of previous proposals, and a description of the performance criteria used to evaluate them. Afterwards, Section 3 provides a brief overview of GP. In Section 4, we describe how the problem of interest point detection can be solved through GP. In particular, we review our previous single objective proposal and introduce the multiobjective approach. Then, Section 5 presents the experimental setup and results. Finally, a summary and concluding remarks are given in Section 6.
Section snippets
Interest point detection
Interest points are simple point features, image pixels that are salient or unique when compared with neighboring pixels. These features are used by a wide range of applications in image analysis, computer vision and photogrammetry [3], [33]. The algorithms used to detect interest points analyze the intensity patterns within local image regions and only make weak assumptions regarding the underlying structure [8]. For instance, another type of point feature are corners; however, these points
Overview of genetic programming
The field of evolutionary computation focuses on the development of search and optimization algorithms that are based on the core principles of Neo-Darwinian evolutionary theory [51]. Evolutionary algorithms are population-based meta-heuristics, where candidate solutions are stochastically selected and modified to produce new, and possibly better, solutions for a particular problem. The selection process favors individuals that exhibit the best performance, and the process is carried out
Previous work: single objective approaches
The first attempts to synthesize interest point operators with GP were presented by Ebner and colleagues [9], [10]. In [9], the goal was to evolve an operator that would detect the same points that are detected with the Moravec detector [53]. Then, in [10] the fitness function was based on the performance of a high-level system that computes the optical flow for a single image sequence. These proposals did produce promising results, but both works suffer from the fact that they fail to
Experimental results
Three goals are pursued with the experimental work:
- 1.
The first goal is to study the relationship between the three performance criteria in order to determine if a conflict exist between them. Therefore, the MO-GP is executed using every combination of two or more objectives: (1) Stability–Dispersion; (2) Stability–Information Content; (3) Dispersion–Information Content; and (4) Stability–Dispersion–Information Content.
- 2.
Second, when a Pareto front is obtained, we compare the evolved operators with
Summary and conclusions
In this paper, we present an approach based on mutiobjective genetic programming for the automatic synthesis of image operators that detect interesting image points. Interest point detectors are an important tool for many modern computer vision systems that perform tasks such as scene recognition, object detection and image indexing. However, no previous proposals have explicitly considered the multiobjective nature of this task. In the present work, a MO-GP is used to produce a variety of
Acknowledgments
This research was founded by CONACyT through the Project 155045 – “Evovisión de Cerebros Artificiales en Visión por Computadora”. Corresponding author was supported by scholarship 174785 from CONACYT during the development of parts of this research. Additionally, thanks are given to the support provided by the Departamento en Ingeniería Eléctrica y Electrónica from the Instituto Tecnológico de Tijuana.
References (61)
- et al.
Interest point detection using imbalance oriented selection
Pattern Recognition
(2008) - et al.
Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming
Image and Vision Computing
(2011) - et al.
Object detection in multi-modal images using genetic programming
Applied Soft Computing
(2004) - et al.
Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance
Pattern Recognition Letters
(2006) Generative learning of visual concepts using multiobjective genetic programming
Pattern Recognition Letters
(2007)- et al.
A new accurate and flexible model based multi-corner detector for measurement and recognition
Pattern Recognition Letters
(2005) - et al.
Gray-level corner detection
Pattern Recognition Letters
(1982) - et al.
Assessing the performance of corner detectors for point feature tracking applications
Image and Vision Computing
(2004) - et al.
Performance evaluation of corner detectors using consistency and accuracy measures
Computer Vision and Image Understanding
(2006) - et al.
Local gray value invariants for image retrieval
IEEE Transactions on Pattern Analysis and Machine Intelligence
(1997)