Elsevier

Applied Soft Computing

Volume 12, Issue 8, August 2012, Pages 2566-2582
Applied Soft Computing

Interest point detection through multiobjective genetic programming

https://doi.org/10.1016/j.asoc.2012.03.058Get rights and content

Abstract

The detection of stable and informative image points is one of the most important low-level problems in modern computer vision. This paper proposes a multiobjective genetic programming (MO-GP) approach for the automatic synthesis of operators that detect interest points. The proposal is unique for interest point detection because it poses a MO formulation of the point detection problem. The search objectives for the MO-GP search consider three properties that are widely expressed as desirable for an interest point detector, these are: (1) stability; (2) point dispersion; and (3) high information content. The results suggest that the point detection task is a MO problem, and that different operators can provide different trade-offs among the objectives. In fact, MO-GP is able to find several sets of Pareto optimal operators, whose performance is validated on standardized procedures including an extensive test with 500 images; as a result, we could say that all solutions found by the system dominate previously man-made detectors in the Pareto sense. In conclusion, the MO formulation of the interest point detection problem provides the appropriate framework for the automatic design of image operators that achieve interesting trade-offs between relevant performance criteria that are meaningful for a variety of vision tasks.

Highlights

► Multi-objective genetic programming is applied to the problem of interest point detection. ► Evolved estimators are Pareto optimal solutions when compared to state-of-the-art methods. ► Results are validated on a large set of real-world images.

Introduction

Over the past decade, many researchers have developed computer vision systems that base their core functionality on the detection and description of sparse local features. The basic approach was introduced in [1], [2], and consists of the following steps. First, small image regions centered around salient image pixels, better known as interest points, are detected [3], [4]. Then, each region is described using compact numerical vectors that attempt to capture the main characteristics of local shape and appearance; these are called local image descriptors [5], [6], [7]. The set of local regions and their corresponding descriptors are used to construct models of objects or the scene captured within the image. Afterwards, when a new image is analyzed, this process is repeated and the extracted features are compared with the stored models. In this manner, a vision system can search for correspondences of local features; a simple example is depicted in Fig. 1. This approach provides several advantages for high-level systems: (1) traditional image segmentation, a difficult mid-level task, is not required; (2) the approach is robust to partial occlusions and to several types of geometric and photometric transformations; and (3) the total amount of image information is sharply reduced because only a subset of image regions are analyzed and described by a compact representation.

The performance of vision systems that are based on this approach will depend on the quality of the algorithms used to detect and describe the local features. In both cases, many proposals have been developed over the past thirty years, and most of them have been derived using standard techniques from computer vision research [8], [3]. Recently, however, some researchers have developed Genetic Programming (GP) algorithms for the automatic design of image operators that can detect [9], [10], [11], [12], [13], [14], [15] and describe [16], [17] local image features. The underlying hypothesis of these GP-based proposals is that current methods should not be considered to be optimal in any general sense and that the design of interest point detection can be accomplished through a machine learning approach based on genetic programming. Therefore, these works have proposed to use evolutionary algorithms to search for specialized image operators that can outperform the standard techniques.

Following this line of research, this paper presents a multiobjective GP (MO-GP) approach to design interest point operators. The main contribution is that the problem of interest point detection is formulated in terms of MO optimization, which is unique. Moreover, we show that MO-GP can produce a diverse set of detectors that achieve different trade-offs between the various performance criteria that are widely used to evaluate interest point detectors. Considering the problem domain, this paper is closely related to other GP applications in computer vision, of which several examples are reviewed next.

In computer vision, most problems are often ill-defined and considered to be intrinsically very difficult. Therefore, some researchers have attempted to reduce the complexity of designing vision systems by relying on automated methods, such as evolutionary computation [18]. In particular, GP has proven to be a useful paradigm in the development of vision algorithms because of its ability to directly generate specialized computer functions or mathematical operators [19], [20]. For instance, some have used GP to search for operators that can detect low-level features that were predefined by human experts, features that posses a clear semantic interpretation such as corners [9] or edges [21]. Also, GP has achieved promising results in measuring a more formal characteristic image structure, the Hölder regularity [22]. Others have used GP to construct domain-specific features that need not be interpretable by a human expert [19], [23], [24]. On the other hand, GP has been used to directly solve a specific visual task, such as image segmentation [25], object recognition and detection [26], [27], texture analysis [28] and content based retrieval [29]. Other recent examples are [16], [17] where GP is used to optimize a part of the SIFT algorithm [2], the most widely used method for region description.

However, all the above cited works pose the problem of evolving vision algorithms in terms of a single objective function. Therefore, despite their promising results, they are limited by the manner in which other objective criteria could be added to help guide the search, especially if conflicts exist among them. Moreover, the evolutionary process, which is normally computationally expensive, needs to be executed several times in order to possibly obtain more than a single type of solution. One manner in which these difficulties could be lessened, or eliminated, is to use a MO approach based on Pareto dominance relations. However, MO search techniques have not gained a wide acceptance within the vision community and only a small number of examples exist. For instance, in [30] a MO genetic algorithm is used to solve the sensor planning problem for vision metrology. Other examples include [31] where MO-GP is used to automatically learn simple visual concepts, and [32] where MO-GP is used to control program size for a feature extraction problem. More recently, in [15] we present a hybrid computer assisted design approach to develop a feature detection algorithm using genetic programming.

This paper presents a MO approach to synthesize image operators that detect interest points. The MO formulation allows us to integrate several performance criteria in a principled manner, and produce a non- dominated set of near-optimal solutions. The evolved operators provide different trade-offs among the various objectives, that are based on standard evaluation criteria. Moreover, experimental results validate the approach and illustrate the intrinsic MO nature of the interest point detection problem.

This work is a continuation of previous contributions we have developed on this subject. In [11], [14] interest point operators were synthesized by considering a single objective, and in [13], [15] only preliminary results of the MO approach were presented. Therefore, in order to contextualize our work, we list the main contributions made in this paper.

  • This paper studies the problem of interest point detection from the standpoint of MO optimization, an extension of the single objective approach followed in [11], [14] and a continuation of preliminary results published in [13], [15]. The proposal is unique in the MO treatment of this ubiquitous computer vision task providing novel insights for future research.

  • The MO-GP algorithm considers Pareto dominance relations and produces several sets of near-optimal operators. Three well-known performance criteria are considered: (1) Stability; (2) Point Dispersion; and (3) Information Content. Moreover, the MOGP can integrate more objectives in a straightforward and principled manner.

  • Finally, several novel detectors are presented, each exhibiting different trade-offs between each of the considered objectives. These operators provide novel tools for vision applications that outperform, in the Pareto sense, many previously proposed methods. Moreover, the performance of the evolved operators is validated with an extensive database of 500 test images of real world scenes.

The remainder of this paper is organized as follows. Section 2 presents an introduction to interest point detection, a review of previous proposals, and a description of the performance criteria used to evaluate them. Afterwards, Section 3 provides a brief overview of GP. In Section 4, we describe how the problem of interest point detection can be solved through GP. In particular, we review our previous single objective proposal and introduce the multiobjective approach. Then, Section 5 presents the experimental setup and results. Finally, a summary and concluding remarks are given in Section 6.

Section snippets

Interest point detection

Interest points are simple point features, image pixels that are salient or unique when compared with neighboring pixels. These features are used by a wide range of applications in image analysis, computer vision and photogrammetry [3], [33]. The algorithms used to detect interest points analyze the intensity patterns within local image regions and only make weak assumptions regarding the underlying structure [8]. For instance, another type of point feature are corners; however, these points

Overview of genetic programming

The field of evolutionary computation focuses on the development of search and optimization algorithms that are based on the core principles of Neo-Darwinian evolutionary theory [51]. Evolutionary algorithms are population-based meta-heuristics, where candidate solutions are stochastically selected and modified to produce new, and possibly better, solutions for a particular problem. The selection process favors individuals that exhibit the best performance, and the process is carried out

Previous work: single objective approaches

The first attempts to synthesize interest point operators with GP were presented by Ebner and colleagues [9], [10]. In [9], the goal was to evolve an operator that would detect the same points that are detected with the Moravec detector [53]. Then, in [10] the fitness function was based on the performance of a high-level system that computes the optical flow for a single image sequence. These proposals did produce promising results, but both works suffer from the fact that they fail to

Experimental results

Three goals are pursued with the experimental work:

  • 1.

    The first goal is to study the relationship between the three performance criteria in order to determine if a conflict exist between them. Therefore, the MO-GP is executed using every combination of two or more objectives: (1) Stability–Dispersion; (2) Stability–Information Content; (3) Dispersion–Information Content; and (4) Stability–Dispersion–Information Content.

  • 2.

    Second, when a Pareto front is obtained, we compare the evolved operators with

Summary and conclusions

In this paper, we present an approach based on mutiobjective genetic programming for the automatic synthesis of image operators that detect interesting image points. Interest point detectors are an important tool for many modern computer vision systems that perform tasks such as scene recognition, object detection and image indexing. However, no previous proposals have explicitly considered the multiobjective nature of this task. In the present work, a MO-GP is used to produce a variety of

Acknowledgments

This research was founded by CONACyT through the Project 155045 – “Evovisión de Cerebros Artificiales en Visión por Computadora”. Corresponding author was supported by scholarship 174785 from CONACYT during the development of parts of this research. Additionally, thanks are given to the support provided by the Departamento en Ingeniería Eléctrica y Electrónica from the Instituto Tecnológico de Tijuana.

References (61)

  • D.G. Lowe

    Object recognition from local scale-invariant features

  • T. Tuytelaars et al.

    Local invariant feature detectors: a survey

    Foundations and Trends in Computer Graphics and Vision

    (2008)
  • K. Mikolajczyk et al.

    A performance evaluation of local descriptors

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2005)
  • L. Trujillo et al.

    Regularity based descriptor computed from local image oscillations

    Optics Express

    (2007)
  • M. Heikkil et al.

    Description of interest regions with local binary patterns

    Pattern Recognition

    (2009)
  • C. Schmid et al.

    Evaluation of interest point detectors

    International Journal of Computer Vision

    (2000)
  • M. Ebner

    On the evolution of interest operators using genetic programming

  • M. Ebner et al.

    Evolving task specific image operator

  • L. Trujillo et al.

    Synthesis of interest point detectors through genetic programming

  • L. Trujillo et al.

    Scale invariance for evolved interest operators

  • L. Trujillo et al.

    Multiobjective design of operators that detect points of interest in images

  • L. Trujillo et al.

    Automated design of image operators that detect interest points

    Evolutionary Computation

    (2008)
  • C.B. Pérez et al.

    Learning invariant region descriptor operators with genetic programming and the f-measure

  • C.B. Perez et al.

    Evolutionary learning of local descriptor operators for object recognition

  • K. Krawiec

    Genetic programming-based construction of features for machine learning and knowledge discovery tasks

    Genetic Programming and Evolvable Machines

    (2002)
  • K. Krawiec et al.

    Overview of object detection and image analysis by means of genetic programming techniques

  • Y. Zhang et al.

    Evolving optimal feature extraction using multi-objective genetic programming: a methodology and preliminary study on edge detection

  • L. Trujillo et al.

    The estimation of hölderian regularity using genetic programming

  • K. Krawiec et al.

    Visual learning by coevolutionary feature synthesis

    IEEE Transactions on Systems, Man, and Cybernetics, Part B

    (2005)
  • Cited by (0)

    View full text