GAUSS: an online algorithm selection system for numerical quadrature

doi:10.1016/S0965-9978(01)00046-1

Advances in Engineering Software

Volume 33, Issue 1, January 2002, Pages 27-36

https://doi.org/10.1016/S0965-9978(01)00046-1 Get rights and content

Abstract

We describe the design and implementation of GAUSS — an online algorithm selection system for numerical quadrature. Given a quadrature problem and performance constraints on its solution, GAUSS selects the best (or nearly best) algorithm. GAUSS uses inductive logic programming to generalize a database of performance data; this produces high-level rules that correlate problem features with algorithm performance. Such rules then serve as the basis for recommending algorithms for new problem instances. GAUSS functions online (new data and information can be incrementally incorporated) and can also provide phenomenological explanations of algorithm recommendations.

Introduction

The subject of this paper is GAUSS — an automatic algorithm recommender system for numerical quadrature. Given a quadrature problem and performance constraints on its solution, GAUSS selects the best (or nearly best) algorithm. The approach we take is to organize a database of test problems and quadrature algorithms, and to accumulate performance data for the given population. This database of performance data is then mined (generalized) to arrive at high-level rules that can form the basis for a recommendation (for future problems).

GAUSS demonstrates a novel approach to the design and implementation of algorithm recommender systems. Traditionally, recommender systems generalize performance data offline and have been restrictive in their coverage of the application domain (e.g. the use of weak learning methods such as neural network training). GAUSS however uses relational descriptions for domain modeling and interacts dynamically with its environment to gather the data needed to mine for recommendations. It is thus an online recommender system. New information about problems and algorithms can be easily incorporated without having to retrain on the old data. In addition, the end-user of the recommender system can query the system for the basis of the recommendations (relational rules that correlate the effect of problem features on algorithm performance).

Two ideas are critical in our presentation below:

•
GAUSS does not evaluate integrals (symbolically or numerically). In addition, it does not require a symbolic form of the integrand, though it can make use of symbolic information if available. The goal of recommender systems is to help novice users or to be used in a completely automatic environment (such as problem solving environments [10]).
•
Systems like GAUSS typically have two phases of development and operation. The first is the collection and generalization of performance information (to yield rules, decision procedures, etc.) while the second is the actual process of algorithm recommendation (and incorporating any feedback/changes). The first phase is time consuming but needs to be done only once. The second phase is comparatively fast, inexpensive, and can be performed as many times as desired.

Section 2 provides a quick overview of the application domain considered in this study and identifies the difficulties in algorithm recommendation. The design of the GAUSS system is detailed in Section 3. Complexity and various practical implementation considerations are described here. Section 4 presents a careful experimental evaluation of the system. Section 5 identifies several important future research directions.

Section snippets

Numerical quadrature

The algorithm recommendation problem addressed by GAUSS is to:

Select an algorithm to evaluate $I= ∫ a b f(x) d x$
so that relative error ϵ_r<θ and N is minimized

where θ is an user-specified error requirement and N is the number of times f(x) is evaluated in [a,b] to yield the desired accuracy. Most quadrature algorithms evaluate I as [6]:

∫ a b f(x) d x≈w_{1} f(x_{1})+w_{2} f(x_{2})+⋯+w_{n} f(x_{n}),

−∞≤a≤b≤+∞

We choose ϵ_r and N as performance criteria because:

•
For most software implementations of integration routines, an absolute

The design of GAUSS

We outline the design of the GAUSS system with an example. Consider the effect of the QUADPACK routine QNG [18] on the integral (Fig. 2, left)³ $∫ 01 x^{1/2} log (x) d x=− 49$

When the relative error requirement is reduced from 0.1 down to 5×10

Experimental studies

In this section, we outline the experimental framework for the evaluation of GAUSS. We have utilized 124 routines from the GAMS cross-index [4], where the category H2a is for the evaluation of one dimensional integrals. In the experiments described below, methods for principal value integrals, interval analysis techniques, parallel methods of numerical integration, Monte Carlo methods and number theoretic methods are excluded. The integrand is also assumed to be either (i) a function available

Extensions

There are several possible extensions to the work presented here:

•
The representational formalism can be enhanced to include so-called description logics or terminological logics that provide more functionality than the logic formalism employed here. In particular, this will permit constructive induction where rules of the form ‘If two or more of these features are present, then use algorithm X’ can be induced. The scheme presented in this paper is a proper subset of this language.
•
The study can

Concluding remarks

The eventual success and acceptance of algorithm recommender systems rely on the expressiveness of their representation(s) and their ability to reason efficiently (and accurately) with such representations. We have shown how GAUSS achieves these objectives by (i) a direct control over the data-generation process, (ii) a rich representation language for recommendation rules and domain-specific context information, and (iii) the use of ILP for generalization. The online capability of systems like

Acknowledgements

We acknowledge suggestions from three anonymous reviewers whose comments greatly improved the presentation and readability of this paper. This research was supported in part by the National Science Foundation (CCR-9311486; EIA-9984317), the Defense Advanced Research Projects Agency through the Army Research Office (DAAH04-94-G-0010), Intel Corporation, and the Purdue Research Foundation.

References (23)

C. de Boor
CADRE: an algorithm for numerical quadrature
F.S. Acton
Real computing made REAL
(1996)
R. Baier et al.
Partial evaluation of numerical programs in FORTRAN
(1994)
A. Berlin et al.
Compiling scientific code using partial evaluation
IEEE Comput
(1990)
Ronald F. Boisvert. The NIST guide to available mathematical software. URL: http://gams.nist.gov,...
I. Bratko et al.
Applications of inductive logic programming
Commun ACM
(1995)
P.J. Davis et al.
Methods of numerical integration
(1984)
L. de Raedt et al.
Using logical decision trees for clustering
(1997)
S. Dzeroski et al.
Relational reinforcement learning
(1998)
E. Gallopoulos et al.
Computer as thinker/doer: problem-solving environments for computational science
IEEE Comput Sci Engng
(1994)

E.N. Houstis et al.

PYTHIA-II: a knowledge/database system for managing performance data and recommending scientific software

ACM Trans Math Software

(2000)

Cited by (11)

Exploring the role of graph spectra in graph coloring algorithm performance
2014, Discrete Applied Mathematics
Citation Excerpt :
These ideas date back to the 1970s, when the Algorithm Selection Problem (ASP) was first posed by Rice [49], and formulated as the task of learning the relationship between a set of features describing problem classes, and the performance of algorithms. The approach was then applied to the task of selecting the best solver for partial differential equations and numerical quadrature [61,32,33,47]. Independently, the same ideas have been applied in the machine learning community, to select the best algorithm for solving classification [10,2] and prediction problems [46,63]—a sub-field known as meta-learning [62] since the approach is to learn about learning algorithm performance.
This paper considers the challenge of recognizing how the properties of a graph determine the performance of graph coloring algorithms. In particular, we examine how spectral properties of the graph make the graph coloring task easy or hard. We address the question of when an exact algorithm is likely to provide a solution in a reasonable computation time, when we are better off using a quick heuristic, and how the properties of the graph influence algorithm performance. A new methodology is introduced to visualize a collection of graphs in an instance space defined by their properties, with an optimal feature selection approach included to ensure that the instance space preserves the topology of an algorithm performance metric. In this instance space we can reveal how different algorithms perform across a variety of instance classes with various properties. We demonstrate the methodology using a sophisticated branch and bound exact algorithm, and the faster heuristic approaches of integer rounding of a linear programming relaxation of the graph coloring formulation, as well as a greedy degree-saturation heuristic. Our results demonstrate that spectral properties of the graph instances can provide useful descriptions of when each of these algorithms will provide the best solution for a given computational effort.
Towards objective measures of algorithm performance across instance space
2014, Computers and Operations Research
This paper tackles the difficult but important task of objective algorithm performance assessment for optimization. Rather than reporting average performance of algorithms across a set of chosen instances, which may bias conclusions, we propose a methodology to enable the strengths and weaknesses of different optimization algorithms to be compared across a broader instance space. The results reported in a recent Computers and Operations Research paper comparing the performance of graph coloring heuristics are revisited with this new methodology to demonstrate (i) how pockets of the instance space can be found where algorithm performance varies significantly from the average performance of an algorithm; (ii) how the properties of the instances can be used to predict algorithm performance on previously unseen instances with high accuracy; and (iii) how the relative strengths and weaknesses of each algorithm can be visualized and measured objectively.
Measuring instance difficulty for combinatorial optimization problems
2012, Computers and Operations Research
Discovering the conditions under which an optimization algorithm or search heuristic will succeed or fail is critical for understanding the strengths and weaknesses of different algorithms, and for automated algorithm selection. Large scale experimental studies – studying the performance of a variety of optimization algorithms across a large collection of diverse problem instances – provide the resources to derive these conditions. Data mining techniques can be used to learn the relationships between the critical features of the instances and the performance of algorithms. This paper discusses how we can adequately characterize the features of a problem instance that have impact on difficulty in terms of algorithmic performance, and how such features can be defined and measured for various optimization problems. We provide a comprehensive survey of the research field with a focus on six combinatorial optimization problems: assignment, traveling salesman, and knapsack problems, bin-packing, graph coloring, and timetabling. For these problems – which are important abstractions of many real-world problems – we review hardness-revealing features as developed over decades of research, and we discuss the suitability of more problem-independent landscape metrics. We discuss how the features developed for one problem may be transferred to study related problems exhibiting similar structures.
A sensor fusion framework for online sensor and algorithm selection
2008, Robotics and Autonomous Systems
This paper presents a sensor fusion framework for selecting online the most reliable logical sensors and the most suitable algorithm for fusing sensor data in a robot platform. The framework is rule-based, employing the concept of using the simplest sensor fusion algorithm with the most reliable sensors. The framework is realized by implementing measures that were developed to quantify online sensor performance. Statistical, histogram, time series and graphical analyses demonstrate the advantages of this new framework.
Object-oriented numerical integration - A template scheme for FEM and BEM applications
2006, Advances in Engineering Software
An object-oriented numerical integration template implementation is presented on the basis of the C++ programming language. Aiming its straightforward application in finite and boundary element methods, the design supports integrand objects of scalar, vector or matrix types, so that a single programming statement is able to integrate element matrices and vectors. The integrand can contain singularities like the ones typically found in boundary element methods, allowing the evaluation of both regular and singular integrals under the same programming structure. The use of the proposed design is illustrated through some elementary applications as well as finite element and boundary element code excerpts.
Novel runtime systems support for adaptive compositional modeling in PSEs
2005, Future Generation Computer Systems
Citation Excerpt :
Consider how the interaction between Weaves and runtime recommender systems would work for the task of adaptive numerical quadrature. Let us start with the collection of 120 quadrature algorithms described in [24]. For a given numerical integration problem, the performance goal is to recommend a suitable quadrature routine such that the number of function evaluations is minimized.
Complex problem solving environments (PSEs) rely on runtime systems support for adaptive composition and control of scientific computations. This paper presents a PSE runtime support solution through a novel combination of two technologies—Weaves, a source language independent parallel compositional framework that operates through reverse-analysis of compiled object files, and runtime recommender systems that aid in dynamic knowledge-based application composition. Domain-specific adaptivity is realized by recommendation of code modules, supported by a sophisticated checkpointing framework for runtime control. A core set of “adaptivity schemas” are provided as templates for adaptive composition of large-scale scientific computations. Implementation issues, motivating application contexts, and preliminary results are described.

View all citing articles on Scopus

View full text

GAUSS: an online algorithm selection system for numerical quadrature

Abstract

Introduction

Section snippets

Numerical quadrature

The design of GAUSS

Experimental studies

Extensions

Concluding remarks

Acknowledgements

Real computing made REAL

Partial evaluation of numerical programs in FORTRAN

Compiling scientific code using partial evaluation

IEEE Comput

Applications of inductive logic programming

Commun ACM

Methods of numerical integration

Using logical decision trees for clustering

Relational reinforcement learning

Computer as thinker/doer: problem-solving environments for computational science

IEEE Comput Sci Engng

PYTHIA-II: a knowledge/database system for managing performance data and recommending scientific software

ACM Trans Math Software