On the experimental investigation of Pareto–Lipschitzian optimization

. A well-known example of global optimization that provides solutions within ﬁxed error limits is optimization of functions with a known Lipschitz constant. In many real-life problems this constant is unknown. To address that, we propose a novel method called Pareto Lipschitzian Optimization (PLO) that provides solutions within ﬁxed error limits for functions with unknown Lipschitz constants.In the proposed approach, a set of all unknown Lipschitz constants is regarded as multiple criteria using the concept of Pareto Optimality (PO).


Introduction
The target of this paper is to discuss the results of experimental calculations. The theoretical part of PLO is described and some computing results are presented in [6]. The description of the PLO algorithm is in [6], too. We compare PLO to the existing family of the DIRECT algorithms [3,1,2,4]. The DIRECT algorithms considers only a small subset of PO decisions that are selected by a heuristic rule depending on an adjustable parameter. It means that some PO decisions are preferred to others. In contrast, PLO regards all PO decisions without preferences and is naturally suited to utilize highly parallel computing.

Pareto-Lipschitzian Optimization (PLO)
We start to explain the optimization of Lipschitz functions with unknown constants by considering this one-dimensional example.
Suppose that the interval D = [a, b] ∈ R is partitioned into intervals [a i , b i ], i = 1, . . . , I of lengths l i = b i − a i with midpoints c i = (b i + a i )/2 and the values of the function f ω (x) are known only at the midpoints c i . The unknown Lipschitz constants ω are regarded as different components of multiple criteria. The variables x are represented by the intervals a i x b i and the function f ω (x) is approximated by the lower bounds: (1) f (c i ) − ω l i /2 < f (c j ) − ω l j /2, for at least one ω ∈ Ω. (2) Expressions (1), (2) show that the lower bound of the interval i is increasing with f (c i ) and decreasing with l i for all ω.
In [7] the set of Pareto Optimal (PO) intervals is defined as follows: The PLO algorithm is described in [6] using corresponding theorems 1 and 2.

Comparison of PLO with the DIRECT algorithm
The DIRECT algorithm [3,2] for Lipschitzian optimization with unknown constants is defined as a heuristic without any references to the theory of vector optimization or Pareto optimality. However, it can be explained in terms of PLO as well. The basic idea of DIRECT is to select (and sample within) all Potentially Optimal (PTO) intervals during an iteration. A formal definition of PTO intervals follows.
Here ǫ > 0 is a constant that defines the size of the set of PTO intervals, and f min is the current best value.
One can see that inequality (3) includes all the intervals which are among the best for at least one ω > 0. It means that these intervals are not dominated, thus, they belong to the PO set as well. However, condition (3) defines only a subset of not dominated intervals. The further restriction of this part of PO intervals is provided by condition (4) which depends on the parameter ǫ. Thus the actual performance of DIRECT algorithm is determined by this parameter.

Extension to several dimensions
We define the length l i of the interval a k i z k b k i , k = 1, . . . , K in a compact subset D ∈ R K as the longest length: Then the definition of PO-intervals remains the same.

Sampling
The sampling procedure of the PLO algorithm [6] is similar to that of the DIRECT algorithm [2]. However, the definition of Pareto Optimal intervals, used in PLO, is different from the definition of Potentially Optimal intervals in the DIRECT algorithm.
In this paper, three models that simulates real optimization problems [5] are investigated. The first one, called "Eco Duel", in short, is about a differential game that represents the competition of two servers, using the concept of Nash equilibrium. Here eight optimization variables are used. The second five-dimensional model optimizes the "mixture" of five heuristics for packing rectangular boxes of different size into the container and is called "Packer", in short. The third model "ModelTask" minimizes the deviation of the neuron gate model from the experimental results [8]. Table 1 illustrates the sample of PLO, BA, and MC comparison. In the all instances PLO was the best. BA was better MC with exception of the "ModelTask" model. Figure 1 shows the results of optimal packing of hundred different rectangular packages into the rectangular container. In this example, both PLO and BA were used to optimize the mixture of five greedy heuristics, the function was the total volume of packages in the container. Table 2 illustrates the comparison the results of Monte Carlo (MC), PLO, DIRECT obtained using standard test functions of global optimization [3,1,4]. MC and PLO represent methods intended for parallel computing with no adjustable parameters. The version of DIRECT is for sequential realization with one adjustable parameter ǫ selected by the method authors. This means that we compare algorithms representing different families of optimization methods.
DIRECT was better than PLO for 1 test function: Shekel, m=10. For the function Hartman-3, the results were equal. PLO was better than DIRECT for 5 functions Hartman-6, Brcos, GolPri, and SixH.

Summary
The theoretical novelty of PLO is the definition of the problem of Lipschitzian optimization with unknown Lipschitz constants in terms of Pareto optimality (PO).  The computational contribution is implementation the Pareto-Lipschitzian Optimization (PLO) algorithm as the Java applet with experimental calculations illustrating the PLO efficiency for functions up to 20 variables. We expect that in the future, the extensive computer simulation with various test and real functions will reveal additional aspects of the proposed algorithm and that would be an interesting new investigation.