A penalty algorithm for solving convex separable knapsack problems

Abstract

In this paper, we propose a penalized gradient projection algorithm for solving the continuous convex separable knapsack problem, which is simpler than existing methods and competitive in practice. The algorithm only performs function and gradient evaluations, sums, and updates of parameters. The relatively complex task of the algorithm, which consists in minimizing a function in a compact set, is given by a closed formula. The convergence of the algorithm is presented. Moreover, to demonstrate its efficiency, illustrative computational results are presented for medium-sized problems.

Introduction

We are interested in solving The Continuous Convex Separable Knapsack Problem:

$$\left(CSKP\right):\{\begin{array}{c}{\displaystyle \text{Minimize}f\left(z\right):=\sum _{i=1}^n{f}_i\left(z_i\right),\text{s.t.}}\hfill \\ {\displaystyle \sum _{i=1}^n{b}_i{z}_i=c,}\hfill \\ l_i\le z_i\le u_i,i=1,2,\dots ,n,\hfill \end{array}$$

where $f_i:\mathbb{R}\to \mathbb{R}$ are differentiable and convex functions and b_i > 0, l_i < u_i for all $i=1,2,\dots ,n$ with c > 0 such that ⟨b, l⟩ ≤ c ≤ ⟨b, u⟩, see, for example, [1]. In our approach, we are interested in the problems where $c-〈b,l〉>0$.

The knapsack is a thoroughly studied problem in past literature. A survey regarding algorithms and applications for the nonlinear knapsack problem, also known as the nonlinear resource allocation problem, is presented in [1]. The problem considered by Bretthauer and Shetty [1] is somewhat more general than the problem (1) and as was indicated by the authors, and references therein, there is a variety of applications regarding the knapsack problem; included among them are financial models, production and inventory management, stratified sampling, optimal design of queuing network models in manufacturing, computer systems, subgradient optimization, and health care.

The paper [2] surveys the history and applications of the problem of minimizing a separable, convex and differentiable function over a convex set, defined by bounds on the variables and an explicit constraint described by a separable convex function, as well as algorithmic approaches to its solution. They found that the most common techniques are based on finding the optimal value of the Lagrange multiplier for the explicit constraint, most often through the use of a type of line search procedure. They analyze the most relevant references, especially regarding their originality and numerical findings, summarizing with remarks on possible extensions and future research. More recently in [3] it provides an up-to-date extension of the survey of the literature of the field, complementing the survey in [2] with more then 20 books and articles, totaling over one hundred references methodically analyzed. Besides of that they contributes with an improvement of the pegging (that is, variable fixing) process in the relaxation algorithm, and an improved means to evaluate sub-solutions. Finally they provided a rigorous numerical evaluation of several relaxation (primal) and breakpoint (dual) algorithms, incorporating a variety of pegging strategies, as well as a quasi-Newton method.

With regard to the continuous knapsack problem, which is the object of study of this paper, certain well known methods exist that deserve significant attention. With few variations on its formulation, the most known are multiplier search methods and variable pegging methods. The last one is also called variable fixing techniques.

A substantial portion of papers is focused on a particular case of the problem (1), when the objective function is quadratic with continuous or integer variables. This formulation is called quadratic knapsack problem, including, between them, the support vector machines for training. In this case, the primary techniques used for solving this problem are quasi-Newton and Newton methods, projected gradient method, branch and bound, and relaxation techniques among others, for example [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14].

For the non-quadratic case, relatively less research papers compared with the quadratic case, some of them found in the literature are [1], [3], [15], [16], [17].

In this paper, we present an algorithm for non-quadratic continuous convex separable knapsack problem. However, it includes a particular case of the quadratic problem. We start by reformulating the problem (1) as a penalized upper unbounded problem restricted to the unit simplex. Thus, as proposed by Beck and Teboulle [18], [19], we use a Bregman distance to define a penalized algorithm with explicit projections. Convergence analysis and numerical results are also presented.

The paper is organized as follows. In Section 2, we present the theoretical basis of our work, i.e., a change of variables and explicit projections on the unit simplex. Section 3 is devoted to introducing our algorithm and its convergence analysis. In Section 4, we report our numerical experiments. Finally, we infer overall conclusions in Section 5.