SOLVING QUADRATIC PROGRAMMING PROBLEM WITH LINEAR CONSTRAINTS CONTAINING ABSOLUTE VALUES

In this paper the quadratic programming problem with linear constraints containing absolute values of variables (QPPLCAV) is considered. Hessian matrix is presumed to be positive definite. The problem is transformed to the larger problem with double number of variables with the same number of linear constraints without absolute values and with additional nonnegativity conditions (one inequality containing n absolute values could be ‘directly’ substituted by the system of 2n inequalities without absolute values). This problem may have several solutions. The relations between the original and the transformed problems are studied. In order to obtain stable approximations to the normal solution to the transformed problem corresponding to the unique solution of the original problem a regularization technique is proposed. A numerical example is given.


INTRODUCTION
Quadratic programming (QP) is the simplest nonlinear programming problem.An extensive research in this area starts with the pioneering work of [12].In the paper [3] a comprehensive summary of quadratic programming bibliography (including items corresponding to electrotechnics and informatics, e.g., digital filter design or using Neural Networks to solve LP and QP problems) have been published, and quadratic programming codes have been listed at [4].Authors refer to H. Mittelmann's QP solvers site [14].The numerical optimization including QP can be found, e.g., in books [9,15].A repository of convex quadratic programming problems is introduced in [13].A survey on methods for solving the general quadratic programming problem is presented in [16].There are numerous algorithms for the convex QP, see, e.g., [2, 5-8, 10, 11, 18, 21, 23-26].The comparison of five QP algorithms is given in [19].
Usually, QP problems with linear equality and inequality constraints are considered.In [1] optimal solutions to quadratic programs with quadratic constraints and inequality constraints expressed by means of l p -norms are studied.We have not found papers dealing with linear constraints containing absolute values.
This paper is organized as follows.In Section 2 a brief motivation is given.In Section 3 the quadratic programming problem with linear constraints containing absolute values of variables (QPPLCAV) is introduced.Section 4 contains auxiliary theoretical results necessary to prove the main results.In Section 5 we propose a stable method for solving QPPLCAV.We transform the QPPLCAV to the QP problem with linear constraints without absolute values at a cost of doubling the number of variables, adding only nonnegativity constraints.To stabilize the solution of the transformed problem we introduce a regularization method.A numerical example is given in Section 6.
Let the corresponding relative investments v 1 , v 2 , . . ., v m in these pairs fulfil the constraint where q j > 0 is the margin requirement associated with the j-th pair.
Wealth fractions w i (where n ∑ i=1 w i = 1) invested in particular assets are related to v k by equations where the first asset represents the deposit currency.
An optimization portfolio problem

PROBLEM FORMULATION
Above we have arrived to the problem of the form: , where q j > 0 (6) for all j = 1, n, and C = C > 0 is a symmetric positive definite matrix of order n ( denotes matrix transposition, > 0 stands for positive definitivity of a matrix).
Let us formulate a more general problem which we will denote QPPLCAV: where Q and P are matrices od the size k × n, and s ∈ R k .All elements of matrix Q we consider nonnegative.Inequalities from (6) can be included in (9) using zero rows in matrix Q. Zero rows of matrix P correspond to the inequalities containing only absolute values, corresponding right side values s i we consider positive.
Inequalities in both problems (5)-( 6) and ( 7)-( 8) could be rewritten in form of linear inequalities without absolute values (one inequality containing n absolute values could be 'directly' substituted by the system of 2 n inequalities without absolute values).But the number of such inequalities will rapidly increase for large n even if k -the number of inequalities -is small.
We will transform the problem QPPLCAV to the QP problem of order 2n with the same number of equalities and inequalities adding nonnegativity conditions.Remark 3.1.If we allow the element q i j to be negative, the inequality (9) may define a non-convex domain in R n .For example, consider inequality or simply in a component form The solution to this inequality is a non-convex subset of R 2 , see Fig. 1.

THEORETICAL PRELIMINARIES
In this section we will formulate and prove some auxiliary results from the matrix theory.Lemma 4.1.Let C be a symmetric positive definite matrix of order n, with the eigenvector v corresponding to the eigenvalue λ .Denote C a block matrix of the form Then the matrix C has an eigenvector [v , −v ] corresponding to the eigenvalue 2λ .All eigenvectors of the matrix C corresponding to zero eigenvalues are of the form [w , w ] , w ∈ R n .
Proof.We only multiply matrix C by vector [v , −v ] and we get: So the truth of the first part of claim is evident.Now, suppose that matrix C has an eigenvector of the form [w 1 , w 2 ] , w 1 , w 2 ∈ R n , corresponding to the zero eigenvalue.Then we have Proof.It is evident from the form of the matrix C, that its rows in the lower half are the rows from upper half multiplied by −1.So, because the rank of the matrix C is n, the rank of the matrix C is n, too.Therefore it has n zero eigenvalues.The second part of the statement is direct consequence of Lemma 4.1.
Corollary 4.2.Let C be a symmetric positive definite matrix of order n.Denote C a block matrix of the form (10). Then matrix C is symmetric positive semidefinite matrix.
Proof.The truth of the assertion is evident from (10) and from Corollary 4.1.
In the following we will work with special modifications of the matrix C. We will now study their properties.Lemma 4.2.Let C be a matrix of the form (10) with symmetric positive definite matrix C. Let where I 2n is the identity matrix of size 2n.Then C α is symmetric positive definite matrix with n minimal eigenvalues equal to α.
where vectors v i ∈ R 2n , i = 1, 2n generate an orthonormal basis of the space R 2n , and µ i are nonnegative eigenvalues of the matrix C. Therefore we have n eigenvalues of matrix C are zero, say µ i , i = 1, n. Corresponding eigenvalues of matrix C α are then equal Lemma 4.3.Let C α be a matrix of the form where C is a symmetric positive definite matrix of order n with eigenvalues λ i > 0 and corresponding eigenvectors v i ∈ R n , i = 1, n, and I n is the identity matrix of order n.Let {w i ∈ R n , i = 1, n} be an arbitrary basis of R n .Then the matrix C α has: (i) n eigenvectors of the form v i −v i corresponding to the eigenvalues 2λ i , i = 1, n; (ii) n eigenvectors of the form w i w i corresponding to the eigenvalues equal to 2α, i = 1, n.
Proof.(i) Direct substitution leads to Proof.The claim follows from Lemma 4.2 and Lemma 4.3, respectively.

Transformation of the QPPLCAV problem
As we have mentioned before, in the space R n one inequality with absolute values may be written in a form of 2 n inequalities without absolute values.Even for not very large size n, the number of inequalities may be ultra large.
Therefore we will adopt an idea from [17], where each value x i is considered as the difference of two values x + i − x − i .Specifically, for given value x i let us denote two values: It is evident that both x + i and x − i are nonnegative.Moreover for any value x i one of these values will be zero: if x i ≥ 0, then x − i = 0, if x i ≤ 0, then x + i = 0.It is evident that from Eq. (13) we have also Remark 5.1.Equalities (14) hold only if at least one of the values x + i and x − i is zero.For example, if we take The second number 5 is not the absolute value of the first number −3.
If we calculate 'positive' and 'negative' parts for every component of vector x, we will end up with the corresponding vectors x + and x − .Let us substitute in the problem QPPLCAV for vectors x and |x| values: We arrive at the problem: minimize Remark 5.2.We will not add here the request x + i • x − i = 0 for every i = 1, n.We will discuss this problem later.
We have obtained the transformed problem TQPPLC in the space R 2n with linear constraints without absolute values: Remark 5.3.According to the Corollary 4.1 Hessian matrix C (10) of the problem TQPPLC is positive semidefinite, so the problem TQPPLC may have more then one solution.

Equivalence of the problems QPPLCAV and TQP-PLC
Problems QPPLCAV and TQPPLC are not completely equivalent.The problem QPPLCAV has a unique solution, if its feasible set is nonempty.In such a case the problem TQPPLC may have several solutions.However, in this case there always exists the solution of the problem TQPPLC corresponding to the solution of the problem QPPLCAV.(i) Any feasible solution x of the problem QPPLCAV corresponds to the feasible solution of the problem TQPPLC of the form [x + , x − ] , where x + and x − are defined by (13), with the same value of the objective function.
(ii) For any feasible solution of the problem TQPPLC of the form [x + , x − ] vector x = x + − x − is the ble solution of the problem QPPLCAV with the same value of the objective function.
Proof.(i) Using substitution ( 13)-( 14) we get the problem (15)- (16).Here for every i = 1, n one of the values x + i or is the feasible solution of the problem QPPLCAV with the same value of the objective function -it is sufficient to use relations ( 13)-( 14) to show it.If x + i > 0, x − i > 0 for some i, using ( 14) is not correct: However in both cases we may use vector x = x + − x − with where under min we understand component-wise minimum.It is clear that now i for all i = 1, n, hence obviously the equalities hold true.So, the equality constraints in the problem QPPLCAV for the vector x are satisfied, and the values of the objective functions of TQPPLC for vector [x + , x − ] and QPPLCAV for vector x are the same.Now let us consider the i-th row of the inequality Because vector [x + , x − ] is the feasible solution of the TQPPLC, we get Q| x| + P x ≤ s, and vector x is a feasible solution of QPPLCAV.But So the claim (ii) is proven.
Definition 5.1.The solution x of the minimization problem which minimizes some stabilization functional Ω(x) on the set of all solutions to the minimization problem is called Ω-normal solution of the problem, see [22].The solution [ x + , x − ] defined above is the Ω-normal solution of the problem TQPPLC for stabilization functionals and The normal solution will have at least one zero value for each pair of components x + i and x − i .
x Proof.If the feasibility set of the problem TQPPLC is nonempty, then it is convex.Let us consider two different normal solutions 2 if the vectors x 1 and x 2 are different.We get the contradiction, because both normal solutions should minimize the functional Ω i , i = 1, 2, on the set of solutions.

Stabilization of the solution of the TQPPLC problem
Using Theorem 5.1 first we may find some solution [x + , x − ] to the problem TQPPLC, and then we get the solution x = x + − x − to the original problem QPPLCAV corresponding to the normal solution of the problem TQP-PLC.However, since Hessian matrix C of the form (10) of the problem TQPPLC is only positive semidefinite, it may cause problems by the numerical solution of the problem.We would like to get a good enough approximation to the Ω-normal solution of the problem TQPPLC.
Let us consider the problem TQPPLC with modified Hessian matrix.We arrive to the problem MTQPPLC if we use in (15) the matrix C α of the form (11) instead of the matrix C, α > 0. Let us denote the solution to the problem MTQPPLC by , and the corresponding approximation of the solution to the original problem QPPLCAV by Theorem 5.2.Let x ∈ R n be the solution of the problem QPPLCAV.Then Remark 5.4.Here we suppose, that the feasible set of the problem QPPLCAV is not empty, and so it has a unique solution x ∈ R n .If the feasible set of the original problem is empty and the solution x does not exist, the problem MTQP-PLC with the same (empty) feasible set has no solution too.
Proof.Because the feasible set of the problem QPPLCAV is nonempty, the feasible sets of both problems TQPPLC and MTQPPLC are nonempty, too.So from Corollary 4.3 it follows that the problem MTQPPLC has a unique solution for any α > 0.
The solution x = x + − x − corresponds to the normal solution [ x + , x − ] to the problem TQPPLC with respect to the stabilization functional Ω 2 defined by (19).
For any α > 0 we have 1 2 Comparing the first and the third parts of (20) we conclude, that Now, let us take any sequence 0 < α k → 0. For the corresponding sequence of solutions ] .Using continuity of the matrix product, and taking α k l → 0 in (20) we get 1 2 1 2 , where c = [c , −c ], together with Since vector [ x + , x − ] is the normal solution of the problem TQPPLC, we conclude, using Lemma 5.1, that Since all convergent subsequences of the sequence [x + α k , x − α k ] belonging to the compact in R 2n have the same limit, the sequence [x + α k , x − α k ] is convergent with the same limit.We may conclude that lim Remark 5.5.We can use in (15) the matrix C α of the form (12) instead of the matrix C, α > 0. Then we obtain the problem with positive definite matrix, too.This is another stabilization possibility, which is related with using the stabilization functional of the form (18). Remark 5.6.Theorem 5.2 claims that regularized solution x α converges to the normal solution x as α tends to zero.However, it is not possible to write an upper bound for the discrepancy between these two solutions for the general problem.
Let us consider next example: subject to The set of feasible solutions is shown in Fig. 4. The segment [a 2 , ∞] of the x 2 -axis is the set of all solutions with the normal solution at the point x = [0, a 2 ].
x 1 Consider the regularized problem, changing the functional (22) to Regularized solutions x α are with the difference One can easily see, that the norm of the difference tends to zero as parameter α tends to zero.For a given value α we can choose the value a 2 sufficient large making the distance between regularized solution and the normal solution as large as we like.
The sequence of the reqularized solutions for different values α and for a 1 = 0.3 and a 2 = 3 is shown in Fig. 4 and in the second and the third column of Tab. 1. Columns four and five contain the absolute values of the components differences between regularized solutions for chosen values α and the normal solution.
, with the linear inequalities containing absolute values and with equality constraints We have first rewrite 2 constraints with absolute values into 8 constraints without absolute values.As the result we get the optimal solution of the original problem QPPLCAVx = [0.92,−0.84, 0.75, 0.17] -and the corresponding minimal value of the objective function f min = 1.5333:Next we have solved the problem TQPPLC directly, without regularization.Further we have solved the problem MTQPPLC for values α = 10 −1 , 10 −2 , . . ., 10 −14 .
Even for large value α = 0.1 obtained regularized approximation to the normal solution x 0.1 = [0.87503,−0.75615, 0.75305, 0.12808] is close to the solution x = [0.92,−0.84, 0.75, 0.17] of the original problem.Tab. 2 shows the components of the regularized solutions and the corresponding functional values for different values of parameter α.In the first row the normal solution and its functional value is given, the second row corresponds to the numerical solution of the problem TQPPLC without regularization using MATLAB function quadprog with zero compoments of the initial vector (the solution does not meet the first equality constraint).Tab. 3 shows the differences between components of the normal solution and the regularized solutions, and the differences f α − f between the corresponding functional values.Choosing α = 10 −6 ÷ 10 −5 seems to be reasonable giving the sufficient high precision.

CONCLUSIONS
Quadratic programming problem with linear constraints containing absolute values of variables may lead to very large number of inequalities.Direct solution of the transformed problem TQPPLC with double number of variables and adding nonnegativity constraints may lead and often leads to the correct solution.However, as it is clear from our example, it may be better to use regularized solution solving the regularized problem MTQPPLC with small value α.If given data C, c, A, b, P, Q, and s are not exact, more sophisticated method for choosing the appropriate value of the parameter α should be considered.

1 2 w
C w → min, where Āin w ≤ bin and Āeq w = beq (4) can be transformed using w = f + Mv to the corresponding QP problem with the linear inequality constraint (1) containing absolutes values of variables.
and b ∈ R m are column vectors.Further we will use vector notation for vectors of absolute values: |x| = [ |x 1 |, . . ., |x n | ] .So we may rewrite the inequalities in (8) in a matrix form

Fig. 1
Fig. 1 Nonconvex domain defined by linear constraint with absolute values w 1 because C is the positive definite matrix with kernel consisting of zero vector 0 n only.Corollary 4.1.Let C be a symmetric positive definite matrix of order n.Denote C a block matrix of the form (10). Then matrix C has n zero eigenvalues and n positive eigenvalues that are eigenvalues of matrix C multiplied by 2. All eigenvectors of C corresponding to the positive eigenvalues may be written in the form [v , −v ] , where v ∈ R n are corresponding eigenvectors of matrix C.

Table 1
Regularized solutions and the distance between the regularized and the normal solution for a 1 = 0.3 and a 2 = 3

Table 2
Regularized solutions and functional value for chosen values α a : using MATLAB function quadprog with zero compoments ofthe initial vector b : using Octave function qp with zero compoments of the initial vector

Table 3
The components difference between the regularized and the normal solution and between corresponding functional values for chosen α values