A Method for Solving Classical Smoothing Problems with Obstacles

We study how to reduce the smoothing problem with obstacles to the smoothing problem with weights. A system connecting deviations of the solution from given values and weights is established. An algorithm for solving this equation is proposed and illustrated by examples.


INTRODUCTION
Smoothing problems have been in the field of interest of researchers already for more than 40 years.Both the smoothing problem with obstacles and the problem with weights are problems of reconstructing a function according to some discrete inexact data.Solving a smoothing problem with weights is an easy task: the problem reduces to a linear system of equations.However, in practice the weights are not known.Hence, the smoothing problem with obstacles, where the error bounds on a finite set of knots are given, is much more practical.For the problem with obstacles necessary and sufficient conditions describing the solution are known (see, e.g., [1]) but finding an algorithm to solve this problem is still an open problem.A natural method of adding-removing knots, proposed in [1], can lead to a cycle as shown in [2].
It is known that if the Lagrangian associated to the smoothing problem with obstacles has a saddle point, then its first component is a solution of this problem and the second component defines the weights in the equivalent problem with weights.We have shown [Leetma, E. and Oja, P., unpubl.notes] that the Lagrangian associated to the classical smoothing problem with obstacles always has a saddle point, meaning that there exists an equivalent smoothing problem with weights.The equivalence of problems with obstacles and weights has been studied for example in [3] (univariate case) and in [4] (multivariate case).In [4] the special case of problem with weights is considered where all the weights are positive.But the problem with obstacles has an equivalent problem with positive weights only in an exceptional case: when all the knots in the solution of the smoothing problem with obstacles are active.Under this restriction an equation connecting deviations of the solution from the given values and weights was derived in [4].No attempts have been made to solve this equation.
In this paper we derive an equation connecting deviations and weights in the classical case where the weights are non-negative.We propose a method for solving this equation.The effectiveness of the method has not yet been studied, but as our first example shows, the problem from [2], where the method of addingremoving knots is cycling, can be solved by this method.

NOTATION AND PRELIMINARIES
For given integers r and n, 2r > n ≥ 1, let us denote by L (r) 2 (R n ) the space of functions defined on R n having all partial (distributional) derivatives of order r in L 2 (R n ).Define the operator T : where α = (α 1 , . . ., α n ), α i ≥ 0, α! = α 1 ! . . .α n ! and |α| = α 1 + . . .+ α n .We also need the product and the corresponding seminorm A function of the form with I a finite set and arbitrary X i ∈ R n , X i = X j for i = j, is called a natural spline.Here P r−1 is the space of polynomials of degree not exceeding r − 1 and G is the fundamental solution of the operator ∆ r , where ∆ is the n-dimensional Laplace operator.It is known that for n odd, G(X) = c nr X 2r−n and for n even, G(X) = c nr X 2r−n log X with some constants c nr > 0 and X = x 2 1 + . . .+ x 2 n .It is also known that any natural spline belongs to L (r) For given sets of indexes I 0 , I 1 , I 0 ∩ I 1 = / 0, I 0 ∪ I 1 = I, obstacles ε i > 0, i ∈ I 1 , pairwise distinct datapoints X i ∈ R n , i ∈ I, and values z i ∈ R, i ∈ I, define We consider the minimization problem min as the classical smoothing problem with obstacles.Assume that the zero-valued interpolation problem with polynomials from P r−1 in the knots X i , i ∈ I, possesses a unique solution.The solution of problem (3) exists and is a natural spline.The next proposition (see [1]) characterizes the solution of problem (3).

Proposition 1. A natural spline S of the form (1) such that S ∈ Ω is a solution of problem (3) if and only if the coefficients d i , i ∈ I 1 , of S satisfy the conditions
For the uniqueness of the solution it is sufficient that the interpolation problem with polynomials has only the solution P = 0.
For given sets of indexes I 0 , I 1 , We consider the minimization problem min as the classical smoothing problem with weights.
Assume that the zero-valued interpolation problem with polynomials from P r−1 in the knots X i , i ∈ I 0 ∪ { j ∈ I 1 | w j = 0}, possesses a unique solution.According to the next proposition the solution of problem (5) exists and is a natural spline of form (1). Proposition 2. There exists only one natural spline S of form (1) satisfying and this spline is the unique solution of the smoothing problem with weights.
The proof of Proposition 2 is a slight modification of that of Proposition 1 in [4], where the case w i > 0, i ∈ I 1 , is treated.
Problems (3) and ( 5) are equivalent, i.e. for any problem (3) there exists a problem (5) such that their solutions coincide (I 0 , I 1 , X i and z i do not change) and vice versa.In the next section an equation connecting equivalent problems with obstacles and weights will be derived.

AN EQUATION CONNECTING SMOOTHING PROBLEMS WITH OBSTACLES AND WEIGHTS
For any problem (3) the knot values S(X i ), i ∈ I 1 , are considered unknown, and so are the weights w i , i ∈ I 1 , in equivalent smoothing problem (5).In this section we will derive an equation connecting the deviations z i − S(X i ), i ∈ I 1 , to the weights w i , i ∈ I 1 .This equation will also contain the coefficients d i , i ∈ I 0 , corresponding to the interpolation knots.
Let us define the matrix W = (w i j ) i, j∈I with w ii = w i for i ∈ I 1 , w ii = 1 for i ∈ I 0 , and w i j = 0 for i = j.We also use the notations z = (z i ) i∈I and s = (S(X i )) i∈I , then equations ( 6) can be written as χ : R |I| → R |I| being the projection such that (χd) i = d i , i ∈ I 0 , (χd) i = 0, i ∈ I 1 , and the notation |I| means the number of elements in I. Let X β j , j ∈ J, be a basis in P r−1 .Then natural spline (1) may be presented as with d ∈ kerV T as an equivalent form for (2).From ( 7) and (8) we obtain Take an arbitrary symmetric regular |I| × |I| matrix A and define U = (WV )| (ker(WV )) ⊥ and D = A −1 U. Note that WV : R |J| → R |I| may not be injective but, according to R |J| = ker(WV ) ⊕ (ker(WV )) ⊥ , the operator U : where E is the identity operator.Using the ideas from [4], it can be shown that Π 2 = Π, ran Π = A kerU T and Πx, y = x, Πy for all x, y ∈ R |I| , which means that Π is an orthogonal projection onto the subspace A kerU T .
Let us show that ΠA −1 WV c = 0 for all c ∈ R |J| .Actually, it is sufficient to show that ΠA −1 WV c = 0 for all c ∈ (ker(WV )) ⊥ , which is equivalent to But this holds because for all x ∈ kerU T we have Then equation ( 7) can be written as Now, applying ΠA −1 to (9) and taking (10) into account, we obtain This equation connects the deviations εi , i ∈ I 1 , and coefficients εi , i ∈ I 0 , of the solution of the smoothing problem with obstacles to the weights of the equivalent smoothing problem with weights.Note that the condition ker(WV ) = {0} is equivalent to the assumption about unique solvability of the zero-valued interpolation problem with polynomials from P r−1 in the knots X i , i ∈ I 0 ∪ { j ∈ I 1 | w j = 0}.In practice usually ker(WV ) = {0} and thus U = WV .For example, in the case of cubic splines (n = 1, r = 2) it is sufficient that there are at least two non-zero weights for ker(WV ) being trivial.Assuming ker(WV ) = {0}, we propose a method for finding the weights in problem ( 5) equivalent to a given problem (3).

A METHOD FOR FINDING WEIGHTS
In equation (11) both W and ε are unknown.For our method we take w i = 1, i ∈ I, as guess values.Define N = {i ∈ I 1 | w i = 0} as the set of indexes corresponding to inactive obstacle knots.At the beginning we have N = / 0.
Step 1. (The step of finding ε.) Using (10), equation ( 11) can be written as Adding the conditions V T d = 0, we have a system of |I| + |J| linear equations with |I| unknowns d i , i ∈ I.
Let us solve this system by the standard least squares method.Even if we assume the sufficient uniqueness condition of the solution for the interpolation problem with polynomials as in Proposition 1, it may happen that this system has more than one least squares solution.Then we take the solution with minimal Euclidean norm.Based on (10), determine εi = (−1) r d i w i , i ∈ I \ N.
If N = / 0, we proceed to step 2, otherwise to step 3.
Step 2. (The step of computing missing ε.) Solve the interpolation problem or equivalently, the linear system with (c j ) j∈J and (d i ) i∈I\N as unknowns.In the case of multiple solution we continue with that of minimal Euclidean norm.We also take d i = 0, i ∈ N. The unknown deviations will be computed as If the step preceding this step was step 3, we also need to evaluate the coefficients εi = (−1) r d i , i ∈ I 0 .
Proceed to step 3.
Step 3. (The step of correcting ε.)For the solution, any number |ε i | should not exceed the obstacle values ε i , i ∈ I 1 , and |ε i | corresponding to the active knots should not be less than ε i , i ∈ I 1 \ N. Thus, define the set of indexes corresponding to the deviations that need to be corrected as If K = / 0, we proceed to step 4. Otherwise for i ∈ K we take εi = sign ( εi ) ε i , if εi = 0. We include the knots with εi = 0, i ∈ K, to the set of inactive knots by defining the new set N as If N ∪ I 0 = / 0, we continue at the beginning of step 2, otherwise we proceed to step 4.
Step 4. (The step of finding the weights.)Since equation ( 11) is nonlinear with respect to the weights w i , i ∈ I, we compute the corresponding weights using equations (6).For the interpolation knots take w i = 1, i ∈ I 0 .For the obstacle knots take If all the weights are nonnegative, i.e., w i ≥ 0, i ∈ I, we have got the solution.Otherwise, define N = {i ∈ I 1 | w i ≤ 0}, take w i = 0, i ∈ N, and continue at the beginning of step 1.

EXAMPLES
In [2] we presented a counterexample to the method of adding-removing knots proposed in [1].In this paper in the first example we use the same data and show how our method solves this problem.The implementation of the method goes via equation (11) and we take as A the identity matrix in both following examples.The working process of our algorithm is presented in Table 1.The first line contains the guess values of weights.In step 1 we compute the values εi , i ∈ I \ N, given in the second line.Since at the beginning N = / 0, we continue from step 3 and correct the deviations εi , i ∈ K = {1, 2, 3, 4}.At the end of step 3 the set of indexes corresponding to inactive knots is still empty, i.e.N = / 0, but since I 0 = / 0, we proceed from step 2. Compute the coefficients εi , i ∈ I 0 = {5, 6}.Now K = / 0 and we proceed to step 4. The computed weights and the corrected weights with indexes from N = {1, 2, 4} are presented in Table 1 on lines 5 and 6.Line 7 contains values of εi , i ∈ I \ N, computed in step 1 and so on.
Take notice of lines 14 and 15.In step 3 the deviations ε3 and ε4 are corrected because they exceed the obstacle value 0.7, the deviation ε2 is corrected because at this moment the point x 2 is an active knot.Only the deviation ε1 is left unchanged and the index set corresponding to the inactive knots is defined as N = {1}.Despite that in the next steps the deviation ε1 is corrected and the knot x 1 has been taken into the set of active knots.

Table 1 .
Values of εi and w i computed by the algorithm (Example 1)