Aitken ’ s Δ 2 method extended

Abstract: Aitken’s Δ method is used to accelerate convergence of sequences, e.g. sequences obtained from iterative methods. An explicit assumption in deriving Aitken’s Δ method and establishing acceleration (for linearly convergent sequences) is that consecutive error iterates (or their approximations) have the same sign or have an alternating sign pattern. We extend the standard Aitken’s Δ method to the cases in which consecutive pairs of error iterates in the sequence have alternating signs. Under suitable restrictions, acceleration of convergence is proved. Implementation of our extended method is described. Numerical examples demonstrate the process. An example is included relating our results to results obtained from Richard extrapolation.


PUBLIC INTEREST STATEMENT
Aitken's Δ 2 method is a method used to accelerate convergence of sequences, e.g. sequences of numbers obtained from iterative methods. An explicit assumption in deriving Aitken's Δ 2 method and establishing acceleration (for linearly, relatively slowly convergent sequences) is that consecutive error iterates (or their approximations) have the same sign or have an alternating sign pattern. This paper presents an extension of the standard Aitken's delta-squared method to the cases in which consecutive pairs of error iterates in the sequence have alternating signs. Under suitable restrictions, acceleration of convergence is proved. Implementation of this extended method is described. Numerical examples demonstrate the process. An example is included applying this work to results obtained from Richard extrapolation. Bumbariu (2012) and Buoso, Karapiperi, and Pozza (2015), among others. In this paper, we present new work in which we extend the method to apply to sequences exhibiting a pattern in which the (n + 2)nd error iterate has the opposite sign from the nth error iterate.
In Section 2 the basic Aitken's Δ 2 method is reviewed. Our extension of the basic method is developed in Section 3. Numerical examples illustrate our extended method in Section 4. A summary of results is given in Section 5.

Solving for q yields
The resulting Aitken's Δ 2 approximation is denoted q n and defined as Under the above assumptions, the Aitken's Δ 2 formula provides faster convergence: n=0 is a sequence that converges linearly to the limit q with and the signs of q n − q are the same for each n sufficiently large, then the Aitken's Δ 2 sequence {q} ∞ n=0 converges to q faster than {q n } ∞ n=0 in the sense that Proof Assume that where This implies that q n+1 − q q n − q = + n , n = 1, … , lim n→∞ n = 0. and We have Taking the limit as n → ∞, since the asymptotic error constant satisfies 0 < < 1. ✷ A similar proof holds if the signs of the consecutive error iterates alternate.

Extension of Aitken's Δ 2 method
The Aitken's Δ 2 formula was derived under the assumption that for n sufficiently large, the consecutive error iterates have the same sign (or have a pattern of alternating signs). But what if this is not the case? We extend acceleration results to some cases in which the two pairs of error iterates q n − q and q n+2 − q have opposite signs for n sufficiently large. The error iterate q n+1 − q may have either sign, positive or negative. For example, q n − q > 0 and q n+1 − q > 0 (or q n+1 − q < 0) but q n+2 − q < 0. Therefore, we can begin with the assumption that for n sufficiently large, This gives for n sufficiently large. Solving the resulting quadratic equation (approximation) for q yields There are two cases to consider with respect to the sign choice preceding the square root. For one case, use the plus sign, denoted as: and for the other case, use the minus sign: Each of these two cases has two (sub)cases, as described in Theorem 3.1. An extension of the Aitken's Δ 2 method acceleration theorem for these four cases is as follows: n=0 is a sequence that converges linearly to the limit q, and the pair of error iterates q n − q and q n+2 − q have opposite signs for n sufficiently large, then the extended Aitken's Δ 2 sequences {q +n } ∞ n=0 and {q −n } ∞ n=0 , and have subsequences with n = 4m + 3, 4m + 4, 4m + 2, or 4m + 1, m = 0, 1, 2, … that converge to q faster than {q n } ∞ n=0 in the sense that or Proof There are four cases to consider. For the first case, use the formula for q −n with where Therefore, rearranging directly from Equation (1), in which the algebra manipulations involve adding zero (and multiplying by one in the following algebra) in order to put expressions into more useful forms. Taking the limit as n → ∞, A corresponding proof for the second case uses the formula for q +n , with where The analogous computations yield the result that provided that We do not obtain completely analogous results for this case with q +n . However, this result for q +n , while not holding for all 0 < < 1, does pertain to the more important situations, those which have larger values of , i.e. slower original rates of convergence. If 0 < < √ 2 − 1, the rate of convergence does not worsen, but there is no guaranteed acceleration.
For the third case, using q −n , with where we obtain the result that Otherwise, the rate of convergence does not worsen, but there is no guaranteed acceleration.
For the fourth case, using q +n , with where gives the result that for 0 < < 1. Therefore, the rate of convergence does not worsen, but there is no guaranteed acceleration. ✷ These four cases apply to subsequences/subsets of the sequences that are of interest (see Section 4). Also, note that Theorem 3.1 gives sufficient conditions, which may not be necessary. The proof of Theorem 3.1 does not conflict with the numerical results given in Section 4, but does not give a complete explanation of the method. This is because some information is lost in the process of taking the limit, in particular the signs of the three relevant error terms preceding the square root. This aspect of the method is discussed in Section 4.3.
We need to decide which is the appropriate choice at each iteration. Recall that and If we have a suitable sign pattern, for example, as given in Theorem 3.1, do we use q +n or q −n so as to minimize the nth iteration absolute error? In an actual application, the exact limit value q is unknown. However, if it were known, then to minimize the nth iteration absolute error, Since the exact value, q, is unknown in applications, q can be approximated by q n+3 , or other information about the behavior of the sequence can be used (see Section 4). In applying the standard Aitken's Δ 2 method, the signs of the error terms are similarly unknown.

Numerical results
Three numerical examples are presented. The first example considers a sublinearly convergent sequence, the second example uses a linearly convergent sequence and the third example compares methods.

Example 1
For Example 1, we have the following sequence converging to zero: The notation ⌈n⌉ denotes the ceiling function, defined as This sequence converges sublinearly, not linearly, but, even so, the generalization of Aitken's Δ 2 method accelerates the convergence, as will now be described. Since the sequence limit is zero, the errors are the same as the iterates. Some numerical data are presented in Table 1, and the iterates are represented in Figure 1.
From  Table 1, Example 1 data in bold font): q +1 , q −2 , q −3 , q +4 , q +5 , etc. That is, for n > 1, alternate pairs of iterates, using q − for two consecutive iterates, then q + for the next two consecutive iterates, and continue with this pattern until sufficient accuracy is attained. The justification for this choice of iterates is discussed in Section 4.3.
Again, as with Example 1, the use of {q ±n } ∞ n=1 accelerates convergence compared to the rate of convergence of the sequence {q n } ∞ n=1 , provided that the following choices are made (see Table 2, data in bold font): q +1 , q −2 , q −3 , q +4 , q +5 , etc. i.e. for n > 1, alternate pairs of iterates, using q − for two consecutive iterates, then q + for next two consecutive iterates, and continue with this pattern until sufficient accuracy is attained.

Example 3
For Example 3, the data were obtained from Pomeranz (2011, Sec. 5.1). The nodes used are some of the boundary nodes from a square domain, and the numerically obtained outward normal flux values, q, were computed using the boundary element method. A test problem with known exact flux was used. A Mathematica (Wolfram Research, 2016) computer program was run for three different grid spacings, a coarse grid, an intermediate grid, and a relatively fine grid. Richardson extrapolation was performed at each node using the three flux values computed at that node. The order of the  dominant error term in the flux computation (referred to as a 'p value') was approximated numerically (Pomeranz, 2011, Table 1, p. 2321Smith, 2010, p. 249).
The Richardson flux extrapolation in which the order of convergence was unknown, and which accordingly required results from three different grid spacings, gave results that were identical to the standard Aitken's Δ 2 method results, provided that the three errors (from the coarse, intermediate, and fine grids) had the same sign (Pomeranz, 2011, Sec. 3). Richardson extrapolation does not apply at a given node if the error signs from each of the three grid spacings are not the same. Eight boundary nodes that appeared to present numerical difficulties were selected based on the criterion of having "bad p values", as described in Pomeranz (2011, Sec. 5.1). The behavior of the errors and acceleration of convergence is now examined in detail for this example (Example 3). The difficulties in convergence behavior occurred because the outward normal flux error signs were not as required, and consequently the standard Aitken's formula did not apply. However, when the standard Aitken's    , 9, 17, 18, 19, 21, 27, and 28 were selected because these were the nodes at which complex values (with non-zero imaginary parts) for p, the order of dominant flux error term, were obtained numerically (Pomeranz, 2011, Table 1). This is an indicator of numerical difficulties, since the order of convergence should be a positive real number at each node.
In Example 3, by observing the signs of the outward normal boundary flux errors, we can predict which variant of Aitken's Δ 2 method should be applied. Also, we can confirm that the numerical results agree with the results predicted from the theory/discussion following Equation (2). The data are given in Table 3. The best convergence (values which minimize the absolute error) is indicated by the bold font values in Table 4.
The choice of which approximation is best in the sense of minimizing the absolute error is determined as follows. In this discussion, the exact value for q, the outward normal boundary flux, is known. This would not be the case in practice, so some other information about the expected behavior of the iterates could be used, the next iterate could be used to approximate q, or, in some cases, observed values/pattern of the iterates might suggest which choices give fastest convergence.
Let the error at the nth iteration be defined and denoted as The first decision is whether to use q, the standard Aitken's Δ 2 method value, or one of two values from the extension of the standard method, q+ or q−, as developed in this paper. At a specific node and for iteration n, (1) If (q n − q) ⋅ (q n+2 − q) ≡ e n ⋅ e n+2 > 0, use the standard Aitken's method iterate, q n .
The choice of (i) or (ii) indicates which iterate, q +n or q −n , respectively, minimizes the absolute error, given in Equation (2). These values are displayed in Table 5. The drawback is that to use these formulas directly, the sequence limit, q, is needed. This is not known in practice. Our intent here is to show e n ≡ q n − q, n = 1, … . how the theory and numerical results agree for a specific sample test problem. The theory and the numerical results do agree, as is shown in Tables 5 and 6. In Table 6 we compare the error magnitudes at these eight nodes for each of the three possible formulas for the variants of Aitken's Δ 2 method, q, q+, and q−. The minimum error magnitudes for the flux values at these eight nodes are, respectively, given by formulas q+, q+, q−, q, q, q−, q+, and q+, in agreement with the predictions from the theoretical analysis (see bold font values in Tables 4 and 6).

Conclusions
We have developed an extension of Aitken's Δ 2 method which, although very specialized, accelerates the convergence of iterative sequences of the form described in Sections 1 and 3. A proof for this extension was developed and two numerical examples illustrating application of the method were given. A third numerical example comparing results from this extended method and Richardson extrapolation was presented.