Acceleration of an adaptive generalized Arnoldi method for computing

Abstract: By considering a weighted inner product, an adaptive generalized Arnoldi (GArnoldi) method was constructed by [13] for computing PageRank. In order to accelerate the adaptive GArnoldi method, this paper proposes a new method by using the power method with extrapolation process based on Google matrix’s trace (PET) as an accelerated technique of the adaptive GArnoldi method. The new method is called as GArnoldi-PET method, whose implementation and convergence analysis are discussed in detail. Numerical experiments are used to illustrate the effectiveness of our proposed method.


Introduction
Using the hyperlink structure of web pages, Google's PageRank becomes one of the most successful methods for measuring the importance of each page [1]. From the viewpoint of numerical computations, the core of PageRank problems can be regarded as the problem of solving a dominant eigenvector of the Google matrix A:

Previous work
In this section, we give simple introductions of the PET method and the adaptive GArnoldi method for computing PageRank.

The PET method for computing PageRank
Here, we first give the algorithmic version of the PET method for PageRank problems as follows, see [6] for more details.
2 ; 3.4. if r ≤ tol, break; else, goto step 2; endif Now, some illustrations of Algorithm 1 are given as follows.
• In step 1, the parameter µ is the trace of the Google matrix A.
• In step 2, the power method is run m 1 steps, which means the extrapolation technique is not employed to the power method in each iteration, but is used every m 1 power iterations. • In step 3, we can see that the extrapolation strategy based on x (m 1 −1) , x (m 1 ) and µ is easy to implement as given in step 3.1.

The adaptive GArnoldi method for computing PageRank
As described in [13], let G = (g i j ) be an n × n symmetric positive definite (SPD) matrix, then the GArnoldi process based on a weighted inner product is presented as Algorithm 2.

Algorithm 2. The GArnoldi process
Input: an initial vector v 1 , and the steps m of GArnoldi process, a SPD matrix G.
In Algorithm 2, (·, ·) G is a G-inner product defined as (x, y) G = x T Gy, ∀x ∈ R n , y ∈ R n , and · G is a G-norm defined by where Q ∈ R n×n is an orthogonal matrix, D = diag{d 1 , d 2 , · · · , d n } is a n × n diagonal matrix with d i > 0, i = 1, 2, · · · , n, and G = Q T DQ is a diagonalized decomposition of G. Let e m ∈ R m be the m-th co-ordinate vector, then the GArnoldi process has the following relations [13] From Algorithm 2, it is obvious that different SPD matrices G will lead to different GArnoldi methods. Since every SPD matrix can be diagonalized, for simplicity, we let G = diag{d 1 , d 2 , · · · , d n }, d i > 0, i = 1, 2, · · · , n. It is seen that in each outer iteration of the GArnoldi method, we hope to find a vector v satisfying min i , which leads to a weighted least squares problem where d i is actually the weight for the i-th component of residual r i , i = 1, 2, · · · , n. In order to speed up the computation of PageRank problems, Yin et al. [13] changed the weights adaptively according to the changing of the current residual corresponding to the approximate PageRank vector. Therefore, one choice of the matrix G is that where r is the residual vector computed by the last calculation, and n i=1 d i = 1. And the algorithmic version of the adaptive GArnoldi algorithm for computing PageRank is presented as Algorithm 3.

end
Note that, in step 5 of Algorithm 3, s m and u m denote the right and left singular vector of H m+1,m − [I; 0] T associated with the minimal singular value σ m , respectively.

The GArnoldi-PET method for computing PageRank
In order to accelerate the computation of PageRank problems, we develop a new method by combining the PET method with the adaptive GArnoldi method. The new method is called GArnoldi-PET method. Here we first describe the construction of the GArnoldi-PET method, and then discuss its convergence.

The GArnoldi-PET method
As described in the subsection 2.1, based on the trace of the Google matrix A, an extrapolation strategy has been presented to speed up the convergence of the power method. Numerical experiments in [6] have illustrated that the PET method has a faster convergence than the power method when the damping factor α is close to 1. On the other hand, since the Arnoldi method is more computationally intense than applying the same number of iterations of the power method [8], thus it is natural to consider using the extrapolation strategy based on trace and the adaptive GArnoldi method together.
Similar to the construction of the Power-Arnoldi algorithm [8], the mechanism of our proposed method can be presented as follows: Given a unit positive vector x (0) , and an approximate PageRank vector is obtained by iterating the adaptive GArnoldi method (Algorithm 3) for a few times (e.g., 2-3 times). If this approximate PageRank vector does not satisfy our prescribed tolerance, then we run the PET method to obtain another approximate vector with the resulting vector as the initial guess. If this approximate PageRank vector still can not satisfy our accuracy, then we return to Algorithm 3 with the new approximation as the starting vector. Repeating the above procedure until the described accuracy is achieved.
There is a problem about how to control the conversion between the PET method and the adaptive GArnoldi method. Many strategies have been developed to deal with this problem. Here, as given in [8], three parameters β, restart, maxit are used to control the procedure. Let τ (curr) be the residual norm of the current iteration, and τ (prev) be the residual norm of the previous iteration. Computing ratio = τ (curr) /τ (prev) , if ratio > β, then restart = restart + 1. If restart ≥ maxit, then we terminate the PET method and trigger the adaptive GArnoldi method. The specific implementation of the GArnoldi-PET method is given as follows.
Now, some remarks about the GArnoldi-PET method are given as follows.
• As shown in the step 3.16, the matrix G is adaptively changed according to the current residual.
• According to the construction of the GArnoldi-PET method, it is natural to treat the PET method as an accelerated technique for the adaptive GArnoldi method. inner products in the adaptive GArnoldi method and one matrix-vector product in the PET method.

Convergence analysis of the GArnoldi-PET method
Here we discuss the convergence analysis of the GArnoldi-PET algorithm. Particularly, we focus on the procedure when turning from the PET method to the adaptive GArnodli method.
Assume σ(A) denote the set of eigenvalues of the Google matrix A, and its eigenvalues are arranged as 1 = |λ 1 | > |λ 2 | ≥ · · · ≥ |λ n |. Let L m−1 be the set of polynomials of degree not exceeding m − 1, and K m (A, v 1 ) be a Krylov subspace. If (λ i , ϕ i ), i = 1, 2, · · · , n are the eigenpairs of A, and ( λ j , y j ), j = 1, 2, · · · , m are the eigenpairs of H m , then λ j is often used to approximate λ j , and ϕ j = V m y j is applied to approximate ϕ j in the standard Arnoldi method. However, instead of using Ritz vectors ϕ j as approximate eigenvectors, Jia [15] proposed a new strategy such that for each λ j , a unit norm vector u j ∈ K m (A, v 1 ) satisfying the condition is used to approximate ϕ j . Here u j is called a refined approximate eigenvector corresponding to λ j . The convergence of the refined Arnoldi method is given as follows.
Theorem 1 [15]. Assume that v 1 = n i=1 γ i x i with respect to the eigenbasis {x i } i=1,2,··· ,n in which x i 2 = 1, i = 1, 2, · · · , n and γ i 0, let S = [x 1 , x 2 , · · · , x n ], and where u j is a refined approximate eigenvector as above, σ max (S ) and σ min (S ) are the largest and smallest singular value of the matrix S , respectively. Before analyzing the convergence of the GArnoldi-PET algorithm, some useful conclusions are introduced as follows.
Lemma 1 [14]. Let G = diag{d 1 , d 2 , · · · , d n }, d i > 0, 1 ≤ i ≤ n, be a diagonal matrix. Then for any vector x ∈ R n , according to the definitions of the G-norm and the 2-norm, it has Lemma 2 [10]. Let v 1 be the initial vector for the PET method, which is from the previous adaptive GArnoldi method. Then the PET iteration in Algorithm 4 produces the vector v new where k ≥ maxit, η is the normalizing factor, µ is the trace of the matrix A, m 1 is a given number, T is called as the iterative matrix and I is an n × n identity matrix. Theorem 2 [16]. Assume that the spectrum of the column-stochastic matrix P is {1, λ 2 , · · · , λ n }, then the spectrum of the matrix A = αP + (1 − α)ve T is {1, αλ 2 , · · · , αλ n }, where 0 < α < 1, v is a vector with nonnegative elements such that e T v = 1. Theorem 3 [17]. Let P be an n × n column-stochastic matrix. Let α be a real number such that 0 < α < 1. E is the n × n rank-one column-stochastic matrix E = ve T , where e is the n-vector of all ones and v is an n-vector whose elements are all non-negative and sum to 1, A = αP + (1 − α)ve T is the n × n column-stochastic matrix, then its dominant eigenvalue λ 1 = 1, |λ 2 | ≤ α.
In the next cycle of the GArnoldi-PET method, v new 1 will be used as the initial vector for an m-step GArnoldi process, so that the new Krylov subspace The following theorem shows the convergence of the GArnoldi-PET method. Theorem 4. Assume that v 1 = n i=1 γ i x i with respect to the eigenbasis {x i } i=1,2,··· ,n in which x i 2 = 1, i = 1, 2, · · · , n and γ 1 0. Let G = diag{d 1 , d 2 , · · · , d n }, d i > 0, i = 1, 2, 3, · · · , n, S = [x 1 , x 2 , x 3 , · · · , x n ], and where u is taken from the Krylov subspace K m (A, v 1 ), µ = 1 + α l n − 1 with the number of dangling nodes l, k ≥ maxit, σ min (S ) is the smallest singular value of the matrix S .

Numerical experiments
In this section, we test the effectiveness of the GArnoldi-PET method and compare it with the PET method (Algorithm 1) [6], the Power-Arnoldi algorithm (called as PA) [8] and the adaptive GArnoldi method (called as A-Arnoldi) [13] in terms of the number of matrix-vector products (Mv) and the computing time in seconds (CPU). All the numerical results are obtained by using MATLAB 2018b on the Windows 10 (64 bit) operating system with 1.7 GHz Intel(R) Core(TM) i5 CPU and RAM 4.00 GB.
In Table 1, we list the characteristics of test matrices including the matrix size (n), the number of nonzero elements (nnz), the number of dangling nodes (numd) and the density (den) which is defined by den = nnz n×n × 100. For a fair comparison, all algorithms use the same initial guess x (0) = v = e/n with e = [1, 1, · · · , 1] T . The tolerance is chosen as tol = 10 −8 . The values of the damping factor α are 0.990, 0.993, 0.995 and 0.997, respectively. The parameter β = α − 0.1. We run the thick restarted Arnoldi procedure, with the number of approximate eigenpairs p, two times per cycle in the Power-Arnoldi method. Similarly, we run the adaptive GArnoldi procedure two times per cycle in the GArnoldi-PET method. In addition, for describing the efficiency of the GArnoldi-PET method, we define Example 1. The first test matrix is the wb-cs-stanford matrix, which contains 9914 pages, 36854 links and 2861 dangling nodes. It is available from https://sparse.tamu.edu/Gleich/wb-cs-stanford. In this example, we set the parameters m = 5, p = 3, maxit = 6 and m 1 = 40. Numerical results of the PET method, the adaptive GArnoldi method, the Power-Arnoldi algorithm and the GArnoldi-PET algorithm are reported in Table 2. Figure 1 plots the convergence history of the four methods with different values of α.
From Table 2, we can see that the Power-Arnoldi algorithm works better than the PET method and the adaptive GArnoldi method in terms of the number of matrix-vector products and the computing time. However, the GArnoldi-PET algorithm performs the best. For example, when α = 0.997, the Power-Arnoldi algorithm needs 0.1473 seconds to reach the desired accuracy, while the GArnoldi-PET algorithm only uses 0.1038 seconds, and the speedup is 29.53%.
From Figure 1, it is easy to find that the GArnoldi-PET algorithm has a faster convergence speed than the PET method and Power-Arnoldi algorithm, even though its iteration counts are slightly inferior to the adaptive GArnoldi method. Obviously, only the number of iterations can not describe the whole story.  Example 2. The second test matrix is the web-Stanford matrix, which contains 281903 nodes, 2312497 links and 172 dangling nodes. It is available from https://sparse.tamu.edu/SNAP/ web-Stanford. In this example, we choose the parameters m = 5, p = 3, maxit = 12 and m 1 = 35. Numerical results of the PET method, the adaptive GArnoldi method, the Power-Arnoldi algorithm and the GArnoldi-PET algorithm are given in Table 3. Figure 2 depicts the convergence of the four methods with different values of α.
From Table 3, it observes that the GArnoldi-PET algorithm outperforms the other three methods in terms of the number of matrix-vector products and the computing time. Although the speedup is only 6.57% relative to the Power-Arnoldi algorithm when α = 0.993. However, when α increases, e.g., α = 0.997, the Power-Arnoldi algorithm needs 13.7626 seconds to reach the desired accuracy, the GArnoldi-PET algorithm only takes 10.7224 seconds, and the speedup becomes 22.09%.
From Figure 2, it shows that the GArnoldi-PET algorithm converges faster than the PET method and the Power-Arnoldi algorithm. When α is close to one, e.g., α = 0.995 and α = 0.997, the iteration counts of the GArnoldi-PET algorithm are less than those of the adaptive GArnoldi method. This suggests that our new algorithm has some potential.    Table 4. Figure 3 shows the convergence curves of the four methods with different values of α.
From Table 4, we also see that the GArnoldi-PET algorithm makes great improvements on the PET method, the adaptive GArnoldi method and the Power-Arnoldi algorithm in terms of the number of matrix-vector products and the computing time. For a large damping factor such as α = 0.997, the Power-Arnoldi algorithm takes 56.5771 seconds to reach the desired accuracy, while the GArnoldi-PET algorithm takes 41.9286 seconds to achieve the same accuracy, and the speedup is 25.89%.
From Figure 3, we again find that the GArnoldi-PET algorithm converges faster than the PET method, the adaptive GArnoldi method and the Power-Arnoldi algorithm. For different values of α, the iteration counts of the GArnoldi-PET algorithm are the least.

Conclusions
In this paper, by combining the PET method with the adaptive GArnoldi method, we propose a new method called as GArnoldi-PET method for accelerating the computation of PageRank problems. Its construction and theoretical analysis can be found in Section 3. Numerical results in Section 4 show that our proposed method is quite efficient and better than the existing methods, especially when the damping factor is close to 1. However, much research still needs further study. For example, determining the optimal choice of the parameters, or considering to use some preconditioning strategies as given in [18][19][20] for the GArnoldi method.