A convergence analysis of SOR iterative methods for linear systems with weak H-matrices

Abstract It is well known that SOR iterative methods are convergent for linear systems, whose coefficient matrices are strictly or irreducibly diagonally dominant matrices and strong H-matrices (whose comparison matrices are nonsingular M-matrices). However, the same can not be true in case of those iterative methods for linear systems with weak H-matrices (whose comparison matrices are singular M-matrices). This paper proposes some necessary and sufficient conditions such that SOR iterative methods are convergent for linear systems with weak H-matrices. Furthermore, some numerical examples are given to demonstrate the convergence results obtained in this paper.


Introduction
In this paper we consider the solution methods for the system of n linear equations where A D .a ij / 2 C n n and is nonsingular, b; x 2 C n and x unknown. In order to solve system (1) by iterative methods, the coefficient matrix A D .A ij / 2 C n n is split into where M 2 C n n is nonsingular and N 2 C n n . Then, the general form of iterative methods for (1) can be described as follows: The matrix H D M 1 N is called the iterative matrix of the iteration (3). It is well-known that (3) converges for any given x .0/ if and only if .H / < 1 (see [1][2][3][4][5]), where .H / denotes the spectral radius of the matrix H . Thus, to establish the convergence results of iterative methods, we mainly study the spectral radius of the iteration matrix in iteration (3). For simplicity, we let A D I L U; (4) where I is the identity matrix, L and U are strictly lower and strictly upper triangular, respectively. According to the standard decomposition (4), the iteration matrices for SOR iterative methods of (1) are listed in the following. The forward, backward and symmetric SOR methods (FSOR, BSOR and SSOR) iteration matrices are H F SOR.!/ D .I !L/ 1 OE.1 !/I C !U ; (5) H BSOR.!/ D .I !U / 1 OE.1 !/I C !L; (6) and where ! 2 .0; 1/ is the overrelaxation parameter.
Recently, the class of strong H -matrices including strictly or irreducibly diagonally dominant matrices (whose comparison matrices are nonsingular M -matrices) has been extended to encompass a wider set, known as the set of general H matrices (whose comparison matrices are M -matrices). In a recent paper, a partition of the n n general H matrix set H n was obtained in [6][7][8]. Here, we give a different partition: general H matrix set H n are partitioned into two mutually exclusive classes: the strong class, H S n , where the comparison matrices of all general H matrices are nonsingular, and the weak class, H W n , where the comparison matrices of all general H matrices are singular, in which singular and nonsingular H matrices coexist. As is shown in [1][2][3][4][5][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27], classical iterative methods such as Jacobi, Gauss-Seidel, symmetric Gauss-Seidel, JOR and SOR(SSOR) iterative methods for linear systems, whose coefficient matrices are strong H matrices including strictly or irreducibly diagonally dominant matrices, are convergent to the unique solution of (1) for any choice of the initial guess x .0/ . However, the same need not be true in case of some iterative methods for the linear systems with weak H matrices. Let us investigate the following example. It is verified that both A and B are weak H matrices and not strong H matrices, since their comparison matrices are both singular M matrices. How can we get the convergence on FSOR-, BSOR-and SSOR iterative methods for linear systems with this class of matrices without direct computations?
In last years, Zhang et al. in [28] and Zhang et al. in [29] studied the convergence of Jacobli, Gauss-Seidel and symmetric Gauss-Seidel iterative methods for the linear systems with nonstrictly diagonally dominant matrices and general H matrices, and established some significant results. In this paper the convergence of FSOR-, BSOR-and SSOR-iterative method will be studied for the linear systems with weak H matrices and some necessary and sufficient conditions are proposed, such that these iterative methods are convergent for the linear systems with this class of matrices. Then some numerical examples are given to demonstrate the convergence results obtained in this paper.
The paper is organized as follows. Some notations and preliminary results about general H -matrices are given in Section 2. The main results of this paper are given in Section 3, where we give some necessary and sufficient conditions on convergence for FSOR-, BSOR-and SSOR-iterative methods of linear systems with weak H -matrices. In Section 4, some numerical examples are given to demonstrate the convergence results obtained in this paper. Future work is given in Section 4.

Preliminaries
In this section we give some notations and preliminary results relating to the special matrices that are used in this paper. C m n .R m n / will be used to denote the set of all m n complex (real) matrices. Z denotes the set of all integers. Let˛Â hni D f1; 2; ; ng Z. For nonempty index sets˛;ˇÂ hni, A.˛;ˇ/ is the submatrix of A 2 C n n with row indices in˛and column indices inˇ. The submatrix A.˛;˛/ is abbreviated to A.˛/. Let A 2 C n n ,˛ hni and˛0 D hni ˛. If A.˛/ is nonsingular, the matrix is called the Schur complement with respect to A.˛/, indices in both˛and˛0 are arranged with increasing order. We shall confine ourselves to the nonsingular A.˛/ as far as A=˛is concerned. Let A D .a ij / 2 C m n and B D .b ij / 2 C m n , A ı B D .a ij b ij / 2 C m n denotes the Hadamard product of the matrices A and B. A matrix A D .a ij / 2 R n n is called nonnegative if a ij 0 for all i; j 2 hni. A matrix A D .a ij / 2 R n n is called a Z matrix if a ij Ä 0 for all i 6 D j . We will use Z n to denote the set of all n n Z matrices.
The comparison matrix of a given matrix A D .a ij / 2 C n n , denoted by .
It is clear that .A/ 2 Z n for a matrix A 2 C n n . The set of equimodular matrices associated with A, denoted by For n 2, an n n complex matrix A is reducible if there exists an n n permutation matrix P such that where A 11 is an r r submatrix and A 22 is an .n r/ .n r/ submatrix, where 1 Ä r < n. If no such permutation matrix exists, then A is called irreducible. If A is a 1 1 complex matrix, then A is irreducible if its single entry is nonzero, and reducible otherwise.
holds for all i 2 hni. If inequality in (13) holds strictly for all i 2 hni; A is called strictly diagonally dominant by row. If A is irreducible and the inequality in (13) holds strictly for at least one i 2 hni, A is called irreducibly diagonally dominant by row. If (13) holds with equality for all i 2 hni; A is called diagonally equipotent by row.
D n .SD n ; ID n / and DE n will be used to denote the sets of all n n (strictly, irreducibly) diagonally dominant matrices and the set of all n n diagonally equipotent matrices, respectively.
holds for all i 2 hni. If inequality in (14) holds strictly for all i 2 hni; A is called generalized strictly diagonally dominant. If (14) holds with equality for all i 2 hni; A is called generalized diagonally equipotent.
We denote the sets of all n n generalized (strictly) diagonally dominant matrices and the set of all n n generalized diagonally equipotent matrices by GD n .GSD n / and GDE n , respectively.  More importantly, under the condition of "reducibility", we have the following conclusion.
Lemma 2.7 (See [6,29] (15) of A, each irreducible diagonal square block R i i is generalized diagonally dominant and has at least one generalized diagonally equipotent principal submatrix.
The following definitions and lemmas come from [28,29].
Definition 2.10. Any complex matrix A D .a rs / 2 C n n has the following form: where Á 2 R, jAj D .ja rs j/ 2 R n n and E iÂ D .e iÂ rs / 2 C n n with Â rs 2 R and Â rr D 0 for r; s 2 hni: The matrix E iÂ is called a ray pattern matrix of the matrix A. If the ray pattern matrix E iÂ of the matrix A given in (16) is a ray pattern matrix, then A is called a ray matrix.
R n denote the set of all n n ray matrices. Obviously, if a matrix A 2 R n , then A 2 R n for all 2 C.
; a nn /. Then A 2 R n if and only if there exists an n n unitary diagonal matrix D such that n is singular if and only if the matrix A has at least either one zero principal submatrix or one irreducible principal submatrix ; a i k i k /.

Main results
In numerical linear algebra, the successive overrelaxation iterative method, simultaneously introduced by Frankel (1950) [33] and Young (1950) [34], is a famous iterative method used to solve a linear system of equations. This iterative method is also called the accelerate Liebmann method by Frankel (1950) [33] and the other many subsequent researchers. Kahan (1958) [35] calls it the extrapolated Gauss-Seidel method. It is often called the method of systematic overrelaxation. Frankel showed that for the numerical solutation of the Dirichlet problem for a rectangle, successive overrelaxation iterative method gave substantially larger (by an order of magnitude) asymptotic rates of convergence than those for the point Jacobi and point Gauss-Seidel iterative methods with suitable chosen relaxation factor. Young (1950) [34] and Young (1954) [36] showed that these conclusions held more generally for matrices satisfying his definition of propertly A, and that these results could be rigorously applied to the iterative solution of matrix equations arising from discrete approximations to a large class of elliptic partial differential equations for general regions. Later, this iterative method was developed as three iterative methods, i.e., the forward, backward and symmetric successive overrelaxation (FSOR-, BSOR-and SSOR-) iterative methods. Though these iterative methods can be applied to any matrix with non-zero elements on the diagonals, convergence is only guaranteed if the matrix is strictly or irreducibly diagonally dominant matrix, Hermitian positive definite matrix, strong H matrix and consistently ordered p-cyclic matrix. Some classic results on convergence of SOR iterative methods are as follows:  (5), (6) and (7), respectively, and therefore the sequence fx .i/ g generated by the FSOR-, BSOR-and SSOR-scheme (3), respectively, converges to the unique solution of (1) for any choice of the initial guess x .0/ . Theorem 3.2 (See [37]). Let A 2 H S n . Then the sequence fx .i/ g generated by the FSOR-, BSOR-and SSOR-scheme (3), respectively, converges to the unique solution of (1) for any choice of the initial guess x .0/ . [4,5,38]). Let A 2 C n n be a Hermitian positive definite matrix. Then the sequence fx .i/ g generated by the FSOR-, BSOR-and SSOR-scheme (3), respectively, converges to the unique solution of (1) for any choice of the initial guess x .0/ .
In this section, we mainly study convergence of SOR iterative methods for the linear systems with weak H -matrices. Proof. The sufficiency can be proved by contradiction. We assume that there exists an eigenvalue of H F SOR such that j j 1. According to Equality (5), Since j j 1, 1 C ! ¤ 0. Hence, equality (17) yields i.e.
The following will prove the necessity by contradiction. Assume that A 2 R n . Then it follows from Lemma 2.11 that there exists an n n unitary diagonal matrix D such that A D I L U D A D I DjLjD 1 DjU jD 1 and Sice A D I L U 2 DE n and is irreducible, Lemma 2.4 shows that I jLj jU j 2 H W n \ R n and is irreducible. Lemma 2.12 shows that I jLj jU j is singular and hence det .I jLj jU j/ D 0. Therefore, (27) yields det .I H F SOR.!/ / D 0, which shows that 1 is an eigenvalue of H F SOR.!/ . Then, we have that .H F SOR.!/ / 1, i.e. FSOR-method doesn't converge. This is a contradiction. Thus, the assumption is incorrect and hence, A … R n . This completes the necessity.
In the same way, we can prove that for ! 2 .0; 1/; BSOR-method converges, i.e. .H BSOR.!/ / < 1 if and only if A … A 0 n . Here, we finish the proof. In what follows we will propose the convergence result for SSOR iterative method for linear system with weak H -matrices including nonstrictly diagonally dominant matrices. Firstly, the following lemma will be given for the convenience of the proof.  Proof. The sufficiency can be proved by contradiction. We assume that there exists an eigenvalue of H SSOR such that j j 1. According to equalities (5), (6) and (7), Then, B. ; !/ D B=R is the Schur complement of B with respect to the principal submatrix R. Since B. ; !/ D B=R is singular, Lemma 3.6 shows that B is also singular. Again since A is irreducible, both L ¤ 0 and U ¤ 0. As a result, B is also irreducible. Since A D I L U D .a ij / 2 DE n with unit diagonal entries, ja ij j; i D 1; 2; ; n: Thus, for all ! 2 .0; 1/ and j j 1, we have that both hold for all i 2 N D f1; 2; ; ng. Immediately, we obtain ja ij j; i D 1; 2; ; n: " The latter two equalities of (34) indicate that Â D 2k ; where k is an integer and thus D e i2k D 1, and there exists an n n unitary diagonal matrix D such that D 1 AD D I jLj jU j, i.e. A 2 A 0 n . However, this contradicts A … A 0 n . According to the proof above, we have that .H SSOR.!/ / < 1, i.e. SSOR-method converges.
The following will prove the necessity by contradiction. Assume that A 2 A 0 n . Then there exists an n n unitary diagonal matrix D such that A D I L U D I DjLjD 1 DjU jD 1    Proof. The conclusion of this theorem is easy to be obtained form Lemma 2.12, Theorem 3.1 and Theorem 3.7.  Proof. From (5), (6), (7), (38) and (39), Therefore, similarly as in the proof of Theorem 3.9, we have with Theorem 3.7 that for ! 2 .0; 1/, .H SSOR.!/ / D . O H SSOR.!/ / < 1, i.e. the sequence fx .i/ g generated by the SSOR iterative schemes (7) converges to the unique solution of (1) for any choice of the initial guess x .0/ if and only if A … R n . Proof. If A 2 H W n is irreducible, it follows from Theorem 3.9 that the conclusion of this theorem is true.  (5) and (6) converges to the unique solution of (1) for any choice of the initial guess x .0/ . This completes the proof. Proof. Similar to the proof of Theorem 3.11, the conclusion of this theorem is easy to be obtained form Theorem 3.1, Theorem 3.2 and Theorem 3.10.

Numerical examples
In this section, some numerical examples are given to demonstrate the convergence results obtained in this paper.
It is easy to see that A n 2 DE n is irreducible. Since i.e. the sequence fx .i / g generated by the FSOR-, BSOR-and SSOR-iterative schemes (5), (6) and (7) Table 1.
It is shown in Table 1 and Fig. 1 that: (i) the changes of .H F SOR.!/ / and .H BSOR.!/ / are identical with ! increasing. They gradually decrease from 1 to 0.358 with ! increasing from 0 to 0.7 while they gradually increase from 0.358 to 1.284 with ! increasing from 0.7 to 1. This shows the optimal value of ! should be ! opt 2 .0:50; 0:80/ such that the SOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess x 0 .  The optimal value of ! should be ! opt 2 .0:40; 0:60/ such that the SOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess x 0 . It follows from Table 1 and Fig. 1 that SSOR iterative method is superior to the other two SOR iterative methods.
Although A 2 DE 6 are reducible, there is not any principal submatrix A k (k < 6) in A such that D 1 A k A k 2 R k , Lemma 2.12 shows that A is nonsingular. Therefore, Theorem 3.5 and Theorem 3. i.e. the sequence fx .i / g generated by the FSOR-, BSOR-and SSOR-iterative schemes (5), (6) and (7) converges to the unique solution of (1) for any choice of the initial guess x .0/ .
The computations on Matlab 7.0 of PC yield some comparison results on the spectral radius of SSOR iterative matrices, see Table 2.  then decreases gradually to 0.342 at ! D 0:7. Finally, it increases quickly from 0.342 to 0.585 with ! increasing from 0.7 to 1. The optimal value of ! should be ! opt 2 .0:40; 0:60/ such that the SOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess x 0 . It follows from Table 2 and Fig. 2 that SSOR iterative method is superior to the other two SOR iterative methods.

Further work
In this paper some necessary and sufficient conditions are proposed such that SOR iterative methods, including FSOR, BSOR and SSOR iterative methods, are convergent for linear systems with weak H -matrices. The class of weak H -matrices with singular comparison matrices is a subclass of general H matrices [29] and has some theoretical problems. In particular, the convergence problem on AOR iterative methods for this class of matrices is an open problem and is a focus of our further work.