Abstract
The state of the art iterative method for solving large linear systems is the conjugate gradient (CG) algorithm. Theoretical convergence analysis suggests that CG converges more rapidly than steepest descent. This paper argues that steepest descent may be an attractive alternative to CG when solving linear systems arising from the discretization of ill-posed problems. Specifically, it is shown that, for ill-posed problems, steepest descent has a more stable convergence behavior than CG, which may be explained by the fact that the filter factors for steepest descent behave much less erratically than those for CG. Moreover, it is shown that, with proper preconditioning, the convergence rate of steepest descent is competitive with that of CG.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Å. Björck, A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations, BIT, 28 (1988), pp. 659-670.
Å. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, PA, 1996.
R. H. Chan and M. K. Ng, Conjugate gradient methods for Toeplitz systems, SIAM Rev., 38 (1996), pp. 427-482.
P. J. Davis, Circulant Matrices, Wiley, New York, 1979.
H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers, Dordrecht, 2000.
M. Hanke, On lanczos based methods for the regularization of discrete ill-posed problems, BIT, 41 (2001), pp. 1008-1018.
M. Hanke, J. G. Nagy, and R. J. Plemmons, Preconditioned iterative regularization for ill-posed problems, in L. Reichel, A. Ruttan and R. S. Varga (eds.), Numerical Linear Algebra, de Gruyter, Berlin, 1993, pp. 141-163.
P. C. Hansen, Regularization tools: A Matlab package for the analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), pp. 1-35.
P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, Philadelphia, PA, 1997.
L. Kaufman, Maximum likelihood, least squares, and penalized least squares for PET, IEEE Trans. Med. Imag., 12 (1993), pp. 200-214.
M. E. Kilmer and D. P. O'Leary, Choosing regularization parameters in iterative methods for ill-posed problems, SIAM J. Matrix Anal. Appl., 22 (2001), pp. 1204-1221.
J. G. Nagy, K. M. Palmer, and L. Perrone, Iterative methods for image restoration: A Matlab object oriented approach. http://www.mathcs.emory.edu/~nagy/RestoreTools, 2002.
J. G. Nagy and Z. Strakoš, Enforcing nonnegativity in image reconstruction algorithms, in D. C. Wilson (ed), Mathematical Modeling, Estimation, and Imaging, Vol. 3461, SPIE, 2000, pp. 182-190.
D. P. O'Leary and J. A. Simmons, A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems, SIAM J. Sci. Stat. Comp., 2 (1981), pp. 474-489.
Y. Saad, Iterative Methods for Sparse Linear Systems, PWS Publishing Company, Boston, 1996.
C. R. Vogel, Solving ill-conditioned linear systems using the conjugate gradient method, Technical Report, Department of Mathematical Sciences, Montana State University, 1987.
C. R. Vogel, Computational Methods for Inverse Problems, SIAM, Philadelphia, PA, 2002.
Rights and permissions
About this article
Cite this article
Nagy, J.G., Palmer, K.M. Steepest Descent, CG, and Iterative Regularization of Ill-Posed Problems. BIT Numerical Mathematics 43, 1003–1017 (2003). https://doi.org/10.1023/B:BITN.0000014546.51341.53
Issue Date:
DOI: https://doi.org/10.1023/B:BITN.0000014546.51341.53