Skip to main content
SpringerLink
Log in

Solution of sparse rectangular systems using LSQR and CRAIG

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

We examine two iterative methods for solving rectangular systems of linear equations: LSQR for over-determined systemsAx ≈ b, and Craig's method for under-determined systemsAx = b. By including regularization, we extend Craig's method to incompatible systems, and observe that it solves the same damped least-squares problems as LSQR. The methods may therefore be compared on rectangular systems of arbitrary shape.

Various methods for symmetric and unsymmetric systems are reviewed to illustrate the parallels. We see that the extension of Craig's method closes a gap in existing theory. However, LSQR is more economical on regularized problems and appears to be more reliable if the residual is not small.

In passing, we analyze a scaled “augmented system” associated with regularized problems. A bound on the condition number suggests a promising direct method for sparse equations and least-squares problems, based on indefiniteLDL T factors of the augmented matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Arioli, I. S. Duff and P. P. M. de Rijk,On the augmented system approach to sparse least-squares problems, Numer. Math., 55 (1989), pp. 667–684.

    Google Scholar 

  2. Å. Björck,Iterative refinement of linear least squares solutions I, BIT, 7 (1967), pp. 257–278.

    Google Scholar 

  3. Å. Björck,A bidiagonalization algorithm for solving ill-posed systems of linear equations, Report LITH-MAT-R-80-33, Dept. of Mathematics, Linköping University, Linköping, Sweden, 1980.

    Google Scholar 

  4. Å. Björck,Pivoting and stability in the augmented system method, in D. F. Griffiths and G. A. Watson (eds.),Numerical Analysis 1991: Proceedings of the 14th Dundee Conference, Pitman Research Notes in Mathematics 260, Longman Scientific and Technical, Harlow, Essex, 1992.

    Google Scholar 

  5. J. E. Craig,The N-step iteration procedures, J. Math. and Phys., 34, 1 (1955), pp. 64–73.

    Google Scholar 

  6. A. Dax,On row relaxation methods for large constrained least-squares problems, SIAM J. Sci. Comp., 14 (1993), pp. 570–584.

    Google Scholar 

  7. I. S. Duff and J. K. Reid,Exploiting zeros on the diagonal in the direct solution of indefinite sparse symmetric linear systems, ACM Trans. Math. Softw., to appear.

  8. D. K. Faddeev and V. N. Faddeeva,Computational Methods of Linear Algebra, Freeman, London, 1963.

    Google Scholar 

  9. R. W. Freund,Über einige CG-ähnliche Verfahren zur Lösung linearer Gleichungssysteme, Ph.D. Thesis, Universität Würzburg, FRG, 1983.

  10. P. E. Gill, M. A. Saunders and J. R. Shinnerl,On the stability of Cholesky factorization for quasi-definite systems, SIAM J. Mat. Anal., 17(1) (1996), to appear.

  11. G. H. Golub and W. Kahan,Calculating the singular values and pseudoinverse of a matrix, SIAM J. Numer. Anal., 2 (1965), pp. 205–224.

    Google Scholar 

  12. G. H. Golub and C. F. Van Loan,Unsymmetric positive definite linear systems, Linear Alg. and its Appl., 28 (1979), pp. 85–98.

    Google Scholar 

  13. P. C. Hansen,Test matrices for regularization methods, SIAM J. Sci. Comput., 16(2) (1995), pp. 506–512.

    Google Scholar 

  14. G. T. Herman, A. Lent and H. Hurwitz,A storage-efficient algorithm for finding the regularized solution of a large, inconsistent system of equations, J. Inst. Math. Appl., 25 (1980), pp. 361–366.

    Google Scholar 

  15. D. P. O'Leary, Private communication, 1990.

  16. C. Lanczos,An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Standards, 45 (1950), pp. 255–282.

    Google Scholar 

  17. P. Matstoms,Sparse QR factorization in MATLAB, ACM Trans. Math. Software, 20(1) (1994), pp. 136–159.

    Google Scholar 

  18. C. C. Paige,Bidiagonalization of matrices and solution of linear equations, SIAM J. Numer. Anal., 11 (1974), pp. 197–209.

    Google Scholar 

  19. C. C. Paige,Krylov subspace processes, Krylov subspace methods and iteration polynomials, in J. D. Brown, M. T. Chu, D. C. Ellison, and R. J. Plemmons, eds.,Proceedings of the Cornelius Lanczos International Centenary Conference, Raleigh, NC, Dec. 1993, SIAM, Philadelphia, 1994, pp. 83–92.

    Google Scholar 

  20. C. C. Paige and M. A. Saunders,Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12(4) (1975), pp. 617–629.

    Google Scholar 

  21. C. C. Paige and M. A. Saunders,LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8(1) (1982), pp. 43–71.

    Google Scholar 

  22. C. C. Paige and M. A. Saunders,Algorithm 583. LSQR: Sparse linear equations and least squares problems, ACM Trans. Math. Software, 8(2) (1982), pp. 195–209.

    Google Scholar 

  23. M. A. Saunders,Cholesky-based methods for sparse least squares: The benefits of regularization, Report SOL 95-1, Dept. of Operations Research, Stanford University, California, USA, 1995.

    Google Scholar 

  24. R. J. Vanderbei,Symmetric quasi-definite matrices, SIAM J. Optim., 5(1) (1995), pp. 100–113.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Systems Optimization Laboratory, Department of Operations Research, Stanford University, 94305-4022, Stanford, CA, USA

    Michael A. Saunders

Authors
  1. Michael A. Saunders
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Dedicated to Professor Åke Björck in honor of his 60th birthday

Partially supported by Department of Energy grant DE-FG03-92ER25117, National Science Foundation grant DMI-9204208, and Office of Naval Research grant N00014-90-J-1242.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Saunders, M.A. Solution of sparse rectangular systems using LSQR and CRAIG. Bit Numer Math 35, 588–604 (1995). https://doi.org/10.1007/BF01739829

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01739829

Key words

Search

Navigation