Skip to main content
SpringerLink
Log in

REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems

  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The package REGULARIZATION TOOLS consists of 54 Matlab routines for analysis and solution of discrete ill-posed problems, i.e., systems of linear equations whose coefficient matrix has the properties that its condition number is very large, and its singular values decay gradually to zero. Such problems typically arise in connection with discretization of Fredholm integral equations of the first kind, and similar ill-posed problems. Some form of regularization is always required in order to compute a stabilized solution to discrete ill-posed problems. The purpose of REGULARIZATION TOOLS is to provide the user with easy-to-use routines, based on numerical robust and efficient algorithms, for doing experiments with regularization of discrete ill-posed problems. By means of this package, the user can experiment with different regularization strategies, compare them, and draw conclusions from these experiments that would otherwise require a major programming effert. For discrete ill-posed problems, which are indeed difficult to treat numerically, such an approach is certainly superior to a single black-box routine. This paper describes the underlying theory gives an overview of the package; a complete manual is also available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R.C. Allen, Jr., W.R. Boland, V. Faber and G.M. Wing, Singular values and condition numbers of Galerkin matrices arising from linear integral equations of the first kind, J. Math. Anal. Appl. 109 (1985) 564–590.

    Google Scholar 

  2. R.S. Anderssen and P.M. Prenter, A formal comparison of methods proposed for the numerical solution of first kind integral equations, J. Austral. Math. Soc. (Series B) 22 (1981) 488–500.

    Google Scholar 

  3. C.T.H. Baker, Expansion methods, chapter 7 in [18].

  4. C.T.H. Baker,The Numerical Treatment of Integral Equations (Clarendon Press, Oxford, 1977).

    Google Scholar 

  5. C.T.H. Baker and M.J. Miller (eds.),Treatment of Integral Equations by Numerical Methods (Academic Press, New York, 1982).

    Google Scholar 

  6. D.M. Bates, M.J. Lindstrom, G. Wahba and B.S. Yandell, GCVPACK — routines for generalized cross validation, Commun. Statist.-Simul. 16 (1987) 263–297.

    Google Scholar 

  7. M. Bertero, C. De Mol and E.R. Pike, Linear inverse problems with discrete data: II. Stability and regularization, Inverse Problems 4 (1988) 573–594.

    Google Scholar 

  8. Å. Björck, A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations, BIT 28 (1988) 659–670.

    Google Scholar 

  9. Å. Björck, Least squares methods, in:Handbook of Numerical Analysis, Vol. 1, eds. P.G. Ciarlet and J.L. Lions (Elsevier, Amsterdam, 1990).

    Google Scholar 

  10. Å. Björck and L. Eldén, Methods in numerical algebra for ill-posed problems, Report LiTH-MAT-R33-1979, Dept. of Mathematics, Linköping University (1979).

  11. H. Brakhage, On ill-possed problems and the method of conjugate gradients, in:Inverse and Ill-Posed Problems, eds. H.W. Engl and C.W. Groetsch (Academic Press, Boston, 1987).

    Google Scholar 

  12. T.F. Chan and P.C. Hansen, Some applications of the rank revealing QR factorization, SIAM J. Sci. Stat. Comp. 13 (1992) 727–741.

    Google Scholar 

  13. J.A. Cochran,The Analysis of Linear Integral Equations (McGraw-Hill, New York, 1972).

    Google Scholar 

  14. D. Colton and R. Kress,Integral Equation Methods for Scattering Theory (Wiley, New York, 1983).

    Google Scholar 

  15. L.J.D. Craig and J.C. Brown,Inverse Problems in Astronomy (Adam Hilger, Bristol, 1986).

    Google Scholar 

  16. J.J.M. Cuppen, A numerical soluiton of the inverse problem of electrocardiology, Ph.D. Thesis, Dept. of Mathematics, Univ. of Amsterdam (1983).

  17. L.M. Delves and J.L. Mohamed,Computational Methods for Integral Equations (Cambridge University Press, Cambridge, 1985).

    Google Scholar 

  18. L.M. Delves and J. Walsh (eds.),Numerical Soluution of Integral Equations (Clarendon Press, Oxford, 1974).

    Google Scholar 

  19. J.B. Drake, ARIES: a computer program for the solution of first kind integral equations with noisy data, Report K/CSD/TM43, Dept. of Computer Science, Oak Ridge National Laboratory (October 1983).

  20. M.P. Ekstrom and R.L. Rhodes, On the application of eigenvector expansions to numerical deconvolution, J. Comp. Phys. 14 (1974) 319–340.

    Google Scholar 

  21. L. Eldén, A program for interactive regularization, Report LiTH-MAT-R-79-25, Dept. of Mathematics, Linköping University, Sweded (1979).

    Google Scholar 

  22. L. Eldén, Algorithms for regularization of ill-conditioned least-squares problems, BIT 17 (1977) 134–145.

    Google Scholar 

  23. L. Eldén, A weighted pseudoinverse, generalized singular values, and constrained least squares problems, BIT 22 (1982) 487–501.

    Google Scholar 

  24. L. Eldén, A note on the computation of the genralized cross-validation function for illconditioned least squares problems, BIT 24 (1984) 467–472.

    Google Scholar 

  25. H.W. Engl and J. Gfrerer, A posteriori parameter choice for general regularization methods for solving linear ill-posed problems, Appl. Numer. Math. 4 (1988) 395–417.

    Google Scholar 

  26. R.D. Fierro and J.R. Bunch, Collinearity and total least squares, Report, Dept. of Mathematics, Univ. of California, San Diego (1991), to appear in SIAM J. Matrix Anal. Appl.

    Google Scholar 

  27. R.D. Fierro, P.C. Hansen and D.P. O'Leary, Regularization and total least squares, manuscript in preparation for SIAM J. Sci. Comp.

  28. R. Fletcher,Practical Optimization Methods. vol. 1, 2nd ed. (Wiley, Chichester, 1987).

    Google Scholar 

  29. G.H. Golub and C.F. Van Loan,Matrix Computations, 2nd ed. (Johns Hopkins, Baltimore, 1989).

    Google Scholar 

  30. G.H. Golub and U. von Matt, Quadratically constrained least squares and quadratic problems, Numer. Mathematik 59 (1991) 561–580.

    Google Scholar 

  31. C.W. Groetsch,The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind (Pitman, Boston, 1984).

    Google Scholar 

  32. C.W. Groetsch,Inverse Problems in the Mathematical Sciences (Vieweg Verlag, Wiesbaden), to be published.

  33. C.W. Groetsch and C.R. Vogel, Asymptotic theory of filtering for linear operator equations with discrete noisy data, Math. Comp. 49 (1987) 499–506.

    Google Scholar 

  34. J. Hadamard,Lectures on Cauchy's Problem in Linear Differential Equations (Yale University Press, New Haven, 1923).

    Google Scholar 

  35. M. Hanke, Regularization with differential operators. An interative approach, J. Numer. Funct. Anal. Optim. 13 (1992) 523–540.

    Google Scholar 

  36. M. Hanke, Iterative solution of undertermined linear systems by transformation to standard form, submitted to J. Num. Lin. Alg. Appl.

  37. M. Hanke and P.C. Hansen, Regularization methods for large-scale problems, Report UNIC-92-04, UNI.C (August 1992), to appear in Surveys Math. Ind.

  38. P.C. Hansen, The truncated SVD as a method for regularization, BIT 27 (1987) 543–553.

    Google Scholar 

  39. P.C. Hansen, Computation of the singular value expansion, Computing 40 (1988) 185–199.

    Google Scholar 

  40. P.C. Hansen, Perturbation bounds for discrete Tikhonov regularization, Inverse Problems 5 (1989) L41-L44.

    Google Scholar 

  41. P.C. Hansen, Regularization, GSVD and truncated GSVD, BIT 29 (1989) 491–504.

    Google Scholar 

  42. P.C. Hansen, Truncated SVD solutions to discrete ill-posed problems with ill-determined numerical rank, SIAM J. Sci. Statist. Comp. 11 (1990) 503–518.

    Google Scholar 

  43. P.C. Hansen, Relations between SVD and GSVD of discrete regularization problems in standard and general form, Lin. Alg. Appl. 141 (1990) 165–176.

    Google Scholar 

  44. P.C. Hansen, The discrete Picard condition for discrete ill-posed problems, BIT 30 (1990) 658–672.

    Google Scholar 

  45. P.C. Hansen, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev. 34 (1992) 561–580.

    Google Scholar 

  46. P.C. Hansen, Numerical tools for analysis and solution of Fredhold integral equations of the first kind, Inverse Problems 8 (1992) 849–872.

    Google Scholar 

  47. P.C. Hansen, Regularization Tools, a Matlab package for analysis and solution of discrete ill-posed problems; Version 2.0 for Matlab 4.0, Report UNIC-92-03 (March 1993). Available in PostScript form by e-mail via netlib (netlib@research.att.com) from the library NUMERALGO.

  48. P.C. Hansen and D.P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, Report CS-TR-2781, Dept. of Computer Science, Univ. of Maryland (1991), to appear in SIAM J. Sci. Comp.

  49. P.C. Hansen, D.P. O'Leary and G.W. Stewart, Regularizing properties of conjugate gradient iterations, work in progress.

  50. P.C. Hansen, T. Sekii and H. Shibahashi, The modified truncate SVD method for regularization is general form, SIAM J. Sci. Statist. Comp. 13 (1992) 1142–1150.

    Google Scholar 

  51. B. Hofmann,Regularization for Applied Inverse and Ill-Posed Problems (Teubner, Leipzig, 1986).

    Google Scholar 

  52. T. Kitagawa, A deterministic approach to optimal regularization — the finite dimensional case, Japan J. Appl. Math. 4 (1987) 371–391.

    Google Scholar 

  53. R. Kress,Linear Integral Equations (Springer, Berlin, 1989).

    Google Scholar 

  54. C.L. Lawson and R.J. Hanson,Solving Least Squares Problems (Prentice-Hall, Englewood Cliffs, 1974).

    Google Scholar 

  55. P. Linz, Uncertainty in the solution of linear operator equations, BIT 24 (1984) 92–101.

    Google Scholar 

  56. Matlab Reference Guide (The Math Works, MA, 1992).

  57. K. Miller, Least squares methods for ill-posed problems with a prescribed bound, SIAM J. Math. Anal. 1 (1970) 52–74.

    Google Scholar 

  58. V.A. Morozov,Methods for Solving Incorrectly Posed Problems (Springer, New York, 1984).

    Google Scholar 

  59. F. Natterer,The Mathematics of Computerized Tomography (Wiley, New York, 1986).

    Google Scholar 

  60. F. Natterer, Numerical treatment of ill-posed problems, in —.

    Google Scholar 

  61. D.P. O'Leary and J.A. Simmons, A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems, SIAM J. Sci. Stat. Comp. 2 (1981) 474–489.

    Google Scholar 

  62. C.C. Paige and M.A. Saunders, LSQR: an algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software 8 (1982) 43–71.

    Google Scholar 

  63. R.L. Parker, Understanding inverse theory, Ann. Rev. Earth Planet Sci. 5 (1977), 35–64.

    Google Scholar 

  64. D.L. Phillips, A technique for the numerical solution of certain integral equations of the first kind, J. ACM 9 (1962) 84–97.

    Google Scholar 

  65. F. Santosa, Y.-H. Pao, W.W. Symes and C. Holland (eds.),Inverse Problems of Acoustic and Elastic Waves (SIAM, Philadelphia, 1984).

    Google Scholar 

  66. C. Ray Smith and W.T. Grandy, Jr. (eds.),Maximum-Entropy and Bayesian Methods in Inverse Problems (Reidel, Boston, 1985).

    Google Scholar 

  67. G. Talenti,Inverse Problems, Lecture Notes in Mathematics, 1225 (Springer, Berlin, 1986).

    Google Scholar 

  68. H.J.J. te Tiele, A program for solving first kind Fredholm integral equations by means of regularization, Comp. Phys. Comm. 36 (1985) 423–432.

    Google Scholar 

  69. A.N. Tikhonov, Solution of incorrectly formulated problems and the regularization method, Dokl. Adak. Nauk. SSSR 151 (1963) 501–504 [Soviet Math. Dokl. 4 (1963) 1035–1038].

    Google Scholar 

  70. A.N. Tikhonov and V.Y. Arsenin,Solutions of Ill-Posed Problems (Winston, Washington, DC, 1977).

    Google Scholar 

  71. A.N. Tikhonov and A.V. Goncharsky,Ill-Posed Problems in the Natural Sciences (MIR, Moscow, 1987).

    Google Scholar 

  72. A. van der Sluis, The convergence behavior of conjugate gradients and Ritz values in various circumstances, in:Iterative Methods in Linear Algebra, eds. R. Bauwens and P. de Groen (North-Holland, Amsterdam, 1992).

    Google Scholar 

  73. A. van der Sluis and H.A. van der Vorst, SIRT- and CG-type methods for iterative solution of sparse linear least-squares problems, Lin. Alg. Appl. 130 (1990) 257–302.

    Google Scholar 

  74. J.M. Varah, On the numerical solution of ill-conditioned linear systems with applications to illposed problems, SIAM J. Numer. Anal. 10 (1973) 257–267.

    Google Scholar 

  75. J.M. Varah, A practical examination of some numerical methods for linear discrete ill-posed problems, SIAM Rev. 21 (1979) 100–111.

    Google Scholar 

  76. J.M. Varah, Pitfalls in the numerical solution of linear ill-possed problems, SIAM J. Sci. Statist. Comp. 4 (1983) 164–176.

    Google Scholar 

  77. C.R. Vogel, Optimal choice of a truncationlevel for the standard SVD solutio of linear first kind integral equations when data are noisy, SIAM J. Numer. Anal. 23 (1986) 109–117.

    Google Scholar 

  78. C.R. Vogel, Solving ill-conditioned linear systems using the conjugate gradient method, Report, Dept. of Mathematical Sciences, Montana State University (1987).

  79. G. Wahba,Spline Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59 (SIAM, Philidelphia, 1990).

    Google Scholar 

  80. G.M. Wing, Condition numbers of matrices arising from the numerical solution of linear integral equations of the first kind, J. Integral Equations 9 (Suppl.) (1985) 191–204.

    Google Scholar 

  81. G.M. Wing and J.D. Zahrt,A Primer on Integral Equations of the First Kind (SIAM, Philidelphia, 1991).

    Google Scholar 

  82. H. Zha and P.C. Hansen, Regularization and the general Gauss-Markov linear model, Math. Comp. 55 (1990) 613–624.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. UNI.C (Danish Computing Center for Research and Education), Technical University of Denmark, Building 305, DK-2800, Lyngby, Denmark

    Per Christian Hansen

Authors
  1. Per Christian Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by C. Brezinski and M. Redivo Zaglia

This work was supported by grants from Augustinus Fonden, Knud Højgaards Fond, and Civ. Ing. Frants Allings Legat.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hansen, P.C. REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems. Numer Algor 6, 1–35 (1994). https://doi.org/10.1007/BF02149761

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02149761

Keywords

Search

Navigation