Abstract
This chapter deals with iterative methods for nonlinear ill-posed problems. We present gradient and Newton type methods as well as nonstandard iterative algorithms such as Kaczmarz, expectation maximization, and Bregman iterations. Our intention here is to cite convergence results in the sense of regularization and to provide further references to the literature.
Keywords
- Bregman Iteration
- Gauss-Newton Type Method
- Landweber Iteration
- Minimal Error Method
- Iterative Regularization
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akcelik, V., Biros, G., Draganescu, A., Hill, J., Ghattas, O., Waanders, B.V.B.: Dynamic data-driven inversion for terascale simulations: real-time identification of airborne contaminants. In: Proceedings of SC05. IEEE/ACM, Seattle (2005)
Bachmayr, M., Burger, M.: Iterative total variation schemes for nonlinear inverse problems. Inverse Prob. 25, 105,004 (2009)
Bakushinsky, A.B.: The problem of the convergence of the iteratively regularized Gauss-Newton method. Comput. Math. Math. Phys. 32, 1353–1359 (1992)
Bakushinsky, A.B.: Iterative methods without degeneration for solving degenerate nonlinear operator equations. Dokl. Akad. Nauk. 344, 7–8 (1995)
Bakushinsky, A.B., Kokurin, M.Y.: Iterative Methods for Approximate Solution of Inverse Problems. Mathematics and Its Applications, vol. 577. Springer, Dordrecht (2004)
Bauer, F., Hohage, T.: A Lepskij-type stopping rule for regularized Newton methods. Inverse Prob. 21, 1975–1991 (2005)
Bauer, F., Kindermann, S.: The quasi-optimality criterion for classical inverse problems. Inverse Prob. 24(3), 035,002 (20 pp) (2008)
Bauer, F., Hohage, T., Munk, A.: Iteratively regularized Gauss-Newton method for nonlinear inverse problems with random noise. SIAM J. Numer. Anal. 47, 1827–1846 (2009)
Baumeister, J., Cezaro, A.D., Leitao, A.: On iterated Tikhonov-Kaczmarz regularization methods for ill-posed problems. ICJV (2010). doi:10.1007/s11263-010-0339-5
Baumeister, J., Kaltenbacher, B., Leitao, A.: On Levenberg-Marquardt Kaczmarz regularization methods for ill-posed problems. Inverse Prob. Imaging 4, 335–350 (2010)
Benning, M., Brune, C., Burger, M., Mueller, J.: Higher-order tv methods - enhancement via bregman iteration. J. Sci. Comput. 54(2-3), 269–310 (2013). In honor of Stanley Osher for his 70th birthday
Bertero, M., Boccacci, P.: Introduction to Inverse Problems in Imaging. Institute of Physics Publishing, Bristol (1998)
Bissantz, N., Hohage, T., Munk, A., Ruymgaart, F.: Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J. Numer. Anal. 45, 2610–2636 (2007)
Bissantz, N., Mair, B., Munk, A.: A statistical stopping rule for MLEM reconstructions in PET. IEEE Nucl. Sci. Symp. Conf. Rec 8, 4198–4200 (2008)
Blaschke, B., Neubauer, A., Scherzer, O.: On convergence rates for the iteratively regularized Gauss-Newton method. IMA J. Numer. Anal. 17, 421–436 (1997)
Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comp. Math. Math. Phys. 7, 200–217 (1967)
Brune, C., Sawatzky, A., Burger, M.: Bregman-EM-TV methods with application to optical nanoscopy. In: Tai, X.-C., et al. (ed.) Proceedings of the 2nd International Conference on Scale Space and Variational Methods in Computer Vision. Lecturer Notes in Computer Science, vol. 5567, pp. 235–246. Springer, New York (2009)
Brune, C., Sawatzky, A., Burger, M.: Primal and dual Bregman methods with application to optical nanoscopy. Int. J. Comput. Vis 92, 211–229 (2011)
Burger, M., Kaltenbacher, B.: Regularizing Newton-Kaczmarz methods for nonlinear ill-posed problems. SIAM J. Numer. Anal. 44, 153–182 (2006)
Burger, M., Resmerita, E., He, L.: Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81(2-3), 109–135 (2007)
Cai, J.F., Osher, S., Shen, Z.: Convergence of the linearized Bregman iteration for l1-norm minimization. Math. Comput. 78, 2127–2136 (2009)
Cai, J.F., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Math. Comput. 78, 1515–1536 (2009)
Cai, J.F., Osher, S., Shen, Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2, 226–252 (2009)
Colonius, F., Kunisch, K.: Output least squares stability in elliptic systems. Appl. Math. Optim. 19, 33–63 (1989)
Combettes, P.L., Pesquet, J.C.: A proximal decomposition method for solving convex variational inverse problems. Inverse Prob. 24, 065,014 (27 pp) (2008)
Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 57, 1413–1457 (2004)
De Cezaro, A., Haltmeier, M., Leitao, A., Scherzer, O.: On steepest-descent-Kaczmarz methods for regularizing systems of nonlinear ill-posed equations. Appl. Math. Comput. 202, 596–607 (2008)
Dembo, R., Eisenstat, S., Steihaug, T.: Inexact Newton’s method. SIAM J. Numer. Anal. 14, 400–408 (1982)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39, 1–38 (1977)
Deuflhard, P., Engl, H.W., Scherzer, O.: A convergence analysis of iterative methods for the solution of nonlinear ill-posed problems under affinely invariant conditions. Inverse Prob. 14, 1081–1106 (1998)
Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)
Egger, H.: Fast fully iterative Newton-type methods for inverse problems. J. Inverse Ill-Posed Prob. 15, 257–275 (2007)
Egger, H.: Y-Scale regularization. SIAM J. Numer. Anal. 46, 419–436 (2008)
Egger, H., Neubauer, A.: Preconditioning Landweber iteration in Hilbert scales. Numer. Math. 101, 643–662 (2005)
Eicke, B., Louis, A.K., Plato, R.: The instability of some gradient methods for ill-posed problems. Numer. Math. 58, 129–134 (1990)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Engl, H.W., Zou, J.: A new approach to convergence rate analysis of tiknonov regularization for parameter identification in heat conduction. Inverse Prob. 16, 1907–1923 (2000)
Glowinski, R., Tallec, P.L.: Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989)
Green, P.J.: Bayesian reconstructions from emission tomography data using a modified EM algorithm. IEEE Trans. Med. Imaging 9, 84–93 (1990)
Green, P.J.: On use of the EM algorithm for penalized likelihood estimation. J. R. Stat. Soc. Ser. B (Methodological) 52, 443–452 (1990)
Haber, E.: Quasi-Newton methods for large-scale electromagnetic inverse problems. Inverse Prob. 21, 305–323 (2005)
Haber, E., Ascher, U.: A multigrid method for distributed parameter estimation problems. Inverse Prob. 17, 1847–1864 (2001)
Haltmeier, M., Leitao, A., Scherzer, O.: Kaczmarz methods for regularizing nonlinear ill-posed equations I: convergence analysis. Inverse Prob. Imaging 1, 289–298 (2007)
Haltmeier, M., Kowar, R., Leitao, A., Scherzer, O.: Kaczmarz methods for regularizing nonlinear ill-posed equations II: applications. Inverse Prob. Imaging 1, 507–523 (2007)
Haltmeier, M., Leitao, A., Resmerita, E.: On regularization methods of EM-Kaczmarz type. Inverse Prob. 25(7), 17 (2009)
Hanke, M.: A regularization Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems. Inverse Prob. 13, 79–95 (1997)
Hanke, M.: Regularizing properties of a truncated Newton-CG algorithm for nonlinear inverse problems. Numer. Funct. Anal. Optim. 18, 971–993 (1997)
Hanke, M.: The regularizing Levenberg-Marquardt scheme is of optimal order. J. Integr. Equ. Appl. 22, 259–283 (2010)
Hanke, M., Neubauer, A., Scherzer, O.: A convergence analysis of the Landweber iteration for nonlinear ill-posed problems. Numer. Math. 72, 21–37 (1995)
He, L., Burger, M., Osher, S.: Iterative total variation regularization with non-quadratic fidelity. J. Math. Imaging Vis. 26, 167–184 (2006)
Hein, T., Kazimierski, K.: Modified Landweber iteration in Banach spaces – convergence and convergence rates. Numer. Funct. Anal. Optim. 31(10), 1158–1184 (2010)
Hochbruck, M., Hönig, M., Ostermann, A.: A convergence analysis of the exponential euler iteration for nonlinear ill-posed problems. Inverse Prob. 25, 075,009 (18 pp) (2009)
Hochbruck, M., Hönig, M., Ostermann, A.: Regularization of nonlinear ill-posed problems by exponential integrators. Math. Mod. Numer. Anal. 43, 709–720 (2009)
Hohage, T.: Logarithmic convergence rates of the iteratively regularized Gauß-Newton method for an inverse potential and an inverse scattering problem. Inverse Prob. 13, 1279–1299 (1997)
Hohage, T.: Iterative methods in inverse obstacle scattering: regularization theory of linear and nonlinear exponentially ill-posed problems. Ph.D. thesis, University of Linz (1999)
Hohage, T.: Regularization of exponentially ill-posed problems. Numer. Funct. Anal. Optim. 21, 439–464 (2000)
Hohage, T., Werner, F.: Iteratively regularized Newton methods with general data misfit functionals and applications to Poisson data. Numer. Math. 123, 745–779 (2013)
Jin, Q.: Inexact Newton-Landweber iteration for solving nonlinear inverse problems in Banach spaces. Inverse Prob. 28(6), 15 (2012)
Kaltenbacher, B.: Some Newton type methods for the regularization of nonlinear ill-posed problems. Inverse Prob. 13, 729–753 (1997)
Kaltenbacher, B.: On Broyden’s method for ill-posed problems. Numer. Funct. Anal. Optim. 19, 807–833 (1998)
Kaltenbacher, B.: A posteriori parameter choice strategies for some Newton type methods for the regularization of nonlinear ill-posed problems. Numer. Math. 79, 501–528 (1998)
Kaltenbacher, B.: A projection-regularized Newton method for nonlinear ill-posed problems and its application to parameter identification problems with finite element discretization. SIAM J. Numer. Anal. 37, 1885–1908 (2000)
Kaltenbacher, B.: On the regularizing properties of a full multigrid method for ill-posed problems. Inverse Prob. 17, 767–788 (2001)
Kaltenbacher, B.: Convergence rates for the iteratively regularized Landweber iteration in Banach space. In: Hmberg, D., Trltzsch, F. (eds.) System Modeling and Optimization. 25th IFIP TC 7 Conference on System Modeling and Optimization, CSMO 2011, Berlin, Germany, September 12–16, 2011. Revised Selected Papers, pp. 38–48. Springer, Heidelberg (2013)
Kaltenbacher, B., Hofmann, B.: Convergence rates for the iteratively regularized Gauss-Newton method in Banach spaces. Inverse Prob. 26, 035,007 (2010)
Kaltenbacher, B., Neubauer, A.: Convergence of projected iterative regularization methods for nonlinear problems with smooth solutions. Inverse Prob. 22, 1105–1119 (2006)
Kaltenbacher, B., Schicho, J.: A multi-grid method with a priori and a posteriori level choice for the regularization of nonlinear ill-posed problems. Numer. Math. 93, 77–107 (2002)
Kaltenbacher, B., Tomba, I.: Convergence rates for an iteratively regularized Newton-Landweber iteration in Banach space. Inverse Prob. 29, 025010 (2013). doi:10.1088/0266-5611/29/2/025010
Kaltenbacher, B., Neubauer, A., Ramm, A.G.: Convergence rates of the continuous regularized Gauss-Newton method. J. Inverse Ill-Posed Prob. 10, 261–280 (2002)
Kaltenbacher, B., Neubauer, A., Scherzer, O.: Iterative Regularization Methods for Nonlinear Ill-Posed Problems. Radon Series on Computational and Applied Mathematics. de Gruyter, Berlin (2008)
Kaltenbacher, B., Schöpfer, F., Schuster, T.: Iterative methods for nonlinear ill-posed problems in Banach spaces: convergence and applications to parameter identification problems. Inverse Prob. 25, 065,003 (19 pp) (2009)
Kaltenbacher, B., Kirchner, A., Veljovic, S.: Goal oriented adaptivity in the IRGNM for parameter identification in PDEs I: reduced formulation Inverse Prob. 30, 045001 (2014)
Kaltenbacher, B., Kirchner, A., Vexler, B.: Goal oriented adaptivity in the IRGNM for parameter identification in PDEs II: all-at once formulations Inverse Prob. 30, 045002 (2014)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. ETNA 38, 233–257 (2011)
Kindermann, S., Neubauer, A.: On the convergence of the quasioptimality criterion for (iterated) Tikhonov regularization. Inverse Prob. Imaging 2, 291–299 (2008)
King, J.T.: Multilevel algorithms for ill-posed problems. Numer. Math. 61, 311–334 (1992)
Kowar, R., Scherzer, O.: Convergence analysis of a Landweber-Kaczmarz method for solving nonlinear ill-posed problems. In: Romanov, V.G., Kabanikhin, S.I., Anikonov, Y.E., Bukhgeim, A.L. (eds.) Ill-Posed and Inverse Problems, pp. 69–90. VSP, Zeist (2002)
Krein, S.G., Petunin, J.I.: Scales of Banach spaces. Russ. Math. Surv. 21, 85–160 (1966)
Kügler, P.: A derivative free Landweber iteration for parameter identification in certain elliptic PDEs. Inverse Prob. 19, 1407–1426 (2003)
Kügler, P.: A derivative free Landweber method for parameter identification in elliptic partial differential equations with application to the manufacture of car windshields. Ph.D. thesis, Johannes Kepler University, Linz, Austria (2003)
Langer, S.: Preconditioned Newton methods for ill-posed problems. Ph.D. thesis, University of Göttingen (2007)
Langer, S., Hohage, T.: Convergence analysis of an inexact iteratively regularized Gauss-Newton method under general source conditions. J. Inverse Ill-Posed Prob. 15, 19–35 (2007)
Leitao, A., Marques Alves, M.: On Landweber-Kaczmarz methods for regularizing systems of ill-posed equations in Banach spaces. Inverse Prob. 28, 104,008 (15 pp) (2012)
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)
Mülthei, H.N., Schorr, B.: On properties of the iterative maximum likelihood reconstruction method. Math. Methods Appl. Sci. 11, 331–342 (1989)
Natterer, F., Wübbeling, F.: Mathematical Methods in Image Reconstruction. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2001)
Neubauer, A.: Tikhonov regularization of nonlinear ill-posed problems in Hilbert scales. Appl. Anal. 46, 59–72 (1992)
Neubauer, A.: On Landweber iteration for nonlinear ill-posed problems in Hilbert scales. Numer. Math. 85, 309–328 (2000)
Neubauer, A.: The convergence of a new heuristic parameter selection criterion for general regularization methods. Inverse Prob. 24, 055,005 (10 pp) (2008)
Neubauer, A., Scherzer, O.: A convergent rate result for a steepest descent method and a minimal error method for the solution of nonlinear ill-posed problems. ZAA 14, 369–377 (1995)
Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation based image restoration. SIAM Multiscale Model. Simul. 4, 460–489 (2005)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes: The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)
Resmerita, E., Engl, H.W., Iusem, A.N.: The expectation-maximization algorithm for ill-posed integral equations: a convergence analysis. Inverse Prob. 23, 2575–2588 (2007)
Rieder, A.: On the regularization of nonlinear ill-posed problems via inexact Newton iterations. Inverse Prob. 15, 309–327 (1999)
Rieder, A.: On convergence rates of inexact Newton regularizations. Numer. Math. 88, 347–365 (2001)
Rieder, A.: Inexact Newton regularization using conjugate gradients as inner iteration. SIAM J. Numer. Anal. 43, 604–622 (2005)
Sawatzky, A., Brune, C., Wübbeling, F., Kösters, T., Schäfers, K., Burger, M.: Accurate EM-TV algorithm in PET with low SNR. Nuclear Science Symposium Conference Record, 2008. NSS ’08, pp. 5133–5137. IEEE, New York (2008)
Scherzer, O.: A modified Landweber iteration for solving parameter estimation problems. Appl. Math. Optim. 38, 45–68 (1998)
Schöpfer, F., Louis, A.K., Schuster, T.: Nonlinear iterative methods for linear ill-posed problems in Banach spaces. Inverse Prob. 22, 311–329 (2006)
Schöpfer, F., Schuster, T., Louis, A.K.: An iterative regularization method for the solution of the split feasibility problem in Banach spaces. Inverse Prob. 24, 055,008 (20pp) (2008)
Schuster, T., Kaltenbacher, B., Hofmann, B., Kazimierski, K.: Regularization Methods in Banach Spaces. Radon Series on Computational and Applied Mathematics. de Gruyter, Berlin/New York (2012)
Stück, R., Burger, M., Hohage, T.: The iteratively regularized gauss-newton method with convex constraints and applications in 4pi-microscopy. Inverse Prob. 28, 015,012 (2012)
Vardi, Y., Shepp, L.A., Kaufman, L.: A statistical model for positron emission tomography with discussion. J. Am. Stat. Assoc. 80, 8–37 (1985)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer Science+Business Media New York
About this entry
Cite this entry
Burger, M., Kaltenbacher, B., Neubauer, A. (2015). Iterative Solution Methods. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0790-8_9
Download citation
DOI: https://doi.org/10.1007/978-1-4939-0790-8_9
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-0789-2
Online ISBN: 978-1-4939-0790-8
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering