Skip to main content
SpringerLink
Log in

MiKM: multi-step inertial Krasnosel’skiǐ–Mann algorithm and its applications

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we first introduce a multi-step inertial Krasnosel’skiǐ–Mann algorithm (MiKM) for nonexpansive operators in real Hilbert spaces. We give the convergence of the MiKM by investigating the convergence of the Krasnosel’skiǐ–Mann algorithm with perturbations. We also establish global pointwise and ergodic iteration complexity bounds of the Krasnosel’skiǐ–Mann algorithm with perturbations. Based on the MiKM, we construct some multi-step inertial splitting methods, including the multi-step inertial Douglas–Rachford splitting method (MiDRS), the multi-step inertial forward–backward splitting method, multi-step inertial backward–forward splitting method and and the multi-step inertial Davis–Yin splitting method. Numerical experiments are provided to illustrate the advantage of the MiDRS over the one-step inertial DRS and the original DRS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J. Optim. 14, 773–782 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch, H., Peypouquet, J., Redont, P.: A dynamical approach to an inertial forward–backward algorithm for convex minimization. SIAM J. Optim. 24, 232–256 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Peypouquet, J., Redont, P.: Backward–forward algorithms for structured monotone inclusions in Hilbert spaces. J. Math. Anal. Appl. 457, 1095–1117 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  4. Aubin, J.P., Frankowska, H.: Set-Valued Analysis. Birkhäuser, Boston (1990)

    MATH  Google Scholar 

  5. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)

    Book  MATH  Google Scholar 

  6. Bot, R.I., Csetnek, E.R.: An inertial forward–backward–forward primal-dual splitting algorithm for solving monotone inclusion problems. Numer. Algorithm 71, 1–22 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bot, R.I., Csetnek, E.R., Hendrich, C.: Inertial Douglas–Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)

    MathSciNet  MATH  Google Scholar 

  8. Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 20, 103–120 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Censor, Y.: Weak and strong superiorization: between feasibility-seeking and minimization. An. St. Univ. Ovidius Constanta Ser. Mat. 23, 41–54 (2015)

    MathSciNet  MATH  Google Scholar 

  10. Censor, Y., Davidi, R., Herman, G.T.: Perturbation resilience and superiorization of iterative algorithms. Inverse Probl. 26, 065008 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  11. Censor, Y., Zaslavski, A.J.: Strict Fej\(\acute{e}\)r monotonicity by superiorization of feasibility-seeking projection methods. J. Optim. Theory Appl. 165, 172–187 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Chen, C., Chan, C.H., Ma, S., Yang, J.: Inertial proximal ADMM for linearly constrained separable convex optimization. Siam J. Imaging Sci. 8, 2239–2267 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen, P., Huang, J., Zhang, X.: A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 29, 025011 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  15. Combettes, P.L., Glaudin, L.E.: Quasi-nonexpansive iterations on the affine hull of orbits: from Mann’s mean value algorithm to inertial methods. SIAM J. Optim. 27, 2356–2380 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  16. Combettes, P.L., Pennanen, T.: Generalized Mann iterates for constructing fixed points in Hilbert spaces. J. Math. Anal. Appl. 275, 521–536 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Combettes, P.L., Pesquet, J.C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20, 307–330 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Combettes, P.L., Vũ, B.C.: Variable metric forward–backward splitting with applications to monotone inclusions in duality. Optimization 63, 1289–1318 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  19. Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. Set-Valued Var. Anal. 25, 829–858 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  20. Dong, Q.L., Cho, Y.J., Rassias, T.M.: General inertial Mann algorithms and their convergence analysis for nonexpansive mappings. In: Rassias, T.M. (ed.) Applications of Nonlinear Analysis, pp. 175–191. Springer, Berlin (2018)

    Chapter  Google Scholar 

  21. Dong, Q.L., Cho, Y.J., Zhong, L.L., Rassias, T.M.: Inertial projection and contraction algorithms for variational inequalities. J. Glob. Optim. 70, 687–704 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dong, Q.L., Gibali, A., Jiang, D., Ke, S.H.: Convergence of projection and contraction algorithms with outer perturbations and their applications to sparse signals recovery. J. Fixed Point Theory Appl. 20, 16 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Dong Q.L., Li, X.H., Cho, Y.J., Rassias, T.M.: Multi-step inertial Krasnosel’skiǐ–Mann iterations on the affine hull of orbits. J. Glob. Optim. (under review)

  24. Dong, Q.L., Yuan, H.B.: Accelerated Mann and CQ-algorithms for finding a fixed point of a nonexpansive mapping. Fixed Point Theory Appl. 2015, 125 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Dong, Q.L., Yuan, B.H., Cho, Y.J., Rassias, T.M.: Modified inertial Mann algorithm and inertial CQ-algorithm for nonexpansive mappings. Optim. Lett. 12, 87–102 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  27. Eckstein, J., Svaiter, B.F.: A family of projective splitting methods for the sum of two maximal monotone operators. Math. Program. Ser. B 111, 173–199 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  28. Garduño, E., Herman, G.T.: Superiorization of the ML-EM algorithm. IEEE Trans. Nucl. Sci. 61, 162–172 (2014)

    Article  Google Scholar 

  29. Glowinski, R., Le Tallec, P. (eds.): Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989)

    MATH  Google Scholar 

  30. Herman, G.T., Davidi, R.: Image reconstruction from a small number of projections. Inverse Probl. 24, 045011 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  31. Herman, G.T., Garduño, E., Davidi, R., Censor, Y.: Superiorization: an optimization heuristic for medical physics. Med. Phys. 39, 5532–5546 (2012)

    Article  Google Scholar 

  32. Iiduka, H.: Iterative algorithm for triple-hierarchical constrained nonconvex optimization problem and its application to network bandwidth allocation. SIAM J. Optim. 22, 862–878 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Iiduka, H.: Fixed point optimization algorithms for distributed optimization in networked systems. SIAM J. Optim. 23, 1–26 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  34. Krasnosel’skiǐ, M.A.: Two remarks on the method of successive approximations. Usp. Mat. Nauk 10, 123–127 (1955)

    MathSciNet  Google Scholar 

  35. Liang, J.W.: Convergence rates of first-order operator splitting methods. In: Optimization and Control [math.OC]. Normandie Université; GREYC CNRS UMR 6072, English (2016)

  36. Liang, J.W., Fadili, J., Peyré, G.: Convergence rates with inexact non-expansive operators. Math. Program. Ser. A 159, 403–434 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  37. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  38. Lorenz, D.A., Pock, T.: An inertial forward–backward algorithm for monotone inclusions. J. Math. Imaging Vis. 51, 311–325 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  39. Mainge, P.E., Gobinddass, M.L.: Convergence of one-step projected gradient methods for variational inequalities. J. Optim. Theory Appl. 171, 146–168 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  40. Mainge, P.E.: Numerical approach to monotone variational inequalities by a one-step projected reflected gradient method with line-search procedure. Comput. Math. Appl. 72, 720–728 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  41. Mainge, P.E.: Convergence theorems for inertial KM-type algorithms. J. Comput. Appl. Math. 219, 223–236 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  42. Malitski, Yu.: Projected reflected gradient method for variational inequalities. SIAM J. Optim. 25, 502–520 (2015)

    Article  MathSciNet  Google Scholar 

  43. Mann, W.R.: Mean value methods in iteration. Proc. Am. Math. Soc. 4, 506–510 (1953)

    Article  MathSciNet  MATH  Google Scholar 

  44. Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 279, 045009 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  45. Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. 29, 341–346 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  46. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate O(1/\(k^2\)). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  47. Nesterov, Y.E.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004)

    Book  MATH  Google Scholar 

  48. Ogura, N., Yamada, I.: Non-strictly convex minimization over the fixed point set of an asymptotically shrinking nonexpansive mapping. Numer. Funct. Anal. Optim. 23, 113–137 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  49. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic, New York (1970)

    MATH  Google Scholar 

  50. Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72, 383–390 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  51. Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3, 28–41 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  52. Peng, Z., Wu, T., Xu, Y., Yan, M., Yin, W.: Coordinate friendly structures, algorithms and applications. Ann. Mater. Sci. Appl. 1(1), 57–119 (2016)

    MathSciNet  MATH  Google Scholar 

  53. Peypouquet, J.: Convex Optimization in Normed Spaces: Theory, Methods and Examples. Springer, Berlin (2015)

    Book  MATH  Google Scholar 

  54. Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. U.S.S.R. Comput. Math. Math. Phys. 4, 1–17 (1964)

    Article  Google Scholar 

  55. Polyak, B.T.: Introduction to Optimization. Optimization Software (1987)

  56. Raguet, H., Fadili, J., Peyré, G.: A generalized forward–backward splitting. SIAM J. Imaging Sci. 6, 1199–1226 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  57. Xu, H.K.: Averaged mappings and the gradient-projection algorithm. J. Optim. Theory Appl. 150, 360–378 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  58. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667–681 (2013)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to express our thanks to Dr. Jingwei Liang for his help in program and for having drawn the authors’ attention to Davis–Yin’s splitting method.

Author information

Authors and Affiliations

  1. Tianjin Key Lab for Advanced Signal Processing and College of Science, Civil Aviation University of China, Tianjin, 300300, People’s Republic of China

    Q. L. Dong & X. H. Li

  2. LSEC, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, People’s Republic of China

    J. Z. Huang

  3. School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

    J. Z. Huang

  4. Department of Mathematics Education and RINS, Gyeongsang National University, Jinju, 660-701, Korea

    Y. J. Cho

  5. School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, People’s Republic of China

    Y. J. Cho

  6. Department of Mathematics, National Technical University of Athens, Zografou Campus, 15780, Athens, Greece

    Th. M. Rassias

Authors
  1. Q. L. Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Z. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. X. H. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Y. J. Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Th. M. Rassias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Q. L. Dong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supported by National Natural Science Foundation of China (No. 71602144) and Open Fund of Tianjin Key Lab for Advanced Signal Processing (No. 2016ASP-TJ01).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, Q.L., Huang, J.Z., Li, X.H. et al. MiKM: multi-step inertial Krasnosel’skiǐ–Mann algorithm and its applications. J Glob Optim 73, 801–824 (2019). https://doi.org/10.1007/s10898-018-0727-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-018-0727-x

Keywords

Search

Navigation