Skip to main content
SpringerLink
Log in

The operator splitting schemes revisited: primal-dual gap and degeneracy reduction by a unified analysis

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

We revisit the operator splitting schemes proposed in a recent work of [Some extensions of the operator splitting schemes based on Lagrangian and primal-dual: A unified proximal point analysis, Feng Xue, Optimization, 2022, doi: 10.1080/02331934.2022.2057309], and further analyze the convergence of the generalized Bregman distance and the primal-dual gap of these algorithms within a unified proximal point framework. The possibility of reduction to a simple resolvent is also discussed by exploiting the structure and possible degeneracy of the underlying metric.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We say a convex function f is proximable, if the proximity operator of f has a closed-form representation or at least can be solved efficiently up to high precision [17]. This property is also called ‘simple’ [17] or ‘with inexpensive proximity operator’ [36].

  2. Note that the well-known proximal forward-backward splitting (PFBS) algorithm may not in general be applied to solve (1), since neither f nor g is assumed to be differentiable with a Lipschitz continuous gradient.

  3. For any pair of \(({\textbf{c}},{\textbf{c}}^{\prime })\), the quantity of \(\Pi ({\textbf{c}},{\textbf{c}}^{\prime })\) is generally only a difference, but not a distance, since it is not guaranteed to be non-negative.

References

  1. Briceño Arias, L., Combettes, P.: A monotone+skew splitting model for composite monotone inclusions in duality. SIAM J. Control. Optim. 21(4), 1230–1250 (2011)

    Article  MathSciNet  Google Scholar 

  2. Briceño Arias, L., Roldán, F.: Resolvent of the parallel composition and the proximity operator of the infimal postcomposition. Opt. Lett. (2022). https://doi.org/10.1007/s11590-022-01906-5

    Article  Google Scholar 

  3. Bai, J., Zhang, H., Li, J.: A parameterized proximal point algorithm for separable convex optimization. Optim. Lett. 12, 1589–1608 (2018)

    Article  MathSciNet  Google Scholar 

  4. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. CMS Books in Mathematics, Springer, New York, NY (2017)

    Book  Google Scholar 

  5. Beck, A.: First-order methods in optimization. SIAM-Society for Industrial and Applied Mathematics (2017)

  6. Boţ, R., Csetnek, E.: On the convergence rate of a forward-backward type primal-dual primal-dual splitting algorithm for convex optimization problems. Optimization 64(1), 5–23 (2014)

    Article  MathSciNet  Google Scholar 

  7. Boţ, R.I., Hendrich, C.: Convergence analysis for a primal-dual monotone+skew splitting algorithm with applications to total variation minimization. J. Math. Imaging Vis. 49, 551–568 (2014)

    Article  MathSciNet  Google Scholar 

  8. Bredies, K., Chenchene, E., Lorenz, D.A., Naldi, E.: Degenerate preconditioned proximal point algorithms. SIAM J. Optim. 32(3), 2376–2401 (2022)

    Article  MathSciNet  Google Scholar 

  9. Bredies, K., Sun, H.: A proximal point analysis of the preconditioned alternating direction method of multipliers. J. Optim. Theory Appl. 173, 878–907 (2017)

    Article  MathSciNet  Google Scholar 

  10. Cai, J., Osher, S., Shen, Z.: Linearized bregman iterations for compressed sensing. Math. Comput. 78, 1515–1536 (2009)

    Article  MathSciNet  Google Scholar 

  11. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  12. Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. Ser. A 159(1–2), 253–287 (2016)

    Article  MathSciNet  Google Scholar 

  13. Chouzenoux, E., Pesquet, J.C., Repetti, A.: A block coordinate variable metric forward-backward algorithm. J. Glob. Optim. 66, 457–485 (2016)

    Article  MathSciNet  Google Scholar 

  14. Combettes, P., Pesquet, J.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20(2), 307–330 (2012)

    Article  MathSciNet  Google Scholar 

  15. Combettes, P., Pesquet, J.: Fixed point strategies in data science. IEEE Transact. Signal Process. 69, 3878–3905 (2021)

    Article  MathSciNet  Google Scholar 

  16. Combettes, P., Wajs, V.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    Article  MathSciNet  Google Scholar 

  17. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable, and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  Google Scholar 

  18. Drori, Y., Sabach, S., Teboulle, M.: A simple algorithm for a class of nonsmooth convex-concave saddle-point problems. Oper. Res. Lett. 43(2), 209–214 (2015)

    Article  MathSciNet  Google Scholar 

  19. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1), 293–318 (1992)

    Article  MathSciNet  Google Scholar 

  20. Frankel, P., Garrigos, G., Peypouquet, J.: Splitting methods with variable metric for Kurdyka-Łojasiewicz Functions and General Convergence Rates. J. Optim. Theory Appl. 165(3), 874–900 (2015)

    Article  MathSciNet  Google Scholar 

  21. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

    Book  Google Scholar 

  22. Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis d’ordure un et la résolution par pénalisation-dualité d’une classe de problèmes de dirichlet non linéaires. Revue Fr. Autom. Inf. Rech. Opér. Anal. Numér. 2, 41–76 (1975)

  23. Goldstein, T., Osher, S.: The split Bregman method for \(\ell _1\)-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  Google Scholar 

  24. Gonçalves, M.L.N., Marques, A.M., Melo, J.G.: Pointwise and ergodic convergence rates of a variable metric proximal alternating direction method of multipliers. J. Optim. Theory Appl. 177, 448–478 (2018)

    Article  MathSciNet  Google Scholar 

  25. He, B., Ma, F., Yuan, X.: An algorithmic framework of generalized primal-dual hybrid gradient methods for saddle point problems. J. Math. Imaging Vis. 58(2), 279–293 (2017)

    Article  MathSciNet  Google Scholar 

  26. He, B., Xu, M., Yuan, X.: Block-wise ADMM with a relaxation factor for multiple-block convex programming. J. Oper. Res. Soc. China 6, 485–505 (2018)

    Article  MathSciNet  Google Scholar 

  27. He, B., Yuan, X.: On the \({\cal{O} }(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numerical Analysis 50(2), 700–709 (2012)

    Article  MathSciNet  Google Scholar 

  28. He, B., Yuan, X.: On non-ergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)

    Article  MathSciNet  Google Scholar 

  29. He, B., Yuan, X.: A class of ADMM-based algorithms for three-block separable convex programming. Comput. Optim. Appl. 70, 791–826 (2018)

    Article  MathSciNet  Google Scholar 

  30. Kiwiel, K.: Proximal minimization methods with generalized bregman functions. SIAM J. Control. Optim. 35(4), 1142–1168 (1997)

    Article  MathSciNet  Google Scholar 

  31. Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  Google Scholar 

  32. Ma, F., Ni, M.: A class of customized proximal point algorithms for linearly constrained convex optimization. Comp. Appl. Math. 37, 896–911 (2018)

    Article  MathSciNet  Google Scholar 

  33. Martínez-Legaz, R.S.B.J.E.: On bregman-type distances for convex functions and maximally monotone operators. Set-Valued Var. Anal. 26, 369–384 (2018)

  34. Nemirovski, A.: Prox-method with rate of convergence \({\cal{O} }(1/t)\) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004)

    Article  MathSciNet  Google Scholar 

  35. Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4(2), 460–489 (2005)

    Article  MathSciNet  Google Scholar 

  36. O’Connor, D., Vandenberghe, L.: Primal-dual decomposition by operator splitting and applications to image deblurring. SIAM J. Imaging Sci. 7(3), 1724–1754 (2014)

    Article  MathSciNet  Google Scholar 

  37. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: IEEE Int. Conf. on Computer Vision, pp. 1133–1140 (2009)

  38. Rockafellar, R.T.: Convex analysis. Princeton Landmarks in Mathematics and Physics, Princeton University Press (1996)

  39. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer, Grundlehren der Mathematischen Wissenschaft, vol. 317 (2004)

  40. Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    Article  MathSciNet  Google Scholar 

  41. Tran-Dinh, Q., Fercoq, O., Cevher, V.: A smooth primal-dual optimization framework for nonsmooth composite convex minimization. SIAM J. Optim. 28(1), 96–134 (2018)

    Article  MathSciNet  Google Scholar 

  42. Vũ, B.: A splitting algorithm for coupled system of primal-dual monotone inclusions. J. Optim. Theory Appl. 164, 993–1025 (2015)

    Article  MathSciNet  Google Scholar 

  43. Valkonen, T., Pock, T.: Acceleration of the PDHGM on partially strongly convex functions. J. Math. Imaging Vis. 59(3), 394–414 (2017)

    Article  MathSciNet  Google Scholar 

  44. Xue, F.: On the metric resolvent: nonexpansiveness, convergence rates and applications. arXiv preprint: arXiv:2108.06502 (2021)

  45. Xue, F.: On the nonexpansive operators based on arbitrary metric: a degenerate analysis. RM (2022). https://doi.org/10.1007/s00025-022-01766-6

    Article  Google Scholar 

  46. Xue, F.: Some extensions of the operator splitting schemes based on Lagrangian and primal-dual: a unified proximal point analysis. Optimization (2022). https://doi.org/10.1080/02331934.2022.2057309

    Article  Google Scholar 

  47. Yan, M.: A new primal-dual algorithm for minimizing the sum of three functions with a linear operator. J. Sci. Comput. 76, 1698–1717 (2018)

    Article  MathSciNet  Google Scholar 

  48. Yan, M., Yin, W.: Self Equivalence of the Alternating Direction Method of Multipliers, pp. 165–194. Springer, Cham. (2016)

  49. Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for \(\ell _1\)-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

    Article  MathSciNet  Google Scholar 

  50. Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)

    Article  MathSciNet  Google Scholar 

  51. Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Report 08-34, UCLA (2008)

Download references

Acknowledgements

I am gratefully indebted to the anonymous reviewers and the editor for helpful discussions, particularly related to the convergence analysis of PPA (Lemma 1), the notion of infimal postcomposition (Lemma 2), the generalized Bregman distance and primal-dual gap (Sects. 3.3, 3.4, 4.2 and 5.2 ), and for bringing references [34, 37, 47] to my attention.

Author information

Authors and Affiliations

  1. National key laboratory, Beijing, China

    Feng Xue

Authors
  1. Feng Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Xue.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, F. The operator splitting schemes revisited: primal-dual gap and degeneracy reduction by a unified analysis. Optim Lett 18, 155–194 (2024). https://doi.org/10.1007/s11590-023-01983-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-023-01983-0

Keywords

Search

Navigation