Skip to main content
SpringerLink
Log in

An augmented Lagrangian method for distributed optimization

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

We propose a novel distributed method for convex optimization problems with a certain separability structure. The method is based on the augmented Lagrangian framework. We analyze its convergence and provide an application to two network models, as well as to a two-stage stochastic optimization problem. The proposed method compares favorably to two augmented Lagrangian decomposition methods known in the literature, as well as to decomposition methods based on the ordinary Lagrangian function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Berge, C.: Espaces Topologiques Functions Multivoques. Dunod, Paris (1959)

  2. Berger, A.J., Mulvey, J.M., Ruszczyński, A.: An extension of the DQA algorithm to convex stochastic programs. SIAM J. Optim. 4(4), 735–753 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertsekas, D.: Extended monotropic programming and duality. J. Optim. Theory Appl. 139(2), 209–225 (2008)

    Article  MathSciNet  Google Scholar 

  4. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific, Belmont (1997)

    Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  6. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  7. Eckstein, J.: The alternating step method for monotropic programming on the connection machine CM-2. ORSA J. Comput. 5(1), 84 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  8. Eckstein, J.: Augmented Lagrangian and Alternating Direction Methods for Convex Optimization: A Tutorial and Some Illustrative Computational Results. Rutcor Research Report, Rutgers (2012)

  9. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)

    MATH  Google Scholar 

  11. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  12. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  13. Kiwiel, K.C., Rosa, C.H., Ruszczynski, A.: Proximal decomposition via alternating linearization. SIAM J. Optim. 9(3), 668–689 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  14. Mulvey, J., Ruszczyński, A.: A diagonal quadratic approximation method for large scale linear programs. Oper. Res. Lett. 12, 205–215 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  15. Powell, M.J.D.: A Method for Nonlinear Constraints in Minimization Problems. Optimization. Academic Press, London (1969)

    Google Scholar 

  16. Rockafellar, R.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control 12, 268–285 (1973)

    Article  MathSciNet  Google Scholar 

  17. Rockafellar, R.: Augmented lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. 1, Res. 97–116 (1976)

  18. Rockafellar, R.: Network Flows and Monotropic Optimization. Wiley, New York (1984)

    MATH  Google Scholar 

  19. Rockafellar, R.: Monotropic programming: a generalization of linear programming and network programming. In: Convexity and Duality in Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 256, pp. 10–36. Springer, Berlin (1985)

  20. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  21. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  22. Rockafellar, R.T., Wets, R.B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 1–23 (1991)

    Article  MathSciNet  Google Scholar 

  23. Ruszczyński, A.: On convergence of an augmented Lagrangian decomposition method for sparse convex optimization. Math. Oper. Res. 20, 634–656 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  24. Ruszczyński, A.: Decompostion methods. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming, Handbooks in Operations Research and Management Science, pp. 141–211. Elsevier, Amsterdam (2003)

    Google Scholar 

  25. Ruszczyński, A.: Nonlinear Optim. Princeton University Press, Princeton (2006)

    Google Scholar 

  26. Ruszczyński, A., Shapiro, A.: Optimality and duality in stochastic programming. In: Shapiro, A., Ruszczyński, A. (eds.) Stochastic Programming, Handbooks in Operations Research and Management Science, pp. 65–139. Elsevier, Amsterdam (2003)

  27. Shakkottai, S., Srikant, R.: Network optimization and control. Found. Trends Netw. 2(3), 271–379 (2007)

    Article  MATH  Google Scholar 

  28. Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming. MPS-SIAM Series on Optimization. MPS-SIAM (2009)

  29. Stephanopoulos, G., Westerberg, A.W.: The use of hestenes method of multipliers to resolve dual gaps in engineering system optimization. J. Optim. Theory Appl. 15, 285–309 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  30. Tatjewski, P.: New dual-type decomposition algorithm for nonconvex separable optimization problems. Automatica 25(2), 233–242 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  31. Watanabe, N., Nishimura, Y., Matsubara, M.: Decomposition in large system optimization using the method of multipliers. J. Optim. Theory Appl. 25(2), 181–193 (1978)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the two anonymous referees whose comments helped improve the paper.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC, 27708, USA

    Nikolaos Chatzipanagiotis & Michael M. Zavlanos

  2. Department of Mathematics, Stevens Institute of Technology, Hoboken, NJ, 07030, USA

    Darinka Dentcheva

Authors
  1. Nikolaos Chatzipanagiotis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Darinka Dentcheva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael M. Zavlanos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos Chatzipanagiotis.

Additional information

This work was supported by the NSF awards DMS #1311978, CNS #1261828 and CNS #1302284.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chatzipanagiotis, N., Dentcheva, D. & Zavlanos, M.M. An augmented Lagrangian method for distributed optimization. Math. Program. 152, 405–434 (2015). https://doi.org/10.1007/s10107-014-0808-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-014-0808-7

Keywords

Mathematics Subject Classification

Search

Navigation