Abstract
In this paper, we analyze the exponential method of multipliers for convex constrained minimization problems, which operates like the usual Augmented Lagrangian method, except that it uses an exponential penalty function in place of the usual quadratic. We also analyze a dual counterpart, the entropy minimization algorithm, which operates like the proximal minimization algorithm, except that it uses a logarithmic/entropy “proximal” term in place of a quadratic. We strengthen substantially the available convergence results for these methods, and we derive the convergence rate of these methods when applied to linear programs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D.P. Bertsekas, G.S. Lauer, N.R. Sandell, Jr. and T.A. Posbergh, “Optimal short term scheduling of large-scale power systems,”IEEE Transactions on Automatic Control AC-28 (1982) 1–11.
D.P. Bertsekas and J.N. Tsitsiklis,Parallel and Distributed Computation: Numerical Methods (Prentice-Hall, Englewood Cliffs, NJ, 1989).
D.P. Bertsekas, “Necessary and sufficient conditions for a penalty method to be exact,”Mathematical Programming 9 (1975) 87–99.
D.P. Bertsekas, “Newton's method for linear optimal control problems,” in:Proceedings of the Symposium on Large Scale Systems (Udine, 1976) pp. 353–359.
D.P. Bertsekas,Constrained Optimization and Lagrange Multiplier Methods (Academic Press, New York, 1982).
L.M. Bregman, “The relaxation method of finding the common point convex sets and its application to the solution of problems in convex programming,”USSR Computational Mathematics and Mathematical Physics 7 (1967) 200–217.
Y. Censor and S.A. Zenios, “The proximal minimization algorithm with D-functions,” (1989), to appear in:Journal of Optimization Theory and Applications.
G. Chen and M. Teboulle, “Convergence analysis of a proximal-like minimization algorithm using Bregman functions,” (1990), to appear in:SIAM Journal on Optimization.
G.B. Dantzig,Linear Programming and Extensions (Princeton University Press, Princeton, NJ, 1963).
J. Eckstein, “Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming,” (1990), to appear in:Mathematics of Operations Research.
A.V. Fiacco and G.P. McCormick,Nonlinear Programming: Sequential Unconstrained Minimization Techniques (Wiley, New York, 1968).
E.G. Golshtein and N.V. Tretjiakov, “Modified Lagrangians in convex programming and their generalizations,”Mathematical Programming Studies 10 (1979) 86–97.
C.D. Ha, “A generalization of the proximal point algorithm,”SIAM Journal on Control and Optimization 28 (1990) 503–512.
A.J. Hoffman, “On approximate solutions of systems of linear inequalities,”Journal of Research of the National Bureau of Standards 49 (1952) 263–265.
B.W. Kort and D.P. Bertsekas, “A new penalty function method for constrained minimization,” in:Proceedings of the 1972IEEE conference on decision and control (New Orleans, 1972) pp. 162–166.
G.S. Lauer, D.P. Bertsekas, N.R. Sandell, Jr. and T.A. Posbergh, “Optimal solution of large-scale unit commitment problems,”IEEE Transactions on Power Systems and Apparatus 101 (1981) 79–86.
D.G. Luenberger,Linear and Nonlinear Programming (Addison-Wesley, Reading, MA, 1984).
F.J. Luque, “Nonlinear proximal point algorithms,” Ph.D. Thesis, Department of Civil Engineering and Operations Research Center, Massachusetts Institute of Technology (Cambridge, MA, 1984).
F.J. Luque, “The nonlinear proximal point algorithm,” Report LIDS-P-1598, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology (Cambridge, MA, 1986).
O.L. Mangasarian and T.-H. Shiau, “Lipschitz continuity of solutions of linear inequalities, programs and complementarity problems,”SIAM Journal of Control and Optimization 25 (1987) 583–595.
B. Martinet, “Regularisation d'inéquations variationelles par approximations successives,”Revue Française d'Automatique et Informatique Recherche Opérationalle 4 (1970) 154–159.
V.H. Nguyen and J.J. Strodiot, “On the convergence rate of a penalty function method of exponential type,”Journal of Optimization Theory and Applications 27 (1979) 495–508.
B.T. Poljak and N.V. Tretjakov, “An iterative method for linear programming and its economic interpretation,”Matecon 10 (1974) 34–46.
S.M. Robinson, “Bounds for errors in the solution set of a perturbed linear program,”Linear Algebra and its Applications 6 (1973) 69–81.
R.T. Rockafellar,Convex Analysis (Princeton University Press, Princeton, NJ, 1970).
R.T. Rockafellar, “New applications of duality in convex programming,” in:Proceedings of the Conference on Probability (Brasov, 1971) pp. 73–81.
R.T. Rockafellar, “A dual approach to solving nonlinear programming problems by unconstrained minimization,”Mathematical Programming 5 (1973) 354–373.
R.T. Rockafellar, “Monotone operators and the proximal point algorithm,”SIAM Journal on Control and Optimization 14 (1976) 877–898.
Author information
Authors and Affiliations
Additional information
Research supported by the National Science Foundation under Grant DDM-8903385, and the Army Research Office under Grant DAAL03-86-K-0171.
Rights and permissions
About this article
Cite this article
Tseng, P., Bertsekas, D.P. On the convergence of the exponential multiplier method for convex programming. Mathematical Programming 60, 1–19 (1993). https://doi.org/10.1007/BF01580598
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01580598