Skip to main content
SpringerLink
Log in

On averaging and representation properties of the BFGS and related secant updates

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

In this paper we present several representation theorems and averaging theorems for members of the difference class of secant updates introduced by Brodlie et al. (J Inst Math Appl 11:73–82, 1973). Major contributions are that the integral form of the mean-value theorem leads to a proof that the BFGS update is pointwise the infinite average of all the updates on the one-dimension manifold in the Dennis class that connects the DFP secant update to the Greenstadt update, and that it can be expressed as the pointwise average of these latter two updates. Analogous results hold for all secant updates that belong to the difference class. These results contribute new understanding of the structural properties of the highly popular BFGS secant update and other updates from the difference class.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brodlie, K., Gourlay, A., Greenstadt, J.: Rank-one and rank-two corrections to positive definite matrices expressed in product form. J. Inst. Math. Appl. 11, 73–82 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  2. Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19, 577–593 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  3. Broyden, C.G.: Quasi-Newton methods and their application to function minimisation. Math. Comput. 21, 368–381 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  4. Broyden, C.G.: The convergence of a class of double-rank minimization algorithms 1. General considerations. IMA J. Appl. Math. 6(1), 76–90 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  5. Broyden, C.G.: The convergence of a class of double-rank minimization algorithms 2. The new algorithm. IMA J. Appl. Math. 6(3), 222–231 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  6. Broyden, C.G., Dennis J.E., Jr., Moré, J.: On the local and superlinear convergence of quasi-Newton methods. IMA J. Appl. Math. 12, 223–246 (1973)

  7. Byrd, R.H., Tapia, R.A., Zhang, Y.: An SQP augmented Lagrangian BFGS algorithm for constrained optimization. SIAM J. Optim. 2, 210–241 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  8. Contreras, M., Tapia, R.: Sizing the BFGS and DFP updates. J. Optim. Theory Appl. 78, 93–108 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  9. Davidon, W.C.: Variable metric method for minimization. Argonne National Laboratory/University of Chicago, Technical Paper ANL-5990 (1959)

  10. Davidon, W.C.: Optimally conditioned optimization algorithms without line searches. Math. Program. 9, 1–30 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dennis, J.E., Jr.: On some methods based on Broyden’s secant approximation. In: Numerical Methods for Non-Linear Optimization, Conference, Dundee, pp. 19–34 (1972)

  12. Dennis J.E., Jr., Moré, J.: A characterization of superlinear convergence and its application to quasi-Newton methods. AMS Math. Comput. 28(126), 549–560 (1974)

  13. Dennis J.E., Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, New York (1983)

  14. Fletcher, R.: A new approach to variable metric algorithms. Comput. J. 13(3), 317–322 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  15. Fletcher, R., Powell, M.: A rapidly convergent descent method for minimization. Comput. J. 6, 163–168 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gay, D.M.: Representing symmetric rank-2 updates. In: National Bureau of Economic Research, Working Paper 124 (1976)

  17. Goldfarb, D.: Family of variable-metric methods derived by variational means. AMS Math. Comput. 24(109), 23–26 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  18. Greenstadt, J.: Variations on variable-metric methods. AMS Math. Comput. 24, 1–22 (1970)

    MathSciNet  MATH  Google Scholar 

  19. Papakonstantinou, J., Tapia, R.: On the generation of classes of symmetric rank-2 secant updates and the maximality of the Davidon class. Optim. Methods Softw. 27(4–5), 919–929 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–65. Academic Press, New York (1970)

    Google Scholar 

  21. Schnabel, R.B.: Analyzing and improving quasi-Newton methods for unconstrained optimization. Cornell University, Ph.D. thesis (1977)

  22. Shanno, D.: Conditioning of quasi-Newton methods for function minimization. AMS Math. Comput. 24, 647–656 (1970)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The author, quite belatedly, thanks David Gay and Bobby Schnabel for copies of their unpublished papers that introduced this so-called area of representation and averaging theory for secant updates and motivated the present work. He thanks two referees, the associate editor, and the editor for generous efforts leading to recommendations that much improved the presentation. He thanks one of the referees for strong encouragement to construct Theorem 6. It was not present in the original version of the paper.

Author information

Authors and Affiliations

  1. Department of Computational and Applied Mathematics, Rice University, Houston, TX, USA

    Richard Tapia

Authors
  1. Richard Tapia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Tapia.

Additional information

This work was supported in part by funds associated with the Maxfield-Oshman Chair in Engineering at Rice University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tapia, R. On averaging and representation properties of the BFGS and related secant updates. Math. Program. 153, 363–380 (2015). https://doi.org/10.1007/s10107-014-0807-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-014-0807-8

Keywords

Mathematics Subject Classification

Search

Navigation