Skip to main content
SpringerLink
Log in

Introduction to Extremum Seeking

  • Chapter
Stochastic Averaging and Stochastic Extremum Seeking

Part of the book series: Communications and Control Engineering ((CCE))

Abstract

The motivation behind extremum seeking methodology is discussed and the advances in the field of extremum seeking of the last 15 years are reviewed. Then a basic introduction to stochastic extremum seeking is presented, including how it relates to standard deterministic extremum seeking with periodic perturbations and what ideas are behind the study of stability of the resulting stochastic nonlinear system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adetola V, Guay M (2006) Adaptive output feedback extremum seeking receding horizon control of linear systems. J Process Control 16:521–533

    Article  Google Scholar 

  2. Adetola V, Guay M (2007) Guaranteed parameter convergence for extremum-seeking control of nonlinear systems. Automatica 43:105–110

    Article  MathSciNet  Google Scholar 

  3. Ariyur KB, Krstic M (2003) Real-time optimization by extremum seeking control. Wiley, Hoboken

    Book  MATH  Google Scholar 

  4. Ariyur K, Krstic M (2004) Slope seeking: a generalization of extremum seeking. Int J Adapt Control Signal Process 18:1–22

    Article  MATH  Google Scholar 

  5. Aumann RJ (1964) Markets with a continuum of traders. Econometrica 32(1):39–50

    Article  MathSciNet  MATH  Google Scholar 

  6. Banaszuk A, Ariyur KB, Krstic M, Jacobson CA (2004) An adaptive algorithm for control of combustion instability. Automatica 40:1965–1972

    Article  MathSciNet  MATH  Google Scholar 

  7. Becker A, Kumar PR, Wei CZ (1985) Adaptive control with the stochastic approximation algorithm: geometry and convergence. IEEE Trans Autom Control 30:330–338

    Article  MathSciNet  MATH  Google Scholar 

  8. Becker R, King R, Petz W, Nitsche W (2006) Adaptive closed-loop separation control on a high-lift configuration using extremum seeking. AIAA paper 2006-3493

    Google Scholar 

  9. Becker R, King R, Petz W, Nitsche W (2007) Adaptive closed-loop separation control on a high-lift configuration using extremum seeking. AIAA J 45:1382–1392

    Article  Google Scholar 

  10. Binetti P, Ariyur KB, Krstic M, Bernelli F (2003) Formation flight optimization using extremum seeking feedback. AIAA J Guid Control Dyn 26:132–142

    Article  Google Scholar 

  11. Brunn A, Nitsche W, Henning L, King R Application of slope-seeking to a generic car model for active drag control. Preprint

    Google Scholar 

  12. Carnevale D, Astolfi A, Centioli C, Podda S, Vitale V, Zaccarian L (2009) A new extremum seeking technique and its application to maximize RF heating on FTU. Fusing Eng Des 84:554–558

    Article  Google Scholar 

  13. Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  14. Choi J-Y, Krstic M, Ariyur KB, Lee JS (2002) Extremum seeking control for discrete time systems. IEEE Trans Autom Control 47:318–323

    Article  MathSciNet  Google Scholar 

  15. Cochran J, Krstic M (2009) Nonholonomic source seeking with tuning of angular velocity. IEEE Trans Autom Control 54(4):717–731

    Article  MathSciNet  Google Scholar 

  16. Cochran J, Kanso E, Kelly SD, Xiong H, Krstic M (2009) Source seeking for two nonholonomic models of fish locomotion. IEEE Trans Robot 25:1166–1176

    Article  Google Scholar 

  17. Creaby J, Li Y, Seem JE (2009) Maximizing wind turbine energy capture using multivariable extremum seeking control. Wind Eng 33:361–387

    Article  Google Scholar 

  18. DeHaan D, Guay M (2005) Extremum-seeking control of state-constrained nonlinear systems. Automatica 41:1567–1574

    Article  MathSciNet  MATH  Google Scholar 

  19. Favache A, Guay M, Perrier M, Dochain D (2008) Extremum seeking control of retention for a microparticulate system. Can J Chem Eng 86:815–827

    Article  Google Scholar 

  20. Foster DP, Young HP (2003) Learning, hypothesis testing, and Nash equilibrium. Games Econ Behav 45:73–96

    Article  MathSciNet  MATH  Google Scholar 

  21. Foster DP, Young HP (2006) Regret testing: learning to play Nash equilibrium without knowing you have an opponent. Theor Econ 1:341–367

    Google Scholar 

  22. Frihauf P, Krstic M, Başar T (2011) Nash equilibrium seeking with infinitely-many players. In: Proceedings of 2011 American control conference, San Francisco, CA, USA, June 29–July 1, pp 3059–3064

    Google Scholar 

  23. Fudenberg D, Levine DK (1998) The theory of learning in games. MIT Press, Cambridge

    MATH  Google Scholar 

  24. Gelfand SB, Mitter SK (1991) Recursive stochastic algorithms for global optimization in ℝd. SIAM J Control Optim 29:999–1018

    Article  MathSciNet  MATH  Google Scholar 

  25. Gelfand SB, Mitter SK (1993) Metropolis-type annealing algorithms for global optimization in ℝd. SIAM J Control Optim 31:111–131

    Article  MathSciNet  MATH  Google Scholar 

  26. Green EJ (1984) Continuum and finite-player noncooperative models of competition. Econometrica 52(4):975–993

    Article  MathSciNet  MATH  Google Scholar 

  27. Guay M, Perrier M, Dochain D (2005) Adaptive extremum seeking control of nonisothermal continuous stirred reactors. Chem Eng Sci 60:3671–3681

    Article  Google Scholar 

  28. Guay M, Dochain D, Perrier M, Hudon N (2007) Flatness-based extremum-seeking control over periodic orbits. IEEE Trans Autom Control 52:2005–2012

    Article  MathSciNet  Google Scholar 

  29. Henning L, Becker R, Feuerbach G, Muminovic R, Brun A, Nitsche W, King R (2008) Extensions of adaptive slope-seeking for active flow control. Proc Inst Mech Eng, Part I, J Syst Control Eng 222:309–322

    Article  Google Scholar 

  30. Jafari A, Greenwald A, Gondek D, Ercal G (2001) On no-regret learning, fictitious play, and Nash equilibrium. In: Proceedings of the 18th international conference on machine learning

    Google Scholar 

  31. Killingsworth NJ, Krstic M (2006) PID tuning using extremum seeking. IEEE Control Syst Mag 26:70–79

    Article  MathSciNet  Google Scholar 

  32. Killingsworth NJ, Krstic M, Flowers DL, Espinoza-Loza F, Ross T, Aceves SM (2009) HCCI engine combustion timing control: optimizing gains and fuel consumption via extremum seeking. IEEE Trans Control Syst Technol 17:1350–1361

    Article  Google Scholar 

  33. Kim K, Kasnakoglu C, Serrani A, Samimy M (2009) Extremum-seeking control of subsonic cavity flow. AIAA J 47:195–205

    Article  Google Scholar 

  34. King R, Becker R, Feuerbach G, Henning L, Petz R, Nitsche W, Lemke O, Neise W (2006) Adaptive flow control using slope seeking. In: Proceedings of the 14th IEEE Mediterranean conference on control and automation, June 28–30, pp 1–6

    Google Scholar 

  35. Krieger JP, Krstic M (2011) Extremum seeking based on atmospheric turbulence for aircraft endurance. AIAA J Guid Control Dyn 34:1876–1885

    Article  Google Scholar 

  36. Krstic M (2000) Performance improvement and limitations in extremum seeking control. Syst Control Lett 39:313–326

    Article  MathSciNet  MATH  Google Scholar 

  37. Krstic M, Wang HH (2000) Stability of extremum seeking feedback for general nonlinear dynamic systems. Automatica 36:595–601

    Article  MathSciNet  MATH  Google Scholar 

  38. Kumar PR, Varaiya P (1986) Stochastic systems: estimation, identification and adaptive control. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  39. Lei P, Li Y, Chen Q, Seem JE (2010) Extremum seeking control based integration of MPPT and degradation detection for photovoltaic arrays. In: Proceedings of 2010 American control conference, Baltimore, MD, USA, June 30–July 2, pp 3536–3541

    Google Scholar 

  40. Li P, Li Y, Seem JE (2009) Extremum seeking control for efficient and reliable operation of air-side economizers. In: Proceedings of 2009 American control conference, St. Louis, MO, USA, June 10–12, pp 20–25

    Chapter  Google Scholar 

  41. Li S, Başar T (1987) Distributed algorithms for the computation of noncooperative equilibria. Automatica 23:523–533

    Article  MATH  Google Scholar 

  42. Li Y, Rotea MA, Chiu GT-C, Mongeau LG, Paek I-S (2005) Extremum seeking control of a tunable thermoacoustic cooler. IEEE Trans Control Syst Technol 13:527–536

    Article  Google Scholar 

  43. Ljung L (1978) Strong convergence of a stochastic approximation algorithm. Ann Stat 6:680–696

    Article  MathSciNet  MATH  Google Scholar 

  44. Ljung L (2001) Recursive least-squares and accelerated convergence in stochastic approximation schemes. Int J Adapt Control Signal Process 15:169–178

    Article  MATH  Google Scholar 

  45. Ljung L, Pflug G, Walk H (1992) Stochastic approximation and optimization of random systems. Birkhäuser, Basel

    Book  MATH  Google Scholar 

  46. Luo L, Schuster E (2009) Mixing enhancement in 2D magnetohydrodynamic channel flow by extremum seeking boundary control. In: Proceedings of the 2009 American control conference, St. Louis, MO, USA, June 10–12, pp 1530–1535

    Chapter  Google Scholar 

  47. Manzie C, Krstic M (2009) Extremum seeking with stochastic perturbations. IEEE Trans Autom Control 54:580–585

    Article  MathSciNet  Google Scholar 

  48. Moase WH, Manzie C, Brear MJ (2009) Newton-like extremum-seeking part I: theory. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 3839–3844

    Google Scholar 

  49. Moase WH, Manzie C, Brear MJ (2009) Newton-like extremum-seeking part II: simulation and experiments. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 3845–3850

    Google Scholar 

  50. Moeck JP, Bothien MR, Paschereit CO, Gelbert G, King R Two-parameter extremum seeking for control of thermoacoustic instabilities and characterization of linear growth. AIAA paper 2007-1416

    Google Scholar 

  51. Murugappan S, Gutmark E, Acharya S, Krstic M (2000) Extremum seeking adaptive controller of swirl-stabilized spray combustion. Proc Combust Inst 28:731–737

    Article  Google Scholar 

  52. Nešić D, Tan Y, Moase WH, Manzie C (2010) A unifying approach to extremum seeking: adaptive schemes based on estimation of derivatives. In: Proceedings of the 49th IEEE conference on decision and control, Atlanta, GA, USA, December 15–17, pp 4625–4630

    Google Scholar 

  53. Ou Y, Xu C, Schuster E, Luce TC, Ferron JR, Walker ML, Humphreys DA (2008) Design and simulation of extremum-seeking open-loop optimal control of current profile in the DIII-D tokamak. Plasma Phys Control Fusion 50:115001

    Article  Google Scholar 

  54. Peterson K, Stefanopoulou A (2004) Extremum seeking control for soft landing of an electromechanical valve actuator. Automatica 29:1063–1069

    Article  MathSciNet  Google Scholar 

  55. Ren B, Frihauf P, Krstic M, Rafac RJ (2012) Laser pulse shaping via extremum seeking. Control Eng Pract 20:678–683

    Article  Google Scholar 

  56. Rotea MA (2000) Analysis of multivariable extremum seeking algorithms. In: Proceedings of the 2000 American control conference, Chicago, IL, USA, June 28–30, pp 433–437

    Google Scholar 

  57. Schuster E, Torres N, Xu C (2006) Extremum seeking adaptive control of beam envelope in particle accelerators. In: Proceedings of the 2006 IEEE conference on control applications, Munich, Germany, October 4–6, pp 1837–1842

    Chapter  Google Scholar 

  58. Schuster E, Xu C, Torres N, Morinaga E, Allen CK, Krstic M (2007) Beam matching adaptive control via extremum seeking. Nucl Instrum Methods Phys Res, Sect A, Accel Spectrom Detect Assoc Equip 581:799–815

    Article  Google Scholar 

  59. Shamma JS, Arslan G (2005) Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria. IEEE Trans Autom Control 53(3):312–327

    Article  MathSciNet  Google Scholar 

  60. Sharma R, Gopal M (2010) Synergizing reinforcement learning and game theory—a new direction for control. Appl Soft Comput 10(3):675–688

    Article  Google Scholar 

  61. Shitovitz B (1973) Oligopoly in markets with a continuum of traders. Econometrica 41(3):467–501

    Article  MathSciNet  MATH  Google Scholar 

  62. Stanković MS, Stipanović DM (2009) Stochastic extremum seeking with applications to mobile sensor networks. In: Proceedings of the 2009 American control conference, St. Louis, MA, USA, June 10–12, pp 5622–5627

    Chapter  Google Scholar 

  63. Stanković MS, Stipanović DM (2009) Discrete time extremum seeking by autonomous vehicles in a stochastic environment. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 4541–4546

    Google Scholar 

  64. Stanković MS, Stipanović DM (2010) Extremum seeking under stochastic noise and applications to mobile sensors. Automatica 46:1243–1251

    Article  MATH  Google Scholar 

  65. Stanković MS, Johansson KH, Stipanović DM (2010) Distributed seeking of Nash equilibrium in mobile sensor networks. In: Proceedings of IEEE conference on decision and control, Atlanta, GA, USA, December 15–17, pp 5598–5603

    Google Scholar 

  66. Tan Y, Nešić D, Mareels IMY (2006) On non-local stability properties of extremum seeking controllers. Automatica 42:889–903

    Article  MATH  Google Scholar 

  67. Wang H-H, Krstic M (2000) Extremum seeking for limit cycle minimization. IEEE Trans Autom Control 45:2432–2437

    Article  MathSciNet  MATH  Google Scholar 

  68. Wang H-H, Krstic M, Bastin G (1999) Optimizing bioreactors by extremum seeking. Int J Adapt Control Signal Process 13:651–669

    Article  MATH  Google Scholar 

  69. Wang H-H, Yeung S, Krstic M (2000) Experimental application of extremum seeking on an axial-flow compressor. IEEE Trans Control Syst Technol 8:300–309

    Article  Google Scholar 

  70. Wehner W, Schuster E (2009) Stabilization of neoclassical tearing modes in tokamak fusion plasmas via extremum seeking. In: Proceedings of the 3rd IEEE multi-conference on systems and control (MSC 2009), Saint Petersburg, Russia, July 8–10

    Google Scholar 

  71. Wiederhold O, Neuhaus L, King R, Niese W, Enghardt L, Noack BR, Swoboda M (2009) Extensions of extremum-seeking control to improve the aerodynamic performance of axial turbomachines. In: Proceedings of the 39th AIAA fluid dynamics conference, San Antonio, TX, USA

    Google Scholar 

  72. Zhang C, Arnold D, Ghods N, Siranosian A, Krstic M (2007) Source seeking with nonholonomic unicycle without position measurement and with tuning of forward velocity. Syst Control Lett 56:245–252

    Article  MathSciNet  MATH  Google Scholar 

  73. Zhang XT, Dawson DM, Dixon WE, Xian B (2006) Extremum-seeking nonlinear controllers for a human exercise machine. IEEE/ASME Trans Mechatron 11:233–240

    Article  Google Scholar 

  74. Zhu M, Martinez S (2010) Distributed coverage games for mobile visual sensor networks. SIAM J Control Optim (submitted). Available at http://arxiv.org/abs/1002.0367

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Southeast University, Nanjing, People’s Republic of China

    Shu-Jun Liu

  2. Department Mechanical & Aerospace Engineering, University of California, San Diego, La Jolla, California, USA

    Miroslav Krstic

Authors
  1. Shu-Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miroslav Krstic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miroslav Krstic .

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag London

About this chapter

Cite this chapter

Liu, SJ., Krstic, M. (2012). Introduction to Extremum Seeking. In: Stochastic Averaging and Stochastic Extremum Seeking. Communications and Control Engineering. Springer, London. https://doi.org/10.1007/978-1-4471-4087-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4087-0_2

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4086-3

  • Online ISBN: 978-1-4471-4087-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation