Skip to main content
SpringerLink
Log in

Parametric Optimization: Stochastic Gradients and Adaptive Search

  • Chapter
  • First Online:
Simulation-Based Optimization

Part of the book series: Operations Research/Computer Science Interfaces Series ((ORCS,volume 55))

Abstract

This chapter focusses on simulation-based techniques for solving stochastic problems of parametric optimization, also popularly called static optimization problems. Such problems have been defined in Chap. 3.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bibliography

  1. M.H. Alrefaei, S. Andradóttir, A simulated annealing algorithm with constant temperature for discrete stochastic optimization. Manag. Sci. 45(5), 748–764 (1999)

    Article  Google Scholar 

  2. M.H. Alrefaei, S. Andradóttir, A modification of the stochastic ruler method for discrete stochastic optimization. Eur. J. Oper. Res. 133, 160–182 (2001)

    Article  Google Scholar 

  3. S. Andradóttir, Simulation optimization, in Handbook of Simulation, ed. by J. Banks, chapter 9 (Wiley, New York, 1998)

    Google Scholar 

  4. H. Arsham, A. Feurvergerd, D.L. McLeish, J. Kreimer, R.Y. Rubinstein, Sensitivity analysis and the what-if problem in simulation analysis. Math. Comput. Model. 12(2), 193–219 (1989)

    Article  Google Scholar 

  5. R.R. Barton, J.S. Ivey, Nelder-Mead simplex modifications for simulation optimization. Manag. Sci. 42(7), 954–973 (1996)

    Article  Google Scholar 

  6. R.E. Bechhofer, T.J. Santner, D.J. Goldsman, Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons (Wiley, New York, 1995)

    Google Scholar 

  7. D.P. Bertsekas, Non-linear Programming (Athena Scientific, Belmont, 1995)

    Google Scholar 

  8. S. Bhatnagar, Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Trans. Model. Comput. Simul. 15(1), 74–107 (2005)

    Article  Google Scholar 

  9. S. Bhatnagar, V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization. Probab. Eng. Inf. Sci. 12, 519–531 (1998)

    Article  Google Scholar 

  10. S. Bhatnagar, V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization. Simulation 79(10), 569–580 (2003)

    Article  Google Scholar 

  11. S. Bhatnagar, H.J. Kowshik, A discrete parameter stochastic approximation algorithm for simulation optimization. Simulation 81(11), 757–772 (2005)

    Article  Google Scholar 

  12. I. Bohachevsky, M. Johnson, M. Stein, Generalized simulated annealing for function approximation. Technometrics 28, 209–217 (1986)

    Article  Google Scholar 

  13. S. Brooks, B. Morgan, Optimization using simulated annealing. The Statistician 44, 241–257 (1995)

    Article  Google Scholar 

  14. Y. Carson, A. Maria, Simulation optimization: methods and applications, in Proceedings of the 1997 Winter Simulation Conference, Atlanta, 1997, pp. 118–126

    Google Scholar 

  15. A. Cauchy, Méthode génserale pour la résolution des systéms d’équations simultanées. C. R. Sci. Paris 25, 536–538 (1847)

    Google Scholar 

  16. H.S. Chang, M.C. Fu, J. Hu, S.I. Marcus, Simulation-Based Algorithms for Markov Decision Processes (Springer, London, 2007)

    Book  Google Scholar 

  17. G. Deng, M.C. Ferris, Variable-number sample path optimization. Math. Program. Ser. B 117, 81–109 (2009)

    Article  Google Scholar 

  18. M. Dorigo, T. Stützle, Ant Colony Optimization (MIT, Cambridge, MA, 2004)

    Google Scholar 

  19. T. Feo, M. Resende, Greedy randomized adaptive search procedures. J. Global Optim. 6, 108–133 (1995)

    Article  Google Scholar 

  20. M.J. Fielding, Optimisation by simulated annealing, PhD thesis, Department of Mathematics, The University of Melbourne, 1999

    Google Scholar 

  21. B.L. Fox, G.W. Heine, Probabilistic search with overrides. Ann. Appl. Probab. 5, 1087–1094 (1995)

    Article  Google Scholar 

  22. M.C. Fu, Optimization via simulation: theory vs. practice. INFORMS J. Comput. 14(3), 192–215 (2002)

    Google Scholar 

  23. M.C. Fu, Optimization via simulation: a review. Ann. Oper. Res. 53, 199–247 (1994)

    Article  Google Scholar 

  24. M.C. Fu, J.Q. Hu, Conditional Monte Carlo-Gradient Estimation and Optimization Applications (Kluwer Academic, Boston, 1997)

    Book  Google Scholar 

  25. S.B. Gelfand, S.K. Mitter, Simulated annealing with noisy or imprecise energy measurements. J. Optim. Theory Appl. 62(1), 49–62 (1989)

    Article  Google Scholar 

  26. M. Gendreau, J.-Y. Potvin, Handbook of Metaheuristics (Springer, New York, 2010)

    Book  Google Scholar 

  27. L. Gerencser, S.D. Hill, Z. Vago, Optimization over discrete sets by SPSA, in Proceedings of the Winter Simulation Conference, Phoenix, 1999, pp. 466–470

    Google Scholar 

  28. S. German, D. German, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 6, 721–741 (1984)

    Article  Google Scholar 

  29. F. Glover, Tabu search: a tutorial. Interfaces 20(4), 74–94 (1990)

    Article  Google Scholar 

  30. F. Glover, J.P. Kelly, M. Laguna, New advances and applications of combining simulation and optimization, in Proceedings of the 1996 Winter Simulation Conference, Coronado, 1996, pp. 144–152

    Google Scholar 

  31. F. Glover, M. Laguna, Tabu Search (Kluwer Academic, Norwell, 1998)

    Google Scholar 

  32. F. Glover, M. Laguna, R. Marti, Fundamentals of scatter search and path relinking. Control Cybern. 29(3), 653–684 (2000)

    Google Scholar 

  33. D. Goldsman, B.L. Nelson, Comparing systems via simulation, in Handbook of Simulation, ed. by J. Banks, chapter 8 (Wiley, New York, 1998)

    Google Scholar 

  34. W.B. Gong, Y.C. Ho, W. Zhai, Stochastic comparison algorithm for discrete optimization with estimation, in Proceedings of the 31st Conference on Decision Control, Tucson, 1992, pp. 795–800

    Google Scholar 

  35. A. Gosavi, On step-sizes, stochastic paths, and survival probabilities in reinforcement learning, in Proceedings of the 2008 Winter Simulation Conference, Miami (IEEE, 2008)

    Google Scholar 

  36. A. Gosavi, Codes for neural networks, DP, and RL in the C language for this book (2014), http://web.mst.edu/~gosavia/bookcodes.html

  37. B. Hajek, Cooling schedules for optimal annealing. Math. Oper. Res. 13, 311–329 (1988)

    Article  Google Scholar 

  38. S.S. Heragu, Facilities Design, 3rd edn. (CRC, Boca Raton, 2008)

    Google Scholar 

  39. Y.C. Ho, X.R. Cao, Perturbation Analysis of Discrete Event Dynamic Systems (Springer, Boston, 1991)

    Book  Google Scholar 

  40. Y.C. Ho, R. Sreenivas, P. Vakili, Ordinal optimization of discrete event dynamic systems. J. DEDS 2(2), 61–88 (1992)

    Google Scholar 

  41. Y. Hochberg, A.C. Tamhane, Multiple Comparison Procedures (Wiley, New York, 1987)

    Book  Google Scholar 

  42. J.H. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, Ann Arbor, 1975)

    Google Scholar 

  43. L.J. Hong, B.L. Nelson, Discrete optimization via simulation using COMPASS. Oper. Res. 54(1), 115–129 (2006)

    Article  Google Scholar 

  44. R. Hooke, T.A. Jeeves, Direct search of numerical and statistical problems. ACM 8, 212–229 (1966)

    Article  Google Scholar 

  45. J. Hu, M.C. Fu, S.I. Marcus, A model reference adaptive search method for global optimization. Oper. Res. 55, 549–568 (2007)

    Article  Google Scholar 

  46. S.H. Jacobson, L.W. Schruben, A harmonic analysis approach to simulation sensitivity analysis. IIE Trans. 31(3), 231–243 (1999)

    Google Scholar 

  47. J.-S.R. Jang, C.-T. Sun, E. Mizutani, Neuro-Fuzzy and Soft Computing (Prentice Hall, Upper Saddle River, 1997)

    Google Scholar 

  48. J. Kennedy, R.C. Eberhart, Y. Shi, Swarm Intelligence (Morgan Kaufmann, San Francisco, 2001)

    Google Scholar 

  49. J. Kiefer, J. Wolfowitz, Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23, 462–466 (1952)

    Article  Google Scholar 

  50. S.-H. Kim, B.L. Nelson, A fully sequential procedure for indifference-zone selection in simulation. ACM Trans. Model. Comput. Simul. 11, 251–273 (2001)

    Article  Google Scholar 

  51. S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing. Science 220, 671–680 (1983)

    Article  Google Scholar 

  52. J.P.C. Kleijnen, Sensitivity analysis and optimization in simulation: design of experiments and case studies, in Proceedings of the 1995 Winter Simulation Conference, Arlington, 1995, pp. 133–140

    Google Scholar 

  53. J.P.C. Kleijnen, Design and Analysis of Simulation Experiments (Springer, New York, 2007)

    Google Scholar 

  54. B. Kristinsdottir, Z.B. Zabinsky, G. Wood, Discrete backtracking adaptive search for global optimization, in Stochastic and Global Optimization, ed. by G. Dzemyda, V. Saltenis, A. Zilinskas (Kluwer Academic, Dordrecht/London, 2002), pp. 147–174

    Chapter  Google Scholar 

  55. J.C. Lagarias, J.A. Reeds, M.H. Wright, P.E. Wright, Convergence properties of the Nelder–Mead simplex method in low dimensions. SIAM J. Optim. 9(1), 112–147 (1998)

    Article  Google Scholar 

  56. A.M. Law, W.D. Kelton, Simulation Modeling and Analysis (McGraw Hill, New York, 1999)

    Google Scholar 

  57. M. Lundy, A. Mees, Convergence of the annealing algorithm. Math. Program. 34, 111–124 (1986)

    Article  Google Scholar 

  58. N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equation of state calculations by fast computing machines. J. Chem. Phys 21, 1087–1092 (1953)

    Article  Google Scholar 

  59. O. Molvalioglu, Z.B. Zabinsky, W. Kohn, The interacting particle algorithm with dynamic heating and cooling. J. Global Optim. 43, 329–356 (2019)

    Article  Google Scholar 

  60. J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  Google Scholar 

  61. R. Pasupathy, B.W. Schmeiser, Retrospective-approximation algorithms for multidimensional stochastic root-finding problems. ACM TOMACS 19(2), 5:1–5:36 (2009)

    Google Scholar 

  62. G.C. Pflug, Optimization of Stochastic Models: The Interface Between Simulation and Optimization (Kluwer Academic, Boston, 1996)

    Book  Google Scholar 

  63. D.T. Pham, D. Karaboga, Intelligent Optimisation Techniques: Genetic Algorithms, Tabu Search, Simulated Annealing, Neural Networks (Springer, New York, 1998)

    Google Scholar 

  64. E.L. Plambeck, B.R. Fu, S.M. Robinson, R. Suri, Sample path optimization of convex stochastic performance functions. Math. Program. 75, 137–176 (1996)

    Google Scholar 

  65. W.H. Press, S.A. Tuekolsky, W.T. Vetterling, B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing (Cambridge University Press, Cambridge, 1992)

    Google Scholar 

  66. A.A. Prudius, S. Andradóttir, Balanced explorative and exploitative search with estimation for simulation optimization. INFORMS J. Comput. 21, 193–208 (2009)

    Article  Google Scholar 

  67. Y. Rinott, On two-stage selection procedures and related probability-inequalities. Commun. Stat. Theory Methods A7, 799–811 (1978)

    Article  Google Scholar 

  68. H.E. Romeijn, R.L. Smith, Simulated annealing and adaptive search in Global optimization. Probab. Eng. Inf. Sci 8, 571–590 (1994)

    Article  Google Scholar 

  69. H.E. Romeijn, R.L. Smith, Simulated annealing for constrained global optimization. J. Global Optim. 5, 101–126 (1994)

    Article  Google Scholar 

  70. R.Y. Rubinstein, A. Shapiro, Sensitivity Analysis and Stochastic Optimization by the Score Function Method (Wiley, New York, 1983)

    Google Scholar 

  71. G. Rudolph, Convergence of canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96–101 (1994)

    Article  Google Scholar 

  72. S. Sarkar, S. Chavali, Modeling parameter space behavior of vision systems using Bayesian networks. Comput. Vis. Image Underst. 79, 185–223 (2000)

    Article  Google Scholar 

  73. L. Shi, S. Olafsson, Nested partitions method for Global optimization. Oper. Res. 48(3), 390–407 (2000)

    Article  Google Scholar 

  74. L. Shi, S. Olafsson, Nested partitions method for stochastic optimization. Methodol. Comput. Appl. Probab. 2(3), 271–291 (2000)

    Article  Google Scholar 

  75. L. Shi, S. Olafsson, Nested Partitions Method, Theory and Applications (Springer, New York, 2008)

    Google Scholar 

  76. R.L. Smith, Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 32, 1296–1308 (1984)

    Article  Google Scholar 

  77. J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332–341 (1992)

    Article  Google Scholar 

  78. J.C. Spall, Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control (Wiley, New York, 2003)

    Book  Google Scholar 

  79. H. Szu, R. Hartley, Fast simulated annealing. Phys. Lett. A 122, 157–162 (1987)

    Article  Google Scholar 

  80. T. Tezcan, A. Gosavi, Optimal buffer allocation in production lines using an automata search, in Proceedings of the 2001 Institute of Industrial Engineering Research Conference, Dallas, 2001

    Google Scholar 

  81. M.A.L. Thathachar, P.S. Sastry, Learning optimal discriminant functions through a cooperative game of automata. IEEE Trans. Syst. Man Cybern. 17, 73–85 (1987)

    Article  Google Scholar 

  82. P. van Laarhoven, E. Aarts, Simulated Annealing: Theory and Applications (Kluwer Academic, Dordrecht, 1987)

    Book  Google Scholar 

  83. I.-J. Wang, J.C. Spall, Stochastic optimization with inequality constraints using simultaneous perturbations and penalty functions, in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, 2003

    Google Scholar 

  84. R.R. Wilcox, A table for Rinott’s selection procedure. J. Qual. Technol. 16, 97–100 (1984)

    Google Scholar 

  85. D. Yan, H. Mukai, Stochastic discrete optimization. SIAM J. Control Optim. 30, 594–612 (1992)

    Article  Google Scholar 

  86. Z.B. Zabinsky, Stochastic Adaptive Search for Global Optimization (Springer, New York, 2003)

    Book  Google Scholar 

  87. Z. Zabinsky, Random search algorithms for simulation optimization, in Handbook of Simulation Optimization (forthcoming), ed. by M. Fu (Springer, New York, 2014)

    Google Scholar 

  88. Z.B. Zabinsky, D. Bulger, C. Khompatraporn, Stopping and restarting strategy for stochastic sequential search in global optimization. J. Global Optim. 46, 273–286 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering Management and Systems Engineering, Missouri University of Science and Technology, Rolla, MO, USA

    Abhijit Gosavi

Authors
  1. Abhijit Gosavi
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this chapter

Cite this chapter

Gosavi, A. (2015). Parametric Optimization: Stochastic Gradients and Adaptive Search. In: Simulation-Based Optimization. Operations Research/Computer Science Interfaces Series, vol 55. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7491-4_5

Download citation

Publish with us

Policies and ethics

Search

Navigation