Abstract
This paper proves convergence of a sample-path based stochastic gradient-descent algorithm for optimizing expected-value performance measures in discrete event systems. The algorithm uses increasing precision at successive iterations, and it moves against the direction of a generalized gradient of the computed sample performance function. Two convergence results are established: one, for the case where the expected-value function is continuously differentiable; and the other, when that function is nondifferentiable but the sample performance functions are convex. The proofs are based on a version of the uniform law of large numbers which is provable for many discrete event systems where infinitesimal perturbation analysis is known to be strongly consistent.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ho, Y. C., andCao X. R.,Perturbation Analysis of Discrete Event Dynamic Systems, Kluwer Academic Publishers, Boston, Massachusetts, 1991.
Rubinstein, R. Y., andShapiro, A.,Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method, John Wiley and Sons, New York, New York, 1993.
Shapiro, A., andWardi, Y.,Nondifferentiability of the Steady-State Function in Discrete Event Dynamic Systems, IEEE Transactions on Automatic Control, Vol. 39, pp. 1707–1711, 1994.
Clarke, F. H.,Optimization and Nonsmooth Analysis, John Wiley and Sons, New York, New York, 1983.
Rockefellar, R. T.,Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.
Robinson, S. M.,Convergence of Subdifferentials under Strong Stochastic Convexity, Management Science (to appear).
Shapiro, A., andWardi, Y.,Convergence Analysis of Stochastic Algorithms, Mathematics of Operations Research (to appear).
Correa, R., andLemaréchal, C.,Convergence of Some Algorithms for Convex Minimization, Mathematical Programming, Vol. 62, pp. 261–275, 1993.
Demyanov, V. F., andVasilev, L. V.,Nondifferentiable Optimization, Optimization Software, Publications Division, New York, New York, 1985.
Fu, M. C.,Optimization via Simulation: A Review, Annals of Operations Research, Vol. 53, pp. 199–247, 1994.
Chong, E. K. P., andRamadge, P. J.,Optimization of Queues Using Infinitessimal Perturbation Analysis-Based Stochastic Algorithm with General Update Times, SIAM Journal on Control and Optimization, Vol 31, pp. 698–732, 1993.
L'ecuyer, P., andGlynn, P.,Stochastic Optimization by Simulation: Convergence Proofs for the GI/G1 Queue in Steady State, Management Science, Vol. 40, pp. 1562–1578, 1994.
Chong, E. K. P., andRamadge, P. J.,Stochastic Optimization of Regenerative Systems Using Infinitestimal Perturbation Analysis, IEEE Transactions on Automatic Control, Vol. 39, pp. 1400–1410, 1994.
Shantikumar, J. G., andYao, D. D.,Second-Order Stochastic Properties of Queuering Systems, Proceedings of the IEEE, Vol. 77, pp. 162–170, 1989.
Bartusek, J. D., andMarkowski, A. M.,On Stochastic Approximations Driven by Sample Averages: Convergence Results via the ODE Method, Manuscript, Institute for Systems Research, University of Maryland, 1993.
Dupuis, P., andSimha, R.,On Sampling Controlled Stochastic Approximation, IEEE Transactions on Automatic Control, Vol. 36, pp. 915–924, 1991.
Meheshwari, S., andMukai, H.,An Optimization Algorithm Driven by Probabilistic Simulation, Proceedings of the Conference on Decision and Control, Athens, Greece, pp. 1703–1705, 1986.
Yan, D., andMukai, H.,An Optimization Algorithm with Probabilistic Simulation, Journal of Optimization Theory and Applications, Vol. 79, pp. 345–371, 1993.
Wardi, Y.,Stochastic Algorithms with Armijo Stepsizes for Minimization of functions, Journal of Optimization Theory and Applications, Vol. 64, pp. 399–417, 1990.
Hiriart-Urruty, J. B., andLemaréchal, C.,Convex Analysis and Minimization Algorithms, Part 1, Springer Verlag, Berlin, Germany, 1993.
Hiriart-Urruty, J. B., andLemaréchal, C.,Convex Analysis and Minimization Algorithms, Part 2, Springer Verlag, Berlin, Germany, 1993.
Shapiro, A.,Asymptotic Properties of Statistical Estimators in Stochastic Programming, Annals of Statistics, Vol. 17, pp. 841–858, 1989.
Wardi, Y.,Interchangeability of Expectation and Differentiation of Waiting Times in GI/G1 Queues, Stochastic Processes and Their Applications, Vol. 45, pp. 141–154, 1993.
Glasserman, P.,Gradient Estimation via Perturbation Analysis, Kluwer Academic Publishers, Boston, Massachusetts, 1991.
Asmussen, S.,Applied Probability and Queues, John Wiley and Sons, New York, New York, 1987.
Author information
Authors and Affiliations
Additional information
Communicated by W. B. Gong
Rights and permissions
About this article
Cite this article
Shapiro, A., Wardi, Y. Convergence analysis of gradient descent stochastic algorithms. J Optim Theory Appl 91, 439–454 (1996). https://doi.org/10.1007/BF02190104
Issue Date:
DOI: https://doi.org/10.1007/BF02190104