Natalie M. Steiger
Skip Abstract Section
Abstract
We introduce ASAP3, a refinement of the batch means algorithms ASAP and ASAP2, that delivers point and confidence-interval estimators for the expected response of a steady-state simulation. ASAP3 is a sequential procedure designed to produce a confidence-interval estimator that satisfies user-specified requirements on absolute or relative precision as well as coverage probability. ASAP3 operates as follows: the batch size is progressively increased until the batch means pass the Shapiro-Wilk test for multivariate normality; and then ASAP3 fits a first-order autoregressive (AR(1)) time series model to the batch means. If necessary, the batch size is further increased until the autoregressive parameter in the AR(1) model does not significantly exceed 0.8. Next, ASAP3 computes the terms of an inverse Cornish-Fisher expansion for the classical batch means t-ratio based on the AR(1) parameter estimates; and finally ASAP3 delivers a correlation-adjusted confidence interval based on this expansion. Regarding not only conformance to the precision and coverage-probability requirements but also the mean and variance of the half-length of the delivered confidence interval, ASAP3 compared favorably to other batch means procedures (namely, ABATCH, ASAP, ASAP2, and LBATCH) in an extensive experimental performance evaluation.
- Alexopoulos, C. and Goldsman, D. 2004. To batch or not to batch? ACM Trans. Model. Comput. Simul. 14, 1 (Jan.), 76--114. Google Scholar
- Alexopoulos, C. and Seila, A. F. 1998. Output data analysis. In Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice}, J. Banks, Ed. John Wiley & Sons, New York, NY, 225--272.Google Scholar
- Amemiya, T. and Wu, R. Y. 1972. The effect of aggregation on prediction in the autoregressive model. J. Amer. Statist. Assoc. 67, 339 (Sept.), 628--632.Google Scholar
- Bickel, P. J. and Doksum, K. A. 1977. Mathematical Statistics: Basic Ideas and Selected Topics. Holden-Day, San Francisco, CA.Google Scholar
- Box, G. E. P. 1954. Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification. Ann. Math. Stat. 25, 290--302.Google Scholar
- Box, G. E. P., Jenkins, G. M., and Reinsel, G. C. 1994. Time Series Analysis: Forecasting and Control. 3rd Ed. Prentice Hall, Englewood Cliffs, NJ. Google Scholar
- Chow, Y. S. and Robbins, H. 1965. On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Ann. Math. Stat. 36, 457--462.Google Scholar
- Fishman, G. S. 1996. Monte Carlo: Concepts, Algorithms, and Applications. Springer-Verlag, New York, NY.Google Scholar
- Fishman, G. S. 1998. LABATCH.2 for analyzing sample path data {online}. Department of Operations Research, University of North Carolina, Chapel Hill, NC. Available at <http://www.unc.edu/depts/or/downloads/tech_reports/fishman/uncor97-04.ps>.Google Scholar
- Fishman, G. S. and Yarberry, L. S. 1997. An implementation of the batch means method. INFORMS J. Comput. 9, 3, 296--310.Google Scholar
- Forsythe, G. E., Malcolm, M. A., and Moler, C. B. 1977. Computer Methods for Mathematical Computations. Prentice-Hall, Englewood Cliffs, NJ. Google Scholar
- Fox, B. L., Goldsman, D., and Swain, J. J. 1991. Spaced batch means. Oper. Res. Lett. 10, 5 (July), 255--263.Google Scholar
- Fuller, W. A. 1996. Introduction to Statistical Time Series. 2nd Ed. John Wiley & Sons, New York, NY.Google Scholar
- Jenkins, G. M. 1954. An angular transformation of the serial correlation coefficient. Biometrika 41, 1/2, 261--265.Google Scholar
- Johnson, N. L., Kotz, S., and Balakrishnan, N. 1994. Continuous Univariate Distributions, Vol. 1. 2nd Ed. John Wiley & Sons, New York, NY.Google Scholar
- Kang, K. and Schmeiser, B. W. 1987. Properties of batch means from stationary ARMA time series. Oper. Res. Lett. 6, 1 (March), 19--24.Google Scholar
- Lada, E. K., Wilson, J. R., and Steiger, N. M. 2003. A wavelet-based spectral method for steady-state simulation analysis. In Proceedings of the 2003 Winter Simulation Conference, S. Chick, P. J. Sánchez, D. Ferrin, and D. J. Morrice, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 422--430. Available at <http://www.informs-cs.org/wsc03papers/052.pdf>. Google Scholar
- Lada, E. K., Wilson, J. R., Steiger, N. M., and Joines, J. A. 2004a. Performance evaluation of a wavelet-based spectral method for steady-state simulation analysis. In Proceedings of the 2004 Winter Simulation Conference, R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 694--702. Available at <www.informs-sim.org/wsc04papers/084.pdf>. Google Scholar
- Lada, E. K., Wilson, J. R., Steiger, N. M., and Joines, J. A. 2004b. Performance of a wavelet-based spectral procedure for steady-state simulation analysis. INFORMS J. Comput. to appear. Available at <ftp.ncsu.edu/pub/eos/pub/jwilson/lada04joc.pdf>. Google Scholar
- Malkovich, J. F. and Afifi, A. A. 1973. On tests for multivariate normality. J. Amer. Statist. Assoc. 68, 341 (March), 176--179.Google Scholar
- Nádas, A. 1969. An extension of a theorem of Chow and Robbins on sequential confidence intervals for the mean. The Ann. Math. Stat. 40, 2, 667--671.Google Scholar
- Royston, J. P. 1982a. An extension of Shapiro and Wilk's W test for normality to large samples. Appl. Stat. 31, 2, 115--124.Google Scholar
- Royston, J. P. 1982b. Algorithm AS 181. The W test for normality. Appl. Stat. 31, 176--180.Google Scholar
- Satterthwaite, F. E. 1941. Synthesis of variance. Psychometrika 6, 5, 309--316.Google Scholar
- Satterthwaite, F. E. 1946. An approximate distribution of estimates of variance components. Biometrics Bull. 2, 6, 110--114.Google Scholar
- Searle, S. R. 1982. Matrix Algebra Useful for Statistics. John Wiley & Sons, New York, NY.Google Scholar
- Steiger, N. M. 1999. Improved batching for confidence interval construction in steady state simulation. PhD thesis, Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Available at <http://www.lib.ncsu.edu/etd/public/etd-19231992992670/etd.pdf>.Google Scholar
- Steiger, N. M. and Wilson, J. R. 1999. Improved batching for confidence interval construction in steady-state simulation. In Proceedings of the 1999 Winter Simulation Conference, P. A. Farrington, H. B. Nembhard, D. T. Sturrock, and G. W. Evans, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 442--451. Available at <http://www.informs-cs.org/wsc99papers/061.PDF>. Google Scholar
- Steiger, N. M. and Wilson, J. R. 2000. Experimental performance evaluation of batch-means procedures for simulation output analysis. In Proceedings of the 2000 Winter Simulation Conference, R. R. Barton, J. A. Joines, K. Kang, and P. A. Fishwick, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 627--636. Available at <http://www.informs-cs.org/wsc00papers/084.PDF>. Google Scholar
- Steiger, N. M. and Wilson, J. R. 2001. Convergence properties of the batch means method for simulation output analysis. INFORMS J. Comput. 13, 4, 277--293. Google Scholar
- Steiger, N. M. and Wilson, J. R. 2002a. An improved batch means procedure for simulation output analysis. Manage. Sci. 48 12, 1569--1586. Google Scholar
- Steiger, N. M. and Wilson, J. R. 2002b. ASAP software and user's manual. Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Available at <ftp.ncsu.edu/pub/eos/pub/jwilson/installasap.exe>.Google Scholar
- Steiger, N. M., Lada, E. K., Wilson, J. R., Alexopoulos, C., Goldsman, D., and Zouaoui, F. 2002. ASAP2: An improved batch means procedure for simulation output analysis. In Proceedings of the 2002 Winter Simulation Conference, E. Yücesan, C.-H. Chen, J. L. Snowdon, and J. M. Charnes, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 336--344. Available at <http://www.informs-cs.org/wsc02papers/043.pdf>. Google Scholar
- Steiger, N. M., Lada, E. K., Wilson, J. R., Joines, J. A., Alexopoulos, C., and Goldsman, D. 2003. ASAP3 software and user's manual. Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Available at <ftp.ncsu.edu/pub/eos/pub/jwilson/installasap3.exe>.Google Scholar
- Steiger, N. M., Lada, E. K., Wilson, J. R., Joines, J. A., Alexopoulos, C., and Goldsman, D. 2004. Steady-state simulation analysis using ASAP3. In Proceedings of the 2004 Winter Simulation Conference, R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ. 672--680. Available at <www.informs-sim.org/wsc04papers/081.pdf>. Google Scholar
- Stuart, A. and Ord, J. K. 1994. Kendall's Advanced Theory of Statistics, Volume 1: Distribution Theory. 6th. Ed. Edward Arnold, London.Google Scholar
- Suárez-González, A., López-Ardao, J. C., López-García, C., Rodríguez-Pérez, M., Fernández-Veiga, M., and Sousa-Vieira, M. E. 2002. A batch means procedure for mean value estimation of processes exhibiting long range dependence. In Proceedings of the 2002 Winter Simulation Conference, E. Yücesan, C.-H. Chen, J. L. Snowdon, and J. M. Charnes, Eds. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 456--464. Available at <http://www.informs-cs.org/wsc02papers/057.pdf>. Google Scholar
- Tew, J. D. and Wilson, J. R. 1992. Validation of simulation analysis methods for the Schruben-Margolin correlation-induction strategy. Oper. Res. 40, 1, 87--103. Google Scholar
- Welch, B. L. 1956. On linear combinations of several variances. J. Amer. Stat. Assoc. 51 273, 132--148.Google Scholar
- Welch, P. D. 1983. The statistical analysis of simulation results. In Computer Performance Modeling Handbook, S. S. Lavenberg, Ed. Academic Press, New York NY, 268--329.Google Scholar
Index Terms
- ASAP3: a batch means procedure for steady-state simulation analysis
Recommendations
To batch or not to batch?
When designing steady-state computer simulation experiments, one may be faced with the choice of batching observations in one long run or replicating a number of smaller runs. Both methods are potentially useful in the course of undertaking simulation ...
A Batch Means Methodology for Estimation of a Nonlinear Function of a Steady-State Mean
We study the estimation of steady-state performance measures from an â d -valued stochastic process Y = { Y t : t â 0} representing the output of a simulation. In many applications, we may be interested in the estimation of a steady-state ...
Steady-state simulation analysis using ASAP3
WSC '04: Proceedings of the 36th conference on Winter simulationWe discuss ASAP3, a refinement of the batch means algorithms ASAP and ASAP2. ASAP3 is a sequential procedure designed to produce a confidence-interval estimator for the expected response of a steady-state simulation that satisfies user-specified ...
Comments