Seong-Hee Kim
Abstract
We develop fully sequential procedures for comparison with a standard. The goal is to find systems whose expected performance measures are larger or smaller than a single system referred to as a standard and, if there is any, to find the one with the largest or smallest performance. The general formulation of comparison with a standard gives the standard a special status and tries to protect it when its performance is better than or even equal to performance measures of all the other alternatives. Therefore, the problem cannot be formulated as the selection of the best and a specialized procedure is required. Our procedures allow for unequal variances across systems, the use of common random numbers, and known or unknown expected performance of the standard. Experimental results are provided to compare the efficiency of the procedure with other existing procedures.
- Bechhofer, R. and Turnbull, B. 1978. Two (k+1)-decision selection procedures for comparing k normal means with a specified standard. J. Amer. Statis. Assoc. 73, 385--392.Google Scholar
- Boesel, J., Nelson, B., and Kim, S.-H. 2003. Using ranking and selection to clean up after a simulation search. Oper. Res. 51, 814--825. Google Scholar
- Chen, H.-C., Chen, C.-H., Dai, L., and Yücesan, E. 1997. New development of optimal computing budget allocation for discrete event simulation. In Proceedings of the 1997 Winter Simulation Conference, S. Andradóttir, K. Healy, D. Withers, and B. Nelson, Eds. IEEE, Piscataway, New Jersey, 334--341. Google Scholar
- Chen, H.-C., Chen, C.-H., and Yücesan, E. 2000. Computing efforts allocation for ordinal optimization and discrete event simulation. IEEE Trans. Auto. Cont. 45, 960--964.Google Scholar
- Chick, S. 1997. Selecting the best system: A decision-theoretic approach. In Proceedings of the 1997 Winter Simulation Conference, S. Andradóttir, K. Healy, D. Withers, and B. Nelson, Eds. IEEE, Piscataway, New Jersey, 326--333. Google Scholar
- Chick, S. and Inoue, K. 2001a. New two-stage and sequential procedures for selecting the best simulated system. Oper. Res. 49, 1609--1624. Google Scholar
- Chick, S. and Inoue, K. 2001b. New procedures for identifying the best simulated system using common random numbers. Manage. Sci. 47, 1133--1149. Google Scholar
- Fabian, V. 1974. Note on Anderson's sequential procedures with triangular boundary. Annals Statis. 2, 170--176.Google Scholar
- Goldsman, D. and Nelson, B. 1998. Comparing systems via simulation. In Handbook of simulation: Principles, Methodology, Advances, Applications, and Practice, J. Banks, Ed. John Wiley & Sons, New York, 273--306.Google Scholar
- Hartmann, M. 1991. An improvement on Paulson's procedure for selecting the population with the largest mean from k normal populations with a common unknown variance. Sequential Analysis 10, 1--16.Google Scholar
- Hong, L. and Nelson, B. 2005. The tradeoff between sampling and switching: New sequential procedures for indifference-zone selection. IIE Trans., to appear.Google Scholar
- Kim, S.-H. and Nelson, B. 2001. A fully sequential procedure for indifference-zone selection in simulation. ACM Trans. Model. Comput. Simul. 11, 251--273. Google Scholar
- Law, A. and Kelton, D. 2000. Simulation Modeling and Analysis, 3rd ed. McGraw-Hill, New York. Google Scholar
- Nelson, B. and Barnerjee, S. 2001. Selecting a good system: Procedures and inference. IIE Trans. 33, 149--166.Google Scholar
- Nelson, B. and Goldsman, D. 2001. Comparisons with a standard in simulation experiments. Manage. Sci. 47, 449--463. Google Scholar
- Nelson, B., Swann, J., Goldsman, D., and Song, W. 2001. Simple procedures for selecting the best simulated system when the number of alternatives is large. Oper. Res. 49, 950--963. Google Scholar
- Paulson, E. 1952. On the comparison of several experimental categories with a control. Annals Math. Statis. 23, 239--246.Google Scholar
Index Terms
- Comparison with a standard via fully sequential procedures
Recommendations
A fully sequential procedure for indifference-zone selection in simulation
We present procedures for selecting the best or near-best of a finite number of simulated systems when best is defined by maximum or minimum expected performance. The procedures are appropriate when it is possible to repeatedly obtain small, incremental ...
Comparisons with a Standard in Simulation Experiments
We consider the problem of comparing a finite number of stochastic systems with respect to a single system designated as the "standard" via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any ...
Comparison with a Standard via All-Pairwise Comparisons
We develop two sequential procedures to compare a finite number of designs with respect to a single standard. The goal is to identify a good design, and ensure that the standard is chosen when other alternatives are not better than the standard. We give ...
Reviews
James Speybroeck
The focus of this very well-written paper is the determination of a system whose expected performance measures are larger or smaller than a single system, with the goal of subsequently presenting the smallest and largest system. After a brief, but interesting introduction, consisting of the paper's organization and a history of such experimentation, Kim defines the problem and lists basic assumptions. In the next section, the author lays out a generic fully sequential procedure, followed by a customized procedure. Section 3 summarizes the results of an experiment to compare the two-stage procedure with a fully sequential procedure. Section 4 details the statistical and mathematical theorems used in the study. The conclusion briefly summarizes the findings. The author states that, with or without the use of the common random number (CRN) scheme, the estimated nominal probability of correct selection (PCS) of fully sequential procedures (FSP) was over .95. He indicates that overall experiments showed that that FSP is uniformly superior to NG (a generic procedure) under any configuration, in terms of the total number of observations. There is a short reference section. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments