Abstract
Sequential rules are explored in the context of null hypothesis significance testing. Several studies have demonstrated that the fixed-sample stopping rule, in which the sample size used by researchers is determined in advance, is less practical and less efficient than sequential stopping rules. It is proposed that a sequential stopping rule called CLAST (composite limited adaptive sequential test) is a superior variant of COAST (composite open adaptive sequential test), a sequential rule proposed by Frick (1998). Simulation studies are conducted to test the efficiency of the proposed rule in terms of sample size and power. Two statistical tests are used: the one-tailed t test of mean differences with two matched samples, and the chi-square independence test for twofold contingency tables. The results show that the CLAST rule is more efficient than the COAST rule and reflects more realistically the practice of experimental psychology researchers.
Article PDF
Similar content being viewed by others
References
Allison, D. B., Silverstein, J. M., &Gorman, B. S. (1996). Power, sample size estimation, and early stopping rules. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.),Design and analysis of singlecase research (pp. 335–371). Mahwah, NJ: Erlbaum.
Arnold, D. H., &Harvey, E. A. (1998). Data monitoring: A hypothesistesting approach for treatment-outcome research.Journal of Consulting & Clinical Psychology,66, 1030–1035.
Becker, R. A., Chambers, J. M., &Wilks, A. R. (1988).The new S language: A programming environment for data analysis and graphics. New York: Chapman & Hall.
Cohen, J. (1988).Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Fiske, D. W., &Jones, L. V. (1954). Sequential analysis in psychological research.Psychological Bulletin,51, 264–275.
Frick, R. W. (1998). A better stopping rule for conventional statistical tests.Behavior Research Methods, Instruments, & Computers,30, 690–697.
Ghosh, B. K., &Sen, P. K. (Eds.) (1991).Handbook of sequential analysis. New York: Dekker.
Hays, W. L. (1988).Statistics (4th ed.). Philadelphia: Holt, Rinehart & Winston.
Jennison, C., &Turnbull, B. W. (2000).Group sequential methods with applications to clinical trials. Boca Raton, FL: Chapman & Hall.
Kimball, A. W. (1950). Sequential sampling plans for use in psychological test work.Psychometrika,15, 1–15.
Kirk, R. E. (1995).Experimental design: Procedures for the behavioral sciences (3rd ed.). Belmont, CA: Brooks/Cole.
Lachin, J. M. (1981). Introduction to sample size determination and power analysis for clinical trials.Controlled Clinical Trials,2, 93–113.
Lai, T. L. (2001). Sequential analysis: Some classical problems and new challenges.Statistica Sinica,11, 303–408.
Lan, K. K. G., &DeMets, D. L. (1989). Changing frequency of interim analysis in sequential monitoring.Biometrics,45, 1017–1020.
O’Brien, P. C., &Fleming, T. R. (1979). A multiple testing procedure for clinical trials.Biometrics,35, 549–556.
O’Brien, R. G., &Muller, K. E. (1993). Unified power analysis fort-tests through multivariate hypotheses. In L. K. Edwards (Ed.),Applied analysis of variance in behavioral science (pp. 297–344). New York: Dekker.
Siegmund, D. (1985).Sequential analysis: Tests and confidence intervals. New York: Springer.
Siegmund, D. (1994). A retrospective of Wald’s sequential analysis: Its relation to challenge-point detection and sequential clinical trials. In S. S. Gupta & J. O. Berger (Eds.),Statistical decision theory and related topics (pp. 9–33). New York: Springer.
Venables, W. N., & Smith, D. M. (2001).An introduction to R. Retrieved February 16, 2005 from http://www.r-project.org/.
Vos, H. J. (2001). A minimax procedure in the context of sequential testing problems in psychodiagnostics.British Journal of Mathematical & Statistical Psychology,54, 139–159.
Wald, W. (1947).Sequential analysis. New York: Dover.
Wetherill, G. B., &Glazebrook, K. D. (1986).Sequential methods in statistics. London: Chapman & Hall.
Whitehead, J., &Brunier, H. (1990). The double triangular test: A sequential test for the two-sided alternative with early stopping under the null hypothesis.Sequential Analysis,9, 117–136.
Winer, B. J. (1971).Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was partially supported by Grants BSO2003-08908 from the Ministerio de Educación y Ciencia of Spain and 06/HSE/0005/2004 from the Comunidad de Madrid, Spain.
Rights and permissions
About this article
Cite this article
Botella, J., Ximénez, C., Revuelta, J. et al. Optimization of sample size in controlled experiments: The CLAST rule. Behavior Research Methods 38, 65–76 (2006). https://doi.org/10.3758/BF03192751
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03192751