Abstract
The valid use of a randomization test with single-case data typically involves the choice of a test statistic prior to gathering data. In situations in which it is difficult to anticipate the form of the effect, the early specification of the test statistic can be problematic. The purpose of this paper is to demonstrate methods that will allow single-case researchers to conduct valid randomization tests in situations in which they wish to delay the specification of the test statistic until the data have been observed. A concrete example is given to motivate the use, explicate the logic, and demonstrate the conduct of this method.
Article PDF
Similar content being viewed by others
References
Allison, D. B., Franklin, R. D., &Heshka, S. (1992). Reflections on visual inspection, response guided experimentation, and Type I error rate in single-case designs.Journal of Experimental Education,61, 45–51.
Baer, D. M. (1977). Perhaps it would be better not to know everything.Journal of Applied Behavior Analysis,10, 167–172.
DeProspero, A., &Cohen, S. (1979). Inconsistent visual analysis of intrasubject data.Journal of Applied Behavioral Analysis,12, 573–579.
Edgington, E. S. (1980a). Overcoming obstacles to single-subject experimentation.Journal of Educational Statistics,5, 261–267.
Edgington, E. S. (1980b). Validity of randomization tests for one-subject experiments.Journal of Educational Statistics,5, 235–251.
Edgington, E. S. (1992). Nonparametric tests for single-case experiments. In T. R. Kratochwill & J. R. Levin (Eds.),Single-case research design and analysis: New directions for psychology and education (pp. 133–157). Hillsdale, NJ: Erlbaum.
Edgington, E. S. (1995).Randomization tests (3rd ed.). New York: Marcel Dekker.
Ferron, J. (1993, April).Suggested solutions to problems facing the use of randomization tests with single-case designs. Paper presented at the annual meeting of the American Educational Research Association, Atlanta.
Ferron, J., &Onghena, P. (1996). The power of randomization tests for single-case phase designs.Journal of Experimental Education,64, 231–239.
Ferron, J., &Ware, W. (1994). Using randomization tests with responsive single-case designs.Behavior Research & Therapy,32, 787–791.
Ferron, J., &Ware, W. (1995). Analyzing single-case data: The power of randomization tests.Journal of Experimental Education,63, 167–178.
Greenwood, K. M., &Matyas, T. A. (1990). Problems with the application of interrupted time series analysis for brief single-subject data.Behavioral Assessment,12, 355–370.
Kazdin, A. E. (1980). Obstacles in using randomization tests in singlecase experimentation.Journal of Educational Statistics,5, 253–260.
Knapp, T. J. (1983). Behavior analysts’ visual appraisal of behavior change in graphic display.Behavioral Assessment,5, 155–164.
Koehler, M. J., &Levin, J. R. (1996, April).A sharper analytical tool for the multiple-baseline design. Paper presented at the annual meeting of the American Educational Research Association, New York.
Kratochwill, T. R. (1992). Single-case research design and analysis: An overview. In T. R. Kratochwill & J. R. Levin (Eds.),Single-case research design and analysis: New directions for psychology and education (pp. 1–14). Hillsdale, NJ: Erlbaum.
Kromrey, J. D., &Foster-Johnson, L. (1996). Determining the efficacy of intervention: The use of effect sizes for data analysis in singlesubject research.Journal of Experimental Education,65, 73–93.
Levin, J. R., Marascuilo, L. A., &Hubert, L. J. (1978). N = nonparametric randomization tests. In T. R. Kratochwill (Ed.),Single subject research: Strategies for evaluating change (pp. 167–196). New York: Academic Press.
Marascuilo, L. A., &Busk, P. L. (1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects.Behavioral Assessment,10, 1–28.
Matyas, T. A., &Greenwood, K. M. (1990). Visual analysis of singlecase time series: Effects of variability, serial dependence, and magnitude of intervention effects.Journal of Applied Behavior Analysis,23, 341–351.
Matyas, T. A., &Greenwood, K. M. (1991). Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data.Behavioral Assessment,13, 137–157.
McCleary, R., &Welsh, W. N. (1992). Philosophical and statistical foundations of time-series experiments. In T. R. Kratochwill & J. R. Levin (Eds.),Single-case research design and analysis: New directions for psychology and education (pp. 41–91). Hillsdale, NJ: Erlbaum.
Onghena, P. (1992). Randomization tests for extensions and variations of ABAB single-case experimental designs: A rejoinder.Behavioral Assessment,14, 153–171.
Onghena, P. (1994).The power of randomization tests for single-case designs. Unpublished doctoral dissertation, Katholieke Universiteit Leuven.
Onghena, P., &Edgington, E. S. (1994). Randomization tests for restricted alternating treatments designs.Behavior Research & Therapy,32, 783–786.
Park, H. S., Marascuilo, L., &Gaylord-Ross, R. (1990). Visual inspection and statistical analysis in single-case designs.Journal of Experimental Education,58, 311–320.
Parsonson, B. S., &Baer, D. M. (1986). The graphic analysis of data. In A. Poling & W. R. Fuqua (Eds.),Research methods in applied behavior analysis: Issues and advances (pp. 157–186). New York: Plenum.
Parsonson, B. S., &Baer, D. M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.),Single-case research design and analysis: New directions for psychology and education (pp. 15–40). Hillsdale, NJ: Erlbaum.
Skinner, B. F. (1963). Operant behavior.American Psychologist,18, 503–515.
Stocks, J. T., &Williams, M. (1995). Evaluation of single subject data using statistical hypothesis tests versus visual inspection of charts with and without celeration lines.Journal of Social Research,20, 105–126.
Toothaker, L. E., Banz, M., Noble, C., Camp, J., &Davis, D. (1983). N=1 designs: The failure of ANOVA-based tests.Journal of Educational Statistics,8, 289–309.
Wampold, B. E., &Worsham, N. L. (1986). Randomization tests for multiple-baseline designs.Behavioral Assessment,8, 135–143.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors thank Patrick Onghena and three anonymous reviewers for comments on an earlier version of this manuscript. L.F.-J. is currently a Faculty Research Statistics Specialist at the Amos Tuck School of Business, Dartmouth College.
Rights and permissions
About this article
Cite this article
Ferron, J., Foster-johnson, L. Analyzing single-case data with visually guided randomization tests. Behavior Research Methods, Instruments, & Computers 30, 698–706 (1998). https://doi.org/10.3758/BF03209489
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03209489