Some researchers have argued for the use of or have attempted to make use of randomization tests in single-subject research. To address this tide of interest, the authors of this article describe randomization tests, discuss the theoretical rationale for applying them to single-subject research, and provide an overview of the methodological research into the Type I error rates and statistical power of randomization tests in single-subject designs. It is found that randomization tests generally are an awkward fit to single-subject designs, and they are found to have low statistical power in single-subject contexts. It is therefore recommended that single-subject researchers make sparing, if any, use of randomization tests to analyze their data.
Barlow, D.H., & Hayes, S.C. ( 1979). Alternating treatments design: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12, 199-210.
2.
Barlow, D.H., & Hersen, M. ( 1984). Single case experimental designs: Strategies for studying behavior change. New York: Pergamon Press.
3.
Birnbrauer, J.S. ( 1981). External validity and experimental investigation of individual behaviour. Analysis and Intervention in Developmental Disabilities , 1, 117-132.
4.
Busk, P.L., & Marascuilo, L.A. (1988). Autocorrelation in single-subject research: A counterargument to the myth of no autocorrelation. Behavioral Assessment, 10, 229-242.
Edgington, E.S. ( 1980). Validity of randomization tests for one-subject experiments . Journal of Educational Statistics, 5, 235-251.
9.
Edgington, E.S. ( 1987). Randomization tests (2nd ed.). New York: M. Dekker.
10.
Ferron, J., Foster-Johnson, L., & Kromrey, J.D. ( 2003). The functioning of single-case randomization tests with and without random assignment. The Journal of Experimental Education, 71, 267-288.
11.
Ferron, J., & Onghena, P. ( 1996). The power of randomization tests for single-case phase designs. Journal of Experimental Education, 64, 231-239.
12.
Ferron, J., & Sentovich, C. ( 2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70, 165-178.
13.
Ferron, J., & Ware, W. ( 1994). Using randomization tests with responsive single-case designs. Behaviour Research and Therapy, 32, 787-791.
14.
Ferron, J., & Ware, W. ( 1995). Analyzing single-case data: The power of randomization tests. Journal of Experimental Education, 63, 167-178.
15.
Fisher, R.A. ( 1935). The design of experiments. Edinburgh : Oliver & Boyd.
16.
Good, P.I. ( 1994). Permutation tests: A practical guide to resampling methods for testing hypotheses. New York: Springer-Verlag.
17.
Hains, A.H., & Baer, D.M. ( 1989). Interaction effects in multielement designs: Inevitable, desirable, and ignorable. Journal of Applied Behavior Analysis , 22, 57-69.
18.
Kazdin, A.E. ( 1980). Obstacles in using randomization tests in single-case experimentation. Journal of Educational Statistics, 5, 253-260.
19.
Kazdin, A.E. ( 1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.
Koehler, M.J., & Levin, J.R. ( 1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods , 3, 206-217.
22.
Levin, J.R., Marascuilo, L.A., & Hubert, L.J. ( 1978). N = 1 nonparametric randomization tests. In T. R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 167-196). New York: Academic Press.
23.
Marascuilo, L.A., & Busk, P.L. ( 1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10, 1-28.
Matyas, T.A., & Greenwood, K.M. (1990). Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341-351.
26.
Mielke, P.W., & Berry, K.J. ( 2007). Permutation methods: A distance function approach (2nd ed.). New York: Springer.
27.
Ninness, C., Newton, R., Saxon, J., Rumph, R., Bradfield, A., Harrison, C., & Vasquez, E., III. (2002). Small group statistics: A Monte Carlo comparison of parametric and randomization tests. Behavior and Social Issues, 12, 53-63.
28.
Onghena, P., & Edgington, E.S. (1994). Randomization tests for restricted alternating treatments designs. Behaviour Research and Therapy, 32, 783-786.
29.
Peres-Neto, P.R., & Olden, J.D. ( 2001). Assessing the robustness of randomization tests: Examples from behavioural studies. Animal Behaviour, 61, 79-86.
30.
Pesarin, F. ( 2001). Multivariate permutation tests with applications in biostatistics . West Sussex, UK: Wiley.
31.
Pitman, E.J.G. ( 1937a). Significance tests which may be applied to samples from any populations. Supplement to the Journal of the Royal Statistical Society, 4, 119-130.
32.
Pitman, E.J.G. ( 1937b). Significance tests which may be applied to samples from any populations: II. The correlation coefficient test. Supplement to the Journal of the Royal Statistical Society, 4, 225-232.
33.
Pitman, E.J.G. ( 1938). Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika , 29, 322-335.
34.
Sidman, M. ( 1960). Tactics of scientific research: Evaluating experimental data in psychology. New York: Basic Books.
35.
Sierra, V., Solanas, A., & Quera, V. ( 2005). Randomization tests for systematic single-case designs are not always appropriate. Journal of Experimental Education , 73, 140-160.
36.
Singer, J.D., & Willett, J.B. ( 2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.
37.
Todman, J.B., & Dugard, P. ( 2001). Single-case and small-n experimental designs: A practical guide to randomization tests. Mahwah, NJ: Lawrence Erlbaum.
38.
Ulman, J.D., & Sulzer-Azaroff, B. (1975). Multielement baseline design in educational research. In E. Ramp & G. Semb (Eds.), Behavior analysis: Areas of research and application (pp. 377-391). Englewood Cliffs, NJ: Prentice-Hall.
