Several issues regarding the effects of autocorrelated errors on Type I error in ordinary least-squares models are clarified. Although autocorrelated errors have a large effect on both omnibus F tests and tests on individual intervention effect coefficients in many applications, there are exceptions that have not been pointed out previously. It is demonstrated that under certain conditions, distortion in Type I error is far less than is predicted by asymptotic theory. It is shown that these exceptions occur because the effect of autocorrelated errors is dependent on (a) the type of parameters (e.g., level change and/or slope change) required in the model, (b) the number of variables in the design matrix, and (c) the sample size. Because existing time-series methods perform poorly with small samples, this may be a useful finding in some situations; however, a better general solution is to use a recently developed small-sample method.
Get full access to this article
View all access options for this article.
References
1.
Agras, W. S. , Barlow, D. H., Chapin, H. N., Abel, G. G., & Leitenberg, H. (1974). Behavior modification of anorexia nervosa. Archives of General Psychiatry, 30, 279-286.
2.
Barlow, D. H. , & Hersen, M. (1985). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Pergamon.
3.
Berry, W. D. , & Lewis-Beck, M. S. (1986). Interrupted time series. In W. D. Berry & M. S. Lewis-Beck (Eds.), New tools for social scientists: Advances and applications in research methods (pp. 209-240). Beverly Hills, CA: Sage.
4.
Bradley, J. V. (1978). Robustness?British Journal of Mathematical and Statistical Psychology, 31, 144-152.
5.
Cook, T. D. , & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Chicago: Rand McNally.
6.
Durbin, J. , & Watson, G. S. (1950). Testing for serial correlation in least squares regression I. Biometrika, 37, 409-428.
7.
Durbin, J. , & Watson, G. S. (1951). Testing for serial correlation in least squares regression II. Biometrika, 38, 159-178.
8.
Epstein, L. H. , Hersen, M., & Hemphill, D. P. (1974). Music feedback in the treatment of tension headache: An experimental case study. Journal of Behavior Therapy and Experimental Psychiatry, 5, 59-63.
9.
Glass, G. V. , Willson, V. L., & Gottman, J. M. (1975). Design and analysis of time series experiments. Boulder: Colorado Associated University Press.
10.
Gottman, J. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge, UK: Cambridge University Press.
11.
Gottman, J. M. , & Glass, G. V. (1978). Analysis of interrupted time-series experiments. In T. R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 197-235). New York: Academic Press.
12.
Hibbs, D. A. (1974). Problems of statistical estimation and causal inference in time-series regression models. In H. L. Costner (Ed.), Sociological methodology (pp. 252-308). San Francisco: Jossey-Bass.
13.
Huitema, B. E. , & McKean, J. W. (1991). Autocorrelation estimation and inference with small samples. Psychological Bulletin, 110, 291-304.
14.
Huitema, B. E. , & McKean, J. W. (1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3, 104-116.
15.
Huitema, B. E. , & McKean, J. W. (In press). Design specification issues in least-squares intervention models. Educational and Psychological Measurement.
16.
Huitema, B. E. , McKean, J. W., & Zhao, J. (1996). The runs test for autocorrelated errors: Unacceptable properties. Journal of Educational and Behavioral Statistics, 21, 390-403.
17.
Jaeger, R. M. (Ed.). (1997). Complementary methods for research in education (2nd ed.). Washington, DC: American Educational Research Association.
18.
Johnston, J. (1984). Econometric methods. New York: McGraw-Hill.
19.
Kahner, D. , Moler, C., & Nash, S. (1988). Numerical methods and software. Englewood Cliffs, NJ: Prentice Hall.
20.
Liberman, R. P. , Davis, J., Moon, W., & Moore, J. (1973). Research design for analyzing drugenvironment-behavior interactions. Journal of Nervous and Mental Disease, 156, 432-439.
21.
Malinvaud, E. (1970). Statistical methods of econometrics. Amsterdam, the Netherlands: North-Holland.
22.
Manly, B.F.J. (1992). The design and analysis of research studies. London: Cambridge University Press.
23.
McCain, L. J. , & McCleary, R. (1979). The statistical analysis of the simple interrupted timeseries quasi-experiment. In T. D. Cook & D. T. Campbell (Eds.), Quasi-experimentation: Design & analysis issues for field settings (pp. 233-293). Boston: Houghton Mifflin.
24.
McKnight, S. D., McKean, J. W., & Huitema, B. E. (1999). A double bootstrap method to analyze linear models with autoregressive error terms. Manuscript submitted for publication.
25.
Mohr, L. B. (1995). Impact analysis for program evaluation (2nd ed.). Thousand Oaks, CA: Sage.
26.
Mood, A. M. (1940). The distribution theory of runs. Annals of Mathematical Statistics, 11, 367-392.
27.
Ostrom, C. W., Jr. (1990). Time series analysis regression techniques (2nd ed.). Newbury Park, CA: Sage.
28.
Padia, W. L. (1973). Effect of autocorrelation on probability statements about the mean. Unpublished master’s thesis, Laboratory of Educational Research, University of Colorado.
29.
Porterfield, J. , Blunden, R., & Blewitt, E. (1980). Improving environments for profoundly handicapped adults: Using prompts and social attention to maintain high group engagement. Behavior Modification, 4, 225-241.
