This article discusses theproblem of Type I error in behavioral science research and describes three measures of error rate: experimentwise, per-experiment, andpercentage error rates. The author computes error rates for a sample of behavioral science literature using these three procedures, with the sample comprising seven articles published in The Journal of Applied Behavioral Science. The results of the analysis suggest the presence of a substantial incidence of Type I error in the research. The author presents possible solutions to these problems.
Get full access to this article
View all access options for this article.
References
1.
Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
2.
Bock, R.D., & Haggard, E.A. (1968). The use of multivariate analysis in behavioral research. In D.K. Whitela (Ed.), Handbook ofmeasurement and assessment in behavioral science. Reading, MA: Addison-Wesley.
3.
Cook, T.D., & Campbell, D.T. (1979). Quasiexperimentation: Design & analysis issues for field settings. Boston: Houghton Mifflin.
4.
Cooley, W.W., & Lohnes, P.R. (1971). Multivariate data analysis. New York: John Wiley.
5.
Cowles, M., & Davis, C. (1982). On the origins of the .05 level of statistical significance. American Psychologist, 37, 553-558.
6.
Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco, CA: Jossey-Bass.
7.
Gatio, J. (1978). Multiple comparisons within ANOVA using orthogonal or nonorthogonal components. Educational and Psychological Measurement, 38, 901-904.
8.
Harris, R.J. (1975). A primer of multivariate statistics. New York: Academic Press.
9.
Hummell, T.J., & Sligo, J.R. (1971). Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76,49-57.
10.
Jacobs, K.W. (1976). A table for the determination of experimentwise error (alpha) from independent comparisons. Educational and Psychological Measurement, 36, 899-903.
11.
Keppel, G. (1982). Decision and analysis: A researcher's handbook (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
12.
Marriott, F.H.C. (1974). The interpretation of multiple observations. London: Academic Press.
13.
Petrinovick, L.F., & Handyck, C.P. (1969). Error rates for multiple comparison methods: Some evidence concerning the frequency of erroneous conclusions. Psychological Bulletin, 71, 43-54.
14.
Rosenthal, R., & Rosnow, R.L. (1984). Essentials of behavioral research: Methods and data analysis. New York: McGraw-Hill.
