This article describes the Bonferroni multiple-comparison procedure, and makes a case for researchers’ more frequent and appropriate use of it. The procedure is discussed as a test that facilitates investigation of precise and powerful a priori multiple comparisons. Characteristics of the Bonferroni procedure are described in relation to the more familiar Scheffe post hoc multiple-comparison method, and a step-by-step guide for comparing and choosing between the two is provided. The Bonferroni procedure is discussed in detail in the context of one-factor analysis-of-variance designs. Application of the technique is then considered in the context of factorial designs, analyses of covariance, univariate repeated-measures analyses, multivariate analyses of variance, and recent sequential hypothesis-testing extensions. To aid the presentation, an example from the field of management is included.
Get full access to this article
View all access options for this article.
References
1.
Bailey, B., Jr. (1977). Tables of the Bonferroni I statistic. Journal of the American Statistical Association, 72: 469-477.
2.
Betz, M. A. & Levin, J. R. (1982). Coherent analysis-of-variance hypothesis-testing strategies: A general model. Journal of Educational Statistics, 7: 193-206.
3.
Dayton, C. M. & Schafer, W. D. (1973). Extended tables off and Chi square for Bonferroni tests with unequal error allocation. Journal of the American Statistical Association, 68: 78-83.
4.
Dunham, R. B., Pierce, J. L., & Castaiteda, M. B. (1987). Alternative work schedules: Two field quasi-experiments. Personnel Psychology, 40: 215-242.
5.
Dunn, O. J. (1961). Multiple comparisons among means. American Statistical Association Journal, 56: 52-64.
6.
Gabriel, K. R. (1969). Simultaneous test procedures: Some theory of multiple comparisons. Annuls of Mathematical Statistics, 40: 224-250.
7.
Games, P. A. (1977). An improved I table for simultaneous control on 9 contrasts. Journal of the American Statistical Association, 72: 531-534.
8.
Gardner, W. L. & Martinko, M. J. (1988). Impression management: An observational study linking audience characteristics with verbal self-presentations. Academy of ManagementJournal, 31: 42-65.
9.
Hedge, J. W. & Teachout, M. S. (1992). An interview approach to work sample criterion measurement. Journal of Applied Psychology, 77: 453-461.
10.
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75: 800-802.
11.
Hochberg, Y. & Tamhane, A. C. (1987). Multiple comparison procedures. New York: Wiley.
12.
Holland, B. S. & Copenhaver, M. D. (1988). Improved Bonferroni-type multiple testing procedures. Psychological Bulletin, 104: 145-149.
13.
Holm, S. (1977). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6: 65-70.
14.
Jandreau, S. & Bever, T. (1992). Phrase-spaced formats improve comprehension in average readers. Journal of Applied Psychology, 77: 143-146.
15.
Jaccard, J., Becker, M. A., & Wood, G. (1984). Pairwise multiple comparison procedures: A review. Psychological Bulletin, 96: 589-596.
16.
Keselman, H. J., Keselman, J. C., & Games, P. A. (1991). Maximum familywise Type I error rate: The least significant difference, Newman-Keuls, and other multiple comparison procedures. Psychological Bulletin, 110: 155-161.
17.
Keselman, H. J., Keselman, J. C., & Shaffer, J. P. (1991). Multiple pairwise comparisons of repeated measures means under violation of multisample sphericity. Psychological Bulletin, 110: 162-170.
18.
Kirk, R. E. (1982). Experimental design: Procedures for the behavioral sciences. Belmont: Brooks/Cole.
19.
Klockars, A. J. & Hancock, G. R. (1992). Power of recent multiple comparison procedures as applied to a complete set of planned orthogonal contrasts. Psychological Bulletin, 111: 505-510
20.
Levin, J. R. (1985). Some methodological and statistical “bugs” in research on children’s learning. Pp. 205-233 in M. Pressley & C.J. Brainerd (Eds.), Cognitive learning and memory in children, New York: Springer-Verlag.
21.
Levin, J. R., Serlin, R. C., & Seaman, M. A. (In press). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin.
22.
Lin, T. R., Dobbins, G. H., & Farh, J. L. (1992). A field study of race and age similarity effects on interview ratings in conventional and situational interviews. Journal of Applied Psychology, 77: 363-371,
23.
Marascuilo, L. A. & Levitt, J. R. (1970). Appropriate post hoc comparisons for interaction and nested hypotheses in analysis of variance designs: The elimination of type IV errors. American Educational Research Journal, 1: 397-421
24.
Marascuilo, L. A. & Levitt, J. R. (1976). A note on the simultaneous investigation of interaction and nested hypotheses in two-factor analysis of variance designs. American Educational Research Journal, 13: 61-65.
25.
Marascuilo, L. A. & Levitt, J. R. (1983). Multivariate statistics in the social sciences: A researcher’s guide. Monterey, CA: Brooks/Cole.
26.
Marascuilo, L. A. & McSweeney, M. (1977). Nonparametric and distribution-free methods for the social sciences. Monterey, CA: Brooks/Cole.
27.
Maxwell, S. E. (1980). Pairwise multiple comparisons in repeated measures designs. Journal of Educational Statistics, 5: 269-287.
28.
Mento, A. J., Locke, E. A., & Klein, H. J. (1992). Relationship of goal level to valence and instrumentality. Journal of’Applied Psychology, 77: 395-405.
29.
Miller. R. G.. Jr. (1966). Simultaneous statistical inference. New York: McGraw-Hill.
30.
Miller. R. G.. Jr. (1981). Simultaneous statistical inference. New York: Springer-Verlag.
31.
Petrinovich, L. & Hardyck, C. D. (1969). Error rates for multiple comparison methods: Some evidence concerning the frequency of erroneous conclusions. Psychological Bulletin, 71: 43-54.
32.
Ralston, D. A., Anthony, W. P., & Gustafson, D. J. (1985). Employees may love flextime. but what does it do to the organization’s productivity?Journal of Applied Psychology, 70: 272-279.
33.
Rogan, J. C., Keselman, H. J., & Mendoza, J. L. (1979). Analysis of repeated measurements. British Journalof Mathematical and Statistical Psychology, 32: 269-286.
34.
Scheffe, H. (1953). A method for judging all contrasts in the analysis of variance. Biometrika, 40: 87-104.
35.
Seaman, M.’ A., Levin, J. R., & S&in, R. C. (1991). New developments in pairwise multiple comparisons: Some powerful and practicable procedures. Psychological Bulletin, 110: 577-586.
36.
Shaffer, J. P. (1979). Comparison of means: An F test followed by a modified multiple range procedure. Journal of Educational Statistics, 81: 826-831.
37.
Shaffer, J. P. (1986). Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association, 81: 826-831
38.
Sidak, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions, Journal of’the American Statistical Association, 62: 626-633.
39.
Steckler, N. A. & Rosenthal, R. (1985). Sex differences in nonverbal and verbal communication with bosses, peers, and subordinates. Journal of Applied Psychology, 70: 157-163.
40.
Streufert, S. & Pogash, R. M., Roache, J., Gingrich, D., Landis, R., Severs, W. & Lonardi, L., & Kantner, A. (1992). Effects of alcohol intoxication on risk taking, strategy, and error rate in visuomotor performance. Journal of Applied Psycho&v, 77: 515-524.
41.
Yukl, G. & Tracey, B. (1992). Consequences of influence tactics used with subordinates, peers and the boss. Journal of Applied Psychology, 77: 525-535.
