This paper deals with the problem of multiple inference in psychiatric research, an issue which arises whenever a researcher has to make more than one statistical inference in a single research study. It frequently arises in psychiatric research because of multivariate study designs, with subjects being measured on more than one dependent variable with the intention of studying differences between groups in mean scores. The disadvantages of the commonly adopted strategy of using multiple univariate tests (e.g. multiple t-tests) are outlined. Two broad strategies — Bonferroni-adjusted univariate tests and multivariate statistical analysis — are introduced. Their advantages and disadvantages are discussed in terms of their usefulness in confirmatory and exploratory research in psychiatry.
References
1.
BaileyB. J. R. (1977) Tables of the Bonferroni t statistic. Journal of American Statistical Association, 72, 469–477.
2.
BirdK. D. (1975) Simultaneous contrast testing procedures for multivariate experiments. Multivariate Behavioral Research, 10, 343–352.
3.
BirdK. D.Hadzi-PavlovicD. (1983) Simultaneous test procedures and the choice of a test statistic in MANOVA. Psychological Bulletin, 93, 167–178.
4.
BishopY. M. M.FienbergS.HollandP. W. (1975) Discrete Multivariate Analysis, Cambridge, Ma., M.I.T. Press.
5.
BonnettD. G.BentlerP. M. (1983) Goodness-of-fit procedures for the evaluation and selection of log-linear models. Psychological Bulletin, 93, 149–166.
6.
CampbellD. T. (1969) Reforms as experiments. American Psychologist, 24, 409–429.
7.
CohenJ. (1977) Statistical Power Analysis for the Behavioural Sciences. New York, Academic Press.
8.
CohenJ.CohenP. (1983) Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Second Edition. Hillsdale, Lawrence Erlbaum.
9.
DaytonC. M.SchaferW. D. (1973) Extended tables of t and chi-square for Bonferroni tests with unequal error allocation. Journal of the American Statistical Association, 68, 78–83.
10.
FinnJ. D. (1969) Multivariate analysis of repeated measures data. Multivariate Behavioral Research, 4, 391–413.
11.
GlantzS. A. (1980) Biostatistics: how to detect, correct and prevent errors in the medical literature. Circulation, 61, 1–7.
12.
GlassG. V.PeckhamP. D.SandersJ. R. (1972) Consequences of failure to meet assumptions underlying fixed effects analyses of variance and covariance. Review of Educational Research, 42, 237–288.
13.
GoodmanL. A. (1964) Interactions in multidimensional contingency tables. Annals of Mathematical Statistics, 35, 632–646.
14.
GoreS. M.JonesI. G.RytterE. C. (1977) Misuse of statistical methods: critical assessment of articles in British Medical Journal from January to March 1976. British Medical Journal, 1, 85–87.
15.
GroveW. M.AndreasenN. C. (1982) Simultaneous tests of many hypotheses in exploratory research. Journal of Nervous and Mental Diseases, 170, 3–8.
16.
HarrisR. J. (1975) A Primer of Multivariate Statistics. New York, Academic Press.
17.
HarrisR. J. (1979) Multivariate Statistics: When Will Experimental Psychology Catch Up? Centennial Symposium: Psychology and Mathematics, American Psychological Association Meeting, New York.
18.
HummelT. J.SligoJ. R. (1971) Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76, 49–57.
19.
JonesD. R.RushtonL. (1982) Simultaneous inference in epidemiological studies. International Journal of Epidemiology, 11, 276–282.
20.
KahnemanD.SlovicP.TverskyA. (Eds) (1983) Judgment Under Uncertainty: Heuristics and Biases. Cambridge, Cambridge University Press.
21.
LarzelereR. E.MulaikS. A. (1977) Single-sample tests for many correlations. Psychological Bulletin, 84, 557–569.
22.
MeierP. (1975) Statistics and medical experimentation. Biometrics, 31, 511–529.
23.
MillerR. G. (1981) Simultaneous Statistical Inference. Second Edition. New York, Springer-Verlag.
24.
MorrisonD. F. (1976) Multivariate Statistical Methods. Second Edition. New York, McGraw-Hill.
25.
MostellerF.TukeyJ. W. (1977) Data Analysis and Regression. Reading, Ma., Addison-Wesley.
26.
NunnallyJ. C. (1978) Psychometric Theory. Second Edition. New York, McGraw-Hill.
27.
OlsonC. L. (1976) On choosing a test stastistic in multivariate analysis of variance. Psychological Bulletin, 83, 579–586.
28.
PoorD. D. S. (1973) Analysis of variance for repeated measures designs: two approaches. Psychological Bulletin, 80, 204–209.
29.
RodgerR. S. (1967) Type 1 errors and their decision basis. British Journal of Mathematical and Statistical Psychology, 20, 51–62.
30.
RodgerR. S. (1973) Confidence intervals for multiple comparisons and the misuse of the Bonferroni inequality. British Journal of Mathematical and Statistical Psychology, 26, 58–60.
31.
RyanT. A. (1959) Multiple comparisons in psychological research. Psychological Bulletin, 56, 26–47.
32.
TabachnickB. G.FidellL. S. (1983) Using Multivariate Statistics. New York, Harper and Row.
33.
TukeyJ. W. (1969) Analyzing data: sanctification or detective work?American Psychologist, 24, 83–91.
34.
TukeyJ. W. (1977) Some thoughts on clinical trials, especially problems of multiplicity. Science, 198, 679–684.
35.
WhiteS. J. (1979) Statistical errors in papers in the British Journal of Psychiatry. British Journal of Psychiatry, 135, 336–342.
