Scores, that is, ordinal data with few categories, are frequently encountered in practical biopharmaceutical research, but rarely in statistical textbooks and computer programs. This has caused inefficient and even incorrect procedures to appear in the literature, for example, application of the standard Chi-square test or dichotomization and subsequent evaluation of the resulting 2 × 2 table. Four different tests are recommended: 1. the Wilcoxon-Mann-Whitney test, 2. the Mantel-Haenszel test, 3. the arbitrary score test, and 4. all possible score tests (Streitberg-Röhmel), either asymptotic or preferably exactly based on permutational distributions. The discussion includes the relative merits of these tests, alternative procedures, and related areas such as effect size measures and sample size calculations. Two examples from the literature are used to demonstrate that test statistics and other information are easily obtained by using the personal computer (PC) program TESTIMATE.
AndersonB. Methodological Errors in Medical Research. Oxford: Blackwell Scientific Publications; 1990.
2.
McKelvieSJ. Graphic rating scales—How many categories?Br J Psychol. 1978;69:185–202.
3.
De JongeH. Deficiencies in clinical reports for registration of drugs. Stat Med.1983;2: 155–166.
4.
WoodwardJAOverallJE. The Significance of Treatment Effects in Ordered Category Data. Psychometric Laboratory Reports, No. 53, Dept. of Psychiatry, The University of Texas Medical Branch, Galveston, Texas; 1976.
5.
MosesLEEmersonJDHosseiniH. Analyzing data from ordered categories. N Eng J Med.1984;311: 442–448.
6.
MostellerFGilbertJPMcPeekB. Controversies in design and analysis of clinical trials. In: ShapiroSHLouisThA, ed. Clinical Trials. New York: Marcel Dekker; 1983:13–64.
7.
WhiteheadJ. Sample size calculations for ordered categorical data. Internal Report Version 2. Dept of Applied Statistics, University of Reading, UK; 1993.
8.
HarkinsRDAlbrechtR. Design and analysis of clinical trials for antiinfective drug products, Drug Inf J.1990;24:213–224.
9.
MantelN. Chi-square tests with one degree of freedom; extensions of the Mantel-Haenszel procedure. J Am Stat Assoc. 1963;58: 690–700.
10.
MantelNHaenszelW. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst. 1959;22: 719–748.
11.
KochGGEdwardsS. Clinical efficacy trials with categorical data. In: PeaceKE ed. Biopharmaceutical Statistics for Drug Development. New York: Marcel Dekker, 1988:403–457.
12.
ZimmermannHRahlfsVW. Methoden der Auswertung von Ordinaldaten mit wenigen Ausprägungen und ihre Berechnung mit Standardpaketen; Paper 3. Konferenz über die wissenschaftliche Anwendung von Statistik-Software. 3–6. März 1985, Neuherberg, Gesellschaft für Strahlen- und Umweltforschung in Zusammenarbeit mit dem Zentrum für Umfragen, Methoden und Analysen (ZUMA) und der Arbeitsgruppe Statistische Auswertungssysteme der GMDS.
13.
EmersonJDMosesLE. A note on the Wilcoxon-Mann-Whitney test for 2xK ordered tables. Biometrics. 1985;41: 303–309.
14.
StreitbergBRöhmelJ. Exact distributions for permutation and rank tests: an introduction to some recently published algorithms. Statistical Software Newsletter. 1986;12: 10–17.
15.
MantelN. Ridit analysis and related ranking procedures—use at your own risk. Am J Epidem. 1979;109: 25–29.
16.
GraubardBIKornEL. Choice of column scores for testing independence in ordered 2xK contingency tables. Biometrics. 1987;43: 471–476.
17.
ZimmermannH. Erweiterung des Mantel-Haenszel tests auf K (>2) Gruppen. Biom J.1984;26: 223–224.
18.
SchneiderB. The choice of statistical tests. Arzneim Forsch Drug Res. 1991;41: 869–884.
19.
SnedecorGWCochranWG. Statistical Methods. Ames, Iowa: The Iowa State University Press;1980.
20.
HeerenTD'AgostinoR. Robustness of the two independent samples t-test when applied to ordinal scaled data. Stat Med.1987;6: 79–90.
21.
BrossIDJ. How to use ridit analysis. Biometrics. 1958;14: 18–38.
22.
SelvinS. A further interpretation on the interpretation of ridit analyses. Am J Epidem. 1977;105: 16–20.
23.
TaguchiG. Statistical Analysis (Japanese). Tokyo, Japan: Maruzen;1966.
24.
TaguchiG. A new statistical analysis method for clinical data, the accumulating analysis, in contrast with the Chi-square test. Saishin Igaku (The Newest Medicine). 1974;29:806–813.
25.
NairVN. Testing in industrial experiments with ordered categorical data. Technometrics. 1986;28: 283–291.
26.
NairVN. Chi-squared-type tests for ordered alternatives in contingency tables. J Am Stat Assoc. 1987;82: 283–291.
HettmanspergerTP. Statistical Inference Based on Ranks. New York: John Wiley & Sons;1984.
29.
StreitbergBRöhmelJ. Simultane Anwendung aller möglichen Tests zu einem vorgelegten Test problem: P-Wert Schranken für monotone Permutationstests im 2-Stichprobenproblem. In: BauerPHommelGSonnemannE. ed. Multiple Hypothesenprüfung, Symposium, 6. und 7. November 1987. Berlin: Springer-Verlag;1988:86–97.
30.
KimeldorfGSampsonARWhitakerLR. Min and Max Scorings for two-sample ordinal data. J Am Stat Assoc. 1992;87: 241–247.
31.
GardnerMJAltmanDG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J.1986;292: 746–750.
32.
ColditzGAMillerJNMostellerF. Measuring gain in the evaluation of medical technology. Intl J Tech Assess Health Care. 1988;4: 637–642.
33.
AgrestiA. Generalized odds ratio for ordinal data. Biometrics. 1980;36: 59–67.
34.
BernhardGAlleMHerboldMMeyersW. Investigation on the reliability of some elementary nonparametric methods in statistical analysis systems. Statistical Software Newsletter. 1988;14: 19–26.
35.
MehtaCRPatelNR. A network algorithm for the exact treatment of the 2xK contingency table. Comm Statist-Simula Computa. 1980;9: 649–664.
36.
MehtaCRPatelNR. A network algorithm for performing Fisher's exact test in RxC contingency tables. J Am Stat Assoc. 1983;78: 427–434.
37.
ZimmermannH. Exact calculation of permutational distributions for two dependent samples. Biom J.1985a;27: 349–352.
38.
ZimmermannH. Exact calculation of permutational distributions for two independent samples. Biom J.1985b;27: 431–434.
