Cluster randomized trials commonly employ multiple endpoints. When a single summary of treatment effects across endpoints is of primary interest, global methods represent a common analysis strategy. However, specification of the required joint distribution is non-trivial, particularly when the endpoints have different scales. We develop rank-based interval estimators for a global treatment effect referred to here as the “global win probability, or the mean of multiple Wilcoxon Mann–Whitney probabilities, and interpreted as the probability that a treatment individual responds better than a control individual on average. Using endpoint-specific ranks among the combined sample and within each arm, each individual-level observation is converted to a “win fraction” which quantifies the proportion of wins experienced over every observation in the comparison arm. An individual’s multiple observations are then replaced with a single “global win fraction” by averaging win fractions across endpoints. A linear mixed model is applied directly to the global win fractions to obtain point, variance, and interval estimates adjusted for clustering. Simulation demonstrates our approach performs well concerning confidence interval coverage and type I error, and methods are easily implemented using standard software. A case study using public data is provided with corresponding R and SAS code.
O’BrienPGellerN. Interpreting tests for efficacy in clinical trials with multiple endpoints. Control Clin Trials1997; 18: 222–227.
6.
RistlRUrachSRosenkranzG, et al.Methods for the analysis of multiple endpoints in small populations: A review. J Biopharm Stat2019; 29: 1–29.
7.
O’BrienP. Procedures for comparing samples with multiple endpoints. Biometrics1984; 40: 1079–1087.
8.
LachinJM. Applications of the Wei-Lachin multivariate one-sided test for multiple outcomes on possibly different scales. PLoS ONE2014; 9: e108784.
9.
LoganBRTamhaneAC. On O’Brien’s OLS and GLS tests for multiple endpoints. IMS Lect Notes Monogr Ser2004; 47: 76–88.
10.
OffenWChuang-SteinCDmitrienkoA, et al.Multiple co-primary endpoints: Medical and statistical solutions: A report from the multiple endpoints expert team of the pharmaceutical research and manufacturers of America. Drug Inf J2007; 41: 31–46.
11.
SankohAHuqueMRussellH, et al.Global two-group multiple endpoint adjustment methods applied to clinical trials. Drug Inf J1999; 33: 119–140.
12.
HuangPTilleyBWoolsonR, et al.Adjusting O’Brien’s test to control type I error for the generalized nonparametric Behrens–Fisher problem. Biometrics2005; 61: 531–539.
13.
HuangPWoolsonRO’BrienP. A rank-based sample size method for multiple outcomes in clinical trials. Stat Med2008; 27: 3084–3104.
14.
SchulzKFAltmanDG, et al.CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. J Pharmacol Pharmacot2010; 1: 100–107.
15.
AcionLPetersonJJTempleS, et al.Probabilistic index: An intuitive non-parametric approach to measuring the size of treatment effects. Stat Med2006; 25: 591–602.
16.
DongGHuangBVerbeeckJ, et al.Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes. Pharm Stat2023; 22: 20–33.
17.
ZouG. Confidence interval estimation for treatment effects in cluster randomization trials based on ranks. Stat Med2021; 40: 3227–3250.
18.
ZouGSmithEJZouL, et al.A rank-based approach to design and analysis of pretest-posttest randomized trials, with application to COVID-19 ordinal scale data. Contemp Clin Trials2023; 126: 107085.
19.
ZouGZouL. A nonparametric global win probability approach to the analysis and sizing of randomized controlled trials with multiple endpoints of different scales and missing data: Beyond O’Brien–Wei–Lachin. Stat Med2024; 43: 5366–5379.
20.
ZouGZouLChoiY. Distribution-free approach to the design and analysis of randomized stroke trials with the modified Rankin scale. Stroke2022; 53: 3025–3031.
21.
ZouGZouLQiuSF. Parametric and nonparametric methods for confidence intervals and sample size planning for win probability in parallel-group randomized trials with Likert item and Likert scale data. Pharm Stat2023; 22: 418–439.
22.
WightDRaabGHendersonM, et al.Limits of teacher delivered sex education: Interim behavioural outcomes from randomised trial. BMJ2002; 324: 1430–1435.
23.
RahlfsVZimmermannH. Effect size measures and their benchmark values for quantifying benefit or risk of medicinal products. Biomet J2019; 61: 973–982.
24.
LachinJM. Some large-sample distribution-free estimators and tests for multivariate partially incomplete data from two populations. Stat Med1992; 11: 1151–1170.
25.
BuyseM. Generalized pairwise comparisons of prioritized outcomes in the two-sample problem. Stat Med2010; 29: 3245–3257.
26.
AgrestiA. Generalized odds ratios for ordinal data. Biometrics1980; 36: 59–67.
27.
PocockSAritiCCollierT, et al.The win ratio: A new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. Eur Heart J2012; 33: 176–182.
28.
DongGHoaglinDQiuJ, et al.The win ratio: On interpretation and handling of ties. Stat Biopharm Res2020; 12: 99–106.
29.
BrunnerEVandemeulebroeckeMM’utzeT. Win odds: An adaptatation of the win ratio to include ties. Stat Med2021; 40: 3367–3384.
30.
MannHWhitneyD. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat1947; 18: 50–60.
31.
SunXXuW. Fast implementation of DeLong’s algorithm for comparing the areas under correlated receiver operating characteristic curves. IEEE Signal Process Lett2014; 21: 1389–1393.
32.
HoeffdingW. A class of statistics with asymptotically normal distribution. Ann Math Stat1948; 19: 293–325.
33.
van BreukelenGJCandelMJBergerMP. Relative efficiency of unequal versus equal cluster sizes in cluster randomized and multicentre trials. Stat Med2007; 26: 2589–2603.
34.
WangBHarhayMSmallD, et al.On the mixed-model analysis of covariance in cluster-randomized trials. arXiv preprint 2021, ArXiv:2112.00832. Forthcoming in Statistical Science.
DeLongEDeLongDClarke-PearsonD. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics1988; 44: 837–845.
37.
SenP. On some convergence properties of U-statistics. Calcut Stat Assoc Bull1960; 10: 1–18.
38.
BrunnerEMunzelUPuriM. The multivariate nonparametric Behrens–Fisher problem. J Stat Plan Inference2002; 108: 37–53.
39.
RubarthKSattlerPZimmermannH, et al.Estimation and testing of Wilcoxon–Mann–Whitney effects in factorial clustered data designs. Symmetry2021; 14: 1–34.
40.
EldridgeSAshbyDFederG, et al.Lessons for cluster randomized trials in the twenty-first century: A systematic review of trials in primary care. Clin Trials2004; 1: 80–90.
41.
KahanBForbesGAliY, et al.Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: A review, reanalysis, and simulation study. Trials2016; 17: 1–8.
42.
IversNTaljaardMDixonS, et al.Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: Review of random sample of 300 trials, 2000-8. BMJ2011; 343: d5886.
43.
EldridgeSAshbyDKerryS. Sample size for cluster randomized trials: Effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol2006; 35: 1292–1300.
44.
AdamsGGullifordMUkoumunneO, et al.Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol2004; 57: 785–794.
45.
YangSMoerbeekMTaljaardM, et al.Power analysis for cluster randomized trials with continuous coprimary endpoints. Biometrics2023; 79: 1293–1305.
46.
WangJCaoJZhangS, et al.A flexible sample size solution for longitudinal and crossover cluster randomized trials with continuous outcomes. Contemp Clin Trials2021; 109: 106543.
47.
LiFTurnerEPreisserJ. Sample size determination for gee analyses of stepped wedge cluster randomized trials. Biometrics2018; 74: 1450–1458.
48.
WicklinR. Simulating Data with SAS. Cary, North Carolina: SAS Institute, 2013.
49.
XuJ. On the bias in the AUC variance estimate. Pattern Recognit Lett2024; 178: 62–68.
50.
YuC. Nonparametric Methods for Analysis and Sizing of Cluster Randomization Trials with Baseline Measurements. PhD Thesis, Western University, London, ON, CA, 2023. Electronic Thesis and Dissertation Repository. 9697. https://ir.lib.uwo.ca/etd/9697.
51.
ShuDZouG. Revisiting sample size planning for receiver operating characteristic studies: A confidence interval approach with precision and assurance. Stat Methods Med Res2023; 32: 748–759.
52.
RauchGJahn-EimermacherABrannathW, et al.Opportunities and challenges of combined effect measures based on prioritized outcomes. Stat Med2014; 33: 1104–1120.
