Does treatment effect depend on control event rate? Revisiting a meta-analysis of suicidality and antidepressant use in children

Abstract

Background Weisberg et al. (Clin Trials 2009, 6: 109—18) illustrate that entry criteria in randomized trials can lead to results that are biased for the target population. Presenting data from a previously published meta-analysis of the effect of antidepressants in children on suicidality, they suggest a relationship between the control event rate and relative risk.

Purpose Our purpose is threefold: (1) to show that natural methods of demonstrating a relationship between the control event rate and relative treatment benefit are fraught with danger, (2) to develop better methods, and (3) to assess whether there is evidence of such a relationship in the suicidality meta-analysis. Methods We propose an improved graphical method and an exact and approximate test of whether the treatment effect increases (or decreases) with the control event rate.

Results The apparent relationship in the naive plot of relative risk against control rate is actually no stronger than what would be expected by chance. Results of our test do not support the conclusion that the odds ratio for suicidality varies with the control rate (one-sided p = 0.39).

Limitations We are not able to apply the exact test to this data set. Simulation results indicate that the relationship would have to be very strong to detect it with these sample sizes and control event rates.

Conclusions The difficulty in showing that the treatment effect in clinical trials differs by control rate is caused by 3 factors: (1) artificial correlation between a relative risk and control rate, (2) regression dilution bias because the independent variable — the control proportion — is subject to random variability, and (3) low power because we are trying to detect an interaction. Our graph and test are useful tools. Using them, we found no support for a relationship between the control event rate and treatment effect on suicidality. Clinical Trials 2010; 7: 109—117. http://ctj.sagepub.com

Get full access to this article

View all access options for this article.

References

Weisberg HI , Hayden VC , Pontes VP

Selection criteria and generalizability within the counterfactual framework: explaining the paradox of antidepressant-induced suicidality?

Clin Trials 2009; 6: 109-18.

Hammad TA , Lauphren T. , Racoosin J. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychia 2006; 63: 332-9.

Senn S. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials (letter to the editor). Stat Med 1994; 13: 293-6.

Sharp SJ , Thompson SG , Altman DG The relation between treatment benefit and underlying risk in meta-analysis . Brit Med J 1996; 313: 735-8.

Schmid CH , Lau J. , McIntosh MW , Cappelleri JC An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med 1998; 17: 1923-42.

McIntosh M. The population risk as an exploratory variable in research synthesis of clinical trials. Stat Med 1996; 15: 1713-28.

Sharp SJ , Thompson SG Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. Stat Med 2000; 19: 3251-74.

Greenland S. , Robins JM Estimation of a common effect parameter from sparse follow-up data. Biometrics 1985; 41: 55-68.

L’Abbé K. , Detsky A. , O’Rourke K. Meta-analysis in clinical research. Ann Intern Med 1987; 107: 224-33.

10.

Carroll RJ , Ruppert D. , Stefanski LA , Crainiceanu CM Measurement Error in Nonlinear Models: A Modern Perspective, (2nd edn). Chapman and Hall/CRC, New York, 2006.

11.

Rücker G. , Schwarzerl G. , Carpenter J. , Olkin I. Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta-analysis with zero cells. Stat Med 2009; 28: 721-38.

12.

Gart JJ , Zweifel JR On the bias of various estimators of the logit and its variance with application to quantal bioassay. Biometrika 1967; 54: 181-7.

13.

Breslow N. Odds ratio estimators when the data are sparse. Biometrika 1981; 68: 73-84.

14.

Gart JJ , Pettigrew HM , Thomas DG The effect of bias, variance estimation, skewness and kurtosis of the empirical logit on weighted least squares analyses. Biometrika 1985; 72: 179-90.

15.

Sweeting MJ , Sutton AJ , Lambert PC What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 2004; 23: 1351-75.

16.

Bradburn MJ , Deeks JJ , Berline JA , Localio AR Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007; 26: 53-77.

17.

Fleiss JL , Levin B. , Paik MC Statistical Methods for Rates and Proportions, (3rd edn). John Wiley & Sons, New York, 2003.

18.

Fay MP , Kim HJ , Haschey M. On using truncated sequential probability ratio test boundaries for Monte Carlo implementation of hypothesis tests. J Comput Graph Stat 2007; 16: 946-67.

19.

Yusuf S. , Peto R. , Lewis J. , et al. Beta blockade during and after myocardial infarction: an overview of the randomized trials. Prog Cardiovasc Dis 1985; 27: 335-71.

20.

Gelman A. , Shor B. , Bafumi J. , Park D. Rich state , poor state, red state blue state: what’s the matter with Connecticut? Q J Polit Sci 2007; 2: 345-67.