Statistical issues in multisite effectiveness trials: the case of brief strategic family therapy for adolescent drug abuse treatment

Abstract

The statistical development of the multisite Brief Strategic Family Therapy (BSFT) Trial of the National Institute on Drug Abuse's Clinical Trials Network provides a useful, real example of how an effectiveness trial can differ from an efficacy trial. In particular, two design elements distinguish this effectiveness trial from an efficacy trial. First, because the goal of the trial is to show that the use of BSFT would be an improvement on current practice, it was decided to compare BSFT to treatment as usual at each location. This decision ensures that the trial has the most ecological validity to the participating community treatment providers. Second, the desire to generalize the results to general clinical practice dictates that variability (in effect) across community treatment providers be estimated using a random effects model. These two decisions jointly influence the sample size calculations. Allowing variation in treatment as usual, will increase the variability in effect sizes across sites and estimation of this variability as a random effect necessitates a larger sample size (both number of community treatment providers and participants per community treatment provider), than is the case for a fixed site effect estimate. Details of these effects and their implications for the statistical design are presented.

Get full access to this article

View all access options for this article.

References

1. Johnston LD , O'Malley PM , Bachman JG . National Institutes of Health. Monitoring the future: national results on adolescent drug use. Overview of key findings 2002 (NIDA: NIH Publication No. 03-5374). Bethesda, MD: National Institutes of Drug Abuse, 2003.

2. Substance Abuse and Mental Health Services Administration (SAMHSA) . Mid-year 2000 preliminary emergency department data from the drug abuse warning network. (DHHS Publication No. 01-3502). Washington, DC: U.S. Government Printing Office, 2001.

3. Kazdin AE . Psychotherapy for children and adolescents. In Bergin AE , Garfield SL eds. Handbook of psychotherapy and behavior change. New York: Wiley & Sons, 1994: 543–594.

4. Liddle HA , Dakof GA . Family based treatment for adolescent drug use: state of the science. In Rahdert E et al. eds. Adolescent drug abuse: clinical assessment and therapeutic interventions [NIDA Research Monograph #156, NIH Publication 95-3908]. Rockville, MD: National Institute on Drug Abuse, 1995.

5. Stanton MD , Shadish WR . Outcome, attrition, and family-couples treatment for drug abuse: a meta-analysis and review of the controlled, comparative studies. Psychol Bull 1997; 122: 170–191.

6. Robbins MS , Szapocznik J , Santisteban DA , Hervis O , Mitrani VB , Schwartz, SJ . Brief strategic family therapy for hispanic youth. In Kazdin AE , Weisz JR eds. Evidence-based psychotherapies for children and adolescents. New York: Guilford, 2003.

7. Szapocznik J , Robbins MS , Mitrani VB , Santisteban D , Hervis O , Williams RA . Brief strategic family therapy. In Kaslow F ed. Comprehensive handbook of psychotherapy: Volume 4. New York: Wiley, 2002.

8. Szapocznik J , Kurtiness WM . Breakthroughs in family therapy with drug abusing problem youth. New York: Springer Publishing Company, 1989.

9. Szapocznik J , Hervis O , Schwartz SJ . Brief strategic family therapy for adolescent drug abuse (NIDA Treatment Manuals Series). Rockville, MD: National Institute on Drug Abuse, 2003.

10.

10. Flay BR . Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med 1986; 15: 451–474.

11.

11. Robbins MS , Bachrach K , Szapocznik J. Bridging the research–practice gap in adolescent substance abuse treatment: the case of brief strategic family therapy. J Subst Abuse Treat 2002; 23: 123–132.

12.

12. Clarke GN . Improving the transition from basic efficacy research to effectiveness studies: methodological issues and procedures. J Consult Clin Psychol 1995; 63: 718–725.

13.

13. Henggeler SW , Melton GB , Brondino MJ , Scherer DG . Multisystemic therapy with violent and chronic juvenile offenders and their families: the role of treatment fidelity in successful dissemination. J Consult Clin Psychol 1997; 65: 821–833.

14.

14. Weiss B , Weisz JR . Assessing the effects of clinic-based psychotherapy with children and adolescents. J Consult Clin Psychol 1989; 57: 741–746.

15.

15. Kazdin AE . Comparative outcome studies of psychotherapy: methodological issues and strategies. J Consult Clin Psychol 1986; 54: 95–105.

16.

16. Brown H , Prescott R. Applied Mixed Models in Medicine. London: John Wiley & Sons, 1999.

17.

17. Hocking RR . Methods and applications of linear models: regression and the analysis of variance. Hoboken, NJ: John Wiley & Sons, 2003.

18.

18. Raudenbush SW , Liu X. Statistical power and optimal design for multisite randomized trials. Psychol Methods 2000; 5: 199–213.

19.

19. Mikulich SK , Zerbe GO , Feaster DJ . Some ramifications of treating “site” as random in multi-center clinical trials. Controll Clin Trials 2003; 24: 43S–2405.

20.

20. Snijders T , Bosker, R. Multilevel analysis: an introduction to basic and advanced multilevel modeling. London, Sage Publications, 1999.

21.

21. Raudenbush SW , Bryk AS . Hierarchical linear models: applications and data analysis methods. London: Sage Publications, 2002.

22.

22. Fattinger KE , Sheiner LB , Verotta D. A new method to explore the distribution of interindividual random effects in non-linear mixed effects models. Biometrics 1995; 51: 1236–1251.

23.

23. Muthén LK , Muthén BO . Mplus user's guide version 3. Los Angeles, CA: Muthén & Muthén, 1998–2004.

24.

24. Cohen J. Statistical power analyses for the behavioral sciences. Hillsdale, NJ: Lawerence Erlbaum Associates, 1988.

25.

25. Hedeker D , Gibbons RD , Waternaux C. Sample size estimation for longitudinal designs with attrition: comparing time-related contrasts between two groups. J Educ Behav Stat 1999; 24: 70–93.

26.

26. Muthén LK , Muthén BO . How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling 2002; 4: 599–620.