Including all individuals is not enough: Lessons for intention-to-treat analysis

Abstract

Background

Intention-to-treat (ITT) analysis requires all randomised individuals to be included in the analysis in the groups to which they were randomised. However, there is confusion about how ITT analysis should be performed in the presence of missing outcome data.

Purposes

To explain, justify, and illustrate an ITT analysis strategy for randomised trials with incomplete outcome data.

Methods

We consider several methods of analysis and compare their underlying assumptions, plausibility, and numbers of individuals included. We illustrate the ITT analysis strategy using data from the UK700 trial in the management of severe mental illness.

Results

Depending on the assumptions made about the missing data, some methods of analysis that include all randomised individuals may be less valid than methods that do not include all randomised individuals. Furthermore, some methods of analysis that include all randomised individuals are essentially equivalent to methods that do not include all randomised individuals.

Limitations

This work assumes that the aim of analysis is to obtain an accurate estimate of the difference in outcome between randomised groups and not to obtain a conservative estimate with bias against the experimental intervention.

Conclusions

Clinical trials should employ an ITT analysis strategy, comprising a design that attempts to follow up all randomised individuals, a main analysis that is valid under a stated plausible assumption about the missing data, and sensitivity analyses that include all randomised individuals in order to explore the impact of departures from the assumption underlying the main analysis. Following this strategy recognises the extra uncertainty arising from missing outcomes and increases the incentive for researchers to minimise the extent of missing data.

Get full access to this article

View all access options for this article.

References

Newell

. Intention-to-treat analysis: Implications for quantitative and qualitative research. Int J Epidemiol 1992; 21: 837–41.

McKnight

Sidani

Figueredo

. Missing Data: A Gentle Introduction. The Guilford Press, New York, 2007.

PSI Missing Data Expert Group. Missing data: Discussion points from the PSI missing data expert group. Pharm Stat 2010; 9: 288–97.

National Research Council, Panel on Handling Missing Data in Clinical Trials, Committee on National Statistics, Division of Behavioral and Social Sciences and Education. The Prevention and Treatment of Missing Data in Clinical Trials. The National Academies Press, Washington, DC, 2010.

Little

RJA

Rubin

. Statistical Analysis with Missing Data (2nd edn). Wiley, Hoboken, NJ, 2002.

Altman

Schulz

Moher

et al. The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Ann Intern Med 2001; 134: 663–94.

Hollis

Campbell

. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 1999; 319: 670–74.

Committee for Proprietary Medicinal Products (CPMP). Points to consider on missing data, 2001. Available at: http://www.emea.europa.eu/pdfs/human/ewp/177699EN.pdf

Altman

. Missing outcomes in randomized trials: Addressing the dilemma. Open Med 2009; 3(2): e51–53.

10.

European Medicines Agency. Guideline on missing data in confirmatory clinical trials, 2010. Available at: http://www.ema.europa.eu/ema/pages/includes/document/open_document.jsp?webContentId=WC500096793

11.

Schulz

Altman

Moher

. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c332.

12.

White

Horton

Carpenter

Pocock

. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ 2011; 342: d40.

13.

White

. Uses and limitations of randomization-based efficacy estimators. Stat Methods Med Res 2005; 14: 327–47.

14.

Center for Drug Evaluation and Research (CDER). Guidance for industry: Antiretroviral drugs using plasma HIV RNA measurements – Clinical considerations for accelerated and traditional approval, 2002. Available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM070968.pdf (accessed 18 October 2010).

15.

Wood

White

Thompson

. Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin Trials 2004; 1: 368–76.

16.

Kenward

Molenberghs

. Last observation carried forward: A crystal ball? J Biopharm Stat 2009; 19: 872–88.

17.

Molenberghs

Thijs

Jansen

et al. Analyzing incomplete longitudinal clinical trial data. Biostatistics 2004; 5: 445–64.

18.

Lane

. Handling drop-out in longitudinal clinical trials: A comparison of the LOCF and MMRM approaches. Pharm Stat 2008; 7: 93–106.

19.

Carpenter

Kenward

Evans

White

. Re: Last observation carry-forward and last observation analysis (Shao J, Zhong B, Stat Med 2003; 22: 2429–41). Stat Med 2004; 23: 3241–42.

20.

Siddiqui

Hung

O’Neill

. MMRM vs. LOCF: A comprehensive comparison based on simulation study and 25 NDA datasets. J Biopharm Stat 2009; 19: 227–46.

21.

Heyting

Tolboom

JTBM

Essers

JGA

. Statistical handling of drop-outs in longitudinal clinical trials. Stat Med 1992; 11: 2043–61.

22.

Shao

Zhong

. Last observation carry-forward and last observation analysis. Stat Med 2003; 22: 2429–41.

23.

Sutton

Gilbert

. Effectiveness of individually tailored smoking cessation advice letters as an adjunct to telephone counselling and generic self-help materials: Randomized controlled trial. Addiction 2007; 102: 994–1000.

24.

Verbeke

Molenberghs

(ed.). Linear Mixed Models in Practice. Springer-Verlag, New York, 1997.

25.

Potthoff

Tudor

Pieper

Hasselblad

. Can one assess whether missing data are missing at random in medical studies? Stat Methods Med Res 2006; 15: 213–34.

26.

Rubin

. Multiple Imputation for Nonresponse in Surveys. John Wiley and Sons, New York, 1987.

27.

van Buuren

. Flexible Imputation of Missing Data. CRC Press, Boca Raton, FL, 2012.

28.

Royston

. Multiple imputation of missing values. Stata J 2004; 4: 227–41.

29.

Novo

Schafer

. Package NORM. R package version 1.0-9.2, 2010.

30.

van Buuren

Oudshoorn

CGM

. Multivariate imputation by chained equations: MICE V1.0 user’s manual. TNO Report PG/VGZ/00.038. TNO Preventie en Gezondheid, Leiden, 2000. Available at: http://www.multiple-imputation.com/

31.

SAS Institute Inc. SAS/STAT 9.1 User’s Guide (Ch. 46). SAS Institute Inc., Cary, NC, 2004.

32.

StataCorp. Stata Multiple-Imputation Reference Manual. Stata Press, College Station, TX, 2009.

33.

Robins

Wang

. Inference for imputation estimators. Biometrika 2000; 87(1): 113–24.

34.

Meng

. Multiple-imputation inferences with uncongenial sources of input. Stat Sci 1994; 9: 538–58.

35.

Royston

Carlin

White

. Multiple imputation of missing values: New features for mim. Stata J 2009; 9: 252–64.

36.

Carpenter

Kenward

. Missing Data in Clinical Trials – A Practical Guide. Publication RM03/JH17/MK. National Institute for Health Research, Birmingham, 2008. Available at: http://www.pcpoh.bham.ac.uk/publichealth/methodology/projects/RM03_JH17_MK.shtml

37.

White

Moodie

Thompson

Croudace

. A modelling strategy for the analysis of clinical trials with partly missing longitudinal data. Int J Methods Psychiatr Res 2003; 12: 139–50.

38.

Lachin

. Statistical considerations in the intent-to-treat principle. Control Clin Trials 2000; 21: 167–89.

39.

Lavori

. Clinical trials in psychiatry – Should protocol deviation censor patient data? Neuropsychopharmacology 1992; 6: 39–48.

40.

Lavori

Dawson

. Designing for intent-to-treat. Drug Inf J 2001; 35: 1079–86.

41.

Little

Yau

. Intent-to-treat analysis for longitudinal studies with dropout. Biometrics 1996; 52: 1324–33.

42.

Sterne

JAC

White

Carlin

. Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ 2009; 338: b2393.

43.

Sabin

Lepri

Phillips

. A practical guide to applying the intention-to-treat principle to clinical trials in HIV infection. HIV Clin Trials 2000; 1: 31–38.

44.

Shih

. Problems in dealing with missing data and informative censoring in clinical trials. Curr Control Trials Cardiovasc Med 2002; 3: 4.

45.

Kenward

Goetghebeur

EJT

Molenberghs

. Sensitivity analysis for incomplete categorical tables. Stat Model 2001; 1: 31–48.

46.

White

Carpenter

Evans

Schroter

. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clin Trials 2007; 4: 125–39.

47.

Rotnitzky

Robins

Scharfstein

. Semiparametric regression for repeated outcomes with nonignorable nonresponse. J Am Stat Assoc 1998; 93: 1321–39.

48.

Daniels

Hogan

. Reparameterizing the pattern mixture model for sensitivity analyses under informative dropout. Biometrics 2000; 56: 1241–48.

49.

Molenberghs

Kenward

Goetghebeur

. Sensitivity analysis for incomplete contingency tables: The Slovenian plebiscite case. J Roy Stat Soc C-App 2001; 50: 15–29.

50.

Matts

Launer

Nelson

Miller

Dain

. A graphical assessment of the potential impact of losses to follow-up on the validity of study results. Stat Med 1997; 16: 1943–54.

51.

Hollis

. A graphical sensitivity analysis for clinical trials with non-ignorable missing binary outcome. Stat Med 2002; 21: 3823–34.

52.

Rubin

Stern

Vehovar

. Handling ‘don’t know’ survey responses: The case of the Slovenian plebiscite. J Am Stat Assoc 1995; 90: 822–28.

53.

Liang

Zeger

. Longitudinal data analysis of continuous and discrete responses for pre-post designs. Sankhya Ser B 2000; 62: 134–48.

54.

Liu

Mogg

Mallick

Mehrotra

. Should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials? Stat Med 2009; 28: 2509–30.

55.

White

Thompson

. Adjusting for partially missing baseline measurements in randomised trials. Stat Med 2005; 24: 993–1007.

56.

Kenward

White

Horton

Carpenter

. Re: Should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials? (Liu GF et al., Stat Med 2009; 28: 2509–30). Stat Med 2010; 29: 1455–56.

57.

Riper

Kramer

Smit

et al. Web-based self-help for problem drinkers: A pragmatic randomized trial. Addiction 2008; 103: 218–27.

58.

Burns

Creed

Fahy

. Intensive versus standard case management for severe psychotic illness: A randomised trial. UK700 trial group. Lancet 1999; 353: 2185–89.

59.

Greenland

Finkle

. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol 1995; 142: 1255–64.

60.

White

Kalaitzaki

Thompson

. Allowing for missing outcome data and incomplete uptake of randomised interventions, with application to an Internet-based alcohol trial. Stat Med 2011; 30: 3192–207.

61.

Royston

. Multiple imputation of missing values: Update. Stata J 2005; 5: 188–201.

62.

Royston

. Multiple imputation of missing values: Further update of ice, with an emphasis on categorical variables. Stata J 2009; 9: 466–77.

63.

Wood

White

Hillsdon

Carpenter

. Comparison of imputation and modelling methods in the analysis of a physical activity trial with missing outcomes. Int J Epidemiol 2005; 34: 89–99.

64.

Zeger

Liang

Albert

. Models for longitudinal data: A generalized estimating equation approach. Biometrics 1988; 44: 1049–60.