Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research

Abstract

The objective of this research was to identify determinants of the magnitude of intracluster correlation coefficients (ICCs) in cluster randomized trials from the field of implementation research. A survey of experts was conducted to generate a priori hypotheses of factors that might affect ICC size. Hypotheses were tested on empirical estimates of ICCs calculated from 21 implementation research datasets, mainly from the UK. Effects of setting (primary or secondary care), type of variable (process or outcome), type of measurement (objective or subjective), prevalence of outcome and size of cluster were tested. In total, 220 ICCs were available (range 0 to 0.415). Significant differences in ICC magnitude were found. The ICCs were significantly higher for process than for outcome variables, and for secondary care outcomes compared with primary care outcomes. The effects of prevalence and size were less clear cut. There was no evidence to suggest that type of measurement affected ICC size. In conclusion, accurate estimates of ICCs are essential for sample size calculations for cluster randomized trials of professional behaviour change interventions. This study demonstrates that ICCs are sensitive to a number of trial factors, particularly setting and outcome type. These factors must be considered when planning such cluster randomized trials.

Get full access to this article

View all access options for this article.

References

1. Eccles M , Grimshaw JM , Campbell MK , Ramsay CR . Research designs for studies evaluating the effectiveness of change and improvement strategies. Quality and Safety in Health Care 2003; 12: 47–52.

2. Gail MH , Mark SD , Carroll RJ , Green SB , Pee D . On design considerations and randomization-based inference for community intervention trials. Stat Med 1996; 15: 1069–1092.

3. Donner A . Some aspects of the design and analysis of cluster randomization trials. Appl Stat 1998; 47: 95–113.

4. Ukoumunne OC , Gulliford MC , Chinn S , Sterne JAC , Burney PGJ . Methods for evaluating areawide and organization based interventions in health and health care: a systematic review. Health Technol Assess 1999; 3: iii–92

5. Murray DM . Design and analysis of group randomized trials. New York: Oxford University Press, 1998.

6. Donner A , Klar N . Design and analysis of cluster randomization trials in health research. London: Arnold; 2000.

7. Murray DM , Varnell SP , Blitstein JL . Design and analysis of group randomized trials: a review of recent methodological developments. Am J Public Health 2004; 94: 423–432.

8. Donner A , Birkett N , Buck C . Randomization by cluster. Sample size requirements and analysis. Am J Epidemiol 1981; 114: 906–914.

9. Campbell MK , Grimshaw JM , Steen N . Sample size calculations for cluster randomized trials. JHSRP 2000; 5: 12–16.

10.

10. Smeeth L , Ng E . Intraclass correlation coefficients for cluster randomized trials in primary care: data from the MRC trial of the assessment and management of older people in the community. Control Clin Trials 2002; 23: 409–421.

11.

11. Butler C , Bachmann M . Design and analysis of studies evaluating smoking cessation interventions where effects vary between practices or practitioners. Fam Prac 1996; 13: 402–407.

12.

12. Donner A . An empirical study of cluster randomization. Int J Epidemiol 1982; 11: 283–286.

13.

13. Siddiqui O , Hedeker D , Flay BR , Hu FB . Intraclass correlation estimates in a school-based smoking prevention study. Am J Epidemiol 1996; 144: 425–433.

14.

14. Gulliford MC , Ukuomunne OC , Chinn S . Components of variance and intraclass correlations for the design of community based surveys and intervention studies: data from the health survey for England 1994. Am J Epidemiol 1999; 149: 876–883.

15.

15. Murray DM , Blitstein JL . Methods to reduce the impact of intraclass correlation in group randomized trials. Eval Rev 2003; 27: 79–103.

16.

16. Mason J , Wood J , Freemantle N . Designing evaluations of interventions to change professional practice. JHSRP 1999; 4: 106–111.

17.

17. Donner A , Koval JJ . The estimation of intraclass correlation in the analysis of family data. Biometrics 1980; 36: 19–25.

18.

18. Donner A , Koval JJ . Design considerations in the estimation of intraclass correlation. Ann Hum Gen 1982; 46: 271–277.

19.

19. Rogers WH . Regression standard errors in clustered samples. Stata Technical Bulletin 1993; 3: 88–94.

20.

20. Williams RL . A note on robust variance estimation for cluster-correlated data. Biometrics 2000; 56: 645–646.

21.

21. NHS Cervical Screening Programme . Cervical Screening: a pocket guide. Sheffield: Health Education Authority, 2001.

22.

22. Adams G , Gulliford MC , Ukoumunne OC , Chinn S , Campbell MJ . Geographical and organizational variation in the structure of primary care services: implications for study design. JHSRP 2003; 8: 87–93.

23.

23. Ouellette JA , Wood W . Habit and intention in everyday life: the multiple processes by which past behaviour predicts future behavior. Psychol Bull 1998; 124: 54–74.

24.

24. Feng Z , Grizzle JE . Correlated binomial variates: properties of estimator of intraclass correlation and its effect on sample size calculation. Stat Med 1992; 11: 1607–1614.

25.

25. Hauck WW , Gilliss CL , Donner A , Gortner S . Randomization by cluster. Nursing Res 1991; 40(6): 356–358.

26.

26. Adams AS , Soumerai SB , Lomas J , Ross-Degnan D . Evidence of self-report bias in assessing adherence to guidelines. Int J Qual Health Care 1999; 11: 187–192.

27.

27. Eccles M , Ford G , Duggan S , Steen N . Are postal questionnaire surveys of reported activity valid: an exploration using general practitioner management of hypertension in older people. BJGP 1999; 49: 35–38.

28.

28. Turner RM , Omar RZ , Thompson SG . Using the intra-cluster correlation coefficient in Bayesian approaches for cluster randomized trials. Proceedings of the International Society of Clinical Biostatistics 2000: 49-49.

29.

29. Turner RM , Omar RZ , Thompson SG . Bayesian methods of analysis for cluster randomized trials with binary outcome data. Stat Med 2001; 20: 453–472.

30.

30. Kreft I , De Leeuw J . Introducing multilevel modelling. London: Sage, 1998.

31.

31. Duncan C , Jones K , Moon G . Context, composition and heterogeneity: using multilevel models in health research. Soc Sci Med 1998; 46: 97–117.

32.

32. Paterson L , Goldstein H . New statistical methods for analysing social structures: an introduction to multilevel models. Br Educ Res J 1991; 17: 387–393.