Retention of Children and Their Families in the Longitudinal Outcome Study of the Comprehensive Community Mental Health Services for Children and Their Families Program

Abstract

Predictors of retention of participants in a longitudinal study and heterogeneity between communities were investigated using a multilevel logistic regression model. Data from the longitudinal outcome study of the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families program and information on communities participating in the evaluation were used to examine retention of participants at 6-, 12-, 18-, and 24-month follow-up interviews. Findings revealed that both child and family characteristics and community characteristics significantly influenced participation for up to 24-month follow-up. Specifically, behavioral and emotional problems at intake, referral source, proportion of total staff devoted to interviewers, and total amount of incentive paid out at baseline influenced retention of participants. In addition, significant random effects at both Level 2 and Level 3 indicated that retention varied significantly between communities and among children even after controlling for child and family characteristics.

Keywords

retention in longitudinal study incentive multilevel model heterogeneity

Get full access to this article

View all access options for this article.

References

Achenbach

Rescorla

(2001). Manual for ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.

Allison

(2002). Missing data. Thousand Oaks, CA: Sage.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Bender

B. G.

Iklé

D. N.

DuHamel

Tinkelman

(1997). Retention of asthmatic patients in a longitudinal clinical trial. Journal of Allergy and Clinical Immunology, 99,197–203.

Campanelli

P. A.

O’Muircheartaigh

. (2002). The importance of experimental control in testing the impact of interviewer continuity on panel survey nonresponse. Quality and Quantity, 36, 129–144.

Center for Mental Health Services. (2004). The comprehensive community mental health services for children and their families program: Evaluation findings—Annual report to Congress, 2004. Atlanta, GA: Macro International Inc.

Collins

R. L.

Ellickson

P. L.

Hays

R. D.

Mccaffrey

D. F.

(2000). Effects of incentive size and timing on response rates to a follow-up wave of a longitudinal mailed survey. Evaluation Review, 24, 347. doi:10.1177/0193841X0002400401

de Graaf

Bijl

R. V.

Smit

Ravelli

Vollebergh

W. A. M

. (2000). Psychiatric and socioeconomic attrition in a longitudinal study: The Netherlands mental health survey and incidence study. American Journal of Epidemiology, 152, 1039–1047.

Dillman

(2000). Mail and Internet surveys: The tailored design method. New York, NY: John Wiley & Sons, Inc.

10.

Giusti

Little

R. J.

(2011). A sensitivity analysis of nonignorable nonresponse to income in a survey with a rotating panel design. Journal of Official Statistics, 27, 211–229.

11.

Goldstein

Browne

Rasbash

. (2002). Partitioning variation in generalized linear multilevel models. Understanding Statistics, 1, 223–32.

12.

Graham

J. W.

Donaldson

S. I.

(1993). Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow-up data. Journal of Applied Psychology, 78, 119–128.

13.

Groves

R. M.

Couper

M. P.

(1998). Nonresponse in household interview surveys. New York, NY: John Wiley and Sons.

14.

Harris

P. M.

(1998). Attrition revisited. American Journal of Evaluation, 19, 293–305.

15.

Hills

D. H.

Willis

R. J.

(2001). Reducing panel attrition: A search for effective policy instruments. The Journal of Human Resources, 36, 416–438.

16.

Howard

K. I.

Krause

M. S.

Orlinsky

D. E.

(1986). The attrition dilemma: Toward a new strategy for psychotherapy research. Journal of Consulting and Clinical Psychology, 54, 106–110.

17.

Hutt

L. E.

(2003). Paying research subjects: Historical considerations. Health Law Institute, 12, 16–21.

18.

Jäckle

Lynn

(2008) Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology, 34, 105–117.

19.

Larsen

Merlo

(2005). Appropriate assessment of neighborhood effects on individual health: Integrating random and fixed effects in multilevel logistic regression. American Journal of Epidemiology, 161, 81–88.

20.

Laurie

(2008). Minimizing panel attrition. In Menard

(Ed.), Handbook of longitudinal research: Design, measurement, and analsysis (pp. 167–184). Oxford, England: Elsevier, Inc.

21.

Laurie

Lynn

. (2008). The use of respondent incentives on longitudinal surveys. Institute of Social and Economic Research, University of Essex. Retrieved from http://www.iser.essex.ac.uk/publications/book/chapter/1669

22.

Laurie

Smith

Scott

(1999). Strategies for reducing nonresponse in a longitudinal panel survey. Journal of Official Statistics, 15, 269–282.

23.

Lepkowski

J. M.

Couper

M. P.

(2002). Nonresponse in the second wave of longitudinal household surveys. In Groves

R. M.

Dilman

D. A.

Eltinge

J. L.

Little

R. J.

(Eds.), Survey Nonresponse (pp. 259–273). New York, NY: John Wiley & Sons, Inc.

24.

Little

R. J. A

.(1988). A test of missing completely at random for multivariate data with missing values. Journal of American Statistical Association, 83, 1198–1202.

25.

Little

R. J. A.

Rubin

D. B.

(2002). Statistical analysis with missing data. New York, NY: John Wiley & Sons.

26.

Lynn

(2006). Attrition and non-response. Journal of the Royal Statistical Society Series A (Statistics in Society), 169, 393–394.

27.

Lynn

Buck

Burton

Jäckle

Laurie

. (2005). A review of methodological research pertinent to longitudinal survey design and data collection (Working papers of the Institute for Social and Economic Research (ISER), No.2005–29). Retrieved from http://www.iser.essex.ac.uk/publications/book/chapter/1669

28.

Manteuffel

Stephens

R. L.

Brashears

Krivelyova

Fisher

S. K.

(2008). Evaluation results and systems of care: A review. In Stroul

B. A.

Blau

G. M.

(Eds.), The system of care handbook: Transforming mental health services for children, youth, and families (pp. 25–69). Baltimore, MD: Paul H. Brookes Publishing Co., Inc.

29.

Merlo

Chaix

Ohlsson

Beckman

Johnell

Hjerpe

Rastam

Larsen

(2006). A brief conceptual tutorial of multilevel analysis is social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. Journal of Epidemiology and Community Health, 60, 290–297.

30.

Moineddin

Matheson

F. I.

Glazier

R. H.

(2007). A simulation study of sample size for multilevel logistic regression models. BMC Medical Research Methodology, 7, 1–10.

31.

Moser

D. K.

Dracup

Doering

L. V.

(2000). Factors differentiating dropouts from completers in a longitudinal, multicenter clinical trial. Nursing Research, 49, 109–116.

32.

Rabe-Hesketh

Skrondal

Pickles

. (October 2004). GLLAMM manual. (U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 160). Retrieved from http://www.bepress.com/ucbbiostat/paper160

33.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.

34.

Ribisl

K. M.

Walton

M. A.

Mowbray

C. T.

Luke

D. A.

Davidson II

W. S.

Bootsmiller

B. J

. (1996). Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: Review and recommendations. Evaluation and Program Planning, 19, 1–25.

35.

Rubin

D. B.

(1987). Multiple imputation for nonresponse in surveys. New York, NY: John Wiley & Sons.

36.

Ryan

E. A.

Hayman

L. L.

(1996). The role of the family coordinator in longitudinal research: Strategies to recruit and retain families. Journal of Family Nursing, 2, 325–335.

37.

Schafer

J. L.

(1997). Analysis of incomplete multivariate data. London, England: Chapman & Hall.

38.

Schafer

J. L.

(1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8, 3–15.

39.

Schafer

J. L.

Graham

J. W.

(2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177.

40.

Siddique

Harel

Crespi

C. M

. (2012). Addressing missing data mechanism uncertainty using multiple-model multiple imputation: Application to a longitudinal clinical trial. The Annals of Applied Statistics , 6, 1814–1837.

41.

Spybrook

Raudenbush

S. W.

Congdon

Martinez

. (2009). Optimal design for longitudinal and multilevel research: Documentation for the optimal design software (Version 2.0). Retrieved from www.wtgrantfoundation.org

42.

Stroul

B. A.

Friedman

(1994). A system of care for children and youth with severe emotional disturbances (Rev. ed.). Washington, DC: Georgetown University, Child Development Center, CASSP Technical Assistance Center.

43.

Sullivan

C. M.

Rumptz

M. H.

Campbell

Eby

K. K.

Davidson

W. S.

(1996). Retaining participants in longitudinal community research: A comprehensive protocol. Journal of Applied Behavioral Science, 32, 262–276.

44.

Vach

Blettner

(1991). Biased estimation of the odds ratio in case-control studies due to the use of ad hoc methods of correcting for missing values for confounding variables. American Journal of Epidemiology, 134, 895–907.

45.

Watson

Wooden

. (2006). Identifying factors affecting longitudinal survey response. Melbourne Institute of Applied Economic and Social Research. Retrieved from http://www.melbourneinstitute.com/hilda/Biblio/cp/Watson_Wooden_Identifying_Factors_AffectingLongitudinal_Survey_Response%20_MOLS.pdf

46.

Werthiemer

Miller

F. G.

(2008). Payment for research participation: A coercive offer? Journal of Medical Ethics, 34, 389–392.

47.

Wolke

Waylen

Samara

Steer

Goodman

Ford

Lambert

(2009). Selective drop-out in longitudinal studies and non-biased prediction of behavior disorders. The British Journal of Psychiatry, 195, 249–256.