Abstract
Introduction/Objectives:
This paper reports on participant retention from an ongoing prospective, multi-site cohort caries risk study involving parent/infant pairs. The objectives were to: (1) compare the retention rates at each intermediate contact (every 4 months) and dental visit (every 18 months) across the 3 clinical sites, (2) assess primary caregivers’ perceptions at the end of the study about the retention efforts used in this longitudinal study, and (3) determine whether primary caregiver baseline demographic characteristics and child’s baseline caries experience were associated with retention.
Methods:
1325 primary caregiver-child pairs recruited at the child’s first birthday were followed for 36 months at 3 sites. Dental visits occurred at children’s ages of approximately 12, 30, and 48 months. Telephone/email intermediate contacts with the primary caregiver occurred 6 times between dental visits. The outcome variable was the retention rates at each dental visit and each intermediate contact. Primary caregivers’ perceptions of intermediate contacts were evaluated. Retention rates were compared by maternal age, race, ethnicity, Medicaid status, yearly household income, baseline caries experience (defined as decayed, missing due to caries, or filled tooth surfaces) at 12 months, and the number of teeth erupted.
Results:
1325 primary caregiver/infant pairs were enrolled and completed the first in-person dental visit, 1062 pairs (80%) completed the second visit and 985 (74%) completed the third. Most primary caregivers were female (94%), with a mean age of 29 years and 667 (50%) self-identified as White, 544 (41%) as Black, and 146 (11%) as Hispanic. The percentages of successful intermediate contacts were 95% at 4 months decreasing to 82% at 34 months. Almost all 964 (98%) of 985 primary caregivers reported at the last visit that they were comfortable/very comfortable with 4-month intermediate contacts. The multivariable analysis showed that primary caregivers who were older (OR = 1.07; 95% CI, 1.04-1.09) and White (OR = 1.52; 95% CI, 1.12-2.06) were more likely to complete the study.
Conclusions:
Retention strategies were focused on frequent routine contact and increasing monetary incentives. Those strategies may have resulted in retention exceeding the proposed goals. At the end of the study, primary caregivers were comfortable with the 4-month intermediate contacts.
Introduction
In longitudinal studies, after participant recruitment, retention is a key aspect of maintaining the integrity of the study.1-3 Participant retention is the continuous engagement of the participant in a study. 4 If participant retention is not achieved, the study can be jeopardized by not having enough statistical power or having reduced study validity.3,5 In the planning and budgeting phases of a research proposal, participant retention needs to be considered as a critical component. 6 This paper reports on participant retention from an ongoing prospective multi-site study of dental caries risk involving parent/infant pairs.7,8
Participant retention varies across studies in the literature, with ranges reported between 0% and 79%. 9 Acceptable retention rates often are deemed adequate within the 70% to 80% range. While retention of 95% is not likely to introduce bias, retention less than 80% raises concerns about the study validity, if this is not accounted for in the sample size calculations or there is non-random attrition.10,11 Studies that include both parents and childen as subjects are more complicated, as study procedures need to account for different factors for the adults and children. A multi-site study that attributed its parent/child retention success to program structure achieved a 97% retention rate at 1 site for a 5-year period, with the other 3 sites’ retention rates ranging from 84% to 95%. 12 Factors to which they attributed their success included use of dedicated, consistent study staff over time at each site, childcare provided for the family, and other tangible supports, such as birthday cards and presents. 12
In a systematic review of 28 longitudinal population-based cohort studies, retention strategies were reviewed and categorized as cash or gift incentives, reminder methods (repeat visits or repeat questionnaires), and other methods. 13 Findings indicated that incentives were associated with higher retention rates and that, the better the incentive, the higher the retention rates. 13 Maintenance of updated contact information is another strategy that has been shown to be beneficial in maintaining high retention.14,15
Literature on retention of dental subjects longitudinally with primary caregiver/child pair participants is sparse. In order to contribute to the retention literature associated with dental studies in young children, the objectives of this paper were to: (1) compare the retention rates at each intermediate contact (every 4 months) and each dental visit (every 18 months) across the 3 clinical sites, (2) assess primary caregivers’ perceptions at the end of the study about the retention efforts used in this longitudinal study, and (3) determine whether primary caregiver baseline demographic characteristics and child’s baseline caries experience were associated with retention.
Methods
This report used data collected from a study to develop and validate a dental caries risk tool for use in primary healthcare settings for practitioners to easily identify children at high risk for caries. 8 This longitudinal study was a prospective cohort, 3-year, multi-site study managed and coordinated at the University of Michigan (Ann Arbor, MI) Data and Clinical Coordinating Center, which provided overall study administration (Clinical Trial Number: NCT01707797). The coordinating center arranged that site data could be uploaded to their secure website within 5 days of collection and provided monthly feedback to all sites regarding retention.
Site Selection/Training/Calibration
The universities conducting participant recruitment and retention for this project were Duke University (Duke) in Durham, North Carolina; Indiana University (IU) in Indianapolis, Indiana; and the University of Iowa (UI) in Iowa City, Iowa. Each clinical site was selected because of its unique connection with a medical primary care practice-based research network, which facilitated participant recruitment. Institutional Review Board approval was obtained at all 4 universities.
Prior to study recruitment, 7 project staff from the 3 sites convened with the coordinating center staff for 3 days. The study teams were oriented to the study, protocol details were reviewed, and training and calibration of dental examiners was conducted. Two examiners at each site were calibrated with the principal investigator as the gold standard. Additional calibrations of dental examiners were held prior to each of the second and third dental examinations. Calibrations were conducted on children with multiple carious lesions. Examiners were trained annually. Minimum intra-and inter-examiner agreement was established as kappa >.75 for identification of cavitated lesions ICDAS >5, and kappa >.55 for identification of ICDAS >3. All examiners met and significantly exceeded these calibration thresholds. Caries was scored using the International Caries Detection and Assessment System (ICDAS) criteria as the number of tooth surfaces that were decayed (non-cavitated level with distinct visible changes in enamel or more involved), filled, or missing due to caries (dmfs). 16
In this study, the young child participants received 3 dental screening examinations and primary caregivers completed questionnaires. The dental examinations and questionnaires were at baseline when the infant was 12 months of age (±3 months) and at 2 follow-up times when the children were ages 30 and 48 months. See Figure 1 for a depiction of the study sequence and age progression of the study. Two different timeframes are being reported—the age progression of the child and the timetable of the study itself.
Timeframe of study dental visits and intermediate contacts by children’s age.
Participant Inclusion Criteria
Each primary caregiver was the individual primarily responsible for the child’s health, housing, and safety and was the parent or legal guardian. Inclusion criteria for the primary caregiver were to be at least 18 years of age or older or an emancipated minor, able to read and speak English and/or Spanish, provide written informed consent for themselves and the child, complete the study questionnaires, allow 3 dental examinations of the child over 3 years, and respond to intermediate contacts between examination visits. Inclusion criteria for the child was to be 9 to 15 months of age at the time of the first in-person dental examination, be generally healthy, allow examination of the oral cavity, not be in foster care, not require antibiotic and/or sedative medication prior to dental procedure, and not have a history of uncontrolled epilepsy, undergoing cancer therapy, or having an unrepaired congenital heart defect.
Study sites were chosen so that the overall population recruited in this longitudinal cohort study would be diverse in terms of Medicaid status, race and ethnicity, and rural/urban residence. The Medicaid population sometimes is considered to be a high risk population, so non-Medicaid families were recruited to include more subjects not considered high risk.
Retention goals
Retention goals were established during study planning for the multisite cohort study. Planning showed that 1326 pairs of primary caregivers and infants would need to be enrolled across the 3 sites. Attrition rates between in-person examination periods (ie, over a period of 18 months each) were projected to be 25% in Indiana and North Carolina, and 30% in Iowa (due to the rural population and based on the Iowa Fluoride Study).17,18 Attrition was tracked throughout the study via a study termination log which monitored screen failures, early terminations, and withdrawals. Reasons and timings for those events were documented.
Primary Caregiver Questionnaire Development
The primary caregiver questionnaire was a self-administered 52-item questionnaire completed by the participants at each of the 3 dental visits. Thirty-two items were about the child, 18 about the parent, and 2 about the parent’s perceptions of their provision of dental and health care for their child. This instrument was pilot-tested starting with 143 items and pared down to 52 after redundant, invariant, and independent items were eliminated. 19 The 52-item questionnaire was then reworded in think aloud sessions with 5 respondents matching the target population and later administered to 125 primary caregivers where items were reworded for clarity. 8 The questionnaire has favorable construct and criterion validity for the 1325 children participants. 8 The goal of the parent study is the validation of this instrument.
From the baseline visit, primary caregiver’s date of birth, race, ethnicity, household income, Medicaid coverage, children’s number of dmfs, number of teeth erupted, and completed/not completed dental visit were used for data analysis. Race choices were White/Caucasian, African American or Black, Asian, Native Hawaiian or Pacific Islander, American Indian or Alaskan Native, and other (specify). Ethnicity was either Spanish/Hispanic/Latino or not. Household income was a categorical variable with the following categories: less than $5000, $5000-$9999, $10 000-19 999, $20 000-29 999, $30 000-39 999, $40 000-49 999, $50 000-59 999, $60 000-69 999, $70 000-79 999, $80 000-99 999, $100 000 or more, and don’t know. Medicaid health insurance coverage was assessed as yes/no. From the child’s dental examination, the tooth information was determined.
Retention Interventions
To help enhance retention, participants were asked at the baseline dental visit to provide their address, email (if they had one), and 3 possible telephone numbers of theirs (eg, home, work, and cellular), as well as their preferred method of contact. In addition, they were asked to provide names/telephone numbers, addresses, and email, as well as the preferred method of contact (including text messages), for 3 individuals who would know their whereabouts should loss of contact occur with them. It was considered loss to follow-up when a telephone number was dialed and a recording said, “this number is no longer in service” or someone answered the telephone and said, “this is my new telephone number.” After no response from telephone and email contact, a letter was sent to the participant and, if the address had changed, the letter was returned to the sender. Loss to follow-up could also have been due to the participant choosing not to respond to any of our contact efforts. Contact information was collected at each of the 3 dental visits.
Intermediate contacts with the primary caregiver occurred approximately every 4 months between dental visits (by phone, email, or mail) and twice by mail between dental visits to help enhance retention (See Figure 1). After the baseline visit when the children were 12 months of age, intermediate contacts were made at follow-up times of 4, 8, 12, and 16 months, and the first follow-up dental exam and questionnaire were done when the children were approximately 30 months of age. Intermediate contacts continued after the second in-person visit at follow-up times of 22, 26, 30, and 34 months, with the third in-person dental exam and questionnaire when the children were approximately 48 months of age.
All primary caregivers who consented to the study were entered into a tracking database designed by each site. During the intermediate contact telephone calls, the research assistants asked questions regarding any changes in this child’s primary caregiver or current contact information, if their child visited the dentist, if fluoride varnish was applied at the dental visit, and if a non-dental (medical) visit occurred was fluoride varnish applied since the previous contact. The method for intermediate contact was by telephone first, then email, followed by a letter, with up to 3 attempts total to reach the primary caregiver. For each successful intermediate contact where the primary caregiver provided answers, the primary caregiver was sent $10.
The primary caregiver was sent a letter reminding them of their next study appointment time about 2 weeks prior to the scheduled visit. In addition, a postcard was sent annually for the child’s birthday. The appointment letter and birthday card were considered contacts; however, no compensation was provided. Primary caregivers received $50 compensation for completion of the first dental visit, $60 for the second, and $75 for the third. Thus, in total, each caregiver had the potential to receive $265 for 3 dental visits and 8 intermediate contacts over the 36-month time-period, with increasing remuneration over time. At all 3 sites, parking was provided at no cost to primary caregivers during their dental visits. All 3 sites gave the child a toothbrush, sticker, and/or book after the visit.
Primary Caregiver Perceptions of Retention Efforts
At the third dental visit, primary caregivers completed a new form, the Intermediate Contact Review. The form included 3 questions: (1) Comfort level with being contacted every 4 months, with the following foils: uncomfortable, somewhat uncomfortable, neutral, comfortable, and very comfortable; (2) Select all responses that reflect your level of comfort with the contacts: I expected the communication, I appreciated the reminders, It did not take much time, I enjoyed the study participation, It took too much time, Payment was not high enough, and Other; and (3) Rank the top 3 ways we could have contacted you in order of preference, with the following choices: phone, email, letter, text message, Facebook or other social media, and other.
Data Analysis
Preliminary descriptive analyses were completed to assess the distributions of responses for all key variables including sex, age, race, ethnicity, Medicaid, annual household income, the number of dmfs, and teeth erupted. One-way analysis of variance was used to assess relationships with variables with continuous responses. The Pearson chi-square test was used to assess relationships with variables with discrete responses. Retention rates for the in-person dental visits were reported as percentages of participants who completed the visit compared to the start of the study. Retention rates at the intermediate contacts were calculated similarly. At the end of the study, retention was calculated 2 ways, to include both (1) participants who completed visits 1, 2, and 3 and (2) those who completed visits 1 and 3. This was done because some participants skipped the second visit, but then came to the third visit, so they were included as finishing the study.
A generalized linear mixed model using the SAS GLIMMIX procedure was used to estimate a binomial model predicting whether participants were retained at the end of the study. The site variable (Duke, IU, or UI) was specified as the random intercept in the model to account for the correlation among participants within each site. An unconditional model with only a random intercept, but no predictors, was used to estimate variability among sites. First, each of the potential associated factors was tested separately with the outcome variable in the model. Variables tested included 1325 primary caregiver’s age, race, ethnicity, household income, Medicaid coverage, dmfs count, and number of teeth at baseline. Variables with P < .15 in the univariate predictor models were included in the multivariable analyses, with P < .05 considered statistically significant. Household income and Medicaid status were highly correlated, so only Medicaid status was included in the final model. All analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC).
Results
Across the 3 study sites, 1325 primary caregiver/infant pairs were enrolled at baseline, with 434 at Duke, 543 at IU, and 348 at UI. One thousand and sixty-two pairs (80%) completed the second visit and 985 (74%) completed the third visit. Most primary caregivers were female (94%), with a mean age of 29 years and range of 18 to 56 years (See Table 1). Six hundred and sixty-seven (50%) of the primary caregivers self-identified as White, 544 (41%) as Black, 31 (2%) as Asian, 9 (1%) as American Indian, and 74 (6%) as other; 146 (11%) were Hispanic. Eight hundred and eleven (61%) primary caregivers were enrolled in Medicaid. Children were 49% female, with a mean age of 11.4 months at baseline.
Baseline Demographic Characteristics of Primary Caregiver Participants by Site (n = 1325).
Overall, the percentage of completed intermediate contacts decreased from 95% after 4 months, when the children were 16 months of age, to 89% after 16 months, when the children were 28 months of age (See Table 2). The second in-person visit, when the child was approximately 30 months of age, was completed by 1062 (80%) of the participants, ranging from 74% at IU to 88% at UI. Overall, the completed percentage of intermediate contacts was stable, varying from 81% to 88% (See Table 2). The third in-person visit, at age 48 months, was completed by 985 (74%) of the participants, ranging from 71% at IU to 79% at UI. UI had the highest retention rate at 88% at dental visit 2 and 79% at dental visit 3, followed by Duke with 81% and 74%, respectively, and IU with 74% and 71%, respectively, but the rates were not significantly different by site.
Participants Completing Each of the 3 Visits and Intermediate Contacts.
The window for all 3 dental visits was ±3 months, thus the children could be younger or older than 12, 30, or 48 months by up to 3 months.
Of the final 985 responding primary caregivers, 964 (97.9%) reported at the third visit that they were comfortable/very comfortable with the 4-month intermediate contacts. Only 21 (2.1%) of respondents were neutral or uncomfortable with the contacts. Six hundred and ninety-two (70%) expected to receive a communication, 865 (88%) appreciated the reminders, 805 (82%) reported the contacts did not take much time, and 793 (81%) enjoyed study participation. Only 3 (0.3%) reported the contact duration was lengthy. Eleven (1%) respondents reported that reimbursement was too low. Participants ranked their preferred methods of contact. Most, 601 (61%) of the primary caregivers preferred contact by telephone, followed by email for 241 (24%), text for 134 (14%), and letter for 9 (1%) participants (See Table 3).
Primary Caregivers’ Perceptions of Retention Efforts.
At the second dental visit, when children were age 30 months, 263 (20%) participants were lost to follow-up. At the third dental visit, when the children were age 48 months, 340 (26%) participants did not complete the third dental visit. Overall attrition was 26% for all sites, with 985 of the 1325 subjects remaining at the end of the study.
Those caregivers retained in the study had a mean age of 29.4 years versus 26.7 for those lost to follow-up (P < .001). Five hundred and fifty-eight (57%) of the 985 retained in the study were enrolled in Medicaid compared to 253 (74%) among those lost to follow-up (P < .001). Five hundred and forty (55%) of those retained in the study were White, compared to 127 (37%) among those lost to follow-up (P < .001). Four hundred and eighteen (42%) of those retained reported annual income greater than or equal to $40 000 compared with 90 (26%) of those lost to follow-up (P < .001).
In the unconditional GLIMMIX procedure with only random intercept, the variability in retention rates across sites was not statistically significant. In the generalized linear mixed modeling with univariate predictors, participants who were retained at the end of the study were older and more likely to be White, with higher household income, and not on Medicaid (See Table 4). The baseline dental caries experience of the child, number of erupted teeth, and ethnicity were not related to retention at the end of the study. The multivariable analysis showed that primary caregivers who were older were more likely to complete the study (see Table 4). The odds ratio for primary caregiver age was 1.07 (95% CI, 1.04-1.09), which means for a 1-year increase in primary caregiver age, we expect to see the odds of completing the study increase by 7%. Primary caregivers who were White were more likely to complete the study compared to primary caregivers who were Black (OR = 1.52; 95% CI, 1.12-2.06), indicating that the odds of White primary caregivers completing the study were 1.52 times the odds of Black primary caregivers. Medicaid coverage was not significantly related to retention in the study in the multivariable analyses.
Generalized Linear Mixed Model of Factors Associated With Retention at the Conclusion of the Study (n = 1325).
Household income and Medicaid status were highly correlated, so only Medicaid status was considered in the multivariable model.
The odds ratio was for a 1 unit increase in each of the factors.
Discussion
Participant retention was planned for during the proposal writing and implementation planning phases of this longitudinal study, using increasing monetary reimbursement, updating of contact logs, and frequent contacts every 4 months. At each time-period, each of the 3 sites retained more participants than planned in the original study proposal. Those participants who stayed in the study were significantly different from those who left the study. Younger and Black participants were more likely not to finish the study. The finding with Blacks is consistent with other research where minorities have been found to be lost to follow-up at higher rates.20-22
Overall, the retention rate was 74% at the third visit, which exceeded the study goal of 55%. This compares favorably to the retention rates of 75% at 6 months and 64% at 12 months in another longitudinal study conducted with a low-income, ethnically-diverse, urban population of mothers, where there were no extra site visits and only routine physician visits for nutritional anticipatory guidance during early childhood. 23 However, our retention rate fell short of the 97% achieved in the multi-site study at 1 site over 5 years who were following lead-exposed children. 12 In another study, retention rates varied at 12 months from 53% to 92% across 4 sites, while retention ranged from 54% to 85% at 36 months for 3 sites. 1 The study used similar strategies to our study, but also involved the community in development of retention strategies. 1
In longitudinal studies, attrition is a reality. 10 Reasons for attrition in this dental study were mainly 2, investigator-determined exclusion and loss to follow-up, the latter which is often unavoidable. 24 Investigator-determined exclusion was mostly due to changes in primary caregiver, such as foster care or primary caregiver being jailed, as the new primary caregivers had not consented to continue participation. Loss to follow-up occurred in part because the contact information for the primary caregiver and their respective contacts was inaccurate, even though home, cell, and work telephone numbers, addresses, and email addresses were collected at each of the dental visits. A potential intervention in future studies would be to provide a pre-paid study cell phone for each participant to retain throughout the study.
Being white and an older primary caregiver were independent predictors for retention in our study. Both the Duke and IU sites had more Black participants than the UI site, while the UI site’s participants were older than the Duke and IU participants, which may explain the consistently higher retention rates at UI than the other 2 sites. Medicaid status was controlled for in the final multivariable model because it is an important indicator of primary caregivers’ socioeconomic status. Other factors among the 3 sites which were not assessed that could have influenced the retention tendency. It is more important than ever to find ways to encourage the participation of underrepresented minorities in studies on dental caries, given the lower retention rate in our study and high prevalence of caries among those with low socioeconomic status and minority groups.25,26 Over-sampling of minority populations could be warranted in studies where it is important that minorities be retained, which would compensate for lack of retention to ensure enough participants remain in the study. In addition to over-sampling for retention of minorities, offering child care services, home visits, and assistance with transportation could be other effective methods for retention. 9 Our dental study used strategies across the 4 retention strategy domains (barrier-reduction, community-building, improvement of follow-up rates with each wave, and tracing strategies) suggested by Teague et al, 9 but additional strategies are warranted to focus on the subjects who were lost to follow.
In our study, 61% of the participants had low income, less than $40 000 annually, and the relatively small amount of remuneration for telephone calls and study visits may have been helpful for basic needs. In a systematic review and meta-analysis of retention strategies in longitudinal studies with participants’ average age of 30 years, cohorts with more female participants reported higher retentions rates than those studies with more male participants. 9 Primary caregivers were predominantly the children’s mothers and thus female, which may have helped with our retention. 9
Strengths and Limitations
Several study limitations should be considered. Participants were a convenience sample and not a representative sample. The educational level of the primary caregivers was not assessed at the baseline or second dental visits and was collected only at the time of the third in-person visit. Thus, that variable was not considered in the model, and it is unknown if it would have had an impact.
Inclusion/exclusion criteria were the same at each site. Minorities were targeted specifically at Duke and IU, but not at UI, which had a rural population target and lower statewide minority counts. Studying and comparing the participant retention at each site was not a planned outcome for the parent study. If a participant was not reached at 1 intermediate contact, they were not terminated from the study. They were retained and the next intermediate contact was attempted, in an ongoing effort to retain participants.
Only participants remaining in the study at the third dental visit were asked their perceptions regarding retention strategies. This could have led to bias and less informative findings than if all had been asked at each visit.
Conclusion
Planning for retention and maintaining regular contact in this longitudinal dental study contributed to the good retention that exceeded the initial retention goals for each site and overall. Retention strategies were focused on frequent routine contacts, which over time became expected by the participants, and providing monetary incentives of increasing value. At the end of the 36-month longitudinal follow-up, primary caregivers reported high comfort levels with 4-monthly intermittent contacts. Future research could assess methods to track participants lost to follow-up and using social media for retention as an alternate form of contact.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by National Institutes of Health-NIDCR- U01 DE021412 and NIH CTSA grants: TR000006 (Indiana University), U54 TR001356 (University of Iowa), and UL1TR000433 (University of Michigan).
