Response to the Letter to the Editor From Mayen et al Regarding “Clinical Trial Recruitment and Retention of College Students With Type 1 Diabetes via Social Media: An Implementation Case Study”

We are delighted to address the comments of Mayen et al ¹ regarding our report and applaud them for helping to enliven conversation about these evolving methods. ² Mayen et al suggest that our described methodology ³ does not align with that of a case study. We disagree. The study’s substantive aim involved a comparative effectiveness pilot trial. Here, we present the study’s methodological aim, an evaluation of the implementation of social media-based recruitment. We framed this report as a case study since our goal was to describe the details of our methodological approach and to share empirical and operational insights, recognizing the limitations of a single report.

Mayen et al also raise issue with sample representativeness (a limitation we explore). Many studies, regardless of recruitment strategy, do not formally assess sample representativeness and may be subject to external validity threats without providing data to reveal them. We present a transparent and comprehensive accounting of how our sample may (or may not) represent the target population (see table 1 and appendix 2), including a discussion of potential ways to mitigate these issues in future work (see table 3). Despite similarities between our analytic sample and the sampling frame, we do state that our sample may not be representative of all college students with type 1 diabetes. Notably, we did not explicitly exclude noninternet users (in fact, several participants accessed the study because of a personal referral, see figure 1), but reliance on social media outreach likely biases our sample toward internet users (a potential limitation inherent to all internet-based recruitment).

They also call for a different design that evaluates “direct recruitment during a medical consultation” ¹ as a control comparator. The substantive aim of this trial was to evaluate the effectiveness of competing spokespersons for delivering an educational intervention (the intervention materials are being evaluated in a separate, “traditional” randomized controlled trial), so our recruitment method for this trial was not the main “treatment” under investigation. Their proposed design might be appropriate for future work wishing to compare social media to traditional recruitment directly.

Finally, Mayen et al express frustration that trial results were not included in this article. Space restrictions precluded the presentation of our methodological approach (and associated evaluation) alongside a report of trial results. Publication of a methods paper describing trial design in advance of publication of trial results is not uncommon. Stay tuned for trial findings as we are preparing them for submission now!

While we agree that our paper is not sufficient for definitively proving that “Internet recruitment is reliable for scientific studies” (per Mayen et al comments ¹ ), our experience provides evidence toward the feasibility of recruiting hard-to-reach populations via social media and of generating “a sample that is largely representative of the user-base from which the sample was drawn.” ³ We stand by our conclusions and reiterate our call for future work to evaluate the effectiveness and acceptability of these methods further.