Comparing online crowdsourcing with clinic patient enrollment: Findings from the IP4IC Study on interstitial cystitis/bladder pain syndrome

Abstract

Introduction

Interstitial cystitis/bladder pain syndrome (IC/BPS) manifests as urinary symptoms including urgency, frequency, and pain. The IP4IC Study aimed to establish a urine-based biomarker score for diagnosing IC/BPS. To accomplish this objective, we investigated the parallels and variances between patients enrolled via physician/hospital clinics and those recruited through online crowdsourcing.

Methods

Through a nationwide crowdsource effort, we collected surveys from patients with history of IC/BPS. Study participants were asked to complete the validated instruments of Interstitial Cystitis Symptom Index (ICSI) and Interstitial Cystitis Problem Index (ICPI), as well as provide demographic information. We then compared the survey responses of patients recruited through crowdsourcing with those recruited from three specialized tertiary care urology clinics engaged in clinical research.

Results

Survey responses of 1300 participants were collected from all 50 states of the USA via crowdsourcing and 319 from a clinical setting. ICSI and ICPI were similar for IC/BPS patients diagnosed by the physicians in clinic and self-reported by subjects via crowdsourcing stating they have a history of previous physician diagnosis of IC/BPS. Surprisingly, ICSI and ICPI were significantly lower in crowdsourced control than in-clinic control subjects.

Conclusion

The IP4IC Study provides valuable insights into the similarities and differences between patients recruited through clinics and those recruited through online crowdsourcing. There were no significant differences in disease symptoms among these groups. Individuals who express an interest in digital health research and self-identify as having been previously diagnosed by physicians with IC/BPS can be regarded as reliable candidates for crowdsourcing research.

Introduction

Online crowdsourcing platforms present an effective means of enlisting participants for research endeavors, including disease biomarker studies, as demonstrated in our own investigation. These platforms facilitate access to a broad and diverse population, potentially bolstering both the size and diversity of the participant sample. Recruitment methods employed online may involve leveraging social media advertisements, email campaigns, online forums, or dedicated study websites. Conversely, patient recruitment within a clinical setting entails engaging individuals who seek healthcare services at hospitals and clinics.

The primary focus of the IP4IC Study was the development of a urine-based biomarker score for the diagnosis of Interstitial Cystitis/Bladder Pain Syndrome (IC/BPS). To achieve this objective, we meticulously examined the similarities and disparities between patients recruited through physician/hospital clinics and those recruited via online crowdsourcing.

Our study sought to identify potential biomarkers for IC/BPS, an enduring and debilitating condition that affects the urinary bladder. Presently, there is a lack of specific diagnostic tests for IC/BPS, with diagnosis predominantly relying on patient-reported symptoms and the exclusion of other similar clinical presentations. In addressing this gap, we utilized social media platforms to recruit participants and collect urine samples, aiming to develop a non-invasive, swift, and cost-effective diagnostic test using urine-based biomarkers. ¹ We aimed to assess the cost-effectiveness, efficiency, and demographic representation of online recruitment compared to in-clinic recruitment by collecting urine samples from both sources.^2–4

During the execution and presentation of the IP4IC study, a central question that has been posed to us pertains to the similarity or disparity between self-reported IC/BPS patients obtained through crowdsourcing and those diagnosed by physicians in clinical settings.

Materials and methods

Online recruitment

All study materials were approved by the Beaumont Health Institutional Review Board (IRB# 2019–266), and participant consent was documented both in the clinic and online. We created a research study website and used a Health Insurance Portability and Accountability Act (HIPAA)-compliant survey using SurveyMonkey to screen and enroll participants. ⁵

We partnered with the Interstitial Cystitis Association (ICA), a national IC/BPS patient advocacy group. The ICA sent emails to their members, and links to the recruitment website were also circulated through their newsletters. Additionally, the ICA created and circulated social media posts on Instagram, Facebook, and TikTok to reach even more participants. ⁵

The study was also directly advertised using Google Ads, with the keywords “IC,” “IC/BPS,” “interstitial cystitis,” and “urology study.” Finally, search engine optimization (SEO) of the study webpage allowed users to easily find and directly signup for the study, as well as share it through email or social media applications. The participants were instructed to find two other sample donors of similar age without a bladder disease health history to serve as control participants.

Clinic participation collection

The participants were physically present at three reputable urology clinics located within academic tertiary medical centers known for their expertise and research in IC/BPS. These clinics possessed IRB-approved research capabilities. The participants visited these clinics during their regular clinical appointments to provide urine samples. The entire process of recruitment, obtaining consent, collecting urine samples, completing surveys, and enrolling participants took place on the same day. ⁶

Subject inclusion criteria

Subjects who returned a urine sample were eligible for inclusion. Control subjects with high symptom scores for OAB, multiple sclerosis, pregnancy, urinary tract infection, high pain (≥7/10), and diabetes were excluded. All included subjects who completed the online survey received and returned a urine collection kit.

Analysis

The participants self-reported their ICSI and ICPI demographic data on the survey forms provided. Date of birth, age, gender, race, and ethnicity were listed in the survey. Chi-squared statistical tests were used for ICSI and ICPI (Table 1, Figure 1).

Figure 1.

ICSI score (1a) and ICPI score (1b) among the four groups. **** ICSI: p = 0.000; ICPI: p = .000; n.s.: not significant; ICPS: 0.260; ICPS: 0.666.

Table 1.

Demographic data of the four groups.

Variable	Statistic	Clinic IC	Clinic control	Crowdsource IC	Crowdsource control
Age	n	78	218	536	718
	Mean (std)	54.8 (15.9)	62.0 (17.3)	50.1 (16.3)	47.7 (16.9)
	Median	54	65	50	46
	Min, max	25, 86	22, 94	19, 91	19, 95
Sex	n (%)
Female		71 (91%)	148 (67.9%)	513 (95.7%)	311 (43.3%)
Male		7 (9%)	68 (31.2%)	21 (3.9%)	399 (55.6%)
Unknown, missing, or not reported		-	2 (0.9%)	2 (0.4%)	8 (1.1%)

Results

After 3 months and 2 weeks from the first enrollment date, online sample collection was stopped with a total of 1300 samples collected. Out of the1300 urine samples, 36 were excluded due to not meeting the inclusion criteria. With the total accepted online sample of 1264, we averaged 90.3 samples per week. In-clinic recruitment started at the same time; however, sample collection took longer due to a slower influx of participants. As of December 2021, 319 IC/BPS and control samples were collected at the two clinical sites combined. Of the 319 urine samples, 23 were excluded due to not meeting the inclusion criteria. With the total accepted clinic samples of 296, we averaged 5.7 samples per week. The first week of online recruitment had the highest number of enrolled participants with the highest sample return rate in the third week. There was a 2-week average turnaround time from initial online enrollment to sample return. ⁵ The rate of enrollment and return tapered down slowly over time but peaked slightly at weeks 8 and 10 when the ICA sent reminder emails to the participants. Enrollment at the clinical sites was at a steady rate with no major spikes or drops.

Overall, 93% of the online-recruited participants identify as white, while 60% of the clinic-recruited participants identify as white (88% of the full population). All other racial groups made up 7% of the online-recruited and 40% of the clinic-recruited (12% of the full population). A total of 6% of the total online-recruited participants identify as Hispanic, whereas 30% of the clinic-recruited participants identify as Hispanic.

 Figure 1(a) highlights the differences between the clinical and crowdsource ICSI/ICPI levels within the control and IC groups. We demonstrate that ICSI/ICPI levels are significantly different between clinics vs. crowdsource in the control groups. Table 1 provides a summary of the age and sex information across groups. In addition to the results in Figure 1(b), Chi-squared tests were performed in an alternate arrangement, comparing the control vs. IC groups in crowdsource (ICSI, p = 4.9 × 10⁻¹²⁹; ICPI, p = 1.0 × 10⁻¹⁴⁴) and clinic (ICSI, p = 2.4 × 10⁻⁵; ICPI, p = 0.017).

Discussion

The findings of this study demonstrate that utilizing contactless online recruitment methods, such as social media and website SEO, is a compelling and innovative approach in the discovery and validation of clinical biomarkers. ¹

Recently, we have shown that collecting biomarkers through crowdsourcing is more cost-effective and time-efficient compared to traditional in-person recruitment at clinics. Our calculations indicate that online urine sample collection, at a cost of $81.45 per sample, proved to be 80% cheaper with a 70% faster return rate. ⁶ In-person recruitment at clinical sites is constrained by their physical location, whereas online recruitment allowed us to engage the participants from all 50 states, effectively minimizing geographical bias in research studies. Through online networks, we were able to reach individuals who are typically underrepresented, and at a significantly reduced cost.

The IP4IC Study, with a focus on IC/BPS, provides valuable insights into the comparability of patients recruited through physician/hospital clinics and those recruited through online crowdsourcing. Our research indicates that within this motivated participant group, which includes individuals diagnosed with IC/BPS and specialized healthcare providers experienced in managing the condition, no significant differences were observed. Therefore, individuals who actively participate in crowdsource research and self-report a prior diagnosis of IC/BPS by physicians can be considered reliable candidates for clinical trials conducted through crowdsourcing. The significant differences in ICSI/ICPI for the controls between clinic and crowdsourcing call for further investigation.^7,8

A key difference noted in this study was the difference between controls collected in the clinic vs. crowdsourcing. It is intuitive that entirely healthy people will not be consulting a urologist or urogynecologist in the clinic. This difference could have profound effects on biomarker studies and therapeutic clinical trials depending on how controls are obtained. The crowdsource group shows a greater difference in the distribution of ICSI/ICPI scores between the control and IC groups as opposed to the clinical subjects, who, in the control group, may have confounding lower urinary tract symptom disorders.

The limitations of the study include sampling bias as the online sample collection had a predominantly white and non-Hispanic participant pool (93% white, 6% Hispanic). In contrast, the clinic-recruited participants had a more diverse racial and ethnic composition. There were also different recruitment rates and timing variability between online and clinic settings as well as participant identification by racial and ethnic categories relied on self-reporting.

Conclusions

The IP4IC Study sheds light on the potential of online recruitment methods and the reliability of self-reported data from individuals with IC/BPS, further emphasizing the value of crowdsourcing in research endeavors. Levels of subjective diagnostic tests for IC/BPS (ICSI and ICPI) were found to significantly differ between control groups obtained via clinic vs. obtained via crowdsourcing, which may be an important consideration in future work in the development of diagnostics and treatment for IC/BPS.