Abstract
Background
Evidence on effective approaches for the recruitment of diverse populations remains limited. This study aimed to assess whether a co-developed study information video would improve participant ethnic diversity within a paediatric study cohort.
Methods
Pre-post study nested within the Uveitis in childhood prospective national inception cohort study, UNICORNS. Intervention of interest: study information and recruitment video with format and content co-developed with a patient expert group. Outcome of interest: the proportion of families from ethnic minority groups, and of families with primary caregivers without a higher educational degree who were recruited to the study, and the proportion of families who engaged in study activities (returned patient reported outcome metrics following recruitment) pre and post intervention implementation.
Results
During the study period, 191 children were recruited to UNICORNS (110 pre-intervention era, 81 post-intervention era). A larger proportion of families recruited in the post-intervention era were of an ethnic minority background (36% versus 20%, Fisher’s exact test p=0.02). A larger proportion of families returned self-descriptions of their sociodemographic background (100% versus 87%, Fisher’s exact test p=0.0003) and a larger proportion returned study specific patient reported outcome measures (70% versus 55%, Fisher’s exact test p=0.003).
Conclusion
Co-created digital research information resources can be an effective tool for ensuring better representation in research studies and may also support participant engagement overall. Our reported methods and findings should support other researchers in developing patient-centred and patient co-developed strategies and tools for the purposes of equitable access to medical research.
Keywords
Introduction
The under-representation of participants from ethnic minority groups in medical research has been widely discussed, 1 with postulated explanatory factors including a failure to adopt diverse recruitment strategies, 2 and a lack of trust born of the long-standing neglect or misuse of data from under-represented communities.3–5 The adverse impacts of a mismatch between research populations, and the real world populations the research is intended to benefit, are numerous: significant discoveries particular to populations may go unnoticed or therapeutic benefits and harms identified in research may not translate to all groups within a community.6,7
Poor recruitment and retention in clinical and health studies can lead to delays, protocol changes, additional costs, and sometimes the abandoning of studies altogether. 8 Slower recruitment usually results in failure to reach the target study sample within the necessary timeframe, impacting statistical power, feasibility and translational value of studies.9,10 To address this, research investigators are increasingly examining the use of digital tools to identify, recruit, and retain study participants. 10 These tools, which include videos, online advertisements, and social media, may also be appropriate for targeting less frequently included groups.8–13 However, there is a limited evidence base on the impact of research methods designed to ensure a more diverse participant population in clinical research. 14
We sought to improve recruitment rates and diversity, and engagement post recruitment, within a prospective longitudinal rare childhood disease study. UNICORNS (Uveitis in Childhood Prospective National Cohort Study) is an observational study that aims to describe disease natural history amongst a nationally representative group of children with non-infectious uveitis (a chronic, complex inflammatory disease) and to identify the determinants of adverse visual and developmental outcomes. 15 Alongside longitudinal collection of clinical data, families and children are asked to self-report using patient reported outcome measures (PROMs) capturing quality of life and health utility outcomes. Whilst the UK patterns of the distribution of childhood uveitis is unclear, patient ethnicity and socioeconomic status have been postulated as predictors of poor outcomes for affected children 16 ; therefore, multiple approaches to support diverse recruitment, such as the development of a patient expert study steering group, social media channels, and regular participant study newsletters, were embedded within study design. UNICORNS opened to recruitment in March 2020. Despite these strategies, it was recognised that additional steps needed to be taken to ensure a diverse participant group, particularly with the onset of the global COVID-19 pandemic.
It has been suggested that complex interventions co-developed by patient groups are more effective because they are informed by the lived experience of members of the target population.10,17 The development of a video study information and recruitment tool specifically aimed at improving recruitment diversity was suggested and championed by the UNICORNS patient expert team, the Childhood Uveitis Studies Steering group. The group, formed in 2019, consists of three young people living with childhood uveitis, and three parents of (other) affected children, and had supported had earlier work to demonstrate the positive impact of co-developed patient disease education video resources. 18 We hypothesised that a co-developed recruitment video would improve diversity within the study patient population and increase participant engagement post recruitment and interrogated this hypothesis using a pre-post (also known as before-and-after) study design.
Methods
We undertook a quantitative pre-post study embedded within a multicentre inception cohort study, the Uveitis in Childhood Prospective National Cohort Study (UNICORNS). 15 Ethics approval for this work was granted by the Health Research Authority London – London – Bloomsbury Research Ethics Committee, REC reference 20/LO/0661. The researchers have followed the principles for human research outlined in the Declaration of Helsinki.
Cohort study participants
Participants for UNICORNs were recruited across 32 United Kingdom National Health Service (NHS) hospitals. Initial approach was undertaken by the managing clinical team during routine care. The managing clinical teams were asked to achieve “Good Clinical Practice” (GCP) certification. GCP, the recommended training in the UK for health professionals undertaking patient-facing research, includes training on representative recruitment and inclusive recruitment approaches. Families of eligible children (those newly diagnosed with the target disorder, non-infectious uveitis, during the first 18 years of life) were handed a study pack and invitation letter. The study pack contained a participant information sheet, assent form for older children, consent form for parents and young people aged 16 years or over, and a family background questionnaire which collected data on self-described ethnic background and socioeconomic markers. Study packs had been co-developed with patient involvement groups to ensure the use of accessible and inclusive language. Families who wished to participate then completed and returned study documentations postally to the central research team (Figure 1). Recruited families were then provided, postally, with four patient reported outcome measures (Strengths and Difficulties Questionnaire, Child Health Utility 9D, Children’s Sleep Habits Questionnaire, and a general evaluation visual analogue score) to return to the study team. Participant flow chart
Intervention development
A video development virtual meeting was held with the Childhood Uveitis Study Steering Group. During this initial discussion, key target aims for the intervention, as agreed by the group by consensus, comprised keeping the video short (less than 3 min), using lay, non-coercive language, and centring the importance of capturing the diversity of voices of those affected by disease. The script and video story board were then drafted in Microsoft PowerPoint™ and circulated for group approval. In the approved version of the video (available using the QR code provided in a supplemental document), the key content of the video was agreed by consensus (supplementary document).
The research team (CI & RA) and clinical research nurse recorded their sections together (using a smartphone with a microphone attachment) over a 43-min period. The parent recorded their section at home and sent in their video for use. Editing was undertaken by the study RA (requiring a total of 2 h of video editing time using Adobe Premiere Pro) and following approval by the patient expert group, the final 2 min 35 s duration video was uploaded to YouTube™. This platform was chosen to host the video due to the high penetration amongst the target population, as advised by our patient expert group, and because of the ability to autogenerate closed captions in a comprehensive range of non-English languages. A leaflet embedded with a QR code video link was placed within the study packs sent out to identified eligible families across all sites.
Assessment of video impact
The pre-post study design allowed for measurement of our pre-defined outcomes of interest before and after implementation of the intervention. Implementation was defined as the handing over of a QR-code containing study pack to an eligible family (vs a non-QR code containing pack as used prior to this intervention), and the posting of a QR code leaflet alongside the PROMs containing pack (vs the PROMs packs without this QR code leaflet sent to families prior to this intervention). Variables assessed were ethnic diversity (self-reported) and socioeconomic diversity of recruited participants (recruitment outcome) and of recruited participating children and families returning PROM questionnaires (engagement outcome). Recruitment was defined as return to the study team by the family (or young person for those aged 16 – 18 years) of a completed consent form. Engagement was defined as the return of completed Patient Reported Outcome Measures (PROMs) post recruitment. The study period was set as 1st March 2021 to 31st January 2022 (pre-intervention) and 1st February 2022 to 30th September 2022 (post-intervention). The UNICORN recruitment data between March 2020 and 28th Feb 2021 were not used. These data were omitted as this period was considered to have non-comparable recruitment figures, due to the limited number of hospitals which were open to recruitment during that time (many NHS centres having closed recruitment to non-COVID research).
Analysis
The outcome measures were the proportion of families from ethnic minority backgrounds (categorized according to the standard Office of National Statistics categories used in the UK), 19 and of families with primary caregivers with a degree of higher level of education (a standard self-reported socio-economic status measure used in child health research in the UK), 20 who were either recruited or engaged, comparing the pre- and post-intervention implementation eras. Frequency data are presented with 95% confidence intervals. We undertook a Fisher’s exact test to test the hypothesis of a positive impact on recruitment and engagement following implementation of the video.
To ensure ethical involvement of patients and their families in co-development, we followed the UK’s National Institute of Health Research’s (NIHR) INVOLVE guidance for involving members of the public and patients in research. 21
Role of the funding source
The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. ALS and SK had full access to all study data.
Results
Baseline Characteristics of the UNICORN study recruits pre and post intervention.
a14 of the 110 participants pre-intervention declined to submit any data on family background.
Self-reported ethnicity
Pre intervention, 19.8% (95% CI 12% - 29%) of 96 participants identified as being from an ethnic minority background, whilst post intervention 35.8% of 81 participants (95% CI 25% - 47%) identified as being from an ethnic minority background. This difference was statistically significant (Fe test p = .02).
Socioeconomic status
Primary care giver level of education, used here are a marker of family socioeconomic status (with graduate level education being a marker of higher family socio-economic status), did not differ significantly within the recruited study cohort following the intervention. Although fewer families had this marker of higher socioeconomic status following the intervention (43%, 95% CI 32%–55%, graduate primary care providers post vs 46%, 95% CI 36%–56%, pre intervention) this difference did not reach statistical significance (Fe test p = .77).
Return rate for PROMs
During the pre-intervention period, 48% (53/110, 95% CI 39% - 58%) of families returned completed questionnaires. Of these 53 families, 28, 52% (95% CI 39%–67%) of the primary care givers in the family were non-degree holders and 13, 25% (95% CI 14%–38%) identified as being from an ethnic minority background. During the post-intervention period, 71% (57/81, 95% CI 59% – 80%) of families returned completed questionnaires (vs pre intervention Fe test p = .003). Of those, 61% (35/57, 95% CI 48%–74%) were non-degree holders and 19% (11/57, 95% CI 10%–32%) identified as being from an ethnic minority background. There was no significant difference in the distribution of educational status or ethnicity of those families returning PROMs post versus pre the video intervention (on Fisher’s exact testing).
Discussion
From this pre-post study assessing changes in participant diversity and engagement in an on-going cohort study after implementation of a co-developed video-based intervention, we report significant positive impact on the ethnic diversity within the recruited participant sample as well as a significant positive impact on overall engagement. There was no impact on the socioeconomic characteristics of the cohort as measured using primary care giver education level.
These findings suggest that this approach was an effective, accessible and family centred method for recruitment, retention and creation of a diverse participant population. Similar positive impact on global recruitment has since been reported by other investigators. 22 The use of video media provided the opportunity to reach a wider audience, particularly those who prefer to engage more with visual digital content, 23 and those potential participants with limited literacy skills. 24 The ethnic diversity of the researchers, clinician and affected family member in the video may have added to video impact, and the representation 25 this presented, as well as the ‘humanisation’ of research, may have made the study less impersonal, enabling trust building. 26
Current obstacles to research involvement from ethnic minority groups vary.2,3,5 Poor communication of the information needed to understand the concept of research, its importance and what it involves for them as participants may play a role. The video intervention developed for this study aimed to address this, emphasising the importance of the research and equitable participation to achieve clinically important representation, as well as directing participants to other sources of information. Secondly, lack of awareness of research may be due to limited use or access to services and poor communication / relationship between researchers and the very communities who are most important to reach.27,28 The co-development approach for this video ensured the involvement of affected children, young people and families in production of the intervention, supporting engagement and building relationships. Thirdly, concerns about adverse effects, potential costs associated with participating and general mistrust of research due to perceptions or histories of poor participant experience may be obstacles for some communities.28,29 The video content briefly addressed the burden of participation, in order to address these concerns. Lastly, lack of language and cultural competency of researchers may limit the ability to engage less frequently heard communities in research. Investigators seeking to involve families must be aware of the nuances involved in the experience of people from traditionally under-represented groups, for example hesitation to participate because of perceived or actual stigmatisation following diagnosis of a particular health condition.27–29 The ethnically diverse nature of the team involved in video production, and the description within the video of the availability of written patient facing documentation in multiple languages, may have reassured participants of the cultural competency of the team.
There are several resources available to help researchers in increasing diversity in their recruiting process, most notably the REP-EQUITY toolkit, which presents a framework and checklist with which to design approaches aimed at capturing a representative and equitable sample in health research.
30
Other resources include the UK’s NIHR INVOLVE inclusivity framework, designed to support active public involvement in research,
31
the INCLUDE Ethnicity Framework for improving recruitment and retention of participants from ethnic minority groups to randomised controlled trials,
32
the Farooqi et al toolkit for involving participants from Black, Asian and Minority Ethnicity background,
27
and the US National Academies Improving Representation in Clinical Trials and Research Guidelines.
4
The recommendations in these resources advise investigators to ensure adequate funding is in place, use inclusive approaches, focus on communication strategies and build trust within the populations of interest (Figure 2). There is, however, a paucity of evidence on the impact of implementing interventions developed in line with these recommendations. Our findings suggest that application of the evidence presented in these toolkits results in interventions which will encourage diverse participation in future research and presents a useful model for the development of similarly successful interventions by other investigators. Mapping and comparing the recommendations, guidelines and toolkits for increasing diversity amongst research participants. BAME = Black Asian and Minority Ethnicity, NIHR = National Institute of Health Research.
Our findings on the absence of a differential effect of the intervention on the characteristics of those who remained engaged in the study following recruitment may be limited by the overall success of the intervention on engagement. As the proportion of those engaged increased was only half of all recruits, versus two thirds recruited following the intervention, a larger sample size might have supported an exploration of differential impact. Our work has other limitations. This work, which examines the inclusion, or exclusion, of different ethnic groups in paediatric research, has used a broad categorisation for ethnicity (minority vs majority, i.e. White ethnic background). Whilst this allowed us to undertake analysis of outcomes for our rare disease cohort, a more detailed level of analysis (i.e. by individual self-defined race/ethnicity) may have allowed us to report on differential patterning for recruitment and engagement. However, the focus for this work was on ensuring representation of under-represented groups as a whole. Future work will need to take a more granular approach to ensure that the interventions applied have significant reach for the populations of interest. Another potential study limitation was a difference in the study population pre and post intervention. Of the 37 actively recruiting sites, three were only recruiting participants post implementation of the video intervention. However, the 34 locations at which the study was actively recruiting pre and post intervention included the most ethnically diverse populations within the UK, including London. 17 Consequently, the study sampling base did not become increasingly diverse following the opening of the three new sites, thus this cannot be responsible for an increasingly diverse study sample. As it has been postulated that research teams who are diverse are more likely to recruit diverse research participant populations, it might be possible that the increased diversity seen in the UNICORN study sample was in part due to some change in the characteristic of the collaborating NHS teams. However, as the approach to families was made by those already providing direct clinical care and those teams changed very little (staff changes in only two of the 31 sites), it is unlikely that this was a factor in our study.
The metric used by this study for intervention implementation was a ‘proxy’ assessment, that is, sending of a link to watch the video intervention, rather than a direct measure of whether or not the video was reviewed, completeness of viewing, or comprehension or retention of video contents. The pragmatic decision not to use a survey to collect these more robust data was intended to prevent survey fatigue (as families were already being asked to complete study PROMs) and allowed a more ‘real world’ assessment of intervention implementation but prevents an assessment of the casual relationship between the video intervention and the change seen post intervention. Any ‘real-world’ assessment brings with it the risk of confounding factors, with potential differences between the implementation of the intervention or the outcome metric such as seasonal change. This study does however benefit from some stability in study environment across the pre and post periods, with the central study research team (ALS and SK) interacting with families staying the same, and with the local collaborating teams making the first approach to families (i.e. the family’s managing senior clinicians) staying the same.
The digital format could have excluded the most vulnerable or hard to reach groups who were without the means of accessing the video, specifically those from lower socioeconomic backgrounds who had limited access to digital devices or the internet. The language format (spoken English) of the video may have been inaccessible for non-native English speakers. However, our use of the YouTube platform , which enables the auto generation of closed captions in a comprehensive range of languages, will in part have addressed this. This was a relatively small study: however, as a rare disease study for a disease with an incidence of approximately one per 10,000 children, 33 a larger sample size would have required a lengthy recruitment window.
Future work should involve a larger sample size, more detailed capture of intervention implementation, family socioeconomic vulnerabilities, alongside a qualitative exploration of participant perceptions and experiences, and the perceptions and experiences of those who declined the invitation to participate. This should enable more granular examination of intervention impact, and in particular the role of the intersection of socioeconomic status and ethnicity. Interventions aimed at making research more equitable must consider how to address the obstacle of poverty, and the resultant instability, stripping of autonomy and poor mental health experienced by those living in or at risk of socioeconomic deprivation. Significant societal change is needed to address these concerns, and whilst this societal change may be outside of the ‘circle of power’ of individual research teams, the design of inclusive, understanding and flexible recruitment methods is very much within scope for investigators. Crucially, systems level change, such as adequate recognition of inequity through reimbursement and renumeration and mandating the inclusion of strategies to address inequitable participation within study protocols and grant applications, is needed to drive change. 4
Our study benefitted from the input of a disease specific patient group. Complex intervention research calls for strong and early engagement with patient groups to help inform how and whether the intervention will be suitable, implementable, affordable, scalable, and adaptable across contexts. 17 The short duration of our video, as advised by the patient group, was beneficial, with studies suggesting that short length videos may be optimal, as viewers are more likely to complete watching the video and better retain information. 9 Although no formal co-development framework was used in the co-development process, the principles of ownership and understanding for all by all, openness, commitment to sharing power, clear communication, and respect, which are central to the subsequently published co-production frameworks, 34 were kept at the heart of our group activities.
Conclusion
This study demonstrates a feasible, accessible and scalable approach to co-developing a digital video intervention which resulted in a positive impact on the diversity of recruited study participants, and participant engagement. Other research teams should consider using videos as a format to increase participant diversity and engagement. This, however, should be part of multiple approaches, with the main aim being to make research “available, accessible, aware, appropriate, and acceptable”, 35 with targeted strategies for specific groups to ensure true equity.
Supplemental Material
Supplemental Material - Reducing research participant disparities through use of a co-developed video intervention: A cohort nested pre-post study
Supplemental Reducing research participant disparities through use of a co-developed video intervention: A cohort nested pre-post study by Salomey Kellett, Jugnoo S Rahi, Ameenat Lola Solebo and on behalf of the UNICORN Study Group in Research Methods in Medicine & Health Sciences
Footnotes
Acknowledgments
We acknowledge the input of Christine Twomey, clinical nurse specialist, and the Childhood Uveitis Study steering group in the development of the video intervention. We are grateful to the collaborating clinical teams who form the UNICORN study group: Dr Valerija Tadic Greenwich University; Prof Andrew D Dick, Institute of Ophthalmology; Mr Harry Petrushkin, Moorfields Eye Hospital; Dr Jane Ashworth, Manchester Eye Hospital; Mr Alan Connor, Newcastle Eye Centre, Royal Victoria Infirmary; Ms Eibhlín McLoone, Royal Belfast Hospital; Dr Srilakshmi Sharma, Oxford Eye Hospital; Mr Eleftherios Agorogiannis, Oxford Eye Hospital; Mr Patrick Watts, Cardiff Eye Hospital; Mr Ed Hughes, Sussex Eye Hospital; Ms Ailsa Elizabeth Ritchie, Evalina Unit, St Thomas’s Hospital; Prof Rachel F Pilling, Bradford Teaching Hospital; Mr James Benzimra, Royal Devon and Exter Foundation Trust; Ms Catherine Marsh, University Hospitals Devon; Mr Daniel Pharoah, James Paget Hospital; Dr Conrad Schmoll, Princess Alexandra Eye Hospital, Edinburgh; Ms Dhanes Thomas, Moorfields Eye Hospital; Mr Jerald William, Birmingham Children’s Hospital / Royal Wolverhampton Hospitals NHS Trust; Ms Jessy Choi, Sheffield Children’s Hospital; Mr Anas Injarie and Mr Narman Puvanachandra, Norfolk and Norwich NHS Trust; Mr Vernon Long and Mr Ashish Kumar, Leeds Teaching Hospitals NHS Trust; Mr Jose Gonzalez-Martin, Alder Hey Hospital, Leeds; Ms Archana Pradeep and Dr Kishore Warrior, Nottingham University Hospital; Ms Samira Anwar Leicester Hospital; Ms Brinda Muthusamy, Addenbrookes Hospital; Mr M. Ashwin Reddy, Royal London Hospital; Ms Kate Bush, Royal Bournemouth Hospital; Mr Kevin Falzon, Harrogate Hospital; Dr Eoghan Millar, Royal Hospital for Children, Glasgow; Ms Una O’Colmain, Ninewells Hospital Dundee; Ms Aabgina Shafi, Mid Yorkshire Hospitals NHS Trust.
Author contributions
The study was conceptualised by Ameenat Lola Solebo. All authors contributed to the study design. Material preparation, data collection and analysis were performed by Salomey Kellett and Ameenat Lola Solebo. The first draft of the manuscript was written by Salomey Kellett, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute for Health and Care Research (NIHR) Clinician Scientist award (grant number CS-2018-18-ST2-005) and Biomedical Research Centre Small grant. This work was undertaken at UCL Institute of Child Health / Great Ormond Street Hospital for children which received a proportion of funding from the Department of Health’s NIHR Biomedical Research Centers funding scheme. Views expressed are those of the authors and not necessarily those of the NHS/NIHR. The sponsor had no involvement in: (1) study design; (2) the collection, analysis, and interpretation of data; (3) the writing of the report; and (4) the decision to submit the paper for publication.
Ethical statement
Data availability statement
Data sharing: Individual level data are not being made available for this study involving human research participant data. Consent for publication of raw data was not obtained (as this approach may have led to differential non-inclusion of certain patient groups). The dataset could in theory pose a threat to confidentiality. Guidance was sought from relevant IRB.
Supplemental Material
Supplemental material for this article is available online.
Appendix
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
