The role of people with intellectual disability in intellectual disability research: A systematic review of Delphi studies

Abstract

The design of intellectual disability research warrants critical examination, as the knowledge produced through these approaches informs evidence-based practices. People with lived experience should be recognised as experts in understanding their own bodies, conditions, and treatment. This systematic review examined the design of Delphi studies to assess the extent to which people with intellectual disability are included as experts. The Delphi method, which gathers structured feedback from experts, offers insight into how lived experience is valued as a source of knowledge. Across fifty-five publications reporting on forty-nine separate Delphi studies that met our inclusion criteria, only nine involved people with intellectual disability directly. In contrast, family and informal caregivers were more frequently represented as experts, and their perspectives carried greater influence. The findings of this review include guidance for practitioners and researchers to facilitate greater participatory roles of people with intellectual disability.

Introduction

The Convention on the Rights of Persons with Disabilities (CRPD 2007), as a key international treaty, advocates for the full and effective participation of persons with disabilities in all aspects of society. It emphasises the right to be heard and included in decision-making processes that affect people with disabilities. Critical examination of the design of intellectual disability research is urgently needed. The knowledge produced through these approaches, informs evidence-based practices. It is therefore necessary for practitioners themselves to understand the role people with intellectual disability have had as informants of this evidence base, as they are responsible for the design and delivery of frontline services.

In light of the CRPD’s (2007) call for inclusion and participation, this study investigates whether and how people with intellectual disability are incorporated into research about them, which uses the Delphi method. The Delphi method has unique opportunities for incorporating lived experience, as expert status is conferred to participants and a process of structured feedback is followed. We argue that the extent to which the Delphi method incorporates the expertise of people with intellectual disability, offers a window to make sense of a broader social reality: In developing evidence-based practice, to what degree are people with intellectual disability understood as authorities of their own experience? The research design intentions evident in Delphi studies can be interpreted as a meaningful case study, to produce tangible insights toward answering this broader question.

The Delphi method produces authoritative knowledge, through structured consultation in areas where there is little existing evidence (Chalmers and Armour, 2019). Delphi studies have a well-established utility within the Health Sciences, where the method has been used primarily with a clinical focus around intellectual disability. However, the method is also commonly used in practice settings to forecast healthcare and community demands, to guide policy development and inform problem solving (Chalmers and Armour, 2019). Delphi approaches offer scope to broaden who is considered an ‘expert’ (Linstone and Turoff, 1975).

Experts involved in Delphi panels are purposively recruited to make evaluations based on their specialist medical, research or technical backgrounds. To understand more fully the context and impact of findings developed through formal/professional forms of knowledge, the value of lived experience as an epistemology is increasingly recognised (Björne, 2020; Merrells et al., 2019). Delphi expert panels can be comprised of people with diverse lived and/or formal/professional backgrounds. The potential to facilitate structured feedback and dialogue between experts of diverse backgrounds serves the imperative of inclusive health research and greater patient involvement (Currie et al., 2022).

There is no single accepted standard for a Delphi study and the parameters for administration and reporting results differ widely (Humphrey-Murto and de Wit, 2019). Traditional Delphi studies were administered through post, with hard copies of correspondence sent to panel experts for consensus scoring (Linstone and Turoff, 1975). Contemporary Delphi studies are often conducted online or over the telephone, conditions which are themselves reported as ‘modifications’ (Jandhyala, 2020). This established practice of modification within the method could feasibly engender greater possibilities for Delphi studies to be designed in ways that facilitate access needs and expand the avenues of direct participation available to ‘patient populations’. This is particularly relevant for people with intellectual disability, who are considered a ‘hard to reach’ population. Despite facing poorer health outcomes, they are often excluded from directly participating in mainstream health research (Johnson and Walmsley, 2003; Mulhall et al., 2020). Delphi studies are a worthwhile avenue to explore in remediating this situation, given both the modifications possible and the recognition that people living with the conditions under investigation can themselves be considered ‘experts’.

As part of a necessary agenda for Delphi method studies, Humphrey-Murto and de Wit (2019) call for analysis of stakeholder participation. This review responds to that call, while also interpreting Delphi studies as an aggregate dataset, to analyse how people with intellectual disabilities’ lived experience is utilised. Rix and colleagues’ (2020) systematic review of participatory research involving people with intellectual disability and/or sensory impairments, identifies a need for analysis beyond the types of activities carried out. Conceptualising ‘participation’ instead requires an understanding of how these activities are facilitated. By examining the role(s) of people with intellectual disability in Delphi studies and how access modifications are reported, this review responds to Rix and colleagues’ (2020) call.

The following research questions guided our review:

(1) What role(s) have people with intellectual disability had in the Delphi method?

In terms of:

a. What direct participatory roles have people with intellectual disability had?

b. What details are given about the intersectional identities of participants with intellectual disability?

c. How do the roles of people with intellectual disability compare to those of other experts?

(2) How are Delphi studies being used in the field of intellectual disability?

Namely:

a. Delphi focus: What topic areas are Delphi studies producing knowledge about?

b. Delphi objective: How is this Delphi evidence intended to be used in practice?

(3) What accessibility modifications to the Delphi method have been made, to enable participation of people with intellectual disability?

Methods

This systematic review is reported using the PRISMA guidelines. Critical appraisal and quality assessment tools were not used, as the outcomes and evidence base of included studies are not being evaluated (Yeo et al., 2022).

Eligibility criteria

Studies were included in the review if written in English and (i) the Delphi method was named and the main features used; (ii) the population investigated were adults (≥18 years) with intellectual disability. The first criterion recognises there is no single definitive description of the Delphi method. To ensure distinction from other methods of group consensus, studies were included if they adopted the four generally accepted key features of the Delphi method: (1) anonymity between respondents; (2) gathering data that is statistically quantitative; (3) iteration over multiple rounds; (4) controlled feedback (Lindqvist and N¨ardanger cited in Chalmers and Armour, 2019; Currie et al., 2022; Lyons et al., 2018; Sulewski et al. 2017).

The second criterion recognises language around intellectual disability has evolved over time. In this review we refer to ‘people with intellectual disability’ while recognising the shifts and geographical variation in language. Studies included in this review have used terms such as ‘mental retardation’ (Annison and Young, 1980) or ‘mentally handicapped’ (Fourie et al., 1999), reflecting the clinical language used at the time of these publications. These labels were included in our search terms to make sure relevant studies were captured.

Similarly, terms like ‘learning disability’ and ‘mental disability’ may overlap in an Australian context with the criteria for intellectual disability. For example, writing in the UK context, Matthews and Hegarty (1997) refer to ‘people with learning disabilities’. In the Australian context, the population group they have studied would be described as ‘people with intellectual disability’. Terms such as ‘developmental disability’ and ‘cognitive disability’ were therefore also included. Due to the rate of comorbidity with Autism (Wark and Kingstone, 2019) and Dementia (Janicki and Dalton, 2000), particularly Alzheimer's disease among people with Down’s Syndrome, (Sheehan et al., 2014), terms relating to Dementia and neurodivergent conditions were also included in the search terms. This was to make sure articles reporting distinct findings for people with intellectual disability diagnosed with these comorbid conditions were included, even if the authors had not used a derivative of the term ‘intellectual disability’ in their abstract, title, or keywords.

During screening, these articles were reviewed to ensure that their focus, reflected our focus on intellectual disability, and not Dementia, Alzheimer’s, cognitive impairment, neurodivergent conditions or specific learning difficulties. To understand the specificity of intellectual disability research, (notwithstanding the challenges in unequivocal delineation of this field), studies were excluded upon full text review if they investigated these groups as isolated clinical populations in their studies, rather than as conditions co-occurring with intellectual disability. This eligibility criterion required findings related to the cohort with intellectual disability to be distinguishable from those diagnosed with only Dementia or Autism.

Studies were considered eligible for this systematic review when the population investigated were adults (>18 years) described as having a formal diagnosis of intellectual disability (DSM-5) or mental retardation under the DSM-IV. The review also considered studies that clearly specified the participants’ intellectual disability within the title, abstract, or content, and studies that focused on adults who self-identified with intellectual disability or who used services specifically for people with intellectual disability.

Studies were excluded if: (i) the consensus building process did not include the main features of the Delphi method (defined above); (ii) no specific findings were reported on adults with intellectual disability; (iii) they were abstract-only reports, case studies, study protocols, reviews, dissertations, doctoral theses, or non-human analyses; (iv) published in languages other than English, and (v) there was no separation in results and part of the sample consisted of populations who had otherwise been excluded, e.g. adolescent/paediatric populations and/or people diagnosed with (only) conditions other than intellectual disability.

Search strategy

Systematic searching was undertaken across: Medline Complete, Cochrane Central Trials, CINAHL, APA PsycINFO, Embase, PubMed and citation searching of key articles was completed within Scopus. The electronic search strategy (Supplementary 1) was created by a Health Librarian (FB) and represented the concepts of intellectual disability and Delphi method. It was independently peer-reviewed using the Peer Review of Electronic Search Strategies (PRESS) standard (CADTH 2016) by a second Health Librarian (RW). The PRESS standard provides validation of the quality of the database search strategy. No limits were applied to publication date, language, or publication type. The searches were current as of 17/11/2021.

Grey literature

Additionally, Google Advanced Search was used to identify grey literature on 29/10/2021. The grey literature PRISMA is presented in Supplementary 2. Only the first ten pages of results (100 hits) were screened. This reflects the focus of the review being that of methodological design and resultant knowledge production in peer reviewed work. Duplicates occurring within the Google search itself and within the database search described previously were removed (n = 48). The remaining fifty-two results yielded no additional items that met the eligibility criteria.

Google was searched to ensure our database strategy had captured the range of published literature intended and there was no evidence to broaden this search, i.e. no relevant Delphi studies (e.g., commissioned by government agencies) that met our inclusion criteria were identified. While we acknowledge it is a limitation of the work that a more extensive search of grey literature was not conducted, the specificity of our strategy allows for the meaningful examination of Delphi design intentions and the conferral of ‘experthood’ within institutions.

Selection process

Four reviewers (DKP, AO, SB-O, CJG) conducted the eligibility assessment using Covidence (www.covidence.org). Papers were screened based on title and abstract, with those potentially meeting the selection criteria undergoing full text review. The agreement of two reviewers was required at both the screening and full text review stages. Disagreements between reviewers were resolved through deliberation between authors.

Data extraction

Prior to the commencement of screening, all variables for which data were sought to meet our objectives were listed and are defined in the section that follows. Any assumptions made about missing or unclear information has been identified in Supplementary 4 endnotes. Studies that met the selection criteria had the following data extracted: (i) Intersectionality; (ii) Role(s) of people with intellectual disability; (iii) Delphi panel composition; (iv) Delphi objective; (v) Access modifications.

Intersectionality

Guided by the intersectional categories articulated in Kelly-Brown and collegues’ (2022) systematic review of cancer care, we set out to extract details where participants with intellectual disability were described in terms of their: (a) race/ethnicity; (b) sexual orientation; (c) gender identity; (d) age; (e) disability co-occurrence; (f) socioeconomic status; (g) geographic location.

Role(s) of people with intellectual disability

We classified the involvement of people with intellectual disability in each Delphi study into one of four categories: (1) consultants-only; (2) experts-only; (3) consultants and experts; (4) no role. For consistent application in this review, these terms have been defined in relation to two key participatory activities associated with Delphi studies, namely the development of Delphi items and consensus scoring.

Consultants-only were involved specifically in the development of Delphi items, that is the material to be evaluated in the Delphi and experts-only participated as panellists in the consensus scoring Delphi rounds. Consultants and experts were involved in both key activities. The category of no role was applied to studies where the topic of investigation pertained to intellectual disability, but people with intellectual disability were not involved in either of these activities. In some publications, people with intellectual disability were described as having peripheral roles. For example, Aznar and colleagues’ (2012) describe people with intellectual disability being involved in a pilot phase occurring after the completion a Delphi study. Here, we categorised participation as no role because people with intellectual disability had no direct influence on the knowledge produced through the Delphi method.

In some cases, studies were run parallel to a Delphi that involved people with intellectual disability. In Chaplin and colleagues’ (2012) study, ‘service user experts’ participate in focus groups. We have coded people with intellectual disability as having a consultant role in this case, given it is clear that the cohort informed the development of Delphi items; in round two, additional items were added to the Delphi in response to their feedback (Chaplin et al., 2012, p. 19). Chaplin and colleagues’ (2012) describe the group as ‘experts’, however they do not fit within our review definition of experts, as they did not participate in the consensus scoring rounds. The participation of people with intellectual disability in this Delphi was therefore categorised as consultants-only.

In other cases, it is unclear whether people with intellectual disability were involved at all. For example, in evaluating a screening questionnaire for “adults who are likely to have an intellectual disability” [emphasis added], McKenzie and colleagues’ (2020, p. 459) describe semi-structured interviews with a group of ‘homeless service users’. The screening tool is designed to identify a likelihood of intellectual disability among mainstream homeless service users (i.e., this was not an intellectual disability specific service). The impact of these interviews (if any) in developing Delphi items is not reported, thus people with intellectual disability are coded as having no role in this Delphi.

In contrast, Lemmey and colleagues’ (2021, p. 102) refer to the inclusion of an ‘athlete representative’ on a Delphi panel, undertaken to group athletes with intellectual disability into categories that reflect similar activity limitations. The specific focus on athletes with intellectual disability makes it logical that the ‘athlete representative’ is a person with intellectual disability. This athlete representative participated in consensus scoring, meaning the participation of people with intellectual disability in this study has been categorised as experts-only.

Panel composition

Delphi panel experts were coded into four categories: (1) paid-experts (2) unpaid-experts (3) people with intellectual disability and (4) uncategorisable. In this review, ‘lived experience’ is operationalised to deliberately refer to both people with intellectual disability and their family, friends, caregivers, advocates (unpaid-experts). When referring to the distinct experiences between these groups we use the direct terms people with intellectual disability, or unpaid-expert.

It was not viable to make distinctions between the professional backgrounds of paid-experts, as they could not be equally distinguished across all papers reviewed. Petry and colleagues’ (2007, p. 336) describe recruitment of three distinct categories: theory-experts (researchers), practice-experts (who are paid to provide direct care, education, or therapy) and experience-experts (who are involved with people with intellectual disability outside of paid employment, aligning with our definition of unpaid-experts). For the purposes of our review, the groups Petry and colleagues’ (2007) refer to as theory-experts and practice-experts are combined and counted as paid-experts. It is common for intellectual disability researchers to also be frontline practitioners. This is reflected in Lyons and colleagues’ (2018) study, where expert backgrounds are represented as overlapping percentages, with the actual number of experts representing each category not reported.

The distinctions we made for this review were determined by how the composition of expert panels were reported by the authors, in the first round of consensus scoring. This demonstrates the number of people who were purposively recruited to make evaluations from a particular type of expert experience. Whether reported by the authors or not, the possibility for overlap exists. For example, a researcher (paid-expert) might also be a caregiver (unpaid-expert). The intention is to examine how various experience backgrounds are purposively recruited and reported in conferring expert status through the Delphi method.

Delphi objective

This involved extracting data that addressed why consensus was sought. It either aligned directly with the research question, or sought to answer a specific part of the research question.

Access modifications

This involved recording the recruitment strategy and modifications related to the Delphi, which could be reasonably (if not explicitly) attributed to facilitate direct participation of people with intellectual disability as experts and/or consultants.

Data analysis

Our thematic analysis was guided by the approach of Caton and Chapman (2016) in their systematic review of social media and people with intellectual disability. During independent data extraction of the fifty-five papers, the reviewers made notes of initial ideas for codes and themes. Upon completion, the reviewers met to refine these themes. Clear definitional parameters were then finalised through discussion between the two authors. In reaching agreement this process delivers sound data, but as Caton and Chapman (2016, p. 127) similarly identify, results that rely on researcher judgement are not always replicable.

In all fifty-five papers, first cycle coding involved identifying the topic areas where the Delphi is being used to produce knowledge (Delphi focus). Second cycle coding involved identifying how this evidence is intended to be used in practice (Delphi objective). Descriptive statistics were used to report the number and percentage of publications aligned to each theme. In the nine studies where people with intellectual disability had a participatory role that directly influenced the Delphi, this process of thematic analysis was repeated to group access modifications. Discussion took place between the two authors to finalise themes (Caton and Chapman 2016). Supplementary 10 outlines the features of these nine studies that led to measures being coded within each of the modification categories.

We acknowledge that not all modifications may have been reported in the studies we reviewed. Likewise, the frequency with which these themes appear, does not necessarily equate to being most crucial to enabling accessibility (Caton and Chapman, 2016). For these reasons, we have not provided descriptive statistics for the instance of each measure. These modification categories are intended to group the practical examples we identified, to inform the design of future Delphi studies, and enable greater direct participation of people with intellectual disability.

Relative frequency synthesis

To understand the role people with intellectual disability have on Delphi panels, specifically in relation to other experts, panel composition was analysed through two stages of relative frequency analysis. CJG and DKP independently returned to each publication to purposively re-extract panel composition data to ensure reliability. DKP then took screen shots of the segment(s) of all fifty-five articles where this data was derived and copied these into a shared document with a short commentary. AO and DKP then used this document to resolve the few discrepancies remaining and confirm which studies contained uncategorisable data. Studies excluded from relative frequency analysis due to uncategorisable panel composition data, are addressed in Supplementary 5.

This resulted in forty-three Delphi panels from which to report on expert composition. Microsoft Excel was used to analyse panel composition. First, for each individual Delphi study, expert types were calculated by dividing the number of experts representing each category of experience (paid-experts, unpaid-experts, people with intellectual disability) by the total number of panelists. Where a one was scored, this meant 100% of the panel was comprised of a particular expert group. Where representation across the three expert groups was mixed, the score was converted to a percentage. Secondly, the relative frequency scores for each individual panel, were aggregated to reflect a total for each of the three experience types. These were then divided by the number of studies eligible for relative frequency analysis, to indicate the overall concentration of representation.

Results

2136 citations were imported into Covidence and after the removal of 1155 duplicates, 981 were subject to title/abstract screening. 132 were assessed for eligibility at full text review and a final sample of fifty-five publications were retained (the Published Literature PRISMA is available in Supplementary 3). In forty-six of these, people with intellectual disability are the topic of investigation only. Supplementary 10 details the nine studies where people with intellectual disability have a direct participatory role within the Delphi.

Participatory roles

In 83% (n = 46) of the fifty-five publications reviewed, people with intellectual disability were the topic of investigation but had no participatory role. In the 16% (n = 9) of studies remaining where people with intellectual disability had a participatory role: in 88% (n = 8) of cases, they were included in the panel of experts. In 11% (n = 1) they were consultants-only, involved in the development of Delphi items but not in the expert panel. In 55% (n = 5) of these nine studies, people with intellectual disability were recruited as experts-only. In 33% (n = 3) people with intellectual disability had roles as both consultants and experts.

Intersectionality

Very few details are reported pertaining to the identities of people with intellectual disability who have had direct participatory roles in the Delphi method. Lyons and colleagues’(2018, p. 209) include a table of demographic characteristics, however these are an aggregate of all experts, and it is not possible to discern the details relating specifically to the identities of participants with intellectual disability. In 66% (n = 6) of the nine studies, binary (male/female) gender configurations are reported (Bonell et al., 2012, p. 905; Chaplin et al., 2012, p. 18; Khayatzadeh-Mahani et al., 2020, p. 8; Lemmey et al., 2021, p. 102; Vicente et al., 2019, p. 208).

Chaplin and colleagues (2012, p. 19) and Bonell and colleagues (2012, p. 905) also report the psychiatric co-morbidities of their participants, as this is relevant to the mental health focus of their respective studies. Bonell and collegues (2012) sought to understand the differences between two racial groups’ experiences accessing mental health services, and therefore additionally grouped their participants as either ‘Black British’ or ‘White British’.

Role of people with intellectual disability relative to other experts

Supplementary 6 reflects the aggregate composition of panels, comprising of 92% of paid-experts. At 3%, people with intellectual disability are represented less on panels than their family, friends, caregivers and other (unpaid-expert) allies (5%). Of the forty-three panels at this stage of analysis, 79% (n = 34) comprised entirely of paid-experts (Supplementary 7). Of the remaining nine studies, there are far fewer unpaid-experts than paid-experts; four included unpaid-experts, but no people with intellectual disability (Supplementary 8). The relative frequency of paid-experts in these panels was not greater than the point at which authors had pre-defined consensus. This means that even if paid-experts delivered uniformly similar responses, reaching consensus required these assessments to be supported from (at least) a small number of unpaid-experts.

The remaining five studies included people with intellectual disability on their panels (Supplementary 9). In contrast to the mixed paid/unpaid-expert panels, in three of these five studies the concentration of paid-experts meant that people with intellectual disability were not included in numbers that would allow them to impact the outcome of consensus, if the paid-expert group were to collectively respond differently (Khayatzadeh-Mahani et al., 2020; Lemmey et al., 2021; Noorlandt et al., 2021). The two outliers were authored by Vicente and colleagues’ (2019) and Bonell and colleagues’ (2012).

Vicente and collegues (2019) established three separate panels comprising ten ‘professionals’ (paid-experts) six ‘relatives’ (unpaid-experts), and five people with intellectual disability. In this case the participation of people with intellectual disability was segregated, rather than modifying the study to facilitate full and equal participation. In the third round, where the final retention or removal of items was determined, only the paid-experts were engaged. In Bonell and colleagues’ (2012) study, the panel is comprised entirely of people with intellectual disability. However, the wording of the Delphi items required participants to reflect on their experiences of mental health services as individuals. The researchers then extrapolated this scoring to draw relationships between the (racial) collective communities in which they were embedded. In other words, the expert status of participants was situated within their experience as consumers (individual service users), not as authorities of knowledge about intellectual disability. Aside from Bonell and collegues (2012), any time people with intellectual disability were recruited alongside other expert types, the concentration of paid-experts already exceeded the predetermined consensus point. The material impact of representation afforded through these conditions was therefore limited.

Delphi focus

Of the studies reviewed, clinical research was the most common topic area where Delphi studies were used to produce knowledge, accounting for 42% (n = 23) of the publications. The intended practical applications of this Delphi knowledge were primarily: indicator development (n = 5), practice agreement (n = 5) and instrument development (n = 5). Some clinical areas of research focus necessitated panel experts with specialist medical knowledge (paid-experts). However, two studies included unpaid-experts on their panels (without professional clinical training) to speak on behalf of people with intellectual disability, rather than enable their direct participation (Nouwens et al., 2020; Ouellette-Kuntz et al., 2019). Chaplin and collegues’ (2012) was the only clinical study where people with intellectual disability had a direct participatory role. Their consultant-only role influenced the development of Delphi items, but only paid-experts were included on the panel with consensus scoring authority.

Non-clinical research

The remaining 58% (n = 32) publications were focused on research in service development (n = 9), participation (n = 8), professional development (n = 6), community support (n = 4), rights (n = 2), quality of life (n = 2), and safety (n = 1). While lived experience (unpaid-experts OR people with intellectual disability) may be considered central to approaching these topics, only eight had recruited experts to capture this knowledge. Bonell and collegues (2012) recruited a panel entirely of people with intellectual disability. Only four of the remaining seven studies included people with intellectual disability as panel experts (Khayatzadeh-Mahani et al., 2020; Lemmey et al., 2021; Noorlandt et al., 2021; Vicente et al., 2019). In all four cases, unpaid-experts are represented on these panels in greater numbers than people with intellectual disability. In the remaining three articles, only unpaid-experts were recruited (Aznar et al., 2012; Hempe et al., 2013; Petry et al., 2007). Again, even when lived experience is an intentional feature of the study, unpaid-experts are more likely to speak on behalf of people with intellectual disability than people with intellectual disability themselves.

Delphi objective

Delphi studies are inherently directed toward a specific objective for practical application. In the fifty-five publications reviewed, the knowledge produced through the Delphi method was used to inform planning/methodology (n = 11), practice agreement (n = 10) and instrument development (n = 8) indicator development (n = 7), support needs (n = 6), screening/ diagnosis/ classification (n = 5), guideline development (n = 5), forecast (n = 2), service rating (n = 1).

Accessibility modifications

Six modification categories were identified through thematic analysis, namely Communication, Specific techniques, Digital/technology, Relational/recruitment, Material/for comment and Responsive/flexible. As Rix and collegues’ (2020) call for greater attention to how participatory activities are undertaken, we have prioritised reporting specific examples of how modifications were enabled in practice. These are not discrete classifications, as some measures could be coded across multiple categories. For example, Sulewski and colleagues’ (2017) use of telephone interviews might relate to both Digital/technology while also being a Communication measure.

Discussion

Overall, this review has found that people with intellectual disability are frequently the topic of Delphi investigations, but rarely have participatory roles in the knowledge produced about them. The lack of lived experience representation results in material impacts, given the wide practical application of Delphi knowledge. Importantly, it raises pertinent questions about whether the research methods used in intellectual disability research align with the principles and spirit of the CRPD (2007).

We have examined the modifications made to facilitate inclusion of people with intellectual disability in several studies. However, some authors considered it not possible for people with intellectual disability to have a direct participatory role, describing such modifications as ‘non-existent’ (Frankena et al., 2019, p. 2), or the method itself as ‘not suitable’ (Hempe et al., 2013, p. 678).

These claims are challenged by the nine studies where people with intellectual disability did have direct participatory roles in the Delphi method. Approaches to modifications that facilitated their access were varied, which reflects people with intellectual disability do not have homogenous access needs. To develop modifications that enable participation, engagement at an individual level is needed with people with intellectual disability, during both research design and implementation. It is worth noting inclusive research studies are more commonly drawn upon in the discussion and elaboration of results. Rix and colleagues (2020) emphasise access and participation for people with intellectual disabilities requires deliberate and ongoing facilitation at all stages of research. The present review similarly contributes to participatory action/ inclusive research knowledge by demonstrating how people with intellectual disability can contribute to the design development of Delphi studies.

The discussion which follows outlines how the six modification groupings identified in our review were enabled in practice. While these groupings emerged through the analysis of Delphi studies, they may broadly apply to methods in intellectual disability research, particularly those employing a group consensus approach. In doing so, these modification groupings offer a practical guide to enable greater direct participation of people with intellectual disability, to ensure lived experience knowledge from this perspective is captured.

Communication modifications involve changes to written or verbal communications beyond data collection, including consent, participant information, and dissemination of findings. Except for Lemmey and colleagues (2021), the eight other studies eligible for this stage of review all reported communication related modifications, making this the most common modification type. Examples of communication modifications included using easy read formats, providing examples or synonyms to assist with understanding, reading materials aloud, and using plain language. For instance, Vicente and colleagues (2019) used a group interview to assist with comprehension and provided information in oral format. In addition to telephone and email, Khayatzadeh-Mahani and colleagues’ (2020) utilised multiple modes of communication including in-person meetings and inviting participants to visit. Likewise, Bonell and colleagues (2012) used several adaptive communication strategies, including making an informal assessment of communication abilities while gaining written consent, using pictorial diagrams to assist with understanding, and reading aloud Delphi items during face-to-face interviews.

Relational/Recruitment modifications involve recruitment through more than one organisation and/or specific details regarding the mode or focus for recruitment, details around efforts toward relationship building, and specifying intentionally choosing researchers with experience working with people with intellectual disability. Several studies highlighted the importance of building relationships with people with intellectual disability and disability organisations to increase engagement. For instance, Khayatzadeh-Mahani and colleagues (2020) partnered with disability organisations and visited potential participants to build trust and create a “safe space” for participation. Noorlandt and colleagues (2021) recruited participants through two distinct groups/organisations, while Chaplin and colleagues (2012) allowed members of an existing community group to choose whether or not to participate in data collection, which occurred during their regular community group meetings. Bonell and colleagues (2012) specifically chose research team members who had experience working with people with intellectual disability, to administer Delphi item interviews.

Material/for comment modifications involve instances where people with intellectual disability have a material impact on the knowledge produced, the administration of research, or knowledge translation. Examples of this modification include providing feedback on the layout, language, and images used in research materials or contributing to the development of content. For instance, Chaplin and colleagues (2012) asked participants to provide feedback on an accessible prototype template, commenting on the layout and the support needed to use it. Bonell and colleagues’ (2012) included a personalised survey in the second round of their study, allowing participants to provide feedback on each item individually. Likewise, Noorlandt and colleagues’ (2021) sought feedback on a draft document, which resulted in adjustments being made to the icons used to assist with understanding.

Digital/technology modifications involved changes to technical or digital elements that were identified as barriers, such as internet access and email accounts. For example, Chaplin and colleagues (2012) presented their in-person method of consultation as an access modification due to a reported difficulty that participants had in accessing computers and email accounts. Likewise, Noorlandt and colleagues (2021) reported that a survey was completed on paper in a group to address accessibility issues related to using digital technology. Conversely, Lyons and colleagues (2018) recorded participants’ oral responses and comments online to overcome access barriers presented by in-person participation. The variation in these examples reflect that for some participants, the online administration of the Delphi was considered more accessible, while for others reasoning was offered for face-to-face delivery. Importantly, people with intellectual disability do not have homogenous access needs. This variation highlights the importance of considering the needs of the individual participants engaged in research design and implementation.

Specific techniques involved facilitating methods and/or approaches that are intentionally modified to support the participation of people with intellectual disability. For example, Chaplin and colleagues (2012) asked participants to vote on the best indicators and effective coping strategies, and similar answers were merged with group agreement. In Vicente and colleagues (2019), a group interview was conducted to assist with comprehension and provide support, and a short time frame was used to minimise respondent fatigue. Khayatzadeh-Mahani and colleagues (2020) used a focus group to allow discussion of personal experiences, while Noorlandt and colleagues’ (2021) used a focus group to discuss the first draft document.

Finally, studies that provided Responsive/flexible modifications allowed options for participants to negotiate modifications that suited their individual needs and preferences. Importantly again, these were presented as options for participants, rather than a one-size-fits-all approach. Chaplin and colleagues (2012) and Bonell and colleagues (2012) report providing explanations of Delphi items and elaborations where necessary. Lyons and colleagues (2018) offered a flexible option for those who had difficulty attending in-person sessions and conducted evaluations over the telephone for three participants. Khayatzadeh-Mahani and colleagues (2020) identified the allocation of time and resources for participant engagement, including suitable locations and travel.

These examples encouragingly demonstrate that the Delphi method can be modified to facilitate the direct participation of people with intellectual disability. The most common modification (Communication) involved specific adjustments during data collection. In many cases this pertained to the format of the items in which Delphi scoring was delivered. Researchers met with people with intellectual disability (online, telephone or face-to-face) to read items aloud and/or explain key terms in plain English. Deliberate efforts were made to ensure items were understood, through images/diagrams or the opportunity for participants to ask questions. Some studies reported how participation was structured to minimise fatigue or provide participants with space to discuss their personal experiences as a pragmatic measure toward comprehension. In other cases, authors attempted to meet these aims by developing partnerships with disability organisations and/or employing a dedicated advisory group for oversight.

Some studies describe other, bespoke conditions to include people with intellectual disability in peripheral aspects, rather than adapting the methodology to apply equally to all participants. Most studies conferred limited authoritative power to people with intellectual disability, by reducing their collective authority (relative to the expert backgrounds of the rest of the panel and the definition of consensus) and/or segregating their participation outside the Delphi method.

Limitations of the reviewed studies

Papers with identified participatory roles contained scant details on intersectional identities, the nature of participation and how this influenced the Delphi. This patterned lack of detail reflects similar findings of other intellectual disability focused systematic reviews (Bigby, 2012; Caton and Chapman, 2016; Mullen, 2018; Rix et al., 2020). The quantum of ‘unclear’ participants with intellectual disability (identified in Supplementary 10) demonstrates the frequent lack of information provided, including details pertaining to the numbers of people with intellectual disability involved in a study, whether expert/consultant groups were discrete or overlapping and the methods of recruitment.

This issue in reporting was exacerbated by unclear language or incomplete descriptions of the panel composition. For example, in Khayatzadeh-Mahani and colleagues (2020 p. 5), the proportion of people with intellectual disability who participated in the study overall is unclear because ‘PWDD and their families/caregivers’ are collapsed into the one group. Likewise, Lyons and colleagues (2018 p. 210) describe the participation of an ‘individual with a disability/self-advocate’, meaning the actual number of people with intellectual disability who have participated is unclear.

In instances where people with intellectual disability clearly had direct participatory roles, there are few examples of diversity in representation from this broad group. Most authors reported the gender of participants, and some identified the exclusion of more ‘profound’ intellectual disability as a limitation of their work. Vicente and colleagues (2019 p. 214) notes convenience sampling was used, and acknowledge this limits the representativeness of participation, both in terms of gender and intellectual disability aetiology. In such cases, it may have been outside the authors’ capacity to purposively recruit for diversity among people with intellectual disability. This suggests a gap in existing Delphi studies and their limitations in intersectionally relevant knowledge production.

Strengths and limitations of the current review

The strengths of this review include employing the relative frequency approach to synthesise panel composition, which may be replicated in future Delphi method studies. Clear definitions for inclusion/exclusion criteria and data extraction have facilitated meaningful qualitative examination. However, we acknowledge that, even with this care, there are limitations in applying a contemporary definition of intellectual disability across time and space. Likewise, it was not within the scope of our review to analyse research investigating populations <18 with intellectual disability and/or other cognitive conditions.

Conclusion

People with intellectual disability are commonly excluded from research about them that uses the Delphi method. When they are included, it is mostly in numbers that limit their potential impact on the study findings, relative to paid-experts. The established expectation for modifications within Delphi methodology occasions significant scope for facilitating access needs. This has not yet resulted in an increased participatory role for people with intellectual disability. Studies which have included people with intellectual disability in direct participatory roles influencing the Delphi, offer a foundation to inform the design of future inclusive approaches. However, current practices have been limited to facilitating access in terms of data collection (as experts and/or consultants). There are opportunities to build on this foundation by incorporating people with intellectual disability in co-design roles as peer researchers assisting with Delphi administration.

Supplemental Material

Supplemental Material - The role of people with intellectual disability in intellectual disability research: A systematic review of Delphi studies

Supplemental Material for The role of people with intellectual disability in intellectual disability research: A systematic review of Delphi studies by Diana K Piantedosi, Amie O’Shea in Journal of Intellectual Disabilities