Abstract
People with lived experiences are often excluded from development of solutions and decision-making related to health research and policy. To describe and demonstrate how high-quality engagement supports partner and project outcomes. The ultimate project outcome was to identify health priorities desired by people with intellectual and/or developmental disability (IDD) and the people who support achieving those priorities, including caregivers, clinicians, and payers/regulators. This capacity-building project implemented and evaluated methods of engagement of IDD self-advocates, caregivers/partners, clinicians, payers/regulators, and researchers. Our reliance on a variety of engagement approaches, but particularly graphic illustration and other visual engagement, yielded productive conversations to advance areas of priority. Partners felt satisfied with engagement and continued to participate at multiple points throughout the 2-year project. We identified nine illustrated priority health outcomes useful for research, practice, and policy change. Our engagement and priority-setting approach resulted in findings that partners found compelling personally and professionally.
Plain Language Summary
Keywords
Introduction
Individuals with intellectual and developmental disabilities (IDDs) comprise a large and growing (7.4–16 million), heterogeneous group of people with a variety of functional, developmental, and medical needs (Zablotsky & Black, 2020). Often, clinicians categorize individuals with IDD based on diagnosis (e.g., “Down syndrome”) or the level of cognitive functioning (e.g., “mild intellectual disability”) for practice and research (Sulkes, n.d.); however, diagnosis and level of functioning are not the central component of a fulfilled and healthy life, nor do they define the outcomes that matter to individuals with IDD (Brown et al., 2013).
Patient-centered outcomes research (PCOR) requires the engagement of partners with lived experience to evaluate the impact of desired interventions on outcomes that matter, usually implemented in real-life settings with clinicians who are practicing within payer networks. Although PCOR involving persons with IDD has increased in recent years (e.g., Bendixen et al., 2022; Kramer et al., 2024; Schwartz et al., 2020; St. John et al., 2018, 2022), this community has historically been excluded from meaningful engagement about what matters and from intervention studies needed to address prioritized health outcomes (Ally et al., 2018; Feldman et al., 2014). PCOR capacity-building requires identification of outcomes that matter to persons with IDD and their families, clinicians, and payers or regulators, who fundamentally need to address the priorities through research and practice. Individuals with IDD point out that not being involved limits their autonomy and self-determination and are calling for changes (Boyd et al., 2015; Elbard, 2015). Methodological approaches for effective engagement in pre-research (the prioritization of topics, the identification of outcomes, and the planning for future research collaboration) and actual engagement during PCOR are limited. Publications in this area illustrate innovation in using multiple adapted and tailored approaches for iterative feedback and information gathering (e.g., Nicholson et al., 2023), technology approaches (e.g., Hwang et al., 2022), peer supports (e.g., Pfeiffer et al., 2024), and prioritization processes (e.g., Davis et al., 2022). Furthermore, engagement is often seen as a stand-alone activity, when engagement activities and the quality of engagement directly influence partner outcomes and future research outcomes. This methodology article aims to describe and demonstrate how high-quality engagement supports partner outcomes and project outcomes. The project outcome was to identify health priorities desired by people with IDD, caregivers or care partners (referred to “caregivers” throughout), clinicians, and payers/regulators for future PCOR.
Methods
Ethics Review
This project was deemed Not Human Subjects Research by Augusta University. Partners from the various communities were considered co-developers of engagement approaches and the resulting products and therefore were offered compensation and authorship during project implementation.
Project Team
The positionality of the project team is broad and includes people with IDD, caregivers of people with IDD, academic researchers, clinicians who specialize in working with people with IDD, general clinicians, payers, and non-profit leaders in advocacy and policy. The project team included a Leadership Team, 5-member Core Team, and 13-member Advisory Council. The project team co-led the project and informed all actions. All members of the project team were offered compensation for their time in accordance with published preferences (Shore & Benevides, 2018).
Geographic Location
This project was of U.S. national scope.
Partner Identification
The initial project was funded to include 58 partners from four groups in Year 1 to identify priority areas: people with IDD, caregivers of people with IDD, clinicians delivering health care to people with IDD, and payers or regulators of health care. To identify interested partners, we used known networks to invite or share registration information with advocacy groups, researchers, health care providers, and disability networks. We intentionally invited partners representing different lived experiences. For example, we asked known networks to share with people with different racial or ethnic backgrounds, genders, ages, geographic location, or clinical experience. Interested individuals completed a brief registration survey. These partners were compensated hourly for their time.
Additional funding in Year 1 allowed us to replicate our approaches with a second group of partners to engage in priority-identification activities. This phase was led by a different university team to triangulate initial priorities. This second phase included 73 additional people and is described elsewhere (Nicholson et al., 2025). In Year 2, we had funding to include 35 partners in agenda-setting and prioritization activities. We also hosted meetings with 27 policymakers to understand their perspectives related to the identified priorities and agenda (described in Pham et al., 2024).
Setting
We used virtual engagement methods for Year 1 activities due to COVID restrictions by the funder. In Year 2, we met virtually and also held a 1-day meeting in Washington, D.C. Travel reimbursement (e.g., flights, hotel, and food) and compensation for time were offered to all partners. Partners with IDD were also offered travel compensation for a support person traveling with them.
Capacity-Building Engagement Process
The design of this project, Individuals with Intellectual and/or Developmental Disability Engaged, Aligned, and Leading (IIDDEAL), followed the early steps of Collective Impact (Kania & Kramer, 2011) for building capacity: (a) development of norms; (b) developing consensus on the agenda with all involved parties; and (c) identifying measures that all parties agree are important. In this manuscript, we describe Steps 1 and 2. To develop norms and structures, we established a Leadership Team, Core Team, and Advisory Council comprised of partners from the four groups we were engaging (people with IDD, caregivers, clinicians, and payers/regulators). The Leadership Team met weekly for 2 years, and the Core Team met monthly for 2 years. Our Advisory Council met quarterly for 2 years and provided insight into what we were learning prior to moving to the next stage of the project. These teams worked to generate written norms for working together, developed a structure for identifying priorities and the questions to elicit priorities, generated networks of people to invite to the project, and helped share information about the project. The Core Team developed and approved the meeting discussion questions, participated in leading the meetings eliciting priorities, and interpreted what people shared. To develop consensus on an agenda related to health outcome priorities, we did the following: (a) identified priority outcomes via a series of 1-hr virtual meetings, first with people with IDD 1-on-1, then in four separate virtual group meetings with key partners held 1-month apart (people with IDD, caregivers, clinicians, payers, or regulators, sequentially); (b) after each group meeting, we shared resulting priorities, requested feedback, and revised illustrated priorities based on feedback; (c) engaged in prioritization activities and a 1-day in-person agenda-setting meeting with 35 partners from diverse perspectives (described in Pham et al., 2024); and (d) met with 27 policymakers to identify opportunities or barriers to the agenda (described in Pham et al., 2024). Thus, payers/regulators and clinicians contributed additional priorities and/or health care elements supporting priorities but did not subtract from any priorities previously identified by people with IDD or caregivers. Over the course of the two-year project, we held thirty-seven meetings with partners. For priority identification and revision: 16 1-on-1 meetings to elicit priorities from adult partners with IDD, one group meeting with adult partners with IDD, one group meeting with caregivers, two group meetings with clinicians, and two group meetings with payers. During this phase, approximately 50% of the partners with IDD requested preparation time with our team to learn about what to expect prior to 1-on-1 meetings and/or group meeting. For consensus-building, we held five meetings over a 6-month period. After the two-year project, we hosted two policy summits.
During our group meetings, partners were allowed to build on the previous priorities identified by earlier partners but were not allowed to remove any priorities identified by prior groups. We used thematic classification of ideas in Miro® and MaxQDA to identify priority areas and descriptive quotes as well as graphic illustration of priorities to summarize what we learned. We invited feedback on the final priorities, goals, and illustrations from all project partners and invited authorship on the resulting health outcomes framework. We then publicly released the health outcomes framework and invited community input through sharing of the content with known networks.
Evaluating Engagement Quality and Partner Outcomes
Patient-Centered Outcomes Research Institute (PCORI; 2023)’s Engagement Theory of Action, outlined in Table 1, illustrates the complex interplay between engagement and project outcomes. This model outlines Engagement Activities (e.g., planning and supporting partners, managing engagement, facilitating diversity in partnerships, and supporting engagement at a high level throughout the research life cycle) and Engagement Quality (e.g., partner experiences and inclusiveness). These directly influence Partner Outcomes (e.g., sustainability, attrition/retention of partners, capacity to continue collaborating) and Project Outcomes (e.g., perceptions of the importance of the project/research, acceptability of the output). Engagement quality was measured in terms of use of engagement resources, participation during meetings, accommodations requested and implemented, anonymous satisfaction surveys (Supplemental Appendix 2 survey questions), and unsolicited email or other communications. Descriptive statistics and frequency of topics raised in open-ended questions were used to understand survey responses. Partner outcomes were also captured through numeric retention/attrition and agreement to continue to participate via a signed Memorandum of Understanding (MOU).
Engagement Activities, Engagement Quality, Partner Outcomes, and Project Outcomes.
Note: Arrows indicate authors’ perceived influence of engagement activities on engagement quality, partner outcomes, and project outcomes.
Adapted from: Patient-Centered Outcomes Research Institute (2023).
Project Outcomes
The primary project outcome was the identification of health outcome priorities of value to all project partner groups. We also aimed to ensure that communication of outcomes was accessible to all partners.
Results
Level of Engagement Reach and Diversity of Perspectives
In Phase 1 of the project, we budgeted for 58 partners (16 people with IDD, 16 caregivers, 13 clinicians, and 13 payers/regulators). We over-invited partners with expected attrition between invitation to join and attending the first meeting. Partner demographics illustrate diversity in geographical reach, age, race, and ethnicity (Table 2). Most caregivers and clinicians who attended were female. Payers/regulators had a greater proportion of White individuals compared to the other partner groups. Of our collaborators with IDD, four (26.7%) reported using assistive/augmentative communication as their primary form of communication, and two (12.5%) attended the meeting with a caregiver/partner support person.
Registered Partners for Priority-Setting Meetings.
Some questions were multiple-select so totals may not equal 100%.
Measures of Engagement Quality
We met our engagement goal with a total of 58 partners attending planned Year 1 meetings, inclusive of 15 individuals with IDD (94%), 16 caregivers (100%), 11 clinicians (85%), and 16 payers/regulators of health insurance services (123%). Optional preparation meetings with our team to review content and anticipated discussion questions were attended by 100% of our partners with IDD; no caregivers, clinicians, or payers/regulators attended preparation meetings. 100% of partners contributed or participated during group meetings reviewing priority outcomes either through spoken language, chat, or use of Miro® Board. Two partners with IDD requested to schedule separate (non-group) meetings to review priorities due to communication preferences; these requests were honored. Adaptations requested for these meetings included using brief questions of less than five words, use of yes/no questions, and use of visuals on the screen to ascertain which content elicited greatest attention by the partner.
Response rates for Year 1 post-meeting survey completion varied: 56% of people with IDD, 38% of caregivers, and 31% of payers/regulators. No clinician data are reported, as the clinician survey failed to launch due to technical difficulties in Qualtrics. Satisfaction with communication was high: partners with IDD (M = 4.89, SD = 0.31, range = 4–5), caregivers (M = 5.00, SD = 0, range = 5–5), and payers/regulators (M = 4.6, SD = 0.49, range = 4–5). Satisfaction with listening to partners was high: partners with IDD (M = 5.00, SD = 0, range = 5–5), caregivers (M = 4.33, SD = 1.49, range = 1–5), and payers/regulators (M = 4.6, SD = 0.49, range = 4–5). The caregiver respondent rating “1” (not satisfied) for listening is possibly an error because their open-ended comment was “Made to feel relaxed and comfortable to participate.” Satisfaction with support for participation was high: partners with IDD (M = 4.89, SD = 0.31, range = 4–5), caregivers (M = 5.00, SD = 0, range = 5-5), and payers/regulators (M = 4.2, SD = 0.40, range = 4-5). 100% of individuals participating in meetings and responding to surveys indicated that visual tools and graphic illustration supported their engagement and understanding. Preparation videos sent prior to meetings were rated as helpful by 88% of partners with IDD and 75% of caregivers, and 1-on-1 preparation meetings were rated by 100% of partners with IDD as helpful. Closed captioning was helpful for 38% of partners with IDD and 50% of caregivers; most people reported not using it. No engagement approaches were rated as not helpful. Qualitative feedback received on anonymous surveys or through communications with partners revealed that “drawings” were mentioned most frequently as what people liked the most about being able to share their health priorities. For example, people with IDD said: “[I liked] the artist who was drawing what I was doing—explained my story in my eyes and their eyes.” Reasons why people expressed appreciation for the graphic illustration included, “I felt heard. Thank you!,” and “I appreciated the opportunity to share my experiences. I loved the drawing and the advocacy element of the exercise.” Caregivers used the following words when asked to share what they liked most: “The collective collaboration method used” and “Engaging.” People found value in being in safe and supportive spaces. One caregiver shared, “It was comfortable to share. In our breakout session, there were a couple parents with similar challenges, it was comfortable to be honest. The sticky note process was cool, first time I’ve seen that virtually. I feel like we covered the questions thoroughly.” And similarly, payers/regulators reflected that what was most helpful about the meeting was “hearing from other payers!” Another caregiver of a non-speaking adult who was supporting her child in the process noted that her child spent a long time looking at the illustration on the screen and had reduced wandering during the meeting when graphic illustrations were used on screen. For people with IDD and caregivers, there were no negative responses related to engagement when asked “What did you like least?”; only “Not Applicable” responses were received. Payers/regulators reported lack of time for discussion as what they liked the least.
Retention in Subsequent Activities—Measure of Partner Outcomes
Of the 58 individuals attending initial meetings in Year 1, 43.1% replied with interest and participated in subsequent project activities 2 years after their initial engagement. At the conclusion of the project, we invited individuals and organizations whom we had engaged during the project to sign a Memorandum of Understanding agreeing to participate in future activities related to the project; 42 partners signed MOUs. 90% of MOUs agreed to share the priorities with their networks; 92% agreed to learn more in the future about next steps; and 55% agreed to share the priorities on their websites publicly. Our Core Team (7 members) and Advisory Council (13 members) met regularly during the 2-year project. Attendance ranged from 50% to 100% for Core Team members attending monthly meetings; all Core Team members/organizations initially invited were retained at the end of the project period. Advisory Council attendance at quarterly meetings ranged from 54% to 100%; all Advisory Council members were retained at the end of the project period.
Priority Health Outcomes that Matter that are Important and Accessible—Measure of Project Outcomes
Nine primary health outcomes were reported by key partners as critical to positive health for people with IDD (Supplemental Appendix Figures): (a) Making healthy choices; (b) Improving my emotional or mental health; (c) Improving my physical health, reducing pain, and increasing energy; (d) Addressing sexuality, gender, reproductive health, and parenting; (e) Being able to do the things I love or need to do; (f) Family care partner/giver wellness and support; (g) Workforce development of health care providers; (h) System supports; and (i) Payer and regulator priorities.
Discussion
Our approach to engagement was uniquely intended to create action among diverse audiences: people with IDD, caregivers, clinicians, and payers/regulators. In this article, we aimed to describe how engagement activities resulted in perceived engagement quality, partner outcomes, and our project outcome of identifying health outcome priorities for future PCOR. We identified that partners felt graphic illustrations and visual engagement approaches were easy to use and engaging for diverse partners. Although health literacy approaches to incorporate visuals into patient care are not new (Mansky, 2018; Osborne, 2006), the intended use of illustrations was innovative and highly successful in gaining engagement from different partners. Illustrations were useful in both allowing partners to tell their story, as well as in communicating priority outcomes in accessible ways. We found that partners appreciated having easy-to-understand illustrations that communicated priorities efficiently and effectively. Illustrations may be an important and distinct approach that is different from using Plain Language, a frequently used approach to communication with diverse audiences.
Open-ended feedback from partners on surveys revealed that partners appreciated having a similar peer-group in which to initially discuss priorities. This approach was used prior to a mixed-partner group meeting. Open-ended feedback as well as participation data during the meetings suggests that similar peer-groups create safe spaces, and psychological safety could be an important component to ask about in future engagement surveys. We debated whether to mix partners with different lived experiences early in the project (vs later). Our findings may be useful to other teams hoping to bring together partners with many different perspectives.
There are some limitations to this work. This project was not designed as a research study, so we did not implement methods as traditionally found in scientific articles, such as a sampling plan or engage in a traditional qualitative inquiry. Furthermore, engagement activities are not static nor always methodologically defined in a “protocol” because the nature of engaging people is to listen and learn and to evolve approaches based on how engagement is proceeding. In listing the broad engagement activities, we recognize that this fails to quantify the extent of time spent working with partners to assure their input was reflected in each iteration of the work. By defining engagement outcomes narrowly through things that can be counted, we are unable to illustrate the power of relationships that were developed and supported the return of our partners through multiple rounds of meetings. These types of impacts are found in very personal and humbling emails about the impact of the work on people, none of which are appropriate to share in a manuscript.
Conclusions
Through activities aimed to promote high-quality engagement, partner outcomes, and project outcomes, we identified consensus about the key health priorities desired in practice and policy by four groups: people with IDD, caregivers, clinicians, and payers/regulators. Graphic illustration and other visual approaches resulted in communication of actionable priorities accessible to a wide variety of audiences (Supplemental Appendix). It is feasible and important to engage a variety of parties with different ideas of “outcomes that matter.”
Supplemental Material
sj-docx-1-otj-10.1177_15394492251385448 – Supplemental material for Engagement to Identify Health Priorities of People With Intellectual and/or Developmental Disability
Supplemental material, sj-docx-1-otj-10.1177_15394492251385448 for Engagement to Identify Health Priorities of People With Intellectual and/or Developmental Disability by Teal W. Benevides, Hoangmai H. Pham, May-Lynn Andresen, Madelyn R. Bahr, Tim Corey, Joanne Nicholson, Kristen Faughnan, Jennifer E. Jaremski, Carolyn Langer, Vincent Siasoco, Alexis Hernandez-Hons and Stephen M. Shore in OTJR: Occupational Therapy Journal of Research
Footnotes
Acknowledgements
This work is a collective body of activities through which many people shared their views and contributed to materials, including Jeffrey Bennett, Jordan Sestak Bruce, Oanh Bui, Maverly Dennis, Vanessa Gonzalez, Laura Hart, Sue Hingle, Patti Killingsworth, Susan Platkin, Denise Resnik, and others on our Advisory and Core Team. We have asked all individuals who contributed in our IIDDEAL meetings whether they wanted their names to be listed as contributing authors or whether they wished to remain unnamed. We acknowledge the contributions of the unnamed contributors as important to this work. AI was used for the formatting of references in this manuscript.
Ethical Considerations
This project’s activities were reviewed by Augusta University’s Institutional Review Board protocol 1830929-1 and were identified as “Not Research.”
Consent to Participate
Informed consent is not applicable to this project.
Author Contributions
TWB and HHP: conceptualization, information curation, formal analysis of partner input, funding acquisition, methodology, project administration, resources, supervision, and writing—original draft.
TC: resources, information curation, and visualization.
M-LA, MRB, JN, KF, and JEJ: project administration, information curation, and writing—original draft.
SMS, VS, CL, and AH-H: methodology and writing–original draft.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by Patient-Centered Outcomes Research Institute (PCORI), Eugene Washington PCORI Engagement Award, EASC-IDD-00241 to Augusta University, and is also funded by Arnold Ventures to Institute for Exceptional Care. This work is solely the responsibility of the project leads and does not necessarily represent the views of PCORI®, its Board of Governors, or its Methodology Committee.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Data sharing is not applicable to this article, as no data sets were generated or analyzed during the current study. It was considered “Not Research” by the funder and institution.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
