Sage Journals: Discover world-class research

Abstract

Purpose: The purpose of this project was to identify, develop, and assess the feasibility of a complex intervention to support independent public transportation for individuals with cognitive disabilities in a midsized urban setting. Methods: We used the Medical Research Council framework to develop and evaluate a complex intervention. The complex intervention was an education and training program that supported personal navigation for individuals with disabilities (PNID) and was part of the Smart Columbus Mobility Assistance for People with Cognitive Disabilities (MAPCD) project. The identification, development, and feasibility of the PNID education and training program was iterative and included seven stages, which started with engaging with external stakeholders and ended with describing future directions for the implementation of the complex intervention. Results: The result was an evidence-based, theory-informed, and stakeholder supported education and training program to support public transportation for individuals with cognitive disabilities. The intervention was based on frameworks, models, and theories including a sociotechnical architecture model. The sociotechnical architecture was operationalized using the WayFinder system, a personal transportation assistant that consists of both a smartphone application and a web-based portal. The education and training program consisted of five activities. Conclusions: The PNID education and training program and MAPCD project provide a framework for the identification, development, implementation, and evaluation of sociotechnical architectures that support independent public transportation for individuals with disabilities in midsize urban settings.

Keywords

public transportation innovative public transportation services and technologies disabilities information technologies (cellphone based apps)

Individuals with disabilities and older adults often have difficulty accessing public rights-of-way (sidewalks, street crossings), shared-use paths, and public transportation while traveling from one location (e.g., home) to another location (e.g., work). The lack of efficient transportation available to these individuals, particularly individuals with cognitive disabilities, creates barriers to gaining and maintaining employment in the community ( 1 ). Cognitive disabilities encompass a range of impairments and include intellectual disabilities, and can result from brain injuries, Alzheimer’s disease and other dementia diagnoses, stroke, and mental illness ( 2 ). The availability of accessible transportation for individuals with cognitive disabilities is an important part of eliminating barriers and creating employment and educational, social, and recreational opportunities. Transportation is critical to successful employment, and individuals with disabilities rely on public transportation at a higher rate than individuals without a disability. People aged 18 to 64 with disabilities use local transit including buses, subways, and commuter rail, for a higher share of trips than people without disabilities. Conversely, some individuals with disabilities report not using local transit at all. In fact, they report not being able to access transportation, at a rate 2.5 times greater than people without a disability ( 3 ). With regard to the different types of transportation, 18.4% of individuals with disabilities identified fixed-route bus service as their primary mode of mobility. Finally, nearly 50% of individuals with disabilities reported the public transportation system was inadequate, 26% indicated that routes to stops and stations were inaccessible, and 20% indicated inaccessible stops and stations ( 4 ). The comparatively higher reliance on public transportation in comparison to individuals without disabilities, in particular fixed-route bus systems, and the high rate of barriers to accessing public transportation demonstrate the need to develop and implement programs that support transportation for individuals with disabilities.

Developing Complex Interventions

Interventions, such as programs that support independent travel in the community, might leverage technology-based solutions (e.g., personal navigation applications) to support individuals with disabilities to access public transportation. We sought to develop a complex intervention, as defined by the Medical Research Council (MRC), to educate and train individuals with disabilities to access public transportation safely ( 5 ). Several dimensions can qualify a program to be considered a complex intervention. These include, but are not limited to, the “number and difficulty of behaviors required by those delivering or receiving the intervention,”“number and variability of outcomes,” and “degree of flexibility or tailoring of the intervention permitted” ( 6 , p. 7). Often, complex interventions are described as interventions that contain several interacting components. Though the concept of complex interventions was originally developed to describe interventions in the healthcare setting, it has been applied equally to describe interventions in a community setting ( 5 ). In developing a complex intervention to support independent transportation for individuals with cognitive disabilities, the research team partnered with local government and academic and community organizations within a midsize urban metropolitan region.

Personal Navigation Intervention for Individuals with Cognitive Disabilities

The Smart Columbus program, a $40 million citywide grant award from the United States Department of Transportation (U.S. DOT), was a program to develop and implement emerging transportation-related technologies. The program involved several projects supporting the transportation goals of residents and visitors of the city of Columbus. One of the projects was the Mobility Assistance for People with Cognitive Disabilities (MAPCD). The purpose of the MAPCD project was to “build upon COTA’s existing framework to maximize mobility and independence while maintaining safety. The [MAPCD] project [sought] to provide a solution [that allowed] users with cognitive disabilities to safely transition from using paratransit services to using fixed-route bus service, and to attract new users with cognitive disabilities who were not previously using paratransit services” ( 7 ). To meet this purpose, the MAPCD project aimed to evaluate and implement smartphone technology to minimize transportation barriers for those with cognitive disabilities. The MAPCD project team consisted of the City of Columbus, the Central Ohio Transit Authority (COTA), and The Ohio State University (OSU) in addition to numerous stakeholders from across the Columbus metropolitan region.

The Smart Columbus MAPCD project proposed a personal travel assistant (PTA) as a technology-based intervention to support transportation for individuals with cognitive disabilities. The PTA is based on a sociotechnical architecture that supports the traveler and the traveler’s support community (e.g., travel trainer, transit operators, care partners and other stakeholders) during transportation ( 8 ). The sociotechnical architecture leverages the following technologies: 1) mobile, wireless, location-aware smartphones, and 2) mobile GPS technology on transit vehicles. The PTA incorporates a mobility agent and a support community feedback system. The mobility agent generates just-in-time prompts and detects interruptions in the trip (e.g., off-route, delays) based on bus information and smartphone information. The support community feedback system provides real-time feedback to the traveler’s support community based on the bus and traveler position ( 8 ). The PTA provides a foundation for a complex intervention that meets the broader goals of supporting transportation for residents and visitors of Columbus, with a focus on individuals with cognitive disabilities.

In partnership with the MAPCD project team, the OSU team proposed a personal navigation for individuals with disabilities (PNID) education and training program based on the Human, Activity, Assistive Technology (HAAT) model and the assistive technology (AT) service delivery process. The HAAT model describes the human, activity, AT, and context, and is designed to support the implementation of technology for individuals with disabilities ( 9 ). The AT service delivery process describes the iterative steps necessary when implementing technology for individuals with disabilities ( 9 ). The combined PTA, HAAT model, and AT services delivery process provide the foundation for a complex intervention to support independent transportation for individuals with cognitive disabilities.

The purpose of the PNID program was to identify, develop, and assess the feasibility of a complex intervention to support independent public transportation for individuals with cognitive disabilities in a midsize urban setting. The development of the PNID program was guided by the MRC framework for the development and evaluation of complex interventions ( 10 , 11 ) and aligns with the preconditions and pre-implementation phases of the Replicating Effective Programs (REPs) framework ( 12 , 13 ), which is one of many implementation science frameworks ( 14 ). The REPs framework provides a structure to develop systematic and effective strategies and interventions. There are four phases: preconditions (e.g., identifying a need, target population, and intervention structure and strategy), pre-implementation (e.g., intervention packaging and community input), implementation, and maintenance and evolution. This report provides a transparent and detailed description of the identification, development, and feasibility-testing processes for the PNID program as an implementation strategy, which could be used by other midsize urban settings. A detailed description of the results of the intervention are beyond the scope of this paper and can be referenced in an article by Culter Harris and colleagues ( 15 ).

Methods

We used the seven stages described by Müller and colleagues, which provides an example implementation of the MRC framework, to describe the development and implementation of the PNID program ( 16 ). Though the article was published following the completion of the MAPCD project, it provided an excellent template for describing the PNID program (Figure 1). In stage one, the key stakeholders and team members were identified. Next, we determined the existing evidence on similar interventions and identified the theoretical foundation of the intervention (Stages 2 and 3). Then the training was designed, and the trainers were provided training on how to implement the intervention (Stage 4 and 5). In Stage 6, we developed processes to address the feasibility and acceptability of the intervention-based feedback from the travelers, travel partners, program instructors, and community mobility professionals. In Stage 7, we identified future directions for the implementation and sustainability of the complex intervention in midsize urban settings. This study was approved by OSU’s IRB Committee.

Figure 1.

Gant chart of stages of intervention development of the PNID program in the MRC framework.

Results

Stage 1: Identifying and Establishing Stakeholder Roles and Contributions

Stage 1 consisted of identifying and establishing stakeholder interest in supporting transportation for individuals with disabilities. The identification of stakeholders occurred on an ad hoc basis before the start of the MAPCD project. The Ohio State University Wexner Medical Center (OSUWMC) Assistive Technology Center (ATC) identified a persistent issue through the Driver’s Rehabilitation Program. The issue focused on the lack of transportation options for individuals with disabilities who could not acquire a driver’s license, did not have access to a personal vehicle, or did not want to drive. Through networking activities and serendipity, a core group of stakeholders started working together to better understand the issue and identify potential solutions to support individuals with disabilities.

Foundational activities to address the issue included the Pre-trip, Departure, En-route, and Arrival (PDEA) program, the Pre-Vocational Integrated Education and Campus Experience (PIECE) program, and the Community Wayfinding for Individuals with Disabilities (CWID) program (Figure 1). The foundational activities provided the motivation for the PNID program. The PDEA program was an unfunded project that provided the foundation for future collaborations, and increased awareness of the need for accessible transportation beyond the physical context of ramps, kneeling buses, wheelchair tiedowns, occupant restraint systems, and signage. The PIECE program was one of those future collaborations and was implemented through the OSU Nisonger Center in 2017 and 2018. The PIECE program was designed to provide individuals with intellectual and developmental disabilities (IDDs) with the vocational skills that are necessary for employment. As part of the PIECE program, individuals with IDDs completed an internship with a local company or organization. The program instructors in the PIECE program identified a need to support independent public transportation for individuals with IDDs. The PIECE program led to the creation of the CWID program, which was submitted to the OSU Connect and Collaborate Grants Program in 2018. Though it was unfunded, it further increased awareness among community stakeholders of the need for accessible transportation options and created the framework that later became the foundation for the PNID program.

In parallel with the activities that were taking place at OSU/OSUWMC, the City of Columbus won the U.S. DOT Smart City Challenge in 2016, which became Smart Columbus. Smart Columbus was funded through U.S. DOT and included eight individual projects. OSU was a contributing partner in the submission and would support the Smart Columbus program through in-kind investments. The parallel activities across the OSU/OSUWMC communities and the City of Columbus led to a natural partnership to develop and evaluate the PNID program under the auspices of the MAPCD project.

Stage 2: Determining Existing Evidence on Interventions

Several of the activities during Stage 2 overlapped with the activities in Stage 1. As noted in the methods, we leveraged research evidence, expert opinion, and stakeholder input to develop and evaluate the PNID program. First, the research evidence focused on transportation as it related to individuals with disabilities ( 5 , 13 , 17 , 18 ), AT devices and services ( 9 , 19 –22), the diffusion of innovation ( 23 ), and education design practices ( 24 ). Second, the expert opinion was derived from the PDEA and CWID program proposals. These proposals provided the foundation for the current state of the art in independent public transportation for individuals with disabilities. The stakeholder input was derived through formal and informal interactions during the PIECE program and Smart Columbus. Finally, the evidence from stakeholders was acquired during several working groups and community engagement activities as part of the Smart Columbus trade study ( 7 ). The purpose of the trade study was to evaluate currently available PTA systems based on the needs of travelers with cognitive disabilities, travel trainers, travel coaches, community specialists, and other stakeholders as it relates to public transportation. The Smart Columbus team conducted several meetings with the Smart Columbus Mobility Assistance Working Group, the Smart Columbus Connect Linden Community Outreach Program, and the COTA Mobility Advisory Board. The Smart Columbus team conducted several product demonstrations with PTA system developers (i.e., AbleLink Technologies and Mass Factory), including interviews of PTA system designers and engineers. Finally, the Smart Columbus team leveraged the results from the OSU Nisonger PIECE Program, to round out the evidence used in the trade study. The research evidence, expert opinion, and stakeholder input identified the underlying issues with public transportation for individuals with disabilities in the Columbus metropolitan region, as well as the potential solutions.

Stage 3: Identifying the Foundation of Intervention

Theoretical foundations were identified and selected during Stage 3, which informed the intervention development, implementation, and evaluation. The theoretical foundations are based on frameworks, models, and theories (FMTs) in technology, mainstream and AT, innovation, service delivery, disability, and education and were selected by the research team (who are professionals in the field of AT) informed by their clinical practice and research training. The FMTs that were used for the development and evaluation of the PNID program include the World Health Organization—International Classification of Functioning, Disability, and Health framework ( 25 ), the human, the activity, the AT, and the context (HAAT) model ( 8 ), the AT service delivery model ( 9 ), the diffusion of innovation theory ( 22 ), and the Integrated Course Design process ( 24 ). The FMTs, as the underlying foundation of the complex intervention, are critical for its continued adoption and sustainability. The continued use of technology is based on a closed-loop feedback system where individuals continually evaluate the perceived relative advantage of technology over parallel interventions ( 25 ). Therefore, the FMTs provided a foundation for complex interventions. The FMTs, in conjunction with evidence-based practice (research evidence, professional expertise, and stakeholder input), provided a foundation for the development and evaluation of the PNID program.

Stage 4: Designing the Education and Training Package Intervention

The PNID program was funded in the summer of 2018 by OSU and was integrated within the MAPCD project (SMART@OhioState). Before the development of the PNID program, the City of Columbus conducted a trade study in collaboration with OSU to evaluate PTA systems (Stage 2). The systems included the Transit app (https://transitapp.com/), the App&Town Campagnon app (https://www.appandtown.com/compagnon.html), and the AbleLink WayFinder app (https://www.ablelinktech.com/index.php?id=33). Based on the trade study, conducted in 2017, the WayFinder system was selected for implementation during the Smart@Columbus Go-Live period (April 29, 2019 to April 28, 2020). The trade study’s completion was a key milestone in the MAPCD project. On selection of the WayFinder system, the project team began developing the implementation strategies for the PNID program, which included educational materials, training strategies, logistical support, and an evaluation plan.

Description of Intervention Implementation

The PNID program used the WayFinder system as a PTA to improve community mobility and independence for adults with cognitive disabilities. The WayFinder system included the WayFinder app (iOS or Android), SMART Route Builder (web), SMART Route Library (web), and SMART Travel Manager (web). The PNID education program included five activities: 1) assessment of abilities; 2) assessment of travel experience; 3) didactic, lab and simulation training; 4) real-world training; and 5) satisfaction and sustainability (Table 1). The WayFinder system and the PNID program provided the foundation for independent travel by individuals with cognitive disabilities.

Table 1.

Personal Navigation for Individuals with Disabilities Program for Travelers

Activity no.	Activity description
1	Assessment of abilities
2	Assessment of travel experience
3	Didactic, lab and simulation training a. Safety training i. Safety overview and discussion ii. AbleLink travel training curriculum: interactive video iii. Safety quiz b. Public transportation training i. Public transportation content instruction 1. AbleLink travel training curriculum: interactive video 2. Public transportation training at COTA facility ii. Public transportation quiz c. General smartphone training i. Smartphone overview and discussion ii. Smartphone interactive hands-on lab iii. Smartphone task analysis iv. Smartphone training satisfaction survey d. WayFinder app training i. WayFinder app overview and discussion ii. WayFinder interactive hands-on lab iii. WayFinder app task analysis iv. WayFinder app training satisfaction survey e. Fixed-route bus system simulation training
4	Real-world training
5	Satisfaction and sustainability

The travel training program started with assessments of abilities and travel experience. The assessments included the travelers’ attributes and experiences and allowed the program instructors to customize the training to the traveler’s individual needs. The travelers completed five training sessions: 1) safety; 2) public transportation; 3) smartphone; 4) WayFinder app; and 5) fixed-route bus system simulation. The training sessions were customized to the individual. The first four training sessions (i.e., safety, public transportation, smartphone, and WayFinder app training) were conducted in a classroom setting where the participants engaged in didactic training, mock assessments, and online training videos. The fifth training took place at the COTA Mobility Services Facility to simulate the task before accessing public transit. After each training session, the travelers completed either a quiz or task analysis to assess their learning retention, and a satisfaction survey to assess the training session. The safety and the public transportation training included quizzes to assess travelers’ comprehension. The program instructor completed a task analysis following the WayFinder app and the smartphone training.

Customizing the Training Program

Twenty-eight individuals completed at least some portions of the training program. It is beyond the scope of this paper to report participant-specific data as this manuscript is focused on the development of the intervention; however, case studies highlighting participants’ experiences are referenced in Culter Harris and colleagues ( 15 ). The training program was customized to the unique requirements of each traveler. Training was modified if the traveler had prior experience with the activities addressed in the training and could demonstrate knowledge of the training content. In addition, training was modified to accommodate the traveler’s access to transportation. Some participants did not complete the public transportation training at the COTA Mobility Services Facility with a simulated bus, and instead practiced transportation skills on a fixed-route bus system (i.e., OSU’s Campus Area Bus System [OSU CABS]). In those instances, providing the practice on a fixed-route bus system offered an equitable alternative solution because the participants were still able to practice the transportation skills. Finally, travelers had a range of experience with smartphone devices. Therefore, the smartphone training was omitted if the traveler already owned a personal smartphone, had significant experience using one, or both.

Stage 5: Training the Program Instructors

Program instructors were graduate and undergraduate students in the School of Health and Rehabilitation Science and are authors two, three, four, five, six, and seven of this manuscript. Program instructors were trained by the ninth author. Program instructors were tasked with conducting the training. The PNID training program for program instructors included didactic classroom activities, interactive activities to simulate real-world travel experiences in the classroom and on a fixed-route bus, apprenticeship activities to return demonstrate the training program activities, and a final reflection to demonstrate the acquisition of knowledge and skills and to learn from the experience of other program instructors. The six program instructors were recruited through the School of Health and Rehabilitation Science and selected on their interest and experience engaging with individuals with disabilities.

The classroom activities included a review of the PNID program and a review of the WayFinder system, followed by interactive activities to simulate real-world travel experiences. The classroom activities were created based on the lessons learned during Stages 1 to 3. The first step in the classroom activities was a didactic lecture that provided an overview of the PNID program. The second step included an interactive activity where research personnel would take on both the role of the traveler and the role of the program instructor. The third step included a trip on a fixed-route bus system and/or in a personal vehicle to simulate the interaction between the traveler, travel partner, and program instructor in a low-stakes setting. During the simulated route completion, the program instructors had the opportunity to use the various components of the WayFinder system, including the app, route builder, route library, and travel manager. The simulated route completion allowed the program instructors to apply the skills they had acquired during the classroom activities and demonstrate their proficiency with the steps in the PNID program and the WayFinder system. The fourth step included apprenticeship activities that consisted of scaffolded learning activities (i.e., gradually removing support and prompting as the participants continued to engage with the learning activities), starting with the classroom activities and simulated route completion activities, culminating with observation and mentorship activities with the program instructors. The apprenticeship activities leveraged peer interaction and took advantage of the experience, knowledge, and skills of the research personnel. The four steps to the program instructor training process prepared the program instructors to implement the PNID program.

Stage 6: Feasibility Testing and Reflecting on Lessons Learned

The feasibility-testing stage was designed to implement and evaluate the PNID program in a midsize urban setting. The PNID program leveraged the PTA system ( 8 ), HAAT model ( 9 ), and AT service delivery process ( 9 ) as the foundation of independent transportation for individuals with cognitive disabilities. The HAAT model addressed the complex interaction between the individual with a cognitive disability, the WayFinder system, and the activity of transportation in the context of a fixed-route bus system within a midsize urban setting.

Given the focus on fixed-route bus transportation, COTA provided significant resources throughout the project. COTA supported travelers and travel partners through training sessions at its indoor training facility, provided bus passes for travelers and travel partners, and provided access to buses to create travel training videos. Furthermore, COTA provided access to its General Transit Feed Specification (GTFS) data and access to the bus route waypoints (i.e., bus stops) to integrate the information into the WayFinder system. The GTFS-realtime (RT) provides the real-time location of buses, which is integrated into the WayFinder system. The integration of smartphone technology and transit vehicle technology are key components of a successful PTA.

Finally, the AT service delivery process provided a closed-loop feedback system that aligned with the MAPCD project and the PNID program. The referral, intake, and assessment were completed during Activities 1 and 2 of the PNID program, which aligned with identifying participants, and collecting baseline information about the individuals, as well as about the transportation activities the individual would like to perform (Table 1). Decision making and implementation were completed during Activities 3 and 4 of the program, which included customizing the smartphone and WayFinder app to the individual, and training sessions that addressed safety, public transportation, smartphones, the WayFinder app, and transportation via a fixed-route bus system. PTA performance and an evaluation of its effectiveness were completed during Activity 5 of the program (Table 1).

The PNID program feasibility-testing indicated that the program was challenging owing to scheduling and time barriers for the travelers, as a result, only 17 of the 28 travelers completed the entire program. Some of the challenges for completing the training included a reported lack of time, a reported lack of interest, or conflicts with scheduling appointments. Future versions of the training program could address some of these issues by partnering with existing programs (e.g., day programs for adults with IDD) and creating parallel training modules that use multiple learning modalities (e.g., remote training, hands-on training, simulation, and real-world training). Partnering with existing programs and creating multiple learning modalities could reduce the barriers to independent transportation and increase the adoption of transportation programs based on PTA architectures.

Overall, the PNID program was well received by the travelers. The travelers successfully passed the quizzes, though it may have taken multiple attempts. They were generally satisfied with the training sessions, and the majority indicated that they were satisfied with each of the quizzes. Finally, the task analyses demonstrated that the travelers could perform the necessary activities independently or identified activities that required additional supports for the travelers. The travelers then worked with their travel partners to develop and implement the supports, as necessary. The quizzes, satisfaction surveys, and task analyses demonstrated key formative and summative components in the training process. They also identified activities for continued training as the travelers transitioned to the implementation phase.

The PNID program has provided a framework for future training programs, including an assessment, training sessions, quizzes, surveys, and measurement tools to assess the program outcomes. Given the travelers’ feedback and interaction with the program, as demonstrated by the quizzes, task analysis instruments, and satisfaction surveys, the current education program provides a foundation for future training programs.

Stage 7: Future Directions

The lessons learned during feasibility testing of the intervention were compiled through stakeholder meetings and dissemination activities (e.g., regional, national, and international workshops and conference presentations). The dissemination activities led to discussions about the sustainability of the MAPCD program locally, regionally, and nationally. Throughout the project, the project team provided information to organizations in Central Ohio, Cuyahoga County, OH, Pittsburgh, PA, Knoxville, TN, Michigan, and Indiana either through Smart Columbus or through AbleLink Technologies. The lessons learned while developing and implementing the PNID program within the MRC framework provide a foundation for the development, execution, and sustained implementation of similar programs in other metropolitan regions.

Discussion

Complex Intervention: Application of the MRC Framework

The PNID program was a complex intervention as defined by MRC. Therefore, the purpose of this report was to describe the PNID program as an implementation strategy, provide insight into the identification and development of the intervention, report on the feasibility of the intervention, and provide lessons learned and future directions. MRC provided guidance for developing and evaluating complex interventions through researchers working with other stakeholders to identify key questions about complex interventions, about the design and conduct of research with an appropriate choice of methods, and evaluating the complex interventions at the end of the implementation. Using the MRC framework for complex interventions ( 5 , 10 , 11 , 25 ), we have illustrated a seven-step method for developing the PNID program: systematically developed, theory-driven, and based on the available evidence. Therefore, this report describes the systematic development of the PNID program and the multiple iterations of the intervention from conceptualization through pre-implementation. In addition, the report provides detailed information about the stakeholders, the infrastructure, and the strategies necessary to implement the PNID program or variations of it in other midsize urban settings. To that end, reporting on participant-specific data is beyond the scope of this paper and will be addressed in future publications, such as the work of Culter Harris and colleagues outlining case studies of the participants’ experiences in this study ( 15 ).

The PNID program was complex owing to the several active ingredients that affected its success. The MRC framework for developing and evaluating complex interventions identified five characteristics of a complex intervention ( 10 , 11 ). First, the PNID had several interacting components that affected the success of the overall transportation system for individuals with cognitive disabilities. Second, the program was dependent on the behavior of several stakeholders, including the individuals who were recipients of the training and technology intervention (i.e., travelers and travel trainers), and the individuals who trained the recipients of the intervention (i.e., program instructors). Third, the program depended on the groups and organizations targeted by the intervention, ranging from the local public transportation systems to organizations that support individuals with IDD. Fourth, the PNID program was a complex intervention based on the number and variability of outcome measures used to measure the program’s success. The outcomes were multifaceted and could change based on the interests of the individual stakeholder groups. Finally, the PNID program was a complex intervention based on the flexibility of the intervention, the capacity and performance of the individual stakeholders (e.g., comfort and experience with smartphones), and the goals of the organizations (e.g., to reduce the number of paratransit trips). These five items clearly defined the PNID program as a complex intervention with numerous moving parts.

The MRC framework for developing and evaluating complex interventions provided a foundation for the development, implementation, and analysis of the PNID program. As is often the case with complex interventions, the PNID program was developed within an existing project, namely the MAPCD project within Smart Columbus. Therefore, numerous boundary conditions were already in place when the PNID was in development. The boundary conditions included funding for portions of the project (e.g., technology and personnel), stakeholders (e.g., City of Columbus, COTA, and OSU) defined timelines (2016 to 2020), external project milestones, existing infrastructure (e.g., COTA and OSU CABS), and an identified technology solution (i.e., technology-based PTA). The boundary conditions therefore constrained the flexibility of the development of the PNID program while also providing existing infrastructure for rapid intervention development and feasibility testing.

The MRC framework identified six core elements that should be considered during each phase of the development, evaluation, feasibility, and implementation process. The core elements include the context, theory, stakeholders, uncertainties, refinement, and economics. During the development of the PNID program, we considered each of the core elements. Though all the elements played an important role in the success of the program, the element that may have played the largest role was the context.

The context played a core role in AT provision and service delivery and is described in the HAAT model ( 9 ). The HAAT model explicitly identified and described the physical, social, cultural, and institutional contexts, and these were identified as key barriers and facilitators to the success of the PNID program. The built environment played a significant role given the location of bus stops (e.g., distance from home) and the physical accessibility of the bus stop (e.g., sidewalks). The social context played a role given the level of support provided by the travel trainers, transit operators, care partners, and other stakeholders. For example, the travel trainer may not have been comfortable using the technology to develop routes for the traveler. In addition, the bus driver on a fixed-route bus system may understand that many individuals with disabilities use public transportation and appreciate the use of the app in public spaces. The cultural context played a role given that individuals with a disability, in particular individuals with a cognitive disability, typically have a vast support system in place. Care partners (e.g., family or independent care provide) may feel it is their role to provide transportation or may not be comfortable with the individual traveling on their own. Finally, the institutional context included boundaries established through organizations and policies that can have an impact on access to transportation. The institutional context affected transportation through funding for public transportation and for technology that supports individuals with cognitive disabilities, in this case, the WayFinder system.

The PNID program used both theory-based and systems-based perspectives ( 26 ). The theory-based perspective aimed to understand how the PNID program could affect an individual with a cognitive disability by using technology to support independent transportation on a fixed-route bus system. The systems perspective aimed to describe the interaction of a new intervention, namely the WayFinder system, within existing systems, specifically COTA, OSU CABS, and private transportation. The two perspectives provided a lens through which the researchers could evaluate the program’s feasibility and identify facilitators and barriers for future implementation.

Throughout the development and evaluation of the PNID program, outcomes were a key consideration. Some of the outcomes addressed the success of the individual, and some addressed the success of the broader program. Given that the program development and feasibility-testing process is a closed-loop system (i.e., implementation leads to further development), outcome measures are necessary because they inform the refinement of the program as the intervention moves from one phase to the next. For the project, we identified both individual- and program-level outcome measures. The individual-level outcome measures included satisfaction, trips taken, task completion, and quiz scores. The program-level outcome measures included adoption, appropriateness, feasibility, and sustainability as described by Proctor and colleagues ( 26 ). The outcome measures were aligned with those described in the MRC framework literature ( 10 , 11 ) and were a combination of qualitative and quantitative measures. These outcome measures were used in the development of the PNID program and when addressing the sustainability of the program. These outcome measures should be addressed in future implementations of this intervention.

In developing and implementing the PNID program we went through several of the phases described in the MRC framework, though not in a completely linear fashion. We suggest that this is partially a characteristic of a complex intervention, as suggested by Craig and colleagues ( 5 ), and partially a result of the project team not being aware of the MRC framework before the start of project development. The program developers and researchers recognize that there was a sense of serendipity in aligning the PNID project to the MRC framework after completing the project. However, we believe this demonstrates two points: first that the PNID project was developed using sound program design and development principles based on the experience and knowledge of the project team in both program development and clinical practice. Second, the MRC framework is an excellent resource that clearly articulates many of the complexities observed when developing, evaluating, and implementing complex interventions and programs. Therefore, we acknowledge the serendipity of aligning the project with the MRC framework on a post hoc basis, and we believe this highlights the true complexity of developing, evaluating, and implementing a complex intervention in a community setting.

PNID Barriers, Facilitators, and Insights

The PNID program and MAPCD project directly addressed the need for independent transportation, which is a barrier to employment and community participation. The program’s overall success was measured through the lens of increased autonomy and independence for individuals with cognitive disabilities. However, the various stakeholders identified numerous barriers and facilitators to implementation during the PNID program and MAPCD project. The barriers that were reported to our team included physical access to the fixed-route bus service, familiarity with technology (e.g., smartphone and computer), cost of technology, travel partner engagement, and traveler willingness to transition from door-to-door service (e.g., paratransit or private transportation) to public fixed-route bus service. These reported barriers were not aspects of participant inclusion or exclusion criteria. When these barriers were reported, the intervention was personalized to meet the participants’ needs. The facilitators included access to the fixed-route bus service, GTFS and GTFS-RT data, route library, access to the COTA Mobility Services Facility, access to the WayFinder system, the creation of a comprehensive training program, and collaboration among stakeholders. In the exit interviews conducted with the travelers, they all reported positive perspectives from the training. Though several of the participants did not report continued use of the training strategies, they stated this was because of the COVID-19 pandemic and travel restrictions.

The facilitators and barriers to implementation for the PNID program provided numerous insights for the future implementation of similar programs. First, we believe that successful implementation of this complex intervention was dependent on the interprofessional collaboration among stakeholders. In the case of the MAPCD project, the interprofessional collaboration included stakeholders from COTA, OSU Nisonger Center, OSU Aphasia Initiative, OSUWMC ATC, Franklin County Board of Developmental Disabilities, and Arc Industries. The stakeholders primarily included AT professionals, care partners, case coordinators, consumers, community specialists, rehabilitation engineers, and travel trainers. Each of these stakeholders were able to provide insight and feedback into the creation of the PNID program through their expertise at each stage of intervention development.

Second, we found that the travel partners that were the most successful were also community specialists for a local agency that supports individuals with IDD. Often community specialists work with the individual with a disability on a daily basis and support the individual’s occupations throughout their day (e.g., leisure participation, friendships, volunteering participation, job performance and maintenance, informal educational participation). Community specialists (i.e., community accessibility specialists or community mobility specialists) are typically motivated by human service activities, which aligns with their job roles, and are not necessarily as interested in emerging technologies. Therefore, future implementation should provide additional training and support for these travel trainers. Our team found that community specialists are uniquely positioned to take on the role of the travel trainer given that they typically have the skills, knowledge, and experience to bus support the traveler.

Finally, we found that leveraging existing community resources, such as the COTA Mobility Services Facility, COTA fixed-route bus service, and OSU CABS fixed-route bus service, was essential to participation in this intervention. With regard to long-term sustainability, the most appropriate organization to fund the technology and training may not be a local transit authority (e.g., COTA), but rather a human services organization (e.g., Franklin County Board of Developmental Disabilities, Arc Industries), or a local or regional governmental organization owing to their mission to meet the needs of individuals with disabilities. However, because it was beyond the scope of this project, we were not able to follow up with COTA to understand their perspectives on aspects of this project and whether they will expand or implement this type of program in a more permanent way, nor to better understand whether this intervention could result in a reduction in demand-responsive or paratransit services if implemented in a scaled-up study. Future research leveraging implementation science methodologies would be beneficial in this area to understand the ability for stakeholders to champion this type of complex intervention.

The MAPCD project, in general, and the PNID program, in particular, provided a framework for future training and implementation programs, both regionally (e.g., Central Ohio and Cuyahoga County) and nationally (e.g., Indianapolis and Pittsburgh). The PNID program included an assessment, several training sessions, quizzes, surveys, and other measurement tools to assess the program outcomes. The MAPCD project provided access to an emerging PTA, specifically the WayFinder system. Given the travelers’ and travel partners’ feedback and interactions with the program, as demonstrated by interviews and outcome measures, the PNID program provided a foundation for independent personal navigation and transportation. However, this is just the first step in creating and implementing a complex intervention that may meet the needs of individuals with disabilities to access public transportation.

Conclusion

The purpose of this project was to identify, develop, and assess the feasibility of a complex intervention to support independent public transportation for individuals with cognitive disabilities. The PNID education and training program is a complex intervention that uses a PTA, specifically the WayFinder system, to support independent transportation on fixed-route bus systems. The identification, development, and feasibility testing of the PNID education and training program was based on the MRC framework. The identification and development of the PNID education and training program leveraged existing models and theories, included multiple stakeholders, and addressed the unique context of public transportation in a midsize urban setting and a large academic setting. The feasibility testing (aka pilot testing) provided an opportunity to learn from stakeholders as we refined and evaluated the program. The PNID education and training program and MAPCD project provide a framework for the identification, development, implementation, and evaluation of future PTA programs for midsize urban settings.

Footnotes

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: J. Faieta, O. Vega, S. Metzler, C.P. DiGiovine; data collection: J. Faieta, K. Culter, E. Mance, A. Stojkov, S. Anderson, O. Vega, S. Metzler, C.P. DiGiovine; analysis and interpretation of results: B.J. Frick Semmler, K. Culter, E. Mance, A. Stojkov, S. Anderson, S. Metzler, C.P. DiGiovine; draft manuscript preparation: B.J. Frick Semmler, C.P. DiGiovine. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded as part of the Smart Columbus project, which is part of the SMART City Federal Initiative.

ORCID iDs

Bethany J. Frick Semmler

Sarah Anderson

Sandra Metzler

Carmen P. DiGiovine

References

Noel

V. A.

Oulvey

Drake

R. E.

Bond

G. R.

Barriers to Employment for Transition-Age Youth with Developmental and Psychiatric Disabilities. Administration and Policy in Mental Health, Vol. 44, 2016, pp. 354–358.

Braddock

Rizzolo

M. C.

Thompson

Bell

Emerging Technologies and Cognitive Disability. Journal of Special Education Technology, Vol. 19, No. 4, 2004, pp. 49–56.

Brumbaugh

Travel Patterns of American Adults with Disabilities [Issue Brief]. BTS Special Reports and Issue Briefs, 2018. https://doi.org/10.21949/1524180.

Bezyak

J. L.

Sabella

S. A.

Gattis

R. H.

Public Transportation: An Investigation of Barriers for People with Disabilities. Journal of Disability Policy Studies, Vol. 28, No. 1, 2017, pp. 52–60.

Craig

Dieppe

Macintyre

Michie

Nazareth

Petticrew

Developing and Evaluating Complex Interventions: The New Medical Research Council Guidance. British Medical Journal, Vol. 337, 2008, p. a1655. https://doi.org/10.1136/bmj.a1655.

Craig

Dieppe

Macintyre

Michie

Nazareth

Petticrew

Developing and Evaluating Complex Interventions: The New Medical Research Council guidance. International Journal of Nursing Studies, Vol. 50, No. 5, 2013, pp. 587–592. https://doi-org.proxy.lib.ohio-state.edu/10.1016/j.ijnurstu.2012.09.010.

Trade Study. Mobility Assistance for People with Cognitive Disabilities (MAPCD) Interface Control Document. SMRT Columbus. 2018. https://smart.columbus.gov/projects/mobility-assistance-for-people-with-cognitive-disabilities.

Carmien

Dawe

Fischer

Gorman

Kintsch

Sullivan

J. F.

Socio-Technical Environments Supporting People with Cognitive Disabilities Using Public Transportation. ACM Transactions on Computer-Human Interaction, Vol. 12, No. 2, 2005, pp. 233–262.

Cook

A. M.

Polgar

J. M.

Encarnação

Activity, Human and Context in Assistive Technologies: Principles and Practices, 5th ed. Elsevier, New York, NY, 2020, pp. 31–55.

10.

Craig

Dieppe

Macintyre

Michie

Nazareth

Petticrew

Developing and Evaluating Complex Interventions: Following Considerable Development in the Field Since 2006. Medical Research Council Journal, 2019, p. 39. http://www.mrc.ac.uk/complexinterventionsguidance

11.

Skivington

Matthews

Simpson

S. A.

Craig

Baird

Blazeby

J. M.

Boyd

K. A.

, et al. A New Framework for Developing and Evaluating Complex Interventions: Update of Medical Research Council Guidance. British Medical Journal, Vol. 374, 2021, p. n2061. https://doi.org/10.1136/bmj.n2061.

12.

Kilbourne

A. M.

Neumann

M. S.

Pincus

H. A.

Bauer

M. S.

Stall

Implementing Evidence-Based Interventions in Health Care: Application of the Replicating Effective Programs Framework. Journal of Implementation Science, Vol. 2, No. 42, 2007, 1–10. https://doi.org/10.1186/1748-5908-2-42

13.

Park

Esfahani

H. N.

Novack

V. L.

Sheen

Hadayeghi

Song

Christensen

Impacts of Disability on Daily Travel Behaviour: A Systematic Review. Transport Reviews, Vol. 43, No. 2, 2023, pp. 178–203. https://doi.org/10.1080/01441647.2022.2060371.

14.

Bauer

M. S.

Damschroder

Hagedorn

Smith

Kilbourne

A. M.

An Introduction to Implementation Science for the Non-Specialist. BMC Psychology Journal, Vol. 3, No. 1, 2015, 1–12. https://doi.org/10.1186/s40359-015-0089-9.

15.

Culter Harris

Frick Semmler

B. J.

Anderson

Mance

Stojkov

Metzler

DiGiovine

C. P.

Innovative Solutions to Support Individuals with Disabilities Accessing Public Transportation: A Case Study. Assistive Technology, Online ahead of print, 2024, pp. 1–10. https://doi.org/10.1080/10400435.2024.2305977

16.

Müller

Lautenschläger

Dörge

Voigt‒Radloff

A Feasibility Study of a Home-Based Lifestyle-Integrated Physical Exercise Training and Home Modification for Community-Living Older People (Part 2): The FIT-at-Home Fall Prevention Program. Disability and Rehabilitation, Vol. 43, No. 10, 2021, pp. 1380–1390. https://doi.org/10.1080/09638288.2019.1700564.

17.

Tessier

Clément

M.A.

Gélinas

Boucher

Croteau

Morin

Turcotte

M. P. S.

Archambault the Impact of Transportation on the Employment of People with Disabilities: A Scoping Review. Transport Reviews, Vol. 44, No. 1, 2024, pp. 85–111. https://doi.org/10.1080/01441647.2023.2229031.

18.

Egger

S. M.

Gemperli

Filippo

Liechti

Gantschnig

B. E.

The Experiences and Needs of Persons with Disabilities in Using Paratransit Services. Disability and Health Journal, Vol. 15, No. 4, 2022, p. 101365. https://doi.org/10.1016/j.dhjo.2022.101365.

19.

Stock

S. E.

Davies

D. K.

Herold

R. G.

Wehmeyer

M. L.

Technology to Support Transportation Needs Assessment, Training, and Pre-trip Planning by People with Intellectual Disability. Advances in Neurodevelopmental Disorders, Vol. 3, No. 3, 2019, pp. 319–324. https://doi.org/10.1007/s41252-019-00117-x.

20.

Davies

D. K.

Stock

S. E.

Holloway

Wehmeyer

M. L.

Evaluating a GPS-Based Transportation Device to Support Independent Bus Travel by People with Intellectual Disability. Intellectual and Developmental Disabilities, Vol. 48, No. 6, 2010, pp. 454–463. https://doi.org/10.1352/1934-9556-48.6.454.

21.

Stock

S. E.

Davies

D. K.

Hoelzel

L. A.

Mullen

R. J.

Evaluation of a GPS-Based System for Supporting Independent Use of Public Transportation by Adults with Intellectual Disability. Inclusion, Vol. 1, No. 2, 2013, pp. 133–144. https://doi.org/10.1352/2326-6988-01.02.133.

22.

Lindsay

Lamptey

D. L.

Pedestrian Navigation and Public Transit Training Interventions for Youth with Disabilities: A Systematic Review. Disability and Rehabilitation, Vol. 41, No. 22, 2019, pp. 2607–2621. https://doi.org/10.1080/09638288.2018.1471165.

23.

Rogers

E. M.

Diffusion of Innovations, 5th ed. Free Press, New York, NY, 2003.

24.

Fink

L. D.

Integrated Course Design. IDEA Paper #42. IDEA Center, Manhattan, Kansas, 2005, pp. 1–7.

25.

Lenker

J. A.

Paquet

V. L.

A New Conceptual Model for Assistive Technology Outcomes Research and Practice. Assistive Technology, Vol. 16, No. 1, 2004, pp. 1–10. https://doi.org/10.1080/10400435.2004.10132069.

26.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administrative Policy in Mental Health, Vol. 38, No. 2, 2011, pp. 65–76. https://doi.org/10.1007/s10488-010-0319-7.

27.

Ramsay

J. E.

Janevic

M. R.

Hogan

C. K.

Edwards

D. L.

Connell

C. M.

Using the Replicating Effective Programs Framework to Adapt a Heart Health Intervention. Health Promotion Practice, Vol. 20, No. 5, 2019, pp. 760–769. https://doi.org/10.1177/1524839918775740.

28.

Bezyak

J. L.

Sabella

Hammel

McDonald

Jones

R. A.

Barton

Community Participation and Public Transportation Barriers Experienced by People with Disabilities. Disability and Rehabilitation, Vol. 42, No. 23, 2020, pp. 3275–3283. https://doi.org/10.1080/09638288.2019.1590469.

29.

Wasfi

Steinmetz-Wood

Levinson

Measuring the Transportation Needs of People with Developmental Disabilities: A Means to Social Inclusion. Disability and Health Journal, Vol. 10, No. 2, 2017, pp. 356–360. https://doi.org/10.1016/j.dhjo.2016.10.008.

30.

World Health Organization. Towards a Common Language for Functioning, Disability and Health: ICF. World Health Organization, 2002, p. 22. Geneva, Switzerland.

Developing a Training Program for a Personal Navigation Intervention for Individuals with Disabilities

Abstract

Keywords

Developing Complex Interventions

Personal Navigation Intervention for Individuals with Cognitive Disabilities

Methods

Results

Stage 1: Identifying and Establishing Stakeholder Roles and Contributions

Stage 2: Determining Existing Evidence on Interventions

Stage 3: Identifying the Foundation of Intervention

Stage 4: Designing the Education and Training Package Intervention

Description of Intervention Implementation

Customizing the Training Program

Stage 5: Training the Program Instructors

Stage 6: Feasibility Testing and Reflecting on Lessons Learned

Stage 7: Future Directions

Discussion

Complex Intervention: Application of the MRC Framework

PNID Barriers, Facilitators, and Insights

Conclusion

Footnotes

Author Contributions

Declaration of Conflicting Interests

Funding

ORCID iDs

References