Abstract
Background
Upper limb deficits are common after stroke, leading to daily activity limitations and potential caregiver dependence. Consistent assessment of upper limb function is crucial for effective rehabilitation, but its inconsistent implementation across clinical settings necessitates tailored strategies for adherence to outcome measurement and optimal patient care.
Objectives
This study systematically analysed past implementation strategies for the Fugl-Meyer Assessment (FMA), a gold-standard post-stroke upper limb outcome measure, to identify gaps and challenges. The aim was to inform future development of tailored strategies, grounded in theory and implementation science, to enhance routine use of the FMA in practice.
Methods
Grounded in the Normalisation Process Theory (NPT), we conducted a qualitative multiple case study in four public hospitals in Singapore. Interviews were held with occupational therapists involved in the original FMA implementation, and documents from the planning and initial stages of implementation were collected. Data analysis included systematic deductive and thematic approaches, within-case and cross-case analysis, and mapping implementation strategies to the NPT.
Results
Interviews with eight participants and document analysis revealed varying implementation activities across the hospitals. While all hospitals focused on education and training, one acute hospital conducted audits and tailored interventions. Organisational contexts influenced implementation, notably differences in care models and commitment to outcome measurement. Strategies primarily aligned with NPT’s cognitive participation and collective action constructs, but strategies for reflexive monitoring and coherence were limited.
Conclusion
Findings underscore the importance of a nuanced approach considering both implementation strategies and practice context. Key targets were identified to inform future tailored strategies for improving the consistency of using post-stroke upper limb outcome measures.
Background
Upper limb deficits are among the most common and persistent consequences of stroke, affecting 77% of stroke survivors with weakness and 63% with sensory impairments.1–4 These impairments can lead to significant limitations in daily activities, with many stroke survivors requiring assistance or being dependent on caregivers. 5 The impact of these impairments and activity limitations often extend beyond physical constraints, contributing to depression and reduced quality of life.6,7 Regaining upper limb function is thus an important rehabilitation goal for stroke survivors. 8
Assessing upper limb function is essential in stroke rehabilitation. Existing stroke rehabilitation guidelines recommend the use of standardised outcome measures,9–11 and recent efforts in stroke rehabilitation have specifically focused on developing evidence-based guidelines for upper limb assessments. 12 Using valid, reliable, and responsive outcome measures allows rehabilitation professionals to quantify the severity of upper limb impairments and activity limitations, guide rehabilitation interventions and assess their effectiveness.13–15 The Fugl-Meyer Assessment (FMA) is a widely recognised gold standard for post-stroke upper limb assessment, recommended for both clinical practice and research.16,17 It also provides a systematic and standardised approach for setting goals, planning, and progressing therapy based on FMA scores. 18 Incorporating the FMA into clinical practice therefore not only ensures evidence-based care but also promotes consistent rehabilitation care across settings.
Despite the emphasis on outcome measures, their implementation has not been adequately addressed. Studies highlight a gap between recommended practices and actual use of outcome measures in clinical practice. 19 For instance, a survey of 300 Canadian physiotherapists found that while 90% agreed on the importance of monitoring treatment outcomes, less than 30% reported routine use of standardised assessments. 20 Similarly, efforts to encourage therapists to use validated outcome measures showed limited success, with less than half incorporating these measures into practice after six years. 21
While existing literature has focused on implementing stroke rehabilitation guidelines, few studies specifically examine the use of standardised outcome measures, and even fewer assess the role of tailored implementation strategies in promoting their use.12,22 Systematic reviews suggest that tailored implementation strategies improve healthcare professionals’ adherence to clinical guidelines and to address barriers to change, 23 highlighting their potential to enhance the consistent use of recommended outcome measures. Therefore, it is beneficial to develop tailored strategies in our local practice setting to enhance the actual use of the FMA in clinical practice.
Our study aimed to analyse past implementation strategies used to introduce the FMA as a routine outcome measure for stroke patients admitted to hospitals. By retrospectively analysing these approaches, we aimed to identify the specific strategies employed and their contexts. This foundational understanding is crucial for informing future efforts to design evidence-based and theory-informed tailored implementation strategies that optimise resources, avoid duplication, and address barriers to consistent use.
Methods
Study design
We used a qualitative multiple case study design, examining multiple cases in their natural settings to understand the implementation process of the FMA as a routine outcome measure. 24 This approach, as described by Stake (2005), 25 allows for comparative analysis and identification of both common and unique factors across four public hospitals within the same network in Singapore.
Theoretical framework
Normalisation Process Theory (NPT)guided our study design, serving both as a classification schema for our data collection and analysis without imposing the theory as a conceptual framework. NPT examines how healthcare practices are embedded and sustained in routine care through four social mechanisms: coherence, cognitive participation, collective action, and reflexive monitoring, further divided into 16 components (Supplemental File 1). 26 We focused on understanding the rationale behind selecting the FMA, the implementation activities, stakeholders involved, and contextual factors, all of which are central elements considered within the NPT framework.
We selected NPT because its focus on the normalisation process aligns with our goal of ensuring the consistent use of the FMA in clinical practice. Unlike frameworks such as the Theoretical Domains Framework, which primarily identifies barriers and facilitators to behaviour change, NPT emphasises how practices become fully integrated over time. 27 This process-oriented lens is essential for exploring how the FMA can transition from a ‘new’ outcome measure to a routine measure in practice.
Additionally, NPT accounts for the dynamic interactions between individual behaviours, collective actions, and organisational contexts, making it particularly suited for our multi-site study across public hospitals. This comprehensive approach facilitated a thorough analysis of past implementation strategies, enabling us to identify factors that may support the long-term integration and sustainability of the FMA in diverse clinical settings. 28
Selection of cases
We defined a case as an occupational therapy department in a hospital that implemented the FMA as a standard outcome measure for stroke rehabilitation. We deliberately chose four different public hospitals to maximise variance, examining whether therapists’ practice contexts influenced FMA implementation. We also focused on a single hospital network as patients transition between hospitals within the network during various stages of stroke recovery. The specific focus on occupational therapy departments was because of the common practice of occupational therapists (OTs) exclusively evaluating post-stroke upper limb function in the local setting.
Our study included a total of four hospitals, which comprised an acute tertiary hospital, an acute regional hospital, and two community hospitals. Both acute hospitals provide hyperacute and acute stroke treatments but diverge in their care models. The tertiary hospital adopts a specialist care model with a dedicated stroke unit, while the regional hospital follows a generalist care model, admitting stroke patients to general medical wards based on capacity. Both community hospitals provide short-term post-acute care and rehabilitation services under a generalist care model.
Data collection
We collected data from multiple sources, including interviews with key informants and documents produced during the planning and initial stages of FMA implementation.
Interviews
Individual semi-structured interviews were conducted with eight OTs directly involved in the original FMA implementation. We employed purposeful sampling, approaching managers and key staff to identify our initial pool of four OTs as key informants who were involved in the pre-implementation (planning/preparation) phase at each hospital. Subsequently, we employed snowball sampling to identify an additional four key informants. This approach allowed us to reconstruct the sequence of events related to the FMA implementation up to the commencement date of our study. The interviews were conducted virtually, lasting approximately 45 to 60 minutes, and were audio recorded for transcription. Interviewers also took field notes during and after the interviews to enhance reflexivity. We developed a semi-structured interview guide designed to elicit details about the activities and processes related to FMA implementation across three stages: (1) Preparation – planning for FMA implementation; (2) Implementation – introducing the FMA to OTs and routine clinical practice; and (3) Sustainment – maintaining routine use of the FMA.
Interviews were conducted by two authors [SXC, JY] who are occupational therapists working at two data collection sites (acute tertiary and regional hospitals, respectively). To facilitate honest feedback, we ensured that interviewers did not interview participants practicing at the same hospital.
Documents
During each interview, the interviewer recorded the various implementation strategies and activities. Participants were also invited to share relevant documents created or utilised for these strategies and activities.
Data analysis
Data analysis was conducted by three authors [SXC, JY, SAMR] using a systematic deductive and thematic approach. Verbatim transcriptions of the audio recordings were imported into NVivo 9 for analysis. Using an initial framework based on the four NPT constructs, each author independently coded the transcripts, with additional open codes identified from the data. To ensure reliability, all transcripts were reviewed by all three authors, who then engaged in discussions to refine themes and develop a comprehensive codebook. 29 A within-case analysis explored unique contextual factors, followed by a cross-case analysis to identify common patterns and discrepancies across different sites. 24
Documents collected were first reviewed to get a sense of their relevance to the implementation activities described by the participants. Thereafter, we applied the same coding structure to code these documents. All implementation activities identified in the data were labelled according to the taxonomy developed by the Cochrane Effective Practice and Organisation of Care (EPOC) group. 30 These implementation strategies were then systematically mapped to the 16 NPT constructs.
Results
We interviewed a total eight participants (two participant per hospital) who were involved in various stages of implementation at their respective hospitals. At the time of the interviews, three participants were still working at the same hospital where they had been involved in the FMA implementation, while the remaining five participants were no longer employed at any of the hospitals. All but one participant was female, and the average years of work experience was approximately nine years.
Additionally, we collected various documents, including FMA administration manuals/guides, scoring sheets, presentation slides, training videos, competency checklists, patient education brochures, and audit data. From our data, we also built a timeline illustrating the implementation of the FMA at each hospital (Figure 1). Timeline of FMA implementation and relevant key events.
Implementation activities at each hospital (within-case analysis)
Summary of implementation strategies used (defined using EPOC taxonomy).
Case A (acute tertiary hospital)
Case A was the first hospital to implement the FMA, driven by the introduction of a stroke rehabilitation coordinated care pathway. Rehabilitation physicians spearheaded this initiative to standardise care and improve patient outcomes. A key outcome of interest within this pathway was post-stroke upper limb function, prompting OTs to select the FMA due to its strong psychometric properties and position as a ‘gold standard’ measure of post-stroke upper limb function. Initial implementation efforts focused on educating and training OTs in FMA administration, as well as establishing work processes, such as standardising scoring sheets. Subsequent efforts shifted towards enhancing OTs’ perceived value of the FMA. These included adopting an evidence-based systematic approach guided by FMA scores in post-stroke upper limb rehabilitation, providing training on FMA score interpretation, and developing patient education materials. Adjustments made during the hospital’s transition to electronic medical records included the introduction of digital scoring sheets and uniform clinical documentation practices.
Case B (acute regional hospital)
Case B followed as the next hospital to implement the FMA. During the initial phases of implementation, the focus was on educating and training OTs in the administration and scoring of the FMA. Concurrently, efforts were made to establish work processes to operationalise the use of the FMA in practice. To ensure competence and consistency among OTs, a competency checklist was developed and implemented.
Case C (community hospital)
Case C was the third to hospital to implement the FMA. Implementation activities at the hospital emphasised education and training, and the establishment of work processes to introduce the FMA into practice. Challenges emerged due to rapid hospital expansion during the COVID-19 pandemic, leading to an influx of new staff and stringent safety protocols. As a result, teleconsultation was adopted as an alternative to in-person coaching to address training needs while adhering to safe distancing measures.
Case D (community hospital)
Case D was the final hospital to implement the FMA, aligning its clinical practices with those of Case C as they were managed under the same administration. Implementation efforts centred on educating and training OTs in the use of the FMA and establishing operational work processes.
Implementation activities across hospitals (between-case analysis)
Influence of organisation contexts on FMA implementation
FMA implementation was influenced by three key organisational contexts. Firstly, the care model of each individual hospital significantly shaped the direction of implementation activities. Case A, a hospital with a specialist care model, focused exclusively on OTs working in the acute stroke and rehabilitation wards, targeting those with expertise in stroke rehabilitation. In contrast, the other three hospitals, following a generalist care model, adopted a broader strategy involving all OTs within their departments. Secondly, the common practice of ‘nesting’, a process where staff from a new hospital are trained at an established hospital before commencing operations, was pivotal in FMA implementation at Case B. Staff at Case B underwent this process at Case A, gaining knowledge and experience with FMA use and implementation, which they applied upon starting operations at their new hospital. Thirdly, Case C implemented the FMA due to its organisational commitment to outcome measurement and therapists’ desire for a standardised post-stroke upper limb assessment. Additionally, as a community hospital complementing Case A, the regular influx of stroke patients from Case A influenced Case C’s choice to adopt the FMA. OTs at Case C not only recognised the FMA’s status as a ‘gold standard’ measure but also appreciated that aligning with Case A in adopting the FMA would ensure continuity of patient care and the consistent tracking of patients’ progress in stroke recovery.
Similarities and differences in implementation activities
We examined key similarities and differences in implementation activities across hospitals (Table 1). Initially, all hospitals emphasised educational activities, adopting published administration manuals and conducting educational meetings. The hospital network’s online learning platform facilitated the dissemination of FMA training videos (created by Case A), enabling consistent training standards across hospitals. Additionally, standardising work processes, such as uniform clinical documentation practices, and appointing key champions to drive the use of the FMA were common implementation activities observed across hospitals.
Three key differences in implementation strategies emerged among the hospitals. Firstly, Case A was the only hospital conducting routine audits, although the audits were not exclusively focused on the FMA. Rather, they were part of broader monitoring for all outcomes within the stroke rehabilitation coordinated care pathway. Moreover, Case A, with nearly a decade of FMA use, was also the only hospital that employed tailored interventions. These interventions aimed to address challenges faced by OTs, focusing on clarifying the purpose and benefits of using the FMA and enhancing the perceived value of the measure. Tailored interventions included training on FMA score interpretation and integrating it into post-stroke upper limb rehabilitation interventions. Secondly, as mentioned above, the rapid hospital expansion due to the COVID-19 pandemic necessitated ongoing OT training while adhering to stringent safety protocols. Both Cases C and D responded by utilising teleconsultation as a training alternative during this period. Lastly, formal evaluation of OTs’ competency in administering the FMA was specifically conducted only at Case B. There, an in-house designed competency checklist detailed assessment components and completed patient evaluations. Conversely, the other three hospitals employed an informal evaluation approach based on performance during coaching sessions with senior staff or supervisors, lacking formal documentation of OTs' competency.
Implementation strategies and the Normalisation Process Theory
Implementation strategies mapped to the Normalisation Process Theory.
Discussion
Outcome measurement remains a cornerstone in post-stroke upper limb rehabilitation, and is essential for identifying impairments and activity limitations, selecting rehabilitation interventions, and determining effectiveness of interventions.13–15 Despite its importance, there is a notable knowledge gap in systematically implementing outcome measures in routine clinical practice to ensure optimal patient care and treatment effectiveness. To address this gap, our study used a theory-driven approach to retrospectively analyse past implementation strategies for the FMA, a gold-standard post-stroke upper limb outcome measure, across multiple hospital settings. This approach enabled us to identify the specific strategies involved, thus laying the groundwork for future efforts to develop targeted, evidence-based implementation strategies.
Findings revealed that all hospitals used multiple implementation strategies, with educational activities, establishing work processes and the presence of key champions as common strategies used. Organisational context, care models, and resource availability were contextual factors that influenced implementation strategies. Using a theory-driven approach and mapping past implementation strategies to NPT constructs, we identified gaps—such as unaddressed constructs—that help explain why the routine use of the FMA continues to be a challenge. This theoretical analysis highlighted specific areas not adequately addressed by past implementation strategies, providing insights into why the FMA has not been consistently integrated into clinical practice.
Use of multifaceted strategies to implement the FMA
In our study, all hospitals employed multifaceted implementation strategies to introduce the FMA into clinical practice. These strategies included educational meetings, distribution of educational materials, use of local opinion leaders, local consensus processes, and reminders. Using multifaceted implementation strategies is consistent with findings from a scoping review that occupational therapists implementing evidence-based practice in stroke rehabilitation employed a median of four implementation strategies, with education-related interventions being one of the most used strategies. 31 In our study, the use of both active and passive strategies by all four hospitals, rather than relying solely on education alone, is encouraging. Active strategies, such as interactive workshops, hands-on training, and individualised feedback, engage healthcare professionals. 32 In contrast, passive strategies, including the distribution of guidelines, informational material, and reminders, require less active engagement from clinicians but still disseminate crucial information. 33 Studies have demonstrated that education alone has limited efficacy in changing healthcare professionals’ clinical practices, suggesting that active and passive strategies may be more effective in achieving the desired practice change. 34
The impact of multifaceted implementation strategies lies not in the aggregated or cumulative effect over single-component strategies. Rather, given that barriers to the routine use of outcome measures exist at individual, managerial, and organisational levels, multifaceted implementation interventions may be more effective than single-component strategies in overcoming these barriers. 35 Moreover, our theoretical framework posits implementing and integrating a new practice, such as the use of the FMA, into routine practice as a product of four social mechanisms. 26 Multifaceted implementation strategies can thus interlink to establish social systems to act on the four social mechanisms to foster changes in behavioural norms and facilitate the desired practice transformation. 36
The task then is to determine what elements should be integrated into multifaceted implementation strategies. This is because a higher number of components increases the complexity of the intervention, which can compromise implementation fidelity and consequently reduce effectiveness. 37 Drawing on findings of a systematic review of effective interventions that promote professional behaviour change in healthcare underpinned by the NPT, the authors found that the most effective interventions emphasised coherence but also highlighted collective action and reflexive monitoring. 36 Healthcare professionals who successfully changed their behaviours likely did so because they understood the rationale behind the changes they were asked to make (coherence), and how their actions (collective action) aligned with external expectations (reflexive monitoring). 36 Mapping of implementation strategies employed by the hospitals in our study to the NPT showed that the multifaceted implementation strategies used acted primarily on the constructs of cognitive participation and collective action, with less focus on reflexive monitoring and coherence. This finding provides valuable insights into areas that need to be addressed in future tailored implementation strategies. While our study did not directly evaluate the effectiveness of past strategies, the theoretical analysis highlights the need to enhance reflexive monitoring and coherence. Potential strategies may include audit and feedback mechanisms, educational outreach initiatives, and reviewing existing reminders to target these NPT constructs.
Impact of organisational contexts on implementation of the FMA
Effective implementation strategies are essential for integrating an outcome measure into practice, but understanding the context is equally crucial. Stroke rehabilitation professionals work within specific contexts shaped by organisational structures, social norms, and group conventions, as outlined in the NPT. 26 NPT examines how social processes influence the implementation, embedding, and integration of practices within specific settings, thus highlighting the importance of contextual factors. Organisational prioritisation of outcome measurement created an environment conducive to initiating FMA implementation, as therapists are more likely to consider using the FMA when there is organisational or managerial emphasis on it. Furthermore, high organisational priority and support for outcome measurement also mean that resources such as time, funding, and training are prioritised to support the implementation of outcome measures in routine practice. While the impact of these organisational contexts on the implementation of post-stroke upper limb outcome measures are unknown, these contextual features are important for evidence-based practice implementation across healthcare settings. 38
Our analysis, limited to a single hospital network, also allowed us to indirectly study the influence of such an organisational structure on FMA adoption. Being part of a hospital network facilitated the spread of FMA use through processes including nesting, resource sharing, and efforts to improve care coordination. The nesting process involved training staff from new hospitals at established hospitals using the FMA, providing them with the opportunity to use and, for some, to implement the measure. These personal experiences with the FMA shaped their perceptions and assessments of its value and effectiveness. Assessments of their own experiences and the benefits of using the FMA influence their commitment to using it, as defined by individual appraisal within the NPT’s reflexive monitoring construct. 26 The positive assessment of the usefulness and effectiveness of the FMA likely influenced their decision to introduce the measure when they moved to the new hospitals.
All four hospitals in our study used a common online learning platform belonging to the hospital network for staff training. This platform streamlined the training process for the FMA, as training videos produced by one hospital were disseminated to all hospitals. This approach ensured a level of consistency in FMA administration and minimised duplication of resources. Additionally, being part of the same hospital network, where patients moved between hospitals within the network at different stages of their stroke recovery, encouraged therapists at newer complementary hospitals to adopt similar practices in using the FMA, enhancing overall care coordination.
Limitations
The current study has some limitations. First, our case selection was restricted to hospitals within a single network. Including hospitals from different networks could have revealed systematic differences that might have enriched the results. However, a range of hospital settings were still included in our study that enabled a robust examination of past implementation strategies. Secondly, our results may be limited by participant selection, as most participants were key individuals involved in the implementation phase. This may have led to a relative underrepresentation of smaller or individual initiatives by ground-level staff or therapists that supported the implementation of the FMA. Lastly, while using the NPT provided a valuable framework for understanding the social processes influencing implementation, it may have limited the exploration of other potentially relevant factors not captured by this theoretical lens.
Conclusion
Stroke rehabilitation guidelines worldwide advocate for the use of outcome measures to assess upper limb function after stroke. However, evidence of effective implementation strategies to implement standardised outcome measures is limited. Commonly employed multifaceted implementation strategies, including education, may not sufficiently address key barriers to their consistent integration into routine practice. This study reveals that current implementation strategies often fail to address key components of the NPT, particularly coherence (understanding how a measure differs from usual practices) and reflexive monitoring (using feedback to sustain its use). These gaps hinder the consistent integration of standardised outcome measures into routine clinical care. By identifying these critical shortcomings and their contextual influences, our findings provide a foundation for developing targeted, theoretically informed strategies to improve the implementation and sustained use of upper limb outcome measures in stroke rehabilitation.
Supplemental Material
Supplemental Material - Implementation of a post-stroke upper limb outcome measure: A qualitative multiple case study
Supplemental Material for Implementation of a post-stroke upper limb outcome measure: A qualitative multiple case study by Silvana X Choo, Joshua Yong, Shaikh Abdullah Bin Mohamed Rafi, Elaine Lum, and Julian Thumboo in Proceedings of Singapore Healthcare
Footnotes
Authors' contributions
SXC reviewed the literature and conceptualised the study, involved in data collection and analysis, drafted, and critically revised the manuscript. JY contributed to data collection and analysis. EL and JT contributed to the study conception and data analysis. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the SingHealth Duke-NUS Academic Medicine Research Grant (Special Category: Health Service Research) Grant number: (AM\HRT016/2021).
Ethical statement
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
