Abstract
Finding, testing and demonstrating efficacy of new treatments for stroke recovery is a multifaceted challenge. We believe that to advance the field, neurorehabilitation trials need a conceptually rigorous starting framework. An essential first step is to agree on definitions of sensorimotor recovery and on measures consistent with these definitions. Such standardization would allow pooling of participant data across studies and institutions aiding meta-analyses of completed trials, more detailed exploration of recovery profiles of our patients and the generation of new hypotheses. Here, we present the results of a consensus meeting about measurement standards and patient characteristics that we suggest should be collected in all future stroke recovery trials. Recommendations are made considering time post stroke and are aligned with the international classification of functioning and disability. A strong case is made for addition of kinematic and kinetic movement quantification. Further work is being undertaken by our group to form consensus on clinical predictors and pre-stroke clinical data that should be collected, as well as recommendations for additional outcome measurement tools. To improve stroke recovery trials, we urge the research community to consider adopting our recommendations in their trial design.
Introduction
Lack of a standardized approach to measurement in stroke recovery research hampers our ability to advance understanding of recovery mechanisms, devise better treatments and consolidate knowledge from a body of research using meta-analyses. 1 As examples, examination of a recent Cochrane Overview of interventions to improve upper limb function after stroke identified 208 unique assessment tools from 243 trials 2 ; another review found more than 100 measures of activities of daily living (ADLs). 3 Furthermore, in most motor rehabilitation trials, measures are taken at arbitrary time points relative to stroke onset, e.g. time of admission to, or discharge from, rehabilitation rather than at standard time points aligned with underlying recovery processes. 4
We must challenge the common assumption that most sensorimotor therapies are universally applicable and will achieve the same benefit for all people with stroke. The magnitude of change and likelihood of achieving clinically meaningful improvement in response to specific therapies will depend on age, stroke severity, and other factors including pre-existing comorbid conditions (e.g. diabetes, cognitive impairment, depression) 5 and pre-stroke lifestyle factors (e.g. social engagement, exercise). 6 The respective contributions of these factors have yet to be fully understood. Going forward, we need to identify the determinants that may help predict responders and non-responders to interventions.
The measurement working group of the Stroke Recovery and Rehabilitation Roundtable (SRRR) 7 was established to develop recommendations for standardized assessment time points and measures to be included in all adult trials of sensorimotor recovery after stroke. Given the current lack of standards for data collection and heterogeneous reports in stroke recovery trials, our expert group also considered pre-stroke clinical, demographic and stroke-related data that should be collected to improve clinical prediction of recovery and characterization of patient cohorts.
The decision to focus on sensorimotor recovery reflects the volume of existing trials in this area, the range of outcomes currently in use across these trials, and the gap in current research that known international initiatives has not addressed (e.g. Core Outcome Measures in Effectiveness Trials Initiative (COMET), National Institute of Neurological Disorders and Stroke Common Data Elements (NINDS CDE), The International Consortium for Health Outcomes Measurement (ICHOM), 8 Improving Research Outcome Measurement in Aphasia (ROMA) 9 and Standardization of Measures in Arm Rehabilitation Trials after Stroke (SMART), Supplementary Table 1). Acknowledging that clinical measures cannot distinguish between true neurological repair (behavioral restitution) and use of compensatory strategies, 10 a second objective was to consider whether we could recommend specific kinetic and/or kinematic outcomes that reflect quality of motor performance in order to better understand the neurophysiological changes that occur when patients improve.11,12 Our overall objective of the roundtable was to provide recommendations that, if applied, could improve the methodology of rehabilitation and recovery trials, help build our understanding of the trajectory of stroke recovery and aid discovery of new and more targeted treatments.
Methods
This consensus generating initiative was registered with the COMET group (http://www.comet-initiative.org/studies/details/910).
Development of core measurement set
Over five stages (Figure 1), our expert group developed the core measurement set. A ‘core’ measure was considered one that should be included in all stroke rehabilitation and recovery trials.
Stages to develop consensus on core measurements.
Stage 1
Establish measurement time point recommendations based on neural repair processes. 13 The SRRR definitions paper contains rationale for time points and description of terms. 10
Stage 2
Selection of stroke impairment constructs (sensorimotor, quality of life and global disability) and measurement tools (n = 72), identified through expert group survey and scoping of existing recommendations and resources (Supplementary Table 1).
Stage 3
Identification of desirable characteristics of a “good” measurement tool using Keeney’s Value Focused Thinking Methodology. 14 Via survey, the expert group elicited 19 desirable properties that subsequently formed six categories: (1) relevance of intended purpose to intended use; (2) responsiveness and sensitivity; (3) reliability; (4) feasibility; (5) ability to run meaningful statistical analysis; and (6) relevance to the International Classification of Functioning and Disability model (ICF model). The expert group members then ranked these six categories in order of importance for appropriate outcome measurement in clinical trials of stroke recovery and rehabilitation. We implemented a structured process used by Utley et al. 15 that aggregated rank-ordered lists produced by individual experts using a robust graph theory-based voting system as a decision-support tool in Microsoft Excel, and this was used to form a prioritized list for the expert group.
Stages 4 and 5
Respondents reviewed source documents pertaining to each tool and then ranked each measurement tool within each construct, considering how each tool overall met the desirable properties at each measurement time point, and elected whether the tool should be a core measure for that time point. For every individual construct at every given time point, Utley et al. 15 process was adopted to aggregate ranked lists by individual experts into a ranked list of preferred tools for the expert group.
Stage 6
Face-to-face discussion occurred at SRRR to finalize consensus on core measurement recommendations. Recommendations were reported back to the entire SRRR group (n = 60) for comment.
With consensus on measurement tools established, we then synthesized the consensus-based standards for the selection of health status measurement instruments (COSMIN) ratings, which evaluated the methodologic quality of the recommended outcome measures. 16
Standardized patient and stroke data
To meet our objectives for recommendations to include standards for collection of baseline patient and stroke related data, including clinical predictors of recovery such as pre-stroke data relevant to how well a person with stroke may engage in rehabilitation, small groups of three to five people reviewed the literature relevant to the area under consideration. These data were presented to the expert group at the SRRR meeting for consideration.
Results
Core outcomes
Box 1 summarizes the general recommendations for measurement in stroke recovery trials. Wherever possible, researchers wishing to test interventions aimed at sensorimotor recovery should commence assessment within seven days of stroke onset, then at set time points up to at least three months post stroke. Note that regardless of when the intervention ends, and the choice of primary endpoint, three month assessment is recommended for all trials. Additional time points may be selected as relevant for a specific research question. For trials testing interventions more likely to work at the level of behavioural compensation, that typically commence later post stroke, acquisition of stroke severity at stroke onset, as well as at time of recruitment to trial, is still strongly recommended.
Consensus Based Recommendations for Measurement in Stroke Recovery & Rehabilitation Trials.
Recovery trials are designed to examine the natural recovery trajectory and/or assess the comparative effectiveness of intervention(s) aimed at biological restitution. Rehabilitation is a process of active biological change in the central nervous system by which a person who has become disabled acquires the knowledge and skills needed for optimum physical, psychological and social function.
SRRR definitions paper (1) contains further descriptions.
Time points of applied measurements should always refer to time post stroke onset.
“Stroke onset” refers to within three days of onset of symptoms.
The National Institutes of Health Stroke Scale (NIHSS) should be used to measure ‘baseline’ stroke severity in all trials, regardless of when the trial starts*. Individual item and total NIHSS scores should be reported. Where possible, NIHSS should be assessed at stroke onset (i.e. within three days of symptoms) for all trials.
For recovery trials, the first outcome assessment should be completed within seven days of stroke onset.
Always include three-month post-stroke assessment points in recovery trials (essential). Consider six months and twelve months (especially for health economics). Later time points are dependent on trial design.
Core measures should be collected in every trial. Stroke severity and outcome measures for each time point are included in Table 1, see below for standard demographics and stroke data, and Supplementary Table 4 for outcome measure training options and cost information. Full measures (not sub-sections) should be collected with exception of the Fugl-Meyer assessment (FMA) for which only the motor part is recommended (Table 2).
Adverse events should be pre-specified and collected according to Good Clinical Practice (GCP).
Recovery trials need to consider serially applied kinematic/kinetic measurements alongside clinical assessments to distinguish between restitution and compensation. A core set of kinetics and kinematic outcomes needs to be established.
Pre-stroke patient-level data (eg physical activity, cognition): No specific recommendation, but this is considered an important area of future enquiry.
*Note: Stroke severity at onset is an important predictor of outcome, so should be routinely collected.
Age: Years and category: 18 - 55, 56 – 74, >75 years
Sex: Male / Female / Other
Ethnicity: self-description
Medical History Vascular risk factors (coronary artery disease, AF, diabetes, hypertension, clinical obesity, smoking and alcohol use, hyperlipidemia) Renal or cardiac failure Prior stroke or transient ischaemic attack (TIA) Co-morbid conditions (cognitive decline, osteoarthritis, other neurological disease)
Premorbid function: Modified Rankin Scale (mRS)
Education: year count
Premorbid walking status: Independent with or without gait aid / with assistance / unable
Premorbid living arrangements Living alone Y / N Living at home / supported accommodation
Stroke severity: National Institute of Health Stroke Scale (NIHSS)
Active hand movement at stroke onset? Y / N
Ability to walk independently at stroke onset? Y / N
Stroke type: Ischaemic / Haemorrhage
Stroke sub-type: Lacunar / Large artery / Other (e.g. carotid dissection) / Undetermined
Stroke location: Cortical: Internal Capsule / Middle cerebral artery (MCA) / Frontal lobe Subcortical: Thalamus / Basal Ganglia Midbrain: Pons / Medulla / Cerebellum Brainstem
Thrombolysis / reperfusion therapy: Y / N
Imaging: Confirmed stroke on imaging Y / N CT obtained Y / N MRI obtained Y / N
Core measures (Table 1, Supplemental Table 2), standard demographics and stroke data (Box 1) should be gathered in all trials. Having evaluated the performance of each measure at each time point, some core measures are not required/relevant at certain time points. COSMIN ratings for each of the recommended outcome measures are shown in Supplementary Table 3. Stroke severity should be assessed using the National Institutes of Health Stroke Score (NIHSS), but we recommended the NIHSS not be used as a measure of recovery. Additional outcome measures, aligned with the trial purpose and target intervention can be added. We could not reach consensus on a participation measure, all exhibited generally inadequate measurement properties. Consensus-based recommendations for additional stroke measures, including sensory measures, will be available in future from the SRRR group (work in progress).
Recommendations of Core Measures to Include in Every Stroke Recovery Trial.
ARAT: action research arm test; EQ-5D: self-report questionnaire of health status; FM: Fugl–Meyer; ICF: international classification of functioning and disability model; NIHSS: national institute of health stroke scale; mRS: modified Rankin Scale (Note: we recommend use of the Simplified mRS in line with ICHOM. 9
Include three month assessment in all trials
NIHSS is recommended to classify stroke severity, but not as an outcome measure.
Two core measures of walking are recommended: (1) Can person walk 10 m independently (gait aids permitted) Y/N? and (2) timed 10-m walk test at comfortable pace if able to walk 10 m independently. For data integrity, do not score (or impute zero) for people who cannot complete this test, rather, record unable/not applicable. Therefore, to avoid missing data and preserve validity, for trials with the primary aim of measuring the progress in walking, we do not recommend 10-m walk as a primary outcome measure. For such trials, an alternative measure should be considered, e.g. a continuous and fully evaluable primary outcome measure such as 6 minute walk.
Evidence to support inclusion of any one tool as a core measure was lacking.
Patient demographics, pre-stroke data and baseline data
Due to the limited time available at the SRRR meeting, a formal consensus approach to determine a standard collection of demographic, pre-stroke or clinical prognostic data was not possible. After small group review and post meeting discussion, recommended patient demographics and baseline data to be included in all trials are shown in Box 1. Considerations for these recommendations are as follows: Patient-level data are used to characterize the study population, enable exploration of factors that may influence recruitment, retention and response to intervention and allow later data pooling with similar studies, at an individual or group level. Baseline patient level data are also used for stratification (i.e. to assign patients with similar potential for recovery to groups at randomization to ensure that treatment groups are balanced.5,17–19 Several prognostic tools developed to determine risk of recurrent stroke and clinical outcomes (e.g. death, disability, and discharge destination) are considered as a valuable approach for selecting or monitoring acute trial participants (see Supplemental Table 4 for summary of tools).20–23 In stroke recovery trials, we have yet to determine the important clinical biomarkers of stroke recovery. 24 Age and initial stroke severity are the strongest predictors of outcome after acute stroke. Some comorbid conditions have also been associated with poorer stroke recovery (e.g. heart failure, chronic kidney disease, diabetes, atrial fibrillation). In addition to the presence of chronic diseases,25,26 social and lifestyle factors, psychological, cognitive and physical factors 7 may impact post-stroke recovery trajectories. Patients’ medical histories usually include presence of comorbidities, living arrangements pre-stroke, degree of social support, occupation and lifestyle factors. Additional factors such as pre-existing depression or mood disorders which are associated with poorer outcome 6 are often ascertained via routine screening procedures in clinical practice and should also be considered in our trials.
Cognitive reserve (the ability to cope with brain damage) is one mechanism postulated to explain differences in functional ability in patients with similar degrees of brain damage. 27 However, we currently lack an adequate measure of cognitive reserve for use in stroke trials. Level of education is a marker for pre-stroke cognitive function and has a role in determining cognitive outcomes in people with vascular dementia; it is similarly associated with aphasia severity and long-term survival after stroke, so we recommend its inclusion here. While the association between physical activity and stroke risk is established,28,29 we found conflicting evidence about the impact of pre-stroke physical activity on stroke severity and outcome. The estimation of pre-stroke physical activity is difficult as self-reported measures are inaccurate, use of the question “How often do you exercise vigorously enough to build up a sweat?” is only moderately correlated with objective measures of fitness, and more general physical activity questionnaires are inadequate. 30 At this stage, we could not recommend inclusion of a specific pre-stroke measure of physical activity; however, validation of such measures should be a priority. Active hand movement and walking at admission (items 10 and 11 in Box 1) are recommended particularly in trials that commence later post stroke where NIHSS at stroke onset could not be gathered. These ‘simple questions’ of baseline performance are included in a number of stroke registries because they are associated with outcome. They may be helpful in retrospective data pooling or analysis.
Measuring quality of motor performance in stroke recovery trials
To improve our understanding about the mechanisms that drive motor recovery, we need to delineate between ‘true neurological repair’ (i.e. restitution) in which neurological impairments are restored towards normal and behavioral compensation strategies.31–33 While clinical measures of activity limitations such as the Action Research Arm Test and walking speed can measure change, they cannot delineate between restitution and compensation. Technological developments in wearable miniaturized sensors, robots and force sensors may make monitoring of kinematics and kinetics feasible as additional outcome parameters to objectively measure improvement in stroke recovery trials.34,35 While desirable, our group determined that reaching consensus with regard to the best kinematic and kinetic parameters to assess sensorimotor control was not feasible in the given time frame. That said, we strongly believe that kinematic and kinetic measures should be considered essential in future trials and be developed accordingly. These tools represent the best way to distinguish behavioral restitution from compensation. Below we offer a brief commentary on those measures that look promising for upper limb and walking recovery.
Some recommendations already exist for gait analysis, 36 standardization of parameters of upper limb coordination of reach to grasp movements 37 and upper limb robotics. 35 Kinematic measures of the movement endpoint, whole trajectories, joint angles as well as correlations between key events in the transport and grasp phases of reaching can be used to address questions about movement quality after stroke. For example, are patients becoming more accurate and less variable? Are trajectories becoming smoother and less curved? Are movements beginning to be controlled out of synergy? Depending on how the kinematics is analyzed, it is possible to determine whether a given movement is compensatory, for example excessive trunk movement, or becoming more similar to a normal movement.31,33,35,38–48 Kinematic data can be obtained during performance of a specific functional task, for example attempting to pick up a glass, or with specially designed non-functional assays, for example the finger individuation index using a cyber-glove or planar reaching tasks. For both the functional tasks and the motor assays, it is possible mathematically to compute a distance between a patient’s global kinematic trajectory and a control data set.
For the lower limb, normalization of the quality of motor performance is strongly reflected by measures of inter-limb coordination such as left-right symmetry in spatio-temporal parameters of gait, and kinetic parameters such as torque and (ground reaction) forces for establishing the (a)symmetry between the hemiplegic and non-hemiplegic side, such as displacement of centre of mass while standing49–51 and walking. 52 All of these measures are seen as important outcomes reflecting the symmetry, and with that, quality of hemiplegic gait performance. Recommending specific measures (and methods) for inclusion in trials is challenging. Broader collaboration between investigators in similar areas of study to align outcomes is encouraged.
Discussion
In our ‘moving rehabilitation research forward’ paper, 7 we outlined why standardising measurement in trials is important to our field. Over the course of a year, the SRRR measurement group undertook extensive literature reviews, consulted with experts in measurement and decision analytics and used a pre-defined methodology to achieve consensus about a core set of valid measures that should be used in every stroke recovery and rehabilitation trial and be assessed at fixed times post stroke. This important step will make stroke trials more comparable, irrespective of their results (i.e. negative, neutral or positive). The pooled data sets from different trials may allow individual patient-level meta-analyses in which responders and non-responders to an intervention can be identified at different recovery phases post stroke (see also recommendations from the SRRR intervention group). 53 Better knowledge of the profiles of patients that benefit most, will not only help in designing better trials in terms of adequate stratification (see for example EXCITE 18 and EXPLICIT stroke trials 19 ) but will also generate new and better hypotheses about how therapies work at different phases post stroke and the underlying mechanisms of recovery.
We achieved our primary goal of developing the core set and providing guidance about standardized demographic and stroke-related data which we recommend to be gathered in all trials. We recognize that the core set may include measures unfamiliar to some researchers, so we have provided links to resources for training in Supplementary Table 2. Assessor training is required for good clinical practice, and standardization of training and certification protocols has been shown to reduce variance in scoring, thereby increasing power and reducing trial costs. 54 We should note that this core set of measures may be insufficient to satisfy the needs for measuring outcomes for a specific stroke recovery or rehabilitation research question. Other outcomes may be added; however, we strongly recommend that researchers be judicious in their choice of additional measures, limit the number of outcomes assessed and identify, a priori, the primary outcome measure and power the trial accordingly. We recommend that researchers follow standards for recording adverse events in trials (e.g. falls, pain, etc.). 55 Since the SRRR meeting, we have expanded the current methods to include a more extensive list of outcomes (e.g. participation, depression, sensation, strength). Similarly, further recommendations for additional outcomes will follow for the use of kinematic and kinetic measures. However, adoption of the current recommendations outlined in this and other papers in this series would alone significantly advance the quality, reproducibility and rigor of stroke recovery and rehabilitation research. There is considerable urgency to do this. Results of most recent phase III and IV trials have been largely neutral or negative. At a time when funding for all clinical trials is undergoing unprecedented scrutiny, convincing funding bodies to commit to further trials will become more difficult in the absence of such self-imposed discipline. We must ensure that our trials are mechanistically well conceptualized, with careful matching of the nature of the intervention and the outcome measure chosen. A recent review of manuscript submissions to one of the major stroke journals found that ‘rehabilitation’ studies reached lower priority scores and were less likely to be accepted for publication. 56 The authors hypothesized that inconsistencies in the described methodology and lower training of rehabilitation specialists in research methods would explain the findings. Adhering to agreed definitions, using standardized measures and reporting recommended outcomes would significantly improve the quality of stroke recovery trials.
Our target was ambitious. Due to the currently limited evidence available, there remain gaps in our knowledge that hamper further progress. We do not know the best behavioral prognostic factors to include in stroke recovery trials, nor how to best stratify patients. While early ‘stroke severity’ is strongly predictive of functional outcome on a group level, other patient and stroke-related factors will influence response to rehabilitation: we need to determine the most powerful prognostic factors of stroke recovery57–60 to optimize stratification.5,18 We have yet to determine which preclinical factors and other risk factors should be included in every stroke recovery trial, but better characterization of our patients, their stroke and their recovery will help us identify ‘recovery phenotypes’.
We agreed that we need to objectively measure quality of motor performance using technology to help distinguish restitution and compensation, although we were unable to recommend a specific set of parameters at this stage. This step is vital to enhance current knowledge about heightened levels of plasticity and its interaction with motor learning early post stroke, and for trials seeking to test interventions targeting brain repair. 61 Recovery trials should start early post stroke, and include both core clinical measures and kinematics assessed serially at standard intervals post stroke. Use of these measurement tools and methods of data synthesis and analysis in clinical trials are still in their infancy but are moving forward quickly as technology evolves. Providing recommendations in this area is a future target of our group.
In this first step, our SRRR group targeted sensorimotor outcomes, not cognitive or other impairments that are the focus of other groups.8,9 While there is more to do in this field, the value and urgency of working towards standardized measurement, in clinical practice and in research, are indisputable. Stroke rehabilitation is a beacon of hope for stroke survivors, but stroke recovery is exceedingly complex and progress towards effective treatments has been frustratingly slow. Reasons for the slow progress are likely attributable to the noise introduced by under-conceptualized mechanistic frameworks and omissions of better ways to categorize our patients and our outcomes. As a research and clinical practice community, we need to work together to combine the latest concepts and findings in neuroscience and experimental psychology with sophisticated clinical trial methodologies and outcomes research.
Supplemental Material
Measurement_Supplementary_Tables – Supplemental material for Standardized Measurement of Sensorimotor Recovery in Stroke Trials: Consensus-Based Core Recommendations from the Stroke Recovery and Rehabilitation Roundtable*
Supplemental material, Measurement_Supplementary_Tables for Standardized Measurement of Sensorimotor Recovery in Stroke Trials: Consensus-Based Core Recommendations from the Stroke Recovery and Rehabilitation Roundtable* by Gert Kwakkel, Natasha A. Lannin, Karen Borschmann, Coralie English, Myzoon Ali, Leonid Churilov, Gustavo Saposnik, Carolee Winstein, Erwin E. H. van Wegen, Steven L. Wolf, John W. Krakauer and Julie Bernhardt in Neurorehabilitation and Neural Repair
Footnotes
Acknowledgements
We thank Paulette van Vliet for her contributions to this paper. We would like to acknowledge Professor Julie Bernhardt for convening this first Stroke Recovery and Rehabilitation Roundtable (SRRR), Ms. Katie Lafferty for moderating discussions at SRRR and Dr Karen Borschmann for project management.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: JB was supported by NHMRC (1058635) and NL holds a NHMRC TRIP fellowship (GNT1112158).
We acknowledge the following organizations for their financial support of the meeting: NHMRC CRE Stroke Rehabilitation & Brain Recovery (Australia), Heart and Stroke Foundation of Canada, Heart and Stroke Foundation Canadian Partnership for Stroke Recovery. An unrestricted educational grant was provided by Ipsen Pharma.The Florey Institute of Neuroscience and Mental Health acknowledges support from the Victorian Government, in particular funding from the Operational Infrastructure Support Grant.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
