A Stroke Recovery Trial Development Framework: Consensus-Based Core Recommendations from the Second Stroke Recovery and Rehabilitation Roundtable

Abstract

A major goal of the Stroke Recovery and Rehabilitation Roundtable (SRRR) is to accelerate development of effective treatments to enhance stroke recovery beyond that expected to occur spontaneously or with current approaches. In this paper, we describe key issues for the next generation of stroke recovery treatment trials and present the Stroke Recovery and Rehabilitation Roundtable Trials Development Framework (SRRR-TDF). An exemplar (an upper limb recovery trial) is presented to demonstrate the utility of this framework to guide the GO, NO-GO decision-making process in trial development.

Introduction

Stroke recovery treatments that set the field on a radical new path are critically needed. ¹ The number of neutral Phase III trials in stroke recovery and rehabilitation suggests that a new approach is required. Recent trials are methodologically rigorous, but often designed with little regard for key, underlying biological mechanisms. The SRRR II Next Trials working group aimed to address the challenge of how we develop the next generation of stroke recovery trials to be both rigorous and aspirational to produce game-changing stroke recovery treatments. The recommendations of this group comprise the current paper.

We propose a new approach to guide the development of stroke recovery treatment trials, incorporating recommendations from SRRR I^1–5 and integrating decision analysis science. Stroke recovery trials in any treatment domain (speech and language, motor, cognition) require critical thinking and evidence gathering to appropriately inform decisions about whether to proceed with or hold off running a comparative effectiveness trial (i.e., the GO, NO-GO decision). To make GO, NO-GO decisions when designing a stroke recovery clinical trial, we should synthesize both preclinical and clinical evidence. Treatments will then be better informed, and this will help maximize the chances of a successful clinical trial.

In this paper we use the term “knowledge units” to refer to important areas for consideration in trial development, such as: HOW MUCH treatment, WHAT are the active ingredients of the treatment, WHO to treat, and WHEN treatment is best delivered. These considerations apply to behavioral treatments, the current cornerstone of treatment approaches. They also apply to ADJUVANT treatments, defined here as those treatments (e.g., drugs, non-invasive brain stimulation) that aim to modify the effect of a primary behavioral intervention. Drugs aimed at promoting neural repair have their own distinct considerations and have been discussed elsewhere.^6–8 Each knowledge unit is necessarily considered separately but we acknowledge that they interact.

Our goal was to create a framework to critically examine key knowledge units when developing stroke recovery treatment trials, which includes a GO, NO-GO decision pathway to guide selection of the most appropriate trial (including phase) given current knowledge.

Methods

To achieve our goal, we undertook a six-stage process outlined in Figure 1.

Figure 1.

Six stage methodological process.

Results

Defining the Knowledge Units

Stage 2 identified five priority knowledge units. The knowledge units were: HOW MUCH treatment to provide (n = 29 responses during consultation phase mentioned dose and intensity), WHEN to provide treatment (n = 29 mentioned “optimal” time window), WHO to provide treatment to (n = 29 mentioned subgroups, stratification and biomarkers), WHAT treatment to provide (behavioral treatment varied according to domain) and possible ADJUVANTS to behavioral treatment (n = 35 such as drugs or non-invasive brain stimulation).

Defining the Criteria

The prioritized evaluation criteria (preclinical and clinical) are presented in Table 1. These criteria were used to judge the importance and confidence of available evidence before reaching a final GO, NO-GO decision to proceed with a stroke recovery trial/experiment.

Table 1.

Prioritized list of evaluation criteria.

Preclinical		Clinical
Rank	Criteria	Rank	Criteria
1	High potential for clinical translation based on appropriately large effect size above what would be expected to occur spontaneously.	1	Prior research suggests that treatment stands to benefit a substantial fraction of the target patient group or provide an appropriately large effect size above what would be expected to occur spontaneously or both.
2	Rationale for intervention, outcome selection, and follow-up mechanistically and behaviorally, well conceptualized and described.	2	Based on appropriate sequence of prior phase studies (e.g., dose ranging, feasibility, safety, efficacy, fidelity, effectiveness) documented and reported.
3	Builds on appropriate efforts to replicate findings in house, externally, across different models (size, location) and across species (2 or more).	3	Rationale for intervention, outcome selection, and follow-up mechanistically and behaviorally well conceptualized and described.
4	Demonstrated in animals with different comorbidities especially cardiovascular and metabolic; across sexes and age spectrum.	4	High potential for clinical implementation and uptake.
		5	Identified potential for target population stratification based on patient characteristics.
		6	Builds on efforts that have been independently replicated.

Development of the Framework

A separate flowchart for each knowledge unit was developed (see Supplemental 1), containing a series of questions to prompt the researcher through the process. A detailed explanation of the purpose of each question within the framework is provided in Figure 2. The Stroke Recovery and Rehabilitation Roundtable Trials Development Framework (SRRR-TDF) includes review of the evidence, rating both the importance and confidence with available evidence to inform the start of the GO, NO-GO decision process. Where there is insufficient knowledge to proceed, research methods and earlier phase trial designs that can fill the knowledge gap need to be prioritized before proceeding to a Phase III recovery trial.

Figure 2.

Guidance on navigating the Stroke Recovery and Rehabilitation Roundtable Trials Development Framework (SRRR-TDF).

Using the SRRR-TDF for the Exemplar Trial: An Upper Limb Recovery Trial

We applied the SRRR-TDF to a hypothetical exemplar trial (see Figure 3 and Supplemental 3 for supporting references) consisting of an upper limb recovery trial using a behavioral treatment (±adjuvant therapy) with the aim of enhancing recovery. This was agreed in the Methods section, see stage 4. Overall, we found that most knowledge units had insufficient evidence to confidently design a protocol for a Phase III trial. In this example, a decision to not go ahead with a Phase III trial should be made. While the completed SRRR-TDF can be seen in Figure 3, for the supporting summary of evidence and gaps for each knowledge unit see Supplemental 2 and Supplemental 3. A brief summary of the issues raised is below.

Figure 3.

Application of the exemplar (upper limb recovery trial) to the Stroke Recovery and Rehabilitation Roundtable Trials Development Framework (SRRR-TDF). (a) HOW MUCH. (b) WHAT. (c) WHO. (d) WHEN. (e) ADJUVANTS. Note: References are contained in Supplemental 3.

HOW MUCH

Clear operational definitions of dose (including repetitions, duration, intensity) and dose schedule and consistent, accurate reporting across both preclinical and clinical studies is crucial. Preclinical studies commonly report repetitions of reaching tasks when describing dose, rather than time (minutes of training) and fail to report intensity. From available preclinical evidence, we are confident that doses >200 reaching repetitions in rodents,^9,10 and >500 repetitions in non-human primates ¹¹ improves reaching outcomes in the early subacute phase post stroke. Accurate data on dose delivered in human trials is rare; dose is often reported simply as minutes of therapy. Systematic reviews of clinical trials show a likely threshold dose (generally reported as total minutes) exists in humans.^12,13 However factors such as active minutes on task, number of repetitions and intensity are not well reported, ¹² this needs to improve. Based on current knowledge, we recommend future trials test doses of more than 1–2 h per day of time on task, as lower doses such as 30 min per day have already been tested and found to be sub-therapeutic,^12,13 and accurately repot repetitions and intensity. Studies have demonstrated that large amounts of time in therapy performing motor training is feasible.^14,15 Both preclinical and clinical experiments testing HOW MUCH are still needed; these must vary dose systematically and hold WHAT and WHEN constant, as well as measure potential harm, to advance this knowledge unit. Dose finding and dose ranging studies could be used (see the section “Approaches to trial design”).^16,17 Experimental designs should also incorporate testing of potential patient stratification variables (WHO), which may influence the patient response.

WHAT

Common upper limb behavioral treatments investigated in trials include constraint-induced movement therapy (CIMT), task specific and repetitive task training, and robotics (Pollack; 503 studies, 18,078 patients ¹⁸ ). Preclinical studies strongly support that when a sufficient dose is prescribed, practice of skilled reaching results in long-term, significant benefits. ¹⁹ However, confidence in WHAT behavioral treatment to use in humans is limited due to poor intervention reporting, conflicting findings, lack of evidence of minimum clinically important differences (MCID), and absence of sustained effects beyond the treatment period. This is the same for ADJUVANT treatments used in combination with behavioral treatments (see the section “ADJUVANT” for further discussion). Successful post-stroke recovery is likely to require more than one component, or “active ingredient.” ²⁰ We need to embrace the necessary complexity of an effective rehabilitation intervention in future trials, rather than studying each component in isolation, particularly as it may be the interactions between ingredients that is key to a successful treatment. ²¹ Elements of successful interventions to date include strengthening, massed practice (repetitions), part practice (shaping) and functional practice. ¹⁴ To achieve these goals, there need to be clear improvements in the way these complex interventions in stroke rehabilitation are developed, monitored and reported, to allow for rigorous testing. Experimental designs should also incorporate testing of potential patient stratification variables (WHO), which may influence the patient response.

WHO

Rarely in upper limb training trials (preclinical or human) do we test whether a treatment has different effects depending on the combination of impairments or biomarker status of an individual. Stratifying or selecting participants based on measurements that align with potential treatment mechanism or other underlying biology is uncommon. ²² Human recovery trials will be more efficient, ²³ and likely more effective, when we can select the patients most likely to respond to a specific treatment; rather than including unselected populations with high treatment-response variability and then clumping dissimilar individuals together in analyses. We agree that the next generation of trials should target the populations with greatest burden; for example people with moderate to severe upper limb impairment, ²⁴ rather than those with little impairment after stroke. We need to further test proposed stratification approaches in comparative effectiveness recovery trials, stratifying by baseline upper limb severity using, for example, Fugl Meyer, finger extension, SAFE scores, motor evoked potential (MEP) status, or models such as PREP,^2,4,23,25 but also by the presence or absence of other key impairments. In human trials, we also need to develop and validate approaches to measuring the “success” of a treatment that incorporates the degree of impairment (and other biological markers) of the patient at trial entry. This would acknowledge the known variability in rate and extent of recovery after stroke. A preclinical research priority is the study of potential mechanisms involved in recovery and responsiveness to training that better reflect lesion profiles that are common in the stroke population. Stratification based on motor ability could be adopted in the staircase or single pellet ⁵ retrieval tasks.

WHEN

Relative to stroke onset, the optimal timing of treatments, relative to stroke onset, to promote upper limb recovery post-stroke is currently unknown and few studies explicitly address this issue. In most clinical studies, WHEN to intervene is largely pragmatic rather than biologically informed. ²² From the available preclinical evidence in rodents (which includes independent studies that have been replicated), we are confident that starting upper limb training ≥5 to 14-days post stroke is feasible, safe and leads to better recovery in comparison to later start points. ²⁶ We are not confident about WHEN to commence upper limb interventions in human trials. The limited understanding of WHEN represents a funding priority. If we can identify the optimal window to intervene, with consideration of both patient subgroups (WHO) and therapy dose (HOW MUCH) and content (WHAT), we may be able to achieve clinical recovery that is more in line with the gains demonstrated from early intervention in preclinical models of stroke.

ADJUVANTS

The potential for applying adjuvant therapies, such as drugs, non-invasive brain stimulation or exercise, to enhance the effects of behavioral training in humans is tantalizing. However, systematic reviews in the field,^18,27–29 highlight the extreme variability of current approaches (HOW MUCH, WHAT, WHO, WHEN) when testing adjuvant treatments such as these in clinical trials. We considered ADJUVANTS in this exercise as a part of a potential multicomponent intervention package (see the “WHAT” section for further discussion). Careful phasing of trial development is urgently needed, in both preclinical and clinical domains, together with agreed definitions and standardization of protocols. Creative study designs might be useful to evaluate multiple therapies in parallel.

Approaches to Trial Design

Simple parallel group 2-arm RCT designs are unlikely to provide sufficient flexibility to answer the questions raised (HOW MUCH, WHAT, WHO, WHEN, ADJUVANTS). Simple trials can be augmented by additional planned data collection and more complex analyses to address questions of treatment moderators, i.e. interactions with treatment (WHO, WHEN) or treatment mediators, i.e. causal modelling of post-randomization variables (HOW MUCH, ADJUVANTS), feeding useful information into the next iteration of trials. ³⁰ Designs to support these questions should not be confused with feasibility designs (pilot RCT) whose primary aim is to ascertain whether it is possible to feasibly run (randomize, collect data, etc.) an RCT of a well-defined and potentially complex intervention, not to ascertain the efficacy of the treatment. Efficient, but more complex designs, enable understanding of multiple treatments, doses, intervention components, within a single trial (e.g., factional or fractional factorial designs, multi-arm, multi-stage (MAMS) designs). ³¹ They may differ in complexity and target individual or multiple questions simultaneously, as well as following different statistical paradigms (i.e., frequentist versus Bayesian). ³² Adaptive designs that allow pre-defined but dynamic modification of trial design during its course based on accumulating data, without undermining trial validity or integrity, may offer solutions to our challenges. ³⁰

Depending on the question, different kinds of adaptations are possible:

HOW MUCH and WHEN – Dose ranging trials allow sequential escalation of multiple dimensions of the dose in order to discover the range of potential doses to inform subsequent dose finding trials. ¹⁶ Prior to phase III, we need to consider greater use of well-known heuristic designs as well as the most current early-phase designs (e.g., the continual reassessment method (CRM) investigating dose escalation) and how to adapt trials methods to the stroke recovery and rehabilitation setting. For dose finding designs, adaptation where we potentially select the optimal doses or treatment time out of a range of potential alternative regimens, can happen dynamically in a responsive fashion to the outcomes observed previously in the trial. ³³ These designs are suitable for both preclinical and clinical research.

WHO – Population enrichment trials, which adapt patient groups with the ultimate goal of selecting those who will benefit most from a particular treatment, such as biomarker enriched trials (http://www.bigted.org/). These adaptations are likely to bring the most benefit once valid recovery biomarkers are identified.

WHAT – here we may learn from Oncology basket and umbrella trials, ³⁴ allowing multiple treatments to be evaluated within one overarching trial protocol. Similar to cancer, which is now understood to be increasingly heterogeneous across biomarker and genetically defined patient subgroups, stroke recovery (WHO considerations) may benefit from informed choices of patient subgroups and recovery trajectories.

Recommendations

Recommendations (Box 1) are based on the above summation of knowledge units.

Box 1.

Consensus recommendations.

1. Researchers interested in developing comparative effectiveness recovery trials in any domain (speech and language, motor, cognition, etc.) should apply the SRRR-TDF when developing trials, ensuring that all “knowledge units” have been considered in the process to inform GO, NO-GO decisions.2. When there is insufficient evidence to support a decision (i.e., NO-GO), the researcher should proceed to fill this knowledge gap. This may prompt an earlier phase design(s) (see the section “Approaches to trial design”), potentially followed by a feasibility study to inform the trial that will answer the primary question, i.e., does the intervention work?3. Funders of stroke research need to support earlier phase programs of research that aim to develop evidence to inform critical knowledge units in stroke recovery. These programs should be prioritized above large Phase III trials that do not have sufficiently strong biological evidence of effect or have positive findings from a single pilot trial. If progress is warranted, recommendations for intervention detailing, monitoring and reporting from the SRRR I should be followed. 3 4. HOW MUCH: To report dose, future trials should report all elements of dose (including repetitions, duration, intensity) and dose schedule, from all arms of the trial. Threshold doses for effectiveness need to be determined.12,13,22 5. WHAT: Effective training is likely to comprise several elements. Studies that aim to optimize promising treatments and use fine grained behavioral outcomes are required. These may identify active ingredients for further testing.6. WHO: Hypothesis-driven studies to identify reliable and valid assessments that distinguish biological subgroups, and responses to treatment are needed across both preclinical and clinical studies. Current potential stratification approaches need further testing in trials.7. WHEN: Complete clinical trials that enroll patients based on days post stroke, in epochs that are time-linked to our current understanding of the neurobiology of recovery. Greater focus on enrolling patients earlier post stroke is needed given the strength of preclinical evidence.8. ADJUVANTS: Clinical and preclinical researchers should jointly design studies to systematically test if training combined with an adjuvant is better than training alone. Careful phasing and reporting of trials is needed.

Discussion

Iterative modelling in developing clinical trials, particularly related to training interventions, in stroke recovery and rehabilitation is currently lacking.^3,22 Recognizing this critical gap, our group developed a framework for stroke recovery research, that incorporates a GO, NO-GO concept to guide the next generation of stroke recovery trials and experiments. This framework complements existing guidelines for complex intervention development.^35,36 We used the SRRR-TDF to test whether we had the evidence to support a GO decision for the conduct of a late phase, upper limb recovery trial. We found that the evidence for most knowledge units is inadequate, indicating that additional, earlier phase trials are needed to address priority gaps. Our group believes that honest and rigorous appraisal of the evidence that underpins essential knowledge units needed in developing a trial will ultimately lead to fewer, but better, trials progressing to Phase III.

In completing the SRRR-TDF exemplar, the core working group drew on systematic reviews, preclinical experiments and clinical trials in our evidence summaries. It was not our intention to conduct a rigorous systematic review of primary literature for each knowledge unit; although doing so would be justified in a real-world scenario. The inherent limitation of addressing each knowledge unit separately (WHO, WHEN, WHAT, etc.) is that rarely can one factor be separated from another. This does not diminish the value of the approach, but rather highlights the importance of identifying key factors to explore in recovery research. Given these complexities, preclinical research may be ideal for testing some of these complex relationships.

Stroke recovery is the next frontier in stroke research. ¹ To achieve major progress, we need significant commitment to filling vital knowledge gaps and aligned investment to realize the enormous potential to optimize recovery after stroke. The goal of this SRRR2 working group extends beyond improving the process by which researchers develop clinical trials and includes setting standards for the critical thinking necessary for the trial/experiment development process. Researchers and funders should prioritize research that can inform the development of stroke recovery treatments built upon strong mechanistic evidence. It is essential that we cease conducting trials and experiments not embedded in a systematic trial development process. This will reduce research waste and strengthen current endeavors to address critical knowledge gaps in stroke recovery. We can learn from related fields that reached similar crossroads; in 1999, acute stroke academic and industry leaders met and published standards for preclinical stroke drug development, ³⁷ and recommended a pathway for translational success. The scientific advances in acute stroke treatments we see today stem from collaboration, alignment and targeted research. To find new and better stroke recovery treatments, we must use a similar approach.

How can Preclinical Science Support Discovery for Clinical Impact?

Better alignment of preclinical studies to clinical realities or constraints should be a priority for the field and funding bodies. The failure to have meaningful two-way interaction has resulted in sub-optimal preclinical experiments and clinical trials, which have ultimately impeded progress. We suggest that an essential step towards enhanced stroke recovery success, albeit currently uncommon, is collaboration of preclinical and clinical scientists to develop studies tailored that specifically target components of knowledge units with NO-GO status for a given trial, thereby strengthening the evidence base for the knowledge unit. This may lead to forward and reverse translation, as well preclinical and clinical studies occurring in parallel.

Conclusion

The emphasis of our group was to build a model for development of comparative effectiveness trials of treatments targeting recovery that would increase the likelihood of trial success and identify vital knowledge gaps that need to be addressed. Although an exemplar from the motor system was used, the approach generalizes across neural systems. We advocate for stronger development of all trials in the stroke recovery and rehabilitation field. Large phase III trials may still be warranted for testing, for example, health service interventions, but these too should meet evidence criteria such as those developed here. While we call for the careful accumulation of evidence, we recognize that exciting new discoveries may result in an evidence leap that warrants rapid acceleration of a given treatment into trials. Funders of research in the stroke recovery field are urged to support earlier phase trial designs that address our knowledge gaps.