Abstract
Animal models have been developed to simulate angiographic vasospasm secondary to subarachnoid hemorrhage (SAH) and to test pharmacologic treatments. Our aim was to evaluate the effect of pharmacologic treatments that have been tested in humans and in preclinical studies to determine if animal models inform results reported in humans. A systematic review and meta-analysis of SAH studies was performed. We investigated predictors of translation from animals to humans with multivariate logistic regression. Pharmacologic reduction of vasospasm was effective in mice, rats, rabbits, dogs, nonhuman primates (standard mean difference of −1.74; 95% confidence interval −2.04 to −1.44) and humans. Animal studies were generally of poor methodologic quality and there was evidence of publication bias. Subgroup analysis by drug and species showed that statins, tissue plasminogen activator, erythropoietin, endothelin receptor antagonists, calcium channel antagonists, fasudil, and tirilazad were effective whereas magnesium was not. Only evaluation of vasospasm >3 days after SAH was independently associated with successful translation. We conclude that reduction of vasospasm is effective in animals and humans and that evaluation of vasospasm >3 days after SAH may be preferable for preclinical models.
Introduction
Outcome from aneurysmal subarachnoid hemorrhage (SAH) has improved. Mortality declined 0.4% per year, after adjustment for age, between 1973 and 2002 (Nieuwkamp et al, 2009). The main changes in management that have occurred are improvements in intensive care, early securing of the ruptured aneurysm, and use of nimodipine. Many other drugs have been tested in animal models but none has proven effective in high-quality clinical trials. Most of these studies tested drugs directed at reducing angiographic vasospasm. This was based on the hypothesis that since angiographic vasospasm is strongly correlated with cerebral infarction (Crowley et al, 2011) and cerebral infarction is correlated with unfavorable clinical outcome on the Glasgow outcome scale (Fergusen and Macdonald, 2007) then reducing vasospasm should improve clinical outcome.
There are many possible reasons for lack of translation of reducing angiographic vasospasm in animal models to improving outcome in humans, including clinical trial design, whether the animal model reproduces the disease and methodological flaws in experimental studies (van der Worp et al, 2010).
Preclinical SAH studies for pharmaceutical reduction of vasospasm differ in terms of animal used, induction of SAH, time course and severity of angiographic vasospasm (Megyesi et al, 2000; Titova et al, 2009) as well in quality of design (i.e., randomization, blinding). All these characteristics could impact the results of experimental treatment and perhaps the chance of translation, but to our knowledge there is no evidence that one model of SAH is better than any other for the purposes of predicting whether the findings in the animal study of a drug in relation to angiographic vasospasm are the same as what is found for its effects on angiographic vasospasm in human clinical trials. A meta-analysis of human clinical trials of drugs to decrease vasospasm showed that some, but obviously not all drugs tested did reduce vasospasm in randomized clinical trials (Etminan et al, 2011). Thus, the primary hypothesis tested here was that some specific animal model or characteristic of one predicts the findings in clinical trials. The idea was that if, for example, one specific species or method of induction of SAH or some such characteristic of an animal model was more likely to be associated with successful translation to prevention of angiographic vasospasm in humans, then maybe that characteristic should be used for preclinical studies. The study of Etminan et al (2011) reported that in the clinical trials, angiographic vasospasm was reduced but there was no significant effect on clinical outcome. Thus, another hypothesis that could be tested would be what animal models or characteristics of them correlate with clinical outcome in humans. The present analysis focuses on angiographic vasospasm.
Materials and methods
We conducted this systematic review and meta-analysis according to the methods recommended for assessing health technologies and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Higgins and Green, 2008; Liberati et al, 2009; Mignini and Khan, 2006; Moher et al, 2009). No language restrictions were used and two reviewers assessed each reference and determined study eligibility.
Criteria for Included Animal Studies
Type of studies
We included preclinical studies in vivo where SAH was simulated by endovascular puncture of a cerebral artery, subarachnoid clot placement, or injection of blood, hemolyzed blood or other purported vasospastic substances into the subarachnoid space.
Type of intervention
We included studies of prophylactic treatment of angiographic vasospasm with any drug tested in both experimental and clinical trials for the same purpose. Intervention was defined as treatment for the purposes of this study if drug administration was started before or under the same anesthetic time as when the SAH was induced in the animal model or within 24 hours of the last injection of blood into the subarachnoid space. We did not include studies where drugs were administered to reverse established angiographic vasospasm. For selecting drugs, we considered a systematic review of drug treatments for humans with SAH and included all drugs tested in randomized, blinded clinical trials (Etminan et al, 2011).
Type of outcome
The primary outcome was cerebral vasospasm evaluated by catheter, CT (computed tomographic), or magnetic resonance angiography or by histological methods.
Search Methods for Identification of Studies
Pubmed and Embase were searched from 1966 to November 2010, using the following terms: (subarachnoid hemorrhage) AND ((statin) OR (tissue plasminogen activator) OR (ticlopidine) OR (erythropoietin) OR (clazosentan) OR (nimodipine) OR (nicardipine) OR (fasudil) OR (tirilazad)) AND (animal OR experimental). The same search was performed for magnesium, endothelin converting enzyme, and endothelin receptor antagonist. The reference lists of the papers were checked to identify other studies not found in the initial search.
Data Collection
For every included study, we extracted data on the species and sex of animals, method of induction of SAH, drug tested, timing and method of delivery of the drug, time and method for outcome assessment, randomization, blinding, sample size estimation, inclusion and exclusion criteria and physiological variables that were measured. Data on the number of animals and degree of vasospasm in treatment and control groups were extracted. If in the same study different doses or delivery way were used, data on the effect of the intervention were extracted for every group and the most effective regimen was used for the analysis, although for assessment of heterogeneity and publication bias, results for the best as well as all (114 comparisons) are provided. If in the same study different drugs were tested, data on the effect of each drug were extracted and analyzed separately. Vasospasm was expressed as the percent reduction in cerebral artery diameter from baseline (in the studies where angiographic evaluation was performed) or compared with control values (in the studies were histological evaluation was performed). If the degree of vasospasm was expressed as reduction in artery cross sectional area, the diameter was calculated using the formula: diameter = 2 × √(area/π). If data were presented only in figures, then values were estimated using digital ruler software.
Methodologic quality of the animal studies was assessed by two reviewers by extracting data on study design, allocation concealment, blinding, number of animals randomized, age of animals, recording of physiologic variables, and inclusion of controls. We did not include funding source since this was not always reported, and we did not use a checklist and a score to assess study quality. This was because according to van der Worp et al (2010), and other sources, use of these is controversial and it is not clear that any particular characteristics are better than any others. We also did not extract data on other variables reported in some other systematic reviews (such as the neuroprotective effect of anesthesia) because in most papers there were insufficient data provided.
To assess the effect of each drug in human trials, the data were obtained from Etminan et al (2011). The outcome measure was radiographic vasospasm, which was defined as a focal or generalized reduction of cerebral arterial caliber on catheter cerebral angiography, or increased cerebral blood flow velocities on transcranial Doppler ultrasound. The included studies used various definitions of angiographic and transcranial Doppler vasospasm, so we used the definitions reported by the investigators of the original studies. If the investigators categorized angiographic vasospasm into no/mild, moderate, and severe vasospasm, we used the number of patients that had moderate-to-severe vasospasm as an outcome event. Transcranial Doppler vasospasm was defined as flow velocity of at least 120 cm/s or peak flow velocity of >200 cm/s. If studies reported angiographic and transcranial Doppler vasospasm, the data on angiographic vasospasm were used (Etminan et al, 2011).
Statistical Analysis
To test which species may reflect more accurately the effect of drug treatment on reducing vasospasm in humans, we analyzed the concordance between the drug effect in human and in the different animals. The drug effect in humans was defined as focal or generalized reduction in cerebral artery diameter on catheter angiography, or increased cerebral blood flow velocities on transcranial Doppler ultrasound, as reported previously (Etminan et al, 2011). If studies reported both catheter angiography and Doppler ultrasound, angiography results were used, and the outcome event was number of patients with moderate or severe vasospasm. The effects were defined as concordant if the drug showed evidence of positive effect in humans as well as in animals or no evidence of effect in humans as well as in animals. Drug effect was defined as discordant if the drug showed evidence of efficacy in animals but not in humans or vice versa.
A random effect standardized meta-analysis was used to assess the effect of pharmacologic reduction of vasospasm in animal studies. Effect sizes were expressed as pooled standard mean difference. Subgroup analyses by drug and animal species were performed. We used meta-regression to analyze the source of heterogeneity, if any. Factors included in the meta-regression were species and sex of animals, method of induction of SAH, drug tested, route of drug delivery, treatment start time (before or after SAH), time and method of outcome assessment, randomization, blinding (for outcome assessment and whether animals belonged to treatment or control groups before surgery), monitoring of physiological variables (temperature, blood pressure, heart rate, oxygenation, intracranial pressure), and monitoring of CO2 (Table 1).
Characteristics of animal studies
SAH, subarachnoid hemorrhage.
Modified cisterna magna injection and prechiasmatic cistern injection.
Tissue plasminogen activator plus endothelin antagonist.
Publication bias was investigated using Begg's and Egger's tests; Begg's funnel plots are shown. For human studies, effect sizes were expressed in pooled risk ratio estimates. To assess predictors of translation from animal to human, univariable and multivariable logistic regressions were used with the dependent variable being concordance/discordance of the drug. Variables entered in the regressions were the same as the variables used for meta-regression. Bonferroni correction was used for multiple comparisons. P<0.05 was considered significant.
Results
A total of 453 studies were identified. Three hundred nineteen studies were excluded after review of the title and abstract showed the study was a review article, clinical studies with no single treatment, in-vitro study or test of a treatment to reverse established angiographic vasospasm. An additional 64 studies were excluded after reading the full text because they did not completely fulfill the inclusion criteria or they reported preliminary or incomplete data. Data from 70 papers were then extracted (Bilginer et al, 2009; Bulsara et al, 2006; Caner et al, 1996; Chang et al, 2010; Chen et al, 2009; Cheng et al, 2009; Chung and Lee, 1993; Cosentino et al, 1993; Espinosa et al, 1984; Findlay et al, 1988, 1989, 1990; Grasso et al, 2002; Gul et al, 2010; Hanggi et al, 2009; Hariton et al, 1993; Hino et al, 1995; Itoh et al, 1993, 1994; Josko et al, 2000; Kanamaru et al, 1990, 1991; Kawada et al, 1999; Kawashima et al, 2000; Kim et al, 1996, 2000; Kita et al, 1998; Kwan et al, 1997, 2001, 2006; Laslo et al, 2006; Lewis et al, 1988; Lin et al, 2007; Macdonald et al, 1998a, 1998b, 2004; Marbacher et al, 2008; Matsui and Asano, 1994; Matsumura et al, 1991; McGirt et al, 2002, 2006; Mesis et al, 2006; Murphy et al, 2008; Nirei et al, 1993; Nosko et al, 1985; Ohkuma et al, 1999; Pasqualin et al, 1991; Roux et al, 1997; Santhanam et al, 2005; Satoh et al, 1992; Seifert et al, 1989; Shigeno et al, 1995; Steinke et al, 1989; Sugawara et al, 2008; Suzuki et al, 1998, 1999; Takanashi et al, 2001a, 2001b; Tang et al, 2008; Turowski et al, 2007; Vollmer et al, 1989; Wanebo et al, 1998; Wanebo et al, 1997; Wang et al, 2010; Willette et al, 1994; Zabramski et al, 1986; Zimmermann et al, 1996; Zuccarello et al, 1989, 1996, 1998). In four papers, two drugs were tested, leading to a total of 74 comparisons for analysis (Cosentino et al, 1993; Hino et al, 1995; Zuccarello et al, 1996, 1998). There were 556 animals assigned to treatment groups and 664 to control groups.
Characteristics of the Studies
Investigators used monkeys (Macaca species), dogs (mongrel or beagles), rabbits (Japanese or New Zealand species), rats (Sprague Dawley or Wistar species) and mice (Table 1; Supplementary Table 1). The most used methods to simulate SAH were two injections of blood into the cisterna magna (dog, rabbit, and rat studies), a single injection of blood into the cisterna magna (rabbit and rat studies) and craniotomy followed by clot placement around the cerebral arteries (all monkey and one dog studies). In the other studies, endovascular perforation (rat and mouse studies), prechiasmatic cistern blood injection (one rat study) (Wang et al, 2010) or modified versions of cisterna magna injection (one rat (Tang et al, 2008) and one dog study) (Zabramski et al, 1986) were used. No studies injected hemolyzed blood or other substances into the subarachnoid space. In 48 studies (52 experiments), vasospasm was assessed using catheter or CT angiography while 22 used histology. In 49 studies (53 experiments), vasospasm was evaluated >3 days after SAH and in 21 experiments evaluation was performed before or on day 3. The degree of vasospasm in the control group was mild (<33%) in 16 experiments, moderate (33 to 66%) in 55 and severe (>66%) in three.
Drugs tested included tirilazad, tissue plasminogen activator, calcium channel antagonists (nimodipine or nicardipine), erythropoietin, statins, fasudil, endothelin antagonists, or endothelin converting enzyme inhibitors, magnesium, and in one study, tissue plasminogen activator plus an endothelin antagonist. There were no experimental studies of ticlopidine. Drugs were administered intracranially in 30 experiments and systemically in 44 (intravenous, oral, or intraperitoneal). In seven cases, treatment was started before the induction of SAH (pretreatment).
Regarding the quality of the studies, none clearly reported a sample size calculation. Only 12% described inclusion and exclusion criteria, 51% used randomization, 20% reported blinding of whether animals belonged to treatment or control group before surgery, and 60% reported blinding of assessment of outcome. Two or more of five physiological variables (heart rate, blood pressure, body temperature, intracranial pressure, oxygenation) were monitored in 80% of studies and blood or end-tidal CO2 was monitored in 78%.
Overall Treatment Effect
The pooled effect size expressed as standard mean difference showed that pharmacologic treatments significantly reduced cerebral vasospasm in the experimental studies (standard mean difference of −1.74;. 95% CI (confidence interval) −2.04 to −1.44; Figure 1). Substantial and significant heterogeneity was detected (heterogeneity χ2 statistic=340.60 (degrees of freedom=73), P=0.0001). In univariate meta-regression including all characteristics reported in Table 1, only the use of calcium antagonists (nicardipine or nimodipine) showed a significant association with the pooled effect size. Blinding induction of SAH revealed a marginal association (P=0.052). In the multivariate meta-regression including study factors with P≤0.15, treatment with calcium antagonists remained significantly associated with the pooled effect size (Supplementary Table 1).

Effect size of 74 animal studies of pharmacologic reduction of vasospasm included in meta-analysis. Gray band represents overall effect with 95% confidence intervals (∗ and ∗∗ are effect of different drugs in the same publication).
Subgroup Analysis by Drug
Subgroup analysis by drug showed that every drug except magnesium was effective for reducing vasospasm. Heterogeneity was detected for every subgroup except for fasudil (Figure 2). Meta-regression was conducted for drug type versus variables significantly (P<0.05) associated in univariate analysis. This showed that the effect of erythropoietin and statins were associated with the method of induction of SAH, while calcium antagonist effect was related to the use of monkeys and mice (Table 2). This means that the effect of erythropoietin and statins are different (higher for statins, smaller for erythropoietin) in studies where cisterna magna double injection model was used compared with studies where cisterna magna single injection was used and that effect of calcium channel antagonists were different in studies where monkeys (smaller effect) or mice (higher effect) where used compared with the other studies.
Subgroup analysis by drug and species

Effect size of the 74 included studies grouped by animal and drug. Gray band represents overall effect with 95% confidence interval. The size of the circle reflects the number of animals included in the different subgroups (ET antagonist: endothelin antagonist and endothelin converting enzyme inhibitor, MgSO4: magnesium).
Subgroup Analysis by Species
Subgroup analysis by species showed that pharmacologic treatment was effective in all species (Figure 2). Again, heterogeneity was detected and meta-regression showed that the pooled effect in rats was associated with the method of induction of SAH, in dogs with use of tissue plasminogen activator, and in monkeys with tirilazad and severity of vasospasm (Table 2). The heterogeneity means that in rats, treatment effect was higher when cisterna magna models were used, that in dogs, tissue plasminogen activator was more effective than other drugs, and that in monkeys, tirilizad was more effective compared with other drugs and that treatment effect was higher when drugs where tested against severe vasospasm.
Publication Bias
Both the Begg's (z-score continuity corrected 6.68, P=0.000) and Egger's test (bias coefficient −4.83; 95% CI −5.96 to −3.69; P=0.000) of heterogeneity in the 74 most effective doses in the pooled studies returned strong evidence of heterogeneity. The results were similar when including all 114 dose regimens tested (Begg's z-score continuity corrected 8.18, P=0.000 and Egger's test bias coefficient −4.79; 95% CI −5.75 to −3.84; P=0.000). Furthermore, Begg's funnel plot showing asymmetric orientation of the data points for both analyses (Figure 3).

Begg's funnel plots with pseudo 95% confidence limits for 74 (upper) or 114 (lower) dose regimen comparisons.
Treatment Effect of Different Drugs in Different Species and Translation
Considering separately the effect of the different drugs in the different species, it is interesting to note that calcium antagonists and endothelin antagonists showed no significant effect in monkeys, statins had no significant effect in dogs, and endothelin antagonists had no significant effects in rats (Table 3). Therefore, the argument that all drugs are effective in animal models is not correct. We next considered data in a systematic review of the effect of pharmaceutical reduction of radiographic vasospasm with different drugs in human clinical trials (Etminan et al, 2011). Drug effects were classified as concordant or discordant based simply on whether or not the study or group of studies demonstrated a statistically significant effect on vasospasm. If, for example, the drug reduced vasospasm in dogs and in humans, or did not do so in either, then it was concordant. Any other result was discordant. Fasudil, endothelin antagonists (clazosentan), and tissue plasminogen activator showed a beneficial effect on angiographic vasospasm in humans while calcium antagonists (nimodipine and nicardipine), erythropoietin, statins, and tirilizad showed no significant effect (Table 3). Since no data were available on the effects of magnesium and tissue plasminogen activator plus endothelin antagonists, experiments using these treatments were excluded from further analysis.
Treatment effect of different drugs in different species
CI, confidence interval.
Two of the four drugs (50%) tested in monkeys presented results concordant with clinical trials (calcium channel antagonists had no significant effects in either, tissue plasminogen activator was significantly effective in both). In dogs, four (statins had no significant effects whereas fasudil, tissue plasminogen activator, and endothelin antagonists/endothelin converting enzyme inhibitors had significant effects) of six drugs (67%) were concordant and in rabbits, two (tissue plasminogen activator and endothelin antagonists had significant effects) of six (33%). One drug (fasudil had significant effect) of four (25%) was concordant in rats while in mice, none of two was concordant (Table 3). Considering animal studies individually, 55% were concordant. In multivariate logistic regression after using a Bonferroni adjustment for multiple testing, the only significant predictor of translation was the day of assessment of vasospasm (continuous or dichotomized ≤ versus > 3 days). Studies where vasospasm was evaluated after day 3 from SAH have results more concordant with human trials than others. Univariate analysis showed angiographic evaluation of vasospasm (versus histology), intracranial delivery of drug (versus other routes), use of randomization, statement of inclusion and exclusion criteria, cisternal injection procedure or fresh blood injection (versus clot placement), and blinding of assessment of outcome were associated with higher probability of translation (P<0.15; Table 4; Supplementary Figure 1).
Results of univariate and multivariate logistic regression for predictors of translation
Discussion
This systematic review found that pharmacologic treatments that are effective at reducing vasospasm in animal models of SAH are also usually effective in humans. However, publication bias was detected, suggesting that some studies remain unpublished and that the data may overestimate the true effect of the drugs. Since one of the objectives of animal experimentation is to develop new treatment for human disease, lack of publishing of negative experimental studies may contribute substantially to unsuccessful translation of results to humans (Sena et al, 2010; van der Worp et al, 2010). On the other hand, methodologic quality of many of the studies was poor, and would limit the interpretation of experimental studies showing treatment benefits, as well as of those showing no benefit. We also found significant heterogeneity and that the effect size was related to some of the study characteristics. The type of SAH model for erythropoietin and statins, animal species for calcium channel antagonists, tissue plasminogen activator for dogs, tirilizad and severity of vasospasm for monkeys were significantly different. This observation suggests that results of experimental studies are related not only to the drug tested but also to the study design and quality and that these factors should be considered when evaluating whether to test a drug in humans.
The other main objective of this work was to assess if any animal used in models of SAH more accurately reflects results of human studies. We observed that not every drug is effective in every species and that in monkey and dog studies the concordance with human studies was higher. However, in multivariate logistic regression after adjustment for multiple testing, only the day of evaluation of vasospasm (>3 days after SAH) correlated with successful translation of results to humans. These observations have practical implications for study design and model selection and it may be that in further preclinical experiments long-term drug effect evaluation should be preferred.
There are a number of limitations to this analysis. Most instances of concordance were positive, in that drugs worked in animals and humans. However, additional important confirmatory data would be studies showing that drugs that were not effective in humans also were not in animals. In studies that show no effect, drug dose, timing and route of administration would also need to be fully explored or at least shown to be equivalent between animal and human studies. There are limited data on some models, such as rats and mice. Only 11 of the 72 (15%) studies used rats or mice, and almost all recent studies use them. Given the paucity of data, we cannot exclude the possibility that these models are equal to dogs and monkeys in terms of translation of results to human. In human trials, angiographic vasospasm frequently was considered as a dichotomous variable (none/mild or moderate/severe), while in animal studies, it is expressed as a continuous variable. For this reason, it was not possible to compare exactly the effects in animals and humans. We focused on prophylactic treatments and in the human studies, drugs were administered before the onset of angiographic vasospasm. In the animal studies, however, since we included some studies where drug administration began after the second blood injection and angiography was not done, it is possible that some of these studies were also treating established vasospasm. The classification of drugs also could be questioned. We categorized magnesium separately since it has calcium antagonist and putative neuroprotective effects. Another option would be to include it in the calcium antagonist category. For calcium antagonists, we searched only for nimodipine and nicardipine since other calcium antagonists have not been studied in randomized clinical trials in humans.
Another issue is the analysis was restricted to evaluation of angiographic or large-artery vasospasm. Most models of SAH differ from human SAH in that animals do not develop cerebral infarctions or other delayed processes that are proposed to be important in humans, such as cortical spreading ischemia and microthromboembolism (Dreier et al, 2009; Vergouwen et al, 2008). The end points in human clinical trials are usually clinical outcome and delayed cerebral ischemia but neurological and functional examination after experimental SAH is still uncommonly used (Jeon et al, 2009; Takata et al, 2008). There is some evidence that reduction of angiographic vasospasm does not correlate with improved outcome in humans (Etminan et al, 2011; Macdonald, 2011; Macdonald et al, 2008). Unfortunately, if this is true and not due to other reasons, such as efficacy of rescue therapy in placebo groups, insensitive outcome measures, drug side-effects and such, animal models will have to be reassessed to ensure they reflect relevant human features if they are to be used to decide whether to advance drugs into clinical trials. The present analysis focused on angiographic vasospasm but subsequent analysis might examine the relation between delayed cerebral ischemia and neurological outcome in animals and humans.
In conclusion, we found evidence that animal models can inform effects of pharmacologic treatments for angiographic vasospasm in human clinical trials. Higher species, longer durations of assessment of vasospasm and injections of fresh blood may be more applicable.
Footnotes
Disclosure/conflict of interest
RL Macdonald is a consultant for Actelion Pharmaceuticals and Chief Scientific Officer of Edge Therapeutics, Inc. Edge Therapeutics is dedicated to developing drugs to improve outcome of patients with subarachnoid hemorrhage and brain injuries.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
