Cannabinoids for the Neuropsychiatric Symptoms of Dementia: A Systematic Review and Meta-Analysis

Abstract

French

Background:

In 2016, the global number of individuals living with dementia was 43.8 million, representing a 117% increase from 1990—mainly due to increases in aging and population growth. Up to 90% of individuals with dementia experience neuropsychiatric symptoms (NPS). However, the limitations of current treatments for NPS have drivent he search for safer pharmacotherapies—including cannabinoids.

Aim:

To assess the efficacy and acceptability of cannabinoids for the treatment of NPS in individuals with dementia.

Design:

Systematic review and meta-analysis of clinical trials.

Setting and participants:

Of 6,902 papers, 9 were eligible (n = 205, 44% female, 78 ± 7 years, 85% Alzheimer disease). Trials were in North America and Europe and explored tetrahydrocannabinol (n = 3), dronabinol (n = 5), or nabilone (n = 1).

Measurement:

Titles/abstracts were independently screened by one reviewer and reviewed by a second. Full-text screening was by two reviewers with discrepancies resolved via a third reviewer. We extracted data on the standardized mean difference (SMD) for several NPS instruments, trial completion, and adverse events. Data were pooled using random-effects models.

Findings:

Cannabinoids led to significant improvements across NPS instruments, including the Cohen Mansfield Agitation Inventory (SMD = −0.80; 95% confidence interval [CI], −1.45 to −0.16), the Neuropsychiatric Inventory (SMD = −0.61; CI, −1.07 to −0.15), and nocturnal actigraphy (SMD = −1.05; CI, −1.56 to −0.54h). Cannabinoids were well-tolerated, with an overall trial completion rate of 93% (193/205) and no serious treatment-related adverse events. Treatment efficacy was associated with baseline dementia severity and dose, but not dementia subtype, age, or sex. The overall study quality was rated as low.

Conclusions:

There is preliminary evidence for the efficacy and tolerability of cannabinoids as treatments for NPS. Population-based studies are needed to characterize their real-world effectiveness and acceptability.

Keywords

Alzheimer disease cannabinoids meta-analysis systematic reviews geriatric psychiatry pharmacotherapy

Background

Globally, the number of individuals living with dementia is increasing, which negatively affects families, communities, and health-care systems ubiquitously.¹ In 2016, the global number of individuals who lived with dementia was 43.8 million (95% uncertainty interval, 37.8 to 51.0), representing a 117% increase from 1990—mainly due to increases in population ageing and growth.¹

Until there are breakthroughs in preventive or curative treatment, dementia will constitute an increasing challenge to health-care systems worldwide.¹ As there is presently no known cure for dementia, there is an ongoing search for effective treatments for the neuropsychiatric symptoms (NPS) of dementia, which can include agitation and aggression.^2
–4 These behavioral symptoms affect up to 90% of all persons with dementia and are highly predictive of caregiver stress and nursing home placement, along with acceleration of cognitive and functional decline.⁵

While atypical antipsychotics are the most commonly used medications for NPS, their efficacy is modest, and their use is associated with an increased risk of multiple serious adverse events, extrapyramidal symptoms, and all-cause mortality.^6

–9 Therefore, the limited efficacy and high-risk profiles of current pharmacotherapies for the treatment of dementia symptoms have driven the search for safer pharmacologic alternatives.¹⁰

Recently, cannabinoids have become a popular treatment for a variety of medical conditions, including chronic pain, and psychiatric disorders,^11
–13 including dementia.^14,15 Available evidence suggests there are multiple neuroprotective properties of cannabinoids.^16

–20 Ligands at the CB₁ cannabis receptors reduce presynaptic neurotransmitter release, including glutamate.^21
–23 As excessive glutamate in the synapse can lead to oxidative stress and damage to neurons promoting neurodegeneration, this is considered a therapeutic mechanism for cannabinoids in dementia.²⁴ As well, in vitro experiments show that cannabinoids at the CB₁ receptor may protect neurons from both excitotoxic^25,26 and hypoxic damage.²⁷ A role for CB₂ receptors, while less apparent in neuroprotection, may also exist via reductions in neuroinflammation.²⁸ As neurodegeneration is a feature common to the various types of dementia, the neuroprotective effects of cannabinoids may therefore be beneficial in slowing the progression of these diseases.

Evidence continues to suggest that cannabinoids, like cannabis, may be therapeutically useful in dementia as they target several underlying pathophysiological processes linked to dementia. Cannabis—notably the psychoactive substrate tetrahydrocannabinol (THC)—reduces neuroinflammation and oxidative stress, confers neuroprotective effects, induces neurogenesis, removes amyloid β, and clears neurofibrillary tangles.^29
–31 The hippocampus and microglia—key players in dementia pathophysiology—are concentrated in CB₁ and CB₂ receptors, respectively, and are modulated by exposure to cannabis.^32,33 Therefore, cannabinoids have the potential to interrupt the disease process as well as treat symptoms in Alzheimer dementia. Preclinical studies in rats have demonstrated THC’s competitive inhibition of the enzyme acetylcholinesterase, paralleling anti-dementia drugs like Donepezil, diminishing acetylcholinesterase-induced amyloid β–peptide aggregation.³⁴ Additionally, intracerebroventricular administration of a synthetic cannabinoid (WIN 55,212-2) prevented cognitive deficits and decreased neurotoxicity among rats.³⁵ This evidence indicates therapeutic promise for cannabinoids in treating disease processes that underlie dementia.

In addition to preclinical studies, several small clinical trials have quantified the efficacy of cannabinoids for NPS among patients with dementia.^36

–42 While some subjective improvements were noted by patients and their caregivers, extant trials have been underpowered to detect clinically significant differences from placebo, limiting informed, evidence-based conclusions.^15,43

–46 Further, any therapeutic benefit of using cannabinoids must be tempered against the body of literature showing clear associations between cannabinoid use and the risk of psychosis, anxiety, and other psychiatric conditions.^47,48 In addition, cannabinoid use by the elderly can lead to impairment that may lead to unintended injuries like falls.⁴⁹

While several reviews^{4,13,15,29,30,45,50

–56} have been published on the topic of cannabinoids for dementia, most have provided only systematic reviews, and not quantitative synthesis. A prior Cochrane systematic review from 2009⁵⁷ identified only a single trial³⁷—precluding any formal analysis. A more recently published meta-analysis of six clinical trials declared that there was no effect of cannabinoids on agitation symptoms or NPS overall.⁵⁸ However, several potential limitations were noted by the authors—such as the small number of studies, the small sample size, the lack of adequate controls, the short trial duration, and the low quality of extant trials—may have diminished their study’s statistical power to detect a significant signal to noise ratio. The authors concluded that their findings were inconclusive due to substantial heterogeneity—which justified the need for additional, rigorous studies.

To address these limitations and provide a unique contribution to the field, we conducted an updated systematic review and meta-analysis. We identified additional studies, which boosted our meta-analysis’ statistical power, which enabled the detection of significant differences in the efficacy of cannabinoids for the treatment of NPS of dementia. We also quantified the acceptability of cannabinoids for the NPS by measuring the occurrence of adverse events and trial completion rates. We identified several potential causes of heterogeneity through extensive meta-regression, subgroup, sensitivity, and publication bias analyses to help put individual trial results into perspective. We also considered a broader range of outcomes—including actigraphy, global assessments of function, and cognitive measures—to complement existing estimates obtained for agitation and aggression that have been reported by previous studies. We also offer suggestions for future studies to address limitations in the available research. For these reasons, this updated meta-analysis aims to offer unique contribution to the field of dementia research.

Methods

We undertook this review using methods consistent with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)⁵⁹ guidelines (checklist provided in Online Appendix 1). We registered our protocol with the Open Science Framework (https://osf.io/et7cm/; https://osf.io/dnsqm).

Search Strategy

We search seven electronic databases (EMBASE, MEDLINE, PsycINFO, CINAHL, PubMed, Cochrane CENTRAL, and ClinicalTrials.gov) for articles published from database inception to January 2019; the search was updated in June 2019. Sets of search strings incorporating both keywords and Medical Subject Headings (MeSH terms) reflecting cannabinoids and dementia were used and are provided in full in Online Appendix 2. Searches were limited to human literature, with no restrictions placed on language or publication type. Citations for papers in languages other than English were read via Google Translate.

Citations were imported into the web-based screening tool, Covidence,⁶⁰ where duplicate citations were removed. Titles and abstracts were screened by one reviewer (AB), and all papers marked as excluded were reviewed by a second person (ACM) to ensure accuracy in first-pass screening. Full-text articles were screened by two independent reviewers (AB and ACM) with discrepancies resolved via consultation with a third reviewer (EH) when needed. Reference lists for relevant systematic reviews identified in the peer-review literature search were hand searched for additional papers not already identified, and we additionally searched table of contents of relevant journals.

Inclusion and Exclusion Criteria

The population of interest was people experiencing NPS of dementia who received cannabinoids—including whole cannabis (from plants), THC, or prescribed synthetic cannabinoids (like nabilone, cannabidiol, or dronabinol). The exact definition of NPS varied across studies, and we summarize the way in which the studies were defined and NPS operationalized in Online Appendix 3. Inclusion criteria for the study population comprised clinical trials of cannabinoid-based interventions to treat NPS (randomized or quasi-randomized controlled trials [RCTs]) where treatment efficacy and acceptability may be reported at trial completion. The primary justification for including only RCTs was to minimize the risk of bias.⁶¹ Inclusion criteria for the study outcomes comprised reporting treatment efficacy and acceptability, or studies where such data could be obtained via contacting study authors. Exclusion criteria comprised observational studies (cohort, case-control, cross-sectional, case reports, case series); works that did not present original data (e.g., letters to the editor, editorials, commentaries); works that did not present treatment efficacy or acceptability for NPS and where such data could not be obtained from study authors; systematic reviews (we hand searched reviews for relevant studies that were assessed independently against the inclusion/exclusion criteria); studies where use of cannabinoids and outcomes were not reported for the same sample of people; clinical trials of interventions with people using cannabinoids for other indications (e.g., clinical trial for epilepsy, multiple sclerosis, or palliative treatments); and studies not using validated instruments to measure NPS (e.g., Neuropsychiatric Inventory [NPI]⁶²) or to diagnose dementia (e.g., Diagnostic and Statistical Manual of Mental Disorders [DSM]).

Data Extraction

The following data were abstracted: study information (i.e., author, journal, and year), study characteristics (i.e., mean age of participants, recruitment setting, country of study, duration of follow-up), participant characteristics (i.e., comorbidities, dementia subtype, baseline severity of NPS, ethnicity), and intervention characteristics (i.e., dementia severity, dementia subtype).

The data extraction form was developed in Microsoft Excel 2016 based on previously conducted reviews,^63
–65 and recommendations were outlined in the PRISMA statement.⁵⁹ Data were independently extracted by one member of the research team (AB) and checked by a second (ACM). Bibliographic information was extracted in addition to study specific information. Data entry was standardized by use of a manual, which contained data entry rules. In instances where data reporting in the publications was incomplete, supplementary information and documents were sourced to locate missing data. If supplementary information could not be located or did not provide the necessary data needed, study authors were contacted by email for additional information.

A quality index based on the Cochrane Risk of Bias Tool for Randomized Controlled Trials was adopted for consistency.⁶⁶ This scale uses a 7-item system to evaluate randomized studies regarding seven domains of quality: randomization, allocation, blinding of participants, blinding of outcome assessors, selection bias, selective reporting, and additional sources of bias. Each item was rated as having a high, low, or unclear risk of bias. Individual assessments for each criterion were pooled to provide an overall risk of bias assessment, where the greater the number of items with a low risk of bias, the higher the methodological quality of the study. Study information necessary for quality assessment was extracted to the Excel template by one reviewer (AB) and double-checked by a second (ACM). Discrepancies were resolved via consultation with a third reviewer (EH) where needed.

Calculation of Treatment Efficacy and Acceptability

Data from measures of NPS, including agitation (Cohen Mansfield Agitation Inventory [CMAI] and the NPI Agitation subscale), overall NPS (NPI total), and cognition (Mini-Mental Status Examination [MMSE]) were extracted from the included studies. Data on changes in weight, systolic blood pressure, and occurrence of adverse events were also extracted. For continuous data, standardized mean differences (SMDs) and 95% confidence intervals (CIs) were calculated using a random-effects model.

Data Analysis

R studio (Version 1.1.463) metafor package was used to conduct DerSimonian and Laird Mantel–Haenszel random-effects meta-analysis to pool efficacy and acceptability estimates.⁶⁷ All analyses were subsequently transferred to Cochrane’s Review Manager 5.3 to generate high-quality plots and graphs.⁶⁸ The random-effects model was used in all meta-analyses as we anticipated high levels of heterogeneity between cohorts. Heterogeneity was quantified using the I ² statistic and described as low (≤25%), moderate (>25% and ≤50%), high (>50% and ≤75%), or substantial (≥75%).^65,69,70 We explored the impact of study characteristics on treatment efficacy and acceptability via subgroup, sensitivity, and meta-regression analyses. Variables included percentage male, baseline age, dementia subtype, severity of dementia (MMSE), study year, and geographic region. Formal comparisons between subgroups were performed using meta-regression, via the metafor package.⁶⁷ Sensitivity analyses included comparisons with leave-one-out, cumulative, subgroup, and fixed-effects meta-analyses. Bubble plots were generated where significant differences were identified between subgroups. The adjusted R ² index was employed to quantify goodness of fit for each model. Statistical significance for all analyses was set at P < 0.05. Publication bias was assessed qualitatively, using funnel plot symmetry, as well as quantitatively, using the Egger and trim-and-fill methods.^71
–73

Results

The search produced 6,903 unique papers, of which 43 were screened in full, and 9^{36

–42,74,75} were included in the review (Figure 1). Online Appendix 4 provides the list of excluded studies.

Figure 1.

PRISMA study flow diagram.

Trials were primarily undertaken in Europe^{36,39

–42,75} or North America^37,38,74 (Table 1). Six were RCTs; the remaining three were quasi-randomized studies. Trials varied widely in size (n = 2 to 50) and reporting of cohort demographics. For example, fewer than half reported the extent of previous treatments for NPS. Medical and psychiatric comorbidity among study participants was inconsistently reported, as were details regarding the duration of dementia, previous treatments, and current medications.

Table 1.

Characteristics of Included Studies.

Study	Study Design	Study Setting	Intervention	Participants	Male (%); age (SD)	Baseline MMSE (SD)	NPS History of Participants
Volicer et al.³⁷	Randomized, DB, PC, crossover trial	Hospital	Dronabinol 2.5 mg po BID for 6 weeks then crossover	15 (3 dropouts)	91.7; 72.7 (4.9)	4.0 (7.4)	Food refusal
Walther et al.³⁶	Retrospective systematic chart review (pre–post intervention)	Hospital	Dronabinol 2.5 mg po qHs for 14 days	6	66.6; 81.5 (6.1)	10.33 (6.28)	Nighttime agitation, daytime rhythm disturbances/sundowning
Mahlberg et al.⁷⁵	Randomized, SB, PC, parallel trial	Hospital	Dronabinol 2.5 mg po qHs for 14 days	24	42; 78.6 (9.7)	Not reported	Agitation, aggression or resistance to care
Walther et al.⁴²	Randomized, DB, PC, crossover trial	Hospital	Dronabinol 2.5 mg po qHs for 14 days	2	100; 78.0 (4.2)	19.5 (3.5)	Circadian rhythm disturbances, verbal aggression, delusions, and apathy
Woodward et al.³⁸	Retrospective systematic chart review (pre–post intervention)	Hospital	Dronabinol 7.0 mg po daily for 4 to 50 days (average = 16.9 days)	40	30; not reported	7.0 (not reported)	Agitation, aggression or resistance to care
Van den Elsen et al.³⁹	Randomized DB PC multicentre Phase II trial	Community and nursing home	THC 1.5 mg po TID for 3 weeks	50	50; 78.4 (7.4)	15.0 (6.8)	Agitation, aggression, and aberrant motor behavior; NPI ≥ 10
Van den Elsen et al.⁴⁰	Randomized DB PC repeated crossover trial	Hospital and community	THC 0.75 mg po BID for 3 days (Weeks 1 to 3) and 1.5 mg po BID for 3 days (Weeks 4 to 6)	22 (2 dropouts)	68; 76.4 (5.3)	16.9 (7.8)	Agitation and aggression; NPI ≥ 10
Shelef et al.⁴¹	Open-label prospective cohort study	Hospital	THC 2.5 to 5 mg po BID for 28 days	11 (1 dropout)	46; 73.2 (8.6)	10.3 (9.4)	Severe agitation and aggressive behavior resulting in hospitalization
Herrmann et al.⁷⁴	Randomized DB PC crossover trial	Community and nursing home	Nabilone 1 to 2 mg po qHs for 6 weeks then crossover	38 (9 dropouts)	77; 87 (10)	6.5 (6.8)	NPI-A ≥3 and clinically stable

Note. SD = standard deviation. MMSE = Mini-Mental Status Examination. NPS = Neuropsychiatric Symptom (of Dementia). DB = double-blind. PC = placebo-controlled. BID = twice daily dosing. po = by mouth. qHs = nightly dosing. SB = single blind. TID = thrice daily dosing. THC = tetrahydrocannabinol. SB = single-blind. NPI = Neuropsychiatric Inventory.

The total sample size was 208 (44% female, mean age 78 [SD = 7] years). Alzheimer disease was the most frequent dementia subtype (85%), followed by mixed (8%), vascular (6%), and frontotemporal dementia (1%). The mean baseline MMSE score of participants was 11 (SD = 7). The main indication for treatment was severe agitation or aggressive behavior (n = 8) or food refusal³⁷ (n = 1). The mean proportion taking an antipsychotic prior to study initiation was 49% (SD = 32%).

Three trials used THC preparations,^39
–41 five used dronabinol,^{36
–38,42,75} and one⁷⁴ used nabilone. Outcome measures reported by studies included the NPI,^{36,38

–42,58,75} the CMAI,^{36

–42,74} nocturnal motor activity (as measured by actigraphy),^{36,38,41,42,75} the MMSE,^39,41,74 and the Clinical Global Impression (CGI).^38,40,41

Efficacy of Cannabinoids for NPS

All trials (n = 208 participants) provided data to inform the pooled efficacy of cannabinoids for NPS using one or more outcome measures. For the CMAI, the SMD was −0.80 (95% CI, −1.45 to −0.16; nine studies; Figure 2), with substantial heterogeneity (I ² = 85%). For the NPI total, the SMD was −0.61 (95% CI, −1.07 to −0.15; seven studies; Online Appendix 5), with high heterogeneity (I ² = 67%). For the NPI-A subscore, the SMD was also −0.61 (95% CI, −0.97 to −0.25; six studies; Online Appendix 6), with moderate heterogeneity (I ² = 50%). For nocturnal motor activity, the SMD was −1.05 (95% CI, −1.56 to −0.54; five studies; Online Appendix 7), with moderate heterogeneity (I ² = 40%). For the MMSE, the SMD was 0.42 (95% CI, 0.07 to 0.78; three studies; Online Appendix 8), with low heterogeneity (I ² = 3%). For the CGI, the SMD was −0.94 (95% CI, −1.24 to −0.64; three studies; Online Appendix 9), with high heterogeneity (I ² = 70%).

Figure 2.

Forest plot from random-effects meta-analysis of the standardized mean difference in the Cohen Mansfield Agitation Inventory.

Factors Associated with Efficacy of Cannabinoids for NPS

We ran a series of meta-regressions and subgroup analyses to explore potential variables that may have accounted for the heterogeneity observed for the effect size of treatment with the CMAI, NPI, NPI-A, and nocturnal motor activity (actigraphy; Table 2). Type of cannabinoid (synthetic vs. THC), percentage male, baseline age, percentage with Alzheimer disease, and geographic region were not associated with any of the outcome measures.

Table 2.

Statistical Significance of Univariate Meta-Regression Analysis for Factors Associated with Efficacy of Cannabinoids for Neuropsychiatric Symptoms of Dementia.

Factor	Cohen Mansfield Agitation Inventory	Neuropsychiatric Inventory	Neuropsychiatric Inventory, Agitation	Nocturnal Motor Activity
Agent (synthetic vs. THC)	0.371	0.996	0.939	0.585
Dose (mg)	0.203	<0.001	0.061	0.209
Baseline MMSE	0.001	0.636	0.210	0.830
Male (%)	0.636	0.261	0.146	0.675
Baseline age	0.176	0.208	0.481	0.061
Alzheimer disease (%)	0.393	0.635	0.833	0.412
Region (North America vs. Europe)	0.065	0.930	0.939	0.285
Design (randomized vs. quasi-randomized)	0.729	0.001	0.047	0.607
Year of study	0.003	0.966	0.907	0.407

Note. THC = tetrahydrocannabinol; MMSE = Mini-Mental Status Examination; bold values indicate meta-regression analyses with P values below 0.05.

For the CMAI, there was an association between effect size and baseline MMSE (with greater effect size noted in those with greater baseline MMSE) and between effect size and year of study (with greater effect sizes noted in earlier studies).

For the NPI—Total and Agitation subscale—there was a positive dose–response relationship (with higher effect sizes noted with higher total daily doses of cannabinoids) and an association with study design (with higher effect sizes noted in quasi-randomized studies relative to randomized trials). Like the CMAI, for nocturnal motor activity, there was an association between effect size and year of study (with greater effect sizes noted in earlier studies).

Sensitivity and Quality Analyses

Overall study quality was rated as low due to inconsistent methods of allocation concealment, incomplete or nonrandomization, a lack of double-blinding, and a high risk of carryover due to concurrent medication use. For summary results of the judged risk of bias across the included studies for each domain, see Online Appendices 10 and 11.

We conducted four sensitivity analyses to assess the robustness of our estimates for the CMAI, NPI, NPI-A, and nocturnal motor activity; these are described in Online Appendix 10.

First, we explored the influence of each study on pooled estimates for efficacy effect size via sensitivity analysis with leave-out-one meta-analysis, allowing the removal of each individual study from the analysis (Online Appendix 12a). This demonstrated that the effect size estimate for the CMAI was influenced by the extreme result of the Volicer 1997³⁷ study, which was an outlier from the remaining studies; when this study was excluded, the CMAI effect size was no longer statistically significant (SMD = −0.359, 95% CI [−0.782 to 0.065]). The estimates for the remaining outcome measures (NPI, NPI-A, and nocturnal motor activity) were robust to leave-out-one sensitivity analysis and remained significant with the removal of any individual study.

Second, we compared the random-effects meta-analysis estimates to a fixed-effects model, which ignores heterogeneity (Online Appendix 12b). For all outcomes, the fixed-effects SMD estimate remained statistically significant but were smaller than the random-effects estimates.

Third, we conducted cumulative meta-analyses to combine studies chronologically to identify when a characteristic or statistically significant change first occurred (Online Appendix 12c). We observed significant time trends in the CMAI, NPI-A, and nocturnal motor activity, but not the NPI total score. This was consistent with the meta-regression measuring the impact of study year on effect size.

Fourth, we conducted subgroup analyses to determine whether the type of cannabinoid preparation influenced the estimates of efficacy (Online Appendix 12d). There was no significant difference in efficacy by synthetic or THC cannabinoid preparations.

Finally, we examined evidence for potential publication bias through funnel plots (Online Appendix 13) and by applying rank correlation tests, Eggers, and the trim-and-fill method. The results did not suggest any evidence to support that a significant bias existed within this review.

Acceptability of Cannabinoids for NPS

The overall rate of completion (defined as participants reaching the primary study endpoint) was 93% (193/208). A total of 15 dropouts occurred across five studies,^{37,39
–41,74} occurring from death (n = 2), dysphagia (n = 2), pneumonia (n = 1), malignancy (n = 2), urinary tract infection (n = 3), seizure (n = 1), elevated INR (n = 1), pneumothorax (n = 1), lethargy (n = 2), and excess as needed (PRN) usage (n = 1). Only lethargy was described by study authors as being potentially related to administration of cannabinoids.

There was no significant association between cannabinoids and weight (P = 0.76), systolic blood pressure (P = 0.20), diastolic blood pressure (P = 0.21), or the occurrence of serious adverse drug events (P = 0.51)—as demonstrated in Online Appendix 14.

Discussion

This systematic review and meta-analysis synthesized the findings of nine trials to appraise available evidence for the use of cannabinoids in the treatment of the NPS associated with dementia. Most participants completed the entirety of treatment, and few serious adverse events were reported. We found consistent evidence for the efficacy of cannabinoids in reducing NPS, which outperformed placebo on all efficacy outcome measures considered (CMAI, NPI, NPI-A, and nocturnal motor activity). Our findings are consistent with the conclusions of several previously published reviews,^{15,29,30,43

–46,50,51,57,76

–80} but these contrast the conclusions of a recent meta-analysis that declared a lack of efficacy for agitation and aggression.⁵⁸

Comparison with Previous Reviews

Although the present review draws on a relatively small amount of literature that has already been subjected to review earlier this year,⁵⁸ the present review aimed to yield novel information and present it in an appropriate and straightforward manner. Both reviews found significant heterogeneity across studies and neither review found differences in the occurrence of adverse events or dropouts due to an adverse event between treatment groups. However, the key differences were that the present review—based on nine studies^{36

–42,74,75}—found a significant effect of cannabinoids on NPS, while the review of Ruthirakuhan and colleagues⁵⁸—which was based on six^{37,39,40,42,74,75} articles—did not. Of note, all six studies identified by Ruthirakuhan and colleagues were included in the present review; hence, there were differences in our conclusions despite overlaps in the studies considered. This reflects potential differences in eligibility criteria across the two reviews—with the present including quasi-randomized trials. Consequently, the three additional studies^36,38,41 considered by this review that were not included by Ruthirakuhan and colleagues were quasi-randomized in design, which may explain the differences in our conclusions. Since the addition or removal of a few studies led to substantially different conclusions across reviews, the conclusions we can draw ought to be done so cautiously. In light of the growing interest in cannabinoids and the unmet need in NPS treatment, we hope this review makes a meaningful contribution to the literature, which may stimulate further research to clarify why there were discrepancies across these two reviews.

For several reasons, the finding of therapeutic efficacy here should not indiscriminately rule in the use of cannabinoids in treating NPS in dementia. Firstly, our meta-analysis was largely restricted to studies using dronabinol and THC—with only one study using nabilone. Dronabinol is a partial agonist at both CB₁ and CB₂ and, via second messenger pathways, inhibits adenylate cyclase and reduces concentrations of Cyclic adenosine monophosphate (cAMP).³⁸ Dronabinol is considered the “purified” version of THC. THC, and THC-like substances alone, may be less efficacious than when paired with other cannabinoids like CBD. The conclusions we can draw are that some cannabinoids appear clinically efficacious to treat dementia-related symptoms relative to THC. Taken together, this reviews’ findings suggest that not all cannabinoids are created equally—particularly when used in the treatment of NPS of dementia. Thus, there may be potential differences between whole cannabis, THC, and synthetic cannabinoids, which may indicate that there may be broader considerations when using cannabinoids therapeutically.

Secondly, there were substantial variations in the dosing of cannabinoids used across studies. For example, dronabinol doses used across studies were moderate but higher than THC doses; however, dronabinol itself has a low bioavailability (roughly 4% to 20%) and, if combined with low dosing, might have indicated inefficacious dosing regimens.⁸¹ Nabilone, another synthetic cannabinoid analogue of THC, is both more potent than dronabinol and has greater bioavailability (approximately 60%) according to the literature—but this did not achieve statistical significance in the present review.^81
–83 As proposed by Russo,⁸⁴ a more complete spectrum of cannabinoids—consistent with the composition of herbal cannabis—might accentuate clinical efficacy in many conditions, creating the so-called entourage effects.^84
–86

Finally, not all dementias have the same neurobiology. Approximately 85% of the patients across studies were diagnosed with Alzheimer; however, our meta-regression did not indicate any association between dementia subtype and treatment efficacy. However, given the small total sample size of our study (n = 208), our meta-analysis was potentially underpowered to detect significant differences across different types of dementia. Therefore, larger studies using appropriate doses of varied cannabinoids would help address the study limitations identified here and formally substantiate cannabinoid’s utility in treating NPS common to dementia.

Limitations

The findings of this meta-analysis cannot be considered without mentioning a few of its limitations. Overall, only a few small studies were identified, and fewer still could be fully meta-analyzed due to inconsistent reporting styles across studies. Most trials did not require a washout period; thus, if participants were already taking other psychotropic medication, the cannabinoid medication they received was not actually in monotherapy, therefore contaminating the effect of the intervention. In terms of aggression and agitation, the outcomes considered across studies largely looked at agitation (the Cohen Mansfield Agitation Inventory, for example). However, across trials, these terms were often used interchangeably despite their contrasting definitions.

As NPS is a broad term, considering it as a single outcome measure is somewhat problematic; however, this was tempered by the fact that the meta-analysis only pooled together trials that were using the same NPS outcome measure (e.g., the NPI), which were all validated measures for us in individuals with dementia. Further, the individuals represented across these trials may not be representative of a large, population-based sample of individuals with dementia, primarily due to a higher proportion of individuals in trials with moderate-to-severe dementia and who were institutionalized. An additional limitation was that one of the component articles³⁷ was about the NPS of food refusal, while the others all included agitation. This is especially important because the CMAI finding of statistical significance disappeared when the paper was excluded, indicating the findings were not robust.

Further, the quality of evidence from the individual studies included in this review was low (using the Cochrane Risk of Bias Tool). In the majority of studies, both the treating clinician and participant were aware of treatment assignment because of the subjective effects of cannabinoids—even if the study was double-blinded. As many participants in the RCTs were crossed over to an open trial of cannabinoids, this precluded follow-up assessments for the control condition. Still, the lack of high-quality data and the unclear risk of bias in many key domains of the included studies prevents definitive conclusions from being made about the usefulness of dronabinol for the treatment of dementia. Further, most of the discussion section is speculative in nature as the precise mechanisms behind the observed findings cannot be confirmed with meta-analysis. As a result, the conclusions of this review should be considered tenuous.

Future Directions

This meta-analysis highlights the need for larger, high-quality, randomized, double-blind, placebo-controlled trials to determine whether additional cannabinoid preparations are clinically efficacious in the treatment of dementia. However, there is also an opportunity for future studies to address potential alternatives to RCTs for evaluating cannabinoid medicines—such as larger, Phase IV effectiveness trials using health administrative data. These studies may also provide a measure of real-world effectiveness compared to traditional efficacy measures from RCTs.

Conclusions

This systematic review and meta-analysis found consistent evidence that cannabinoids are efficacious for the treatment of NPS associated with dementia and are well-tolerated for use in individuals with dementia. However, our findings were not robust and were particularly vulnerable to the small sample sizes as demonstrated in our sensitivity analyses. Thus, while there is growing neurobiological evidence that cannabinoids may be useful in modulating disease processes in dementia, more evidence is needed before they can be recommended for routine use in clinical practice. At present, the use of cannabinoids in individuals with dementia should still be considered an experimental treatment until more clinical data are available. Population-based studies are clearly needed to characterize the real-world effectiveness and acceptability of cannabinoids for the treatment of NPS.

Supplemental Material

Supplemental Material, Data_Collection_Sheet - Cannabinoids for the Neuropsychiatric Symptoms of Dementia: A Systematic Review and Meta-Analysis

Supplemental Material, Data_Collection_Sheet for Cannabinoids for the Neuropsychiatric Symptoms of Dementia: A Systematic Review and Meta-Analysis by Anees Bahji, Arthi Chinna Meyyappan and Emily R. Hawken in The Canadian Journal of Psychiatry

Footnotes

Authors’ Note

All authors are responsible for the integrity of the data and accuracy of data analyses. AB and EH conceived and designed the study. All authors acquired, analyzed, and interpreted all data. AB drafted the manuscript. All authors critically revised the manuscript. EH supervised the study.

Acknowledgments

The authors would like to thank Ms. Sandra Halliday, research librarian at Bracken Health Sciences Library at Queen’s University, for her support in developing the systematic review search strategy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Anees Bahji

Supplemental Material

Supplemental material for this article is available online.

References

Nichols

Szoeke

CEI

Vollset

, et al. Global, regional, and national burden of Alzheimer’s disease and other dementias, 1990–2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol. 2019;18(1):88–106.

Lyketsos

Lopez

Jones

, et al. Prevalence of neuropsychiatric symptoms in dementia and mild cognitive impairment: results from the cardiovascular health study. JAMA. 2002;288(12):1475–1483.

Seitz

Brisbin

Herrmann

, et al. Efficacy and feasibility of nonpharmacological interventions for neuropsychiatric symptoms of dementia in long term care: a systematic review. J Am Med Dir Assoc. 2012;13(6):503–506. e2.

Sherman

Ruthirakuhan

Vieira

, et al. Cannabinoids for the treatment of neuropsychiatric symptoms, pain and weight loss in dementia. Curr Opin Psychiatry. 2018;31(2):140–146.

Livingston

Sommerlad

Orgeta

, et al. Dementia prevention, intervention, and care. Lancet. 2017;390(10113):2673–2734.

Schneider

Dagerman

Insel

. Risk of death with atypical antipsychotic drug treatment for dementia: meta-analysis of randomized placebo-controlled trials. JAMA. 2005;294(15):1934–1943.

Herrmann

Lanctôt

. Do atypical antipsychotics cause stroke? CNS Drugs. 2005;19(2):91–103.

Rochon

Normand

Gomes

, et al. Antipsychotic therapy and short-term serious events in older adults with dementia. Arch Intern Med. 2008;168(10):1090–1096.

Schneider

Dagerman

Insel

. Efficacy and adverse effects of atypical antipsychotics for dementia: meta-analysis of randomized, placebo-controlled trials. Am J Geriatr Psychiatry. 2006;14(3):191–210.

10.

Garay

Citrome

Grossberg

, et al. Investigational drugs for treating agitation in persons with dementia. Expert Opin Investig Drugs. 2016;25(8):973–983.

11.

Häuser

Finn

Kalso

, et al. European Pain Federation (EFIC) position paper on appropriate use of cannabis-based medicines and medical cannabis for chronic pain management. Eur J Pain. 2018;22(9):1547–1564.

12.

Yarnell

. The use of medicinal marijuana for posttraumatic stress disorder: a review of the current literature. Prim Care Companion CNS Disord. 2015;17(3). 2015 May 7 [Epub ahead of print]. doi:10.4088/PCC.15r01786

13.

Nielsen

Germanos

Weier

, et al. The use of cannabis and cannabinoids in treating symptoms of multiple sclerosis: a systematic review of reviews. Curr Neurol Neurosci Rep. 2018;18(2):8.

14.

Aso

Ferrer

. Cannabinoids for treatment of Alzheimer’s disease: moving toward the clinic. Front Pharmacol. 2014;5:37. 2014 Mar 5 [Epub ahead of print]. doi:10.3389/fphar.2014.00037

15.

Walther

Halpern

. Cannabinoids and dementia: a review of clinical and preclinical data. Pharmaceuticals. 2010;3(8):2689–2708.

16.

Alvarez

Lafuente

Rey-Santano

, et al. Neuroprotective effects of the nonpsychoactive cannabinoid cannabidiol in hypoxic-ischemic newborn piglets. Pediatr Res. 2008;64(6):653–658.

17.

Hampson

Grimaldi

Lolic

Wink

Rosenthal

Axelrod

. Neuroprotective antioxidants from marijuana. Ann N Y Acad Sci. 2000;899:274–282.

18.

De Lago

Fernández-Ruiz

. Cannabinoids and neuroprotection in motor-related disorders. CNS Neurol Disord Drug Targets. 2007;6(6):377–387.

19.

Garcia-Orjuela

Alarcon-Franco

Sanchez-Fernandez

, et al. Dependence to legally prescribed opioid analgesics in a university hospital in Medellin-Colombia: an observational study. BMC Pharmacol Toxicol. 2016;17(1):42.

20.

Holgado

Martín-Banderas

Álvarez-Fuentes

, et al. Neuroprotective effect of cannabinoids nanoplatforms in neurodegenerative diseases. J Drug Deliv Sci Technol. 2017;42:4284–4293.

21.

Gerdeman

Lovinger

. CB1 cannabinoid receptor inhibits synaptic release of glutamate in rat dorsolateral striatum. J Neurophysiol. 2001;85(1):468–471.

22.

Rodríguez-Muñoz

Sánchez-Blázquez

Merlos

, et al. Endocannabinoid control of glutamate NMDA receptors: the therapeutic potential and consequences of dysfunction. Oncotarget. 2016;7(34):55840–55862.

23.

Hoffman

Laaris

Kawamura

, et al. Control of cannabinoid CB1 receptor function on glutamate axon terminals by endogenous adenosine acting at A1 receptors. J Neurosci. 2010;30(2):545–555.

24.

Grundy

. The therapeutic potential of the cannabinoids in neuroprotection. Expert Opin Investig Drugs. 2002;11(10):1365–1374.

25.

Hampson

Grimaldi

Axelrod

, et al. Cannabidiol and (-)Delta9-tetrahydrocannabinol are neuroprotective antioxidants. Proc Natl Acad Sci U S A. 1998;95(14):8268–8273.

26.

Shen

Thayer

. Cannabinoid receptor agonists protect cultured rat hippocampal neurons from excitotoxicity. Mol Pharmacol. 1998;54(3):459–462.

27.

Nagayama

Sinor

Simon

, et al. Cannabinoids and neuroprotection in global and focal cerebral ischemia and in neuronal cultures. J Neurosci. 1999;19(8):2987–2995.

28.

Ehrhart

Obregon

Mori

, et al. Stimulation of cannabinoid receptor 2 (CB₂) suppresses microglial activation. J Neuroinflammation. 2005;2:29.

29.

Liu

Chau

Ruthirakuhan

, et al. Cannabinoids for the treatment of agitation and aggression in Alzheimer’s Disease. CNS Drugs. 2015;29(8):615–623.

30.

Ahmed

AIA

van der Marck

van den Elsen

, et al. Cannabinoids in late-onset Alzheimer’s disease. Clin Pharmacol Ther. 2015;97(6):597–606.

31.

Panza

Solfrizzi

Seripa

, et al. Progresses in treating agitation: a major clinical challenge in Alzheimer’s disease. Expert Opin Pharmacother. 2015;16(17):2581–2588.

32.

Weinstein

Livny

Weizman

Brain imaging studies on the cognitive, pharmacological and neurobiological effects of Cannabis in humans: evidence from studies of adult users. Curr Pharm Des. 2016;22(42):6366–6379.

33.

Nader

Sanchez

. Effects of regular cannabis use on neurocognition, brain structure, and function: a systematic review of findings in adults. Am J Drug Alcohol Abuse. 2018;44(1):4–18.

34.

Eubanks

Rogers

Beuscher

, et al. A molecular link between the active component of marijuana and Alzheimer’s disease pathology. Mol Pharm. 2006;3(6):773–777.

35.

Ramírez

Blázquez

Gómez del Pulgar

, et al. Prevention of Alzheimer’s disease pathology by cannabinoids: neuroprotection mediated by blockade of microglial activation. J Neurosci. 2005;25(8):1904–1913.

36.

Walther

Mahlberg

Eichmann

, et al. Delta-9-tetrahydrocannabinol for nighttime agitation in severe dementia. Psychopharmacology (Berl). 2006;185(4):524–528.

37.

Volicer

Stelly

Morris

, et al. Effects of dronabinol on anorexia and disturbed behavior in patients with Alzheimer’s disease. Int J Geriatr Psychiatry. 1997;12(9):913–919.

38.

Woodward

Harper

Stolyar

, et al. Dronabinol for the treatment of agitation and aggressive behavior in acutely hospitalized severely demented patients with noncognitive behavioral symptoms. Am J Geriatr Psychiatry. 2014;22(4):415–419.

39.

van den Elsen

GAH

Ahmed

AIA

Verkes

R-J

, et al. Tetrahydrocannabinol in behavioral disturbances in dementia: a crossover randomized controlled trial. Am J Geriatr Psychiatry. 2015;23(12):1214–1224.

40.

van den Elsen

GAH

Ahmed

AIA

Verkes

R-J

, et al. Tetrahydrocannabinol for neuropsychiatric symptoms in dementia: a randomized controlled trial. Neurology. 2015;84(23):2338–2346.

41.

Shelef

Barak

Berger

, et al. Safety and efficacy of medical cannabis oil for behavioral and psychological symptoms of dementia: an-open label, add-on, pilot study. J Alzheimers Dis. 2016;51(1):15–19.

42.

Walther

Schüpbach

Seifritz

, et al.

Randomized, controlled crossover trial of dronabinol, 2.5 mg, for agitation in 2 patients with dementia

. J Clin Psychopharmacol. 2011;31(2):256–258.

43.

Hill

. Medical marijuana for treatment of chronic pain and other medical and psychiatric problems: a clinical review. JAMA. 2015;313(24):2474–2483.

44.

Preuss

Wong

Koller

. Treatment of behavioral and psychological symptoms of dementia: a systematic review. Psychiatr Pol. 2016;50(4):679–715.

45.

van den Elsen

GAH

Ahmed

AIA

Lammers

, et al. Efficacy and safety of medical cannabinoids in older subjects: a systematic review. Ageing Res Rev. 2014;14:56–64.

46.

Wilkinson

Radhakrishnan

D’Souza

. A systematic review of the evidence for medical marijuana in psychiatric indications. J Clin Psychiatry. 2016;77(08):1050–1064.

47.

Gage

Hickman

Zammit

. Association between cannabis and psychosis: epidemiologic evidence. Biol Psychiatry. 2016;79(7):549–556.

48.

Crippa

Zuardi

Martín-Santos

, et al. Cannabis and anxiety: a critical review of the evidence. Hum Psychopharmacol. 2009;24(7):515–523.

49.

Volkow

Baler

Compton

, et al. Adverse health effects of marijuana use. N Engl J Med. 2014;370(23):2219–2227.

50.

Santibanez

Sepehry

Hsiung

G-YR

. Cannabis and Alzheimer’s disease: a systematic review of the evidence. Alzheimers Dement. 2017;13(7):P614.

51.

Tampi

Young

Tampi

. Cannabinoids for the treatment of behavioral and psychological symptoms of dementia. Neurodegener Dis Manag. 2018;8(4):211–213.

52.

Amanullah

. Synthetic cannabinoids in dementia with agitation: case studies and literature review. 2013; MedicalMarijuana.eu [accessed 2018 Dec 27] http://www.medicalmarijuana.eu/research/synthetic-cannabinoids-dementia-agitation-case-studies-literature-review/.

53.

Hillen

Soulsby

Alderman

, et al. Safety and effectiveness of cannabinoids for the treatment of neuropsychiatric symptoms in dementia: a systematic review. Ther Adv Drug Saf. 2019;10. doi:10.1177/2042098619846993

54.

Mücke

Carter

Cuhls

, et al.

Cannabinoide in der palliativen Versorgung: Systematische Übersicht und Metaanalyse der Wirksamkeit, Verträglichkeit und Sicherheit

Schmerz. 2016;30(1):25–36.

55.

Ijaopo

. Dementia-related agitation: a review of non-pharmacological interventions and analysis of risks and benefits of pharmacotherapy. Transl Psychiatry. 2017;7(10):e1250.

56.

Antonsdottir

Smith

Keltz

, et al. Advancements in the treatment of agitation in Alzheimer’s disease. Expert Opin Pharmacother. 2015;16(11):1649–1656.

57.

Krishnan

Cairns

Howard

. Cannabinoids for the treatment of dementia. Cochrane Database Syst Rev. 2009 ;(2):CD007204.

58.

Ruthirakuhan

Lanctôt

Vieira

, et al. Natural and synthetic cannabinoids for agitation and aggression in Alzheimer’s Disease: a meta-analysis. J Clin Psychiatry. 2019;80(2). 2019 Jan 29 [Epub ahead of print]. doi: 10.4088/JCP.18r12617.

59.

Moher

Liberati

Tetzlaff

, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–1012.

60.

Veritas Health Innovation. Covidence Systematic Review Software. Melbourne, Australia: Veritas Health Innovation; 2019.

61.

Faber

Ravaud

Riveros

, et al. Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review. BMC Med Res Methodol. 2016;16. 2016 Mar 22 [Epub ahead of print]. doi:10.1186/s12874-016-0136-0

62.

Cummings

. The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology. 1997;48(5 Suppl 6):S10–S16.

63.

Bahji

Mazhar

. Treatment of Cannabis dependence with synthetic Cannabinoids: a systematic review. Can J Addict. 2016;7(4):8.

64.

Bahji

Mazhar

Hudson

, et al. Prevalence of substance use disorder comorbidity among individuals with eating disorders: a systematic review and meta-analysis. Psychiatry Res. 2019;273:58–66.

65.

Bahji

Hawken

Sepehry

, et al. ECT beyond unipolar major depression: systematic review and meta-analysis of electroconvulsive therapy in bipolar depression. Acta Psychiatr Scand. 2019;139(3):214–226.

66.

Higgins

JPT

Altman

Gøtzsche

, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;34:3d5928.

67.

Viechtbauer

. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(1):1–48.

68.

Review Manager (RevMan) [Computer program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

69.

Higgins

JPT

Thompson

. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–1558.

70.

Sordo

Barrio

Bravo

, et al. Mortality risk during and after opioid substitution treatment: systematic review and meta-analysis of cohort studies. BMJ. 2017;357:j1550.

71.

Egger

Davey Smith

Schneider

, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–634.

72.

Duval

Tweedie

Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–463.

73.

Begg

Mazumdar

. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088–1101.

74.

Herrmann

Ruthirakuhan

Gallagher

, et al. Randomized placebo-controlled trial of nabilone for agitation in Alzheimer’s disease. Am J Geriatr Psychiatry. 2019;27(11):1161–1173. 2019 May 8 [Epub ahead of print]. doi:10.1016/j.jagp.2019.05.002

75.

Mahlberg

Walther

Actigraphy in agitated patients with dementia. Monitoring treatment outcomes. Z Gerontol Geriatr. 2007;40(3):178–184.

76.

Anand

Khurana

Chawla

, et al. Emerging treatments for the behavioral and psychological symptoms of dementia. CNS Spectr. 2018;23(06):361–369.

77.

Lim

See

Lee

. A systematic review of the effectiveness of medical cannabis for psychiatric, movement and neurodegenerative disorders. Clin Psychopharmacol Neurosci. 2017;15(4):301–312.

78.

Staples

Adcock

. Cannabinoids for behavioural symptoms in adults with dementia: a review of clinical effectiveness and guidelines. Can Agency Drugs Technol Health. 2017:11–23. https://www.cadth.ca/cannabinoids-behavioural-symptoms-adults-dementia-review-clinical-effectiveness-and-guidelines

79.

Weier

Hall

. The use of cannabinoids in treating dementia. Curr Neurol Neurosci Rep. 2017;17(8):56.

80.

Whiting

Wolff

Deshpande

, et al. Cannabinoids for medical use: a systematic review and meta-analysis. JAMA. 2015;313(24):2456–2473.

81.

Ben Amar

Cannabinoids in medicine: a review of their therapeutic potential. J Ethnopharmacol. 2006;105(1–2):1–25.

82.

Lemberger

Rubin

Wolen

, et al. Pharmacokinetics, metabolism and drug-abuse potential of nabilone. Cancer Treat Rev. 1982;9(Suppl):B17–B23.

83.

McGilveray

. Pharmacokinetics of Cannabinoids. Pain Res Manag. 2005;10(Suppl A):15A–22A. 2005 [Epub ahead of print]. doi:10.1155/2005/242516

84.

Russo

. Taming THC: potential cannabis synergy and phytocannabinoid-terpenoid entourage effects. Br J Pharmacol. 2011;163(7):1344–1364.

85.

Russo

. The case for the entourage effect and conventional breeding of clinical cannabis: no “strain,” no gain. Front Plant Sci. 2018 ; 9. 2018 [Epub ahead of print]. doi:10.3389/fpls.2018.01969

86.

Massimino

. In silico discovery of terpenoid metabolism in Cannabis sativa . F1000Research. 2017;6:107.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB