Systematic review and meta-analysis of the placebo effect in panic disorder: Implications for research and clinical practice

Abstract

Objective:

This review aimed to measure the degree of placebo response in panic disorder.

Data Sources:

We searched major databases up to 31 January 2021, for randomized pharmacotherapy trials published in English.

Study Selection:

A total of 43 studies met inclusion criteria to be in the analysis (with 174 separate outcome measurements).

Data Extraction:

Changes in outcome measures from baseline in the placebo group were used to estimate modified Cohen’s d effect size.

Results:

A total of 43 trials (2392 subjects, 174 outcomes using 27 rating scales) were included in the meta-analysis. Overall placebo effect size was 0.57 (95% confidence interval = [0.50, 0.64]), heterogeneity (I²: 96.3%). Higher placebo effect size was observed among clinician-rated scales compared to patient reports (0.75 vs 0.35) and among general symptom and anxiety scales compared to panic symptoms and depression scales (0.92 and 0.64 vs 0.56 and 0.54, respectively). There was an upward trend in effect size over the publication period (r = 0.02, p = 0.002) that was only significant among clinician-rated scales (r = 0.02, p = 0.011). There was no significant publication bias, Egger’s test (p = 0.08).

Conclusion:

We observed a substantial placebo effect size in panic disorder. This effect was more prominent for some aspects of panic disorder psychopathology than for others and was correlated with the source of the assessment and publication year. This finding has implications both for research design, to address the heterogeneity and diversity in placebo responses, and for clinical practice to ensure optimal quality of care.

Systematic review registration number:

PROSPERO, CRD42019125979.

Introduction

Panic disorder (PD) is a common disabling anxiety disorder with worldwide lifetime prevalence varying between 1.7% and 4.7% in general population samples (de Jonge et al., 2016; Goodwin et al., 2005; Kessler et al., 2005). PD is characterized by recurrent, spontaneous episodes of intense fear that begin abruptly and typically last for several minutes to an hour (American Psychiatric Association, 2013). Similar to other anxiety disorders, a stress-diathesis model is recommended for PD (Roy-Byrne et al., 1986). It is believed that stressful life events trigger attacks in a vulnerable person. The factors that contribute to patients’ vulnerability include behavioral, physiological, neurobiological and genetic factors (Admon et al., 2013; Gorman et al., 2000; Park and Kim, 2019; Takaishi et al., 2021).

Studies have identified several classes of medication that are effective in PD treatment to decrease the frequency of attacks, associated anxiety and phobic avoidance. These include selective serotonin reuptake inhibitors (SSRIs), serotonin-norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants (TCAs), monoamine oxidase inhibitors (MAOIs) and benzodiazepines. The severity of illness and the presence of social problems predict poor medication response (Slaap and Den Boer, 2001). Numerous studies have also identified benefits of psychotherapeutic interventions such as cognitive behavioral therapy (CBT) and brief psychodynamic psychotherapy (Carpenter et al., 2018). Most interventional studies in PD are placebo-controlled randomized controlled trials (RCTs). Although the purpose of these studies has been to identify the benefit of the active treatment, interestingly, many of them have shown that placebos are relatively effective in treating PD symptoms as well.

Although there are various ways of defining what constitutes a placebo (Carpenter et al., 2018; Gøtzsche, 1994), it is generally described as an inert substance or procedure which might have a positive effect on a patient, symptom, disease or syndrome (termed the placebo effect or response) (Shapiro, 1964). Some scholars conceptualize placebo response and placebo effect differently. While the former refers to all changes in people receiving placebo, the latter is more specific to changes that are attributable to placebo’s (hypothetical) mechanism of action (Andrea et al., 2018). However, these two terms have been widely used interchangeably in the literature. For the purposes of this paper, we will use the term placebo effect to encompass both the effect and placebo response. Placebos are well known to demonstrate therapeutic effects across a wide variety of medical and psychiatric disorders, including ulcerative colitis (Jairath et al., 2017), pain-related disorders (Klinger et al., 2018; Zunhammer et al., 2018), chronic fatigue (Pollo et al., 2008), restless leg syndrome (Silva et al., 2017), schizophrenia/psychosis (Fraguas et al., 2019; Rutherford et al., 2014), anxiety disorders and depression (Kirsch, 2019) and attention deficit hyperactivity disorder (ADHD) (Sandler et al., 2010). Proposed mechanisms for the placebo effect include conscious and unconscious expectancy effects that can arise through classical conditioning, social observational learning, cultural healing practices, as well as a patient’s own motivation and capacity for emotional regulation (Colloca et al., 2013; Flaten et al., 2011; Kaptchuk, 2011; Kirsch, 1999; Wolf, 1950). Placebo effects can also be influenced by patient personality traits, including empathy (Colloca and Benedetti, 2009), altruism (Mackenbach, 2005), optimism (Geers et al., 2005) and spirituality (Kohls et al., 2011). Imaging studies reveal that placebo effects are mediated via dopaminergic, endocannabinoid, serotonergic and/or endogenous opioid pathways. These pathways share commonalities with the pathophysiology of PD and anxiety disorders in general (Amanzio and Benedetti, 1999; Price et al., 2008; Zubieta and Stohler, 2009). Recently, ‘the placebome’ concept has also been introduced as genetic variations may modulate placebo effects (Hall et al., 2015). Collectively, these findings all indicate the potential importance of placebo effects in the treatment of PD and extant RCTs represent a vast literature examining placebo treatment of PD. Previous studies evaluated the placebo response and compared it between different anxiety disorders (Huppert et al., 2004; Sugarman et al., 2017). However, they have not systematically investigated databases, and they aimed to evaluate placebo response in anxiety disorders and not specifically PD. As a result, the magnitude of the placebo effect in PD has not been systematically characterized. Moreover, in our study, for the first time, we included different efficacy measures from the studies, instead of only the main outcome, to explore the placebo effect’s impact on different characteristics of PD. Understanding the placebo effect’s measures and its correlates could potentially help to advance in evidence-based application of placebo effect in clinical practice through different strategies for PD, and other conditions in general.

This systematic review and meta-analysis aim to address that gap to characterize the magnitude, pattern and correlates of the placebo effect among patients with PD.

Methods

Search strategy

This study has been registered in PROSPERO, international prospective register of systematic reviews: CRD42019125979. Our search terms according to ‘PICO’ instructions were as follows: ‘Panic Disorder*’; ‘Placebo Effect’ OR ‘Placebos’ OR ‘Nocebo Effect’ OR ‘Placebo’ OR ‘Nocebo’ OR ‘Lessebo’; Study Type: ‘Clinical Trial’ OR ‘Controlled Clinical Trial’ OR ‘Randomized Controlled Trial’; and no query for the outcome.

A thorough literature review was conducted using these search terms. Scientific databases, including PubMed, Scopus, Embase, Web of Science and Ovid Medline, and Google Scholar were searched from their respective dates of commencement to 31 January 2021. Moreover, backward and forward citation searching was performed to ensure the inclusion of all relevant studies. The yielded articles were then transferred to the EndNote 17.8 software (Thomson Reuters), removing duplicate publications. Next, the articles’ titles and abstracts were independently reviewed by two reviewers and excluded irrelevant and any remaining duplicate studies. We contacted the corresponding authors of eligible articles when we could not find the full text or obtain sufficient data.

Study selection criteria

Studies had to fulfill the following criteria to be eligible for the meta-analysis: Patients with a diagnosis of PD (according to the Diagnostic and Statistical Manual of Mental Disorders, DSM III, DSM III-R, DSM-IV, DSM-IV-TR or DSM-5), with or without comorbid agoraphobia (e.g. PD with agoraphobia), randomized placebo-controlled trial study design, published in English, at least one pharmacological intervention, applied a standardized primary outcome measure for outcome evaluation and accessible or attainable data on measurements at baseline and final evaluation for the placebo group. Based on this, all placebos in included studies were the placebo equivalent of the pharmacological intervention, or so-called ‘pill placebos’. To address the PICO format of the search strategy, the placebo was considered as the intervention and the comparison and outcome to be the change in outcome measurement values from baseline to the final measures.

Studies that investigated induced-panic, PD recurrence rate (not the treatment/placebo effect), other anxiety disorders in general and those in which the placebo group were simultaneously treated with placebo and other interventions were excluded.

Data extraction

Data were independently extracted from each study by two reviewers. Any inconsistencies in data extraction between reviewers were discussed with the principal investigator of the study (M.A.) to reach a consensus.

The extracted data included general identifiers (e.g. first author, publication date, study country), study specifiers (e.g. sampling setting, PD definition criteria, intervention type, sample size [total, intervention, placebo], follow-up duration, frequency of outcome measurements, subjects’ sex and age [total, intervention, placebo]), study quality assessment and risk of bias measures (e.g. study type/design, random allocation, blinding, analysis [per protocol, PP, or intention-to-treat analysis, ITT], attrition/completion rate [total, intervention, placebo]) (Higgins et al., 2011; Sterne et al., 2019) and outcomes (e.g. measurement tool, baseline and final value/standard deviation [SD] for each measurement tools in placebo groups). When post-scores were not reported, we utilized Becker’s (1988) formula based on pre–post changes in scores and the SD of the baseline. Measurement tools were further categorized as either clinician-administrated or patient self-report.

Quality assessment

In order to evaluate the quality of included studies, we used the Version 2 of the Cochrane tool for assessing risk of bias in randomized trials (RoB 2) (Sterne et al., 2019), with some modifications, as the items about the intervention group and comparison of study group were not applicable in our review (Table 1).

Table 1.

Summary of findings and placebo group effect size from randomized controlled trials in patients with panic disorder.

CI: confidence interval; F/U (w): follow-up duration in weeks; N: placebo group sample size; Male (%): ratio of male subjects in placebo group; Age: mean age in placebo group; Measures #: number of assessments during the trial; Tools #: numbers of measurements tools used in each trial; Completion rate: rate of samples completed the trial in placebo group; Bias: risk of bias; L: low; S: some concerns; H: high; ITT: intention-to-treat vs PP: per protocol analysis; Placebo Effect: effect size based on modified Cohen’s d for paired data.

Data synthesis and statistical analysis

The present meta-analysis fulfilled the criteria of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009). STATA/SE 17.0 (StataCorp LLC, College Station, TX, USA) was used to analyze data. Cohen’s d effect sizes (ESs) were calculated for effectiveness. For studies that included both ITT and PP data, outcomes were measured for each design. In crossover studies, every time the placebo was taken in each arm, the crossover was considered as a separate data set, and we aggregated them to produce a single estimate for every study outcome.

ES calculation

Given that baseline and endpoint measures were not independent, we employed a modified Cohen’s d formula to calculate ESs (Becker, 1988; Morris, 2000, 2008)

$$Cohen\prime sd=\frac{{\overline{x}}_1-{\overline{x}}_2}{\sqrt{S{D}_1^2+S{D}_2^2-2*r*S{D}_1*S{D}_2}}*\sqrt{2\left(1-r\right)}$$

Note that ESs presented here quantify improvement from baseline to endpoint in patients randomized to the placebo arm within the included trials. Each measurement tool’s test–retest reliability stat, r, was retrieved from the literature. When there were several reports on test–retest reliability from different studies, we used the highest ones to arrive at the most conservative/lowest estimate of the ES (Table 1).

Overall, the identified studies used 27 different outcome measurement tools. We categorized them into six groups: Panic Attack Count, Panic Severity Scales, Anxiety Scales, Depression Scales, General Scales and Hemodynamic Measures (Table 1). We also performed a subgroup analysis based on PP or ITT analysis and administration type (clinician-rated vs self/patient-rated questionnaires or hemodynamic measures).

Pooled ES calculation

We used STATA’s ‘metan’ command to estimate a random model pooled ES. Several measures from each study have been aggregated in the meta-analysis to provide an estimate of ES for that study. In addition, we performed a subgroup meta-analysis to estimate the pooled ES for each tool and for tool categories. In other words, data from studies with an identical outcome measurement tool, for example, HAM-A (Hamilton Anxiety Scale), were used in meta-analysis of the ES estimate for that tool. The same also applies for tool groups.

To study the correlates of placebo ES, we used a meta-regression model (‘metareg’ command) to evaluate the interaction between placebo ES and measured covariates, such as calculation type (ITT/PP), subjects’ age/sex and publication date and clinician or patient-administered scales. The I² statistic was utilized to determine the heterogeneity of the current meta-analysis (Higgins and Thompson, 2002).

Publication bias

Egger’s test for small-study effect was performed for assessment of publication bias. Moreover, the publication bias was visually evaluated by funnel plots.

Results

After removing duplicate citations and non-English papers, we screened 1762 studies by the title, of which 1407 were excluded. Among the remaining 355 publications, 312 studies did not fulfill inclusion criteria (Figure 1). The 43 studies included in the final analysis, with total sample size of 2392 in the placebo arm of the studies, were all published between 1986 and 2009. These RCTs involved the study of 12 different active treatments–placebo comparisons (all with at least one medication arm) (Table 1), Supplement, and 174 clinical outcomes measured using 27 tools (mean ± standard error [SE]: 5.6 ± 0.6 outcome measurements by 4.0 ± 0.4 different tools in each study). ESs for the six types of measure and individual measures are shown in Table 2.

Figure 1.

Flow diagram of studies selected and included in systematic review of placebo effect size in patients with panic disorder.

Table 2.

Placebo effect size for different measurement tools and groups in patients with panic disorder.

Freq.: number of studies using each tool/tool group as an outcome measure; R: test–retest reliability for each tool; N: sample size in for studies’ placebo group, most of the studies had used more than one outcome measure tools so it may outcount the total placebo sample size from different studies; Placebo Effect: effect size based on modified Cohen’s d; CI: confidence interval; Weight%: the weight of each tools data on total effect size estimate; STAI: State Trait Anxiety Inventory; BAI: Beck Anxiety Inventory; HAM-A: Hamilton Anxiety Scale; SPRAS: Sheehan’s Patient Rating Anxiety Scale; SCRAS: Sheehan’s Clinician Rating Anxiety Scale; PSS: Phobia Severity Scale (a modification from Mark’s Fear Questionnaire); HSCL: Hopkins Symptom CheckList; PARS: Phobic Avoidance Rating Scale; CASS: Covi Anxiety Scale Schedule; CAS: Clinical Anxiety Scale; BDI: Beck Depression Inventory; HAM-D: Hamilton Depression Scale; MADRS: Montgomery–Asberg Depression Rating Scale; CGI: Clinical global impression; WSDS: Work & Social Disability Scale; SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; P&AS: Panic & Agoraphobia Scale; PASS: Panic Attack Severity Score; PDSS: Panic Disorder Severity Scale; REML: restricted maximum likelihood.

Test of heterogeneity p < 0.001, I²: 99.0%.

The overall placebo ES among all pooled outcomes was 0.57 (95% confidence interval [CI] = [0.50, 0.64]). The ESs ranged from −0.236 to 1.565, and there was a prominent overall heterogeneity (I²: 96.3%) (Table 1).

The pooled ESs also demonstrated significant heterogeneity between both individual tools and tool categories. The largest placebo ES was among general scales (0.92, 95% CI = [0.76, 1.07]) followed by panic severity, anxiety and depression scales, all of which were of similar magnitude (0.34–0.64). The smallest placebo ES was found for hemodynamic measures (0.19, 95% CI = [−0.06, 0.45]) (Figure 2).

Figure 2.

The calculated pooled placebo ES based on study and measurement/tool characteristics among randomized controlled trial studies on the patients with PD.

Among panic attack count outcomes, all three types (situational attacks, spontaneous attacks and total attacks) had a relatively narrow range of placebo ESs (0.17–0.68), while there was a broad range of ESs among panic severity scales (0.03–0.96). Placebo ESs also varied to a slightly lesser degree between anxiety scales (0.16–0.86). Both depression scales (from 0.44 to 0.59) and general scales (from 0.6 to 0.95) showed moderate to high levels of placebo effect among patients with PD. Interestingly, while supine systolic blood pressure (SBP) and supine heart rate (HR) demonstrated non-significant placebo ES estimates (−0.04 and 0.17, respectively), supine diastolic blood pressure (DBP) showed a significant moderate ES for the placebo on patients with PD (0.53, 95% CI = [0.18, 0.88]) (Table 1).

Regarding the source of measurements, clinician-administered tools resulted in higher estimates of placebo ESs (0.75; 95% CI = [0.66, 0.84]) than self-report and objective tools (0.35, 95% CI = [0.28, 0.43] and 0.19, 95% CI = [−0.06, 0.45], respectively, Figure 2).

Note that 89 measurements were based on a PP approach, which excluded data from participants with protocol deviation from the final analysis, and 85 based on an ITT or last observation carried forward (LOCF) model. There was a significant difference in ESs based on their approach to this: PP ES 0.49 (95% CI = [0.41, 0.58]), ITT ES 0.64 (95% CI = [0.55, 0.74]) (Figure 2).

In meta-regression analysis, we included all covariates, such as publication year, follow-up duration, frequency of assessments, number of rating scales, baseline measurement and some items related to placebo subjects (sample size, age, sex and trial completion rate). After adjusting for the covariates, the year of publication was positively associated with higher placebo effect (r = 0.023 per year, 95% CI = [0.010, 0.37], p = 0.001). The upward trend of the placebo effect over the publication period was only significant for clinician-rated measures and showed consistent placebo ES in patient-rated scales (Figure 3). Regarding the study factors correlated with publication year, more recent studies had higher sample sizes (R = 0.028, R² = 0.17; p = 0.005), but no significant change in sample homogeneity was reflected on baseline SD (R = −0.106, R² = 0.011, p = 0.112). While placebo effect from clinician/observer-rated scales shared the same upward trend over time (p = 0.008), there was no significant change in placebo effect over time among patient/self-rated (Figure 3) or hemodynamic measures (p = 0.70 and p = 0.56, respectively).

Figure 3.

Metaregression model for the effect of publication year and placebo group sample size on calculated placebo effect size in patients with panic disorder.

In terms of possible publication bias, Egger’s test for small-study effect was non-significant (p = 0.205).

Discussion

This study is, to our knowledge, the only systematic review and meta-analysis of RCT data to investigate the efficacy of placebos in the treatment of patients with PD. We found a significant medium-high overall placebo ES (0.57; range −0.23 to 1.57) and this finding highlights the substantial positive effects of placebos in PD.

Placebo efficacy in PD is similar to ESs for lorazepam (ES: 0.69; 95% CI = [0.38, 1.01]) (Blaya et al., 2007), duloxetine (0.64; 0.23–1.05) (Simon et al., 2009), a number of psychotherapies (0.92; 0.71–1.12) (Sanchez-Meca et al., 2009) and 20 different medications (ES from 0.33 to 0.96) in anxiety disorders in general (Bandelow et al., 2015). It is also in line with placebo effects in other medical and psychiatric conditions such as pain: 0.66 and 0.89 (Sauro and Greenberg, 2005; Zunhammer et al., 2018), osteoarthritis: 0.51 (Doherty and Dieppe, 2009), intellectual disability: 0.49 (Curie et al., 2015) and depression: 1.69, 1.0 and 0.8.

We also found clinically meaningful heterogeneity of the placebo ES between studies and specific outcome measures (I²: 95–97%), in line with findings for other disorders (Dold and Kasper, 2015; Rief et al., 2009; Walsh et al., 2002). Importantly, such heterogeneity was evident between clinician and patient-rated measures, with the latter showing more modest effects. We further ascertained that the publication year and the characteristics of the outcome measures contributed to the diversity of the placebo effect among the studies.

Historically, randomized placebo-controlled trials are the gold standard for the objective efficacy measurements by subtracting the placebo effect from the overall efficacy (Bondemark and Ruf, 2015).

However, the dynamic between placebo and intervention effect is complex (Howick and Hoffmann, 2018). For example, they may have an additive or synergic interaction. The latter would mean that omitting the components attributed to the placebo effect, totally or partially, would result in an under-estimated intervention effect. Besides the publication year and the characteristics of the outcome measure, we were unable to identify other factors that explained the diversity in the placebo effect among the studies. Unfortunately, clinical trials generally have scarce data on the participants’ (subjective) experiences, including expectations, making it difficult to test hypotheses regarding different mechanisms of the placebo effect (Kelley et al., 2009). Study procedures and participants’/researchers’ subjective state of mind (emotional and cognitive) may contribute to the heterogeneity in the placebo effect. Although it may be presumed that the best research design to estimate the effects of the placebo is comparing placebo with no-intervention (Hróbjartsson et al., 2011), this approach cannot eliminate some inevitable research processes like observations and assessments. Thus, the inherent nature of research interferes with the estimate of the placebo effect (Kelley et al., 2009).

Perhaps the most important implication of our findings is the need to leverage the placebo effect in clinical care, including maximizing attentive care methods. To ensure the quality of care, we suggest that guideline development task forces should amend any intervention recommendations with a set of minimal clinical encounter requirements in terms of quality and follow-up frequency or duration.

Source of measurements

Regarding the source of measurements, our findings indicate a larger ES for clinician-rated outcomes than self-reports (0.75 vs 0.35). This difference was significant both within studies with both types of scales and between different studies and is consistent with prior findings for antidepressants (1.85 vs 0.67) (Rief et al., 2009). Clinician-rated outcomes would inherently involve a relational clinical encounter with the subject, affecting participants’ expectations. The intersubjective nature of a clinical visit may also involve possible biases (e.g. expectation bias) that may lead subjects to under-report negative mental states and clinicians to under-report objective findings (Fava et al., 2003). Our review was not designed to test these potential effects, but they deserve future study.

Huppert et al. (2004) showed a significant pre–post-treatment placebo response in PD and generalized social phobia (d = 0.91 and 1.08, respectively). Furthermore, a meta-analysis was conducted by Sugarman et al. to compare pre–post-treatment placebo response to Food and Drug Administration (FDA)-approved antidepressants in anxiety-related disorders. Obsessive-compulsive disorder had the lowest ES for placebo response (Hedges’ g = 0.49; 95% CI = [0.40, 0.59]), while 10 studies that evaluated the response to placebo in PD showed an ES of 0.94 (95% CI = [0.75, 1.13]) with a prominent heterogeneity (I²: 83.4%) (Sugarman et al., 2017). Our results showed lower ES values which may be due to a higher number of included articles or some methodological differences such as the inclusion of all outcome measurements.

Publication year

We found a significant positive association between the publication year and the ES (a > 0.2 increase for each decade). This has also been found in other systematic reviews that studied the placebo ES in antidepressant trials (Kirsch, 2019; Rief et al., 2009; Walsh et al., 2002), anti-psychotic trials (Dold and Kasper, 2015), dementia (Hyde et al., 2017) and major anxiety disorders (Bandelow et al., 2015). Several factors may potentially contribute, including differences in baseline severity indices, subjects’ age, duration of illness, trial duration and frequency of assessments (Dold and Kasper, 2015; Hróbjartsson and Gøtzsche, 2001; Hyde et al., 2017). Of note, Furukawa et al. conducted two systematic reviews of antidepressant trials for depression which showed a consistent placebo effect over more than two decades (Breilmann et al., 2018; Furukawa et al., 2016). However, after adjusting for baseline severity, age, sex, setting and other co-factors, there was no significant trend in the placebo effect over 25 years. It has been argued that sample characteristics, for example, more homogeneous participants, may contribute, at least partly, to this trend (Furukawa et al., 2016; Rief et al., 2009) although the trend was preserved in our study despite accounting for multiple covariates.

In addition, the time-dependent trend in the placebo effect only emerged in the clinician-rated scales. Prior reviews did not specify the type of measures, so we cannot determine whether this finding is unique to our study. Clinicians’ expectation bias has been reported as a potential source of change in the placebo effect over time (Rief et al., 2009; Walsh et al., 2002) possibly due to greater enthusiasm about the efficacy of the interventions over time (Hróbjartsson et al., 2011).

Symptom domains

We found that General scales (that measure a battery of symptoms and overall function) and other Anxiety scales (that focus on the general anxiety symptomatology rather than PD-specific symptoms) showed higher placebo effects in PD (ES = 0.92, 0.64, respectively) than Depression scales (ES = 0.54) or Panic scales (ES = 0.56). Interestingly, we observed a small- to low-medium ES for Panic attack counts (ES = 0.34) and Hemodynamic measures (ES = 0.19). To the best of our knowledge, our review is the first study that included all rating scales reported, while other published reviews relied on one primary outcome with the most common/valid scale, such as HAM-A or HRSD (Rief et al., 2009; Walsh et al., 2002). This method makes it impossible to compare our domain-specific findings to other reviews. Furthermore, the different magnitudes of placebo effects among various domains of symptoms in patients with the same disorder (in our review PD) indicate the diversity in the mode of action and outcomes in placebo effects. While most reviews distill published data to the primary/main outcome, the large domain-specific heterogeneity within the studies presented here emphasizes the importance of applying a more holistic approach to data aggregation and highlights the complex nature of the placebo effect. In other words, placebos do not have the same impact for different symptoms even within a single controlled trial and presumably within individual subjects.

Limitations

This study has several limitations. First, to extract more comparable and consistent reliable data, we included published papers that reported the measurements data for the placebo group, further limiting the sample of RCTs. A significant group of papers provided insufficient data, or those data were not retrievable either via statistical methods or after contact with the authors. Moreover, we excluded studies that reported just the response rate as their outcome (e.g. a 50% decrease in primary scores) without baseline and final scores. Second, based on our study protocol, we only attempted email contact with the authors, many of whom were unavailable or whose addresses were not identified for most of the studies published before 2000. Third, we included only published studies, which may increase the risk of bias in our findings. Nevertheless, we speculate that our findings are likely robust because the placebo effect was not the primary outcome in any of the studies. Also, the studies with a significant intervention effect (presumably with wide separation from placebo or low placebo responses) are more likely to be published (Walsh et al., 2002), which could have resulted in a more conservative placebo effect estimate. In addition to those, the non-significant small-study effect (Egger’s test) indicated a low risk for publication bias in our data set. Fourth, there were 27 rating scales with different psychometric characteristics in our review. There is a significant challenge in aggregating data from the different scales. To address this concern, we reported both scale-specific and overall ES estimates to bring it for further considerations in future studies. Meanwhile, including different outcome measures is a strength in our study. Other similar reviews overlooked data on non-primary psychopathology domains in their reviews by extracting only one primary or aggregate measure from each study. Fifth, we just included studies in English, which might result in language bias (Jackson and Kuriyama, 2019). Sixth, we used the same r for all studies. In contrast, it might impact the heterogeneity of the effects, decrease the CI and be less conservative in subgroup analyses. A further sensitivity analysis based on pre–post correlation of the studies data also could be helpful. Seventh, we aggregated every single measure to one ES instead of having a hierarchy of measurements so that this approach may underestimate the heterogeneity of the effects. Finally, like most similar reviews, our primary study design prevented us from a more extensive exploration of potential mediators of the placebo effect.

Further directions

Although much remains unknown about the nature and the correlates of the placebo effect, most studies demonstrate the presence of a substantial and heterogeneous placebo effect across a variety of health conditions (Dold and Kasper, 2015; Hyde et al., 2017). For several reasons, including lack of detailed data on potential correlates, systematic reviews have not provided explanations for nuanced placebo effect findings, such as the trend in placebo effects over time. On the other hand, a network meta-analysis, or if possible, a mega-analysis, could potentially provide a better picture on the nature of the interaction between placebo effect and different interventions effect. We believe that more focus on the placebo effect through open-label placebo trials or placebo-oriented designs embedded in conventional RCTs should be considered. Those studies can answer several preliminary but crucial questions in the placebo effect research: (1) What are the sources of heterogeneity in the placebo effect? (2) What is the nature of the interaction between placebo and intervention effect (additive or synergic, for example)? and (3) What are the correlates of the placebo effect that can/should be assessed in controlled trials? Answers to these questions will enhance our understanding and interpretation of placebo arm results in RCTs.

Regarding clinical applications of findings on the placebo effect, there is currently sufficient knowledge to emphasize the role of the placebo and its potential correlates on therapeutic outcomes. We would recommend, where not already done, that clinicians consider applying minimal standards of care rituals to provide sufficient subjective and intersubjective therapeutic expectations. Furthermore, we suggest that clinical guideline taskforces amend intervention recommendations to highlight this issue and to suggest minimal care levels informed by the quality of care in the original trials.

Conclusion

In conclusion, our review demonstrates a substantial placebo effect in the treatment of patients with PD. This finding should be considered in the design of future trials on PD to address intervention efficacy and evaluate factors related to the placebo effect. Clinicians need to be conscious of placebo effects maximizing both placebo and intervention effects while providing the gold standard of care.

Supplemental Material

sj-xlsx-1-anp-10.1177_00048674211068793 – Supplemental material for Systematic review and meta-analysis of the placebo effect in panic disorder: Implications for research and clinical practice

Supplemental material, sj-xlsx-1-anp-10.1177_00048674211068793 for Systematic review and meta-analysis of the placebo effect in panic disorder: Implications for research and clinical practice by Masoud Ahmadzad-Asl, Farnoush Davoudi, Safoura Mohamadi, Fatemeh Hadi, Seyed Aria Nejadghaderi, Seyed Hamidreza Mirbehbahani, Roxana Jabbarinejad, Sadaf Saneh, Mahdi Arshadi, Morteza Naserbakht, Mark Sinyor, Ali Kabir and Ahmadreza Shamshiri in Australian & New Zealand Journal of Psychiatry