Sage Journals: Discover world-class research

Abstract

More than 10% of the general population regularly experience paranoid thoughts. Persecutory delusions occur in one third of psychiatric patients in the United Kingdom and are associated with severe clinical and social impairment. Furthermore, individuals with elevated vulnerability to paranoia interpret ambiguous environmental information more negatively than those with low vulnerability, a cognitive phenomenon called interpretation bias. We conducted a systematic review and meta-analysis of the association between interpretation bias and paranoia. Twenty studies were included, and our meta-analysis indicated that a negative interpretation bias was associated with paranoia both in clinical (standardized mean difference, or SMD = 1.01; 95% confidence interval [CI] = [0.51, 1.52], p < .001) and nonclinical populations (SMD = 1.06; 95% CI = [0.28, 1.85], p = .008). Our results also showed that higher negative interpretation bias was positively correlated with the severity of paranoia, and results were consistent in nonclinical (r = .32; 95% CI = [.21, .43], p < .001) and clinical samples (r = .38; 95% CI = [.27, .48], p < .001). These findings might orient prevention strategies and psychological interventions for paranoia.

Keywords

psychosis cognition interpretation bias paranoia systematic review meta-analysis

Paranoid ideation, including mistrust, suspiciousness, ideas of reference, and persecution, is continuously distributed in the general population, with actual persecutory delusions being placed at the extreme end of the continuum (Bebbington et al., 2013). Genetic heritability and environmental risk factors, such as stressful life events, have been shown to play an important role in the etiology of paranoia (Zavos et al., 2014). However, less is known about the psychological mechanisms implicated in the persistence and severity of paranoid thoughts. From a cognitive perspective, the core of persecutory delusions is unfounded threat beliefs that harm to the self will occur from others (Freeman, 2016; Kuipers et al., 2006). It is thought that this is maintained by a range of cognitive mechanisms, including lack of flexibility to generate alternative explanations and reduced data gathering (Garety et al., 2005).

The cognitive biases operating in psychopathology are generally defined as “the selective processing of pathology congruent information that might confirm a pathological belief” (Savulich, Shergill & Yiend, 2012, p. 516). Cognitive biases refer to specific cognitive mechanisms, such as attention, interpretation, and recall, each of which are thought to operate at different (typically sequential and cyclical) stages of information processing (see Blanchette & Richards, 2010). For example, Beck and Clark (1997) proposed a three-stage model of information processing involving initial registration of threat (perception and attention) and a secondary elaboration and strategic processing phase (interpretation and, later, recall).

Attentional bias is thought of as the preferential selection, for further processing, of one stimulus from among multiple competing stimuli. A bias occurs when the selected stimulus is consistently of one particular type, such as threat in the case of anxiety or paranoid in the case of paranoia. In contrast, interpretation bias is thought to arise once an ambiguous stimulus has already been attended and its multiple possible meanings encoded. Interpretation is the process that resolves the encoded ambiguity into one or other meaning, which results in a single, unambiguous mental representation of the original stimulus. A bias in interpretation occurs when this final unambiguous meaning is consistently of one particular type. Interpretation bias has been more concisely defined as “a consistent tendency to interpret emotionally ambiguous stimuli, situations, or events in a negative (or positive) manner” (Lee, Mathews, Shergill, & Yiend, 2016, p. 26).

To illustrate, consider a patient with paranoid symptoms displaying a paranoid bias in both attention and interpretation entering a room in which there is a security camera. A paranoid attentional bias would result in the camera immediately capturing the patient’s attentional resources and being internally prioritized for further cognitive processing over and above other stimuli (e.g., objects or perhaps people in the room). A paranoid interpretation bias would arise once the possible meanings of the presence of the security camera had been processed and just one selected for further encoding and elaboration (e.g., “it is there to protect me” or “it is there to spy on me”). A person with a spider phobia entering the same room would be unlikely to display similar biases in processing toward the camera but might well do so toward, for example, a small black mark in the corner of a distant wall (which might or might not be a spider).

Interpretation biases have been researched extensively in affective disorders. Numerous studies suggest that negative interpretation biases are present in social phobia (Amin, Foa, & Coles, 1998), anxiety (Mathews & Mackintosh, 2000), and depression (Hirsch, Meeten, Krahé, & Reeder, 2016). Likewise, techniques designed to reduce negative interpretation biases have shown benefits across a range of psychological disorders (Jones & Sharpe, 2017).

Interpretation bias is usually seen as a mechanism that is the same across psychopathologies (indeed, across all cognitive function healthy or otherwise) but that operates on different types of content dependent on the disorder in question. Thus, the underlying cognitive mechanism is the same regardless of psychopathology, but its manifestation varies according to disorder, especially in terms of the kind of material on which the bias operates and the output of the biased process.

The assumption of a common mechanism derives from early theoretical views (e.g., Neisser, 1967) on the cyclical nature of information processing in general, in which interpretation is considered a discrete stage of an iterative cognitive cycle, occurring after perception and attention and involving the resolution of multiple competing meanings when the information that has been attended has some element of ambiguity. Biases might operate on any of these stages, interpretation bias being one example. These early theories have then informed the development of many subsequent specific models of biased cognition in which biases in the process of interpretation (of ambiguity) are purported to contribute to the maintenance of the disorder.

The manner in which interpretation bias is proposed to differ between different disorders is well captured by the notion of content specificity (Mathews & MacLeod, 1994; Yiend, Barnicot, Williams, & Fox, 2018), which refers to the idea that a bias will be stronger when the information being processed has more direct relevance to the disorder and its symptoms. Thus, although anxiety, depression, and paranoia might all be hypothesized to show interpretation biases favoring negative information in general, information that permits more specifically paranoid interpretations would be expected to elicit the strongest biases in this population. For example, biases in panic disorder should be most closely associated with interpretations concerning the meaning of bodily sensations as a sign of death or disease, whereas biases in paranoia should be closely associated with interpretations reflecting a threat of harm to the self, such as the stare of a stranger as malicious. Some studies have sought to empirically test this notion in paranoid samples (Savulich, Freeman, Shergill, & Yiend, 2015).

Although many cognitive processes involved in psychotic symptoms have been well researched, such as jumping to conclusions (So, Siu, Wong, Chan, & Garety, 2016) and attributional phenomena (Bentall, Corcoran, Howard, Blackwood, & Kinderman, 2001), no reviews that we are aware of have been carried out specifically on interpretation bias in paranoia, as defined above (however, for a narrative review, see Savulich et al., 2012). Biased interpretations are a contributing factor in the maintenance of persecutory delusions and co-occur with other cognitive biases in influencing the severity of paranoia. People with persecutory delusions also show greater externalizing attributional bias (the tendency to attribute to others the cause of negative events), reduced data gathering or jumping to conclusions (Dudley, Taylor, Wickham, & Hutton, 2016), reasoning biases that prevent the processing of alternative explanations, and belief inflexibility (Murphy, Bentall, Freeman, O’Rourke, & Hutton, 2018).

Over recent years, the literature examining the role of interpretation bias in paranoia has expanded because interpretation bias may be directly implicated in the onset and maintenance of the disorder (e.g., Yiend et al., 2019). The suggested mechanism for this is that an enhanced tendency to select paranoia-relevant information for further processing (via a hard-wired or acquired interpretation bias) will lead to an artificially increased perception of danger of harm to the self, which will enhance and maintain the matching mood and symptoms (e.g., interpreting a stranger’s stare as malicious is likely to support paranoid beliefs and increase distress about being at risk of observation by others). This in turn will promote further biased processing, and a cycle of reciprocal causation would be established (e.g., Mathews, 1990). In paranoia, an enhanced tendency to select emotionally ambiguous information, such as the stare of a stranger or the scream of a child, might lead to an increased sense of threat and a perception of danger coming from the environment, which in turn can maintain paranoid symptoms and lead to further biased processing.

The main objective of this study was to review the literature on biased interpretation in clinical and subclinical paranoia. We also conducted a meta-analysis comparing the mean differences in interpretation bias among patients with psychosis, control participants, and individuals with psychotic or paranoid traits. Furthermore, in a second meta-analysis, we analyzed separately the correlation between interpretation bias and severity of paranoid symptoms in clinical and nonclinical samples.

Method

Search strategies

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting of this systematic review and meta-analysis (Liberati et al., 2009; Moher, Liberati, Tetzlaff, Altman, & PRISMA Group, 2009). A literature search was conducted on MEDLINE (via PubMed), PsycInfo, and Web of Science using the following search term: ((interpret* or process* or judg* or perceiv* or cognition or recogni*) and (ambigu* or unclear* or uncertain* or vague or indetermin* or indefinite or obscure or neutral or emotionless) and (impair* or bias* or erroneous or error* or delusion* or difficult*)) AND (delusion* or paranoi* or psychosis or psychotic or schizoa* or schizoph* or schizoty* or bipolar or manic or mania).

This search term enabled the filtering of studies that did not examine interpretation biases and paranoia or psychosis while also including studies using relevant measures and tasks that, although not explicitly described as interpretation bias, matched our cognitive experimental definition stated earlier. The search dated from 1990 to June 2019. In addition, we contacted authors to request details of any further published or unpublished studies and manually searched reference sections of relevant review articles, book chapters, empirical articles, and issues of journals to identify any studies that had not been included in the literature databases.

Inclusion and exclusion criteria

A study was included if it met the following criteria:

The authors assessed interpretation bias, defined as the tendency to interpret emotionally ambiguous information in an adaptive (e.g., positive) or maladaptive (e.g., negative) manner (Savulich et al., 2015).

Interpretation bias was assessed in any of the following groups: (a) patients presenting with clinically diagnosed psychosis or paranoid symptoms (diagnosis of schizophrenia or related disorders, paranoid personality disorder, or bipolar type I disorder) according to the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM–V; American Psychiatric Association, 2013) and ICD (World Health Organization, 1992); (b) general population, control samples, or individuals presenting with subclinical psychotic symptoms, which in each case used a standardized instrument to measure paranoid or related traits (e.g., trait measures of paranoia, schizotypy, or psychoticism); or (c) the study design included experimental designs in which interpretation bias was experimentally manipulated (only the premanipulation data included in analysis), cross-sectional case-control comparisons, prospective designs in which the relationship between psychosis/paranoid symptoms and interpretation bias was investigated as a function of time (only baseline data comparing groups included in analysis).

Any of the following criteria were grounds for exclusion: (a) studies including participants with organic etiology of psychosis or substance-induced psychosis with no way to exclude these participants’ data; (b) review studies, letters, dissertations, abstracts, case studies, commentaries; (c) studies not written in English or in peer-reviewed journals.

Data synthesis and analysis

Data (including information such as sample, outcome measures, results) were extracted by J. Kang and reviewed by A. Trotta using a spreadsheet (Excel 2008 for Mac; Microsoft, Redmond, WA), and then checked. Any disagreements were resolved via a discussion between J. Kang and A. Trotta, with reference to the senior authors’ opinion if necessary.

Extracted data included (a) study characteristics (authors and year of publication); (b) study design; (c) sample characteristics (size, psychiatric diagnosis, method of diagnosis); (d) measure of interpretation bias; (e) measure of paranoia, psychosis, or both; and (f) relevant results. We contacted the authors if there was a possibility of participant duplication between studies.

Statistical analyses were carried out using the meta-analysis commands of Stata (Version 15; StataCorp, College Station, TX). Effect sizes were computed using correlation coefficients, means, and standard deviations as available. Standardized mean difference using Cohen’s d (Cohen, 1969) was calculated by dividing the mean difference in interpretation bias between individuals with and without paranoid symptoms by the pooled within-groups standard deviation. Summary meta-analysis involving correlation coefficients were done with a Fisher z transformation of the correlation coefficients. The results were then back transformed into a pooled correlation coefficient (r; Hedges & Olkin, 1985).

All analyses were stratified by population (clinical or nonclinical) to assess whether findings differed across sample types. We ran sensitivity analysis to identify the source of heterogeneity associated to the variety of bias tasks. We identified two main groups of tasks: (a) self-report measures and (b) experimental tasks, including semantic ambiguity represented in text passages (e.g., similarity-rating test or scrambled-sentences task) and situational or behavioral ambiguity representative of real-life scenarios (e.g., facial expressions or incidental movements).

Standardized effect sizes were meta-analyzed using random-effects models. Heterogeneity between studies was assessed with the Cochran’s Q test (DerSimonian & Laird, 1986). A statistically significant Q value indicates true heterogeneity in effect sizes beyond random error; the I² statistic was calculated to express the proportion of variation among studies that was due to heterogeneity (Higgins, Thompson, Deeks, & Altman, 2003). The results are displayed using a forest plot.

Further exploration of heterogeneity was carried out using metaregression analyses for testing effects of the following potential moderator variables: population studied (a two-level factor: clinical vs. nonclinical population), year of publication, quality score, age, gender, measure of paranoia, and interpretation-bias task. All moderators were otherwise entered separately to the metaregression because of the small number of included studies (Borenstein, Hedges, Higgins, & Rothstein, 2009).

The presence of publication bias was assessed informally by visual inspections of funnel plots, which represent a plot of a study’s precision (1/SE) against effect size. The absence of studies in the right bottom corner (low precision and small effect sizes) of a funnel plot is usually taken as an indication of publication bias. In addition, Egger’s test of publication bias was used to formally assess, based on the nature and direction of results, whether there was a tendency for selective publication of studies (Egger, Smith, Schneider, & Minder, 1997).

A significance level of p < .05 was used for the random-effects model, homogeneity, publication bias, and metaregression analyses.

Risk of bias in individual studies

To critically examine and reduce the effect of data selection in the review, a risk of bias assessment tool was created adapting the criteria from the Agency for Healthcare Research and Quality (2014). The tool, detailed in Table 1, included the following criteria: (a) selection bias, (b) performance bias, (c) detection bias, (d) attrition bias, (e) reporting bias. J. Kang and A. Trotta rated the studies included in analysis independently, and any differences were resolved by discussion.

Table 1.

Characteristics of the Main Measures of Interpretation Bias of the Studies Included in the Systematic Review

Interpretation-bias task	Description of task^a	Specificity to paranoia^b
Experimental measures representative of real-life scenarios
Gesture-interpretation task (White et al., 2016)	Participants viewed 60 three-second video clips in which an actor made a gesture that was ambiguous in terms of whether it was directed toward or away from the viewer (e.g., gaze directed away but body posture directed toward viewer). Participants were required to interpret whether the gesture was directed at themselves (self-referential).	Not specific to paranoia Interpreting a gesture as relating to oneself does not necessarily indicate the participant has assumed an intention to harm. Self-referential biases are common across a wide range of psychological disorders.
Abbreviated trustworthiness task (Adolphs et al., 1998; Couture et al., 2010)	Participants were shown 42 faces of unfamiliar males and females in natural poses. They were asked to rate the faces on a 7-point scale from −3 to +3 with regard to trustworthiness.	Unclear An expression interpreted as untrustworthy could reflect the viewer’s belief that there is an intention to harm them but could also represent other interpretations. The rating scale used is not sufficiently specific to determine relevance to paranoia.
Interpretation of ambiguous laughter (Green et al., 2011)	An experimental event was presented during the testing session of participants in which a male stooge interrupted the experimenter and following the exit of the stooge and experimenter, a 35-s audio recording of male laughter was played. The experimenter then returned to the room, apologized to the participant, then asked the participant about their explanations of the interruption and laughter in a semistructured interview.	Specific to paranoia The interview is designed to require participants to interpret whether the stooge and experimenter had an intention to harm the participant (i.e., make fun of the participant and laugh as a result).
Letter-string discrimination (Holt et al., 2006)	Participants were shown 108 words (54 neutral and 54 unpleasant) and requested to determine whether the displayed word was neutral or unpleasant.	Not specific to paranoia The task uses unpleasant words (e.g., cancer, horrid) and does not include words that have specific relevance to paranoia (e.g., spy, interrogate).
Experimental tasks of semantic ambiguity
Similarity-ratings task (SRT) (Eysenck et al., 1991; Mathews & Mackintosh, 2000)	Participants encoded ambiguous passages by reading three sentences that describe commonly experienced social situations. Participants were asked to rate disambiguated sentences that offered possible negative or benign/positive interpretations of the passage. Each sentence was rated for its similarity to the meaning of the passage from 1 (very different in meaning) to 4 (very similar in meaning).	Specific to paranoia The version of the SRT used in the studies included in this review was designed to capture interpretation biases specific to paranoia because it specifically measures the participants’ interpretation of the subject’s intention to harm and whether harm is occurring to them in the scenarios. For example, participants read, “After work, you enter the café for a cup of tea. You sit down and start enjoying your tea. Across the room, two people begin to . . . laugh” and are asked to rate the following possible interpretations: “the people are laughing at you” (paranoid interpretation) and “the people are laughing at their jokes” (nonparanoid interpretation).
Scrambled-sentences task (SST) (Wenzlaff, 1993)	Participants were requested to reorder a string of scrambled words to construct grammatically correct statements. Each string of words had the potential to be constructed into both positive and negative meanings. The proportion of negative versus positive sentences formed can be used to determine interpretation bias.	Specific to paranoia The version of the SST used in the studies included in this review captured interpretation biases specific to paranoia because it specifically measured the participants’ interpretations related to intention to harm and whether harm is occurring. For example, the scrambled sentence “hostile the policeman me friendly to was” could be unscrambled to “the policeman was hostile to me” (paranoid interpretation) or “the policeman was friendly to me” (nonparanoid interpretation).
Self-report measures
Ambiguous Intentions Hostility Questionnaire (AIHQ) (Combs et al., 2007)	The AIHQ is an attribution-style questionnaire composed of self-report scales regarding 15 hypothetical, negative situations. Five of these negative scenarios are ambiguous. The participants rate these situations on variety of scales	Specific to paranoia The hostility bias index was used in this review and captures interpretations related to intended harm to the reader. For example, after reading “You walk past a bunch of teenagers at a mall and you hear them start to laugh,” participants write down the reason why the other person/people acted in the stated way toward them. Responses are assessed by two independent raters coding the interpretation given from 1 (not at all hostile) to 5 (very hostile).
Cognitive Biases Questionnaire for Psychosis (Catastrophizing) (Peters et al., 2014)	The Cognitive Biases Questionnaire for psychosis is a 30-vignette task that aims to examine five types of cognitive biases in psychosis using self-report: “intentionalizing,” “catastrophizing,” “dichotomous thinking,” “jumping to conclusions,” and “emotional reasoning.” Fifteen vignettes were about anomalous perception, and 15 vignettes were about threatening situations. There were six vignettes examining each bias, with three examining anomalous perception and three examining threatening situations. Ratings are made by the interviewer with participants’ input, on a 3-point scale (1 = no bias, 2 = presence of bias with some qualification, 3 = presence of bias).	Not specific to paranoia Although designed for psychosis, detailed examination of the content of items shows that the content need not be specifically related to paranoid concerns. The biases captured by the questionnaire could be relevant to a wider range of psychological disorders. For the purposes of this review, the catastrophizing bias score was used because it most closely mapped onto our definition of interpretation bias.
Identification-of-intention vignettes (Turkat et al., 1990)	This task displays 16 vignettes, which consist of 30-s videos depicting short scenes enacted by university-age individuals. In each vignette, one actor is antagonistic to another actor, with one of four intentions: hostile, accidental, prosocial, or ambiguous. After watching each vignette, participants were asked to answer two questions. Question 1 was “Why did he react the way he did?” Question 2 was “How would you respond to the actions of the character?”	Specific to paranoia The task was considered specific to paranoia because each vignette had one possible interpretation reflecting hostile intent toward the reader, in line with our working definition of paranoia. The bias index used in the review captured the proportion of hostile interpretations indicated by response to Question 1.
Internal, Personal, and Situational Attributions Questionnaire (IPSAQ) (Kinderman & Bentall, 1996)	The IPSAQ is a 32-item self-report questionnaire with 16 positive and 16 negative social situations specifically developed in the context of paranoia. The participants are asked to identify a single most likely cause of the situations described and to then categorize this cause as being either internal (something to do with themselves), external–personal (something to do with another person/s), or external–situational (something to do with circumstances/chance). For those causes identified as being external, paranoia is associated with a tendency to attribute negative events to other people as opposed to chance, denoting the “personalizing bias.” For the purposes of this study, although the situations themselves are valenced, the cause of the situations is ambiguous, meaning the task was deemed appropriate as a measure of interpretation bias according to our definition.	Specific to paranoia For the purposes of this review, only the personalizing bias score (number of external personal attributions for negative events/total external attribution for negative events) was used. Having a greater personalizing bias was considered to reflect an interpretation bias specific to paranoia because (a) the cause of the negative events is inherently ambiguous and (b) the associated bias has been shown quite specifically to be characteristic of paranoia as opposed to other traits or vulnerabilities such as depression/anxiety. For example, the cause of the negative event “a friend betrayed the trust you had in her” is ambiguous but is more likely to be interpreted as being something about her than merely circumstances or chance by individuals with elevated levels of paranoia.
Perception-of-criminal-intent vignettes (Jack & Egan, 2016)	Ten neutral situational vignettes, designed to be realistic and mundane, were shown to the participant. Each vignette consists of a neutral situational statement (i.e., “There has been a violent incident. A man is leaning over the victim and has blood on his shirt”), a neutral instruction (i.e., “Based only on the above statement, use the scale to rate how much you agree with following statement”), and a possible conclusion (i.e., “The man has harmed the victim”). Participants were asked to rate their agreement with the conclusion on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree)	Specific to paranoia The task on perception of criminal intention vignettes was deemed a paranoia-specific task because of the specificity of the questions to criminal intent to harm.
Virtual-reality questionnaire (VR) (Freeman et al., 2003)	Fifteen self-report items that examine three domains of paranoia (five questions each): persecutory thoughts about the VR characters (i.e., “They were hostile towards me”), ideas of reference (i.e., “They were watching me”), and positive beliefs (i.e., “I felt very safe in their company”). Participants were asked to rate each statement on a 4-point scale from 0 (do not agree) to 3 (totally agree).	Specific to paranoia The VR questionnaire was deemed to be specific to paranoia because of the specificity of the questions to paranoid thoughts, including persecution and ideas of reference.

Only those aspects of the task relevant to the systematic review of interpretation bias are included in the descriptions.

Specificity to paranoia was assessed by consensus agreement of all authors. Each task was evaluated according to whether the stimulus materials or response options permitted interpretations that were indicative of paranoid concerns, using Freeman and Garety’s (2000) definition of paranoia, which states that the individual believes that harm is occurring or is going to occur to oneself and that the persecutor has an intention to harm.

Results

The search resulted in 4,335 PubMed articles, 4,534 Web of Science articles, 4,055 PsycINFO articles, and four articles from other sources (added from additional searches from a later date), which amounted to a sum of 16,928 results. After the 7,952 duplicates were removed, 9,336 articles remained, and their titles were screened for relevance to the search criteria (Fig. 1).

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart describing the study-selection process.

A total of 8,735 studies were excluded because (a) the study design was not prospective or experimental, (b) the study was published before 1990, (c) the study was not published in English, or (d) the study was irrelevant to the current review. Finally, the remaining 601 articles were screened using the abstract and then the full text, which left a total of 20 articles. The studies included and their relevant findings are summarized in Table 2.

Table 2.

Systematic Review Study Characteristics

Study	Design	Sample (n)	Measure of paranoia	Interpretation-bias task	Interpretation-bias task content	Relevant findings	Risk of bias assessment
General population
Combs & Penn (2004)	Cross-sectional	High subclinical paranoia university students (n = 29) Low subclinical paranoia university students (n = 31)	PS	IPSAQ^a	Real-life situation	No significant between-groups differences on IPSAQ externalizing bias, t(55) = 1.3, p = .18, or personalizing bias, t(55) = 1.3, p = .18	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Combs et al. (2013)	Cross-sectional	High subclinical paranoia university students (n = 26) Low subclinical paranoia university students (n = 31)	PS	AIHQ^a	Real-life situation	Higher nonclinical paranoia group reported higher perceived hostility of ambiguous social situations, t(55) = −4.3, p < .001, d = 1.00.	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Freeman et al. (2003)	Cross-sectional	University students and staff (n = 24)	PS	Interpretation of ambiguous virtual reality^a	Real-life situation	Persecutory thinking in virtual reality was not significantly correlated with the paranoia scale (r = .156).	Selection bias: fair Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Freeman et al. (2005)	Cross-sectional	University students and staff (n = 30)	PS	Interpretation of ambiguous virtual reality^a	Real-life situation	Persecutory thinking in virtual reality was significantly correlated with the paranoia scale (r = .55, p < .002).	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Freeman et al. (2008)	Cross-sectional	University students and local community (n = 200)	GPTS	Interpretation of ambiguous virtual reality^a	Real-life situation	Association between trait levels of paranoia and the occurrence of persecutory thinking in virtual reality (OR = 1.04, p = .001, 95% CI = [1.02–1.07])	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Green et al. (2011)	Cross-sectional	Paranoid attributing university students and local population (n = 49) Nonparanoid attributing university students and local population (n = 9)	PS GPTS	Interpretation of ambiguous laughter^b IPSAQ^a	Real-life situation	Higher mean PS (U = 128.5, p = .05) and GPTS (U = 94.0, p = .03) scores in paranoid attribution group Lower externalization bias scores of IPSAQ in paranoid attribution group (U = 70.5, p = .001)	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Jack & Egan (2016)	Cross-sectional	General population (n = 200)	PS	Perception-of-criminal-intent vignettes^b	Real-life situation	Paranoia correlated with threat overestimation (r = .321 95% CI = [.173, .502], p < .001).	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Savulich et al. (2015)	Cross-sectional	University students and local community (n = 70)	GPTS	SRT^b SST^b	Semantic ambiguity	Correlation present between SRT and GPTS (r = .33, 95% CI = [0.01–0.55], p < .006) and PS (r = .41, 95% CI = [0.19–0.63], p < .001). Low paranoid (n = 17) individuals tended to have more paranoid interpretation in SRT compared with high paranoid (n = 18) individuals, t(17) = 6.49, p < .001, d = 1.33.	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Turkat et al. (1990)	Cross-sectional	Paranoid personality university students (n = 18) Control university students (n = 18)	SCL90R	Identification-of-intention vignettes^b	Real-life situation	Paranoid participants more likely to identify ambiguous situations and hostile (p < .025)	Selection bias: poor Performance bias: good Detection bias: fair Attrition bias: good Reporting bias: good
Clinical population
An et al. (2010)	Cross-sectional	Patients with first episode of schizophrenia (n = 20) Ultra-high risk for psychosis (n = 24) Control participants (n = 39)	PS PANSS	AIHQ (Hostility)^a	Real-life situation	Significant correlation between paranoia scale and AIHQ hostility scores in ultra-high risk for psychosis group (r = .424, p < .05) and first-episode schizophrenic group (r = .573, p < .01) Significant difference (p = .037) in AIHQ hostility scores in patients with first episode of schizophrenia compared with control participants	Selection bias: fair Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Bratton et al. (2017)	Cross-sectional	Patients with schizophrenia (n = 27)	GPTS	AIHQ (Hostility)^a	Real-life situation	No significant correlation between GPTS and AIHQ (hostility) scores	Selection bias: fair Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Combs et al. (2009)	Cross-sectional	Psychiatric patients with persecutory delusions (n = 32) Psychiatric control participants (n = 28) Control participants (n = 50)	PS	AIHQ (Hostility)^a	Real-life situation	Significant difference (p < .05) in AIHQ hostility scores in psychiatric patients with persecutory delusions compared with psychiatric control participants and control participants	Selection bias: fair Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Couture et al. (2010)	Cross-sectional	Patients with schizophrenia (n = 44) High-functioning autistic patients (n = 36) Control participants (n = 41)	PANSS	ATT^a	Real-life situation	Patients with schizophrenia were significantly more likely to classify untrustworthy faces as positive (p = .032) compared with control participants.	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Daalman et al (2013)	Cross-sectional	Clinical voice-hearers (n = 72) Nonclinical voice-hearers (n = 72) Control participants (n = 72)	LSHS	CBQp (Catastrophizing)^a	Real-life situation	Significant difference, F(2, 213) = 28.06, p < .001, in catastrophizing score of CBQp in clinical voice-hearers compared with nonclinical voice-hearers and control participants	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Freeman et al. (2013)	Cross-sectional	Patients with psychosis (n = 130)	State Paranoia SAPS/SANS	Modified SRT^b	Semantic ambiguity	State paranoia was significantly correlated with threat anticipation (r = .21, p = .020).	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Holt et al. (2006)	Cross-sectional	Patients with delusional schizophrenia (n = 16) Patients with nondelusional schizophrenia (n = 16) Control participants (n = 16)	PANSS SAPS/SANS	Letter-string discrimination^a	Semantic ambiguity	Delusional patients were more likely to classify neutral words as unpleasant compared with nondelusional patients, t(30) = 2.00, p = .05, and control participants, t(30) = 2.68, p < .02.	Selection bias: good Performance bias: good Detection bias: fair Attrition bias: good Reporting bias: good
Lahera et al. (2015)	Cross-sectional	Patients with schizophrenia (n = 49) Patients with bipolar disorder (n = 49) Control participants (n = 50)	PANSS	AIHQ (Hostility)^a	Real-life situation	Significant correlation between PANSS total score and AIHQ hostility scores (r = .361, p < .016) Significant difference, F(2, 121) = 14.67, p < .000, in AIHQ hostility scores in patients with schizophrenia compared with control participants	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
Langdon et al. (2013)	Cross-sectional	Patients with psychosis (n = 23) Control participants (n = 19)	SAPS/SANS PS	IPSAQ^a	Real-life situation	No significant correlation between paranoia and IPSAQ externalizing bias (r = .05, p > .05) in patients and near-significant correlation between severity of persecutory delusions, as measured by SAPS, and IPSAQ externalizing bias (r = .406, p = .055) No significant differences between patients and control participants in IPSAQ externalizing bias score	Selection bias: fair Performance bias: good Detection bias: fair Attrition bias: good Reporting bias: good
Savulich et al. (2017)	Cross-sectional	Patients with paranoid schizophrenia (n = 32) Patients with nonparanoid schizophrenia (n = 29) Control participants (n = 29)	GPTS PS	SRT^b SST^b	Semantic ambiguity	Patients with paranoid schizophrenia (2.21, SD = 0.46) had more paranoid interpretations (p < .001) in SRT compared with control participants (1.73, SD = 0.48).	Selection bias: good Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good
White et al. (2016)	Cross-sectional	Patients with schizophrenia (n = 29) Control participants (n = 25)	PANSS	Gesture-interpretation task^b	Real-life situation	Patients with schizophrenia were more likely to judge that ambiguous gestural movements were directed to themselves compared with control participants, t(50.20) = 2.13, p = .038.	Selection bias: fair Performance bias: good Detection bias: good Attrition bias: good Reporting bias: good

Note: AIHQ = Ambiguous Intentions Hostility Questionnaire (Combs et al., 2007); ATT = abbreviated trustworthiness task (Adolphs et al., 1998; Couture et al., 2010); CBQp = Cognitive Biases Questionnaire for psychosis (Peters et al., 2014); GPTS = Paranoid Thoughts Scale (Green et al., 2008); IPSAQ = Internal, Personal, and Situational Attributions Questionnaire (Kinderman & Bentall, 1996); LSHS = Launay-Slade Hallucinations Scale (Larøi et al., 2004); PANSS = Positive and Negative Syndrome Scale (Kay et al., 1987); PS = Paranoia Scale (Fenigstein & Vanable, 1992); SANS = Scale for the Assessment of Negative Symptoms; SAPS = Scale for the Assessment of Positive Symptoms (Andreasen, 1984); SCL90R = Symptom Checklist-90 Revised (Derogatis & Savitz, 1999); SRT = similarity-rating task (Mathews & Mackintosh, 2000); SST = scrambled-sentences task (Wenzlaff, 1993).

Self-report task, rating, questionnaire, or introspective method.

Experimental task.

Participant characteristics

Nonclinical studies

Of the 20 studies included, nine studies examined interpretation bias in a nonclinical population. All studies used university students as their main participant source. In addition, Savulich et al. (2015) included participants from the local community, and Jack and Egan (2016) used social media for further recruitment. The mean age of the nonclinical participants was 30.8 years, and an average 39.1% of the total participants were men.

Clinical studies

A total of 11 studies examined the association between interpretation bias and psychotic symptoms using participants presenting with a clinical psychotic disorder. The studies’ groups of interest were categorized as either schizophrenia (seven studies) or psychosis/delusions (four studies). All of these studies used a form of clinical interview, such as the Structural Clinical Interview for the DSM-5 (SCID-5; First, Williams, Karg, & Spitzer, 2015), to confirm the diagnosis. However, only seven performed any screening of control participants to ensure absence of clinical diagnosis using either the SCID (n = 1), self-report measures (n = 6), or both. Across studies, there was a large variation in the nature of the control sample. For instance, Daalman, Sommer, Derks, and Peters (2013) used a sample of nonclinical participants experiencing auditory hallucinations in addition to control participants, and An et al. (2010) used a group at ultra-high risk for psychosis as an added experimental/control level. Furthermore, a total of three studies—Combs et al. (2009), Holt et al. (2006), and Savulich et al. (2015)—also recruited nonparanoid psychiatric control participants for additional comparison to the paranoid or delusional participants. Only two clinical studies, Freeman et al. (2013) and Bratton, O’Rourke, Tansey, and Hutton (2017), did not include a control group.

The mean age was 38.2 years for participants in experimental groups and 36.4 years for the participants in the control groups. All studies reported the gender distribution of the participants. On average, the experimental sample was 64.6% male, whereas the control sample was 48.2% male.

Measures of paranoid symptoms and interpretation bias

All nonclinical studies used a self-report measure of paranoia; the Paranoia Scale (Fenigstein & Vanable, 1992) was the most common (n = 7). For the purposes of the current study, any tasks resolving emotional ambiguity were deemed to measure interpretation bias. However, there is a vast literature on impaired recognition (Bortolon, Capdevielle, & Raffard, 2015; Kohler, Walker, Martin, Healey, & Moberg, 2010) and detection (Chen, Norton, Ongur, & Heckers, 2007) of facial expressions of emotion in patients with schizophrenia. These studies were excluded from the present review unless there was specific direct relevance to paranoia, for example involving the emotion of trust.

To assess the robustness of the method chosen to investigate interpretation bias, we classed tasks into one of two types: (a) experimental tasks or (b) self-report tasks, questionnaires, or introspective methods. Only two studies used an experimental method: Turkat, Keane, and Thompson-Pope (1990) used identification-of-intention vignettes, and Savulich et al. (2015) used the scrambled-sentences task (SST; Wenzlaff, 1993) and the similarity-rating test (SRT; Mathews & Mackintosh, 2000). In contrast, a total of six studies used self-report tasks, questionnaires, and introspective methods, including the Internal Personal and Situational Attributions Questionnaire (IPSAQ; Kinderman & Bentall, 1996), the Ambiguous Intentions Hostility Questionnaire (AIHQ; Combs, Penn, Wicher, & Waldheter 2007), individually designed vignette-based questionnaires, and self-report measures regarding virtual-reality experiences.

The IPSAQ has 32 items that describe 16 positive and 16 negative social situations in the second person. For each item, the respondent is required to write down a single, most likely, causal explanation for the situation described. The respondent is then required to categorize this cause as being either internal (something to do with the respondent), external–personal (something to do with another person or persons), or external–situational (something to do with circumstances or chance) by circling the appropriate choice.

Furthermore, personal external attributions for negative events have been found to be associated with paranoid ideation (Bentall, Kinderman, & Kaney, 1994). Although the situations themselves are valenced, the cause of the situations remains ambiguous, and thus we considered the task to be an appropriate measure of interpretation bias according to our definition. In our systematic review, we included only the measure of personalizing bias (n of external personal attributions for negative events divided by total external attribution for negative events).

A number of studies have measured interpretation biases using ambiguous images, such as emotional and neutral faces (Neta & Whalen, 2010; Schwarz, Wieser, Gerdes, Mühlberger, & Pauli, 2013; Yoon & Zinbarg, 2008). Facial expressions, in fact, can be considered to be inherently ambiguous, and individual traits such as social anxiety or paranoia have been shown to influence the perception and evaluation of social stimuli (Schoth & Liossi, 2017). One study, Green et al. (2011) used both self-report and experimental methods. These tasks are described in detail in Table 1.

In terms of statistical analysis, five studies performed a simple correlational analysis between the level of paranoia and interpretation bias, whereas in four studies, the measure of paranoia was used to create “mild” and “high” paranoia groups for experimental purposes. However, Savulich et al. (2015) used both methods.

In contrast, clinical studies tended to use measures of paranoia and psychosis that were interview based (n = 3), self-report (n = 4), or both (n = 4). The most commonly used interview measures were the Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, 1987; n = 4), the Scale for the Assessment of Negative Symptoms (SANS; Andreasen, 1983), and Scale for the Assessment of Positive Symptoms (SAPS; Andreasen, 1984; n = 3). The Paranoia Scale (Fenigstein & Vanable, 1992) was the most commonly used for self-report (n = 4). In line with nonclinical studies, studies including clinical populations used a wide variety of interpretation-bias measures: Three studies used experimental measures, including the SRT (Mathews & Mackintosh, 2000), SST (Wenzlaff, 1993), and a gesture-interpretation task (White, Borgan, Ralley, & Shergill, 2016); eight studies used self-report tasks, including the AIHQ (Combs et al., 2007; n = 4), IPSAQ (Kinderman & Bentall, 1996; n = 1), and individually designed tasks (n = 3).

A total of 10 studies were classified as being of good quality, and a further eight studies were classified as being of acceptable quality. The good-quality studies satisfied all the criteria for the risk of bias assessment described in the Supplemental Material available online. Meanwhile, the acceptable-quality studies mainly had a lack of control screening, one study lacked a sample description (Jack & Egan, 2016), and two studies used fewer than 20 participants per group (Holt et al., 2006; Langdon, Still, Connors, Ward, & Catts, 2013).

Only one study was deemed to be of poor quality (Turkat et al., 1990) because there were significant risks of selection bias and detection bias. The study failed to report inclusion and exclusion criteria, sample descriptions, and baseline results and had no evidence of screening for participants. In addition, regarding the detection bias, there was no description of statistical analysis, and there was a relatively small sample size (n = 36).

Overall study findings

Interpretation bias in clinical and nonclinical samples

Of the 20 included studies, 14 reported the mean values of interpretation bias in the paranoid and nonparanoid groups. In general, the studies included demonstrated a greater likelihood of interpretation bias being present in more paranoid individuals, whether it be in clinical patients or the nonclinical population. In studies conducted in clinical participants (10 studies), the mean difference in interpretation bias between groups was significant except in two studies—Langdon et al. (2013) and White et al. (2016). The Langdon et al. study included a sample of patients in the early stages of psychosis with more severe persecutory delusions who showed a more extreme tendency to avoid self-blame for negative events.

However, studies conducted in nonclinical participants (four studies) showed mixed results: Green et al. (2011) and Combs, Finn, Wohlfahrt, Penn, and Basso (2013) demonstrated significant differences; Savulich et al. (2015) demonstrated a significant difference in the SST but not the SRT; and Combs and Penn (2004) found no significant differences in interpretation-bias levels between paranoid and nonparanoid groups.

Association between interpretation bias and paranoia

A total of 11 studies reported a correlation between paranoia levels and interpretation bias, with the majority of studies (10) reporting significant correlations. Of the studies reporting significant results, the correlation coefficients ranged widely, from .156 to .573. However, two studies demonstrated a nonsignificant correlation; Bratton et al. (2017) found the Spearman’s correlation coefficient between the AIHQ hostility score and the GPTS to be .127, whereas Langdon et al. (2013) found a correlation of .05 and −.08 between paranoia scale values with IPSAQ scores of externalizing bias and personalizing bias, respectively, all ps > .333.

Meta-analysis

Analysis 1. Interpretation bias in clinical and nonclinical samples

In addition, we carried out a meta-analysis on the subset of 11 studies in which the mean interpretation-bias score was provided. The meta-analysis for nonclinical studies yielded a standardized mean difference (SMD) of 1.065 (95% CI = [0.28, 1.85], p = .008), which suggested that individuals with subthreshold paranoia had an interpretation-bias score of more than 1 SD above those without paranoia symptoms (Fig. 2). There was significant heterogeneity between this subgroup of studies, Q(3) = 18.31, p < .001, I² = 83.6%.

Fig. 2.

Forest plot of the interpretation bias in individuals with clinical and subclinical paranoia participants compared with control participants. The gray boxes show the effect estimates from the single studies, and the diamonds show the pooled result. The horizontal lines through the boxes and the blue diamonds illustrate the width of the confidence interval. The solid vertical line is the line of no effect (i.e., the position at which there is no difference between individuals with paranoia and the control group). The dashed vertical line represents the overall measure of effect. The percentage weight indicates the influence of an individual study on the pooled result. SMD = standardized mean difference.

SMD in interpretation-bias score between patients with psychosis and unaffected control participants was slightly lower, at 1.014 (95% CI = [0.506, 1.521], p < .001), which suggests that individuals with clinically relevant paranoia had an interpretation-bias score of 1 SD above those without such symptoms (Fig. 2). There was significant heterogeneity in effect size among studies, Q(6) = 37.32, p < .001, I² = 83.9%.

Outliers were defined as studies in which the 95% CI was outside the 95% CI of the pooled studies (on both sides of the confidence interval). In the first meta-analysis, one outlier was identified within the clinical population subgroup (Combs et al., 2009). After removal of the outlier, the overall effect size was still significant but dropped to SMD = 0.775 (95% CI = [0.48, 1.07]), and heterogeneity became nonsignificant, Q(5) = 8.89, p = .113, I² = 37.3%.

We conducted sensitivity analyses to quantify the extent of heterogeneity attributable to each type of interpretation-bias task. Results (see Fig. S1 in the Supplemental Material) show that studies using Likert scales as a measure of the interpretation bias have higher heterogeneity, Q(4) = 39.33, p < .001, I² = 89.8%, than do experimental tasks, Q(5) = 11.29, p = .046, I² = 55.7%, although both remain significant (> 50% and therefore classed as moderate to high according to Higgins et al., 2003).

In the metaregression analyses, there were no effects of population studied, gender, age, measure of paranoid symptoms, and interpretation-bias task (see Table S1 in the Supplemental Material). However, we found a moderating effect of age, meaning that interpretation bias increases significantly with age in individuals with paranoia compared with control participants. Egger’s test for the meta-analyses did not show evidence of significant publication bias (b = 3.50, p = .3869), which was also confirmed by the symmetry of the funnel plot (see Fig. S2 in the Supplemental Material).

Analysis 2. The association between interpretation bias and paranoid symptoms

Figure 3 shows summary data and a forest plot for the random-effects meta-analysis of 11 studies that reported a correlation between interpretation bias and paranoid symptoms in patients. Effect sizes (r) ranged from −.08 to .68. Our analysis showed an overall effect size (r) of .350 (95% CI = [.273, .426]; p < .001), indicating a moderate positive correlation between interpretation bias and severity of paranoid symptoms. Analyses were also stratified according to the population (clinical and nonclinical studies). Results from the meta-analysis showed moderate effect sizes for studies including nonclinical (r = .319; 95% CI = [.21, .43]; p < .001) as well as studies including clinical samples (r = .3789; 95% CI = [.272, .484]; p < .001). Heterogeneity was significant among the clinical studies, Q(5) = 20.09, p = .001, I² = 75.1%, but not among the nonclinical ones, Q(3) = 3.40, p = .334, I² = 11.8%.

Fig. 3.

Forest plot of the correlation between interpretation bias and paranoia. Summary meta-analysis was done with a Fisher z transformation of the correlation coefficients. The gray boxes show the effect estimates from the single studies, and the diamonds show the pooled result. The horizontal lines through the boxes and the blue diamonds illustrate the width of the confidence interval. The solid vertical line is the line of no effect (i.e., the position at which there is no difference between individuals with paranoia and the control group). The dashed vertical line represents the overall measure of effect. The percentage weight indicates the influence of an individual study on the pooled result. ES = effect size.

Results for meta-analysis 2 (see Fig. S2 in the Supplemental Material) show lower heterogeneity for the studies using Likert scales as a measure of the interpretation bias, Q(2) = 3.66, p = .160, I² = 45.4%, compared with experimental tasks, Q(6) = 15.00, p = .020, I² = 60.0%. However, this could be explained by the low number of studies using Likert scales included in this meta-analysis, which prevented comparisons between the two subgroups.

Two outliers were identified within the clinical population subgroup (An et al., 2010; Combs et al., 2009). After removal of the outliers, the overall effect size (r) was slightly reduced to .27 (95% CI = [.19, .36]) but remained still significant, and overall heterogeneity became nonsignificant, Q(7) = 8.06, p = .328, I² = 13.1%.

Inspection of the funnel plot (see Fig. S2 in the Supplemental Material) and Egger’s test for the meta-analyses did not show evidence of significant publication bias (b = −1.04, p = .448).

Discussion

We conducted the first systematic reviews and meta-analyses on the association between interpretation bias and psychosis and paranoia. Our results showed a consistent pattern of interpretation bias associated with both subclinical paranoia and clinical psychosis, and meta-analyses showed moderate effect sizes.

The first meta-analysis examined the difference in mean interpretation-bias scores in nonclinical populations, comparing individuals with or without psychosis symptoms, and in clinical populations, comparing patients with psychosis with control participants. Both analyses demonstrated a greater mean interpretation-bias score, which suggests that an interpretation bias is present in people with psychotic symptoms. Furthermore, the magnitude of the effect was strong and statistically significant both in the nonclinical and the clinical subgroups, which suggests that an interpretation bias is present in individuals vulnerable to the disorder and that it persists at a clinical level of psychosis.

The second meta-analysis confirmed an association between interpretation bias and symptom severity in patients with psychosis, and the magnitude of interpretation bias increased with the severity of psychotic symptoms. This analysis was conducted to determine whether there was a positive relationship between the two variables, which could help in understanding the nature of their association. The majority of the studies included examined the correlation between paranoid symptoms and interpretation bias, except Henry, von Hippel, Ruffman, Perry, and Rendell (2010) and Lahera et al. (2015), who focused on schizophrenia spectrum disorder diagnoses.

These results are consistent with previous studies examining other cognitive processes involving biases in performance related to psychosis. For instance, a wide literature has demonstrated flawed data-gathering processes in psychotic individuals, termed the jumping-to-conclusions bias (So et al., 2016). Together, these and other cognitive phenomena are purported to contribute to symptom maintenance and are a central feature of cognitive models of psychosis (Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001). The results of the present study suggest that there is now sufficient empirical evidence specifically about biased interpretation of ambiguity associated with paranoia to allow preliminary testing of interventions specifically designed to target this bias (e.g., Yiend et al., 2017).

Persecutory delusions may be maintained by biases in the interpretation of emotional ambiguity, and there is increasing evidence consistently showing that bias exists before illness onset (Yiend et al., 2019; Savulich et al., 2020). Conducting further experimental studies to test causality to understand the direction of the association will be an important stepping-stone in the translational application of the findings of our present review.

A pilot study on clinical feasibility of cognitive bias modification for paranoia, which manipulates biased interpretations toward more adaptive processing to reduce paranoia in patients, is currently under way. Further research on recovered patients is also needed, although these designs have been criticized as tests of causality because successful treatments might also remove previously causal factors (Eysenck, Mogg, May, Richards, & Mathews, 1991) or causal factors may remain as latent vulnerabilities despite recovery (Teasdale, 1983).

Only a few studies included in our systematic review were not significantly consistent with these conclusions—Langdon et al. (2013), Combs and Penn (2004), and Bratton et al. (2017). However, Langdon et al. reported that patients with psychosis with significant paranoia did have greater interpretation bias, suggesting that the lack of significant correlation might be due to the measure of paranoid symptoms used in their sample, the Paranoia Scale (Fenigstein & Vanable, 1992), not adequately capturing clinical levels of paranoia. Likewise, Combs and Penn showed that participants with high subclinical paranoia had greater negative perceptions of the experimenter compared with the low subclinical group, indicative of interpretation bias. Finally, Bratton et al.’s nonsignificant findings might be due to the small sample size (n = 27) as well as to the characteristics of the participants included, all forensic patients who had committed a serious offense and were currently receiving secure inpatient care and therefore might present different cognitive mechanisms from the clinical samples included in the other studies.

Paranoia can be conceived as existing along a continuum. Consistent with this, epidemiological studies have estimated that psychotic experiences have an incidence of 2.5% and a prevalence of 7.2% in the general population (Linscott & van Os, 2013). Persecutory delusions have been described as the severest end of a spectrum of psychotic phenomena that occur across the general population (Freeman, Bentall, & Garety, 2008; Freeman, 2016). These are thought to be maintained by the co-occurrence of similarly continuous psychological processes, including worry, low sense of self, emotion dysregulation, and internal anomalous experiences (Freeman, Garety, Kuipers, Fowler, & Bebbington, 2002). Negative interpretation and “appraisal” of anomalous experiences (Bentall, Fernyhough, Morrison, Lewis, & Corcoran, 2007; Garety, Bebbington, Fowler, Freeman, & Kuipers, 2007) might play a similarly crucial role and is in turn associated with greater levels of distress and worries in paranoia (Brett, Heriot-Maitland, McGuire, & Peters, 2014; Lovatt, Mason, Brett, & Peters, 2010). A previous study showed, specifically, that interpretation bias changes incrementally with depression severity in a linear fashion (Lee et al., 2016), with the magnitude of negative interpretive bias (assessed across a range of measures) varying systematically according to severity of depression. Investigating the same question in a clinical sample with paranoia would be informative. A previous study also highlighted significantly attenuated positive interpretation biases in both “at risk mental state” and first episode psychosis compared with control participants (Yiend et al., 2019). Thus, findings to date are consistent with interpretative biases being present before the onset of psychosis and raise the possibility that they could contribute to its onset and development (Yiend et al., 2019).

Limitations

Results of this study highlighted how the literature currently available on interpretation bias in paranoia and psychosis is very limited. The inclusion and exclusion criteria of our systematic review were in fact lenient regarding the measures of paranoia and psychosis and the tasks measuring interpretation bias, which led to significant heterogeneity among the studies included. Consequently, with only nine nonclinical studies and 11 clinical studies included, it was difficult to solely analyze paranoia independent of psychosis. Likewise, the meta-analysis regarding the association between interpretation bias and psychosis had to combine both nonclinical and clinical samples and assume paranoia and psychosis as a single measure, all of which contributed to the heterogeneity in the effect size. However, effects of heterogeneity in measures of both interpretation bias and paranoia/psychosis were addressed with metaregression analyses and by conducting sensitivity analyses.

Furthermore, arising from the heterogeneity of the tasks, there are limitations that are common to the many studies included in the review. Self-report measures, for instance, used for measuring the level of paranoia/psychosis as well as the extent of interpretation bias can be misleading given their susceptibility to strategic or unintentional response biases as well as individual response variability in scoring (Austin, Deary, Gibson, McGregor, & Dent, 1998). Specifically regarding paranoia, self-report tasks may be inaccurate in clinical patients because a patient may not have enough insight to accurately answer. When self-report instruments are used to measure interpretation bias, despite their face validity, they are unlikely to give an accurate reflection of the actual cognitive processes involved in resolving ambiguity (Richards, 2004). Self-report measures are in fact associated with self-presentation and demand effects (assessing judgments or interpretations specific to self-referent descriptions rather than reactions to ambiguous events commonly experienced in daily life).

In addition, there were methodological limitations regarding the systematic review. First, there were only two reviewers examining the search results, and thus, the study is susceptible to subjective selection bias and risk of accidental study omission due to human error. To counteract this, the two reviewers discussed any uncertainty in the decision to include or exclude a study, and, if necessary, consulted one of the senior authors (J. Yiend). Finally, although there was no indication of publication bias in the findings of the meta-analysis, only published articles were included in the study—which may have led to both significant or insignificant results being excluded from the review.

Conclusions

This meta-analysis and systematic review suggests that there is a pattern of greater interpretation bias in paranoia. The results show a significant mean difference in both the subclinical and clinical subgroups, indicating the presence of a higher negative interpretation bias in the subgroups of individuals presenting with paranoid symptoms. In addition, we found evidence of a positive correlation between interpretation bias and paranoia in that the negative interpretation bias increased with the severity of paranoid symptoms. However, it is important to note the heterogeneity of the current data set, which could have provided false negative or positive results. Increasing evidence suggests that persecutory delusions are maintained by biases in the interpretation of emotional ambiguity. Therefore, conducting experimental studies to test causality to understand the direction of the association is a stepping-stone for the development of cost-effective, evidence-based psychological interventions targeting specific cognitive mechanisms underlying paranoia. With further research that examines interpretation bias in paranoia, the question regarding their association should be revisited in the future.

Supplemental Material

Trotta_Supplemental_Material – Supplemental material for Interpretation Bias in Paranoia: A Systematic Review and Meta-Analysis

Supplemental material, Trotta_Supplemental_Material for Interpretation Bias in Paranoia: A Systematic Review and Meta-Analysis by Antonella Trotta, Jungwoo Kang, Daniel Stahl and Jenny Yiend in Clinical Psychological Science

Footnotes

Transparency

Action Editor: Michael F. Pogue-Geile

Editor: Scott O. Lilienfeld

Author Contributions

A. Trotta developed the study concept. All the authors contributed to the study design. Data extraction and quality assessment were performed by J. Kang and A. Trotta. A. Trotta performed the data analysis and interpretation under the supervision of D. Stahl. J. Kang and A. Trotta drafted the manuscript, and J. Yiend and D. Stahl provided critical revisions. All the authors approved the final manuscript for submission. A. Trotta and J. Kang contributed equally to this work.

ORCID iD

Antonella Trotta

Supplemental Material

Additional supporting information can be found at

References

Adolphs

Tranel

Damasio

A. R.

(1998). The human amygdala in social judgment. Nature, 393, 470–474.

Agency for Healthcare Research and Quality. (2014). Methods guide for effectiveness and comparative effectiveness reviews (AHRQ Publication No. 10(14)-EHC063-EF). Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/cer-methods-guide_overview.pdf

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed). Arlington, VA: Author.

Amin

Foa

E. B.

Coles

M. E.

(1998). Negative interpretation bias in social phobia. Behaviour Research and Therapy, 36, 945–957.

S. K.

Kang

J. I.

Parks

J. Y.

Kim

K. R.

Lee

S. Y.

Lee

(2010). Attribution bias in ultra-high risk for psychosis and first-episode schizophrenia. Schizophrenia Research, 118, 54–61.

Andreasen

N. C.

(1983). The Scale for the Assessment of Negative Symptoms (SANS). Iowa City: University of Iowa.

Andreasen

N. C.

(1984). The Scale for the Assessment of Positive Symptoms (SAPS). Iowa City: The University of Iowa.

Austin

E. J.

Deary

I. J.

Gibson

G. J.

McGregor

M. J.

Dent

J. B.

(1998). Individual response spread in self-report scales: Personality correlations and consequences. Personality and Individual Differences, 24, 421–438.

Bebbington

P. E.

McBride

Steel

Kuipers

Radovanovic

Brugha

. . . Freeman

(2013). The structure of paranoia in the general population. The British Journal of Psychiatry, 202, 419–427.

10.

Beck

A. T.

Clark

D. A.

(1997). An information processing model of anxiety: Automatic and strategic processes. Behaviour Research and Therapy, 35, 49–58.

11.

Bentall

R. P.

Corcoran

Howard

Blackwood

Kinderman

(2001). Persecutory delusions: A review and theoretical integration. Clinical Psychology Review, 21, 1143–1192.

12.

Bentall

R. P.

Fernyhough

Morrison

A. P.

Lewis

Corcoran

(2007). Prospects for a cognitive-developmental account of psychotic experiences. The British Journal of Clinical Psychology, 46, 155–173.

13.

Bentall

R. P.

Kinderman

Kaney

(1994). The self, attributional processes and abnormal beliefs: Towards a model of persecutory delusions. Behaviour Research and Therapy, 32, 331–341.

14.

Blanchette

Richards

(2010). The influence of affect on higher level cognition: A review of research on interpretation, judgement, decision making and reasoning. Cognition & Emotion, 24, 561–595.

15.

Bratton

O’Rourke

Tansey

Hutton

(2017). Social cognition and paranoia in forensic inpatients with schizophrenia: A cross-sectional study. Schizophrenia Research, 184, 96–102.

16.

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2009). Introduction to meta-analysis. Hoboken, NJ: Wiley. https://doi.org/10.1002/9780470743386.ch40.

17.

Bortolon

Capdevielle

Raffard

(2015). Face recognition in schizophrenia disorder: A comprehensive review of behavioral, neuroimaging and neurophysiological studies. Neuroscience & Biobehavioral Reviews, 53, 79–107.

18.

Brett

Heriot-Maitland

McGuire

Peters

(2014). Predictors of distress associated with psychotic-like anomalous experiences in clinical and non-clinical populations. The British Journal of Clinical Psychology, 53, 213–227.

19.

Chen

Norton

Ongur

Heckers

(2007). Inefficient face detection in schizophrenia. Schizophrenia Bulletin, 34, 367–374.

20.

Cohen

(1969). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.

21.

Combs

Penn

D. L.

(2004). The role of subclinical paranoia on social perception and behavior. Schizophrenia Research, 69, 93–104.

22.

Combs

D. R.

Finn

J. A.

Wohlfahrt

Penn

D. L.

Basso

M. R.

(2013). Social cognition and social functioning in nonclinical paranoia. Cognitive Neuropsychiatry, 18, 531–548.

23.

Combs

D. R.

Penn

D. L.

Michael

C. O.

Basso

M. R.

Wiedeman

Siebenmorgan

. . . Chapman

(2009). Perceptions of hostility by persons with and without persecutory delusions. Cognitive Neuropsychiatry, 14, 30–52.

24.

Combs

D. R.

Penn

D. L.

Wicher

Waldheter

(2007). The Ambiguous Intentions Hostility Questionnaire (AIHQ): A new measure for evaluating hostile social-cognitive biases in paranoia. Cognitive Neuropsychiatry, 12, 128–143.

25.

Couture

S. M.

Penn

D. L.

Losh

Adolphs

Hurley

Piven

(2010). Comparison of social cognitive functioning in schizophrenia and high functioning autism: More convergence than divergence. Psychological Medicine, 40, 569–579.

26.

Daalman

Sommer

I. E. C.

Derks

E. M.

Peters

E. R.

(2013). Cognitive biases and auditory verbal hallucinations in healthy and clinical individuals. Psychological Medicine, 43, 2339–2347.

27.

Derogatis

L. R.

Savitz

K. L.

(1999). The SCL-90-R, Brief Symptom Inventory, and Matching Clinical Rating Scales. In Maruish

M. E.

(Ed.), The use of psychological testing for treatment planning and outcomes assessment (pp. 679–724). Mahwah, NJ: Erlbaum.

28.

DerSimonian

Laird

(1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–188.

29.

Dudley

Taylor

Wickham

Hutton

(2016). Psychosis, delusions and the “jumping to conclusions” reasoning bias: A systematic review and meta-analysis. Schizophrenia Bulletin, 42, 652–665.

30.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629–634.

31.

Eysenck

M. W.

Mogg

May

Richards

Mathews

(1991). Bias in interpretation of ambiguous sentences related to threat in anxiety. Journal of Abnormal Psychology, 100, 144–150.

32.

Fenigstein

Vanable

P. A.

(1992). Paranoia and self-consciousness. Journal of Personality and Social Psychology, 62, 129–138.

33.

First

M. B.

Williams

J. B. W.

Karg

R. S.

Spitzer

R. L.

(2015). Structured Clinical Interview for DSM-5 Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association.

34.

Freeman

(2016). Persecutory delusions: A cognitive perspective on understanding and treatment. Lancet Psychiatry, 3, 685–692.

35.

Freeman

Bentall

Garety

P. A.

(2008). Persecutory delusions: Assessment, theory and treatment. Oxford, England: Oxford University Press.

36.

Freeman

Dunn

Fowler

Bebbington

Kuipers

Emsley

. . . Garety

(2013). Current paranoid thinking in patients with delusions: The presence of cognitive-affective biases. Schizophrenia Bulletin, 39, 1281–1287.

37.

Freeman

Garety

P. A.

Bebbington

Slater

Kuipers

Fowler

. . . Dunn

(2005). The psychology of persecutory ideation II: A virtual reality experimental study. The Journal of Nervous and Mental Disease, 193, 309–315.

38.

Freeman

Garety

P. A.

Kuipers

Fowler

Bebbington

P. E.

(2002). A cognitive model of persecutory delusions. British Journal of Clinical Psychology, 41, 331–347.

39.

Freeman

Pugh

Antley

Slater

Bebbington

Gittins

. . . Garety

(2008). Virtual reality study of paranoid thinking in the general population. The British Journal of Psychiatry, 192, 258–263.

40.

Freeman

Slater

Bebbington

P. E.

Garety

P. A.

Kuipers

Fowler

. . . Vinayagamoorthy

(2003). Can virtual reality be used to investigate persecutory ideation? The Journal of Nervous and Mental Disease, 191, 509–514.

41.

Garety

P. A.

Bebbington

Fowler

Freeman

Kuipers

(2007). Implications for neurobiological research of cognitive models of psychosis: A theoretical paper. Psychological Medicine, 37, 1377–1391.

42.

Garety

P. A.

Freeman

Jolley

Dunn

Bebbington

P. E.

Fowler

D. G.

. . . Dudley

(2005). Reasoning, emotions, and delusional conviction in psychosis. Journal of Abnormal Psychology, 114, 373–384.

43.

Garety

P. A.

Kuipers

Fowler

Freeman

Bebbington

P. E.

(2001). A cognitive model of the positive symptoms of psychosis. Psychological Medicine, 31, 189–195.

44.

Green

C. E. L.

Freeman

Kuipers

Bebbington

Fowler

Dunn

Garety

P. A.

(2008). Measuring ideas of persecution and social reference: The Green et al. Paranoid Thought Scales (GPTS). Psychological Medicine, 38, 101–111.

45.

Green

C. E. L.

Freeman

Kuipers

Bebbington

Fowler

Dunn

Garety

P. A.

(2011). Paranoid explanations of experience: A novel experimental study. Behavioural and Cognitive Psychotherapy, 39, 21–34.

46.

Hedges

L. V.

Olkin

(1985). Statistical methods for meta-analysis. London, England: Academic Press.

47.

Henry

J. D.

von Hippel

Ruffman

Perry

Rendell

P.G.

(2010). Threat perception in schizophrenia-spectrum disorders. Journal of the International Neuropsychological Society, 16, 805–812.

48.

Higgins

J. P. T.

Thompson

S. G.

Deeks

J. J.

Altman

D. G.

(2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.

49.

Hirsch

C. R.

Meeten

Krahé

Reeder

(2016). Resolving ambiguity in emotional disorders: The nature and role of interpretation biases. Annual Review of Clinical Psychology, 12, 281–305.

50.

Holt

D. J.

Titone

Long

L. S.

Goff

D. C.

Cather

Rauch

S. L.

. . . Kuperberg

G. R.

(2006). The misattribution of salience in delusional patients with schizophrenia. Schizophrenia Research, 83, 247–256.

51.

Jack

Egan

(2016). Paranoid thinking, cognitive bias and dangerous neighbourhoods: Implications for perception of threat and expectations of victimisation. International Journal of Social Psychiatry, 62, 123–132.

52.

Jones

E. B.

Sharpe

(2017). Cognitive bias modification: A review of meta-analyses. Journal of Affective Disorders, 223, 175–183.

53.

Kay

S. R.

Fiszbein

Opler

L. A.

(1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13, 261–276.

54.

Kinderman

Bentall

R. P.

(1996). A new measure of causal locus: The internal, personal and situational attributions questionnaire. Personality and Individual Differences, 20, 261–264.

55.

Kohler

C. G.

Walker

J. B.

Martin

E. A.

Healey

K. M.

Moberg

P. J.

(2010). Facial emotion perception in schizophrenia: A meta-analytic review. Schizophrenia Bulletin, 36, 1009–1019.

56.

Kuipers

Garety

Fowler

Freeman

Dunn

Bebbington

(2006). Cognitive, emotional, and social processes in psychosis: Refining cognitive behavioral therapy for persistent positive symptoms. Schizophrenia Bulletin, 32, S24–S31.

57.

Lahera

Herrera

Reinares

Benito

Rullas

González-Cases

Vieta

(2015). Hostile attributions in bipolar disorder and schizophrenia contribute to poor social functioning. Acta Psychiatrica Scandinavica, 131, 472–482.

58.

Langdon

Still

Connors

M. H.

Ward

P. B.

Catts

S. V.

(2013). Attributional biases, paranoia, and depression in early psychosis. British Journal of Clinical Psychology, 52, 408–423.

59.

Larøi

Marczewski

Van der Linden

(2004). Further evidence of the multi-dimensionality of hallucinatory predisposition: Factor structure of a modified version of the Launay-Slade Hallucinations Scale in a normal sample. European Psychiatry, 19, 15–20.

60.

Lee

J. S.

Mathews

Shergill

Yiend

(2016). Magnitude of negative interpretation bias depends on severity of depression. Behaviour Research and Therapy, 83, 26–34.

61.

Liberati

Altman

D. G.

Tetzlaff

Mulrow

Gøtzsche

P. C.

Ioannidis

J. P. A.

. . . Moher

(2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLOS Medicine, 339, Article b2700. doi:10.1136/bmj.b2700

62.

Linscott

R. J.

van Os

(2013). An updated and conservative systematic review and meta-analysis of epidemiological evidence on psychotic experiences in children and adults: On the pathway from proneness to persistence to dimensional expression across mental disorders. Psychological Medicine, 43, 1133–1149.

63.

Lovatt

Mason

Brett

Peters

(2010). Psychotic-like experiences, appraisals, and trauma. The Journal of Nervous and Mental Disease, 198, 813–819.

64.

Mathews

(1990). Why worry? The cognitive function of anxiety. Behaviour Research and Therapy, 28, 455–468.

65.

Mathews

Mackintosh

(2000). Induced emotional interpretation bias and anxiety. Journal of Abnormal Psychology, 109, 602–615.

66.

Mathews

MacLeod

(1994). Cognitive approaches to emotion and emotional disorders. Annual Review of Psychology, 45, 25–50.

67.

Moher

Liberati

Tetzlaff

Altman

D. G., &

PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62, 1006–1012.

68.

Murphy

Bentall

R. P.

Freeman

O’Rourke

Hutton

(2018). The paranoia as defence model of persecutory delusions: A systematic review and meta-analysis. Lancet Psychiatry, 5, 913–929.

69.

Neisser

(1967). Cognitive psychology. New York, NY: Appleton-Century-Crofts.

70.

Neta

Whalen

P. J.

(2010). The primacy of negative interpretations when resolving the valence of ambiguous facial expressions. Psychological Science, 21, 901–907.

71.

Peters

E. R.

Moritz

Schwannauer

Wiseman

Greenwood

K. E.

Scott

. . . Garety

P. A.

(2014). Cognitive Biases Questionnaire for psychosis. Schizophrenia Bulletin, 40, 300–313.

72.

Richards

(2004). Anxiety and the resolution of ambiguity. In Yiend

(Ed.), Cognition emotion, and psychopathology: Theoretical, empirical and clinical directions (pp. 130–148). New York, NY: Cambridge University Press.

73.

Savulich

Edwards

Assadi

Guven

Leathers-Smith

Shergill

Yiend

(2020). Biased interpretation in paranoia and its modification. Journal of Behavior Therapy and Experimental Psychiatry, 69, Article 101575. doi:10.1016/j.jbtep.2020.101575

74.

Savulich

Freeman

Shergill

Yiend

(2015). Interpretation biases in paranoia. Behavior Therapy, 46, 110–124.

75.

Savulich

Shergill

Yiend

(2012). Biased cognition in psychosis. Journal of Experimental Psychopathology, 3, 514–536.

76.

Savulich

Shergill

Yiend

(2017). Interpretation biases in clinical paranoia. Clinical Psychological Science, 5, 985–1000.

77.

Schoth

D. E.

Liossi

(2017). A systematic review of experimental paradigms for exploring biased interpretation of ambiguous information with emotional and neutral associations. Frontiers in Psychology, 8, Article 171. doi:10.3389/fpsyg.2017.00171

78.

Schwarz

K. A.

Wieser

M. J.

Gerdes

A. B.

Mühlberger

Pauli

(2013). Why are you looking like that? How the context influences evaluation and processing of human faces. Social Cognitive and Affective Neuroscience, 8, 438–445.

79.

S. H.

Siu

N. Y.

Wong

Chan

Garety

P. A.

(2016). Jumping to conclusions data-gathering bias in psychosis and other psychiatric disorders. Two meta-analyses of comparisons between patients and healthy individuals. Clinical Psychology Review, 46, 151–167.

80.

Steinman

S. A.

Teachman

B. A.

(2010). Modifying interpretations among individuals high in anxiety sensitivity. Journal of Anxiety Disorders, 24, 71–78.

81.

Teasdale

J. D.

(1983). Negative thinking in depression: Cause, effect or reciprocal relationship? Advances in Behaviour Research and Therapy, 5, 3–26.

82.

Turkat

I. D.

Keane

S. P.

Thompson-Pope

S. K.

(1990). Social processing errors among paranoid personalities. Journal of Psychopathology and Behavioral Assessment, 12, 263–269.

83.

Wenzlaff

R. M.

(1993). The mental control of depression: Psychological obstacles to emotional well-being. In Wegner

D. M.

Pennebaker

J. W.

(Eds.), Century psychology series. Handbook of mental control (pp. 239–257). Englewood Cliffs, NJ: Prentice-Hall.

84.

White

T. P.

Borgan

Ralley

Shergill

S. S.

(2016). You looking at me?: Interpreting social cues in schizophrenia. Psychological Medicine, 46, 149–160.

85.

World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva, Switzerland: Author.

86.

Yiend

Allen

Lopez

Falkenburg

Tseng

McGuire

(2019). Interpretation biases precede the onset of psychosis. Behavior Therapy, 50, 718–731.

87.

Yiend

Barnicot

Williams

Fox

(2018). The influence of positive and negative affect on emotional attention. Journal of Behavior Therapy and Experimental Psychiatry, 61, 80–86.

88.

Yiend

Trotta

Meek

Dzafic

Baldus

Crane

. . . Peters

(2017). Cognitive Bias Modification for paranoia (CBM-pa): Study protocol for a randomised controlled trial. Trials, 18, Article 298. doi:10.1186/s13063-017-2037-x

89.

Yoon

K. L.

Zinbarg

R. E.

(2008). Interpreting neutral faces as threatening is a default mode for socially anxious individuals. Journal of Abnormal Psychology, 117, 680–685.

90.

Zavos

H. M.

Freeman

Haworth

C. M.

McGuire

Plomin

Cardno

A. G.

Ronald

(2014). Consistent etiology of severe, frequent psychotic experiences and milder, less frequent manifestations: A twin study of specific psychotic experiences in adolescence. JAMA Psychiatry, 71, 1049–1057.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB