Abstract
The diagnosis of a specific mood or anxiety disorder using the criteria set out in the DSM-IV requires the use of extensive exclusion criteria. The DSM-IV commonly uses two types of exclusion criteria: those that are concerned with the hierarchical relationship between one mental disorder and another, and those that are concerned with whether a given mental disorder is due to the direct physiological effect of a substance or a general medical condition [1]. This paper will focus on the substance-induced and general medical condition exclusion criteria.
Large epidemiological surveys use interviews that apply the DSM-IV diagnostic criteria in order to determine caseness of the common mental disorders. It is important that cases are accurately defined otherwise future research programmes and clinical trials could face the problem of both false negatives and false positives. The use of substances and the experience of medical conditions are common in the general population and it follows that their impact (by virtue of the physical exclusion criteria) on the diagnosis of a mental disorder could influence the prevalence rate derived from epidemiological surveys. Indeed, in a Canadian sample Patten and Williams found that if every respondent who made a physical attribution to their depressive symptoms was automatically excluded from a diagnosis then the prevalence rate of lifetime major depression would decrease from 12.2% to 10.8% (a change of 13% of the base prevalence estimate) [2]. It is conceivable that previously published Australian prevalence estimates of the common mental disorders may likewise be influenced by automatically excluding patients with a medical or substance use (MSU) attribution for their psychiatric symptoms.
Patten and Williams used the World Mental Health version of the Composite International Diagnostic Interview (WMH-CIDI) to make a diagnostic assessment of the substance-induced and general medical condition exclusion criteria [2]. To date, the most cited prevalence estimates of the Australian population are derived from the National Survey of Mental Health and Well-being (NSMHWB) using the CIDI version 2.1 (CIDI 2.1) [3]. The CIDI 2.1 assesses the MSU exclusion criteria in a manner different to that of the WMH-CIDI. Goldney
Despite these criticisms the impact of the CIDI 2.1 on the MSU exclusion criteria has received relatively little attention. The purpose of the present study was to examine data from the NSMHWB and determine the impact of the MSU exclusion criteria when assessing prevalence estimates measured using the CIDI 2.1.
Method
Sample
The data are from the Australian NSMHWB. The sample was weighted to match the sex and age distribution of the Australian population. For more information on the design and sample refer to Andrews
Diagnostic assessment
The CIDI 2.1 was used to determine the 12 month prevalence of common DSM-IV mood and anxiety disorders, including; major depression, dysthymia, bipolar disorder, social phobia, obsessive–compulsive disorder (OCD), generalized anxiety disorder (GAD), panic disorder, and agoraphobia. In the course of the diagnostic interview the endorsement of key symptoms is then checked against the MSU exclusion criteria using an interview structure specific to the CIDI 2.1 known as the probe flow chart (PFC). The PFC consists of a series of questions that were designed to determine whether a psychiatric symptom was clinically significant and was not the result of substance use or a physical illness/injury, the MSU exclusion criteria.
The PFC is automatically initiated, by either interviewer instructions or a computer during a computer-assisted interview, when the psychiatric symptom is endorsed by the respondent. The respondent is then asked if they consulted a doctor or any other health professional about their symptom. If the respondent consulted a doctor or health professional they are probed further about the doctor's diagnosis, specifically if the doctor believed the symptom/s were the result of a substance or an illness/injury. The interviewer is given 11 options to choose from regarding the doctor's diagnosis of the respondent's symptom/s (nerves, stress, anxiety, depression, mental illness, no definite diagnosis, medication, drugs, alcohol, physical illness, and physical injury). If the respondent did not consult a doctor or mental health professional they are probed for their self-belief about what may have caused the psychiatric symptom. First, the interviewer asks if the symptom was ever the result of a substance or physical illness/injury; if the respondents says yes, the interviewer then asks if the symptom was always the result of a substance or physical illness/injury. See Figure 1 for a representation of the PFC and possible responses.
Simplified version of the probe flow chart containing the relevant questions to assess substance-induced and general medical condition exclusion criteria for one symptom of depression. The response to each question is followed by the appropriate interviewer instruction.
Each response to the PFC questions is coded and a series of computerized scoring algorithms systematically combine the responses, depending on the doctor's diagnosis and the respondent's self-belief that the symptom/s were always the result of a substance or illness/injury, in order to determine the final outcome. A code between 1 and 5 is given as the final outcome on the PFC as follows: 1, the symptom was not present; 2, the symptom was present but not clinically significant; 3, the symptom was present, clinically significant, but was always due to the use of medication/drugs/alcohol; 4, the symptom was present, clinically significant, but always due to either the use of medication/drugs/alcohol or a general medical condition; and 5, the symptom was present, clinically significant, and not always due to medication/drugs/alcohol or a general medical condition. The scoring algorithms for CIDI 2.1 [6] are designed in such a way that only symptoms assessed by the PFC with a code of 5 count toward the diagnostic decision of a given disorder. Therefore, the algorithms automatically exclude a respondent from meeting full criteria for a diagnosis if they have previously indicated that at least one psychiatric symptom is always the result of substance use or an illness/injury.
Data analysis
Eleven symptoms for eight common mental disorders (depression, dysthymia, mania, social phobia, agoraphobia, panic disorder, GAD, and OCD) that used the PFC to assess the MSU exclusion criteria were examined. First, the percentage of respondents who endorsed each symptom, regardless of the exclusion criteria, was calculated by selecting every respondent who scored a 2, 3, 4, or 5 on the PFC. Second, out of those people who endorsed the symptom we calculated the percentage of respondents coded as 3 or 4 on the PFC.
New 12 month DSM-IV prevalence estimates were generated without the MSU exclusion criteria operationalized, and compared to the standard DSM-IV prevalence estimates. It should be noted that the large amount of true negatives in the present data would have inflated the overall agreement rates, therefore Cohen's kappa was also calculated. Due to the skip system implemented by the CIDI 2.1, it was possible to calculate new prevalence rates only for dysthymia, social phobia, agoraphobia, panic disorder, and GAD. Data analysis was conducted using the SUDAAN software package [7] in order to account for the complex sampling design of the NSMHWB.
Results
The weighted percentages of respondents who endorsed each of the 11 symptoms and the outcome of probing are listed in Table 1. The highest symptom endorsement rate was for the key symptom of social phobia (16.2%), while the lowest symptom endorsement rate was for the key symptom of OCD (0.6%). Two symptoms, one relating to depression and the other relating to mania, were frequently (13.8% and 19.6%, respectively) attributed as always the result of either a substance or general medical condition. The attribution of symptoms to substances or general medical conditions was less common for symptoms relating to dysthymia, social phobia, agoraphobia, panic disorder, GAD, and OCD.
Endorsement rates for common mental disorder symptoms
†Percentages were generated from the total number of respondents who endorsed each psychiatric symptom (e.g. 7.6% of respondents who endorsed feeling sad, empty, or depressed indicated that their symptom was due to a substance or medical condition).
‡Actual unweighted numbers.
New prevalence estimates ignoring the physical exclusion criteria are presented in Table 2 along with the standard DSM-IV prevalence estimates. For the majority of mental disorders there is only a 0.1% absolute increase in the prevalence estimates when the MSU exclusion criteria are ignored. The relative impact of this 0.1% difference on the individual disorders is dependent on the base prevalence estimate. For instance, the difference between the prevalence estimates of social phobia is 0.1% of the general population, but this equates to a relative difference of 5.4% of the total number of respondents originally diagnosed with social phobia. For respondents diagnosed with dysthymia, the 0.1% difference in prevalence estimates equates to a relative difference of 9.7% of the base prevalence estimate. The relative impact of the substance-induced and general medical condition exclusion criteria on the base prevalence estimate of the disorders examined ranged from no change, for GAD, to an increase of 12%, for agoraphobia.
Impact of exclusions on prevalence of selected common mental disorders
CI, confidence interval. Relative difference = (prevalence if exclusions are ignored-standard prevalence)/standard prevalence.
†Actual unweighted numbers.
Assuming that the standard DSM-IV diagnosis can be considered the gold standard, the overall agreement rates showed ≥98% agreement between the standard prevalence and the prevalence if the MSU exclusion criteria are ignored. Dysthymia, social phobia, agoraphobia, panic disorder with agoraphobia, and panic disorder without agoraphobia all had a strong level of agreement, with κ> 0.9. The new prevalence rate for GAD had perfect agreement with the DSM-IV gold standard, at κ = 1.0.
Discussion
The present results demonstrate that on average 7% of survey respondents who endorsed a clinically significant psychiatric symptom, attributed the symptom(s) to an MSU etiology. The rate of attributions ranged across the disorders (1.6–19.6%), with OCD being least likely and mania being most likely to have an MSU attribution. This is perhaps not surprising considering the nature of the symptoms experienced in the two disorders. Previous evidence has shown that increased irritability is often a common side-effect of many medications and/or general medical conditions and can result in the diagnosis of secondary mania [8]. With regard to OCD, however, there is little substantial and consistent evidence for external (environmental) influences on the development of OCD symptomatology [9].
The formal diagnostic criteria of DSM-IV dictate that all respondents who endorsed physical attributions for their symptoms should be excluded from a final diagnosis. We assessed the impact of ignoring this exclusionary practice on the absolute prevalence estimates. On average there was a 0.1% increase in the absolute prevalence estimates across a range of mental disorders. Interpreting the results this way suggests that the absolute impact of the exclusion criteria on the standard prevalence of each disorder is minor. Contrary to the Goldney
One unexpected finding from the present study was that although 5.5% of respondents endorsed an MSU attribution for GAD symptoms, the absolute impact of ignoring the MSU exclusion criteria on GAD prevalence was zero. This observation led us to consider the proportion of patients who endorsed an MSU attribution who went on to be excluded due to other exclusionary criteria (which were still operationalized in the diagnostic algorithm). When examining the unweighted numbers, we found that for social phobia 24 out of 37 respondents who attributed a MSU cause to their symptoms went on to meet other exclusion criteria or were subthreshold cases. This was true for the 100% of patients with MSU attributions for GAD and 58% of patients with MSU attributions for dysthymia. In fact, this pattern was observed to some extent for all the disorders examined. This finding reinforces the notion that the absolute impact of the MSU exclusion criteria on the diagnosis of disorders should be viewed in the context of the other diagnostic inclusion and exclusion criteria.
The validity of the attributions offered by respondents is another important factor when considering the true impact of the MSU exclusion criteria on prevalence. Patten and Williams found that approximately 30% of MSU attributions for psychiatric symptoms offered by their sample were considered implausible by clinical experts [2]. The present data rely on the PFC to identify MSU attributions, but this method does not assess the plausibility of the respondent's MSU attribution. Instead, it automatically assumes that any MSU attributions provided by respondents are plausible reasons for exclusion. If the Patten and Williams estimates [2] were applicable to the present dataset and we had ignored the MSU exclusion criteria for 30% of the MSU attribution respondents who were likely to have given implausible attributions, the average absolute impact on prevalence may have been around 0.07%. The Patten and Williams estimates [2] apply only to the symptoms of depression, however, and we have no evidence that the estimates are applicable to the present sample. Further research is needed to formally assess attribution validity in an Australian sample across disorders.
Regardless of the validity of the attributions, what remains is that ignoring the MSU exclusion criteria has a minor impact on absolute prevalence of disorders in the present Australian sample. This raises the question of whether it is in fact necessary, useful or practical to operationalize the MSU exclusion criteria in epidemiological studies and exactly how it should be done. Option 1 is perhaps to abolish the assessment of these MSU exclusions in large surveys and accept slightly inflated prevalence rates. Option 2 may be to continue to assess the MSU exclusion criteria but adopt methods such as the CIDI PFC because they are relatively brief and inexpensive. The availability of the data at least allows the researcher to choose whether to operationalize the criteria in their scoring algorithms or not. One would, however, have to accept the lack of clinical validity of the MSU attributions sampled. Perhaps the best option is to adopt the WMH method of recording respondent attributions verbatim and having clinicians review responses later for validity. This approach is similar to one taken by Goodman
With the increasing propensity to record service contact, patient records, disease codes, and prescriptions electronically, data linkage projects may be a novel way to learn more about MSU attribution validity. Although large surveys in Australia are beginning to adopt data linkage (e.g. the 45 And Up study), like other National Mental Health Survey initiatives the recent Australian and New Zealand surveys have not. Although the cost involved with data linkage and the time lag in making service data available are somewhat prohibitive, the usefulness of this approach for mental health survey warrants exploration.
The present study contained two limitations that should be kept in mind when interpreting results. First, the present findings are dependent on the CIDI 2.1, which implements a skip system (meaning that all questions are not administered to all participants). Such skips meant that the present study was unable to examine the effect that the MSU exclusion criteria have on a final DSM-IV diagnosis of depression, mania (and therefore bipolar disorder), and OCD. The symptom endorsement rate for depression is one of the highest compared to the other common mental disorders, and DSM-IV acknowledges the potential for some depressive presentations to be caused by specific medical conditions (e.g. Parkinson's disease, stroke, hyper- and hypothyroidism) [1]. Therefore, future studies that examine the effect of the PFC on determining a final diagnosis of depression using a skip-free dataset are needed. Second, the present findings are specific to a sample of the Australian general population. The results may not generalize to different cultures or subsamples of the population, for instance, the elderly, samples of chronically ill individuals, or samples of clinical populations with a higher incidence of drug and alcohol abuse. Studies in these samples will be informative.
In conclusion, the present study found that MSU attributions to psychiatric symptoms were made by a small percentage of survey respondents (on average 7%). For the majority of common mood and anxiety disorders assessed by the CIDI 2.1, the MSU exclusion criteria have a minor absolute impact on determining case definition in the Australian general population. Improving efficient but valid assessment of the MSU exclusion criteria in epidemiological surveys is dependent on greater awareness by researchers of the complexity of operationalizing these criteria and efforts to study the impact more comprehensively and communicate results.
Footnotes
Acknowledgements
This study was supported by NHMRC programme grant no. 350833. The National Survey of Mental Health and Well-being was an initiative of, and funded by, the Australian Commonwealth Department of Health and Family Services as part of the National Mental Health Strategy.