Abstract
The Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project represents an integration of research methodology into a community-based outpatient practice affiliated with an academic medical centre. The MIDAS project is the largest clinical epidemiological study using semi-structured interviews to assess a wide range of psychiatric disorders in a general clinical outpatient practice. In an early report from the MIDAS project, we found that across diagnostic categories clinicians using unstandardized, unstructured clinical interviews underrecognized diagnostic comorbidity, compared with the results of semi-structured interviews. Moreover, we found that the patients often wanted treatment for symptoms of disorders that were diagnosed as comorbid, rather than principal, conditions. This highlighted the importance, from the patient’s perspective, of conducting thorough diagnostic interviews to diagnose disorders that are not related to the patient’s chief complaint because patients often desire treatment for these additional diagnoses. While several of the initial papers from the MIDAS project identified problems with the detection of comorbid disorders in clinical practice, regarding the diagnosis of bipolar disorder we observed the emergence of an opposite phenomenon—clinician overdiagnosis. The results from the MIDAS project, along with other studies of diagnosis in routine clinical practice, have brought to the forefront the problem with diagnosis in routine clinical practice. An important question is what do these findings suggest about the community standard of care in making psychiatric diagnoses, and whether and how the standard of care should be changed? The implications are discussed.
The impetus for initiating the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, in which research assessment methods would be incorporated into routine clinical practice, was my 7-year stint as a research assistant in the University of Iowa department of psychiatry during which time I participated in several studies requiring the administration of semi-structured diagnostic interviews. As a resident in psychiatry, and then as an attending, I found myself essentially administering a quasi-semistructured interview, though without the actual interview guide before me. Consequently, my evaluations were longer than those of my colleagues, and, on average, I made more diagnoses.
In the MIDAS project I formalized what I was doing clinically, and I expanded its scope beyond my own practice to our entire outpatient clinical group. That is, a structured initial diagnostic evaluation is administered to psychiatric outpatients presenting for treatment. An expanded version of the Structured Clinical Interview for the Diagnostic and Statistical Manual for Mental Disorders (DSM), Fourth Edition (SCID) 1 has thus far been administered to 3800 psychiatric outpatients. The Rhode Island MIDAS project thus represents an integration of research methodology into a community-based outpatient practice affiliated with an academic medical centre. Our group predominantly treats people with medical insurance (including Medicare but not Medicaid) on a fee-for-service basis, and it is distinct from the hospital’s outpatient residency training clinic that predominantly serves lower income, uninsured, and medical assistance patients. Data on referral source was recorded for the last 2000 patients enrolled in the study. Patients were most frequently referred from primary care physicians (29.7%), psychotherapists (17.4%), and family members or friends (17.7%). After the study was underway and running smoothly, the Structured Interview for DSM-IV Personality (SIDP-IV) 2 was introduced and more than 2000 patients were evaluated on DSM-IV, axis II. We then changed the methodology again and stopped administering the entire SIDP-IV, and instead only administered the borderline personality disorder (BPD) questions. From the outset we assumed that comprehensive structured interviews were unlikely to be incorporated into many other clinical practices; thus, we developed a self-administered questionnaire to screen for the most common DSM-IV axis I disorders diagnosed in outpatient settings. 3 –5 The goal was to develop a measure with good psychometric properties that could be incorporated into routine clinical practice.
Overview of the Methods of the MIDAS Project
Patients who call our practice are offered the option of receiving a standard clinical evaluation or a more comprehensive diagnostic interview. Patients are told that the comprehensive diagnostic interview lasts half a day. Patients are asked to arrive at 8:00 AM, are given some questionnaires to complete, and then are interviewed with the SCID, the SIDP-IV, and the Family History Research Diagnostic Criteria. 6
The diagnostic raters are highly trained and monitored throughout the project to minimize rater drift. After the interview, the diagnostic rater drafts a clinical report that is given to the treating psychiatrist before the psychiatrist meets with the patient. The report traces the history of present illness, describes the patients’ past psychiatric history including prior treatment efforts, and includes sections on medical history, family history, psychosocial history, and a mental status examination. The production of this comprehensive report has facilitated the success of the clinical research integration because clinicians directly benefit by having at their disposal a 6- to 12-page typed report before seeing the patient.
Evidence of Underdiagnosis and Underrecognition of Comorbidity in Routine Practice
An early report from the MIDAS project examined whether diagnostic comorbidity is less frequently identified by a routine clinical evaluation than a semi-structured diagnostic interview. 7 The recognition of comorbidity has important clinical significance. Comorbidity predicts poorer outcome for patients with depressive and anxiety disorders, and the presence of multiple psychiatric disorders is associated with greater levels of psychosocial impairment. Five hundred patients underwent a routine unstructured clinical interview. Subsequent to the ascertainment of the first sample, a second sample of 500 patients was collected though the people in this sample were interviewed with the SCID. Diagnoses were based on the DSM-IV criteria and followed the DSM-IV diagnostic hierarchies. The 2 groups had similar demographic characteristics and scored similarly on symptom questionnaires.
Most patients in the SCID sample were diagnosed with 2 or more disorders, compared with the minority of patients in the clinical sample (64.8%, compared with 36.6%, X2 = 79.5, df = 1, P < 0.001; OR 3.1, 95% CI 2.5 to 4.1). The relative difference in comorbidity rates between the SCID and clinical samples increased with increasing number of diagnoses (3 or more diagnoses: 36.0%, compared with 7.6%, X2 = 118.3, df = 1, P < 0.001; OR 6.3, 95% CI 4.7 to 10.0; 4 or more diagnoses: 17.6%, compared with 1.6%, X2 = 73.7, df = 1, P < 0.001; OR 13.1, 95% CI 6.3 to 27.4).
The data in Table 1 shows the difference between the clinical and SCID samples in prevalence rates of specific DSM-IV axis I disorders. Fifteen disorders were more frequently diagnosed in the SCID sample, and these differences cut across mood, anxiety, eating, somatoform, and impulse control disorder categories. Chronic psychotic disorders were infrequent in both patient series, and the SCID and clinicians diagnosed current substance use disorders (SUDs) with equal frequency.
Frequency of current DSM-IV axis I disorders in clinical and SCID samples, n = 500.
DSM = Diagnostic and Statistical Manual of Mental Disorders; SCID = Structured Clinical Interview for DSM-IV, NOS = not otherwise specified, n.s.= nonsignificant.
aFisher exact test.
bIn the SCID group, impulse control disorders were assessed in a subset of 409 people out of the full sample of 500 people.
While it may be the largest and first such study, the MIDAS project has not been the only one to examine the thoroughness of clinical diagnostic interviews. Shear et al 8 studied diagnostic accuracy in 2 community mental health centres, one in urban Pittsburgh and the other in rural western Pennsylvania. They interviewed 164 psychiatric outpatients with the SCID after they were evaluated clinically. More diagnoses were made on the SCID. More than one-third of patients were diagnosed with adjustment disorder by the clinicians, compared with only 7% by the SCID interviewers. Only 13% of the patients diagnosed by clinicians were given an anxiety disorder diagnosis whereas more than one-half (53%) of the patients interviewed with the SCID were diagnosed with a current anxiety disorder. One-half of the patients with a current primary diagnosis of major depressive disorder (MDD) on the SCID were diagnosed with adjustment disorder by clinicians. Shear et al 8 concluded that clinicians’ diagnoses are often inaccurate, and that this poses a barrier to the implementation of treatments that have proven effective for specific disorders.
In another study 9 of community mental health patients, this one conducted in Texas, psychiatric nurses administered the SCID to patients as a test of the use of research diagnostic procedures in clinical practice. They found that supplementing information from the patients’ charts with the information from the SCID resulted in more than 5 times as many comorbid conditions being diagnosed.
Miller et al 10 compared diagnoses of 56 psychiatric inpatients evaluated with the traditional diagnostic assessment, the SCID, and a computer-assisted diagnostic evaluation. Consistent with the other studies they found that diagnoses were missed by the unstructured clinical diagnostic evaluation compared with the computer-assisted interview.
In the MIDAS project we also examined underdiagnosis of specific disorders. One of the first reports from the MIDAS project focused on body dysmorphic disorder (BDD). 11 The underdiagnosis of BDD had been consistently described in case series and research reports. 12 –14 There are some studies of the prevalence of BDD in psychiatric patients; however, these studies were limited to patients with selected axis I disorders. The MIDAS project was the first to assess the presence of BDD in an unselected sample of patients presenting for treatment in an outpatient psychiatric setting. In a sample of 500 patients interviewed with the SCID, 16 (3.2%) patients were diagnosed with BDD. BDD was the principal diagnosis for 3 (0.6%) patients and an additional diagnosis for 13 (2.6%) patients. In a separate sample of 500 patients seen in the practice who were evaluated with a standard, unstructured clinical interview the prevalence of BDD was 0%.
In a report of the impact of research interviews on clinicians’ diagnostic practice we focused on the diagnosis of BPD. 15 We hypothesized that the diagnosis of BPD during the initial evaluation is influenced by the amount of information clinicians have available to them at the interview, and if clinicians are provided with information indicative of a diagnosis of BPD then the diagnosis will be made. Consistent with this we found that the frequency of BPD diagnoses assigned by clinicians increased more than 20-fold when the information from the SIDP-IV was presented to the clinicians before their evaluation (9.2%, compared with 0.4%; χ2 = 31.97, df = 1, P < 0.001).
In another publication we focused on the underdetection of anxiety disorders in patients with depression because of the high frequency of this comorbidity and the potential impact this comorbidity might have on treatment planning.
16
At the time of this analysis our sample size had grown and we compared the frequency of anxiety disorders in 610 patients given a principal diagnosis of nonbipolar MDD who were evaluated with an unstructured clinical interview and 300 patients also given a principal diagnosis of nonbipolar MDD who were evaluated with the SCID. More current anxiety disorders were diagnosed in the SCID than the non-SCID sample (1.0±1.1, compared with 0.3
Frequency of current DSM-IV anxiety disorders in patients with a principal diagnosis of major depressive disorder in clinical and SCID samples.
DSM = Diagnostic and Statistical Manual of Mental Disorders; SCID = Structured Clinical Interview for DSM-IV.
A Consumer’s Perspective on the Relevance of Detecting Comorbid Conditions
While information regarding diagnostic comorbidity has prognostic value, such information may not be immediately useful if patients have minimal interest and (or) willingness for treatment directed toward the comorbid conditions that are not the primary reason for seeking treatment. As part of our modification of the SCID, for all current disorders patients are asked if the symptoms of each diagnosed disorder were a reason (or one of the reasons) for seeking treatment. 17 Nearly all patients wanted treatment for their MDD, and more than 85% of patients with panic disorder, PTSD, and generalized anxiety disorder (GAD) indicated that the symptoms of these disorders were a reason for seeking treatment. Between one-half to two-thirds of patients with social phobia, obsessive–compulsive disorder (OCD), intermittent explosive disorder, BDD, and SUDs reported that the symptoms of these disorders were a reason for seeking treatment. Only 30% of people with specific phobia indicated that their phobic fears were a reason for seeking treatment.
We conducted a similar analysis in the study of anxiety disorders in patients with depression. Table 3 shows that the patients with depression evaluated with the SCID most often wanted treatment of their symptoms of GAD, panic disorder, and PTSD. One-half to two-thirds of patients wanted treatment of social phobia, OCD, and specific phobia. Overall, 86% of the patients with depression with at least one anxiety disorder wanted their treatment to address a comorbid anxiety disorder.
Desire for treatment for current DSM-IV comorbid anxiety disorders in SCID patients with a principal diagnosis of major depressive disorder.
DSM = Diagnostic and Statistical Manual of Mental Disorders; SCID = Structured Clinical Interview for DSM-IV.
Evidence of Overdiagnosis of Bipolar Disorder
While several of the initial papers from the MIDAS project identified problems with the detection of comorbid disorders in clinical practice, regarding the diagnosis of bipolar disorder (BD), over the years we observed the emergence of an opposite phenomenon—clinician overdiagnosis. That is, numerous patients presenting to our practice reported that they had been previously diagnosed with BD, yet a history of a manic or hypomanic episode was not elicited during an evaluation that included both a semi-structured interview and a clinical assessment.
To be sure, several research reports have suggested that BD is underrecognized, and that many patients, particularly those with MDD, have, in fact, BD. 18 –27 However, there is also evidence of overdiagnosis. 28,29
The largest study of overdiagnosis and underdiagnosis of BD was done in the MIDAS project. 30 Prior to the SCID interview, 700 patients completed a self-administered questionnaire which asked them whether they had been previously diagnosed by a health care professional with BD or manic depressive disorder. Family history information was obtained from the patients regarding their first-degree relatives. Diagnoses were blind to the results of the self-administered scale. Slightly more than 20% of the sample reported that they had been previously diagnosed with BD (n = 145, 20.7%), significantly higher than the 12.9% rate based on the SCID. More than one-half (56.6%, n = 82) of 145 patients who reported that they had been previously diagnosed with BD were not diagnosed with BD based on the SCID. Patients with SCID-diagnosed BD had a significantly higher morbid risk of BD than patients who self-reported a previous diagnosis of BD that was not confirmed by the SCID (Table 4). Patients who self-reported a previous diagnosis of BD that was not confirmed by the SCID did not have a significantly higher morbid risk for BD than the patients who were negative for BD by self-report and the SCID (Table 4). The results of the study suggested that BD is often overdiagnosed, and the family history analyses supported the validity of the diagnostic procedures. We also diagnosed BD in 27 patients who had not been previously diagnosed with BD. Thus, 3 times as many patients were overdiagnosed with BD than underdiagnosed (82, compared with 27).
Morbid risks for bipolar disorder in first-degree relatives of psychiatric outpatients who reportedly were previously diagnosed with bipolar disorder (BD) that was not confirmed by the Structured Clinical Interview for DSM-IV (SCID), patients diagnosed with BD based on the SCID, and patients without BD.
a90 probands.
b82 probands.
c528 probands.
dGroup A had higher morbid risk for BD than group B (χ2 = 6.35, df = 1, P < 0.02) and group C (χ2 = 27.32, df = 1, P < 0.001). There was no significant difference between groups B and C (χ2 = 1.21, df = 1, P = 0.27).
In a follow-up to our initial paper on BD overdiagnosis, we examined whether there was a particular diagnostic profile associated with BD overdiagnoses. 31 We compared the diagnostic profiles of the 82 patients who reported having been previously diagnosed with BD which was not confirmed when interviewed with the SCID to the 528 patients who were not diagnosed with BD. The patients overdiagnosed with BD were 4 times more likely to be diagnosed with BPD compared with patients who were not diagnosed with BD (24.4%, compared with 6.1%, P < 0.001). A previous diagnosis of BD also was associated with significantly higher lifetime rates of MDD, PTSD, impulse control disorders, and eating disorders, though only the association with impulse control disorders remained significant after controlling for the presence of BPD. This suggested that psychiatric outpatients overdiagnosed with BD were characterized by more axis I and axis II diagnostic comorbidity in general, and BPD in particular.
Bipolar Disorder Overdiagnosis: Validity or Unreliability?
The usual paradigm for determining whether a disorder is underdiagnosed is to recruit a cohort of patients without the disorder, re-evaluate them, and then demonstrate that some patients are then diagnosed with the disorder of interest. Analogously, to demonstrate a disorder is overdiagnosed one begins with a cohort of patients with the diagnosis, re-evaluate them, and then demonstrate that some patients are not diagnosed with the disorder. Because psychiatric diagnosis is not perfectly reliable it is to be expected that after a re-evaluation some patients who had been diagnosed with the disorder, or initially determined to not have the disorder, will be diagnosed differently. It is for this reason that it is important to examine validity after the re-evaluation. For example, in our study of BD overdiagnosis we used a family history of BD to validate our diagnostic procedure.
Unanswered Questions
The results from the MIDAS project, along with studies of diagnosis in routine clinical practice from Pittsburgh, Texas, and Los Angeles, have brought to the forefront the problem with diagnosis in routine clinical practice. Why is so much comorbidity being missed? We have speculated that the frequency of missed diagnoses, and maybe even misdiagnosis, is due to the insufficient amount of time allocated to the diagnostic examination. For patients with multiple diagnoses it takes a fair amount of time to confirm the presence of the necessary features to make each diagnosis. Since the initial evaluation also includes an assessment of prior treatments, medical history, family history, developmental history, and psychosocial history the clinician must prioritize how much time to spend in each of these areas. It is easy to limit one’s focus to the disorder associated with a patient’s chief complaint and ignore comorbid disorders that the patient does not spontaneously mention. With the increasing use of electronic health records, it will be important to study their impact on diagnostic practice. On the one hand, clinicians often express frustration with electronic medical records because they are less efficient and more time consuming. Conversely, protocols in electronic records can be set up to increase the thoroughness of the assessment and this might reduce problems with underdiagnosis, or with diagnostic accuracy more generally.
Diagnostic trends do not occur in a vacuum. Just as underdiagnosis might be a function of reduced reimbursement rates and pressures to conduct less time consuming assessments, during the past 2 decades there has been an increasing emphasis on biological treatment approaches. Related to this, fewer psychiatrists are doing therapy. 32 It is therefore not surprising that after years of concerns being raised with the underrecognition of BD 23,33 the pendulum has swung and overdiagnosis may now be a more frequent problem. And, as would be predicted given the superficial overlap in phenomenology, BD overdiagnosis is greatest in patients with BPD.
Are these findings any cause for alarm? No research has yet examined the clinical significance of the gap between researchers’ and clinicians’ diagnostic practices. Specifically, we are not aware of any studies that have addressed the important question of whether more accurate and comprehensive research diagnostic evaluations improve outcomes. In fact, one could argue that patients’ outcomes are not more likely to be worse, even if diagnoses are missed, because of the broad spectrum of activity of the new generation of medications. Medications, such as selective serotonin reuptake inhibitors and serotonin-norepinephrine reuptake inhibitors have been found to be effective for depression, almost all anxiety disorders, eating disorders, impulse control disorders, SUDs, attention-deficit hyperactivity disorder, and some somatoform disorders. Atypical antipsychotics are helpful in nonbipolar as well as bipolar depression, and there is some evidence of benefit in anxiety disorders. In short, most of the disorders for which people seek outpatient care have been found to be responsive to at least one of the new generation of antidepressants (ADs) or antipsychotics. Thus, it is possible that accurate and comprehensive diagnostic evaluations are not critical after gross diagnostic class distinctions (for example, psychotic disorder, compared with mood disorder) are made. This is consistent with the results of a survey of psychiatrists’ attitudes about DSM-III and DSM-III-R conducted more than 20 years ago. 34 In that survey only a minority of psychiatrists rated the DSMs as being very important for treatment planning, determining prognosis, patient management, and understanding patients’ problems.
Nevertheless, on common sense grounds it seems logical that greater diagnostic accuracy will improve outcome. If you are not aware of a comorbid condition’s presence, then isn’t it less likely to be successfully treated? If you are not aware that a patient has BPD rather than BD then wouldn’t a referral for evidence based psychotherapy be less likely? And wouldn’t overprescription of mood stabilizers with the accompanying increased side effect burden be more likely in the patient overdiagnosed with BD?
More complete and accurate diagnostic evaluations might influence whether or not a medication is prescribed (for example, an AD is more likely to be inappropriately prescribed if the patient is incorrectly diagnosed with MDD instead of adjustment disorder), choice of medication (for example, a selective serotonin reuptake inhibitor should be preferentially chosen for a patient with depression if a comorbid OCD is recognized), the number of medications prescribed (for example, a mood stabilizer should be added to an AD in a patient with depression with a history of manic episodes), and the prescription of psychotherapy (for example, cognitive-behavioural therapy is preferred to supportive therapy for a patient diagnosed with a specific anxiety disorder instead of adjustment disorder).
Whether or not improved diagnostic practice would result in improved outcome, it is important to recognize that diagnosis has more than one clinically relevant function. In addition to optimizing outcome, diagnosis is important for predicting treatment outcome. We would expect that a greater percentage of the variance in outcome would be predicted by comprehensive evaluations than by clinical diagnoses. Again, this is an unstudied question.
Throughout this paper I have discussed diagnosis from a categorical perspective. This is not to suggest that the categorical compared with the dimensional debate has been settled, or that the categorical approach is more valid than the dimensional approach. The boundaries between mental disorder and no disorder, and between disorders, are not sharp and well-demarcated, and the lack of distinct boundaries is likely responsible, in part, for high rates of diagnostic comorbidity. In the area of personality disorder research, compared with personality disorder categorical diagnoses, personality disorder dimensions are more reliable, stable over time, and account for more variance in measures of psychosocial morbidity. 35 The superiority of the dimensional representation of the personality disorders is not surprising because the transformation of a continuously distributed variable into a dichotomy sacrifices some information. Accordingly, a dimensional model for personality disorder classification was strongly considered for DSM-5, and a proposal was included in the Appendix. 36 Likewise, the National Institute of Mental Health Research Domain Criteria initiative is examining the validity of latent dimensions that cut across contemporary diagnostic categories. 37 Examination of the impact of changing a categorical to a dimensional approach toward classification on patient outcome will be a challenge for future researchers. Regarding diagnostic accuracy and error, the terms over- and underdiagnosis, which follow the categorical approach, will likely change to over- and underrating the severity of dimensional constructs.
For now, the psychiatric field continues to predominantly rely on the categorical approach toward diagnosis. In this context, the data documenting the problems with current clinical diagnostic practice are clear and consistent. Both overdiagnosis and underdiagnosis are problems, though the relative rates of each seem to vary by disorder. BD and adjustment disorder seem to be overdiagnosed. MDD might also be overdiagnosed, particularly in primary care settings where most primary care physicians report that they do not use the DSM criteria. 38 However, studies demonstrating the clinical benefit of state-of-the-art diagnostic practice have not yet been conducted. Consequently, it is premature to suggest that changes in the training of psychiatrists should be implemented in the absence of replicated research demonstrating the effect of less than optimal diagnostic performance on outcome.
In concluding, it is worth reflecting on the change in the discourse on diagnosis over the past 4 decades. DSM-III, the first officially sanctioned diagnostic system to incorporate specified inclusion and exclusion criteria for psychiatric diagnosis, was published only 35 years ago. A generation of psychiatrists has been trained to use specified diagnostic criteria. The empirical justification for the radical change in how psychiatric disorders were defined were the studies documenting problems with diagnostic reliability when diagnoses were based on earlier systems, 39 and other studies demonstrating that high levels of reliability could be achieved when diagnoses were derived from semi-structured interviews and based on specific criteria. 40 Validity was not so much an issue, except for some studies that demonstrated that the more narrow definition of schizophrenia proposed for DSM-III was more valid than DSM-II’s broader definition. 41 The clinical utility of the new diagnostic system was assumed, though improvement in patient outcomes was not the focus of attention.
During the past 3 decades we have witnessed a revolution in the treatment of psychiatric disorders. Pharmacotherapies and psychotherapies have been repeatedly demonstrated to be effective for a wide range of DSM-III, DSM-III-R, DSM-IV, or DSM-5 defined disorders. Consequently, it would seem more important now than 35 years ago that accurate diagnoses be made. The next generation of research on diagnosis (for example, changes in diagnostic criteria, or changes in diagnostic practice, understanding of latent dimensions) will hopefully attend to the most salient aspect of psychiatric treatment—the outcome of care. Depending on the results of these studies, the integration of the assessment methods of researchers into routine clinical practice, which we have done in the MIDAS project, might become common.
While a diagnostic determination is an important function of the intake evaluation, it is not its sole objective. Other integral functions of the intake evaluation include additional history taking (for example, past psychiatric history, prior treatment efforts, medical history, life events, social supports, coping style, family history, and developmental history), providing education about the disorder and treatment options, establishment of a therapeutic alliance, and identification of obstacles of treatment. Striving for improved diagnostic practice should not come with a cost of sacrificing the important details of a patient’s life story to make a diagnosis, or diagnoses. 42 It has been our experience during the past 20 years of the MIDAS project that the nondiagnostic functions of the initial evaluation are enhanced, rather than undermined, by good diagnostic practice and this has enabled us to successfully integrate the diagnostic assessment methods of researchers into our clinical practice.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.