Abstract
There are high rates of psychological morbidity in the general community [1]; and high rates of attendance at general practices by patients with psychological problems highlight the importance of understanding the recognition of mental illness in general practice [2]. However, defining ‘caseness’ is a major challenge in understanding recognition. Different groups may consider caseness differently, with patients considering caseness in terms of problems: general practitioners (GPs) in terms of management and researchers in terms of diagnostic classifications [3].
Different instruments have been used in studies of recognition of common mental disorders, including self-administered general screening questionnaires [4], disorder-specific scales [5–7] and interviewer or clinicianadministered schedules designed to detect well-defined psychological problems [3]. Physician definition of cases has been sourced from specific diagnoses recorded in case notes, from rating scales of physical and psychological severity completed during consultation with the patient [3], [8] and from physician interviews [6]. The interpretation of ‘physician opinion’ to define caseness has also varied, with some studies requiring agreement with various instruments at the level of specific diagnosis [9] and other studies accepting a broader definition of psychological and emotional complaints [10–13].
Implicit in comparisons between practitioner opinion and diagnostic instruments, such as the Composite International Diagnostic Interview (CIDI), has been the assumption that the instrument represents a ‘gold standard’ or provides a definitive diagnosis against which the GP's clinical opinion can be measured. Based on such comparisons it has been widely reported that primary care clinicians miss, or fail to identify as cases, a substantial proportion of patients [12], [14], [15]. This paper compares GP clinical opinion with instruments completed by, or administered to, patients to evaluate the differential levels of agreement.
Method
Data were collected as part of the cross-sectional phase of the Mental Health and General Practice Investigation (MaGPIe) study, a study of the prevalence, outcomes and management of common mental disorders in New Zealand general practice. Methods are described in detail elsewhere [2].
General practitioners were selected at random from a list of all 299 known GPs in a geographical area encompassing the administrative health districts around and between Wellington City and Palmerston North in the lower North Island of New Zealand, yielding a mix of urban, small town and rural practices. GPs were eligible to participate in the study if they were currently practising at least half-time without restriction (e.g. due to ill health or compulsory supervision).
The General Health Questionnaire (GHQ-12) was completed by a total of 3414 consecutive eligible adult patients, approximately 50 from the practice of each participating GP. Patients were eligible to complete the GHQ-12 if they were 18-years-old or over, read English well enough to understand and complete the GHQ-12 screening instrument and were about to consult the index GP for their own health concerns. The GP completed an Encounter Form for every patient aged 18 or over, whom they saw, over the study period. The Encounter Form included rating scales of the extent to which the presenting symptoms were physical or psychological and an assessment of the overall severity of the patient's physical and psychological disorders in the past 12 months.
Based on GHQ-12 strata, 8% of patients with scores of 0 or 1, 30% of patients with scores of 2–4 and all patients with scores of 5 or more were invited to participate in an in-depth interview and subsequent 12-month longitudinal study. The measures used in the in-depth interview were based on the World Health Organization's Collaborative Study of Psychological Problems in General Health Care [3] and included: a computerized interviewer-delivered version of the Composite International Diagnostic Interview (CIDI) version 2.1 (12-month version); the World Health Organization's Disability Assessment Schedule (WHODAS) version II [16] as an assessment of patient self-reported physical and psychological disability; the Somatic and Psychological Health Report (SPHERE-12) [17] a self-report screening tool to detect psychosomatic and psychological symptoms, included as a tool designed for use in general practice settings; as well, there was a range of socio-demographic questions. GPs completed a more detailed Patient Management Questionnaire about problems, history and current management for each patient selected for the in-depth interviews.
The Wellington and Manawatu-Whanganui Ethics Committees approved the methods and procedures used in the study.
Outcome measures
General practitioner recognition
There were three levels of GP recognition of common mental disorders; recognition of psychological symptoms only, recognition of clinically significant problems and explicit diagnosis [18]. ‘Levels’ of psychological problems in the past 12 months were defined using data from two sources: the GP's Encounter Form rating of severity of psychological disorder, and the GP's Patient Management Questionnaire about psychological disorders diagnosed in the past 12 months. GP recognition of ‘any psychological issue’ was defined as any report of psychological symptoms, distress or disorder whatsoever. ‘Clinically significant psychological symptoms’ required identification as a mild, moderate or severe case of psychological disorder from the Encounter Form, or reporting any definite psychological disorder on the Patient Management Questionnaire. Unclear responses to this question were resolved by a consensus of opinion by the study team clinicians. ‘Explicit psychiatric diagnosis’ was defined if the GP reported any definite-named psychological disorder on the Patient Management Questionnaire.
GHQ-12 cases
GHQ-12 caseness was identified as patients with a score of 5 or higher using the ‘0011’ scoring method [19].
CIDI cases
Data were scored using WHO scoring algorithms to produce DSM-IV diagnoses from CIDI v2.1. Data are reported in this paper for both 12-month and 1-month recency.
SPHERE cases
SPHERE cases are either psychosomatic (soma6) or psychological (psych6) cases or both using the scoring framework described by Hickie et al. [17] Psych6 cases were based on six psychological items and soma6 were based on six psychosomatic items.
WHODAS cases
Data were scored using a WHO-supplied scoring algorithm coded in Statisical Package for the Social Sciences syntax to produce a continuous variable between 0.00 and 99.99. Since no population norms were available, severity relative to the MaGPIe sample was estimated via quintile divisions among those with disability, after excluding: (i) those whose self-reported disability was due solely to physical causes with no psychological component; and (ii) those whose disability score was zero due to reporting no problems during the introductory questions of the WHODAS and hence skipping the section. Quintile 1 represented patients with the lowest levels of psychological disability and quintile 5 the highest.
Statistical methods
Statistical analyses were carried out using Statistical Analysis Software (SAS) version 8.2. Data were weighted to adjust for differences in probability of being sampled using the method of Kish [20]. Prevalence estimates were derived using the SAS procedure SURVEYMEANS, which adjusted rates and proportions for the probability of selection and standard errors and 95% confidence intervals (CI) for the effects of clustering among general practitioners.
Results
Seventy of the 78 (90%) randomly selected eligible general practitioners agreed to participate. GHQ screening questionnaires were completed by 3414 of 3687 (93%) eligible general practice attenders. Of the 1151 selected for interview based on the GHQ strata, 311 refused further contact, 19 became ineligible for the in-depth interview (because of limited language skills or worsening illness) and 33 were not traceable, yielding 788 interviews (70% response). Two interviews were lost after completion, leaving 786 interviews. Eleven patients did not consent to their data being linked with data from their GP, leaving 775 patients for inclusion in the analyses in this paper.
General practitioners recognized at least some symptoms of psychological disorders in the past 12 months in 56.4% (95% CI = 49.3–63.5) of patients (Table 1). Different levels of GP recognition have been described previously. Overall 35.5% (95% CI = 29.5–41.5) of patients had CIDI diagnosable disorders in the past 12 months and 17.9% (95% CI = 14.2–21.7) in the past month; 43.0% (95% CI = 37.7–48.3) reported psychosomatic symptoms, psychological symptoms, or both, in the SPHERE questionnaire and 40.1% (95% CI = 35.4–44.9) had some form of psychological disability arising from their symptoms as assessed by the WHODAS.
Comparison between GP identification of patients with psyc hological symptoms and identificationby different instruments
Overall, 49.6% (95% CI = 44.2–55.0) of patients were not identified by any of the GHQ-12, the CIDI or SPHERE-12 (Fig. 1) and 8.7% (95% CI = 7.1–10.2) were identified by all three instruments. Of the patients identified by at least one tool, only 17.2% (95% CI = 14.5–19.9) were identified by all three measures. GPs were more likely to give a specific diagnosis to patients identified by more than one tool and gave a specific diagnosis to 51.8% (95% CI = 42.6–60.9) of patients identified as cases by all three instruments. Only 14.9% (95% CI = 9.8–20.1) of patients identified by all three tools were not identified by the GP (Table 2). Among the patients identified by all three instruments, those with the highest levels of disability were most likely to be identified by GPs (Table 3). GPs recognized psychological symptoms for 79.5% (95% CI = 69.3–89.7) of patients with WHODAS scores in quintiles 4 and 5.
Percentage of general practice attenders identified by each of the GHQ-12, the SPHERE-12, CIDI − 1-month. Rates of identification are estimated by weighting the sample of 775 according to the probability of selection. 95% CI are adjusted for clustering within general practitioners. GPID represents the weighted proportion of patients identified by the instrument or combination of instruments who were also recognized as having at least some symptoms of psychological disorder by the GP they were consulting with.
Comparison between g eneral pr actitioner identification and patients identified by one instrument or combinations of instruments
Relationship between disability and g eneral pr actitioner identification of pr oblems in the last 12 months
Discussion
The study reported in this paper is part of a larger longitudinal study of the recognition and subsequent management of common mental disorders in general practice attenders: the MaGPIe study. The response rates of 90% from GPs and 70% from patients are sufficient to provide some assurance that the results obtained from this study are broadly representative of patients attending New Zealand general practice.
Comparison between screening and diagnostic instruments and the GP's clinical opinion of psychological disorder has demonstrated substantial variation between instruments and between clinical opinion and the screening and diagnostic instruments. Such variation is not unexpected as the instruments being compared were devised and included in the MaGPIe study, for different purposes. The GHQ-12 was developed as a screening instrument and has been widely used in case-finding in a range of different community and healthcare settings [3]. A high GHQ-12 score indicates the presence of symptoms of psychological distress or disorder, especially anxiety or depression [19]. The SPHERE was developed in Australia specifically to screen for common mental disorders in general practice attenders [17]. Underpinning the development of the SPHERE was recognition that patients with psychological morbidity present to general practice with a combination of somatic and psychological symptoms and may not meet the threshold for diagnostic classification systems.
The CIDI was developed for use by non-clinically qualified interviewers in large epidemiological surveys and is based on criteria for mental disorders defined by both the ICD-10 and DSM-IV. Although the CIDI is a useful marker of diagnosable mental disorder it has the virtue and drawback of being completed by an interviewer with no clinical expertise. It is far from a ‘gold standard’ of disorder, in spite of being treated as such by some researchers [15]. The CIDI has been found to have poor sensitivity when compared with clinical assessments of depression [21].
The use of the CIDI (a psychiatric model of ‘caseness’) may not be appropriate as a sole measure for identifying psychological morbidity in general practice attenders [22]. In the MaGPIe study while GPs identified psychological issues in 70.3% of patients with a CIDIdiagnosable disorder in the past month they also identified psychological issues for 53.4% of patients who did not have a CIDI diagnosis. In the primary care context it may be as important to offer interventions for these patients as well as those meeting the criteria for DSMIV diagnoses. In an exploration of ‘caseness’ in a primary care setting, Klinkman [22] concluded that family physicians appear to respond to meaningful clinical clues in assigning a diagnosis of depression and this is consistent with high rates of recognition of psychological problems in patients with psychological disability in the MaGPIe study.
Comparison of different ways of identifying general practice attenders with mental illness has highlighted the complexity of recognition of psychological symptoms, clinically significant disorder and diagnosis. In understanding rates of recognition of mental disorders by GPs, careful consideration needs to be given to the degree to which any single instrument can be utilized as a diagnostic ‘gold standard’. Further research is required to extend our understanding of the way primary care practitioners identify mental disorders and how this identification corresponds with treatment and outcomes.
In a clinical context, identification of a ‘case’ is central to care. However, there may be as many classifications as purposes [23]. In research it is important that the method of identification is consistent with the question posed. For example, if it is the extent to which GPs identify patients with mental disorders, then recognition of symptoms rather than application of a diagnostic label is important, and a screening tool such as the GHQ-12 or SPHERE may be more closely aligned to paradigms used in primary care. Similarly, if the question relates to identification and management of patients at the severe end of the spectrum, using the CIDI with WHODAS may be most appropriate. The absence of a ‘gold standard’ also has implications for longitudinal studies of primary mental healthcare. Differences in identification also need to be taken into account when considering outcome and change in caseness over time.
In conclusion, it is unlikely that a ‘gold standard’ for identification of common mental disorders will ever exist. Researchers and planners must take care in designing research and in evaluating primary care interventions to understand the complexity of the primary care diagnostic paradigm and the differences between primary and secondary mental health diagnosis and management.
Footnotes
Acknowledgements
We thank participating GPs and other practice staff, patients who participated and research staff. The Health Research Council of New Zealand funded the project (Project grant 99/065) with supplementary funds from the New Zealand Alcohol Advisory Council.