Abstract
The need to provide psychosocial care for patients with cancer has been recognized [1–3], but depression and anxiety are frequently undetected by clinicians in busy cancer clinics [4–6]. Screening patients using validated measures combined with appropriate subsequent management has been recommended by peak bodies such as the US Preventive Services Task Force [7], the Canadian Preventive Services Task Force [8], National Institute for Clinical Excellence in the UK [9] and cancer-specific bodies including the National Comprehensive Cancer Network in the USA [3] and National Breast Cancer Centre in Australia [2].
Although a number of standardized questionnaires to detect common psychopathology such as depression and anxiety have been validated in cancer samples, there is no consensus that any existing questionnaire is ideal for screening purposes [10, 11]. One of the most widely used questionnaires in oncology is the Hospital Anxiety and Depression Scale (HADS) [11, 12]. One review identified 12 validations of the HADS against structured or semi-structured diagnostic interview in oncology samples [13]. The psychiatric diagnoses and cut-off points varied across studies, limiting the ability to generalize findings, but the majority of studies reported sensitivity and specificity of >0.75, which compares favourably with other screening tests [14]. Permission to use the HADS without charge for research purposes can generally be obtained, but in communication with the Australian publishers of the HADS (www.acer.edu.au) we were advised that free use of the HADS for routine screening in clinical practice was unlikely to be approved. Thus we sought to identify a comparable instrument that was freely available and validated in Australian samples for detection of anxiety and depression.
In choosing an instrument for comparison we were mindful of the need to reduce the number of items to a minimum. A UK study into the acceptability of screening for mood disorder among cancer professionals found that lack of time was the primary barrier to screening in clinical practice [15]. In that study the optimum practical screening method was considered to be between one and three verbally asked questions or a short screening questionnaire. A review of ultra-short (fewer than five items) screening instruments found that although they could be used to accurately rule out mood disorders (depression, anxiety and emotional distress) they were not accurate at identifying cases [15]. Thus there remains a need to identify scales that are shorter than established questionnaires but which have comparable performance.
One promising candidate is the PSYCH-6 subscale of the Somatic and Psychological Health Report (SPHERE-12) [16]. The SPHERE-12 was developed as a screening measure for use in Australian general practice to detect common mental disorders. The PSYCH-6 is a six-item psychological symptoms subscale that measures aspects of anxiety and depression and was derived from the General Health Questionnaire (GHQ). The other six items in the SPHERE-12, the SOMA-6 subscale, measure symptoms of chronic fatigue and are derived from the Schedule of Fatigue and Anergia [16].
Use of the SPHERE-12 has been reported in three oncology populations. First, a randomized controlled trial of aromatherapy for radiotherapy patients used both the SPHERE-12 and the HADS as outcome measures for levels of anxiety and depression [17]. The percentage of patients identified as cases at baseline on the PSYCH-6 (23%) was similar to that identified using the HADS subscales at a cut-off point of 7 (HADS Anxiety 27%, HADS Depression 18%).
The second study compared the SPHERE-12 with a fatigue measure (Functional Assessment of Cancer Therapy–Fatigue; FACT-F) in 109 breast cancer patients [18]. They found that SOMA-6 cases had significantly higher FACT-F fatigue scores than did non-SOMA-6 cases. Bennett
The third study to use the SPHERE was a prospective, longitudinal study of women following treatment for breast cancer [19]. Baseline measures were obtained from 176 women at an average of 10 months (range = 5–16 months) following adjuvant chemotherapy treatment. At baseline 30% of women met criteria on both the SOMA-6 and PSYCH-6 subscales; 19% were cases on the SOMA-6 scale only; 6% were cases on the PSYCH-6 scale only and 45% did not meet criteria on either subscale. Caseness at 48 months was estimated to be at least 14% of women who met criteria on both subscales; 4% were cases on the SOMA-6 scale only; and 6% were cases on the PSYCH-6 scale only.
Two of the three studies that have used the SPHERE in oncology populations have concurrently administered other, validated, instruments. Their published reports, however, have not included the measures of agreement, such as Cohen's kappa, positive agreement and negative agreement, which are appropriate to examine the level of agreement between the SPHERE and other instruments in oncology patients.
In summary, the PSYCH-6 scale of the SPHERE-12 is a short, freely available scale that has been validated in Australian general practice and has been successfully administered in oncology samples.
Aims
In this study we examined the concurrent validity of the PSYCH-6 with the HADS in an oncology outpatient sample. Three specific items were examined: (i) the percentage of patients identified as cases of anxiety and depression by the HADS-T and the PSYCH-6; (ii) general agreement between the two scales as continuous variables through correlation analysis; and (iii) measures of agreement between the two scales using dichotomous outcomes as required for clinical implementation. Three measures of agreement were calculated: Cohen's kappa, positive agreement, and negative agreement.
Methods
Subjects
All patients attending Medical Oncology, Radiation Oncology and Haematology outpatient clinics at the then Mater Misericordiae Hospital (now Calvary Mater Newcastle) in Newcastle, New South Wales, Australia during an 8 week period (April–May 2005) were eligible for the study with the exclusion of patients attending for the first time. This hospital provides comprehensive secondary and tertiary referral services for adults with malignant disease in Medical Oncology, Surgical Oncology, Radiation Oncology and Haematology. The project was approved by the Hunter Area Health Service Human Research Ethics Committee (Protocol No. 02/12/11/3.25).
Recruitment
In accordance with the requirements of the Ethics Committee, a three-stage recruitment procedure was followed. Eligible patients were given an information slip by reception staff. Patients who were interested in hearing more about the study kept the slip and were given more detail about the study, including a written information letter, by trained hospital volunteers or a member of the research team. Patients who then agreed to participate in the study provided written consent and completed the HADS and SPHERE-12 (and other measures not reported in this paper) on computer. The survey was conducted in the waiting room so that participants would not be concerned about missing their medical appointment, but there was sufficient privacy such that only the participant and the interviewer could see the participant's answers. The participant could elect to have their scored results and the interpretation of these scores sent in a letter to their treating oncology/haematology doctor.
Clinical information was extracted from participants’ medical records with their written consent by a qualified medical practitioner. Information collected included cancer type, stage, and current (at the time of interview) treatment with chemotherapy, radiotherapy or other therapy.
Measures
In the PSYCH-6 subscale of the SPHERE-12 patients were asked if they had been troubled by (i) feeling nervous or tense; (ii) feeling unhappy and depressed; (iii) feeling constantly under strain; (iv) everything getting on top of you; (v) losing confidence; and (vi) being unable to overcome difficulties [16]. Each item had three response options (never or some of the time; a good part of the time; most of the time), which were scored 0–2, respectively, resulting in a minimum score of zero and a maximum possible score of 12 on each subscale. Higher scores indicated more severe symptomatology. In a validation of the SPHERE-12 in general practice using a structured diagnostic interview [16], caseness for anxiety and depression was indicated by a score of ≥2 on the PSYCH-6 scale.
The HADS is a 14-item scale composed of two subscales containing seven questions on anxiety symptoms and seven questions on symptoms of depression [20]. Each item has four response options that are scored from 0 to 3, resulting in a minimum score of zero and a maximum possible score of 21 on each subscale. Higher scores indicate worse symptomatology. A total score on the HADS (HADS-T) of ≥15 was used to define possible cases [21]. This cut-off point was found to have sensitivity of 80% and specificity of 76% for a diagnosis of either generalized anxiety disorder or major depressive illness using a structured clinical interview (Psychiatric Assessment Schedule) in a sample of >500 English cancer patients [21].
Statistical analysis
SPSS version 14 (SPSS, Chicago, IL, USA) was used to calculate the correlation (Spearman's ρ for non-normally distributed data) between the HADS-T and the PSYCH-6 to determine how well the continuous measures were related.
Measures of agreement were calculated using the DAGStat program, which is freely available on the internet (www.mhri.edu.au/biostats/DAG_Stat) [22]. Cohen's kappa is a measure of agreement that is adjusted for chance [23]. Positive agreement is the number of positive ratings that both raters agree on divided by all the positive ratings for both raters [23]. Conversely, negative agreement is the number of negative ratings that both raters agree on divided by all the negative ratings for both raters [23]. Because clinical implementation requires a dichotomous outcome, measures of agreement were calculated at the recommended cut-off point for the PSYCH-6 (2 +) plus 1 point higher (3 +) and 1 point lower (1 +) to explore the optimum cut-off point for clinical implementation.
Results
Patient characteristics
Subject characteristics
The sample had a mean age of 60 years (SD = 12), ranging from 18 to 88 years and was 52% male (n = 177). Most participants (74%, n = 251) were married or living as married and 49% (n = 168) derived their income from a pension. Sixty-two per cent (n = 214) of participants were currently undergoing some kind of active treatment: chemotherapy for 36% (n = 122); radiotherapy for 24% (n = 83) and this number of patients (24%, n = 83) was also receiving ‘other’ therapy. Patients could be receiving more than one type of therapy.
The two main diagnosis groups were breast cancer (24%, n = 80) and haematological malignancies (24%, n = 82; Table 1).
Percentage of participants identified as cases on the HADS-T and PSYCH-6
Using the HADS-T, 29% (n = 100) of patients were classified as cases. On the PSYCH-6 (2 +) 33% (n = 112) scored above the cut-off point for the scale. At a cut-off point of 1+, 47% (n = 158) of patients were classified as cases, and at a cut-off point of 3+, 27% (n = 91) of patients were classified as cases.
General agreement between the two scales using correlation analysis
A statistically significant positive correlation was obtained between the PSYCH-6 and the HADS-T (Spearman's ρ = 0.741, p < 0.001).
Measures of agreement
Agreement between HADS-T and PSYCH-6 vs cut-off point
HADS-T, total score of the Hospital Anxiety and Depression Scale.
Discussion
This study found substantial agreement between the PSYCH-6 subscale of the SPHERE-12 and the HADS-T in oncology/haematology outpatients. Agreement between the scales was higher at a cut-off point of 3+ on the PSYCH-6 than at the previously published cut-off point of 2+, which had been established for a general practice sample using a structured interview for validation. We would conclude that it would be reasonable to use the PSYCH-6 at a cut-off point of 3+ as an equivalent instrument to the HADS-T for the identification of cases and non-cases of anxiety and depression in the present sample.
Strengths and limitations of the study
The strengths of the study include adequate sample size, involvement of an unselected oncology outpatient sample and the application of appropriate statistics for a cross-validation study of two questionnaires. Kappa, positive agreement and negative agreement are appropriate measure of agreement in studies such as this, in which no one instrument can be considered the gold standard. High positive agreement between raters using the same rating method is analogous to high sensitivity, in that the method could be expected to identify cases well [24]. High negative agreement between raters is analogous to high specificity in that the method could be expected to identify non-cases well [24]. This study found that the PSYCH-6 had somewhat greater agreement with the HADS-T in classification of non-cases than in classification of cases. This is analogous to somewhat higher specificity than sensitivity.
One limitation of the study is common to studies of this type and concerns the accuracy of the established, comparison measure. All questionnaires that purport to detect or diagnose psychological disorder will provide only an approximate measure of a person's mental state. Thus the conclusions of the present study regarding the ability of the PSYCH-6 to detect cases of anxiety and depression in cancer patients are only as valid as the ability of the HADS to correctly identify such cases. Because the purpose of this study was to determine whether a shorter tool such as the PSYCH-6 could be used instead of the 14-item HADS, the limitation raised here is not a major concern. Future research into the PSYCH-6 in this setting would ideally also compare this instrument with a validated structured interview or clinician interview in an oncology sample.
Another limitation to note is the low response rate. It is not known whether the sample accurately reflects the composition of the oncology outpatient population, and so whether the observed prevalence of caseness is an accurate estimate. Kappa is affected by prevalence rates in that low kappa can be obtained despite high percentage agreement between raters when the prevalence of a disorder is low [25]. Thus the findings of this study could not be presumed to apply to other samples. Therefore, some consideration of prevalence rates reported in other oncology samples using the PSYCH-6 may be useful to provide a context for interpretation.
Prevalence rates in other studies and effect on kappa
It is difficult to make direct comparison with the other oncology studies that used the PSYCH-6 because of differences between samples in terms of illness characteristics such as tumour site, stage, time since diagnosis, treatment type and time since treatment. The prevalence, however, of cases on the PSYCH-6 scale in the current study (at the published cut-off of 2 +) was similar to studies with breast cancer patients (33%, 36%, and 31%, respectively) [18, 19], most of whom had completed treatment. A somewhat lower prevalence of cases on the PSYCH-6 (23%) was reported in a study with patients of mixed cancer types (20% breast cancer) who were currently undergoing radiation therapy [17]. In the present study sample 24% of participants had breast cancer and 62% were currently receiving some kind of therapy. Because the prevalence rate in the present study was higher, we would anticipate that lower kappa estimates might obtained in the other studies.
Benefits of a shorter screening instrument
The cumulative time saving for patients and clinicians associated with a six-item scale, compared with a 14-item scale, becomes considerable in settings where routine screening involves thousands of patient visits, as would occur in any major cancer centre. A six-item scale taking 3 min to complete compared with a 14-item scale taking 7 min represents a time saving of 67 h per 1000 occasions of screening. At Calvary Mater Newcastle we provide thousands of occasions of service per year. For each 1000 occasions of screening the time difference between a 6-item and a 14-item scale equates to 1.7 weeks at 40 h per week. Furthermore, we anticipate that a reduced number of items might increase acceptability to patients of screening in clinical practice.
Although the effectiveness of screening has been questioned [26, 27], it has been shown that enhanced (collaborative) care for depression, in which system-level changes in health-care delivery are made, is effective in improving depression outcomes [26]. A recent meta-analysis of 14 studies (12 in primary care and two in general hospital settings) found that screening using questionnaires, if implemented alone, resulted in only marginal improvements in detection of depression and the use of any intervention for depression [27]. More specifically, no effect was found on prescription of antidepressants or depression outcomes and no difference in effect size was observed for study setting (primary care vs general hospital) [27].
Moreover, in the context of cancer care, it is recommended that screening and treatment for anxiety and depression take place within a broader framework of screening for distress arising from a range of sources including pain, fatigue and other physical symptoms along with social, spiritual and financial concerns [3]. In this context a tailored or tiered approach to intervention is considered appropriate: from information provision by the treating medical team, which is universally required, to therapy with specialist mental health professionals for the small but important minority of patients with significant mental health problems [28].
Conclusion
It has been acknowledged that no one instrument is perfectly suited to screening for anxiety and depression among oncology patients and that the optimum choice of instrument depends on the purpose for which it is to be used [10, 12]. This study has demonstrated substantial agreement between the HADS-T and the PSYCH-6 at cut-off points of 2+ and 3+. For oncology outpatients a cut-off score of 3+ on the PSYCH-6 scale is more suitable as a screening test for anxiety and depression (as measured by the HADS-T) than the cut-off point of 2+ proposed for general practice populations. Compared with the HADS-T the PSYCH-6 offers equivalent performance, fewer items and is a freely available instrument suitable for use in oncology populations.
Footnotes
Acknowledgements
The authors wish to thank Mr Stuart Allen for devising the computerized versions of the questionnaires and data support, Dr Heidi Reichler for extracting clinical information, staff and volunteers in the outpatient department for their support of the project, and the research assistants Ms Danielle Anthony, Ms Neta Moses and Mr Jason O'Connor.