Abstract
Between 47 and 86% of injecting drug users (IDUs) have co-occurring lifetime mental health disorders [1–3]. Depressive and anxiety disorders are the most common and have been associated with poorer functional and clinical outcomes. Although prompt identification is an important treatment goal, relatively few studies have explored the reliability and validity of mental health screening tools among individuals with substance use disorders (SUD) [4–6]. The current study aimed to assess the psychometric properties of the Kessler 10 (K10) [7] and the Patient Health Questionnaire (PHQ) [8] in an IDU sample, as well as determine which measure is a better predictor of a current DSM-IV depressive and/or anxiety disorder in this population.
Method
Participants
Participants were 103 IDUs, aged 16 and over, with a current SUD accessing a needle and syringe programme (NSP) in Melbourne, Australia, between February and July 2004.
Measures
Mental health screening tools included the K10 [7], a 10-item measure of psychological distress in the past month, which reliably predicts the presence of a current depressive or anxiety disorder at a cut-off score of 17 [9]. The mood and anxiety disorder sections of the PHQ were also completed to diagnostically assess for the presence of these disorders [8].
The Mini International Neuropsychiatric Interview (MINI) [10] was used to identify current DSM-IV diagnoses of depression, anxiety, psychotic and substance use (most frequently used substance) disorders. The MINI has high levels of reliability and good concordance with gold standard structured diagnostic measures [10, 11]. Substance use was assessed using the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) [12].
Procedure
Ethics approval for the study was obtained from the local research and ethics committee. Recruitment took place approximately 20 h per week. Research interviewers had honours degrees in psychology.
Twenty-one interviews (20.4%) were rated for interrater reliability purposes. Cohen's kappa values of 0.61 and 0.64 were obtained for MINI diagnosis of any affective disorder and psychotic disorders, respectively.
Results
Sample characteristics
The NSP has approximately 2221 client contacts (repeated contacts counted) each month. One hundred and three (23.2%) of the 443 NSP clients approached agreed to participate and completed the interview.
The sample was predominantly male (n = 75, 72.8%), with a mean age of 33.6 years (SD = 8.83; Table 1). All participants met DSM-IV criteria for a current SUD, and 25.2% (n = 26) had a concurrent alcohol use disorder. Almost 70% (n = 69) of participants met DSM-IV criteria for a current affective disorder (any anxiety and/or depressive disorder). The mean K10 score was 28.53 (SD = 8.89; min = 10, max = 49).
Demographic, substance use and clinical characteristics
MINI, Mini International Neuropsychiatric Interview; PHQ, Patient Health Questionnaire.
Reliability and validity of the K10 and PHQ for detecting an affective disorder
The K10 had high internal consistency, with a Cronbach's alpha of 0.84. A cut-off of 27 had the highest χ2 value (χ2=25.72) and the best trade-off across sensitivity and specificity, correctly classifying 78% of cases and 74% of non-cases. The area under the Receiver Operating Characteristic (ROC) curve was 0.86. The PHQ-9 depression scale had high internal consistency (Cronbach's α = 0.89), while the panic (KR-20 = 0.73), and other anxiety symptom scales (KR-20 = 0.67) of the PHQ had moderate levels.
A series of logistic regressions was then conducted to determine if the K10 and PHQ (any affective disorder) were predictive of the presence of a DSM-IV affective disorder. The PHQ (χ2(1) = 32.43, p < 0.005), and the K10 both as a continuous (χ2(1) = 40.25, p < 0.005) and categorical measure (K10 ≥ 27; χ2(1) = 26.06, p < 0.005), significantly predicted the presence of a current affective disorder when they were entered individually into the analysis. The K10 (≥27) had an overall predictive accuracy of 76.7% and accounted for 31.1% (i.e. Nagelkerke R2=0.311) of the variance. The PHQ correctly classified 78.6% of cases and accounted for 37.6% of the variance. Individuals with a positive screen on the K10 were 10 times more likely to have an affective disorder, while those with a positive PHQ screen had nearly 14 times the risk.
Both the K10 and PHQ remained significant predictors of an affective disorder when the measures were entered into a hierarchical logistic regression analysis. Together these measures accounted for 43.8% of the variance and had an overall predictive accuracy of 76.5%. Results were identical when the order of entry was reversed. A composite categorical variable consisting of a positive screen on the K10 and PHQ was also a significant predictor of a DSM-IV affective disorder (χ2(1) = 44.49, p < 0.005), correctly classifying 78.6% of cases, and accounted for 48.8% of the variance. Individuals with a positive screen on both measures were nearly 40 times more likely to have an affective disorder.
Discussion
The K10 demonstrated high levels of reliability and validity for detecting an affective disorder at a cut-off score of 27 within this population. While the K10′s specificity was moderate relative to its sensitivity, this is consistent with the findings of previous psychometric studies in addicted populations [4, 6]. The higher cut-off and mean K10 scores in the current study compared to other national surveys [7, 9], may reflect the high levels of psychological distress experienced by this population. Similar studies utilizing other screening tools (e.g. Beck Depression Inventory) in addicted subjects have also described higher cut-off scores [5].
The PHQ also showed promise as a screening tool, with moderate levels of internal consistency. Individuals with a positive screen on the K10 were 10 times more likely to have an affective disorder, while those with a positive PHQ screen had nearly 14 times the risk. While the combined use of the K10 and PHQ did not improve upon the overall predictive accuracy of either the K10 or PHQ alone, individuals with a positive screen on both measures were nearly 40 times more likely to have an affective disorder.
It is important to consider a number of limitations of the current study. While this is a relatively large sample of IDUs, only 23.2% of clients accessing the NSP were recruited, although multiple contacts by the same client may have been counted. Indeed, the participation rate and characteristics of the current sample are consistent with previous research studies for IDUs [1–3]. Finally, although the MINI is a reliable and valid diagnostic measure, its psychometric properties among substance users are yet to be fully determined. Nonetheless, the current sample size was more than sufficient to perform the necessary ROC analyses [13].
Both the K10 and PHQ appear to be valid and reliable screening tools for DSM-IV affective disorders in this population. We recommend the K10 and PHQ as brief screening and diagnostic tools for detecting these disorders within NSP and drug treatment settings.
Footnotes
Acknowledgements
The authors would like to acknowledge Lisa Wong for her assistance with the interrater reliability study. This study was completed with the assistance of funding from Perpetual Trustees, Flora and Frank Leith Charitable Trust, Jack Brockhoff Foundation, Grosvenor Settlement, Sunshine Foundation and Danks Trust.