Scores on the Patient Health Questionnaire–9 (PHQ-9) are frequently used to assess depression both in research and in clinical practice. The aim was to examine the validity of the PHQ-9 sum score by using Mokken scale analysis (Study I) and cognitive interviews (Study II) on the Icelandic version of PHQ-9. A primary care sample of 618 individuals was used in Study I. The results indicate that the PHQ-9 items are not close enough to perfectly unidimensional for their sum score to accurately order people on the depression severity dimension. In Study II, the sample consisted of 53 individuals, with 28 having a history of depression and 25 not. The findings reveal a number of issues concerning respondents’ use of the PHQ-9. No systematic differences were found in the results of the two groups. The PHQ-9 sum score should thus be interpreted and used with great care. We provide scale revision recommendations to improve the quality of PHQ-9.
AdlerM.HettaJ.IsacssonG.BrodinU. (2012). An item response theory evaluation of three depression assessment instruments in a clinical sample. BMC Medical Research Methodology, 12, Article 84. https://doi.org/10.1186/1471-2288-12-84
2.
AllenR. P.PicchiettiD.HeningW. A.TrenkwalderC.WaltersA. S.MontplaisiJ., & The Participants in the Restless Legs Syndrome Diagnosis and Epidemiology workshop at the National Institutes of Health in collaboration with members of the International Restless Legs Syndrome Study Group. (2003). Restless legs syndrome: Diagnostic criteria, special considerations, and epidemiology. A report from the restless legs syndrome diagnosis and epidemiology workshop at the National Institutes of Health. Sleep Medicine, 4(2), 101–119. https://doi.org/10.1016/S1389-9457(03)00010-8
3.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.).
AuvinenP.KoponenH.KautiainenH.KorniloffK.AhonenT.VanhalaM.MäntyselkäP. (2022). A longitudinal study of restless legs symptoms among patients with depression. Nordic Journal of Psychiatry, 76(8), 584–590. https://doi.org/10.1080/08039488.2022.2038265
6.
BeardC.HsuK. J.RifkinL. S.BuschA. B.BjörgvinssonT. (2016). Validation of the PHQ-9 in a psychiatric sample. Journal of Affective Disorders, 193, 267–273. https://doi.org/10.1016/j.jad.2015.12.075
7.
BernaardsC. A.SijtsmaK. (2000). Influence of imputation and EM methods on factor analysis when item nonresponse in questionnaire data is nonignorable. Multivariate Behavioral Research, 35(3), 321–364. https://doi.org/10.1207/S15327906MBR3503_03
8.
BianchiR.VerkuilenJ.TokerS.SchonfeldI. S.GerberM.BrählerE.KroenkeK. (2022). Is the PHQ-9 a unidimensional measure of depression? A 58,272-participant study. Psychological Assessment, 34(6), 595–603. https://doi.org/10.1037/pas0001124
9.
BoothroydL.DagnanD.MuncerS. (2018). Psychometric analysis of the Generalized Anxiety Disorder Scale and the Patient Health Questionnaire using Mokken scaling and confirmatory factor analysis. Health and Primary Care, 2(4), 1–4. https://doi.org/10.15761/HPC.1000145
CumbeV. F. J.MuanidoA.ManacaM. N.FumoH.ChirucaP.HicksL.de Jesus MariJ.WagenaarB. H. (2020). Validity and item response theory properties of the Patient Health Questionnaire-9 for primary care depression screening in Mozambique (PHQ-9-MZ). BMC Psychiatry, 20, Article 382. https://doi.org/10.1186/s12888-020-02772-0
12.
ElhaiJ. D.ContractorA. A.TamburrinoM.FineT. H.PrescottM. R.ShirleyE.ChanP. K.SlembarskiR.LiberzonI.GaleaS.CalabreseJ. R. (2012). The factor structure of major depression symptoms: A test of four competing models using the Patient Health Questionnaire-9. Psychiatry Research, 199(3), 169–173. https://doi.org/10.1016/j.psychres.2012.05.018
FowlerF. J. (1995). Improving survey questions: Design and evaluation. SAGE.
15.
FriedE. I.NesseR. M. (2015). Depression is not a consistent syndrome: An investigation of unique symptom patterns in the STAR* D study. Journal of Affective Disorders, 172, 96–102. https://doi.org/10.1016/j.jad.2014.10.010
Gonzáles-BlanchC.MedranoL. A.Muñoz-NavarroR.Ruíz-RodríguezP.MorianaJ. A.LimoneroJ. T.SchmitzF.Cano-VindelA. (2018). Factor structure and measurement invariance across various demographic groups and over time for the PHQ-9 in primary care patients in Spain. PLOS ONE, 13(2), Article e0193356. https://doi.org/10.1371/journal.pone.0193356
18.
GranilloM. T. (2012). Structure and function of the Patient Health Questionnaire-9 among Latina and non-Latina white female college students. Journal of the Society for Social Work and Research, 3(2), 80–93. https://doi.org/10.5243/jsswr.2012.6
19.
GriceP. (1989). Studies in the way of words. Harvard University Press.
20.
GunzlerD.SehgalA. R.KauffmanK.DaveyC. H.DolataJ.FigueroaM.HumlA.PencakJ.SajatovicM. (2020). Identify depressive phenotypes by applying RDOC domains to the PHQ-9. Psychiatry Research, 286, Article 112872. https://doi.org/10.1016/j.psychres.2020.112872
21.
HarringtonB. J.KlynL. L.RuegseggerL. M.ThomA.JumbeA. N.MaliwichiM.StocktonM. A.AkibaC. F.GoV.PenceB. W.MaselkoJ.GaynesB. N.MillerW. C.HosseinipourM. C. (2021). Locally contextualizing understandings of depression, the EPDS, and PHQ-9 among a sample of postpartum women living with HIV in Malawi. Journal of Affective Disorders, 281, 958–966. https://doi.org/10.1016/j.jad.2020.10.063
HubertM.VandervierenE. (2008). An adjusted boxplot for skewed distributions. Computational Statistics & Data Analysis, 52(12), 5186–5201. https://doi.org/10.1016/j.csda.2007.11.008
24.
KalpakjianC. Z.ToussaintL. L.AlbrightK. J.BombardierC. H.KrauseJ. K.TateD. G. (2009). Patient Health Questionnaire-9 in spinal cord injury: An examination of factor structure as related to gender. The Journal of Spinal Cord Medicine, 32(2), 147–156. https://doi.org/10.1080/10790268.2009.11760766
25.
KedingA.BöhnkeJ. R.CroudaceT. J.RichmondS. J.MacPhersonH. (2015). Validity of single item responses to short message service texts to monitor depression: An mHealth sub-study of the UK ACUDep trial. BMC Medical Research Methodology, 15, Article 56. https://doi.org/10.1186/s12874-015-0054-6
26.
KesslerR. C.BerglundP.DemlerO.JinR.KoretzD.MerikangasK. R.RushA. J.WaltersE. E.WangP. S. (2003). The epidemiology of major depressive disorder: Results from the national comorbidity survey replication (NCS-R). JAMA, 289(23), 3095–3105. https://doi.org/10.1001/jama.289.23.3095
27.
KesslerR. C.BrownR. L.BromanC. L. (1981). Sex differences in psychiatric help-seeking: Evidence from four large-scale surveys. Journal of Health and Social Behavior, 22(1), 49–64. https://doi.org/10.2307/2136367
28.
KeumB. T.MillerM. J.InkelasK. K. (2018). Testing the factor structure and measurement invariance of the PHQ-9 across racially diverse U.S. college students. Psychological Assessment, 30(8), 1096–1106. https://doi.org/10.1037/pas0000550
29.
KrauseJ. S.ReedK. S.McArdleJ. J. (2010). Factor structure and predictive validity of somatic and nonsomatic symptoms from the Patient Health Questionnaire-9: A longitudinal study after spinal cord injury. Archives of Physical Medicine and Rehabilitation, 91(8), 1218–1224. https://doi.org/10.1016/j.apmr.2010.04.015
30.
KroenkeK.SpitzerR. L.WilliamsJ. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
31.
KumarM.HuangK. Y.OthienoC.KagoyaM.NyongesaV.YatorO.MadegheB.UnutzerJ.SaxenaS.McKayM. (2021). Testing the EPDS and PHQ-9 among peripartum adolescents in Nairobi, Kenya: A mixed-methods study. Clinical Medicine Insights: Psychiatry, 12, 1–11. https://doi.org/10.1177/1179557321991093
32.
LoevingerJ. (1947). A systematic approach to the construction and evaluation of tests of ability. Psychological Monographs, 61(4), 1–49. https://doi.org/10.1037/h0093565
33.
LopezM. B.GoldsteinL. H.RobinsonE. J.VitoratouS.ChalderT.CarsonA.StoneJ.ReuberM. (2021). Validation of the PHQ-9 in adults with dissociative seizures. Journal of Psychosomatic Research, 146, Article 110487. https://doi.org/10.1016/j.jpsychores.2021.110487
34.
MalpassA.DowrickC.GilbodyS.RobinsonJ.WilesN.DuffyL.LewisG. (2016). Usefulness of PHQ-9 in primary care to determine meaningful symptoms of low mood: A qualitative study. British Journal of General Practice, 66(643), e78–e84. https://doi.org/10.3399/bjgp16X683473
35.
MalpassA.ShawA.KesslerD.SharpD. (2010). Concordance between PHQ-9 scores and patients’ experiences of depression: a mixed methods study. British Journal of General Practice, 60(575), e231–e238. https://doi.org/10.3399/bjgp10X502119
MokkenR. J. (1971). A theory and procedure of scale analysis: With applications in political research. De Gruyter Mouton.
39.
MolenaarI. W. (1991). A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Kwantitatieve Methoden, 12(37), 97–117.
40.
MolenaarI. W.SijtsmaK. (2000). User’s manual MSP5 for windows. IEC ProGAMMA.
41.
MonahanP. O.ShachamE.ReeceM.KroenkeK.Ong‘orW. O.OmolloO.YebeiV. N.OjwangC. (2008). Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/AIDS in western Kenya. Journal of General Internal Medicine, 24(2), 189–197. https://doi.org/10.1007/s11606-008-0846-z
42.
Moreno-AgostinoD.ChuaK.-C.PetersT. J.ScazufcaM.ArayaR. (2022). Psychometric properties of the PHQ-9 measure of depression among Brazilian older adults. Aging & Mental Health, 26, 2285–2290. https://doi.org/10.1080/13607863.2021.1963951
National Institute for Health and Care Excellence. (2009a). Depression in adults: Recognition and management (clinical guideline 90). https://www.nice.org.uk/guidance/cg90
45.
National Institute for Health and Care Excellence. (2009b). Depression in adults with a chronic physical health problem: Recognition and management (clinical guideline 91). https://www.nice.org.uk/guidance/CG91
46.
National Institute for Health and Care Excellence. (2011a). Common mental health problems: Identification and pathways to care (clinical guideline 123). www.nice.org.uk/guidance/cg123
47.
National Institute for Health and Care Excellence. (2011b). Depression in adults: Quality standard. www.nice.org.uk/guidance/qs8
48.
Nolen-HoeksemaS. (2001). Gender differences in depression. Current Directions in Psychological Science, 10(5), 173–176. https://doi.org/10.1111/1467-8721.00142
49.
NowokB.RaabG. M.DibbenC.SnokeJ.van LissaC. (2016). synthpop: Generating synthetic versions of sensitive microdata for statistical disclosure control (Version 1.6-0). https://CRAN.R-project.org/package=synthpop
50.
NunnallyJ. C.BernsteinI. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
51.
R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
52.
RichardsonE.RichardsJ. (2008). Factor structure of the PHQ-9 screen for depression across time since injury among persons with spinal cord injury. Rehabilitation Psychology, 53(2), 243–249. https://doi.org/10.1037/0090-5550.53.2.243
53.
RobinsonJ.KhanN.FuscoL.MalpassA.LewisG.DowrickC. (2017). Why are there discrepancies between depressed patients’ Global Rating of Change and scores on the Patient Health Questionnaire depression module? A qualitative study of primary care in England. BMJ Open, 7(4), Article e014519. https://doi.org/10.1136/bmjopen-2016-014519
54.
RyanT. A.BaileyA.FearonP.KingJ. (2013). Factorial invariance of the Patient Health Questionnaire and Generalized Anxiety Disorder Questionnaire. British Journal of Clinical Psychology, 52(4), 438–449. https://doi.org/10.1111/bjc.12028
55.
SheehanD. V.LecrubierY.SheehanK. H.AmorimP.JanavsJ.WeillerE.HerguetaT.BakerR.DunbarG. C. (1998). The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59(Suppl. 20), 22–33.
56.
ShinC.KoY.-H.AnH.YoonH.-K.HanC. (2020). Normative data and psychometric properties of the Patient Health Questionnaire-9 in a nationally representative Korean population. BMC Psychiatry, 20, Article 194. https://doi.org/10.1186/s12888-020-02613-0
57.
SijtsmaK.MolenaarI. W. (2002). Introduction to nonparametric item response theory: Volume 5. SAGE.
58.
SijtsmaK.StraatJ. H.Van der ArkL. A. (2015). Goodness-of-fit methods for nonparametric IRT models. In Van der ArkL. A.BoltD. M.WangW-C.DouglasJ. A.ChowS-M. (Eds.), Quantitative psychology research: The 79th Annual Meeting of the Psychometric Society, Madison, Wisconsin, 2014 (pp. 109–120). Springer. https://doi.org/10.1007/978-3-319-19977-1_9
59.
SijtsmaK.Van der ArkL. A. (2017). A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. British Journal of Mathematical and Statistical Psychology, 70(1), 137–158. https://doi.org/10.1111/bmsp.12078
60.
SilvermanJ. J.GalanterM.Jackson-TricheM.JacobsD. G.LomaxJ. W.RibaM. B.TongL. D.WatkinsK. E.FochtmannL. J.RhoadsR. S.YagerJ. (2015). The American Psychiatric Association practice guidelines for the psychiatric evaluation of adults, third edition. American Journal of Psychiatry, 172(8), 798–802. https://doi.org/10.1176/appi.ajp.2015.1720501
61.
StochlJ.FriedE. I.FritzJ.CroudaceT. J.RussoD. A.KnightC.JonesP. B.PerezJ. (2022). On dimensionality, measurement invariance, and suitability of sum scores for the PHQ-9 and the GAD-7. Assessment, 29(3), 355–366. https://doi.org/10.1177/1073191120976863
62.
StraatJ. H.Van der ArkL. A.SijtsmaK. (2016). Using conditional association to identify locally independent item sets. Methodology, 12(4), 117–123. https://doi.org/10.1027/1614-2241/a000115
Van der ArkL. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5), 1–27.
66.
Van SchuurW. H. (2003). Mokken scale analysis: Between the Guttman scale and parametric item response theory. Political Analysis, 11(2), 139–163. https://doi.org/10.1093/pan/mpg002
67.
VellozaJ.NjorogeJ.NgureK.ThuoN.KiptinnessC.MomanyiR.AyubS.GakuoS.MugoN.SimoniJ.HeffronR. (2020). Cognitive testing of the PHQ-9 for depression screening among pregnant and postpartum women in Kenya. BMC Psychiatry, 20, Article 31. https://doi.org/10.1186/s12888-020-2435-6
68.
VésteinsdóttirV.JoinsonA.ReipsU. D.DanielsdottirH. B.ThorarinsdottirE. A.ThorsdottirF. (2019). Questions on honest responding. Behavior Research Methods, 51, 811–825. https://doi.org/10.3758/s13428-018-1121-9
69.
VésteinsdóttirV.ReipsU.-D.JoinsonA.ThorsdottirF. (2017). An item level evaluation of the Marlowe–Crowne Social Desirability Scale using item response theory on Icelandic Internet panel data and cognitive interviews. Personality and Individual Differences, 107, 164–173. https://doi.org/10.1016/j.paid.2016.11.023
70.
WallerN. G.TellegenA.McDonaldR. P.LykkenD. T. (1996). Exploring nonlinear models in personality assessment: Development and preliminary validation of a negative emotionality scale. Journal of Personality, 64(3), 545–576. https://doi.org/10.1111/j.1467-6494.1996.tb00521.x
71.
WilliamsR.HeinemannA.BodeR.WilsonC.FannJ.TateD. (2009). Improving measurement properties of the Patient Health Questionnaire-9 with rating scale analysis. Rehabilitation Psychology, 54(2), 198–203. https://doi.org/10.1037/a0015529
WillisG. B. (2015). Analysis of the cognitive interview in questionnaire design. Oxford University Press.
74.
YuX.TamW. W. S.WongP. T. K.LamT. H.StewartS. M. (2012). The Patient Health Questionnaire-9 for measuring depressive symptoms among the general population in Hong Kong. Comprehensive Psychiatry, 53(1), 95–102. https://doi.org/10.1016/j.comppsych.2010.11.002
75.
ZijlstraW. P.van der ArkL. A.SijtsmaK. (2007). Outlier detection in test and questionnaire data. Multivariate Behavioral Research, 42(3), 531–555. https://doi.org/10.1080/00273170701384340
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
