Rating scales are commonly used in psychological research and practice. It is important to consider how different item response options used on rating scales, such as those based on frequency or severity, may affect how they measure psychological constructs. In this study, participants were randomly assigned to conditions where they completed two depression and two generalized anxiety rating scales that used either frequency or severity item response options. Descriptive statistics and reliability estimates across depression and generalized anxiety scales using different item response options were similar. Measurement invariance tests indicated that depression and generalized anxiety scales with different item response options could be considered practically invariant. Finally, depression and generalized anxiety scales with different item response options had similar correlations with measures of impairment and other psychological constructs. There were essentially no differences in psychometric and measurement properties of these depression and generalized anxiety scales when using different item response options, suggesting this may not substantially affect the measurement of these constructs.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).
2.
AsparouhovT.MuthénB. (2010). Weighted least squares estimation with missing data. Mplus Technical Appendix, 2010, 1–10.
3.
BeardC.HsuK. J.RifkinL. S.BuschA. B.BjörgvinssonT. (2016). Validation of the PHQ-9 in a psychiatric sample. Journal of Affective Disorders, 193, 267–273. https://doi.org/10.1016/j.jad.2015.12.075
4.
BensonN. F.FloydR. G.KranzlerJ. H.EckertT. L.FeferS. A.MorganG. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey. Journal of School Psychology, 72, 29–48. https://doi.org/10.1016/j.jsp.2018.12.004
5.
Byrd-BredbennerC.EckK.QuickV. (2021). GAD-7, GAD-2, and GAD-mini: Psychometric properties and norms of university students in the United States. General Hospital Psychiatry, 69, 61–66. https://doi.org/10.1016/j.genhosppsych.2021.01.002
6.
CasperW. C.EdwardsB. D.WallaceJ. C.LandisR. S.FifeD. A. (2020). Selecting response anchors with equal intervals for summated rating scales. Journal of Applied Psychology, 105(4), 390–409. https://doi.org/10.1037/apl0000444
7.
Center for Collegiate Mental Health. (2022, January). 2021 annual report (Publication No. STA 22-132).
8.
ChangC. H.CellaD.ClarkeS.HeinemannA. W.Von RoennJ. H.HarveyR. (2003). Should symptoms be scaled for intensity, frequency, or both?Palliative & Supportive Care, 1(1), 51–60. https://doi.org/10.1017/S1478951503030049
9.
CostantiniL.PasquarellaC.OdoneA.ColucciM. E.CostanzaA.SerafiniG.. . . AmerioA. (2021). Screening for depression in primary care with Patient Health Questionnaire-9 (PHQ-9): A systematic review. Journal of Affective Disorders, 279, 473–483. https://doi.org/10.1016/j.jad.2020.09.131
10.
DalalR. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90(6), 1241–1255. https://doi.org/10.1037/0021-9010.90.6.1241
11.
DaoustR.SiroisM. J.LeeJ. S.PerryJ. J.GriffithL. E.WorsterA.LangE.PaquetJ.ChaunyJ.-M.ÉmondM. (2017). Painful memories: Reliability of pain intensity recall at 3 months in senior patients. Pain Research and Management, 2017, 5983721. https://doi.org/10.1155/2017/5983721
12.
ElhaiJ. D.PalmieriP. A.BiehnT. L.FruehB. C.MagruderK. M. (2010). Posttraumatic stress disorder's frequency and intensity ratings are associated with factor structure differences in military veterans. Psychological Assessment, 22(4), 723–728. https://doi.org/10.1037/a0020643
IBM Corp. Released. (2021). IBM SPSS statistics for windows (Version 28.0).
15.
KroenkeK.SpitzerR. L.WilliamsJ. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
LittleT. D. (2013). Longitudinal structural equation modeling. Guilford.
18.
LockeB. D.McAleaveyA. A.ZhaoY.LeiP. W.HayesJ. A.CastonguayL. G.LiH.TateR.LinY. C. (2012). Development and initial validation of the Counseling Center Assessment of Psychological Symptoms-34 (CCAPS-34). Measurement and Evaluation in Counseling and Development, 45, 151–169. https://doi.org/10.1177/0748175611432642
19.
MeadeA. W.BauerD. J. (2007). Power and precision in confirmatory factor analytic tests of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 611–635. https://doi.org/10.1080/10705510701575461
20.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
21.
MordenoI. G.NalipayM.JeninaN.LuzanoJ. G. C.GalelaD. S.FerolinoM. A. L. (2021). Development and validation of a DSM-5-based generalized anxiety disorder self-report Scale: Investigating frequency and intensity rating differences. Current Psychology, 40(11), 5247–5255. https://doi.org/10.1007/s12144-019-00475-8
22.
MuthénL. K.MuthénB. O. (1998-2012). Mplus user’s guide (7th ed.).
23.
PilkonisP. A.ChoiS. W.ReiseS. P.StoverA. M.RileyW. T.CellaD., & PROMIS Cooperative Group. (2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, anxiety, and anger. Assessment, 18(3), 263–283. https://doi.org/10.1177/1073191111411667
24.
PlummerF.ManeaL.TrepelD.McMillanD. (2016). Screening for anxiety disorders with the GAD-7 and GAD-2: A systematic review and diagnostic meta-analysis. General Hospital Psychiatry, 39, 24–31. https://doi.org/10.1016/j.genhosppsych.2015.11.005
25.
RutterL. A.BrownT. A. (2017). Psychometric properties of the Generalized Anxiety Disorder Scale-7 (GAD-7) in outpatients with anxiety and mood disorders. Journal of Psychopathology and Behavioral Assessment, 39(1), 140–146. https://doi.org/10.1007/s10862-016-9571-9
26.
Schermelleh-EngelK.MoosbruggerH.MüllerH. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23–74.
27.
ShermanM. F.SrikenJ.ErfordB. T.SmithH. L.MacInerneyE.NiarhosF.Kipper-SmithA. (2021). Psychometric analysis of CCAPS-34 scores with a large university sample. Measurement and Evaluation in Counseling and Development, 54(4), 219–232. https://doi.org/10.1080/07481756.2020.1843971
28.
SimmsL. J.ZelaznyK.WilliamsT. F.BernsteinL. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological Assessment, 31(4), 557–566. https://doi.org/10.1037/pas0000648
29.
SpectorP. E.BauerJ. A.FoxS. (2010). Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know?Journal of Applied Psychology, 95(4), 781–790. https://doi.org/10.1037/a0019477
30.
SpitzerR. L.KroenkeK.WilliamsJ. B. W.LöweB. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
31.
ZimmermanM.ChelminskiI.McGlincheyJ. B.PosternakM. A. (2008). A clinically useful depression outcome scale. Comprehensive Psychiatry, 49(2), 131–140. https://doi.org/10.1016/j.comppsych.2007.10.006
32.
ZimmermanM.ChelminskiI.YoungD.DalrympleK. (2010). A clinically useful anxiety outcome scale. The Journal of Clinical Psychiatry, 71(5), 534–542. https://doi.org/10.4088/JCP.09m05264blu
33.
ZimmermanM.KerrS. (2019). How should the severity of depression be rated on self-report depression scales?Psychiatry Research, 280, 112512. https://doi.org/10.1016/j.psychres.2019.112512
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
