Suicidal ideation disproportionately impacts minoritized populations; yet, these populations have been historically underrepresented in the development and validation of risk assessments, including the Beck Scale for Suicidal Ideation (BSS). The current study investigated the psychometric properties of the BSS in a combined sample (N = 1,026) of minoritized adults to examine its generalizability across diverse identities. Item response theory (IRT) models were estimated, and parameters including differential item functioning (DIF) were evaluated. Results showed that the original BSS and a 5-item BSS devised in a majority-aligning sample had significant model misspecification in the current study’s diverse sample, but a reduced 14-item and 8-item scale has good properties and may be better-performing alternatives. DIF results supported measurement invariance of the BSS across race/ethnicity, sexual orientation, social class, and history of suicide attempts. Inclusion of certain suicidal behaviors (e.g., writing a note) into an assessment of suicidal ideation may not be effective.
AdamsN.HitomiM.MoodyC. (2017). Varied reports of adult transgender suicidality: Synthesizing and describing the peer-reviewed and gray literature. Transgender Health, 2(1), 60–75. https://doi.org/10.1089/trgh.2016.0036
2.
AuerswaldM.MoshagenM. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468.
BagbyR. M.KeeleyJ. W.WilliamsC. C.MortezaeiA.RyderA. G.SellbomM. (2021). Evaluating the measurement invariance of the Personality Inventory for DSM-5 (PID-5) in Black Americans and White Americans. Psychological Assessment, 34(1), 82–90.
5.
BakerF. B. (2001). The basics of item response theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation.
6.
BatterhamP. J.FtanouM.PirkisJ.BrewerJ. L.MackinnonA. J.BeautraisA.Fairweather-SchmidtA. K.ChristensenH. (2015). A systematic review and evaluation of measures for suicidal ideation and behaviors in population-based research. Psychological Assessment, 27(2), 501.
7.
BeckA. T.KovacsM.WeissmanA. (1979). Assessment of suicidal intention: The Scale for Suicide Ideation. Journal of Consulting and Clinical Psychology, 47(2), 343.
8.
BeckA. T.SteerR. A.CarbinM. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8(1), 77–100.
9.
BeckA. T.SteerR. A.RanieriW. F. (1988). Scale for suicide ideation: Psychometric properties of a self-report version. Journal of Clinical Psychology, 44(4), 499–505.
10.
BelzakW. C. M.BauerD. J. (2020). Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. Psychological Methods, 25(6), 673–690. https://doi.org/10.1037/met0000253
11.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300.
12.
BentlerP. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238.
13.
BentlerP. M. (1995). EQS structural equations program manual (Vol. 6). Multivariate Software Encino, CA.
14.
BonifayW. E.ReiseS. P.ScheinesR.MeijerR. R. (2015). When are multidimensional data unidimensional enough for structural equation modeling? An evaluation of the DETECT multidimensionality index. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 504–516.
15.
BorgognaN. C.BrennerR. E.McDermottR. C. (2021). Sexuality and gender invariance of the PHQ-9 and GAD-7: Implications for 16 identity groups. Journal of Affective Disorders, 278, 122–130.
16.
BovinM. J.MarxB. P.WeathersF. W.GallagherM. W.RodriguezP.SchnurrP. P.KeaneT. M. (2016). Psychometric properties of the PTSD checklist for diagnostic and statistical manual of mental disorders–fifth edition (PCL-5) in veterans. Psychological Assessment, 28(11), 1379.
17.
CanadyV. A. (2023). Mounting anti-LGBTQ+ bills impact mental health of youths. Mental Health Weekly, 33(15), 1–6.
18.
ChaC. B.TezanosK. M.PerosO. M.NgM. Y.RibeiroJ. D.NockM. K.FranklinJ. C. (2018). Accounting for diversity in suicide research: Sampling and sample reporting practices in the United States. Suicide and Life-Threatening Behavior, 48(2), 131–139.
19.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.
20.
ChalmersR. P.CounsellA.FloraD. B. (2016). It might not make a big DIF: Improved differential test functioning statistics that account for sampling variability. Educational and Psychological Measurement, 76(1), 114–140.
21.
ChenW.-H.ThissenD. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.
22.
ChoiJ. L.RogersJ. R.WerthJ. L., Jr. (2009). Suicide risk assessment with Asian American college students: A culturally informed perspective. The Counseling Psychologist, 37(2), 186–218. https://doi.org/10.1177/0011000006292256
23.
ChristensenK. B.MakranskyG.HortonM. (2017). Critical values for Yen’s Q3: Identification of local dependence in the Rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178–194. https://doi.org/10.1177/0146621616677520
24.
ChuC.KleinK. M.Buchman-SchmittJ. M.HomM. A.HaganC. R.JoinerT. E. (2015). Routinized assessment of suicide risk in clinical practice: An empirically informed update. Journal of Clinical Psychology, 71(12), 1186–1200.
25.
ChuJ.RobinettE. N.MaJ. K.ShadishK. Y.GoldblumP.BongarB. (2019). Cultural versus classic risk and protective factors for suicide. Death Studies, 43(1), 56–61.
26.
CoimbraB. M.HoeboerC. M.YikJ.MelloA. F.MelloM. F.OlffM. (2022). Meta-analysis of the effect of racial discrimination on suicidality. SSM - Population Health, 20, 101283. https://doi.org/10.1016/j.ssmph.2022.101283
27.
de BeursD. P.FokkemaM.de GrootM. H.de KeijserJ.KerkhofA. J. (2015). Longitudinal measurement invariance of the beck scale for suicide ideation. Psychiatry Research, 225(3), 368–373.
28.
EngbergM. E.AllenD. J. (2011). Uncontrolled destinies: Improving opportunity for low-income students in American higher education. Research in Higher Education, 52, 786–807.
29.
EngleJ.TintoV. (2008). Moving beyond access: College success for low-income, first-generation students. Pell Institute for the Study of Opportunity in Higher Education. Retrieved from http://www.eric.ed.gov/PDFS/ED504448.pdf.
30.
EsfahaniM.HashemiY.AlaviK. (2015). Psychometric assessment of beck scale for suicidal ideation (BSSI) in general population in Tehran. Medical Journal of the Islamic Republic of Iran, 29, 268.
31.
FoaE. B.McLeanC. P.ZangY.ZhongJ.PowersM. B.KauffmanB. Y.RauchS.PorterK.KnowlesK. (2016). Psychometric properties of the Posttraumatic Diagnostic Scale for DSM-5 (PDS-5). Psychological Assessment, 28(10), 1166–1171. https://doi.org/10.1037/pas0000258
32.
FranklinJ. C.RibeiroJ. D.FoxK. R.BentleyK. H.KleimanE. M.HuangX.MusacchioK. M.JaroszewskiA. C.ChangB. P.NockM. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187–232. https://doi.org/10.1037/bul0000084
33.
HambletonR. K.JonesR. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38–47.
34.
HealyD. J.BarryK.BlowF.WelshD.MilnerK. K. (2006). Routine use of the Beck Scale for Suicide Ideation in a psychiatric emergency department. General Hospital Psychiatry, 28(4), 323–329.
HoffmanM.SteinleyD.TrullT. J.SherK. J. (2019). Estimating transdiagnostic symptom networks: The problem of “skip outs” in diagnostic interviews. Psychological Assessment, 31(1), 73.
37.
JunkerB. W.SijtsmaK. (2000). Latent and manifest monotonicity in item response models. Applied Psychological Measurement, 24(1), 65–81.
38.
KangT.ChenT. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45(4), 391–406.
39.
KeanJ.ReillyJ. (2014). Item response theory. In HammondF.MalecJ.NickT. G. (Eds.), Handbook for clinical research: Design, statistics and implementation (pp. 195–198). Demos Medical Publishing.
40.
KimJ.PikeK.McCauleyE.Vander StoepA. (2019). Ethnic variations of trajectories in suicide ideation and attempt: From middle school to high school. Suicide and Life-Threatening Behavior, 49(2), 432–443.
41.
KlonskyE. D.MayA. M. (2015). The three-step theory (3ST): A new theory of suicide rooted in the “ideation-to-action” framework. International Journal of Cognitive Therapy, 8(2), 114–129.
42.
KroenkeK.SpitzerR. L.WilliamsJ. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.
43.
LaiJ.-S.TeresiJ.GershonR. (2005). Procedures for the analysis of Differential Item Functioning (DIF) for small sample sizes. Evaluation & the Health Professions, 28(3), 283–294. https://doi.org/10.1177/0163278705278276
44.
LiH.XieW.LuoX.FuR.ShiC.YingX.WangN.YinQ.WangX. (2014). Clarifying the role of psychological pain in the risks of suicidal ideation and suicidal acts among patients with major depressive episodes. Suicide and Life-Threatening Behavior, 44(1), 78–88. https://doi.org/10.1111/sltb.12056
45.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
46.
LytleM. C.De LucaS. M.BlosnichJ. R.BrownsonC. (2015). Associations of racial/ethnic identities and religious affiliation with suicidal ideation among lesbian, gay, bisexual, and questioning individuals. Journal of Affective Disorders, 178, 39–45. https://doi.org/10.1016/j.jad.2014.07.039
47.
Maydeu-OlivaresA.JoeH. (2014). Assessing approximate fit in categorical data analysis. Multivariate Behavioral Research, 49(4), 305–328.
48.
McBrideO.van BezooijenJ.AggenS. H.KendlerK. S.FriedE. I. (2023). Quantifying skip-out information loss when assessing major depression symptoms. Journal of Psychopathology and Clinical Science, 132(4), 396.
49.
MeadeA. W. (2010). A taxonomy of effect size measures for the differential functioning of items and scales. Journal of Applied Psychology, 95(4), 728.
50.
MetalskyG. I.JoinerT. E. (1997). The hopelessness depression symptom questionnaire. Cognitive Therapy and Research, 21(3), 359–384.
51.
MillsapR. E. (2012). Statistical approaches to measurement invariance. Routledge.
52.
MirandaR.GallagherM.BauchnerB.VaysmanR.MarroquínB. (2012). Cognitive inflexibility as a prospective predictor of suicidal ideation among young adults with a suicide attempt history: Cognitive inflexibility and suicidal behavior. Depression and Anxiety, 29(3), 180–186. https://doi.org/10.1002/da.20915
53.
MolockS. D.BoydR. C.AlvarezK.ChaC.DentonE.GlennC. R.KatzC. C.MuellerA. S.MecaA.MezaJ. I. (2023). Culturally responsive assessment of suicidal thoughts and behaviors in youth of color. American Psychologist, 78(7), 842–855.
54.
NdeteiD. M.MutisoV. N.WeiszJ. R.OkothC. A.MusyimiC.MuiaE. N.OsbornT. L.SouranderA.WassermanD.MamahD. (2022). Socio-demographic, economic and mental health problems were risk factors for suicidal ideation among Kenyan students aged 15 plus. Journal of Affective Disorders, 302, 74–82. https://doi.org/10.1016/j.jad.2022.01.055
55.
NockM. K.HolmbergE. B.PhotosV. I.MichelB. D. (2007). Self-injurious thoughts and behaviors interview: Development, reliability, and validity in an adolescent sample. Psychological Assessment, 19(3), 309–317.
56.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.
57.
PinnintiN.SteerR. A.RissmillerD. J.NelsonS.BeckA. T. (2002). Use of the Beck Scale for Suicide Ideation with psychiatric inpatients diagnosed with schizophrenia, schizoaffective, or bipolar disorders. Behavior Research and Therapy, 40(9), 1071–1079. https://doi.org/10.1016/S0005-7967(02)00002-5
58.
PodlogarM. C.GutierrezP. M.OsmanA. (2023). Optimizing the Beck Scale for suicide ideation: An item response theory approach among U.S. military personnel. Assessment, 30(4), 1321–1333.
59.
PollakO. H.SheehanA. E.WalshR. F.StephensonA. R.ZellH.MayesJ.LawrenceH. R.BettisA. H.LiuR. T. (2024). Assessment of suicidal thoughts and behaviors in adults: A systematic review of measure psychometric properties and implications for clinical and research utility. Clinical Psychology Review, 112, 102464.
60.
PosnerK.BrodskyB.YershovaK.BuchananJ.MannJ. (2014). The classification of suicidal behavior. In NockM. (Ed.), The Oxford handbook of suicide and self-injury (pp. 7–22). Oxford University Press.
61.
R Core Team. (2021). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
62.
RaschkeN.MohsenpourA.AschentrupL.FischerF.WronaK. J. (2022). Socioeconomic factors associated with suicidal behaviors in South Korea: Systematic review on the current state of evidence. BMC Public Health, 22(1), 129. https://doi.org/10.1186/s12889-022-12498-1
RibeiroJ. D.FranklinJ. C.FoxK. R.BentleyK. H.KleimanE. M.ChangB. P.NockM. K. (2016). Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: A meta-analysis of longitudinal studies. Psychological Medicine, 46(2), 225–236. https://doi.org/10.1017/S0033291715001804
66.
RimkevicieneJ.O’GormanJ.De LeoD. (2015). Impulsive suicide attempts: A systematic literature review of definitions, characteristics and risk factors. Journal of Affective Disorders, 171, 93–104. https://doi.org/10.1016/j.jad.2014.08.044
67.
RockettI. R. H.CaineE. D.ConneryH. S.D’OnofrioG.GunnellD. J.MillerT. R.NolteK. B.KaplanM. S.KapustaN. D.LillyC. L.NelsonL. S.PutnamS. L.StackS.VärnikP.WebsterL. R.JiaH. (2018). Discerning suicide in drug intoxication deaths: Paucity and primacy of suicide notes and psychiatric history. PLoS One, 13(1), e0190200. https://doi.org/10.1371/journal.pone.0190200
68.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137.
69.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36.
70.
RuscioJ.RocheB. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282.
71.
SamejimaF. (2016). Graded response models. In LindenW. J.HambletonR. K. (Eds.), Handbook of item response theory (pp. 123–136). Chapman and Hall/CRC.
72.
ScottN. W.FayersP. M.AaronsonN. K.BottomleyA.de GraeffA.GroenvoldMGundyCKollerMPetersenMASprangersMA; EORTC Quality of Life Group & Quality of Life Cross-Cultural Meta-Analysis Group. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288–295.
73.
ShermanR. M.D’OrioB.RhodesM. N.JohnsonS. G.KaslowN. J. (2014). Racial/ethnic, spiritual/religious, and sexual orientation influences on suicidal behaviors. In NockM. (Ed.), The Oxford handbook of suicide and self-injury (pp. 265–285). Oxford University Press.
74.
SimpsonS. A.LohR. M.GoansC. R. (2023). New data on suicide risk assessment in the emergency department reveal the need for new approaches in research and clinical practice. Psychological Medicine, 53(3), 1122–1123.
75.
SteerR. A.RissmillerD. J.RanieriW. F.BeckA. T. (1993). Dimensions of suicidal ideation in psychiatric inpatients. Behavior Research and Therapy, 31(2), 229–236. https://doi.org/10.1016/0005-7967(93)90090-H
76.
SteinerM. D.GriederS. (2020). EFAtools: An R package with fast and flexible implementations of exploratory factor analysis tools. Journal of Open Source Software, 5(53), 2521.
77.
Substance Abuse and Mental Health Services Administration. (2022). Key substance use and mental health indicators in the United States: Results from the 2021 National Survey on Drug Use and Health (PEP22-07-01–005; Series H-57). Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/report/2021-nsduh-annual-national-report
78.
SwansonS. A.BrownT. A.CrosbyR. D.KeelP. K. (2014). What are we missing? The costs versus benefits of skip rule designs: Skip rule designs. International Journal of Methods in Psychiatric Research, 23(4), 474–485. https://doi.org/10.1002/mpr.1396
79.
ThomasM. L. (2019). Advances in applications of item response theory to clinical assessment. Psychological Assessment, 31(12), 1442–1455. https://doi.org/10.1037/pas0000597
80.
TuckerL. R.LewisC. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.
81.
Van der ArkL. A. (2007). Mokken scale analysis in R. Journal of Statistical Software, 20, 1–19.
82.
WoodsA. M.ZimmermanL.CarlinE.HillA.KaslowN. J. (2013). Motherhood, reasons for living, and suicidality among African American women. Journal of Family Psychology, 27(4), 600–606. https://doi.org/10.1037/a0033592
83.
XiaoY.CerelJ.MannJ. J. (2021). Temporal trends in suicidal ideation and attempts among U.S. adolescents by sex and race/ethnicity, 1991–2019. JAMA Network Open, 4(6), e2113513. https://doi.org/10.1001/jamanetworkopen.2021.13513
84.
YenW. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125–145.
85.
ZhangJ.BrownG. K. (2007). Psychometric properties of the scale for suicide ideation in China. Archives of Suicide Research, 11(2), 203–210. https://doi.org/10.1080/13811110600894652
86.
ZumboB. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
