Despite the Heterosexist Harassment, Rejection, and Discrimination Scale (HHRDS) being a routinely used measure of discrimination, there is no current consensus regarding its fundamental psychometric properties. This study sought to: (a) test the dimensionality of the scale using a classical test theory approach, and (b), if lacking a clear multidimensional structure, use an item response theory approach to develop a concise unidimensional measure and examine differential item and test functioning across various identity factors. In a sample of sexual minority adults (N = 1,243), evaluation of the HHRDS supported a reduced 11-item unidimensional measure. Assessment of differential item functioning suggested differences for some items between cisgender men and cisgender women, cisgender men and gender-diverse individuals, gay/lesbian and bi+ individuals, and white/non-Latine individuals and people of color. However, differential item functioning had minimal impact on total test scores, supporting the use of the revised 11-item HHRDS across groups.
AuerswaldM.MoshagenM. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468–491. https://doi.org/10.1037/met0000200
2.
BakerF. B. (2001). The basics of item response theory. ERIC.
3.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
BentlerP. M. (1995). EQS structural equations program manual (Vol. 6). Multivariate Software, Inc.
6.
BonifayW. E.ReiseS. P.ScheinesR.MeijerR. R. (2015). When are multidimensional data unidimensional enough for structural equation modeling? An evaluation of the DETECT multidimensionality index. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 504–516. https://doi.org/10.1080/10705511.2014.938596
7.
BresinK.NicholasJ. K.CowandA. L.AlachaH. F.RodriguezA. M.ParrottD. J. (2023). The effects of sexual and gender minority stress on relationship functioning: A meta-analysis. Personal Relationships, 30(4), 1208–1231. https://doi.org/10.1111/pere.12511
8.
CanadyV. A. (2023). Mounting anti-LGBTQ+ bills impact mental health of youths. Mental Health Weekly, 33(15), 1–6. https://doi.org/10.1002/mhw.33603
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
11.
ChalmersR. P.CounsellA.FloraD. B. (2016). It might not make a big DIF: Improved differential test functioning statistics that account for sampling variability. Educational and Psychological Measurement, 76(1), 114–140. https://doi.org/10.1177/0013164415584576
12.
ChangC. J.FeinsteinB. A.ChuB. C.SelbyE. A. (2022). Application of minority stress and the interpersonal theory of suicide in bisexual+ versus gay/lesbian young adults. Suicide and Life-Threatening Behavior, 52(4), 725–739.
13.
ChenW.-H.ThissenD. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289. https://doi.org/10.3102/10769986022003265
14.
ChristensenK. B.MakranskyG.HortonM. (2017). Critical values for Yen’s Q3: Identification of local dependence in the Rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178–194. https://doi.org/10.1177/0146621616677520
15.
de LangeJ.BaamsL.van BergenD. D.BosH. M.BoskerR. J. (2022). Minority stress and suicidal ideation and suicide attempts among LGBT adolescents and young adults: a meta-analysis. LGBT health, 9(4), 222–237. 10.1089/lgbt.2021.0106
16.
DeBlaereC.BrewsterM. E.BertschK. N.DeCarloA. L.KegelK. A.PresseauC. D. (2014). The protective power of collective action for sexual minority women of color: An investigation of multiple discrimination experiences and psychological distress. Psychology of Women Quarterly, 38(1), 20–32. https://doi.org/10.1177/0361684313493252
17.
DunnT. L.SzymanskiD. M. (2018). Heterosexist discrimination and LGBQ activism: Examining a moderated mediation model. Psychology of Sexual Orientation and Gender Diversity, 5(1), 13. https://doi.org/10.1037/sgd0000250
18.
DürrbaumT.SattlerF. A. (2020). Minority stress and mental health in lesbian, gay male, and bisexual youths: A meta-analysis. Journal of LGBT Youth, 17(3), 298–314. https://doi.org/10.1080/19361653.2019.1586615
19.
FeinsteinB. A.GoldfriedM. R.DavilaJ. (2012). The relationship between experiences of discrimination and mental health among lesbians and gay men: An examination of internalized homonegativity and rejection sensitivity as potential mechanisms. Journal of consulting and clinical psychology, 80(5), 917. 10.1037/a0029425
20.
FeinsteinB. A.KhanA.ChangC. J.MillerS. A. (2023). Use of the heterosexist harassment rejection and discrimination scale with different sexual orientation, gender, and racial/ethnic groups: An examination of measurement invariance. Assessment, 30(8), 2605–2615. https://doi.org/10.1177/10731911231156135
21.
FigueroaW. S.ZoccolaP. M. (2016). Sources of discrimination and their associations with health in sexual minority adults. Journal of Homosexuality, 63(6), 743–763. https://doi.org/10.1080/00918369.2015.1112193
22.
FriedmanC.LeaperC. (2010). Sexual-minority college women’s experiences with discrimination: Relations with identity and collective action. Psychology of Women Quarterly, 34(2), 152–164. https://doi.org/10.1111/j.1471-6402.2010.01558.x
GoldbachJ. T.Tanner-SmithE. E.BagwellM.DunlapS. (2014). Minority stress and substance use in sexual minority adolescents: A meta-analysis. Prevention Science, 15, 350–363. https://doi.org/10.1007/s11121-013-0393-7
25.
JunkerB. W.SijtsmaK. (2000). Latent and manifest monotonicity in item response models. Applied Psychological Measurement, 24(1), 65–81. https://doi.org/10.1177/01466216000241004
26.
KangT.ChenT. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45(4), 391–406. https://doi.org/10.1111/j.1745-3984.2008.00071.x
27.
KroenkeK.SpitzerR. L.WilliamsJ. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
28.
KleimanS.SpaniermanL. B.SmithN. G. (2015). Translating oppression: Understanding how sexual minority status is associated with White men’s racial attitudes. Psychology of Men & Masculinity, 16(4), 404. https://doi.org/10.1037/a0038797
29.
KroenkeK.StrineT. W.SpitzerR. L.WilliamsJ. B.BerryJ. T.MokdadA. H. (2009). The PHQ-8 as a measure of current depression in the general population. Journal of Affective Disorders, 114(1–3), 163–173. https://doi.org/10.1016/j.jad.2008.06.026
30.
LehavotK.SimoniJ. M. (2011). The impact of minority stress on mental health and substance use among sexual minority women. Journal of consulting and clinical psychology, 79(2), 159. https://doi.org/10.1037/a0022839
31.
Maydeu-OlivaresA.JoeH. (2014). Assessing approximate fit in categorical data analysis. Multivariate Behavioral Research, 49(4), 305–328. https://doi.org/10.1080/00273171.2014.911075
32.
MeadeA. W. (2010). A taxonomy of effect size measures for the differential functioning of items and scales. Journal of Applied Psychology, 95(4), 728–743. https://doi.org/10.1037/a0018966
33.
MeyerI. H. (2003). Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychological bulletin, 129(5), 674.
34.
MeyerI. H.RussellS. T.HammackP. L.FrostD. M.WilsonB. D. M. (2021). Minority stress, distress, and suicide attempts in three cohorts of sexual minority adults: A U.S. probability sample. PLOS ONE, 16(3), e0246827. https://doi.org/10.1371/journal.pone.0246827
35.
MolenaarW.SijtsmaK. (2000). MSP5 for windows user’s manual. iec ProGAMMA.
36.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64. https://doi.org/10.1177/01466216000241003
37.
PalanS.SchitterC. (2018). Prolific. ac-A subject pool for online experiments. Journal of behavioral and experimental finance, 17, 22–27. https://doi.org/10.1016/j.jbef.2017.12.004
38.
R Core Team. (2021). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315
41.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
42.
RogersM. L.HomM. A.JanakiramanR.JoinerT. E. (2021). Examination of minority stress pathways to suicidal ideation among sexual minority adults: The moderating role of LGBT community connectedness. Psychology of Sexual Orientation and Gender Diversity, 8(1), 38.
RuscioJ.RocheB. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282–292. https://doi.org/10.1037/a0025697
46.
SamejimaF. (2016). Graded response models. In van der LindenW. J. (Ed.), Handbook of item response theory (pp. 123–136). Chapman and Hall/CRC.
47.
SassD. A.SchmittT. A.MarshH. W. (2014). Evaluating model fit with ordered categorical data within a measurement invariance framework: A comparison of estimators. Structural Equation Modeling: A Multidisciplinary Journal, 21(2), 167–180. https://doi.org/10.1080/10705511.2014.882658
48.
ShinC.LeeS. H.HanK. M.YoonH. K.HanC. (2019). Comparison of the usefulness of the PHQ-8 and PHQ-9 for screening for major depressive disorder: analysis of psychiatric outpatient data. Psychiatry investigation, 16(4), 300. https://doi.org/10.30773/pi.2019.02.01
49.
SmithE. R.PerrinP. B.SutterM. E. (2020). Factor analysis of the heterosexist harassment, rejection, and discrimination scale in lesbian, gay, bisexual, transgender, and queer people of color. International Journal of Psychology, 55(3), 405–412. https://doi.org/10.1002/ijop.12585
50.
SteinerM. D.GriederS. (2020). EFAtools: An R package with fast and flexible implementations of exploratory factor analysis tools. Journal of Open Source Software, 5(53), 2521. https://doi.org/10.21105/joss.02521
51.
StuckyB. D.EdelenM. O. (2015). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In ReiseS. P.RevickiD. A. (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 183–206). Routledge.
52.
SutterM.PerrinP. B. (2016). Discrimination, mental health, and suicidal ideation among LGBTQ people of color. Journal of counseling psychology, 63(1), 98. https://doi.org/10.1037/cou0000126
53.
SzymanskiD. M. (2006). Does internalized heterosexism moderate the link between heterosexist events and lesbians’ psychological distress?Sex Roles, 54(3), 227–234. https://doi.org/10.1007/s11199-006-9340-4
54.
SzymanskiD. M. (2009). Examining potential moderators of the link between heterosexist events and gay and bisexual men’s psychological distress. Journal of Counseling Psychology, 56(1), 142. https://doi.org/10.1037/0022-0167.56.1.142
55.
SzymanskiD. M.DunnT. L.IkizlerA. S. (2014). Multiple minority stressors and psychological distress among sexual minority women: The roles of rumination and maladaptive coping. Psychology of sexual orientation and gender diversity, 1(4), 412. https://doi.org/10.1037/sgd0000066
56.
SzymanskiD. M.GuptaA. (2009). Examining the relationship between multiple internalized oppressions and African American lesbian, gay, bisexual, and questioning persons’ self-esteem and psychological distress. Journal of Counseling Psychology, 56(1), 110. https://doi.org/10.1037/a0013317
57.
SzymanskiD. M.Henrichs-BeckC. (2014). Exploring sexual minority women’s experiences of external and internalized heterosexism and sexism and their links to coping and distress. Sex Roles, 70, 28–42. https://doi.org/10.1007/s11199-013-0329-5
58.
SzymanskiD. M.IkizlerA. S. (2013). Internalized heterosexism as a mediator in the relationship between gender role conflict, heterosexist discrimination, and depression among sexual minority men. Psychology of Men & Masculinity, 14(2), 211. https://doi.org/10.1037/a0027787
59.
TaherdoostH.SahibuddinS.JalaliyoonN. (2022). Exploratory factor analysis; concepts and theory. Advances in Applied and Pure Mathematics, 27, 375–382.
60.
TuckerL. R.LewisC. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10. https://doi.org/10.1007/BF02291170
VelezB. L.MoradiB.DeBlaereC. (2015). Multiple oppressions and the mental health of sexual minority Latina/o individuals. The Counseling Psychologist, 43(1), 7–38. https://doi.org/10.1037/a0027787
63.
WatsonL. B.GrotewielM.FarrellM.MarshikJ.SchneiderM. (2015). Experiences of sexual objectification, minority stress, and disordered eating among sexual minority women. Psychology of Women Quarterly, 39(4), 458–470. https://doi.org/10.1177/0361684315575024
64.
WuY.LevisB.RiehmK. E.SaadatN.LevisA. W.AzarM.. . .ThombsB. D. (2020). Equivalency of the diagnostic accuracy of the PHQ-8 and PHQ-9: A systematic review and individual participant data meta-analysis. Psychological Medicine, 50(8), 1368–1380. https://doi.org/10.1017/S0033291719001314
65.
YenW. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125–145. https://doi.org/10.1177/014662168400800201
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
