Choices Matter: How Response Options for Survey Questions about Sexual Identity Affect Population Estimates of Its Association with Alcohol,Tobacco,and Other Drug Use

Abstract

This study presents results from a randomized experiment in the 2015–2017 National Survey of Family Growth, where a large national sample of U.S. individuals aged 15–49 was randomly assigned to one of two different versions of a survey question about sexual identity (one with three response options, including heterosexual, gay/lesbian, and bisexual, and one adding the option “something else”). Analyses of changes in the associations of sexual identity with alcohol, tobacco, and other drug use across these treatments revealed evidence of significant differences in the associations that remained robust after adjusting for socio-demographics. The results suggest that when individuals choose their sexual identity from a more limited number of response options, the heterogeneity of the sexual identity subgroups increases, weakening estimated associations of sexual identity with these behaviors. Open-ended questions may therefore be necessary to measure sexual identity and estimate its associations with substance use behaviors accurately in surveys.

Get full access to this article

View all access options for this article.

References

Baldwin

Schick

V. R.

Dodge

van Der Pol

Herbenick

Sanders

S. A.

Fortenberry

J. D.

. 2017. Variation in sexual identification among behaviorally bisexual women in the Midwestern United States: Challenging the established methods for collecting data on sexual identity and orientation. Archives of Sexual Behavior 46:1337–48.

Bates

García Trejo

Y. A.

Vines

. 2019. Are sexual minorities hard-to-survey? Insights from the 2020 Census Barriers, Attitudes, and Motivators Study (CBAMS) survey. Journal of Official Statistics 35:709–29.

Bauer

G. R.

Jairam

J. A.

. 2008. Are lesbians really women who have sex with women (WSW)? Methodological concerns in measuring sexual orientation in health research. Women and Health 48:383–408.

Bishop

G. F.

Hippler

H. J.

Schwar

Stack

. 1988. A comparison of response effects in self-administered and telephone surveys. In Telephone survey methodology, eds. Groves

R. M.

Biemer

P. P.

Lyberg

L. E.

Massey

J. T.

Nicholls

W. L.

Waksberg

, 321–40. New York: Wiley.

Boyd

C. J.

Veliz

P. T.

Stephenson

Hughes

T. L.

McCabe

S. E.

. 2019. Severity of alcohol, tobacco, and drug use disorders among sexual minority individuals and their “not sure” counterparts. LGBT Health 6:15–22.

Copen

C. E.

Chandra

Febo-Vazquez

. 2016. Sexual behavior, sexual attraction, and sexual orientation among adults aged 18-44 in the United States: Data from the 2011–2013 National Survey of Family Growth. National Health Statistics Report 88:1–14.

Diamond

L. M

. 2008. Sexual fluidity: Understanding women’s love and desire. Cambridge, MA: Harvard University Press.

Eliason

M. J.

Radix

McElroy

J. A.

Garbers

Haynes

S. G.

. 2016. The “something else” of sexual orientation: Measuring sexual identities of older lesbian and bisexual women using National Health Interview Survey questions. Women’s Health Issues 26:S71–80.

Eliason

M. J.

Streed

C. G.

Jr . 2017. Choosing “something else” as a sexual identity: Evaluating response options on the National Health Interview Survey. LGBT Health 4:376–79.

10.

Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity in Federal Surveys. 2016. Current measures of sexual orientation and gender identity in federal surveys. Federal Committee on Statistical Methodology. https://nces.ed.gov/FCSM/pdf/buda5.pdf (accessed December 23, 2019).

11.

Fish

J. N.

Hughes

T. L.

Russell

S. T.

. 2018. Sexual identity differences in high-intensity binge drinking: Findings from a US national sample. Addiction 113:749–58.

12.

Galesic

Tourangeau

Couper

M. P.

Conrad

F. G.

. 2008. Eye-tracking data: New insights on response order effects and other cognitive shortcuts in survey responding. Public Opinion Quarterly 72:892–913.

13.

Hatzenbuehler

M. L.

McLaughlin

K. A.

Keyes

K. M.

Hasin

D. S.

. 2010. The impact of institutional discrimination on psychiatric disorders in lesbian, gay, and bisexual populations: A prospective study. American Journal of Public Health 100:452–59.

14.

Heeringa

S. G.

West

B. T.

Berglund

P. A.

. 2017. Applied survey data analysis, 2nd ed. Boca Raton, FL: Chapman and Hall/CRC Press.

15.

Holbrook

A. L.

Green

M. C.

Krosnick

J. A.

. 2003. Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly 67:79–125.

16.

Hughes

McCabe

S. E.

Wilsnack

S. C.

West

B. T.

Boyd

C. J.

. 2010. Victimization and substance use disorders in a national sample of heterosexual and sexual minority women and men. Addiction 105:2130–40.

17.

Jans

Viana

Grant

Cochran

S. D.

Lee

A. C.

Ponce

N. A.

. 2015. Trends in sexual orientation missing data over a decade of the California Health Interview Survey. American Journal of Public Health 105:e43–50.

18.

Katz-Wise

S. L.

2015. Sexual fluidity in young adult women and men: Associations with sexual orientation and sexual identity development. Psychology & Sexuality 6:189–208.

19.

Korchmaros

J. D.

Powell

Stevens

. 2013. Chasing sexual orientation: A comparison of commonly used single-indicator measures of sexual orientation. Journal of Homosexuality 60:596–614.

20.

Lepkowski

J. M.

Mosher

W. D.

Groves

R. M.

West

B. T.

Wagner

. 2013. Responsive design, weighting, and variance estimation in the 2006–2010 National Survey of Family Growth. Vital and Health Statistics 2 158:1–52.

21.

Malhotra

2008. Completion time and response order effects in web surveys. Public Opinion Quarterly 72:914–34.

22.

McCabe

S. E.

Hughes

T. L.

Bostwick

Morales

Boyd

C. J.

. 2012. Measurement of sexual identity in surveys: Implications for substance abuse research. Archives of Sexual Behavior 41:649–57.

23.

McCabe

S. E.

Hughes

T. L.

Bostwick

W. B.

West

B. T.

Boyd

C. J.

. 2009. Sexual orientation, substance use behaviors and substance dependence in the United States. Addiction 104:1333–45.

24.

McCabe

S. E.

Matthews

A. K.

Lee

J. G. L.

Veliz

P. T.

Hughes

T. L.

Boyd

C. J.

. 2018. Tobacco use and sexual orientation in a national cross-sectional study: Age, race/ethnicity and sexual identity-attraction differences. American Journal of Preventive Medicine 54:736–45.

25.

Medley

Lipari

R. N.

Bose

Cribb

D. S.

Kroutil

L. A.

McHenry

. 2016. Sexual orientation and estimates of adult substance use and mental health: Results from the 2015 National Survey on Drug Use and Health. NSDUH Data Review. https://www.samhsa.gov/data/sites/default/files/NSDUH-SexualOrientation-2015/NSDUH-SexualOrientation-2015/NSDUH-SexualOrientation-2015.htm (accessed December 23, 2019).

26.

Meyer

I. H.

2003. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: Conceptual issues and research evidence. Psychological Bulletin 129:674–97.

27.

Miller

Ryan

J. M.

. 2011. Design, development and testing of the NHIS sexual identity question. Questionnaire Design Research Laboratory, Office of Research and Methodology, National Center for Health Statistics. https://wwwn.cdc.gov/qbank/report/Miller_NCHS_2011_NHIS%20Sexual%20Identity.pdf (accessed January 2, 2020).

28.

Mishel

2019. Intersections between sexual identity, sexual attraction, and sexual behavior among a nationally representative sample of American men and women. Journal of Official Statistics 35:859–84.

29.

Ridolfo

Miller

Maitland

. 2012. Measuring sexual identity using survey questionnaires: How valid are our measures? Sexuality Research and Social Policy 9:113–24.

30.

Rosario

Schrimshaw

E. W.

Hunter

Braun

. 2006. Sexual identity development among lesbian, gay, and bisexual youths: Consistency and change over time. Journal of Sex Research 43:46–58.

31.

Savin-Williams

R. C.

Diamond

L. M.

. 2000. Sexual identity trajectories among sexual-minority youths: Gender comparisons. Archives of Sexual Behavior 29:607–27.

32.

Savin-Williams

R. C.

Ream

G. L.

. 2007. Prevalence and stability of sexual orientation components during adolescence and young adulthood. Archives of Sexual Behavior 36:385–94.

33.

Savoca

2000. Measurement errors in binary regressors: An application to measuring the effects of specific psychiatric diseases on earnings. Health Services and Outcomes Research Methodology 1:149–64.

34.

Substance Abuse and Mental Health Services Administration. 2019. Results from the 2018 National Survey on Drug Use and Health: Detailed tables. Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/ (accessed January 2, 2019).

35.

Sykes

Collins

. 1988. Effects of mode of interview: Experiments in the U.K. In Telephone survey methodology, eds. Groves

R. M.

Biemer

P. P.

Lyberg

L. E.

Massey

J. T.

Nicholls

W. L.

Waksberg

. 301–20. New York: Wiley.

36.

Talley

A. E.

Stevens

J. E.

. 2017. Sexual orientation self-concept ambiguity: Scale adaptation and validation. Assessment 24:632–45.

37.

Truman

J. L.

Morgan

R. E.

Gilbert

Vaghela

. 2019. Measuring sexual orientation and gender identity in the National Crime Victimization Survey. Journal of Official Statistics 35:835–58.

38.

Vrangalova

Savin-Williams

R. C.

. 2012. Mostly heterosexual and mostly gay/lesbian: Evidence for new sexual orientation identities. Archives of Sexual Behavior 41:85–101.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.02 MB

0.00 MB

0.01 MB

0.02 MB