Within-Respondent Alignment between Single-Choice and Mark-All-That-Apply Survey Measures for Sexual Identity

Abstract

With the rising disclosure of LGBTQ+ identities, there have been increasing efforts to capture inclusive sexual identity data. Commonly used survey measures are closed ended and single choice (i.e., “select one”) and may be restrictive for sexuality- and gender-diverse respondents. However, more expansive measures may be unwieldy to survey takers and analysts. In this data visualization, the authors compare single-choice and mark-all-that-apply measures for sexual identity. Using survey data (n = 2,511) fielded in the United States and United Kingdom in 2022, the authors analyze within-respondent alignment of a single-choice measure and an expansive mark-all-that-apply measure for sexual identity. Responses were considered “aligned” when respondents provided identical information across both measures. Overall, alignment was 89 percent in the full sample, with the highest rate of alignment occurring among straight respondents. With the mark-all-that-apply measure, 3 percent of the sample selected multiple identities, and about 2 percent selected identities not found in the commonly used single-choice measure.

With growing disclosure of LGBTQ+ identities around the world, there has been increasing interest in capturing sexual identity data. Frequently used closed-ended, single-choice (SC) questions (i.e., “select-one”) are parsimonious (National Academies of Sciences, Engineering, and Medicine 2022) but may capture less or inaccurate information, especially for sexual and gender minorities (Suen et al. 2020). On the flip side, mark-all-that-apply (MATA) measures may provide more accurate information for sexual and gender minorities but increase respondent burden and cause confusion (Smyth et al. 2006).

Several recent studies have evaluated the impacts of question design on sexuality reporting. For instance, one study comparing forced-choice, in which respondents select “yes” or “no” for each response option, and MATA revealed that LGBQ respondents selected more categories in forced-choice than MATA conditions (Brenner and Bulgar-Medina 2018). Another study showed that plurisexual individuals, those attracted to more than one gender, were more likely than monosexuals, those attracted to one, to report secondary sexual orientations (Galupo, Mitchell, and Davis 2015). We build on this work by showing the extent of alignment between MATA and SC, a commonly used question design. We consider responses “aligned” when a respondent provides identical information across different measures.

To examine alignment between SC and MATA sexuality measures, we used original survey data (n = 2,511) collected in 2022 in the United States and United Kingdom. Our SC question replicated commonly used measures on national surveys and had options for “heterosexual or straight”; “gay, lesbian, or homosexual”; “bisexual”; and “prefer not to say.” Our expansive MATA measure had options for “aromantic,” “asexual,” “bisexual,” “fluid,” “gay,” “lesbian,” “pansexual,” “queer,” “questioning or unsure,” “straight (heterosexual),” “prefer not to disclose,” and “additional category/not listed.”

Figure 1 displays response patterns across our SC and MATA measures. The figure is a Sankey diagram consisting of “nodes,” indicating frequency, and “flows” between those nodes, indicating different response pathways. The leftmost and middle nodes represent the SC and MATA measures, respectively. Respondents who selected identities exclusive to the MATA options were placed into “another expansive category,” and respondents who selected multiple options were placed into “multiple categories.” The rightmost nodes show the most common response options for “another expansive category” and “multiple categories.” Further information, including the code and data used to produce this diagram, is provided in the Appendix.

Figure 1.

Sankey diagram displaying within-respondent alignment of single-choice and mark-all-that-apply measures for sexual identity.

Across all sexuality groups, we found an overall alignment rate of 89 percent. Straight respondents had the highest alignment at 95 percent, and “prefer not to disclose” respondents had the lowest alignment at 61 percent. As expected, the expansive MATA measure elicited responses that were not included under the SC measure. We found that 2 percent of respondents selected identities exclusive to the MATA measure, and 3 percent of respondents selected multiple identities. Nonetheless, there were respondents who appeared “inconsistent” across the SC and MATA measures, for instance, selecting “gay, lesbian, or homosexual” in SC and “straight (heterosexual)” (alone) in MATA. This may have resulted from “trolling,” respondent confusion, fatigue, or satisficing. Descriptive tables showing the SC and MATA alignment rates and different MATA category combinations are included in Appendix Tables 1 and 2, respectively. Overall, we conclude that the commonly used SC measure may ignore a small but important group of individuals who have complex, pluralistic sexual identities.

Supplemental Material

sj-docx-1-srd-10.1177_23780231241295789 – Supplemental material for Within-Respondent Alignment between Single-Choice and Mark-All-That-Apply Survey Measures for Sexual Identity

Supplemental material, sj-docx-1-srd-10.1177_23780231241295789 for Within-Respondent Alignment between Single-Choice and Mark-All-That-Apply Survey Measures for Sexual Identity by Dashram Pai, Yan Zhen Zhu and Christina Pao in Socius