No one listens to us,we know this,so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic

Abstract

Background:

We created a survey to assess menstrual side effects after COVID-19 vaccination when we noticed news stories that denied or discounted the experiences of tens of thousands of menstruating and formerly menstruating people who reported experiencing bleeding changes. This survey had an unprecedented response hundreds of times higher than we had anticipated (n = 101,824).

Objectives:

We investigated what motivated our sample to participate without remuneration to understand both general motivations for survey participation as well as why this survey captured the interest of so many.

Design:

We used open-ended responses from our online, mixed-method survey collected from April to October 2021.

Methods:

Using computer-assisted qualitative data analysis tools in R, we conducted a thematic analysis on open-ended responses. We used topic modeling to cluster the data, synthesize responses across 22,737 participants, and inform the themes summarizing the responses to “What is your interest in this project?” We compared and contrasted responses across groups (racial identity, ethnicity, gender) to examine whether the themes were representative across the demographic groups in our study.

Results:

The themes that characterized participants’ interest in participating were vaccine effects and women’s/people’s health, personal experience related to the vaccine, and a love for science and data. We compared responses among demographic subgroups to avoid an overfocus on majority group responses and found the themes were reflected across each group. Lastly, we found our themes reflected multiple types of altruism.

Conclusion:

These results were important in showing how emergent research that focuses on the concerns of potential participants can encourage high response rates from both marginalized and majority communities. Inclusive practices and familiarity with the research team built credibility that engendered trust with the public.

Plain language summary

No one listens to us, we know this, so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic

Despite the success of the COVID-19 vaccine rollout, trust in science was put to the test as tens of thousands of people who currently menstruate or people who used to menstruate (e.g., postmenopausal) reported experiencing changes to their menstrual bleeding or new bleeding. News stories and medical professionals denied or discounted their experiences. In response, we created a survey to assess menstrual side effects after COVID-19 vaccination. The survey had an unprecedented response. Therefore, we investigated our sample’s motivations, in particular, because this was unpaid research. We used computer-assisted tools to synthesize open-ended answers to “How did you hear about this project” and “What is your interest in this study?” To understand what led people to participate, we found participants’ interests were vaccine effects and women’s/people’s health, personal experience related to the vaccine, and a love for science and data. We compared responses among demographic subgroups to avoid an overfocus on the majority responses (e.g., cisgender women or White people), and we found our themes reflected multiple types of altruism. These results were important in showing how research that is responsive to public needs encourages high responses from both marginalized and majority communities. We utilized inclusivity and familiarity to build credibility and create online presences to engender trust with the public. Future clinical research can be informed by our findings. We show strategies for doing ethical clinical research by speaking to people from all groups who may be affected.

Keywords

pandemic qualitative research cultural diversity community participation cross sectional survey

Introduction

During the COVID-19 pandemic, the urgency for vaccine rollout to reduce mortality as well as resume day-to-day activities led to an unprecedented level of vaccine development and testing. This expediency to prove vaccine efficacy and safety undoubtedly saved millions of lives.¹ As often happens, when a treatment hits a wider market and thus a much larger sample size, unexpected or unmeasured side effects can start to emerge, from heavy or breakthrough bleeding among menstruating or formerly menstruating people, to rarer adverse events that lead to blood clots, cardiac complications, or long COVID-like sequelae.^2
–4 Early in the vaccine rollout process, we sought to understand whether menstrual changes were occurring among those who received the COVID-19 vaccines, as well as what increased the risk of experiencing said side effects, using an online survey methodology. We were overwhelmed by the engagement of participants: over 100,000 when we originally anticipated only 500–700.³

Our methodology was emergent, responding in real-time to the needs and interests of the public. Since our original work, many other studies have reported similar findings using prospective methodologies and comparing menstrual experiences among those who were and were not vaccinated.^5
–12 But what we observed most strongly, particularly in the open responses of our survey as well as among social media and emailed responses to our work, was that the oversight by clinical trials to not measure or document possible menstrual changes was especially frustrating to respondents and potential respondents. The surprise and early denial by various medical and public health spokespeople of menstrual changes as potential side effects of the COVID-19 vaccines led to what we saw as breakdowns in scientific trust. Moreover, the growing realization that menstruation is rarely remarked upon or measured in clinical trials when the trial does not directly deal with reproductive or contraceptive treatments revealed structural barriers to equitable healthcare.

In the face of so much mistrust and disappointment, we wondered how and why so many chose to participate in our unpaid research on vaccine side effects. We sought to understand the motivations of our participants using qualitative analysis to explore the extent to which high engagement was particular to this moment in time, our study design, or some other factor, to explore whether this work might offer insight into how to produce engaged and robust participant responses to research. Given that there are differential barriers to participation in medical and health research,^13
–16 we especially were interested in racial, ethnic, and gender diverse respondents’ reasons for participating. The research topics addressed were (1) what reasons participants gave for participating and (2) what altruistic motivations underlied their participation. We also explored the ways in which motivations may be different among different demographic groups, highlighting ways to consider improving the recruitment of marginalized populations. In this way, we explore engagement in unpaid online health research among those who are often explicitly excluded or overlooked.

Methods

The current study used data collected online from people after vaccination for COVID-19 to investigate why people engage in survey health research. The data were from a large, mixed-methods survey on menstruation and vaccine side-effects following COVID-19 vaccination.³ The dataset was collected by the third and last authors from April to October 2021 through snowball sampling from social media and news coverage of research addressing menstrual and other vaccine side effects. Eligibility included all vaccinated adults who were currently or formerly menstruating people. In the present study, we used the open-ended responses for qualitative analysis and demographics for a description of the sample and comparing responses. Open-ended responses to two survey questions (how did you hear about this project and what is your interest in this project) were analyzed in all U.S. residents (N = 22,737). We used content analysis and thematic analysis to explore motivations to participate in unpaid health research and examined whether any differences between demographic groups were evident.

The research was designated as exempt for no more than minimal risk by the University of Illinois Urbana-Champaign IRB# 21811 and Washington University in St. Louis IRB# 202104058. All participants provided informed consent before enrollment in the study through the online survey. Data were collected and managed using Research Electronic Data Capture (REDCap), a secure, web-based software platform designed to support data collection for research studies, hosted at the University of Illinois Urbana-Champaign.^17,18 Data processing, sample demographics, and qualitative analyses were conducted in R (Version 4.1.3; R Core Team, 2022).¹⁹ We followed the COREQ Guidelines when preparing the manuscript.²⁰

Sample

The present data consist of 22,737 U.S. adult respondents (Table 1). In the broader project, there were 101,824 respondents in total who were 18 or older, appropriately consented, not hostile (e.g., swearing or entering nonsense words), not duplicated, and vaccinated against SARS-CoV-2 (see OSF for details https://osf.io/3d8ax/). Participants from the dataset (n = 101,824) were limited to those with responses to both open-ended fields (n = 78,570), currently living in the United States (n = 23,212), and responses written in English (n = 22,737) detected using cld2 package in R.²¹ We analyzed people’s responses to demographics (age, race, ethnicity, gender identity) and two open-ended questions (“What is your interest in this project?” and “How did you hear about this project?”). How people heard about the study was included as further description and context of the sample.

Table 1.

Demographics of the sample.

Variables	All		Gender identity
Variables	All		Woman		Nonbinary		Transgender		Additional identity
Age in years, mean (SD)	33.7	(9.8)	34.9	(9.6)	26.1	(6.4)	24.1	(5.1)	26.3	(7.5)
Gender
Woman	20,544	90.4%	19,783	100%	707	36.0%	35	4.3%	19	12.9%
Transgender^a	769	3.4%	0		0		769	93.4%	0
Man^a	434	1.9%	0		0		434	52.7%	0
Nonbinary^a	2,049	9.0%	0		1,584	80.6%	465	56.5%	0
Genderqueer^a or nonconforming	1,017	4.5%	0		732	37.2%	285	34.6%	0
Two-spirit	28	0.1%	0		22	1.1%	<10		0
Race
Black, African, or African American	812	3.6%	758	3.8%	40	2.0%	11	1.3%	<10
Asian	853	3.8%	770	3.9%	59	3.0%	19	2.3%	<10
Native Hawaiian or Pacific Islander	21	0.1%	20	0.1%	<10		0		0
American Indian or Alaska Native	105	0.5%	92	0.5%	<10		<10		<10
White	18,953	83.4%	16,532	83.6%	1,610	81.9%	682	82.9%	<10
Other	443	1.9%	391	2.0%	37	1.9%	11	1.3%	119	81.0%
Multiracial	1,097	4.8%	855	4.3%	158	8.0%	74	9.0%	10	6.8%
Ethnicity
Hispanic/Latinx	1,883	8.3%	1,650	8.3%	152	7.7%	73	8.9%	<10
Non-Hispanic/Latinx	18,547	81.6%	16,139	81.6%	1,607	81.7%	673	81.8%	119	81.0%
Other	955	4.2%	829	4.2%	85	4.3%	34	4.1%	<10
N	22,737		19,783		1,966		823		147

Four hundred fifty-three of the sample were missing race, 1,352 missing ethnicity, and 146 missing gender. Race and ethnicity were cleaned from multiple survey items to be mutually exclusive, so unreported percentages were missing the demographic.

Gender was cleaned for the top categories to be mutually exclusive, but specific identities listed will equal to more than 100%. For the groupings, woman was defined as those who only identified as women; nonbinary was defined as identifying as nonbinary, genderqueer, or gender nonconforming, or two-spirit but not transgender or man; transgender was defined as identifying as man or transgender and did not rule out also selecting nonbinary or genderqueer or other things; additional identity was defined as those who preferred not to say or whose gender they said was not listed but did not select any of the existing categories; and the rest were missing any gender information.

The survey began in response to anecdotal evidence from the research team and others on social media. Journalists reported on the story leading to further sharing of our survey and resulting in large-scale snowball sampling. Media interviews by the research team emphasized the safety of the vaccines and that the survey existed to begin to understand these phenomena while providing an opportunity for the public to hear about the research. The snowball sampling surpassed any expectations or requirements for sample size. In the current qualitative study, we used participants who wrote in English and were from the United States because we had background context on the roll-out of vaccines and knowledge on the healthcare system that we did not have internationally.

Data analysis

We used a mixture of content analysis and thematic analysis to address our main research questions: (1) what were people’s reasons for participation; and (2) what altruistic motives underlied this interest? These strategies were selected based on the relatively short responses and a survey item that directly related to the research questions (see Supplemental Table S1 for workflow).

Content analysis

We used content analysis to summarize, describe, and compare the data using frequencies and proportions of words or word pairs.^22,23 We calculated single-word and word pair frequencies from the open-ended responses to describe the response’s content.²⁴ For example, a single word from the response “from a work colleague” would be “colleague” and a word pair would be “work colleague.” We explain in depth in the Supplemental Figures S1–S5. We compared single-word and word pair frequencies from item responses across demographic subgroups (racial groups, ethnic groups, and gender groups). These steps, describe and compare, were done cyclically at each analysis step to see whether the same concepts were being expressed by all groups sampled.

Thematic analysis

We then used thematic analysis to answer research question 1 (What is your interest in this project?) and build upon the content analysis with a more interpretative approach.²⁵ The workflow for thematic analysis was to relate the responses using topic modeling^26,27 to themes and then compare the responses from different demographic groups.²⁸ Topic modeling finds abstract topics that occur in the open-ended responses. When performing topic modeling, we varied the number of clusters specified to characterize the participant responses. Within each iteration, we used Latent Dirichlet Allocation, which allows every response to have a mixture of topics and each topic to have a mixture of words (see Supplemental Figures S6 and S7).²⁷ The most common words per topic, defined by betas to measure the association of each word with each topic, were used to decide an appropriate number of clusters. The first author read and reviewed the responses that were clustered together until a point of saturation. In this way, simplicity and balance compared to completeness and robustness was used to decide the final solution referring to the final number of themes that characterize the responses.²⁹ Additionally, people were weighted (defined by gamma) as belonging to each theme from 0% to 100%, meaning there were overlapping themes in many responses. Three coherent themes were found after iterating the topic modeling twice and trying two through six clusters.

Finally, responses were read again based on the thematic categorization to classify the altruistic motives present in each theme (research question 2) using typologies from HIV clinical trial research conducted by Chin and colleagues.³⁰ Political altruism is activism in response to an epidemic. Experiential altruism is intimate personal experiences. Community altruism is helping communities one belongs to as defined by them. Professional altruism refers to being or knowing a healthcare professional.³⁰ The other motives from Chin’s typology (i.e., cultural, religious, familial, moral, and existential) could be found in some individual responses, but they did not describe the broader themes. There was one category of motivation not well captured by these typologies and is here labeled vernacular based on Humphrey’s³¹ discussion of the two major altruistic motives. The altruistic vernacular motive was related to “just help,” “contribute to help others,” and the pure benefit of helping others without explicitly incorporating underlying cultural, religious, or existential motives. Specifics on the analysis process are available in the Supplement and R scripts are available online for reproducibility and transparency (https://osf.io/3d8ax/).

Computer-assisted qualitative data analysis software tools for analysis

We used R software (Version 4.1.3; R Core Team, 2022)¹⁹ with the tidytext,²⁷ textdata,³² and topicmodels²⁶ packages. The functions were used to describe the total words in open-ended survey items, compare frequencies by demographics, and relate words throughout the responses.

Results

Descriptive statistics of the sample

The sample analyzed for this paper was on average 33.4 years old, 87% women, 8.6% nonbinary, and 3.6% transgender men or transgender nonbinary people. The racial identities of respondents, using U.S. Census Bureau categories, were mostly White (83.4%), followed by 4.8% multiracial, 3.8% Asian, 3.6% Black, African, or African American, and 0.6% Native Hawaiian, Native American, or Indigenous People. In addition, 8.3% of the sample identified as Hispanic or Latin American.

Participants learned of the survey through several means, largely social media sites, news articles or journal coverage, web searches for symptoms, and friends or trusted sources who directly shared the survey (Figure 1). The majority of participants heard about the survey on social media sites like Tumblr, Twitter (now X), and Reddit. While “friends” is only listed once in the figure as the seventh most common source, it was often friends or trusted sources that were sharing links to the study on social media. Thus, the count of friends in Figure 1 is almost certainly an underestimate of how often people heard about the study from trusted sources/friends due to overlap with the social media and news coverage sources.

Figure 1.

Summary of responses to how people heard about the study. The terms are color-coded by source type: social media (pink), news (purple), internet (orange), and friends (green). The x-axis is the count of people who wrote-in each word.

Most participants reported hearing about the survey from similar sources regardless of their gender identity, racial identity, or ethnicity. Notable exceptions were that no gender diverse people reported using the Chicago Tribune, a specific news source cited by many as where they heard about this study. Many gender diverse people reported Discord (https://discord.com/), which is a social platform that operates primarily as semi-private chat servers. Lastly, Native American or Indigenous persons were likely to specify Dr. Laurel as a source (previously known on Instagram as @kinggutterbaby). In short, direct sharing across multiple platforms led to wide coverage of the study.

What interests and altruistic motivations sparked participation?

We first define the three themes that characterized peoples’ interest in the study, report the distribution of responses across themes, and describe the similarities of demographic characteristics between themes. We then describe each theme as we integrate and interpret responses about interest in participation in the project in relation to altruistic motivations of participation.

The first research topic, which investigated participants’ interests in the study, demonstrated similarity across demographic subgroups (Supplemental Figures S8–S10). A series of correlations (text in one demographic subgroup correlated to another subgroup until all subgroups were compared) supported that the open-ended responses were similar across gender identities (r > 0.81), racial identities (r > 0.94), and ethnic identities (r > 0.98).

The three broad topics underlying participants’ interests in filling out the survey were (1) vaccines and women’s health, (2) personal experience, and (3) science and research. The three themes were not mutually exclusive. We used gamma values to quantify how strongly each response matches each theme. Then, we classified the responses by themes present in order to visualize the distribution of responses across these overlapping themes (Figure 2). See additional representative quotes for each theme with the person context included (Supplemental Table S3).

Figure 2.

Venn diagram showing the overlap of the themes based on classifying participant responses. Probabilities (gamma) were used to classify people into themes. The classifications were defined as gamma >0.60 = “totally,” gamma <0.249 = “other two themes,” gamma >0.4 = “mostly,” and the rest were roughly equivalent probabilities between the three themes. The numbers in the Venn diagram relate to the percentages of people in each classification (N = 22,719).

After classifying each person’s response into these themes, we again described the participant demographics to compare whether people were equally represented across themes. Each theme was roughly equivalent in members from every racial, ethnic, and gender group represented in our study (Supplemental Table S2), and the average age was 33–34 years for each theme category. The only noticeable trend was that Theme 2, which related personal experiences as the motivation to participate, was slightly higher among cisgender women (13%) than gender diverse people (10%).

Theme 1: vaccine effects and women’s/menstrual health

The first theme was composed of words like “vaccine,” “effects,” “research,” “covid,” “women,” “health,” and “menstrual.” These responses tended to be a call to action for research on the vaccine effects in menstruating people. A lot of responses showed concern for the lack of inclusion of women in vaccine research, in health studies, or in science, more broadly. Representative answers on the call for more inclusive health research included:

“I feel research into women’s health is severely under researched, particularly in regard to our reproductive health and immune systems.”

“I like data and supporting ‘women’s’ interests in healthcare (with the understanding that not all women have periods and not all people who have periods are women)”

Not only did participants point out that medicine often ignores health concerns related to vaginal or uterine health, but they also acknowledged across many responses that this extends beyond cisgender women. Sex assigned at birth may be considered an important demographic category to some due to assumed hormonal and biological differences; however, this is increasingly a contested assumption given that genitals at birth do not necessarily correspond to gonads, genes, or hormones. Therefore, it was interesting to observe that participant comments around the inclusion of women often appeared to be intended to open up broader issues of inclusion related to gender minorities.

Other participants called for changes to the way medical research is done:

“Gender equity in medical research. I hadn’t heard that the vaccines could have impacts/side effects on menstruation (tho[ugh] to be honest, I likely would’ve gotten the shots regardless)”

“A[s] a trans person in public health. I am always interested in supporting surveys especially ones that have gender inclusive research methods in mind.”

These responses highlight that participants saw the underrepresentation of vaginal/uterine health concerns as something that could and should be changed. They even give suggestions for how this could take place.

Within this theme, participants expressed a few different altruistic motivations, mainly political altruism (promoting or showing support for a social issue) and vernacular altruism (a broad psychological motive to help).³⁰ The political altruism motive often appeared as participants expressing concern about the lack of regard they see for people who have uteruses. The vernacular motive was seen in Theme 1 through statements like how “I” might contribute or how “I” was not even aware there was a problem. Within these responses, it was clear that a large portion of the participants experienced no menstrual changes themselves and instead were motivated by the importance of drawing attention to these issues, or simply participating to support research on the topic. Across these responses, the burden of participating was deemed to be low while the benefits were deemed high. For instance:

“I feel that more medical studies and research need to focus on women/female identifying people, and I am happy to contribute to that when I am able.”

“As a professional in science and healthcare, I believe in furthering research into experiences that have largely been ignored to the detriment of large portions of society. This was an easy survey to participate in and helps that goal, so why not?”

There were even participants who had been unaware of the possibility of menstrual changes with COVID-19 vaccination who chose to participate anyway:

“I am always glad to see more research into menstruation, and would be glad to have increased knowledge/guidance around menstrual symptoms from vaccination since I had no idea that menstrual side effects were even a possibility.”

In these calls for further research into health issues relevant to menstruation, vaginal, and uterine health, people expressed concern for what is not known. We interpret these findings to mean that the majority of respondents were not blaming the vaccine as the problem, per se, but broader power structures that made it allowable for clinical trials to not include data collection on menstruation and vaginal bleeding. Even people with severe reactions did not express concerns about taking vaccines in the future, instead expressing a desire for the side effects or warnings to be disclosed from the start.

Therefore, those participants whose answers fell into Theme 1 (vaccines and women’s/people’s health) had largely political and vernacular motivations for participation. In particular, respondents expressed motives supporting change or better treatment and knowledge around certain, often overlapping groups (e.g., assigned female at birth, women, people who have uteruses).

Theme 2: personal experiences related to cycles and periods

The second theme was composed of words like “my,” “period,” “experience,” “not,” and “cycle.” These responses discussed people’s own menstrual-related experiences or close friend’s experiences. There was an interest in participating both for those who had menstrual changes and those who did not; they expressed wanting to contribute their experience of changes (or lack thereof) after the COVID-19 vaccination. We noticed “period” being used to describe both the time of bleeding/menstruation and the menstrual cycle more broadly, such that people who may or may not menstruate still used the term “period” to describe their experiences. This is in line with how it is often used colloquially.

Participants whose responses fell into Theme 2 appeared to have experiential (having had an experience personally and wanting to share that information so others may benefit), community (helping others in your community, or others like you), and vernacular (just wanting to help) altruistic motives for becoming involved in the project.³⁰ In this theme, participants expressed anger, fear, or concern for themselves or the people like them whom they heard experienced side effects.

Among those who had side effects, the side effects themselves were often the motivation for participation (i.e., experiential motivation). For example:

“I want women to know about this because I was freaking out and thinking I might have a serious issue and possible cancer etc. we should be allowed to know how the vaccine can [a]ffect a woman. It is seriously important.”

“I want to help women understand the symptoms of the vaccine. Because it is extremely scary for me that no one told me the side effects I could have.”

Others did not experience menstrual changes but wanted to support an overall principle of vaccine safety (i.e., community motivation). For instance:

“I want others who are hesitant about the vaccine to understand all of the possible side effects and help prove that they are usually minimal.”

“I want to make sure this isn’t skewed to show the conclusion that the covid vaccine causes menstrual issues. It’s safe and worth it.”

Within Theme 2, there was large support for the vaccine, alongside concern for other people who might also experience menstrual changes. These also often included strong emotional responses.

“I think it’s bullshit that medications’ effects on menstruation are seldom studied, and in particular something like this vaccine where nearly everyone should be getting it. I want these effects, if any, documented”

“I want other women to know that the vaccine can cause hormonal imbalances and that they should tell them it’s a side effect so they don’t get it by surprise and get scare[d] like I did.”

There was a subset who expressed concerns that these menstrual changes represented effects on fertility, expressing fear for themselves or their teenagers receiving the vaccine. For example, one respondent remarked “I’m curious about the menstrual changes thousands of women have experienced. I want more than one child and I want to make sure this vaccine hasn’t caused permanent adverse changes,” while another wrote “I want to assure the vaccine is safe for females and doesn’t affect fertility. I have 2 teen daughters I want to protect.” These responses were from a small number of respondents (less than 2.5% mention “fertility”). Overall, Theme 2 reflected how a lack of research and unclear messaging may have increased suspicion and vaccine hesitancy among some members of the public.

Theme 3: general interest in and need for research, data, and science on the subject

The third and final theme was composed of words like “want,” “help,” “health,” “research,” “data,” “contribute,” “science,” and “study/ies.” The final theme was made up of responses showing interest purely to contribute data and the love of science, as well as those who specifically were calling for data on the studied subject (menstrual effects and COVID-19 disease and vaccine) or data from understudied groups. Some respondents were in research or medical fields who participated for the sake of science or had first-hand knowledge and wanted to understand.

Participants in Theme 3 (science and research) related to most of the previous motives plus professional altruism. Professional altruism is when a person has specialized knowledge that influences their desire to take part in a study. For example:

“. . .I’m also a scientist and a medical professional and know how important evidence based research is and how women’s health is always a second class citizen. I know everything dealing with COVID is relatively new and any information we can get regarding this topic is going to be beneficial in evidence based medical practice.”

“I’m a scientist myself and will happily contribute my ‘I got vaccinated and nothing remarkable happened’ datapoint.”

There was one additional common response that was not well represented by the predefined motives that encompassed responses showing that the people want to see data that equally portrays the results, for example, “I just wanted to help give my data to have a diverse and unbiased as possible data set.” From Theme 3 this desire to “increase sample size” or “love science” may best be described by Chin and colleagues’³⁰ schema as “other psychological” motives. In this context, it more specifically indicates enthusiasm based on a love of science for the sake of science—perhaps most closely related to vernacular altruism.

In some cases, these responses appear to represent a worry that the sample could be overrepresented with people who did experience menstrual changes. So, for some, it was their lack of menstrual changes, and desire to see that experience well represented in the dataset that motivated participation. We also found that some people explicitly referenced their racial or gender identity as why they felt a need to participate (e.g., being transgender or being Black), again to ensure that their experiences were represented to guarantee a representative and diverse dataset.

Finally, there were some groupings of responses that did not fit into the three main themes but instead suggested that trust in science, or particular scientists, could be a major motivator to participate. For instance, some respondents expressed familiarity with the primary investigator:

“I’m always down to support Kate Clancy and her studies! I thought this one in particular would be important, due to the unique symptoms I had.”

“I’ve followed Dr. Kate Clancy’s research since the early days of Context & Variation. Super interested in the relationship between environment and menstruation.”

Knowledge of and trust in the researcher, as well as news coverage of the research, contributed to reaching and attracting respondents.

Another smaller subset of respondents expressed that they had nowhere else to go to report their menstrual side effects or find answers:

“I wanted somewhere I could report this I tried calling the CDC and got laughed at. . .”

“I’ve been annoyed that the CDC isn’t asking about this. I inputted my experience into vsafe in the hopes they look at it”

“My gynecologist cannot provide an explanation for my prolonged bleeding, nothing strange came up in blood work or ultrasound. Sick of not knowing why. . .”

At the time of data collection, health providers were often unable to explain the menstrual changes respondents were experiencing, and in some cases, our participants shared that they expressly disbelieved their patients. Some respondents expressed feelings of validation when their search engines yielded media coverage of the current survey, and then the survey itself.

Discussion

The COVID-19 pandemic offers many examples of the ways science and public health communication can break down, as well as the role emergent, responsive research like ours can play. COVID-19 was and is a global event that shined light on health inequities across racially and ethnically diverse people in the United States. People who fell ill, required treatment or hospitalization, and died at higher rates were from Black, Hispanic, Native American, and immigrant communities.^33,34 Reasons for the inequity lie in social and structural determinants of health, such as poverty, unemployment or types of employment, stressors from repeated discrimination, and poor access to health services and healthy food.^34,35 While there was data to track inequities by racial identity, ethnicity, and community, gender diverse people were also likely overaffected.³⁶ Gender diverse people often work frontline jobs, facing similar inequities in the social determinants of health and lack of specialized health services experienced by people of color (of course, gender diverse people of color are likely especially overrepresented here).^35,37 COVID-19 continues to expose pervasive problems of racism, sexism, transphobia, and homophobia and their downstream effects on health and healthcare access.

In this way, the pandemic also offers a unique opportunity for the scientific community to evaluate its practices, particularly as they relate to equity, ethics, and scientific communication. The research that we do, as well as how we communicate our results, are all part of the broader narrative of the science we communicate to our colleagues, our funders, and the public. Therefore, in this research, as part of a larger study on vaccine and menstrual side effects following COVID-19 vaccination, we examined the motivations of our participants to understand how they found our work and why they engaged with it. Our analysis showed the feasibility and flexibility of observational, post-clinical work to reach people where they are, especially marginalized populations, by using responsive, online surveys. We learned our survey circulated through trusted sources, like news outlets, friends and family, and existing online communities on social media platforms (e.g., Tumblr, Discord channels, Instagram). This circulation was done by the participants themselves if they deemed the source credible, the need for the work to be high, or the research team to be trustworthy. Showing care for all people’s input—something members of our team worked to do on social media—was one critical way of building source credibility.

We found people’s reasons for participating sorted into three main, non-mutually exclusive themes. These interest themes were vaccine effects and women’s/people’s health, personal experiences related to cycles/periods, and interest in and calls to research/science. We explored the motivations that seemed to underlie responses of each theme; motivations to participate in health research can be altruistic, other-oriented, financial, or self-oriented.³⁸ As our research was unpaid and there were no direct benefits, we focused our analysis on altruistic motivations. A framework of 10 typologies from HIV research³⁰ helped describe the motives seen in our participant’s responses. Specifically, Chin et al.’s³⁰ political, community, experiential, and professional altruism typologies were common in our data, while cultural, religious, familial, moral, and existential altruism were less common across our themes. Our results were also in line with work by Tamlyn et al.¹⁴ on general research participation. Specifically, Tamlyn’s themes of personal/community benefit, interest in the study topic, and compensation were similar to the motivators for research participation reflected in our first two themes. In contrast to Tamlyn et al.’s compensation motive, the fact that our third theme was interest in research for the love of science or research likely reflects the fact that our study did not provide remuneration. However, it also shows that high interest and engagement with research does not necessarily require financial compensation if participants are motivated by alternative altruistic factors. Similar to Lewis,³⁹ our participants expressed that no one usually listens to us, we know this, and so we participated.

The strengths of our research lie in the extensive sampling that occurred because of the widespread coverage of the project, which was the first of its kind to address menstrual side effects and the COVID-19 vaccine.³ Another strength of the present research is its transparency (all scripts and steps of analysis are available on OSF) and the transferability of the research process that helped us arrive at our findings.⁴⁰ Other researchers can use the same processes in a new dataset with the same or similar survey questions. The corresponding limitation of our sampling is that it was a convenience method highly reliant on snowball sampling. However, the survey was so widely covered that we saw pro- and anti-vaccine sentiments, and—importantly to the original study purpose—we had many people who had no side effects related to bleeding or menstruation following the COVID-19 vaccine.³

One limitation of the current computer-assisted analytic approach is that because we did not hand-code we did not capture similar meanings expressed with different word choices. There are drawbacks both to a hand-coding approach that combines linguistically distinct but similar sentiment data and our current approach which uses each word as uniquely meaningful and different. Analytic methodology limitations exist in terms of the use of lists of stop words, manually handling misspellings/typos, and using the full written word over the root words (e.g., abnormality has the root abnormal). In contrast, our analytic approach was able to use a large quantity of data consisting of people expressing similar sentiments repetitively and give an overarching summarization of the responses and themes efficiently.⁴¹

Health research trials and tribulations

Health research—and more specifically clinical trials for vaccines—is often criticized for preference of internal validity to the exclusion of external validity, meaning the studied effects within a trial may not be generalizable to other contexts or the broader population.^42,43 Clinical trials have more eligibility requirements for participation, and thus more exclusions, than other kinds of health research. For vaccine efficacy studies using randomized control trials, eligibility may be limited by age, common medical conditions, common medication prescriptions, and assigned sex at birth.⁴⁴ The COVID-19 vaccine trials by Pfizer and Moderna, the two biggest U.S. vaccine providers, were mostly tested in the United States with few other trial sites.^45,46 For Moderna, efforts to check the efficacy of the vaccines across racial groups were limited to a binarized comparison between non-Hispanic White people and “communities of color,” their terminology that referred to all other categories of racial identity and ethnicity combined into one heterogeneous group.⁴⁵ Racial group comparisons are often shorthand for studying the actual impacts of racial oppression, a major component of social and structural determinants of health, yet these factors were not well addressed.

To translate clinical research to clinical practice, many argue that external validity is necessary, meaning results found within the studied sample should be expected to apply to people not directly in the study.^47,48 External validity relates to generalizability and means once treatment is deemed safe, researchers should be confident that unstudied factors should not interact and lead to harmful, unexpected side effects. However, there is a long history of clinical research having low external validity because of the homogeneity of the original sample and who was excluded.^13,42,49,50 What’s more, people of color, gender minorities, and/or women have experienced in the past, and continue to experience today, significant medical betrayal and mistreatment that decreases their likelihood of research participation.^13
–15 Even when underserved groups are eligible to participate in research, the research design may not consider different lived experiences—as we saw with the lack of anticipation of menstrual changes with these vaccines. Taken together, exclusionary study design and reluctance to participate can contribute to racial, ethnic, and gender disparities in research and its external validity.^16,43

A benefit of opt-in survey-based health research like ours is that there may be fewer barriers to participation compared to randomized control trials, because surveys can be easily accessible and responsive to respondents.⁵¹ In a moment like a pandemic when people are searching for ways to understand their experiences and make informed decisions, as well as be helpful when many may feel helpless, having an avenue to share and to contribute to research was a major motivator in taking our survey. Survey-based health research and other methodologies can follow participants past the original study period and can be expanded to include a more diverse sample of the population. In short, health survey research can be responsive to people’s reactions after a treatment is rolled out widely.

Conclusion

Our participants expressed quite simply that no one usually listens to us, we know this, and so we participated. This sort of community or political motive to participate was found in our marginalized participants, as well as those in the majority (e.g., cisgender women). Social media and trusted news or known sources were able to disseminate health research to the communities at risk of being overlooked who were actively seeking answers. We also showed that people, especially marginalized voices like transgender and gender diverse people, ethnic minorities, and racial minorities, can be reached for post-clinical survey health studies if they trust the research team and source of recruitment. The themes describing why people participated in a COVID-19 and menstrual side effects study were vaccine and people’s/women’s health, personal experience, and science and research. The themes related to altruistic motivations, such as experiential, community, political, professional, and vernacular motives when no direct benefits were offered. Finally, we emphasize that future health research would benefit from adapting clinical trials to include online, transparent, longer follow-ups surveying the public, and appropriately recognizing the scientific and social contributions of research designs beyond randomized clinical trials.

Supplemental Material

sj-docx-2-whe-10.1177_17455057241285189 – Supplemental material for No one listens to us, we know this, so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic

Supplemental material, sj-docx-2-whe-10.1177_17455057241285189 for No one listens to us, we know this, so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic by Eleanor J Junkins, Samar Chehab, Katharine MN Lee and Kathryn BH Clancy in Women’s Health

Supplemental Material

sj-pdf-1-whe-10.1177_17455057241285189 – Supplemental material for No one listens to us, we know this, so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic

Supplemental material, sj-pdf-1-whe-10.1177_17455057241285189 for No one listens to us, we know this, so we participated: Qualitative evidence from menstruation research during the COVID-19 pandemic by Eleanor J Junkins, Samar Chehab, Katharine MN Lee and Kathryn BH Clancy in Women’s Health

Footnotes

Acknowledgements

Not applicable.

Declarations

ORCID iDs

Eleanor J Junkins

Katharine MN Lee

Supplemental material

Supplemental material for this article is available online.

References

Fitzpatrick

Moghadas

Panday

, et al. Two years of U.S. COVID-19 vaccines have prevented millions of hospitalizations and deaths. New York: The Commonwealth Fund, 2022.

Haelle

. Science [cited 21 December 2023]. Why is it so hard to compensate people for serious vaccine side effects?, https://www.nationalgeographic.com/science/article/why-is-it-so-hard-to-compensate-people-for-serious-vaccine-side-effects (2022, accessed 21 December 2023).

Lee

KMN

Junkins

Luo

, et al. Investigating trends in those who experience menstrual bleeding changes after SARS-CoV-2 vaccination. Sci Adv 2022; 8(28): eabm7201.

Rueda-Fernández

Melguizo-Rodríguez

Costela-Ruiz

, et al. The current status of COVID-19 vaccines. A scoping review. Drug Discov Today 2022; 27(11): 103336.

Alvergne

Woon

Male

Effect of COVID-19 vaccination on the timing and flow of menstrual periods in two cohorts. Front Reprod Health 2022; 4: 952976.

Alvergne

Kountourides

Argentieri

, et al. A retrospective case-control study on menstrual cycle changes following COVID-19 vaccination and disease. iScience 2023; 26(4): 106401.

Blix

Laake

Juvet

, et al. Unexpected vaginal bleeding and COVID-19 vaccination in nonmenstruating women. Sci Adv 2023; 9(38): eadg1391.

Edelman

Boniface

Benhar

, et al. Association between menstrual cycle length and coronavirus disease 2019 (COVID-19) vaccination: a US cohort. Obstet Gynecol 2022; 139(4): 481.

Male

Effect of COVID-19 vaccination on menstrual periods in a retrospectively recruited cohort. medRXiv 2021; 2021: 2021-11.

10.

Nazir

Asghar

Rathore

, et al. Menstrual abnormalities after COVID-19 vaccines: a systematic review. Vacunas 2022; 23: S77–S87.

11.

Qashqari

FSI

Dahlawi

Assaggaf

, et al. Effect of the COVID-19 vaccine on the menstrual cycle among females in Saudi Arabia. Ethiop J Health Sci 2022; 32(6): 1083–1092.

12.

Trogstad

Laake

Robertson

, et al. Heavy bleeding and other menstrual disturbances in young women after COVID-19 vaccination. Vaccine 2023; 41(36): 5271–5282.

13.

Yates

Byrne

Donahue

, et al. Representation in clinical trials: a review on reaching underrepresented populations in research. Association of Clinical Research Professionals, https://acrpnet.org/2020/08/10/representation-in-clinical-trials-a-review-on-reaching-underrepresented-populations-in-research (2020, accessed 12 April 2023).

14.

Tamlyn

Tjilos

Bosch

, et al. At the intersection of trust and mistrust: a qualitative analysis of motivators and barriers to research participation at a safety-net hospital. Health Expect 2023; 26(3): 1118–1126.

15.

Byrd

Edwards

Kelkar

, et al. Recruiting intergenerational African American males for biomedical research studies: a major research challenge. J Natl Med Assoc 2011; 103(6): 480–487.

16.

Wilkins

Schindler

Morris

JC.

Addressing health disparities among minority populations: why clinical trial recruitment is not enough. JAMA Neurol 2020; 77(9): 1063.

17.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42(2): 377–381.

18.

Harris

Taylor

Minor

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.

19.

R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2022.

20.

Tong

Sainsbury

Craig

Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19(6): 349–357.

21.

Ooms

. cld2: Google’s Compact Language Detector 2, https://CRAN.R-project.org/package=cld2 (2020, accessed 10 March 2022).

22.

Mayring

Qualitative content analysis. Forum: Qual Social Res 2004; 1(2): 159–176.

23.

Pope

Mays

. Qualitative research in health care, 3rd ed. Malden, Mass: Blackwell Pub./BMJ Books, 2006.

24.

Ryan

Bernard

HR.

Techniques to identify themes. Field Methods 2003; 15(1): 85–109.

25.

Braun

Clarke

Using thematic analysis in psychology. Qual Res Psychol 2006; 3(2): 77–101.

26.

Grün

Hornik

Topicmodels: an R package for fitting topic models. J Stat Softw 2011; 40(13): 1–30.

27.

Silge

Robinson

Tidytext: text mining and analysis using tidy data principles in R. JOSS 2016; 1(3): 37.

28.

Bazeley

Qualitative data analysis: practical strategies, 2nd ed. London: SAGE Publications Ltd., 2021.

29.

Richards

Handling qualitative data: a practical guide. 4th ed. London: SAGE, 2021.

30.

Chin

Berenson

Klitzman

RL.

Typologies of altruistic and financial motivations for research participation: a qualitative study of MSM in HIV vaccine trials. J Empir Res Hum Res Ethics 2016; 11(4): 299–310.

31.

Humphrey

Varieties of altruism—and the common ground between them. Soc Res 1997; 64(2): 199–209.

32.

Hvitfeldt

. Textdata: download and load various text datasets, https://CRAN.R-project.org/package=textdata (2020, accessed 1 May 2022).

33.

Cooney

Who gets hospitalized for COVID-19? Report shows differences by race and sex, https://www.statnews.com/2020/04/09/hospitalized-covid-19-patients-differences-by-race-and-sex/ (2020, accessed 12 April 2023).

34.

Krouse

HJ.

COVID-19 and the widening gap in health inequity. Otolaryngol Head Neck Surg 2020; 163(1): 65–66.

35.

Baah

Teitelman

Riegel

Marginalization: conceptualizing patient vulnerabilities in the framework of social determinants of health—an integrative review. Nurs Inq 2019; 26(1): e12268.

36.

Gibb

DuBois

Williams

, et al. Sexual and gender minority health vulnerabilities during the COVID -19 health crisis. Am J Hum Biol 2020; 32(5): 23499.

37.

Burgess

Batchelder

Sloan

, et al. Impact of the COVID-19 pandemic on transgender and gender diverse health care. Lancet Diabetes Endocrinol 2021; 9(11): 729–731.

38.

Sugarman

Kass

Goodman

, et al. What patients say about medical research. IRB Ethics Hum Res 1998; 20(4): 1.

39.

Lewis

NA.

What counts as good science? How the battle for methodological legitimacy affects public psychology. Am Psychol 2021; 76(8): 1323–1333.

40.

Anfara

Brown

Mangione

TL.

Qualitative analysis on stage: making the research process more public. Educ Res 2002; 31(7): 28–38.

41.

Jackson

Paulus

Woolf

The walking dead genealogy: unsubstantiated criticisms of Qualitative Data Analysis Software (QDAS) and the failure to put them to rest. Qual Rep 2018; 23(13): 74–91.

42.

Borno

Siegel

Ryan

The problem of representativeness of clinical trial participants: understanding the role of hidden costs. J Health Serv Res Policy 2016; 21(3): 145–146.

43.

Kennedy-Martin

Curtis

Faries

, et al. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials 2015; 16(1): 495.

44.

Van Spall

HGC

Toren

Kiss

, et al. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007; 297(11): 1233.

45.

Baden

El Sahly

Essink

, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med 2021; 384(5): 403–416.

46.

Polack

Thomas

Kitchin

, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020; 383(27): 2603–2615.

47.

Averitt

Weng

Ryan

, et al. Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations. NPJ Digit Med 2020; 3(1): 67.

48.

Wong

Steiner

. Moving from What Works to What Replicates: A New Framework for Evidence Based Policy Analysis [Internet]. ZPID (Leibniz Institute for Psychology Information); 2019 [cited 2024 September 20]. https://www.psycharchives.org/handle/20.500.12034/2025

49.

Carroll

Nich

McLellan

, et al. ‘Research’ versus ‘real-world’ patients: representativeness of participants in clinical trials of treatments for cocaine dependence. Drug Alcohol Depend 1999; 54(2): 171–177.

50.

Fisher

Kalbaugh

CA.

Challenging assumptions about minority participation in US clinical research. Am J Public Health 2011; 101(12): 2217–2222.

51.

Gobat

Gal

Butler

, et al. Talking to the people that really matter about their participation in pandemic clinical research: a qualitative study in four European countries. Health Expect 2018; 21(1): 387–395.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

3.65 MB

0.37 MB