Filter Questions in Symptom Assessment Affect the Prevalence of (A)Symptomatic COVID-19 Cases

Abstract

Background

It has been reported that a substantial number of COVID-19 infections are asymptomatic, with both symptomatic and asymptomatic infections contributing to transmission dynamics. Yet, the share of asymptomatic cases varies greatly across studies. One reason for this could be the measurement of symptoms in medical studies and surveys.

Design

In 2 experimental survey studies (total N > 3,000) with participants from Germany and the United Kingdom, respectively, we varied the inclusion of a filter question on whether participants who tested positive for COVID-19 had experienced symptoms prior to presenting a checklist of symptoms. We measured the reporting of asymptomatic (versus symptomatic) COVID-19 infections.

Results

The inclusion of a filter question increased the reporting of asymptomatic (versus symptomatic) COVID-19 infections. Particularly mild symptoms were underreported when using a filter question.

Conclusions and implications

Filter questions affect the reporting of (a)symptomatic COVID-19 cases. To account for such differences in the estimation of population infection rates, future studies should transparently report the applied question format.

Highlights

• Both symptomatic and asymptomatic infections are important for COVID-19 transmission dynamics.

• In previous research, symptoms have been assessed either with or without a filter question prior to presenting a symptom list.

• We show that filter questions reduce the reporting of asymptomatic infections.

• Particularly mild symptoms are underreported when using a filter question.

Introduction

To monitor and control pandemics, accurately assessing infection cases is key. This is also true for the pandemic of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). ¹ The estimation of infection rates oftentimes relies on testing patients who experience symptoms associated with the novel coronavirus disease (COVID-19), as well as their close contacts. ² To estimate the spread of SARS-CoV-2, it is important to consider both symptomatic and asymptomatic infections because infected patients without symptoms have also been shown to transmit COVID-19 ³ and may be especially relevant in disease dynamics. ⁴ Thus, in the absence of effortful and costly testing regimes with random population samples, the prevalence of largely undiagnosed and therefore hidden asymptomatic patients must be extrapolated from the diagnosed patients in the population. This is usually done by surveys asking for COVID-19 symptoms.

Meta-analyses have shown substantial variation between and across such studies in the relative frequency of symptomatic versus asymptomatic COVID-19 infections.^5–7 Whereas some of this heterogeneity may be explained by different sampling strategies, test regimes, and associated differences in sample characteristics, as well as whether presymptomatic and asymptomatic cases are considered separately, another potential source of such variation is response bias (i.e., differences in responses due to, for instance, the questions asked and their respective formats). ⁸ Here, we investigate the impact of one particularly relevant feature of symptom assessment interviews or surveys on the likelihood of reporting symptomatic versus asymptomatic infections: the inclusion of filter questions.

When participants are asked about their symptom experiences, some studies directly present a list of potential symptoms.^9,10 Other studies start with a filter question, asking whether participants have experienced any symptoms.^11,12 Only when participants respond positively to this filter question are they presented with a list of symptoms to choose from; otherwise, symptoms are not recorded. Previous research indicates that filter questions can lead to substantial underreporting, for instance, of mental health symptoms ¹³ and crimes experiences. ¹⁴ Therefore, we conducted 2 experimental survey studies to test the potential impact of filter questions on the reporting of symptomatic versus asymptomatic COVID-19 infections.

Methods

This research obtained ethical clearance from the University of Erfurt’s Institutional Review Board (No. 20200302/20200501). All participants provided informed consent prior to data collection.

The basic set-up of the 2 studies was as follows: participants were first asked whether they had a positive COVID-19 test result (either rapid or polymerase chain reaction [PCR] test). All participants who reported affirmatively were randomly assigned to 1 of the between-participants conditions. In the filter question condition, participants first answered the question, “Did you experience symptoms?” (yes or no). Only when they responded affirmatively were they asked about what type of symptoms they experienced. In the no-filter question condition, participants were immediately asked about their symptoms (if any) without a preceding filter question (for further details, see below). Participants who reported a positive test result and subsequently responded negatively to the filter question or reported no symptoms in the no-filter question condition were coded as asymptomatic.

Study 1

This online study was part of the German COVID-19 Snapshot Monitoring (COSMO) ¹⁵ survey on August 31/September 1, 2022. The sample (N = 998; 51.2% female, 48.5% male, 0.3% other; age: $\overline{x}$ = 46.92, s = 14.74; for more details, see supplementary information) is quota representative for the German adult population regarding age × gender and federal state.

Participants were asked whether they ever had a positive test result in 2022. Afterward, they were assigned to one of the filter question conditions. To indicate their symptoms, participants had to choose symptom(s) they had experienced from a randomized, multiple-choice list. Participants could also indicate that they had experienced no symptoms or that they did not want to answer. The symptom list included typical symptoms associated with different COVID-19 variants, for example, sore throat, fever, and loss of taste/smell (for a full symptom list, see supplementary information).

Study 2

This preregistered online study was conducted with participants from the United Kingdom invited via Prolific on September 7, 2022. Aiming to not limit participation to individuals who had a positive COVID-19 test to avoid sampling bias, it was advertised as a study on “health and diseases.” To increase the test power compared to study 1, we recruited a gender-balanced sample of 2,178 participants (49.5% female, 49.8% male, 0.7% other; age: $\overline{x}$ = 40.33, s = 13.12; for more details, see supplementary information).

Different from the first study, participants were asked whether they had ever a positive COVID-19 test, whether this was confirmed by a PCR test, how often they had been infected with COVID-19, and when the most recent infection was. They were then assigned to one of the filter question conditions. The symptom assessment was based on the most recent infection. The randomized symptom list was extended based on recent studies ¹¹ (see supplementary information). Moreover, participants had to confirm, yes or no, for each symptom to reduce potentially biased responses due to laziness. Finally, participants had to indicate the severity of each symptom they experienced (7-point Likert-type scale ranging from very mild to very severe).

Results

In study 1, n = 344 (34.5%) of the total N = 998 participants reported a positive test result in 2022 and were thus randomly assigned to the experimental conditions. Results revealed a medium-sized effect of the experimental manipulation (Fisher’s exact test: P = 0.026, odds ratio [OR] = 2.67, 95% confidence interval [CI] [1.08, 7.27]). In the filter question condition, 19 of 168 (11.3%) participants with a positive COVID-19 test reported asymptomatic infection and 8 of 176 (4.5%) did in the no-filter question condition. Hence, the results provide preliminary evidence that the presence of a filter question increases the reporting of asymptomatic COVID-19 infections.

In study 2, n = 1,232 (56.6%) of the total N = 2,178 participants reported that they had been infected with COVID-19 since the beginning of the pandemic. Results showed a large effect of the experimental manipulation (Fisher’s exact test: P < 0.001, OR = 7.91, 95% CI [3.52, 20.90]). In the filter question condition, 48 of 594 (8.1%) of the most recent infections reported by each participant but only 7 of 638 (1.1%) in the no-filter question condition were reported to be asymptomatic. Additional regression analyses (see supplementary information) with test type (PCR test versus rapid test versus unknown) and demographics as control variables indicate the robustness of the obtained effect: 1) it was independent of the time of infection (between January 2020 and September 2022) and 2) it was independent of whether a participant was likely infected with the Omicron variant (infections from January 2022) or with a previous variant (infections before January 2022). Finally, we explored differences in the severity of the reported symptoms in the 2 conditions (Figure 1). Participants in the filter question condition reported fewer symptoms ( $\overline{x}$ = 8.91, s = 5.11) than participants in the no-filter question condition ( $\overline{x}$ = 9.56, s = 4.49, Welch’s t test: P = 0.019, Cohen’s d = −0.13, 95% CI [−0.25, −0.02]). Furthermore, participants in the filter question condition reported more severe symptoms ( $\overline{x}$ = 4.28, s = 1.02) than participants in the no-filter question condition ( $\overline{x}$ = 4.14, s = 1.09; Welch’s t test: P = 0.022, Cohen’s d = 0.13, 95% CI [0.01, 0.25]).

Figure 1

Kernel density of self-reported (A) symptom count and (B) symptom severity by experimental condition. Vertical lines indicate mean values.

Discussion

The results indicate that the self-reported prevalence of symptomatic versus asymptomatic COVID-19 infections is substantially affected by question format. Including a filter question on whether individuals experienced symptoms before asking for specific symptoms decreased reports of COVID-19 symptoms. The effect was robust, for instance, across different samples (from Germany and the United Kingdom) and methods of assessing symptoms. It also appears that filter questions led particularly to underreporting of mild symptoms. The latter effect may be a result of a lack of recall for mild symptoms when not prompted to report them.

While it is not possible to draw definitive conclusions on which method of symptom assessment—with or without a filter question—leads to more accurate reporting, it is plausible that the use of a filter question results in underreporting of (mild) symptoms. As a result, it is likely that asymptomatic cases would be overestimated when compared with both the true number of asymptomatic cases and to estimates obtained without the use of a filter question.

Limitations

A potential limitation of our work is that the participants who reported a positive COVID-19 test likely self-selected into testing, either because they had symptoms or were in contact with positively tested others. If experienced symptoms were indeed the reason why they got tested in the first place, our results likely underestimate the difference between the presence versus absence of a filter question as it could be expected in a random-population sample (where testing occurs independent of experienced symptoms).

Conclusions and Implications

Identifying differences between question formats in COVID-19 symptom reporting is important for at least 3 reasons. First, it may lead to different estimations of the prevalence of symptomatic versus asymptomatic infections, which has been the subject of scholarly discussions ¹⁶ and is important to better understand the mechanisms of the infection dynamics. ⁴ Second, when mainly symptomatic patients are tested, it is important to infer the number of asymptomatic infections to properly estimate COVID-19 transmission dynamics. Third, the results warrant caution when making legal rules (e.g., quarantine) contingent on the subjective assessment of symptoms. The subjective nature of what constitutes a relevant symptom and how severe symptoms need to be is likely to render such legislation completely inapplicable.

In conclusion, it is imperative that models of disease spread, which account for asymptomatic cases and not solely rely on PCR test data from random population samples, consider the method of symptom assessment, specifically, whether a filter question was employed or not. Therefore, it is crucial for studies on the prevalence and type of symptoms to clearly disclose the question format in their methodology. We hope that our findings will raise awareness and contribute to greater transparency and standardization in the assessment symptoms.