Sage Journals: Discover world-class research

Abstract

Social desirability bias is a problem in surveys collecting data on norm violations and compliance. If confronted with sensitive questions, respondents systematically underreport norm violations and overreport norm compliance. This leads to biased survey estimates and poor data quality. To improve the measurement of norm violations and compliance, the item count technique (ICT) has been developed. The ICT anonymizes the question-and-answer process. In their experimental survey (n = 2,510), the authors use the ICT to study norm violations and compliance during the coronavirus disease 2019 (COVID-19) pandemic in Europe. More specifically, they estimate the prevalence of vaccination certificate falsification and self-isolation with COVID-19 symptoms. Estimates obtained using standard direct questioning (DQ; n = 1,006) are compared with ICT estimates (n = 1,504). As a result, the authors find no significant difference between estimates of vaccination certificate falsification (4.8 percent with DQ vs. 4.5 percent with the ICT). At the same time, they find a significant difference between estimates of self-isolation with COVID-19 symptoms (87.7 percent with DQ vs. 76.0 percent with the ICT). Conventional survey measures based on DQ thus likely overestimate the extent of norm compliance in a pandemic such as COVID-19.

Keywords

norm violations norm compliance social desirability bias item count technique COVID-19 pandemic

At the beginning of 2020, the severe acute respiratory syndrome coronavirus-2 began to spread worldwide. By the end of 2021, the number of infections had risen dramatically. One of the first measures to fight the pandemic was the introduction of coronavirus disease 2019 (COVID-19) tests. In many countries, there was a rule or at least a recommendation to self-isolate if one tested positive or had COVID-19 symptoms. Other norms and measures were social distancing, washing one’s hands, and wearing a face mask. One big step in the fight against the pandemic was the rapid development of vaccines. To travel or participate in public life, proof of vaccination became a mandatory requirement in many countries. One example of a proof of vaccination was the introduction of the digital COVID-19 certificate of the European Union (EU) (European Commission 2022). The new rules and norms demanded citizens’ compliance. However, not all citizens followed the new norms (Diekmann 2020). For a survey study it is a challenge to determine empirically the extent of norm violations and compliance with regard to the protection measures during the COVID-19 pandemic.

Reviewing the research literature, there are many empirical studies investigating deviant or compliant behavior in connection with various COVID-19 protection measures. Examples of this research include vaccination hesitancy or refusal (see Al-Qerem and Jarab 2021; Alrabadi et al. 2023; Bendau et al. 2021; Biswas et al. 2021; Dror et al. 2020; Lazarus et al. 2023; Leigh et al. 2022; Shah and Coiado 2022; Troiano and Nardi 2021; Wolter et al. 2022), compliance with the social distancing norm (Becher et al. 2021; Diekmann 2020; Munzert and Selb 2020; Oosterhoff and Palmer 2020; van Rooij et al. 2020), refusal to wash hands (Becher et al. 2021; Oosterhoff and Palmer 2020), and compliance with the obligation to wear face masks (Betsch et al. 2020; Diekmann 2020).

Regarding the falsification of vaccination certificates, there are studies analyzing data from various international dark web marketplaces exploring and describing the buying and selling process, as well as the motives of the actors (Catalani et al. 2023; Childs 2023; Georgoulias et al. 2023). A study in Nigeria identified health care workers’ vaccination hesitancy, corruption, and inadequate salaries as key contributing factors to the circulation of fake COVID-19 vaccination cards (Ali et al. 2024). However, quantitative prevalence estimates for vaccination certificate falsification are rare (Helbing and Krumpal 2024).

Explaining and researching the influence of self-isolation on mental health was the focus of various empirical studies (Ellis, Dumas, and Forbes 2020; Nkire et al. 2021; Sepúlveda-Loyola et al. 2020; Zhao et al. 2020). Some studies investigated attitudes and determinants of norm-compliant behavior in connection with COVID-19 measures such as self-isolation or self-quarantine (Galasso et al. 2020; Kowalski et al. 2020; van Rooij et al. 2020). However, the focus of these studies was more on explaining norm-compliant behavior and less on estimating its prevalence. Comparatively few studies report prevalence estimates with regard to self-isolation during the COVID-19 pandemic. A survey study on self-isolation compliance in Israel concluded that 94 percent of respondents were willing to comply with self-quarantine regulations if their lost wages were compensated (Bodas and Peleg 2020). In a Mexican survey study, 48 percent of respondents reported staying at home when experiencing COVID-19 symptoms (González-Morales et al. 2021). In three survey experiments in Canada, socially desirable answers to sensitive items such as “visiting someone” or “being visited by someone” during COVID-19 lockdown were examined using different questioning techniques (Daoust et al. 2021). Using face-saving question wording, self-reported noncompliance with the COVID-19 lockdown measures increased by up to 11 percent compared with standard question wording. In the face-saving condition, 24 percent of respondents reported that they were visited by someone, and 21 percent reported that they visited someone during COVID-19 lockdown. Using standard question wording, in contrast, respondents substantially underreported norm violations (13 percent had been visited by someone and 10 percent visited someone; see Daoust et al. 2021).

All in all, the effectiveness of public health measures to fight a pandemic depends on the citizens’ compliance with those measures. During the COVID-19 pandemic, new norms of prosocial behavior emerged (Berger and Krumpal 2021; Korn et al. 2020; Tutić, Krumpal, and Haiser 2022), such as the norms to get vaccinated or to self-isolate with COVID-19 symptoms in order to contribute to the collective good of public health. However, if a substantial share of the population free rides and violates these norms, then the success of the public health measures will be at risk. As reviewed earlier, survey studies can in principle estimate the shares of norm violators and compliers in a population. Such information can serve as an indicator of the public acceptance of the new norms and public health regulations. However, social desirability bias is a problem in surveys collecting data on norm violations and compliance (Krumpal 2013; Tourangeau and Yan 2007; Yan 2021). If confronted with such sensitive questions, respondents systematically underreport norm violations and overreport norm-compliant behavior. This leads to biased survey estimates and poor data quality. The item count technique (ICT) was developed to anonymize the data collection process and to reduce social desirability bias in respondents’ self-reports of norm violations and compliance. In our experimental survey study, we use the ICT to estimate the prevalence of vaccination certificate falsification and self-isolation with COVID-19 symptoms in Europe. ICT estimates are compared with estimates obtained using standard direct questioning (DQ) to explore the extent of social desirability bias in conventional survey measures of norm violations and compliance during the COVID-19 pandemic. In the following, the terms norm violations and deviant behavior are used interchangeably and refer to behavior violating a norm (e.g., not self-isolating when experiencing COVID-19 symptoms). In contrast, the term norm compliance describes socially acceptable behavior adhering to a norm (e.g., self-isolating with COVID-19 symptoms).

Background

The Falsification of Vaccination Certificates

The norm-violating behavior investigated in our study is the falsification of vaccination certificates. A vaccination certificate is an official document proving that a person has been vaccinated against a specific infectious disease, in our case COVID-19. Depending on the country and its regulations, vaccination certificates can take various forms (e.g., as a printed vaccination card in paper form or digitally as a QR code on a mobile device). Many countries are using the International Certificate of Vaccination or Prophylaxis (Centers for Disease Control and Prevention 2024). In the EU, each member state is using the digital COVID-19 certificate (European Commission 2022). There are three countries in our study that are not member states of the EU, namely, Switzerland, Iceland, and the United Kingdom. These countries also use a digital COVID-19 certificate (Federal Department of Foreign Affairs 2022; Government of Iceland 2021; Government of the United Kingdom 2023). Thus, it is possible to falsify vaccination certificates in each country in our analysis. We define the falsification of vaccination certificates as all actions that lead to the possession of a vaccination certificate without actually being vaccinated. A popular method of obtaining fake vaccination certificates is to buy them online on the dark web (Childs 2023; Georgoulias et al. 2023). The severity of sanctions for falsifications of vaccination certificates varies by country. In Germany, for example, issuing of fake health certificates is punishable by up to five years’ imprisonment (German Criminal Code §§ 277 and 278), while using fake health certificates is punishable by up to one year in prison (German Criminal Code § 279). In Romania, a person faces up to five years in prison for falsifying vaccination certificates (Heghes 2021). In our study, we cannot provide a comprehensive legal analysis of the criminal liability of falsifying vaccination certificates in all European countries. In general, EU Regulation 2021/953 states that sufficient resources should be made available to prosecute illegal practices in connection with the issuance and usage of the digital COVID-19 certificate. Furthermore, the falsification of vaccination certificates violates social norms. On the one hand faking vaccination certificates is a form of lying and therefore it violates the norm of honesty. On the other hand, faking vaccination certificates prevents the establishment of herd immunity. It should be noted that in our study, no distinction is made between actual falsification and the use of false vaccination certificates.

Self-Isolation with COVID-19 Symptoms

Suppawittaya, Yiemphat and Yasri (2020) distinguished between three types of social restrictions in a pandemic: social distancing, (self-)quarantine and self-isolation. Social distancing means avoiding personal contact with friends and family. Quarantine is mostly used in the context of travel and applies to the separation of people without symptoms but who have been in contact with sick people. Self-isolation refers to the separation of sick individuals from others (Suppawittaya et al. 2020). In addition, self-isolation can also refer to the separation of people who are merely experiencing symptoms. Self-isolation while experiencing COVID-19 symptoms is the second behavior investigated in our study. In contrast to the falsification of vaccination certificates, which is a deviant behavior, self-isolation with COVID-19 symptoms is a norm-compliant behavior. During the COVID-19 pandemic if someone tested positive for a COVID-19 infection, he or she was required to self-isolate from others. The aim was to contain the disease and prevent further infections. In mid-January 2022, all member states of the EU had regulations regarding the isolation of COVID-19 cases, and the prescribed length of the isolation phase varied depending on the specific country (Directorate-General for Health and Food Safety 2021). There was also an official rule to stay at home and self-isolate when experiencing COVID-19 symptoms (European Centre for Disease Prevention and Control 2020). The non-EU countries Switzerland, Iceland, and United Kingdom had similar regulations regarding self-isolation with COVID-19 symptoms (European Observatory on Health Systems and Policies 2024; Federal Office of Public Health 2024; National Health Service 2023). Until today, there still are official rules to self-isolate when experiencing COVID-19 symptoms. The compliance with these rules represents prosocial behavior. If someone with COVID-19 symptoms consciously decides not to self-isolate, he or she will infect other people and thus cause immediate damage to health. This violates the social norm that one should not cause injury to other people through negligence.

The Problem of Social Desirability in Survey Studies

Data on the dark field of norm violations can be collected through surveys. One possibility to gather information is to ask respondents directly to self-report norm-compliant or deviant behavior. However, such direct questions asking about norm violations are often perceived as sensitive by the respondents (see Krumpal 2013; Krumpal and Näher 2012; Tourangeau and Yan 2007). Against this background, we assume that the questions in our study about whether one has falsified their vaccination certificate or about self-isolating when experiencing COVID-19 symptoms are sensitive questions. Answers to such sensitive questions are often distorted by social desirability bias. More specifically, systematic underreporting of deviant behavior (such as faking of vaccination certificates) and overreporting of norm-compliant behavior (such as self-isolation with COVID-19 symptoms) can be expected.

There are multiple ways to counteract social desirability bias in survey responses. Possible solutions discussed in the research literature are forgiving wording of the sensitive questions and sealed-envelope techniques for questionnaires (for an overview, see Krumpal 2013). The data collection strategy of forgiving wording intends to change the social valuation of the sensitive behavior, such that respondents feel encouraged to answer truthfully. Sealed-envelope techniques, in contrast, obscure the connection between the answer to the sensitive question and the identity of the respondent, preventing third parties (such as interviewers, bystanders, or state authorities) from linking specific answers with identifying information. When third parties are the primary source of worry, sealed-envelope techniques can be expected to reduce social desirability bias. An alternative approach to elicit more honest answers to sensitive questions is the anonymization of the question-and-answer process by using so-called dejeopardizing techniques (Lee 1993) such as the randomized response technique (RRT; Warner 1965) or the ICT (Raghavarao and Federer 1979). These special questioning techniques pursue the goal of eliciting truthful answers by increasing the level of anonymity via combining respondents’ answers with random statistical noise (Blair, Coppock, and Moor 2020). Compared with the RRT, the ICT has the advantage that the underlying concept of counting items is quite simple and the method is easy to implement. Only a moderate cognitive burden is imposed on the respondent, likely increasing the respondent’s compliance with the interview protocol. Because of the inconclusive state of research regarding the effects of forgiving wording on social desirability bias (see Näher and Krumpal 2012:1609–13; Tourangeau and Yan 2007:874–75; Yan 2021:121–23) and the high cognitive burden of the RRT (Holbrook and Krosnick 2010), we chose to use the ICT in our study. Previous research shows that compared with standard DQ, the ICT elicits more honest answers to sensitive questions asking about norm violations (Ehler, Wolter, and Junkermann 2021; Li and van den Noortgate 2022). In addition, there is evidence that the ICT can reduce systematic overreporting of socially desirable behaviors. For example, Wolter et al. (2022) showed that the proportion of people self-reporting vaccination against COVID-19 was significantly lower when using the ICT (75 percent) compared with interviews using DQ (85 percent).

To counteract the problem of misreporting to sensitive questions, we use the ICT in our survey to protect the respondents’ privacy and to reduce social desirability incentives. In comparison with direct self-reports, we expect that the ICT reduces underreporting and yields higher prevalence estimates when asking about the falsification of vaccination certificates (a “more-is-better” assumption with regard to norm-violating behavior). Furthermore, we expect that the ICT reduces overreporting and yields lower prevalence estimates when asking about self-isolation with COVID-19 symptoms (a “less-is-better” assumption with regard to norm-compliant behavior). Details on the ICT and its implementation are described in the next section.

Methods

Participants and Study Design

We collected data with an online survey in July 2024. Study participants from 27 European countries were recruited via the panel online provider Prolific (www.prolific.com). As subjects from the United Kingdom are overrepresented in the Prolific online panel, we set a limit for participants from the United Kingdom to 200. All interviews were conducted in English. In the Prolific online panel, participants are not directly invited to a study. Instead, a study appears in the feed for eligible participants until the targeted sample size is reached. To ensure high data quality, we included three attention check questions in our survey (see Appendix A1). Overall, 2,513 participants completed the interviews. Three participants were excluded because they were not from Europe. This leaves us with the final sample size of 2,510 participants. The participants’ distribution across countries can be found in Table A2 in the Appendix. Most participants reside in Poland (n = 367) followed by Portugal (n = 345) and Italy (n = 285). A descriptive analysis of the sample can be found in Table A3 in the Appendix. The subjects in our sample are on average middle aged (M = 32.5 years, SD = 10.6). About two thirds (66.4 percent) of the participants have academic degrees. Of all participants in the final sample, 59.0 percent identify as male and 39.6 percent as female; 1.5 percent of the participants describe themselves as diverse. We compare the ICT with standard DQ. At the beginning of the interviews, participants were randomly assigned to one of the two experimental groups: about 40 percent were assigned to the DQ control group (n = 1,006) and about 60 percent to the ICT group (n = 1,504). The ICT group was oversampled because the ICT is statistically less efficient than standard DQ, and a larger sample size is required to increase statistical power (Krumpal et al. 2018).

DQ

In the DQ group, the two key items measuring norm violations and norm-compliant behavior were stated as follows (dichotomous items; 1 = yes; 0 = no):

Falsification of a COVID-19 vaccination certificate: “Have you ever acquired a COVID-19 vaccination certificate without being vaccinated?”

Self-isolation while experiencing COVID-19 symptoms: “Have you practiced self-isolation with COVID-19 symptoms?”

The ICT

The basic principle of the ICT is that subjects respond to several items in a list at once and count their affirmative responses to the whole item list. This anonymizes the individual response to the sensitive item of interest (Raghavarao and Federer 1979). Compared with other dejeopardizing techniques, such as the RRT, the ICT is intuitively understandable and can easily be implemented in self-administered survey modes, such as online surveys (Quatember 2023). To further increase statistical power, we use the double-list design of the ICT in our study (see Droitcour et al. 2004; Krumpal et al. 2018). To implement the ICT double-list design, the ICT group is divided into two subgroups via randomization (subgroups A and B; all items are dichotomous). In the first question block, subgroup A receives a list with a set of nonsensitive items plus the sensitive item (long list 1 [LL1]); subgroup B receives a list with the same set of nonsensitive items but without the sensitive item (short list 1 [SL1]). Respondents in both subgroups are then requested to report the total number of items that apply to them (i.e., the number of “yes” answers), without responding to each item separately. In the second question block, subgroup A receives another list with a different set of nonsensitive items (SL2; SL1 ≠ SL2); subgroup B also receives the other list with the different set of nonsensitive items plus the sensitive item (LL2). In short: SL2 in subgroup A corresponds to LL2 in subgroup B, apart from the sensitive item. Here again, respondents in both subgroups are asked to report the total number of “yes” answers, without answering to each item separately. Using such a design, the individual answer to the sensitive item remains secret (unless all or none of the answers to the list are “yes”). Compared with the single list ICT, the effective sample size is doubled in the ICT double-list design, because respondents from both subgroups respond to a long list including the sensitive item, which increases the statistical power. Table 1 shows an application example of the ICT double-list design with regard to the measurement of the falsification of a COVID-19 vaccination certificate.

Table 1.

Example of the Item Count Technique Double-List Design.

Subgroup A	Subgroup B
LL1 I wear glasses. Hospitals should be privatized. I smoke cigarettes regularly every day. I drink coffee regularly. I have acquired a COVID-19 vaccination certificate without being vaccinated.	SL1 I wear glasses. Hospitals should be privatized. I smoke cigarettes regularly every day. I drink coffee regularly.
SL2 The government has guided my country well through the pandemic. I have a chronic illness. I have been involved in a car accident once. I wear contact lenses.	LL2 The government has guided my country well through the pandemic. I have a chronic illness. I have been involved in a car accident once. I wear contact lenses. I have acquired a COVID-19 vaccination certificate without being vaccinated.

Note: COVID-19 = coronavirus disease 2019; LL = long list; SL = short list.

The prevalence of the sensitive behavior can be estimated on the basis of mean differences between subgroups A and B (see Droitcour et al. 2004; Kirchner et al. 2013; Krumpal et al. 2018). In the ICT double-list design, two separate prevalence estimates of the sensitive behavior are calculated first (one for each question block):

{\hat{π}}_{1} = {\bar{x}}_{L L 1} - {\bar{x}}_{S L 1}

and

{\hat{π}}_{2} = {\bar{x}}_{L L 2} - {\bar{x}}_{S L 2} .

In a subsequent step, the two separate estimates are combined to obtain the double-list estimator:

\begin{matrix} {\hat{π}}_{D L} = \frac{{\hat{π}}_{1} + {\hat{π}}_{2}}{2} \\ = \frac{({\bar{x}}_{L L 1} - {\bar{x}}_{S L 1}) + ({\bar{x}}_{L L 2} - {\bar{x}}_{S L 2})}{2} \\ = \frac{({\bar{x}}_{L L 1} - {\bar{x}}_{S L 2}) + ({\bar{x}}_{L L 2} - {\bar{x}}_{S L 1})}{2} . \end{matrix}

The sampling variance of the double-list estimator is

\begin{matrix} V a r ({\hat{π}}_{D L}) = \frac{V a r ({\hat{π}}_{1}) + V a r ({\hat{π}}_{2}) + 2 C o v ({\hat{π}}_{1}, {\hat{π}}_{2})}{4} \\ = \frac{V a r ({\bar{x}}_{L L 1} - {\bar{x}}_{S L 2}) + V a r ({\bar{x}}_{L L 2} - {\bar{x}}_{S L 1})}{4} . \end{matrix}

Although theoretically, there is no limit on the length of the lists, they should be kept as short as possible because cognitive load of summing up the single responses increases with the length of the lists and statistical efficiency decreases (Krumpal et al. 2018). Thus, the maximum length of the short list should be no longer than three to five items (Droitcour et al. 2004; Krumpal et al. 2015). Furthermore, to decrease the likelihood of ceiling or floor effects (Junkermann 2022; Kirchner et al. 2013; Kuha and Jackson 2014), the short lists should contain both low- and high-prevalence items. With regard to our study focusing on two key sensitive items, each respondent in the ICT group answered to four lists in total. At the end of each list, respondents were asked to indicate the number of statements that apply to them, but not which statements. Assuming that the respondents trust this offer of answer anonymization, it can be hypothesized that the ICT will elicit more self-reported norm violations and less norm-compliant behavior than standard DQ. The exact wording of the ICT instructions and all the lists used in our study to collect data with regard to the two key sensitive items are documented in the Appendix (see Tables A4 and A5).

To estimate the prevalence, the sampling variance and the standard error on the basis of the ICT double-list estimator, the ictreg command from the R package list¹ was used (Blair and Imai 2012; Imai 2011). Using the ictreg command a linear regression model consisting only of the dependent variable (intercept-only-model) was calculated. This procedure is mathematically identical to the difference-in-mean estimator (see foregoing discussion). To calculate the differences between the experimental groups (including standard errors) the predict.ictreg command was used in the next step (Blair and Imai 2012; Imai 2011). Here the ICT estimates based on the linear model and DQ estimates based on a logistic model were passed to this command. The results are discussed in the next section.²

Results

The prevalence estimates of vaccination certificate falsification and self-isolation with COVID-19 symptoms are shown in Table 2. In the DQ group, 4.8 percent of the respondents admitted to vaccination certificate falsification. The ICT estimate is 4.5 percentage points. The difference between the estimates of vaccination certificate falsification is about 0.3 percentage points and is statistically not significant. As a result, we find no evidence of systematic underreporting of this norm-violating behavior in our study. With regard to the norm-compliant behavior, self-isolation with COVID-19 symptoms, the DQ estimate is 87.7 percentage points and the ICT estimate is 76.0 percentage points. This difference of about 11.7 percentage points between estimates of the two experimental groups is statistically significant (p < .05). We can state therefore that conventional survey measures of norm-compliant behavior are susceptible to overreporting bias and estimates on the basis of DQ overestimate the extent of norm compliance in a pandemic such as COVID-19.

Table 2.

Prevalence Estimates of Vaccination Certificate Falsification and Self-Isolation with COVID-19 Symptoms by Experimental Group.

	ICT	DQ	Difference (ICT − DQ)
Vaccination certificate falsification	4.5%(3.4)	4.8%(0.7)	−0.3%(3.4)
Self-isolation with COVID-19 symptoms	76.0%(3.1)	87.7%(1.0)	11.7%*(3.2)
n	1,504	1,006	—

Note: Values in parentheses are standard errors. COVID-19 = coronavirus disease 2019; DQ = direct questioning; ICT = item count technique.

p < .05 (two-sided z test).

Next, we conducted subgroup analyses of the difference between the ICT and DQ by the respondents’ gender,³ academic degree, age, attitudes toward COVID-19 restrictions in general, and attitudes toward the sensitive behavior specifically. Because of the limited statistical power of the ICT estimates, we restrict our subgroup analyses to bivariate comparisons. The results are shown in Tables 3 and 4.

Table 3.

Subgroup Analysis: Reported Vaccination Certificate Falsification by Experimental Group.

	ICT	DQ	Difference (ICT − DQ)	n
Male	8.8%(4.3)	5.7%(1.0)	3.1%(4.4)	1,480
Female	−2.2%(5.3)	3.8%(1.0)	−6.0%(5.4)	993
Academic degree	7.1%(4.1)	5.4%(0.9)	1.7%(4.1)	1,667
No academic degree	−0.8%(5.8)	3.9%(1.1)	−4.7%(5.9)	843
Younger	3.2%(4.0)	5.8%(1.0)	−2.6%(4.1)	1,542
Older	6.5%(5.6)	3.4%(0.9)	3.1%(5.7)	968
Positive attitude toward COVID-19 measures	7.2%(3.7)	5.0%(0.8)	2.2%(3.8)	2,019
Negative attitude toward COVID-19 measures	−6.8%(7.6)	4.3%(1.5)	−11.1%(7.7)	491
Positive attitude toward faking vaccination certificate	22.6%(10.8)	10.9%(3.0)	11.7%(11.2)	281
Negative attitude toward faking vaccination certificate	2.2%(3.5)	4.1%(0.7)	−1.9%(3.6)	2,229

Note: Values in parentheses are standard errors. “Younger” refers to respondents younger than the average age in the study sample, and “older” refers to respondents older than the average age in the study sample (M = 32.5 years, SD = 10.6 years). COVID-19 = coronavirus disease 2019; DQ = direct questioning; ICT = item count technique.

p < .05 (two-sided z test).

Table 4.

Subgroup Analysis: Reported Self-Isolation with COVID-19 Symptoms by Experimental Group.

	ICT	DQ	Difference (ICT − DQ)	n
Male	70.9%(3.9)	84.7%(1.5)	−13.8%*(4.2)	1,480
Female	84.1%(4.9)	91.8%(1.4)	−7.7%(5.1)	993
Academic degree	79.4%(3.7)	89.7%(1.2)	−10.2%*(3.9)	1,667
No academic degree	69.2%(5.3)	84.0%(2.0)	−14.8%*(5.6)	843
Younger	73.2%(3.9)	89.2%(1.3)	−16.0%*(4.1)	1,542
Older	80.6%(5.0)	85.3%(1.8)	−4.8%(5.3)	968
Positive attitude toward COVID-19 measures	79.8%(3.4)	90.3%(1.0)	−10.6%*(3.5)	2,019
Negative attitude toward COVID-19 measures	60.7%(7.3)	76.9%(3.0)	−16.2%*(7.9)	491
Positive attitude toward self-isolation	80.0%(3.2)	91.0%(1.0)	−11.0%*(3.4)	2,206
Negative attitude toward self-isolation	46.9%(9.3)	64.1%(4.3)	−17.2%(10.3)	304

p < .05 (two-sided z test).

With regard to the falsification of vaccination certificates, there are no statistically significant ICT-DQ differences in the considered subgroups (see Table 3). Descriptively, the largest ICT-DQ difference can be seen between respondents who report a positive attitude toward faking vaccination certificates, with a difference of 11.7 percentage points. Beyond that, we cannot find any meaningful and consistent response patterns across the specific subgroups. Note that some values in the first column (ICT) in Table 3 contain negative estimates (e.g., −2.2 percentage points for female respondents). Of course a negative prevalence does not make sense, but the ICT estimator can be negative by construction. In the case of a low true prevalence, negative estimates may occur by chance. For all negative values in the ICT group, the subgroup specific ICT estimates are not significantly different from zero.

With regard to self-isolation with COVID-19 symptoms, most of the ICT-DQ differences are statistically significant (p < .05) in the subgroups under consideration (see Table 4). Descriptively, the ICT estimates are consistently lower than the DQ estimates in all subgroup comparisons. Looking at the specific subgroups, we can see comparatively large differences for male respondents (13.8 percentage points), respondents with no academic degree (14.8 percentage points), younger respondents (16.0 percentage points), respondents with negative attitudes toward COVID-19 restrictions (16.2 percentage points), and respondents with negative attitudes toward self-isolation (17.2 percentage points). From a social desirability perspective, these results are plausible: Especially younger people often underestimated their personal risk of a COVID-19 infection. At the same time, they believed that COVID-19 is dangerous for society (Franzen and Wöhner 2021). Faced with this dilemma, the pressure from social desirability norms was probably higher for the younger respondents leading to considerable overreporting of norm-compliant behavior in this subgroup. Furthermore, those respondents with negative attitudes toward COVID-19 restriction in general or toward self-isolation in particular were more likely to overreport their own norm-compliant behavior. Overall, we find a robust and consistent pattern of misreporting and upwardly biased estimates of self-isolation with COVID-19 symptoms in all subgroups if DQ is used to collect data.⁴

Discussion

During the COVID-19 pandemic, new social norms emerged (Berger and Krumpal 2021; Korn et al. 2020; Tutić et al. 2022), such as the norm to self-isolate with COVID-19 symptoms and the norm to use proof of vaccination (e.g., as a printed vaccination card in paper form or digitally as a QR code on a mobile device) in order to travel or participate in public life. To fight the pandemic and contribute to the collective good of public health, citizens were expected to comply with these new norms and regulations. Survey studies can in principle monitor the extent of norm compliant and deviant behavior in a society. Such information can be used to evaluate the effectiveness of public health measures and study the public acceptance of those measures. But it should come as no surprise that estimates on norm violations and compliance differ depending on the method of data collection (Krumpal 2013; Tourangeau and Yan 2007; Yan 2021). Confronted with direct questions, respondents systematically overreport norm-compliant behavior and underreport norm-violating behavior. In our study, estimating the prevalence of vaccination certificate falsification and self-isolation with COVID-19 symptoms in Europe, we used the ICT to anonymize the question-and-answer process and elicit more honest answers to the sensitive questions. ICT and DQ estimates were compared to study the extent of social desirability bias in our estimates of the norm violating and compliant behavior. As a result, we find no evidence with regard to underreporting of norm-violating behavior. About 4 percent to 5 percent of respondents stated that they had falsified their vaccination certificate, with only a small and statistically insignificant difference between ICT and DQ estimates falsification (4.5 percent with the ICT vs. 4.8 percent with DQ). At the same time, a substantial and statistically significant difference was found between ICT and DQ estimates with regard to the reports of self-isolation (76.0 percent with the ICT vs. 87.7 percent with DQ). Conventional survey measures based on DQ thus likely overestimate the extent of norm compliance in a pandemic such as COVID-19.

Comparing our results with those of former studies, our prevalence estimates of vaccination certificate falsification are slightly higher than estimates in a survey study by Helbing and Krumpal (2024). Focusing on different forms of norm violations during the COVID-19 pandemic in Germany, Helbing and Krumpal estimated the overall prevalence for vaccination certificate falsification at about 3 percentage points. In addition, they compared standard DQ with the ICT to evaluate social desirability bias in the respondents’ self-reports. As a result, slightly more respondents self-reported vaccination certificate falsification with the more anonymous ICT (3.5 percentage points) than in the interviews using DQ (2.6 percentage points). Similar to our study, this difference was not statistically significant. With regard to the prevalence estimation of self-isolation, González-Morales et al. (2021) reported a much lower estimate of norm-compliant behavior in Mexico, with 48 percent of the study participants willing to self-isolate when experiencing COVID-19 symptoms. It is important to note that this study has limited comparability with our research because of different target populations (Mexican society vs. Europe) and different survey periods. Collecting data using a face-saving questioning technique, Daoust et al. (2021) reported up to 76 percent of respondents who received no visits and 79 percent who did not visit someone during COVID-19 lockdown. These estimates are similar to the ICT estimates of self-isolation obtained in our study. Note, however, that this comparison is also limited because visiting someone during COVID-19 lockdown is not exactly the same as not self-isolating with COVID-19 symptoms.

Regarding the methodological discussion about researching sensitive topics, our study confirms the effectiveness of the ICT in reducing systematic overreporting of socially desirable, norm-compliant behavior. Our findings support the results of Wolter et al. (2022), who investigated social desirability bias in the respondents’ self-reported COVID-19 vaccination status. Comparing the ICT and DQ in an experimental survey, they found significantly lower ICT estimates of COVID-19 vaccination coverage compared with the conventional DQ method. This result was interpreted as evidence that conventional survey studies likely overestimate norm-compliant behavior in a pandemic because of misreporting by survey respondents and that the more anonymous ICT reduces such misreporting. Survey designers who aim to estimate the prevalence of socially desirable behaviors in a pandemic could therefore benefit from using the ICT in their research studies (Wolter et al. 2022).⁵ With regard to the socially undesirable behavior investigated in our study, in contrast, the question arises as to why no significant difference between the ICT and DQ estimates of vaccination certificate falsification could be found. The low prevalence estimates in both experimental groups indicate that vaccination certificate falsification is a behavior that is rather rare in society. We suspect that the ICT and DQ estimates cannot be distinguished statistically because of a lack of precision of the ICT estimator in estimating a small prevalence that is close to zero (Ahlquist 2018; Wolter and Laier 2014), not because it is not relevant from a social desirability perspective.⁶ Thus, the ICT should only be used with caution when investigating behaviors expected to be rather rare in the target population.

Finally, we must note limitations of our research and outline future research perspectives. First, our study sample covers a broad range of respondents from different European societies. Nonetheless, the sample is not representative of the European general population, because the Prolific online panel from which the respondents were recruited cannot be considered representative of the general population in the sense of a complete official register. The ICT proved useful in reducing social-desirability bias in our study sample. However, we must be careful not to prematurely generalize the absolute values of the prevalence estimates of norm-compliance and deviance to other population groups.⁷ In particular, a replication of our method experiment in a general population survey would be a sensible next step for evaluating the robustness of our results.

Second, a limitation of the ICT is that larger sample sizes are necessary to achieve the same level of statistical precision compared with DQ. The low statistical efficiency of the ICT estimator and the higher data collection costs of increasing the sample size can be justified only if the amount of reduction in social desirability bias by the ICT is large. Thus, the use of the ICT does only make sense for sensitive questions in which the response bias is likely to be severe. Furthermore, the low statistical precision of the ICT is particularly problematic for items with a very low (or very high) prevalence (as our results for vaccination certificate falsification illustrate). In these cases, significant differences between ICT and DQ estimates of the sensitive behavior can hardly be detected (Ahlquist 2018; Wolter and Laier 2014).

Another limitation is that the ICT may not be appropriate if the central research goal is to study psychological mechanisms or individual biases and link those biases to behaviors toward others (e.g., in research on racial, gender and other forms of discrimination). This is because the ICT does not address all sources of social desirability bias. In particular, because the ICT does not hide answers to sensitive questions from the respondents themselves, the ICT will not reduce social desirability bias resulting from self-deception or self-identity concerns. Moreover, although the ICT can provide more accurate population-level prevalence estimates, it has clear limitations for individual-level inferences, i.e., estimating differences in prevalence rates or differences in social desirability bias across subgroups defined by individual-level characteristics and linking those differences to behaviors toward others (Blair et al. 2020).

Conclusion

Our study shows that conventional survey measures based on DQ systematically overestimate the extent of norm compliance in a pandemic such as COVID-19 and that the use of dejeopardizing data collection techniques such as the ICT, despite their limitations, can be beneficial for survey data collection (Krumpal et al. 2015; Lee 1993). Follow-up survey studies are invited to take our research as an inspiration to further investigate norm-compliant and deviant behavior in possible future pandemics or other social crisis situations. In this context, the effectiveness of other data collection approaches and methods of response anonymization on social desirability bias in sensitive self-reports could be further explored and compared with conventional survey methods (Yan 2021).

Supplemental Material

sj-docx-1-srd-10.1177_23780231251328202 – Supplemental material for Social Desirability Bias in Measures of Norm Violations and Compliance during the COVID-19 Pandemic: Results of an Experimental Survey in Europe

Supplemental material, sj-docx-1-srd-10.1177_23780231251328202 for Social Desirability Bias in Measures of Norm Violations and Compliance during the COVID-19 Pandemic: Results of an Experimental Survey in Europe by Alexander Helbing and Ivar Krumpal in Socius

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the German Research Foundation via the Heisenberg program (GEPRIS project number 497852959) and by the Open Access Publishing Fund of Leipzig University.

ORCID iD

Alexander Helbing

Ivar Krumpal

Data Availability

Data and replication code for the analysis can be found at .

Supplemental Material

Supplemental material for this article is available online.

1

For detailed information on the R package list see, .

2

R code and data to replicate our analysis can be found at .

3

In our survey, a total of 37 respondents stated that they were diverse when asked about their gender. We excluded these cases from the subgroup analysis because the number of cases is too small to allow a meaningful statistical analysis and conclusion with regard to this group. The exclusion of these cases is therefore for methodological reasons; no one should be discriminated against.

4

In additional analyses, we calculated ICT-DQ differences separately by the health care systems (countries with the Beveridge model vs. countries with the Bismarck model) and separately by the strictness of the pandemic response measures (countries with high strictness vs. countries with low strictness). All in all, the ICT-DQ differences across country groups are similar in size and from a social desirability perspective we find no evidence for a clear interaction pattern between countries’ characteristics and the social desirability norms in these countries. The results of these additional analyses can be found in the Appendix (see ).

5

We recommend the use of the double-list design of the ICT. In comparison with the study of Wolter et al. (2022), who used the single list ICT, we used the double-list ICT, which leads to more precise prevalence estimates (Krumpal et al. 2018). Consider our item measuring self-isolation with COVID 19 symptoms (ICT prevalence estimate: 76 percent with a standard error of 3.1; n = 1,504) in contrast to the item in the study by Wolter et al. (p. 5, ) measuring vaccination coverage (ICT prevalence estimate: 75 percent with a standard error of 4.3; n = 2,115). Comparing the two results (both items are dichotomous), it is evident that despite the smaller sample size in our study, the double-list estimate produces a considerably smaller standard error than the single list estimate in the study by Wolter et al.

6

The topic whether someone falsifies a COVID-19 vaccination certificate has a clear normative dimension. As explained in the introduction section, there was an illegal market for fake vaccination certificates on the dark web (Childs 2023; ), and using a fake vaccination certificate in daily life violated legal and social norms.

7

However, because participants were randomly assigned to either the ICT or the DQ split, we can compare estimates between the experimental groups in our study.

Author Biographies

Alexander Helbing (he/him) studied sociology at the University of Leipzig (BA and MA). He is currently a research associate and doctoral student in the Department of Sociology at the Chemnitz University of Technology. Before that, he was a research assistant in the Department of Sociology at the University of Leipzig and at the Fraunhofer Center for International Management and Knowledge Economy. His research interests are deviant behavior and attitudes in relation to health and medicine (in particular alternative medicine, refusal of vaccination, and the active rejection of conventional medicine).

Ivar Krumpal (he/him) is a senior researcher and holds a Heisenberg Position of the German Research Foundation in the Department of Sociology at the University of Leipzig. His research interests are social norms, deviance, social desirability, medical sociology and decision behavior in moral dilemma situations. His research has recently appeared in The Lancet Digital Health, Frontiers in Sociology, the Journal of Health and Social Behavior and SAGE Open.

References

Ahlquist

John S.

2018. “List Experiment Design, Non-strategic Respondent Error, and Item Count Technique Estimators.” Political Analysis 26(1):34–53.

Ali

Victor E.

Asika

Marvellous O.

Elebesunu

Emmanuel E.

Agbo

Chioma

Antwi

Maxwell H.

2024. “Cognizance and Mitigation of Falsified Immunization Documentation: Analyzing the Consequences for Public Health in Nigeria, with a Focus on Counterfeited COVID-19 Vaccination Cards: A Case Report.” Health Science Reports 7(2):e1885.

Al-Qerem

Walid A.

Jarab

Anan S.

2021. “COVID-19 Vaccination Acceptance and Its Associated Factors among a Middle Eastern Population.” Frontiers in Public Health 9:11632914.

Alrabadi

Nasr

Bany-Melhem

Shouq

Alzoubi

Karem H.

Alzoubi

Osama O.

Masadeh

Majd

Abuhammad

Sawsan

Harun

Sabariah N.

2023. “COVID-19 Vaccination Hesitancy: A Review of the Literature and Recommendations.” Current Reviews in Clinical and Experimental Pharmacology 19(1):26–41.

Becher

Michael

Stegmueller

Daniel

Brouard

Sylvain

Kerrouche

Eric

. 2021. “Ideology and Compliance with Health Guidelines during the COVID-19 Pandemic: a Comparative Perspective.” Social Science Quarterly 102(5):2106–23.

Bendau

Antonia

Plag

Jens

Petzold

Moritz B.

Ströhle

Andreas

. 2021. “COVID-19 Vaccine Hesitancy and Related Fears and Anxiety.” International Immunopharmacology 97:107724.

Berger

Roger

Krumpal

Ivar

. 2021. “Sociology in Times of Pandemic: Metatheoretical Considerations and the Example of the COVID-19 Crisis.” Pp. 67–86 in Rationality in Social Science, edited by I. Krumpal. Wiesbaden, Germany: Springer Fachmedien Wiesbaden.

Betsch

Cornelia

Korn

Lars

Sprengholz

Philipp

Felgendreff

Lisa

Eitze

Sarah

Schmid

Philipp

Böhm

Robert

. 2020. “Social and Behavioral Consequences of Mask Policies during the COVID-19 Pandemic.” Proceedings of the National Academy of Sciences 117(36):21851–53. doi:10.1073/pnas.2011674117.

Biswas

Nirbachita

Mustapha

Toheeb

Khubchandani

Jagdish

Price

James H.

2021. “The Nature and Extent of COVID-19 Vaccination Hesitancy in Healthcare Workers.” Journal of Community Health 46(6):1244–51.

10.

Blair

Graeme

Coppock

Alexander

Moor

Margaret

. 2020. “When to Worry about Sensitivity Bias: A Social Reference Theory and Evidence from 30 Years of List Experiments.” American Political Science Review 114(4):1297–1315.

11.

Blair

Graeme

Imai

Kosuke

. 2012. “Statistical Analysis of List Experiments.” Political Analysis 20:47–77.

12.

Bodas

Moran

Peleg

Kobi

. 2020. “Self-Isolation Compliance in the COVID-19 Era Influenced by Compensation: Findings from a Recent Survey in Israel.” Health Affairs 39(6):936–41.

13.

Catalani

Valeria

Townshend

Honor D.

Prilutskaya

Mariya

Chilcott

Robert P.

Metastasio

Antonio

Banayoti

Hani

. 2023. “Illicit COVID-19 Products Online: A Mixed-Method Approach for Identifying and Preventing Online Health Risks.” PLoS One 18(6):e0287231.

14.

Centers for Disease Control and Prevention. 2024. “International Certificate of Vaccination or Prophylaxis (ICVP).” Retrieved November 1, 2024. https://wwwnc.cdc.gov/travel/page/icvp.

15.

Childs

Andrew

. 2023. “The Distribution of Fake Australian Vaccine Digital Certificates on an Alt-Tech Platform.” Trends in Organized Crime 26(2):136–55.

16.

Daoust

Jean-François

Nadeau

Richard

Dassonneville

Ruth

Lachapelle

Erick

Bélanger

Éric

Savoie

Justin

van der Linden

Clifton

. 2021. “How to Survey Citizens’ Compliance with COVID-19 Public Health Measures: Evidence from Three Survey Experiments.” Journal of Experimental Political Science 8(3):310–17.

17.

Diekmann

Andreas

. 2020. “Entstehung und Befolgung neuer sozialer Normen” [“Emergence of and Compliance with New Social Norms”]. Zeitschrift für Soziologie 49(4):236–48.

18.

Directorate-General for Health and Food Safety. 2021. “EU Health Preparedness: Recommendations for a Common EU Approach Regarding Isolation of COVID-19 Cases and Quarantine of Close Contacts.” Retrieved September 26, 2024. https://health.ec.europa.eu/system/files/2022-02/security_covid-19_recomm-quarantine-isolation_en_0.pdf.

19.

Droitcour

Judith

Caspar

Rachel A.

Hubbard

Michael L.

Parsley

Teresa L.

Visscher

Wendy

Ezzati

Trena M.

2004. “The Item Count Technique as a Method of Indirect Questioning: A Review of Its Development and a Case Study Application.” Pp. 185–210 in Measurement Errors in Surveys, edited by Biemer

P. P.

Groves

R. M.

Lyberg

L. E.

Mathiowetz

N. A.

Sudman

Hoboken, NJ: John Wiley.

20.

Dror

Amiel A.

Eisenbach

Netanel

Taiber

Shahar

Morozov

Nicole G.

Mizrachi

Matti

Zigron

Asaf

Srouji

Samer

, et al. 2020. “Vaccine Hesitancy: The Next Challenge in the Fight against COVID-19.” European Journal of Epidemiology 35(8):775–79.

21.

Ehler

Ingmar

Wolter

Felix

Junkermann

Justus

. 2021. “Sensitive Questions in Surveys.” Public Opinion Quarterly 85(1):6–27.

22.

Ellis

Wendy E.

Dumas

Tara M.

Forbes

Lindsey M.

2020. “Physically Isolated but Socially Connected: Psychological Adjustment and Stress among Adolescents during the Initial COVID-19 Crisis.” Canadian Journal of Behavioural Science/Revue Canadienne des Sciences du Comportement 52(3):177–87.

23.

European Centre for Disease Prevention and Control. 2020. “Infographic: Non-pharmaceutical Measures.” Retrieved September 26, 2024. https://www.ecdc.europa.eu/en/publications-data/infographic-non-pharmaceutical-measures.

24.

European Commission. 2022. “EU Digital COVID Certificate.” Retrieved September 26, 2024. https://commission.europa.eu/strategy-and-policy/coronavirus-response/safe-covid-19-vaccines-europeans/eu-digital-covid-certificate_en#documents.

25.

European Observatory on Health Systems and Policies. 2024. “COVID-19 Health System Response Monitor (HSRM). Iceland. 1.3 Isolation and Quarantine.” Retrieved December 2, 2024. https://eurohealthobservatory.who.int/monitors/hsrm/hsrm-countries/hsrm/iceland/preventing-transmission/isolation-and-quarantine/.

26.

Federal Department of Foreign Affairs. 2022. “Information Sheet COVID-19: Swiss COVID Certificate.” Retrieved September 26, 2024. https://www.eda.admin.ch/dam/mission-onu-omc-aele-geneve/en/documents/COVID-19-Notice-informative-Certificat-Covid-suisse-version_EN.pdf.

27.

Federal Office of Public Health. 2024. “COVID-19: Protect Yourself and Others.” Retrieved September 26, 2024. https://www.bag.admin.ch/bag/en/home/krankheiten/krankheiten-im-ueberblick/coronavirus/covid-19/so-schuetzen-wir-uns.html.

28.

Franzen

Axel

Wöhner

Fabienne

. 2021. “Coronavirus Risk Perception and Compliance with Social Distancing Measures in a Sample of Young Adults: Evidence from Switzerland.” PLoS One 16(2):e0247447.

29.

Galasso

Vincenzo

Pons

Vincent

Profeta

Paola

Becher

Michael

Brouard

Sylvain

Foucault

Martial

. 2020. “Gender Differences in COVID-19 Attitudes and Behavior: Panel Evidence from Eight Countries.” Proceedings of the National Academy of Sciences 117(44):27285–91.

30.

Georgoulias

Dimitrios

Pedersen

Jens M.

Falch

Morten

Vasilomanolakis

Emmanouil

. 2023. “COVID-19 Vaccination Certificates in the Darkweb.” Digital Threats: Research and Practice 4(1):1–17.

31.

González-Morales

Romina

Pérez-Ferrer

Carolina

López-Olmedo

Nancy

Alpuche-Aranda

Celia

Barrientos-Gutiérrez

Tonatiuh

. 2021. “Aislamiento ante Síntomas Sugestivos de COVID-19 o Contacto con un Caso: Cumplimiento, Prevalencia y Factores Asociados. Ensanut 2020 COVID-19” [“Isolation in the Event of Symptoms Suggestive of COVID-19 or Contact with a Case: Compliance, Prevalence, and Associated Factors. Ensanut 2020 COVID-19”]. Salud Publica de Mexico 63(6):773–81.

32.

Government of Iceland. 2021. “COVID 19: Digital Vaccine Certificates.” Retrieved September 26, 2024. https://www.government.is/diplomatic-missions/embassy-article/2021/01/25/COVID-19-Digital-vaccine-certificates/.

33.

Government of the United Kingdom. 2023. “NHS COVID Pass.” Retrieved September 26, 2024. https://www.gov.uk/guidance/nhs-covid-pass.

34.

Heghes

Nicoleta-Elena

. 2021. “Counterfeiting of the EU Digital COVID Certificate. Offenses and Penalties for Forgery and Use of False Documents.” Scientia Moralitas—International Journal of Multidisciplinary Research 6(2):41–48.

35.

Helbing

Alexander

Krumpal

Ivar

. 2024. “Vom Lügen und Fälschen: Abweichendes Verhalten in der COVID-19-Pandemie” [“About Lying and Cheating: Deviant Behavior in the COVID-19 Pandemic”]. Rechtspsychologie 10(3):349–63.

36.

Holbrook

Allyson L.

Krosnick

Jon A.

2010. “Measuring Voter Turnout by Using the Randomized Response Technique: Evidence Calling into Question the Method’s Validity.” Public Opinion Quarterly 74:328–43.

37.

Imai

Kosuke

. 2011. “Multivariate Regression Analysis for the Item Count Technique.” Journal of the American Statistical Association 106(494): 407–416.

38.

Junkermann

Justus

. 2022. “Count-Techniken: Eine Lösung für heikle Fragen?” [“Counting Techniques: A Solution for Sensitive Questions?”]. Dissertation, Fakultät für Sozialwissenschaften und Philosophie, Universität Leipzig, Leipzig.

39.

Kirchner

Antje

Krumpal

Ivar

Trappmann

Mark

von Hermanni

Hagen

. 2013. “Messung und Erklärung von Schwarzarbeit in Deutschland – Eine empirische Befragungsstudie unter besonderer Berücksichtigung des Problems der sozialen Erwünschtheit” [“Measuring and Explaining Undeclared Work in Germany—An Empirical Survey Study with Special Consideration of the Problem of Social Desirability”]. Zeitschrift für Soziologie 42(4):291–314.

40.

Korn

Lars

Böhm

Robert

Meier

Nicolas W.

Betsch

Cornelia

. 2020. “Vaccination as a Social Contract.” Proceedings of the National Academy of Sciences 117(26):14890–99.

41.

Kowalski

Joachim

Marchlewska

Marta

Molenda

Zuzanna

Górska

Paulina

Gawęda

Łukasz

. 2020. “Adherence to Safety and Self-Isolation Guidelines, Conspiracy and Paranoia-Like Beliefs during COVID-19 Pandemic in Poland—Associations and Moderators.” Psychiatry Research 294:113540.

42.

Krumpal

Ivar

. 2013. “Determinants of Social Desirability Bias in Sensitive Surveys: A Literature Review.” Quality & Quantity 47(4):2025–47.

43.

Krumpal

Ivar

Auspurg

Katrin

Jann

Ben

von Hermanni

Hagen

. 2015. “Asking Sensitive Questions: a Critical Account of the Randomized Response Technique and Related Methods.” Pp. 122–36 in Improving Survey Methods: Lessons from Recent Research, edited by Engel

Jann

Lynn

Scherpenzeel

Sturgis

New York: Routledge.

44.

Krumpal

Ivar

Jann

Ben

Korndörfer

Martin

Schmukle

Stefan C.

2018. “Item Sum Double-List Technique: An Enhanced Design for Asking Quantitative Sensitive Questions.” Survey Research Methods 12(2):91–102.

45.

Krumpal

Ivar

Näher

Anatol-Fiete

. 2012. “Entstehungsbedingungen sozial erwünschten Antwortverhaltens: Eine experimentelle Onlinestudie zum Einfluss des Wordings und des Kontexts bei unangenehmen Fragen” [“Conditions for the Development of Socially Desirable Answering Behavior: An Experimental Online Study on the influence of Wording and Context in Unpleasant Questions”]. Soziale Welt 63(1):65–89.

46.

Kuha

Jouni

Jackson

Jonathan

. 2014. “The Item Count Method for Sensitive Survey Questions: Modelling Criminal Behaviour.” Journal of the Royal Statistical Society, Series C (Applied Statistics) 63(2):321–41.

47.

Lazarus

Jeffrey V.

Wyka

Katarzyna

White

Trenton M.

Picchio

Camila A.

Gostin

Lawrence O.

Larson

Heidi J.

Rabin

Kenneth

, et al. 2023. “A Survey of COVID-19 Vaccine Acceptance across 23 Countries in 2022.” Nature Medicine 29(2):366–75.

48.

Lee

Raymond M.

1993. Doing Research on Sensitive Topics. London: Sage Ltd.

49.

Leigh

Jeanna P.

Moss

Stephana J.

White

Trenton M.

Picchio

Camila A.

Rabin

Kenneth H.

Ratzan

Scott C.

Wyka

Katarzyna

, et al. 2022. “Factors Affecting COVID-19 Vaccine Hesitancy among Healthcare Providers in 23 Countries.” Vaccine 40(31):4081–89.

50.

Jiayuan

van den Noortgate

Wim

. 2022. “A Meta-analysis of the Relative Effectiveness of the Item Count Technique Compared to Direct Questioning.” Sociological Methods & Research 51(2):760–99.

51.

Munzert

Simon

Selb

Peter

. 2020. “Can We Directly Survey Adherence to Non-pharmaceutical Interventions?” Survey Research Methods 14(2):205–209.

52.

Näher

Anatol-Fiete

Krumpal

Ivar

. 2012. “Asking Sensitive Questions: The Impact of Forgiving Wording and Question Context on Social Desirability Bias.” Quality & Quantity 46:1601–16.

53.

National Health Service. 2023. “COVID-19 Symptoms and What to Do.” Retrieved September 26, 2024. https://www.nhs.uk/conditions/covid-19/covid-19-symptoms-and-what-to-do/.

54.

Nkire

Nnamdi

Mrklas

Kelly

Hrabok

Marianne

Gusnowski

April

Vuong

Wesley

Surood

Shireen

Abba-Aji

Adam

, et al. 2021. “COVID-19 Pandemic: Demographic Predictors of Self-Isolation or Self-Quarantine and Impact of Isolation and Quarantine on Perceived Stress, Anxiety, and Depression.” Frontiers in Psychiatry 12:553468.

55.

Oosterhoff

Benjamin

Palmer

Cara A.

2020. “Psychological Correlates of News Monitoring, Social Distancing, Disinfecting, and Hoarding Behaviors among US Adolescents during the COVID-19 Pandemic”. PsyArXiv. Retrieved March 16, 2025. https://osf.io/preprints/psyarxiv/rpcy4/rpcy4_v1.

56.

Quatember

Andreas

. 2023. “Efficient Item Count Techniques with One or Two Lists.” Metron 81:5–19.

57.

Raghavarao

Federer

W. T.

1979. “Block Total Response as an Alternative to the Randomized Response Method in Surveys.” Journal of the Royal Statistical Society 41(1):40–45.

58.

Sepúlveda-Loyola

Rodríguez-Sánchez

Pérez-Rodríguez

Ganz

Torralba

Oliveira

D. V.

Rodríguez-Mañas

2020. “Impact of Social Isolation Due to COVID-19 on Health in Older People: Mental and Physical Effects and Recommendations.” Journal of Nutrition, Health & Aging 24(9):938–47.

59.

Shah

Aashka

Coiado

Olivia C.

2022. “COVID-19 Vaccine and Booster Hesitation around the World: A Literature Review.” Frontiers in Medicine 9:1054557.

60.

Suppawittaya

Piwat

Yiemphat

Pakara

Yasri

Pratchayapong

. 2020. “Effects of Social Distancing, Self-Quarantine and Self-Isolation during the COVID-19 Pandemic on People’s Well-Being, and How to Cope with It.” International Journal of Science and Healthcare Research 5(2):12–20.

61.

Tourangeau

Roger

Yan

Ting

. 2007. “Sensitive Questions in Surveys.” Psychological Bulletin 133(5):859–83.

62.

Troiano

Gian M.

Nardi

Alessandra

. 2021. “Vaccine Hesitancy in the Era of COVID-19.” Public Health 194:245–51.

63.

Tutić

Andreas

Krumpal

Ivar

Haiser

Friederike

. 2022. “Triage in Times of COVID-19: A Moral Dilemma.” Journal of Health and Social Behavior 63(4):560–76.

64.

van Rooij

Benjamin

de Bruijin

Anne L.

Folmer

Chris Reinders

Kooistra

Emmeke

Kuiper

Malouke Esra

Brownlee

Megan

Olthuis

Elke

, et al. 2020. “Compliance with COVID-19 Mitigation Measures in the United States”. Amsterdam Law School Legal Studies Research Paper No. 2020-21. Retrieved March 16, 2025. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3582626.

65.

Warner

L. S.

1965. “Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias.” Journal of the American Statistical Association 60(309):63–69.

66.

Wolter

Felix

Laier

Bastian

. 2014. “The Effectiveness of the Item Count Technique in Eliciting Valid Answers to Sensitive Questions: An Evaluation in the Context of Self-Reported Delinquency.” Survey Research Methods 8(3):153–68.

67.

Wolter

Felix

Mayerl

Jochen

Andersen

Henrik K.

Wieland

Theresa

Junkermann

Justus

. 2022. “Overestimation of COVID-19 Vaccination Coverage in Population Surveys Due to Social Desirability Bias: Results of an Experimental Methods Study in Germany.” Socius. doi:10.1177/23780231221094749.

68.

Yan

Ting

. 2021. “Consequences of Asking Sensitive Questions in Surveys.” Annual Review of Statistics and Its Application 8:109–27.

69.

Zhao

Yuqing

Yuanyuan

Tan

Xing

Xiaohui

. 2020. “Mental Health and Its Influencing Factors among Self-Isolating Ordinary Citizens during the Beginning Epidemic of COVID-19.” Journal of Loss and Trauma 25(6–7):580–93.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB