Sage Journals: Discover world-class research

Abstract

Due to the rise of fraudulent responders, either human or automated bots, researchers must carefully monitor online survey data quality. The literature on preventing and detecting fraudulent responders is growing, but the prevalence and experience of researchers experiencing fraudulent responders remains unknown. Through an online survey of researchers who have published studies using online surveys, this study explores researchers’ (N = 675) knowledge and experience with fraudulent responders. Over a third of the participants (37.33%) reported experiencing fraudulent responders that affected their online survey research; however, this number could be higher, as 19.26% of the participants did not know or check for fraudulent responses. Participants who experienced fraudulent responders (EFR) reported statistically more confidence, compared to those who had not experienced fraudulent responders (NEFR), that they could remove (M(EFR) = 2.42; M(NEFR) = 3.17; t(583.10) = 8.36, p < .001) and prevent (M(EFR) = 2.93; M (NEFR) = 3.27; t(541.34) = 3.57, p < .001) fraudulent responders from infiltrating their online surveys. Most participants (59.76%) did not realize their study had been affected by fraudulent responders until the data cleaning phase. This study raises awareness about the risks of fraudulent responders, which not only negatively affect data quality and compromise scientific integrity but can also hinder researchers’ productivity and potentially harm their careers.

Keywords

fraudulent responders online survey research data quality data collection

Introduction

Over the past two decades, online survey research has increased dramatically and become a leading data collection approach among social scientists and other researchers (Chandler & Shapiro, 2016; Daikeler et al., 2020; van Quaquebeke et al., 2022). This is primarily due to the ease of distribution and cost-effectiveness of online surveys in recruiting larger samples as well as enabling researchers to access populations that otherwise would be difficult to reach (Bybee et al., 2022; Chandler & Shapiro, 2016; Coppock, 2019; Donegan & Gillan, 2022; Evans & Mathur, 2018; Hammond, 2018; Lawrence et al., 2023; Wright, 2005). Many researchers use online surveys as they can be easily distributed via email, with anonymous or individualized links, online panels (e.g., Qualtrics, Amazon Mechanical Turk), or social media posts or advertisements (Vehovar et al., 2016).

While there are several benefits to using online surveys in research, it behooves researchers to monitor and maintain online survey data quality. Though not exclusive to online surveys, researchers need to be aware of the potential for carelessness in responses (Ward & Meade, 2023), which occurs when participants do not read the entire survey question or are unable to understand the question due to differences in comprehension capabilities (Lopez & Hillygus, 2018). There is also the possibility that a participant accidentally participates in an online survey more than once (Lawlor et al., 2021). More recently, a critical aspect of data quality that researchers need to be vigilant of is the rise of fraudulent responders (also known as “scammers” or “spammers”), either human or automated bots, to online surveys (e.g., Arevalo et al., 2022; Bonnamy et al., 2025; Caven et al., 2025; Nur et al., 2024).

A fraudulent human responder is someone who is deliberately dishonest in an online survey by assuming a false identity or responding with fake answers (Lopez & Hillygus, 2018) and/or trying to take the survey more than one time while using phony profiles (Zhang et al., 2022). An automated bot, also known as a survey bot, is algorithmically programmed by a human to automatically answer online surveys, frequently more than once, while providing fake data (Kennedy et al., 2021; Storozuk et al., 2020). Importantly, both human and bot fraudulent responders have malicious intent and deliberately misrepresent themselves, which is often done to receive monetary benefit from online survey participation incentives (Bethlehem, 2010; Hulland & Miller, 2018; Kumarasamy et al., 2024; Lawlor et al., 2021; Wright, 2005). In addition to the increased financial expenditure from wrongly providing monetary incentives for fraudulent responders, their data can also have adverse implications on the reliability of research findings if included in the analyses (Agrawal et al., 2025).

While there has been an increase in researchers publishing about their experiences with fraudulent responses (e.g., Arevalo et al., 2022; A. M. Bell & Gift, 2023; C. J. Bell et al., 2020; Brainard et al., 2022; Bybee et al., 2022; Griffin et al., 2022; Kumarasamy et al., 2024; Pozzar et al., 2020; Salinas, 2022), it is still unknown how prevalent this issue is among online survey researchers. Awareness of a technology-related issue, like fraudulent responders accessing online surveys, may strongly influence intentions to implement prevention strategies (Dinev & Hu, 2007). It is unknown if there are differences between those who do and do not have experience with fraudulent responders in their knowledge of fraudulent responders. This study contributes to the literature by addressing the following research questions:

(1) How many online survey researchers have experienced fraudulent responses affecting their surveys?

(2) Are there differences between researchers’ knowledge of fraudulent responders in online surveys between those who have experienced fraudulent responders (EFR) and those who have not experienced fraudulent responders (NEFR) in their surveys?

(3) How do researchers handle fraudulent responders affecting their online surveys?

Methods

Recruitment and Data Collection

We obtained a list of researchers who had published peer-reviewed journal articles between 2017 and 2022 using an online survey by searching the Web of Science Core Collection database. The search terms included “online survey,”“web survey,” or “web-based survey.” The language was set to English and the region to the United States. The search yielded 15,204 publication results, with 12,258 unique email addresses. In this study, we only kept the email addresses from academic institutions (i.e., ending with an “.edu” domain), resulting in 7,799 academic researchers. Initial recruitment emails were sent on May 16, 2023, with follow-up reminder emails sent on May 25th and June 1st. One email failed to deliver, and 655 emails bounced back, resulting in 7,143 successfully delivered emails. To prevent fraudulent responses and ensure data quality, each researcher was invited to participate in the survey via an email containing a personal survey link. The survey link directed the participant to the informed consent, which provided information about the study, including the risk of harm and potential benefits of the research, and explained that they had the option to stop participating in the study at any point. This study was designed to minimize the risk of harm to participants by using a non-invasive online survey for the study design, giving the participants the choice to opt in/stop participating, and not asking sensitive questions. Participants were informed that the results of the study could lead to better quality data and potential policy implications. Those who opted to participate in the online survey clicked the reCAPTCHA box at the bottom of the informed consent webpage and were directed to the online survey. The online survey took approximately 10 minutes to complete. After completing the online survey, participants could enter their name and email address to participate in a drawing for a $25 Amazon gift card (one gift card per every 100 participants). The study procedure was approved by the university’s Institutional Review Board. The survey response rate was 10.6%, or 758 responses. Participants were excluded from the final sample due to completing less than 70% of the survey (n = 59), not conducting online survey research (n = 17), and having missing data on the knowledge of fraudulent responder questions and the level of difficulty to protect the online survey from fraudulent responder question (study’s outcome variables; n = 7). Our final sample included 675 participants who had conducted online survey research.

Measures

In this online survey, participants were asked about their experiences with online surveys, including questions related to fraudulent responders or “scammers.” Throughout the online survey, we opted to use the term “scammer” due to simplicity and ease of reading for the participant. A definition for “scammer” was initially provided for participants, “Scammers refer to human or automated respondents (e.g., bots) who are deliberately fraudulent in their responses (e.g., for monetary incentive, to sway the results) and may complete an online survey multiple times.”

Knowledge of Fraudulent Responders in Online Surveys

Participants rated their level of agreement with five statements pertaining to their knowledge of fraudulent responders in online surveys. The five statements, adapted from Dinev and Hu (2007) included: (1) “I am aware of the online survey scammer problems and consequences,” (2) “I discuss with colleagues and/or people around me about scammers in online surveys,” (3) “I read about the problems of scammers infiltrating online surveys,” (4) “I am confident that I can remove scammers responses from my online survey data,” and (5) “I am confident I can prevent scammers from infiltrating my online surveys.” Responses were measured on a 5-point Likert scale ranging from 1 (completely agree) to 5 (completely disagree).

We asked participants, “How difficult do you think it is to protect online surveys from scammers?” Response was measured on a 5-point Likert scale ranging from 1 (extremely difficult) to 5 (extremely easy), with a 6 (I do not know) option. To understand researchers’ knowledge of college or university provided scam prevention strategies, we asked, “Does your college or university offer any prevention strategies to keep scammers from infiltrating online surveys?” Response options included: Yes, No, Not sure, and I don’t work at a college or university.

Online Survey Experience With Fraudulent Responders

Participants were asked, “Have you experienced scammers affecting your online survey research?” Response options included: Yes, No, and I do not know or I did not check. Responses of No and I do not know or I did not check were considered as No for analysis in this study. Participants who responded Yes received follow-up questions about their experiences with fraudulent responders. First, participants were asked about the phase they noticed the fraudulent responders, “For your most recent online survey that was affected by scammers, at what phase of your study did you notice that your online survey had been affected by scammers? (Select one).” Response options included: Recruitment, Data cleaning, Data analysis, Dissemination, and Other (please specify).

Next, we wanted to know what participants did once they determined their study had been infiltrated by fraudulent responders. We asked two questions. “What did you do when you determined that your study had been affected by scammers? Please select all that apply.” Response options included: Paused data collection, Re-started the study, Revised procedures and/or methods, Attempted to remove the scam responses from the data, Research ways to detect scammer responses in data, Consulted with experts, Cancelled the study, Did nothing, and Other (please specify). “Which tactics did you try to reduce the impact of scammers in your most recent online survey that was affected by scammers? Please select all that apply.” Response options included: Analyzing location using the IP address, Analyzing open-ended responses with illogical or unreasonable answers, Analyzing start times, end times, or total duration for suspicious patterns, Verifying information a second way, Collection meta-information, Evaluating attention checks, Honey pot questions, Including redundant items to check for inconsistent responses, Modifying recruitment strategies, Searching for patterns in the data, and Other (please specify). To determine how long it took participants to address the fraudulent responder issue, we asked, “About how much time did it take you to attempt to address the scamming problem in your most recent online survey that was affected by scammers?” Response options included: 24 hours or less, 2–7 day, 1–2 weeks, 3–4 weeks, 2–3 months, 4–6 months, 7–12 months, and More than 1 year.

Finally, we asked three questions about the quality of the data. “What percentage of the completed responses did you identify as scam responses?” Response options included: 1 (I do not know or remember) or 2 (less than 1%), 3 (1%–9%), 4 (10%–19%), and so on until 12 (90% or more). “How confident are you that you were able to address the scammer problem in your most recent online survey that was affected by scammers?” Response was measured on a 5-point Likert scale ranging from 1 (not confident) to 5 (completely confident). “Were you able to publish with the data from your most recent online survey that was affected by scammers?” Response options included: Yes, Not yet, but I am or will try to publish with this data, or No, I will not publish this data.

Demographic Information

Participants' age, gender, race/ethnicity, and level of education were measured. “What is your age in years?” Response options ranged from 18 to more than 90 in increments of 1 year. “What is your gender?” Response options included: Male, Female, Non-binary or gender non-conforming, and Other (please specify). “What is your race/ethnicity? Please select all that apply.” Response options included: White, Black or African American, Asian, American Indian or Alaskan Native, Native Hawaiian or Other Pacific Islander, and Other (please specify). “What is your highest level of education?” Response options included: Some high school, High school graduate/GED, Some college but no degree, Associate degree or vocational/technical degree, Bachelor’s degree, Master’s degree, and Professional or doctorate degree (e.g., JD, PhD, MD). To determine the role at college or university, participants were asked, “What is your primary role at the college or university?” Response options included: I don’t work at a college or university, Faculty, Postdoctoral researcher, Graduate student, Undergraduate student, Administrator, Staff, or Retired from college or university. Participants who worked at a college or university were asked, “What is your major academic discipline? Please select all that apply.” Response options included: Humanities (e.g., performing arts, visual arts, history, law), Social science (e.g., economics, politics, sociology, psychology), Natural science (e.g., physics, chemistry, biology, geology, astronomy), Formal science (e.g., computer science, mathematics), Applied science (e.g., agriculture, architecture and design, business, engineering, medicine), and Other (please specify). Those who responded Faculty as their primary role at the college or university were also asked a follow-up question, “How many years in total have you been a college or university faculty member?” Response options ranged from 1 (less than 1 year) to 41 (more than 40 years).

Data Analysis

Given the exploratory nature of this study, data were primarily analyzed descriptively. The differences in knowledge of fraudulent responders between the two groups (i.e., experienced fraudulent responders vs. not experienced fraudulent responders) were compared using Welch’s t-tests since the observations were normally distributed, independent, and measured on an interval scale. Welch’s test is also appropriate given unequal group sizes, and it is more robust to violations of normality compared to Student’s t-test, and does not assume homoskedasticity. The level of statistical significance was set at 0.05. The effect size was computed using Cohen’s d to assess the practical significance of the differences.

Results

A little over one-third of the participants (n = 252; 37.33%) reported that they had experienced fraudulent responders (EFR) affecting their online survey research, while about two-thirds reported that they had not experienced fraudulent responders (NEFR) (n = 423; 62.67%). See Figure 1 Experience with Fraudulent Responders.

Figure 1.

Experience with Fraudulent Responders.

Participant Characteristics

Participants in this study were, on average, aged 44.94 years (SD = 11.47). The majority of the participants were female (66.92%), White (87.12%), and had a professional or doctorate degree (92.08%). See Table 1 for participant characteristics. Most of the participants worked as faculty at a college or university (79.52%). Of the participants who were faculty members, they worked in that position, on average, over 12 years (SD = 8.93), and most worked in the social science academic discipline (57.58%). See Table 2 for participant academic information.

Table 1.

Participant Characteristics.

		Not experienced fraudulent responders	Experienced fraudulent responders
	Full sample	(NEFR)	(EFR)
	N = 675	n = 423	n = 252
Demographics	M (SD)/ N (%)
Age^a	44.94 (11.47)	46.98 (11.82)	41.43 (9.91)
	Range: 18-83	Range: 25-83	Range: 18-80
Gender^b
Female	445 (66.92)	273 (65.16)	172 (69.92)
Male	206 (30.98)	141 (33.65)	65 (26.42)
Non-binary/Gender non-conforming	12 (1.80)	5 (1.19)	7 (2.85)
Other	2 (0.30)	0 (0)	2 (0.81)
Hispanic/Latino/Spanish descent^c
Yes	32 (4.81)	25 (5.95)	7 (2.86)
No	633 (95.19)	395 (94.05)	238 (97.14)
Race/ethnicity^d
White	575 (87.12)	364 (87.50)	211 (86.48)
Asian	58 (8.79)	38 (9.13)	20 (8.20)
Black/African American	19 (2.88)	9 (2.16)	10 (4.10)
American Indian/Alaskan Native	6 (0.91)	4 (0.96)	2 (0.82)
Native Hawaiian/Other Pacific Islander	1 (0.15)	1 (0.24)	0 (0)
Other	17 (2.58)	10 (2.40)	7 (2.87)
Education^e
Master’s degree	53 (7.92)	32 (7.60)	21 (8.47)
Professional or doctorate degree	616 (92.08)	389 (92.40)	227 (91.53)

Notes. ^aMissing 8 NEFR responses and 11 EFR responses f. ^bMissing 4 NEFR responses and 6 EFR responses. ^cMissing 3 NEFR responses and 7 EFR responses. ^dMissing 7 NEFR responses and 8 EFR responses. ^eMissing 2 NEFR responses and 4 EFR responses.

Table 2.

Participant Academic Information.

		Not experienced fraudulent responders	Experienced fraudulent responders
	Full sample	(NEFR)	(EFR)
	N = 675	n = 423	n = 252
Academic information	M (SD)/ N (%)
Primary role at the college/ university^a
Faculty	532 (79.52)	333 (79.10)	199 (80.24)
Postdoctoral researcher	28 (4.19)	16 (3.80)	12 (4.84)
Graduate student	41 (6.13)	22 (5.23)	19 (7.66)
Administrator	13 (1.94)	11 (2.61)	2 (0.81)
Staff	25 (3.74)	17 (4.04)	8 (3.23)
Retired from college or university	16 (2.39)	15 (3.56)	1 (0.40)
I don’t work at a college or university	14 (2.09)	7 (1.66)	7 (2.82)
Years as college/university faculty^b	13.66 (8.93)	14.47 (9.43)	12.33 (7.87)
	Range: 1-41	Range: 1-41	Range: 1-41
Major academic discipline^c
Social science	376 (57.58)	199 (47.95)	177 (74.37)
Applied science	173 (26.49)	140 (33.73)	33 (13.87)
Natural science	23 (3.52)	19 (4.58)	4 (1.68)
Humanities	6 (0.92)	5 (1.20)	1 (0.42)
Formal science	6 (0.92)	1 (0.24)	5 (2.10)
Other	95 (14.55)	67 (16.14)	28 (11.76

Notes. ^aMissing 2 NEFR responses and 4 EFR responses. ^bMissing 5 NEFR responses and 1 EFR response. ^cMissing 8 NEFR responses and 14 EFR responses.

Knowledge of Fraudulent Responders

EFR participants reported statistically more agreement, compared to NEFR participants, that they were aware of online fraudulent responder problems and consequences (Cohen’s d = 1.07), discussed the fraudulent responder issue with colleagues and other people around them (d = 1.28), read about the problems of fraudulent responders in online surveys (d = 1.34), were confident that they could remove fraudulent responses from their online survey data (d = 1.18), and were confident they could prevent fraudulent responders from infiltrating their online surveys (d = 1.22). EFR participants thought it was significantly more difficult to protect online surveys from fraudulent responders than NEFR participants (d = 1.41). All observed effects between the two groups were large in magnitude (d > 0.80). See Table 3 for knowledge of fraudulent responders. Overall, the majority of participants (62.78%) were not sure what prevention strategies were offered by their college or university to keep fraudulent responders from infiltrating online surveys. See Figure 2 for knowledge of prevention strategies offered by college or university.

Table 3.

Knowledge of Fraudulent Responders.

	Not Experienced Fraudulent Responders	Experienced Fraudulent Responders
	(NEFR)	(EFR)
	n = 423	n = 252
Knowledge components	M (SD)		df	t	p-Value	Cohen’s d
Knowledge of fraudulent responders
I discuss with colleagues and/or people around me about fraudulent responders in online surveys	3.32 (1.44)	1.52 (0.95)	666.2	19.44	<.001	1.28
I am confident I can prevent fraudulent responders from infiltrating my online surveys	3.27 (1.23)	2.93 (1.19)	541.4	3.57	<.001	1.22
I read about the problems of fraudulent responders infiltrating online surveys	3.25 (1.39)	1.96 (1.26)	567.3	12.30	<.001	1.34
I am confident that I can remove fraudulent responses from my online survey data	3.17 (1.23)	2.42 (1.08)	583.1	8.36	<.001	1.18
I am aware of the online survey fraudulent responders’ problems and consequences	2.44 (1.27)	1.26 (0.60)	646.0	16.25	<.001	1.07
Ability to protect online surveys^a
Level of difficulty to protect online surveys from fraudulent responders	3.07 (1.63)	2.05 (0.95)	672.9	10.26	<.001	1.41

Note. ^aResponse option “I do not know” was not included in the analysis.

Figure 2.

Knowledge of prevention strategies offered by college or university,

How Researchers Handled Fraudulent Responders

Of the EFR participants, about two-fifths (40.24%) reported noticing the issue of fraudulent responders in their online survey during the recruitment phase, while most (51.79%) noticed it during the data cleaning phase. See Figure 3 for the phase respondents noticed that online survey had been affected by fraudulent responders. Once participants determined their study had been affected by fraudulent responders, they took between one to six actions to address this issue, with an average of 2.69 actions (SD = 1.51). The top three actions to address the fraudulent responder issue were attempting to remove the fraudulent responses from the data (88.00%), researching ways to detect fraudulent responders in the data (46.40%), and revising the procedures and/or methods (40.80%).

Figure 3.

Phase of the study when participants noticed that their online survey had been affected by fraudulent responders.

Reducing the impact of fraudulent responders necessitated further actions on the part of participants. Participants tried between one and ten tactics to reduce the impact of fraudulent responders with an average of trying 4.24 tactics (SD = 1.91). The top five tactics used by participants to reduce the impact of fraudulent responders were analyzing start times, end times, or total duration for suspicious patterns (78.23%), analyzing open-ended responses with illogical or unreasonable answers (66.13%), analyzing location using the IP address (56.85%), evaluating attention checks (52.42%), and searching for patterns in the data (50.40%). Most participants (63.96%) were able to address their fraudulent responder problem in less than a week. About one-fifth (20.24%) of the participants reported that they did not know or remember the proportion of fraudulent responses in their data, while almost a third (31.57%) reported that more than 20% of their data was identified as fraudulent. Most participants (69.23%) were fairly or completely confident they were able to address the fraudulent responder problem in their most recent survey that was affected by fraudulent responders, and plan to or have published with that data (90.61%). See Table 4 for more information on how participants handled fraudulent responders.

Table 4.

How Researchers Handled Fraudulent Responders.

Addressing fraudulent responders	N (%) / M (SD)
Steps taken once determined study had been affected by fraudulent responders^a	2.69 (1.51)
Attempted to remove the fraudulent responses from the data	220 (88.00)
Research ways to detect fraudulent responders in data	116 (46.40)
Revised procedures and/or methods	102 (40.80)
Consulted with experts (e.g., IRB, colleagues)	90 (36.00)
Paused data collection	86 (34.40)
Re-started the study	25 (10.00)
Cancelled the study	2 (0.80)
Other	32 (12.80)
Did nothing	4 (1.60)
Tactics tried to reduce the impact^b	4.24 (1.91)
Analyzing start times, end times, or total duration for suspicious patterns	194 (78.23)
Analyzing open-ended responses with illogical or unreasonable answers	164 (66.13)
Analyzing location using the IP address	141 (56.85)
Evaluating attention checks	130 (52.42)
Searching for patterns in the data	125 (50.40)
Modifying recruitment strategies	76 (30.65)
Including redundant items to check for inconsistent responses	74 (29.84)
Verifying information, a second way	60 (24.19)
Collection meta-information	40 (16.13)
Honey pot questions	19 (7.66)
Other	29 (11.69)
Length of time it took to address the problem^c
24 hours or less	77 (31.17)
2–7 days	81 (32.79)
1–2 weeks	40 (16.19)
3–4 weeks	25 (10.12)
2–3 months	12 (4.86)
4–6 months	11 (4.45)
7–12 months	0 (0)
More than a year	1 (0.40)
Percentage of the completed responses identified as fraudulent responses^c
Less than 1%	20 (8.10)
1%–9%	72 (29.15)
10%–19%	27 (10.93)
20%–29%	15 (6.07)
30%–39%	16 (6.48)
40%–49%	4 (1.62)
50%–59%	13 (5.26)
60%–69%	5 (2.02)
70%–79%	10 (4.05)
80%–89%	9 (3.64)
90% or more	6 (2.43)
I do not know or remember	50 (20.24)
Confidence in ability to address the fraudulent response problem^c
Completely confident	42 (17.00)
Fairly confident	129 (52.23)
Somewhat confident	41 (16.60)
Slightly confident	26 (10.53)
Not confident	9 (3.64)
Published the data^d
Yes	112 (45.71)
Not yet, but I am or will try to publish with this data	110 (44.90)
No, I will not publish with this data	23 (9.39)

Notes. N = 252. ^aMissing 2 responses. ^bMissing 4 responses. ^cMissing 5 responses. ^dMissing 7 responses.

Discussion

This study examined researchers’ knowledge of and experience with fraudulent responders infiltrating online surveys. Most of our participants were social scientists, which was expected given the prevalence of survey research in social science disciplines (Cheung et al., 2017; Das et al., 2018; Newman et al., 2021). Over one-third (37.33%) of the researchers in this study experienced fraudulent responders affecting their online survey research; however, this number may be understated, as 19.26% of the participants did not know or did not check if they had experienced fraudulent responders. Given the increasing prevalence of fraudulent responders in online research, researchers have an ethical responsibility to be aware of and assess whether their surveys have been infiltrated by fraudulent responders. In addition, researchers who utilize online surveys must stay up to date with methods regarding data quality in order to reduce any risks associated with fraudulent responders (Yarrish et al., 2019).

There were significant differences found between EFR and NEFR participants in awareness of and confidence in addressing fraudulent responders. Awareness of the consequences of technological issues has been found to directly influence actions in implementing prevention or protection strategies (Dinev & Hu, 2007). Most EFR participants implemented multiple strategies to reduce the impact of fraudulent responders in their survey. However, the majority of them (51.79%) did not realize that their data was infiltrated by fraudulent responders until after the recruitment phase, which could have resulted in a smaller sample size than planned (Bauermeister et al., 2012) and reduced the statistical power of their survey study. Further, only about one-sixth of these participants were completely confident that they addressed the fraudulent responder problem in their data. Given the evolving nature of fraud in online surveys (Griffin et al., 2022; Simone et al., 2024), it is possible that the set of fraud detection strategies participants implemented were inadequate to detect complex fraudulent responses. Future research on fraudulent responders should explore psychosocial determinants as well as the variation in types and strategies used by fraudulent responders. Qualitative research is also needed to better understand which strategies researchers are using to identify and resolve their fraudulent responder issue.

Half of the EFR participants (50%) reported that they did not know if their college or university had prevention strategies or resources to help researchers who experienced fraudulent responders. This percentage rose to almost three-fourths (72%) of the NEFR participants. Given the rise of fraudulent responders, colleges and universities need to invest in prevention strategies and resources for researchers at their institution who are encountering this critical issue. Unfortunately, from our data it was not clear whether colleges and universities, as well as specific disciplines, are aware of these risks and/or providing resources, including training, for students, research staff, and faculty members. Without additional training in this area, it is likely that this problem will continue to increase over time. Based on the responses from this sample of college and university researchers, even if their institution has policies and/or resources to address online survey fraudulent responders, the information is not easily accessible. Future research should explore how the presence or absence of institutional policies, including whether current institutional review boards (IRB) frameworks sufficiently guide researchers in this evolving area, influences researchers’ preparedness and response strategies to address fraudulent responders. Additionally, research should focus on designing educational strategies to increase awareness of online survey fraud among researchers, creating training programs to help researchers use available institutional resources to combat fraud, and providing grounded recommendations to mitigate the impact of fraudulent responders. Another line of research investigating the characteristics of fraud and developing reliable tools to prevent and detect fraudulent responses is also needed to safeguard the scientific integrity of survey research (Agrawal et al., 2025). Research also needs to consider the ethical burden of excluding data due to fraudulent responders.

The adverse impacts of fraudulent responders on the use of scientific resources were reflected in the inability and/or unwillingness of about 9% of our EFR participants to publish their data. Hence, the increase in fraudulent responses in online surveys affects the quality of the data, which, in turn, may affect whether research can be published and thus potentially impact researchers’ career productivity (e.g., annual reviews, progress toward tenure).

Limitations

Although this study provides valuable contributions to the literature, we must acknowledge its limitations. The sample was limited to researchers who published their online survey research, had a United States college or university email address, and chose to participate in the study, which may limit the generalizability of the findings. This study does not reflect the experiences of non-academic researchers or academic researchers who may have experienced online survey fraudulent responders but have not published their research. Additionally, the fraudulent responder experiences for staff and students at colleges and universities in the United States may not be fully reflected, as the sample was predominantly comprised of college and university faculty members. Future research should explore researchers’ knowledge of and experience with fraudulent responders in online surveys among researchers working in different roles/organizations and outside of the United States. Additionally, there was a low response rate (10.6%), which is a well-documented challenge in survey-based research, particularly for online surveys (Daikeler et al., 2020, 2022; Nulty, 2008). We sent two reminder emails and included personal survey links to encourage study participation. While we have a relatively large sample of online survey researchers (N = 675), it is possible that sampling bias may exist, with researchers opting to participate in the study due to interest in the topic or experience with fraudulent responders.

Conclusion

While this study does not provide tools to prevent or detect fraudulent responses, we do offer some guidance to researchers, especially those who have not experienced survey fraud yet. First, awareness of fraudulent responders is the most critical aspect of protecting the integrity of the data. Second, it is crucial to implement necessary protocols for preventing and detecting fraud (e.g., adding honeypot questions, recording metadata) before starting participant recruitment. Third, and related to the previous point, make an effort to get familiar with institutional resources and the survey platform before data collection. For example, some universities subscribe to advanced survey security features that remain unknown to researchers. Fourth, remember that not all fraudulent responders are alike. Hence, if your survey is targeted by fraudulent responders, stay flexible as you prepare the plan to detect and mitigate the effects of fraudulent responses, and document the steps clearly to aid reporting transparency. Fifth, the fraud detection process can not only miss identifying fraudulent responses (false negatives), but it can also mark some legitimate responses as fraudulent responses (false positives). Given this, researchers should closely work with their institute’s ethics board to balance human subjects’ protection (e.g., privacy concerns from collecting identifiable information to detect fraud) and the integrity of data. If providing participation incentives, seek the ethics board’s guidance to appropriately deal with the problem of fraudulent responders. For example, by including a statement in informed consent stating that “responses deemed low-quality or fraudulent will not receive participation incentives.”

In conclusion, this study elucidates the experiences that online survey researchers, who published peer-reviewed journal articles of studies using online surveys between 2017 and 2022, have had with fraudulent responders infiltrating their online surveys. Researchers must be more proactive in preventing and removing fraudulent responses. Colleges and universities must also develop effective educational programs to train researchers on how to prevent online survey fraud and, if it happens, prepare them to systematically address the issue using a set of robust tools and strategies.

Footnotes

ORCID iDs

Amy M. Schuster

Shubham Agrawal

Shelia R. Cotten

Ethical Considerations

This study received ethical approval from the Clemson University IRB (approval # IRB2023-0283) on April 07, 2023.

Consent to Participate

Informed consent was reviewed prior to starting the online survey. Those who opted to participate in the online survey clicked the reCAPTCHA box at the bottom of the informed consent webpage and were directed to the online survey.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Data is available upon reasonable request.

References

Agrawal

Watson

G. P.

Schuster

A. M.

Cotten

S. R.

Tidwell

M. L.

Baker

Wang

(2025). It’s more than money: The true costs of spam in online survey research. Research Ethics. Advance online publication. https://doi.org/10.1177/17470161251357454

Arevalo

Brownstein

N. C.

Whiting

Meade

C. D.

Gwede

C. K.

Vadaparampil

S. T.

Tillery

K. J.

Islam

J. Y.

Giuliano

A. R.

Christy

S. M.

(2022). Strategies and lessons learned during cleaning of data from research panel participants: Cross-sectional web-based health behavior survey study. JMIR Formative Research, 6(6), Article e35797. https://doi.org/10.2196/35797

Bauermeister

J. A.

Pingel

Zimmerman

Couper

Carballo-Diéguez

Strecher

V. J.

(2012). Data quality in HIV/AIDS Web-based surveys: Handling invalid and suspicious data. Field Methods, 24(3), 272–291. https://doi.org/10.1177/1525822X12443097

Bell

A. M.

Gift

(2023). Fraud in online surveys: Evidence from a nonprobability, subpopulation sample. Journal of Experimental Political Science, 10, 148–153. https://doi.org/10.1017/XPS.2022.8

Bell

C. J.

Spruit

J. L.

Kavanaugh

K. L.

(2020). Exposing the risks of social media recruitment in adolescents and young adults with cancer: #beware. Journal of Adolescent and Young Adult Oncology, 9(5), 601–607. https://doi.org/10.1089/jayao.2020.0018

Bethlehem

(2010). Selection bias in web surveys. International Statistical Review, 78(2), 161–188. https://doi.org/10.1111/j.1751-5823.2010.00112.x

Bonnamy

Carr

Lazarus

M. D.

Connell

(2025). Survey sabotage: Insights into reducing the risk of fraudulent responses in online surveys. Anatomical Sciences Education, 18(8), 767–773. https://doi.org/10.1002/ase.70015

Brainard

Killett

Houghton

Bunn

Watts

Mumford

O’Brien

S. J.

Lane

(2022). The wasps are clever: Keeping out and finding bot answers in internet surveys used for health research (2022030243). Preprints. https://doi.org/10.20944/preprints202203.0243.v2

Bybee

Cloyes

Baucom

Supiano

Mooney

Ellington

(2022). Bots and nots: Safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901–911. https://doi.org/10.1080/19419899.2021.1936617

10.

Caven

Yang

Okrainec

(2025). It’s raining bots: How easier access to internet surveys has created the perfect storm. BMJ Open Quality, 14(2), Article e003208. https://doi.org/10.1136/bmjoq-2024-003208

11.

Chandler

Shapiro

(2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12(1), 53–81. https://doi.org/10.1146/annurev-clinpsy-021815-093623

12.

Cheung

J. H.

Burns

D. K.

Sinclair

R. R.

Sliter

(2017). Amazon Mechanical Turk in Organizational Psychology: An evaluation and practical recommendations. Journal of Business and Psychology, 32(4), 347–361. https://doi.org/10.1007/s10869-016-9458-5

13.

Coppock

(2019). Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach. Political Science Research and Methods, 7(3), 613–628. https://doi.org/10.1017/psrm.2018.10

14.

Daikeler

Bošnjak

Manfreda

K. L.

(2020). Web versus other survey modes: An updates and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology, 2020(8), 513–539. https://doi.org/10.1093/jssam/smz008

15.

Daikeler

Silber

Bošnjak

(2022). A meta-analysis of how country-level factors affect web survey response rates. International Journal of Market Research, 64(3), 306–333. https://doi.org/10.1177/14707853211050916

16.

Das

Ester

Kaczmirek

(2018). Social and behavioral research and the Internet: Advances in applied methods and research strategies. Routledge.

17.

Dinev

(2007). The centrality of awareness in the formation of user behavioral intention toward protective information technologies. Journal of the Association for Information Systems, 8(7), 386–408. https://doi.org/10.17705/1jais.00133

18.

Donegan

K. R.

Gillan

C. M.

(2022). New principles and new paths needed for online research in mental health: Commentary on Burnette et al. (2021). International Journal of Eating Disorders, 55(2), 278–281. https://doi.org/10.1002/eat.23670

19.

Evans

J. R.

Mathur

(2018). The value of online surveys: A look back and a look ahead. Internet Research, 28(4), 854–887. https://doi.org/10.1108/IntR-03-2018-0089

20.

Griffin

Martino

R. J.

LoSchiavo

Comer-Carruthers

Krause

K. D.

Stults

C. B.

Halkitis

P. N.

(2022). Ensuring survey research data integrity in the era of internet bots. Quality & Quantity, 56(4), 2841–2852. https://doi.org/10.1007/s11135-021-01252-1

21.

Hammond

(2018). Researching men who pay for sex: Using online methods for recruiting and interviewing. Methodological Innovations, 11(1), Article 205979911876840. https://doi.org/10.1177/2059799118768408

22.

Hulland

Miller

(2018). “Keep on Turkin’”? Journal of the Academy of Marketing Science, 46(5), 789–794. https://doi.org/10.1007/s11747-018-0587-4

23.

Kennedy

Hatley

Lau

Mercer

Keeter

Ferno

Asare-Marfo

(2021). Strategies for detecting insincere respondents in online polling. Public Opinion Quarterly, 85(4), 1050–1075. https://doi.org/10.1093/poq/nfab057

24.

Kumarasamy

Goodfellow

Ferron

E. M.

Wright

A. L.

(2024). Evaluating the Problem of Fraudulent Participants in Health Care Research: Multimethod Pilot Study. JMIR Formative Research, 8, e51530. https://doi.org/10.2196/51530

25.

Lawlor

Thomas

Guhin

A. T.

Kenyon

Lerner

M. D.

Drahota

(2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations, 14(3), Article 20597991211050467. https://doi.org/10.1177/20597991211050467

26.

Lawrence

P. R.

Osborne

M. C.

Sharma

Spratling

Calamaro

C. J.

(2023). Methodological Challenge: Addressing Bots in Online Research. Journal of Pediatric Health Care, 37(3), 328–332. https://doi.org/10.1016/j.pedhc.2022.12.006

27.

Lopez

Hillygus

D. S.

(2018). Why so serious?: Survey trolls and misinformation. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3131087

28.

Newman

Bavik

Y. L.

Mount

Shao

(2021). Data collection via online platforms: Challenges and recommendations for future research. Applied Psychology, 70(3), 1380–1402. https://doi.org/10.1111/apps.12302

29.

Nulty

D. D.

(2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33(3), 301–314. https://doi.org/10.1080/02602930701293231

30.

Nur

A. A.

Leibbrand

Curran

S. R.

Votruba-Drzal

Gibson-Davis

(2024). Managing and minimizing online survey questionnaire fraud: Lessons from the Triple C project. International Journal of Social Research Methodology, 27(5), 613–619. https://doi.org/10.1080/13645579.2023.2229651

31.

Pozzar

Hammer

M. J.

Underhill-Blazey

Wright

A. A.

Tulsky

J. A.

Hong

Gundersen

D. A.

Berry

D. L.

(2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional questionnaire. Journal of Medical Internet Research, 22(10), Article e23021. https://doi.org/10.2196/23021

32.

Salinas

M. R.

(2022). Are your participants real? Dealing with fraud in recruiting older adults online. Western Journal of Nursing Research, 01939459221098468. https://doi.org/10.1177/01939459221098468

33.

Simone

Cascalheira

C. J.

Pierce

B. G.

(2024). A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research. American Psychologist, 79(7), 956–969. https://doi.org/10.1037/amp0001183.supp

34.

Storozuk

Ashley

Delage

Maloney

(2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The Quantitative Methods for Psychology, 16, 472–481. https://doi.org/10.20982/tqmp.16.5.p472

35.

van Quaquebeke

Salem

van Dijke

Wenzel

(2022). Conducting organizational survey and experimental research online: From convenient to ambitious in study designs, recruiting, and data quality. Organizational Psychology Review, 12(3), 268–305. https://doi.org/10.1177/20413866221097571

36.

Vehovar

Toepoel

Steinmetz

(2016). The SAGE Handbook of Survey Methodology. Sage Publications Ltd. https://doi.org/10.4135/9781473957893

37.

Ward

M. K.

Meade

A. W.

(2023). Dealing with careless responding in survey data: Prevention, identification, and recommended best practices. Annual Review of Psychology, 74(1), 577–596. https://doi.org/10.1146/annurev-psych-040422-045007

38.

Wright

K. B.

(2005). Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services. Journal of Computer-Mediated Communication, 10(3), JCMC1034. https://doi.org/10.1111/j.1083-6101.2005.tb00259.x

39.

Yarrish

Groshon

Mitchell

Appelbaum

Klock

Winternitz

Friedman-Wheeler

(2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. The Behavior Therapist, 42(7), 235–242.

40.

Zhang

Zhu

Mink

Xiong

Song

Wang

(2022). Beyond bot detection: Combating fraudulent online survey takers. Proceedings of the ACM Web Conference 2022, 699–709. https://doi.org/10.1145/3485447.3512230

Online Survey Researchers’ Experience With Fraudulent Responders

Abstract

Keywords

Introduction

Methods

Recruitment and Data Collection

Measures

Knowledge of Fraudulent Responders in Online Surveys

Online Survey Experience With Fraudulent Responders

Demographic Information

Data Analysis

Results

Participant Characteristics

Knowledge of Fraudulent Responders

How Researchers Handled Fraudulent Responders

Discussion

Limitations

Conclusion

Footnotes

ORCID iDs

Ethical Considerations

Consent to Participate

Funding

Declaration of Conflicting Interests

Data Availability Statement

References