Abstract
Although the issue of bots and fraudulent participants is well established within quantitative research, in recent years there have been increasing incidences of imposter participants within qualitative research. However, how qualitative researchers conceptualise this challenge and what the perceived impact of these imposter participants are, remains underexplored. This systematic scoping review identified 15 articles published since 2018 addressing the topic of imposter participants and fraudulent data in synchronous qualitative research. The review identified that the majority of current articles are commentaries or case study narratives, with little apparent inter disciplinary engagement. Findings indicate that where recommendations are offered these can be subjective or influenced by discipline, with a lack of an evidence informed approach being adopted. The analysis identified three primary issues for applied qualitative research fields, with threats to data integrity and reliability, threats to research diversity, accessibility and reach, and questions of trust and ethics within research highlighted. Developing evidence-based guidance and ensuring cross-disciplinary engagement will be central to maintaining the relevance, impact, and validity of applied qualitative research.
The use of synchronous online qualitative data collection, involving the process of generating qualitative data using digital or remote methods such as completing one-on-one interviews via web-conferencing technology, became ubiquitous during the COVID-19 pandemic (Hensen et al., 2021). While online data collection was utilised before the pandemic, the shift to remote work necessitated more widespread use of online methods in research. Online methods for generating qualitative data can be advantageous for several reasons, such as extending the reach of recruitment and diversity of sample, supporting larger, more geographically spread participation, and reducing the cost and time needed for data collection activities (Braun et al., 2021; Coulson, 2015). Online approaches are also recognised as enabling researchers to reach more marginalised, minoritized populations that may otherwise not engage, or not have had the opportunity to engage, in research previously (Seide et al., 2023; Tates et al., 2009). However, more recently it has become apparent that there is an emerging issue with individuals engaging in qualitative research who may be misrepresenting their eligibility for research studies.
The problem of bots and fraudulent participants in research is not a new one -bots are computer programmes designed to mimic human responses and will complete online research surveys whereas fraudulent participants are when people participate in research but misrepresent themselves or their responses. The issue of inauthentic participation in qualitative research was raised 20 years ago (Flicker, 2004) and quantitative research has a breadth of studies on the topic (Garcia-Orosa et al., 2023; Griffin et al., 2022). As to date, most instances have occurred online, the response has largely been to look for technological solutions to verify that participants are humans (such as CAPTCHA or security scan monitoring). However, more recently there has been an increasing phenomenon of fraudulent, or misrepresentative, data and imposter participants appearing within qualitative research studies with indications of up to 80% of responses potentially being fraudulent in some studies (O’donnell et al., 2023; Salinas, 2023). Although these levels are reflective of internet-based questionnaire type studies (where participation is asynchronous), it is also becoming apparent to researchers that imposter participants are emerging in “traditional” interview or focus group research (Drysdale et al., 2023; Pellicano et al., 2023). As qualitative researchers are aware language choice is important and the terminology to use to describe this problem is unclear and subjective. For the purposes of this review the team’s preference was to use “imposter” rather than “fraudulent” when referring to ineligible participants to avoid possible inference of criminal intent.
Despite the increasing incidence of this challenge, to date there has not been systematic engagement of the issue, resulting in limited consideration of the unintended consequences of different proposed approaches and solutions. The increasing scale of imposter participants and misrepresentative data for applied researchers across multiple disciplines has generated new interest in how to address the issue. To move forward, in terms of both informing the design of research studies and developing appropriate responses to imposter participants and data they contribute, there is a need to consolidate understanding of the existing literature. Specifically, there is a need to focus on synchronous qualitative data collection research where the problems, potential solutions, and perspectives may be different than those proposed within quantitative domains. To address this, we undertook a scoping review of existing literature with an aim of understanding: • How is this issue currently being conceptualised within synchronous qualitative research? • What are the impacts of imposter participants on applied qualitative research? • What are the pathways to mitigating the risk of imposter participants and misrepresentative data within applied qualitative research?
Methods
This scoping review followed the stage-wise framework developed by Arksey and O’Malley (2005). This included (a) identifying the research question; (b) identifying relevant studies; (c) selecting studies for review; (d) charting the data; and (e) collating, summarizing, and reporting the results. The findings of this review are reported in accordance with the PRISMA-ScR guidelines (Tricco et al., 2018).
Search Strategy
PICO Terms Used to Inform Search Strategy.
Inclusion and Exclusion Criteria.
Results were exported from the databases in complete batches on July 18th, 2023. The synthesis review management software, Covidence, was used to facilitate the deduplication of results. This software was also used to manage the title/abstract and full-text screening phases of the review. A review of published data from the initial data screening and extraction to the 4th of July 2024 was completed along with hand-searching of references resulting in an additional 7 papers being identified and included in the review (see Figure 1). PRISMA Diagram Identification and Screening Process.
Screening
All results were retrieved into Covidence for screening. Using the inclusion and exclusion criteria in Table 2, MVH, CW, MH and RP independently screened all entries by title and abstract in duplicate, with no conflicts to report. Full-text screening was carried out by MH, RP, JE, and AD in duplicate. Results from the screening process are reported in the PRISMA diagram (see Figure 1).
Charting the Data and Data Items
Each article was reviewed in full, with all data extracted by MH and RP independently of each other. Article characteristics, including the location of the publication, the type of publication, and the population described, how imposter participation was conceptualised and described, the described impact of the imposter participants, and how risk was mitigated was extracted. In addition, reviewers identified where author recommendations or methodological processes were present. The data was then subsequently discussed and synthesised to answer the posed research questions. As no disagreements arose, no third researcher was required for extraction. Due to the designs of included papers (i.e., commentaries), a quality appraisal was not possible. A narrative analysis is presented in the results below.
Results
Descriptive Summary and Characteristics of Scoping Review Papers.
aAdopts the use of “scammer participants” or “scam interviews” as proposed by Pellicano et al.
bAdopts Roehl & Harlands definition.
Conceptualization of Imposter Participants
There was similarity across papers in relation to how researchers conceptualised this phenomenon as a problem of misrepresentation of identity, either in full, or partially (e.g., papers 5, 8, 9). However, there was discrepancy in relation to the terminology used when referring to participants (imposters, scammers, fraudulent etc.,) and the framing of the problem. This was reflected in the choice of terminology used, with terms such as “untrustworthy,” “fraudulent,” “scammers” and “subversive,” reflecting where authors attributed intent or malice from participants, contrasting with other papers, (e.g., 1, 5, 10) where broader definitions were used (see Table 3). Differences in conceptualisation were seen to map to differences in discipline, for example, the risk associated with imposter participants’ presence within medical research leading to clear blame being directed towards imposter participants. In contrast, where researchers may have less positivist epistemological positions, or who work with more marginalised groups, expressing a less rigid representation of the problem. The existing literature therefore indicates that disciplinary values and researchers’ epistemological approach influences the conceptualisation of the issue, interpretation of causes and decisions of mitigating actions. Some articles (e.g., 5, 11, 12) discuss the potential differences between those who falsify their identity for financial gain (or other similar motive) compared to participants who exaggerate or embellish their accounts of a study’s phenomenon, with these motivations driven by other factors. Similarly, it was acknowledged, for example in McCaul and Wand (2018), that data integrity can never be guaranteed even with “genuine” participants, due to issues such as non-disclosure of adverse events by trial participants for fear of losing their place within the trial. Nevertheless, it was asserted across all included papers that financial gain was a primary driver of the behaviour seen. Table 3 provides a summary of the way the problem was conceptualised within all papers.
Impacts on Applied Qualitative Research
The analysis of the impacts of imposter participants and fraudulent data on applied qualitative research, can be considered in both philosophical and practical terms. There are three core philosophical issues evident within the literature: the threat to research accessibility and diversity (1,2,3,7,9,14); the question of trust and ethics within research (1,3,5,7,8,9,10,11,12,14,15) and the threat to research data integrity and reliability (1,2,4,5,6,7,8,10,12,13,14,15).
When discussing the threat to accessibility and diversity, papers start from the position that a core strength of qualitative research is its ability to engage with marginalised and minority populations and hear the voices of groups whose individual needs or perspectives risk being lost within quantitative research. Further, there was recognition that the move towards online synchronous data collection, whether driven by convenience, cost, or COVID-19, has further enabled engagement with traditionally underserved populations. Similarly, there are benefits to researchers, who are now less restricted by geography and budget. The risk of imposter participants within online qualitative research, or by using online recruitment methods, was a particular focus within some of the papers identified (4–8) and this was also discussed in practical terms. For example, Bybee et al. (2022), highlighted the problem of imposter participants as being a significant barrier to addressing “real-world problems” such as health inequality, whereas Drysdale et al. (2023), presented the problem as creating an additional barrier to engaging vulnerable, stigmatised, or underserved populations in research studies. Within the papers identified, all authors argue to some extent that the potential inclusion of imposter participants draws attention to issues of trust in relation to both research populations/communities and with regards the researcher/participant relationship. However, differences reflective of discipline and context differences could be seen. For example, McCaul and Wand (2018) took a highly medicalised approach, focusing on the implications of deceptive behaviour for the reliability and trustworthiness of medical research data. Other papers, which were informed by social science perspectives, foregrounded the implications for trusting relationships between researchers and participants. Drysdale et al. (2023), for instance, discussed the bidirectionality of trust and how central trust is to research within marginalised or minority communities. Similarly, Roehl and Harland (2022) and Woolfall (2023) highlight the risks of excluding genuine participants – either from the beginning or due to suspicions regarding the data that is subsequently produced. However, as acknowledged in Pellicano et al. (2023) & Pullen Sansfaçon et al. (2024), the decisions over exclusion being proposed currently within the papers included in the review are, in most cases, being made subjectively. Kumarasamy et al. (2024) was unique in raising the question over whether researchers have a moral duty to identify imposter participants, or risk supporting criminal activity.
Beyond these ethical and philosophical questions, several practical impacts were noted including those specific to participant recruitment and those relevant to data collection often framed by a desire to mitigate the risk (see below). In relation to recruitment the issue of financial incentives and methods of recruitment are consistently discussed. There is a consensus in the papers on the inherent tension between advertising incentives being a driver of imposter activity, but also the need for participants to be appropriately remunerated and respected for their time (e.g., Drysdale et al., 2023; Woolfall, 2023). This tension prevents the emergence of a consensus on solutions, such as removing incentive information from study adverts, with this being suggested by some authors (e.g., Woolfall, 2023) and not others (e.g., Ridge et al., 2023). Some practical implications appear specific to the research location, for example, Andrews et al. (2024), experienced challenges over the provision of “local” gift cards as remuneration (which did not have value outside of Australia) with requests for “globally accessible” gift cards then seen as a potential indicator of imposter activity. Choosing “local” voucher payments was then in turn suggested as a possible means for mitigating risk of engagement by imposters, although in Sharma et al. (2024) where this approach was adopted, it did not appear to be a sufficient deterrent. Further, Santinele Martino et al. (2024) highlights the challenge of payment in terms of weighing up the increasing imposter risk versus ensuring an ethically correct means to compensate participants that could not be seen as discriminatory. 1
A further practical impact relates to consideration of the use of online recruitment adverts, with some indication that certain platforms (X/Tik Tok) may drive imposter activity more than others (Pullen Sansfaçon et al., 2024). The other core focus for practical impacts evident within the literature relates to the data collection itself. The issue of requiring participants to have their web-cameras turned on during synchronous data generation such as interviews and focus groups as a means of visual identification check is discussed (e.g., 1,7,10) but the associated ethical concerns over this becoming a compulsory requirement for participation are also acknowledged in the literature (Drysdale et al., 2023). Some papers do also advocate more stringent participant identification e.g., checking of IP addresses (1); undertaking screening calls (11); providing proof of residency (13), but do not provide indications that these recommendations are evidence-based. Similarly, the practical implications of the time required for additional vetting of participant identity or eligibility criteria and the associated challenge to available resources is evident in the literature (2,4,9,12,14).
Mitigating the Risk
Where authors identified negative impacts, suggestions for mitigation strategies were also included, and it is within these strategies that the review papers diverge most significantly. All the papers, except Drysdale et al. (2023), suggest actions to mitigate the risks of imposter involvement, whilst the Drysdale commentary instead discusses some of the strategies other papers suggest and puts forward several counter arguments to technological solutions, such as automatically screening out individuals using virtual private networks. In contrast, other papers, such as Bybee et al. (2022) provides a set of recommendations for researchers if suspicions are raised when undertaking online synchronous qualitative research: Specifically, 1) to review initial entries for time of completion, speed of completion, and content; 2) Temporarily postpone recruitment; 3) Send supplementary eligibility questionnaires to a sub-group and review for suspicious activity (e.g., missing chunks of expected data); and 4) Implement detection measures. It is not clear however whether some suggestions such as asking participants to telephone researchers, are viable, or viewed as acceptable across different disciplines. Further, there is no agreement from authors over possible mitigating behaviours that researchers should adopt. Further, while some, like Bybee et al. (2022), Pellicano et al. (2023), Ridge et al. (2023), include reference to considering automatic detection measures from the study implementation, Drysdale et al. (2023) make valid points about how certain screening techniques may inadvertently exclude certain groups.
Importantly, some of the suggestions put forth within the papers place additional burden upon the potential participants, and/or require ‘verification’ of identities (e.g., Roehl & Harland, 2022), and as such it is unclear the extent to which these processes might be acceptable, practically possible or ethically permissible. The Pullen Sansfaçon et al. (2024) paper opts instead to suggest broad principles for mitigating the risk: 1) Identify the issue within the research team and determine the teams initial standpoint; 2) Review suspicious activities and identify ethical concerns; 3) Make an ethical decision through a rational resolution of the situation and 4) Establish genuine dialogue with the individuals concerned within he research team (rather than the participants).
Recommendations posed from included papers are based on the personal experience and reflection of the researchers and are often heavily situated within author disciplines. Beyond these recommendations, there is little critical reflection of those these disciplinary perspectives may be influencing selected mitigation strategies, nor how disciplinary considerations may impact acceptability to other researchers. Included papers, largely case studies, letters, and reflective articles contribute to this siloed approach. What is missing in the literature is guidance that engages with researchers across different disciplines and epistemological positions to try and develop some degree of consensus of mitigating strategies. Further, any guidance should show sufficient consideration of the tensions apparent in this topic, such as how researchers balance verification processes with accessibility and inclusivity when designing research studies.
Discussion
This review has highlighted the emerging literature relating to imposter participation in synchronous online data collection. As the first known review on this topic to date, we surface cross-disciplinary philosophical, ethical and practical challenges related to participant recruitment and data collection. This evolving body of literature reflects the increased use of virtual technologies for data collection, particularly in the social sciences. Findings shed light on the emerging and likely enduring challenge of imposter participants and misrepresentative data in qualitative research. Across examined papers, researchers largely conceptualise imposter participants as individuals who have misrepresented their identity, with varying degrees of attributed intent.
Findings in this review suggest that many researchers are grappling with the philosophical and ethical implications of imposter participants. While we have surfaced many divergences on how these challenges are conceptualized, three common messages are clear: the importance of trust in data collection, particularly in relation to marginalised communities; the challenges that imposter participants can present both in terms of the integrity of the data included in a study; and the unintended impacts that can emerge from efforts to limit imposter participation (Drysdale et al., 2023). While there were similarities in the responses to suspected imposter participants, and measures to prevent their participation, we did not find agreement on the appropriateness, or practical application, of these measures.
Through looking across a range of literature within this review, we have begun to identify the role that the discipline of study plays in the responses to imposter participants. Different authors reached different conclusions on appropriate responses depending on their methodological and philosophical standpoints in relation to data, participant representation, and the purpose of their research. We can see a continuum of perspectives on imposter participants, with researchers who conduct research in more medicalised contexts at one pole, and those who conduct research in more social science contexts tending at the other. This is not a ‘hard and fast’ rule, but as demonstrated in Table 3 where research was conducted in more medical-based contexts, language used tended to emphasise or attribute intention in descriptions of participants (“deceptive participants” McCaul & Wand, p236) or was more emotive (“scammer participants” Pelicano et al., p1). In contrast, those authors who come from a more social science perspective tended to use language reflecting the actions, rather than the intentions, of imposter participants (“not the people they initially purported to be” Ridge et al., p941). Further, there were differences in the emphasis placed on data integrity, as opposed to ensuring inclusivity. This was reflected in perspectives on appropriate mitigation strategies, ranging from additional screening questionnaires or even blood tests, which placed further burden on prospective participants, to only assessing for inclusion during, or following, the data collection process. It is important to acknowledge how disciplinary values, and epistemological position will influence research design, researcher interpretation and analysis. Some may perceive that more rigid approaches to mitigating the risk of imposter participants is necessary within certain contexts, e.g., drug development. In contrast, we recognise that there will be qualitative researchers who do not agree with ANY conceptualisation of the idea of imposter participants. Simplistic solutions to the issue are unlikely. But, not attempting to engage across disciplinary and epistemological boundaries to develop understanding of each other’s position, and to develop strategies to maintain applied qualitative research quality, will be a lost opportunity.
Current discourse surrounding imposter participants and misrepresentative data within qualitative research largely stems from anecdotal evidence and personal reflections in commentaries and case studies rather than rigorous evidence-informed analyses. Consequently, this literature base still has key gaps, including a concerning lack of interdisciplinary understanding of the challenges posed by imposter participation, as evidenced by the largely isolated nature of the included papers. We hypothesise that the issue may still be too nascent to have yet incited the type of cross-disciplinary knowledge and collaboration that will be necessary to consider and address its implications. There is a clear need for evidence-based methods to further describe this phenomenon including understanding the reach and impact of the issue on research.
Building from a more complete picture of the nature of imposter participation, there is also an opportunity for the development of guidance, including individual or context specific approaches or strategies along with more systemic solutions. As evident in the case studies included in this review, the global community of qualitative researchers has already begun to share recommendations, such as the utilisation of available detection tools within online recruitment or questionnaire platforms. The appropriateness or otherwise of these perspectives and responses is inherently linked to the research questions that are being addressed, the methods that are utilised, and the contexts in which these take place. There is no ‘one size fits all’ solution to the challenges presented by imposter participants. Instead, researchers need to consider how the risk of imposter participants relates to their research aims and context when designing and implementing studies. Discussing how their research prioritises inclusion and diversification alongside the need for data integrity for example is a key step applied qualitative researchers should undertake as early in the research design process as possible. Additionally, future research endeavours should aim to expand from currently largely siloed team-based solutions to develop evidence-based guidance not only applicable across disciplines but also transferable across different domains within those disciplines. Researchers should be engaging with multidisciplinary publications as a means for disseminating on this topic. Further, it would be beneficial if future research enabled the facilitation of multi-disciplinary participatory projects. This would be particularly beneficial to engage with researchers from across career stage and context to ensure a breadth of perspective is gained to enable the development of guidance that is flexible, feasible and reflective of researchers’, and institutions, needs and views. We anticipate that this sort of guidance will prove useful for researchers and journal editors as both increasingly must contend with the growing prevalence of imposter participation. Importantly, this guidance must consider ways changes in research protocols and procedures could also pose risk to non-imposter participants, to not jeopardize sensitive and hard-won relationships between researchers and participants.
Papers identified in this review probe thorny ethical questions that may arise in navigating solutions or mitigation strategies to these challenges. This includes tensions around trust and use of participant incentives in research. It is understood that one potential negative impact of financial incentives in qualitative research is coercion, but in the context of imposter participants, this incentivization may be a main driving force. However, this is juxtaposed with the need to appropriately recognize participant contributions and expertise. Addressing such dilemmas requires a nuanced approach that balances ethical considerations with practical solutions.
Conclusion
This review sought to scope the evidence of imposter participants in synchronous online qualitative research literature. We explore how the problem of imposter participants and participation is being framed, including how this relates to proposed solutions. While three key considerations were identified across this nascent body of work - the impact of on diversity of research participation, the impact on trust between researchers and participants, and data integrity - the solutions proposed were strongly grounded in disciplinary paradigms. Papers drawing on medicalised perspectives prioritised screening and exclusionary practices, foregrounding data integrity, whereas papers drawn from the social sciences emphasised inclusionary practices and trust. The predominance of case studies, letters, and reflective articles currently available regarding imposter participation contribute to this siloed approach to mitigation strategies, where more empirical work might promote a consideration of how the challenges presented by imposter participants might be approached in a more co-ordinated and systematic fashion. To move forward in developing cross-disciplinary guidance that preserves the core values of applied health research, there is a need to both clearly articulate the disciplinary priorities that inform proposed solutions and to look past them to identify cross-cutting principles for engagement with imposter participants.
Supplemental Material
Supplemental Material - Imposter Participants in Synchronous Qualitative Research: A Systematic Scoping Review
Supplemental Material for Imposter Participants in Synchronous Qualitative Research: A Systematic Scoping Review by Margaret Husted, Anna Dowrick, Robert Porter, Maria Velo Higueras, Carly Whitmore, Jane Evered, Megan Kennedy, and Shannon D. Scott in International Journal of Qualitative Methods
Footnotes
Acknowledgments
SS is funded by a Distinguished Researcher Award (Stollery Science Lab) from the Women and Children’s Health Research Institute at the University of Alberta.
ORCID iDs
Ethical Statement
Author Contributions
This was a collaborative project throughout. All authors were involved in the conceptualisation of the review and identification of the research objectives. The search strategy was developed by MVH, SS, MK, CW & JE with the database searches undertaken by MK. Title and abstract screening was undertaken by MVH, CW, RP & MH, with full-text reviews completed by MH, RP, JE & AD. Data extraction and analysis was undertaken by MH & RP with support from AD. The article draft was led by MH, MVH, RP, AD, CW & JE. The final draft was reviewed, edited & approved by all authors.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: MH has received a British Academy Small Grant to support research on this topic (SRG2324\240097).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
There was no raw data generated or analysed for this systematic review. The search terms, database searches, inclusion, and exclusion protocols used are all presented in the method section for the transparency and reproducibility of our work.
Supplemental Material
Supplemental material for this article is available online.
Note
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
