Abstract
In the wake of COVID-19, numerous research projects moved to online data collection to comply with public health guidelines. Since then, many qualitative projects have continued to use online methods to collect data. While online methods facilitated research continuity, they also introduced new opportunities for deceptive behaviors, particularly misrepresentation and multiple participation. Drawing from a recent project that conducted online interviews with young people who detransition after a gender transition, this article describes how fraudulent interviews were identified and dealt with. We present 12 indicators of potential scams in qualitative interviews, including similarities between participants, the type of information provided, participants’ behaviors, and inconsistencies in the narratives. We discuss our overall experience and, in light of recent literature, present strategies to prevent and deal with scams in qualitative research.
Introduction
Following the COVID-19, many research projects that had initially planned to collect data strategies face to face had to move online (Jones et al., 2021; Lobe et al., 2020; Newman et al., 2021). Studies that collect data online can be advantageous for both researchers and participants: they are cheaper and quicker (Deakin & Wakefield, 2014; Hanna, 2012; Pollet & Saxton, 2019), they can reduce ecological costs due to traveling (Hanna, 2012; Jenner & Myers, 2019), and allow a larger (Pollet & Saxton, 2019) geographically distant (Jenner & Myers, 2019; Miner et al., 2012) and more diverse sample (Upadhyay & Lipkovich, 2020). As they may help reduce some barriers to access and facilitate participation (e.g., for people who would not have participated for safety concerns) (Upadhyay & Lipkovich, 2020), online data collection allows to tap into smaller populations (Miner et al., 2012), and hard-to-reach populations (Miner et al., 2012; Upadhyay & Lipkovich, 2020). It is therefore particularly advantageous to use with marginalized, stigmatized or vulnerable populations (Jenner & Myers, 2019; McInroy, 2016; Upadhyay & Lipkovich, 2020). In addition, online interviews appear to produce similar results than in-person, as the rapport between participant and interviewer is not compromised (Jenner & Myers, 2019). Some even argue that they are more beneficial than in-person interview as they can increase the sense of security, help to build trust and allow greater self-disclosure with interviewer (Upadhyay & Lipkovich, 2020), making it very useful for researching sensitive topics (Jenner & Myers, 2019).
However, online studies carry some risk related to privacy (e.g., participants who do not have access to a private space) and to the exclusion of some participants (e.g., because of disabilities or lack of access to a good internet connection) (Carter et al., 2021) and can lead to sampling bias (Miner et al., 2012). Some are also concerned about deceptive behaviors as online studies increase anonymity for participants (Hanna, 2012), and could help participants to misrepresent themselves or to participate more than once (Chandler & Paolacci, 2017). Such deceptive behaviors are labeled differently depending on the study and there is no consensus on how to refer to it, but some terms are criticized more than others. Some refers to it as « fraudsters » (Chandler et al., 2020; Teitcher et al., 2015), while others use « imposters » (Roehl & Harland, 2022), but Pellicano et al. (2023) suggest avoiding those terms as some population may already feel like imposters or fraudsters about their lived experiences. They instead propose to use « scam,» highlighting the need to take into consideration the specificities of the population in choosing the term used.
This article examines a situation of multiple deceptive behaviors in the context of a study about discontinued gender transitions among youth aged 15–25, also referred to as “detransitioners”. At the moment, there is no consensus about the definition of “detransition”, which generally refers to a discontinuation of a gender transition that can be medical, social and/or legal (Vandenbussche, 2022), and research evidence is still lacking. Detransition is a controversial phenomenon and a sensitive topic as debates around it directly impact both the trans community, by having an effect on access to trans care for youth (Ehrensaft et al., 2018) and the detrans community as it amplifies the phenomenon of detransphobia 1 (MacKinnon, et al., 2022a). There is therefore an important need to gain knowledge through trustable sources.
Researching Detrans Youth Populations
Evidence produced so far shows that people who detransition form a heterogenous group in terms of experiences and identity (Expósito-Campos et al., 2023; Pullen Sansfaçon et al., 2023), and may experience many challenges including having to deal with mixed feelings or ambivalence, body dysphoria or regrets (Pullen Sansfaçon et al., 2023) as well as feeling « like a fraud » during their transition (Sanders et al., 2023). Because of the diversity of experiences, there is a need to produce richer and more nuanced understanding of people who discontinue (Butler & Hutchinson, 2020). As such, use of qualitative interview-based research is helpful (Hildebrand-Chupp, 2020).
This paper begins with an overview of participant dishonesty in research. Then, we present a specific project that was subject to scams. We examine participant recruitment procedures and the signs that raised concerns about ‘scam’ interviews in the project. We then outline the measures implemented to detect and address these ‘scam’ cases.
Next, we discuss some ethical issues related to decisions made for handling such situations as having potentially false data compromises the study’s validity. Indeed, the decisions made on how to handle suspicious data will also have an impact on results. For example, in surveys, a study demonstrated that different decisions regarding how to handle suspicious data lead to different associations between study variables, impacting the study’s conclusions (Bauermeister et al., 2012). It is therefore essential to examine risk for each decision taken.
Throughout this paper, ‘scam’ interviews and ‘scammers’ denote interviews carried out by individuals lacking the required lived experience, possibly with the intent of financial gains.
Scams and Participants’ Dishonesty in Research
Participant’s dishonesty or scam in research is not new. It existed before research was conducted online (Flicker, 2004) and is thought to be frequent in research (Devine et al., 2013). For example, in an online study on HIV (n = 1900), Bowen et al. (2008) found that one third of the submissions were invalid due to being multiple entries. Scam may result from either participating without being eligible or from participating more than once (Bauermeister et al., 2012; Miner et al., 2012). Furthermore, there can be several reasons to falsify data, including financial ones (when the study offers monetary incentives), curiosity about the study or malicious intent such as wanting to corrupt data (Bauermeister et al., 2012). The presence of scammers in a given project can lead to major consequences, notably by increasing costs and leading to incorrect conclusions, as well as possibility of missing unexpected data and therefore creating unappropriated recommendations or guidelines (Bauermeister et al., 2012; Strickland & Stoops, 2020).
Several procedures and strategies have already been proposed to minimize the risk of scammers in online surveys (Bartell & Spyridakis, 2012; Lawlor et al., 2021; Teitcher et al., 2015), such as detecting multiple submissions (Bowen et al., 2008) and how to handle the scam and decisions about the data (Bauermeister et al., 2012).
Although scams have also started to be reported in online qualitative studies, such as in-depth interviews (Pellicano et al., 2023), the phenomenon seems newer. Indeed, literature specifically on qualitative studies is mainly recent (see O’Donnell et al., 2023; Pellicano et al., 2023; Ridge et al., 2023; Roehl & Harland, 2022), except for one article (Flicker, 2004), compared to literature regarding scams in quantitative research or not specifically qualitative (Bauermeister et al., 2012; Bowen et al., 2008; Chandler et al., 2020; Chandler & Paolacci, 2017; Glazer et al., 2021; Grey et al., 2015; Lawlor et al., 2021; McInroy, 2016; Panesar & Mayo, 2023; Strickland & Stoops, 2020; Teitcher et al., 2015). Scams in online qualitative research also appear to be more complex to deal with than in surveys (Ridge et al., 2023). Authors therefore highlight the need to verify and adapt strategies used in quantitative research for qualitative one (Jones et al., 2021) as much as developing new approaches to limit scammers specifically in qualitative studies (Bartell & Spyridakis, 2012).
Presented as an ethical issue in research (Teitcher et al., 2015), especially with vulnerable or marginalized populations (Pellicano et al., 2023), the risk of a scam appears even greater when it concerns small populations, leading to a greater impact on results (Chandler et al., 2020). With marginalized or vulnerable populations, however, only a few studies discuss scams and propose strategies (Pellicano et al., 2023), highlighting the need to produce knowledge and propose up-to-date strategies to deal with scams.
The (De)trans Discourses Study
The case examined in this article is based on data collected within the context of a three-pronged study aimed at better understanding the discourses surrounding detransition from various perspectives. These perspectives include youth who discontinue a transition, trans healthcare providers, and an analysis of media discussions on detransition in both traditional and social media. However, this specific case pertains exclusively to one facet of the study: the in-depth interviews conducted with youth.
Originally, the interviews were planned to be conducted in a face-to-face format. However, due to the global COVID-19 pandemic and the socio-sanitary restrictions in place at the time, the research had to shift to an online format. An advantage of this change was the ability to potentially recruit participants internationally, thus increasing the likelihood of achieving the sampling goal despite the relatively small population. A combination of purposive and snowball sampling methods was employed for recruitment. Invitations (in the form of posters) were circulated within social media groups that were likely to include individuals who had discontinued a transition. These groups were primarily located on platforms such as Facebook, Instagram, and Reddit. As “detransitioners” is a label that may encompass diverse experiences and identities within it (Expósito-Campos, 2021), selection of participants relied on self-identification.
Twenty participants, aged 15–25, who self-identified as detrans individuals or described discontinuing a gender transition, were initially interviewed in the fall of 2020 via Zoom. Although the recruitment goal of 20 participants was achieved, it was noted that 19 of them were assigned female at birth, resulting in a highly homogeneous sample. Consequently, a decision was made to conduct a second wave of recruitment, with the aim of interviewing up to 40 participants in total, while also increasing the representation of individuals assigned male at birth.
The second wave of recruitment was carried out in the winter of 2022, utilizing the same recruitment tools and strategies. However, this time the poster was shared on TikTok and Twitter by an individual with a substantial following interested in detransition. The second wave of recruitment yielded a significantly higher number of interested participants within a shorter timeframe. While the initial wave received 48 emails, relatively evenly distributed over six months, the second wave garnered 29 emails, with 25 of them arriving within just seven days. Initially, the research coordinator did not find this response rate suspicious, attributing it to the wider exposure on TikTok. Additionally, some responses ended up in the spam folder and were not seen promptly.
For both recruitment waves, the poster specified the monetary incentive of $30 CAD, eligibility criteria (i.e., beginning and subsequently discontinuing a gender transition, and being between 15 and 25 years old), as well as examples of reasons that could lead to discontinuation.
Doubts in Recruitment, Interviews, and Data Collection
No suspicion occurred during the first wave of interview. The first doubts about some of the participant’s authenticity were experienced during the first three interviews of wave 2. These three participants had systematically refused to turn their camera on, and some details shared during the interview felt uncanny. For example, in two interviews, the interviewer could hear chickens and motorbikes sounds– those two sounds being very unlikely to be heard during the winter in Northern parts of America where participants declared to be based. Some information shared by those three participants also felt erroneous. For example, one participant first mentioned being assigned male at birth and then declared being rather assigned female at birth later in the interview. Every time this type of situation occurred, the interviewer asked for clarification to ensure that they did not misunderstand the person.
After the end of each of the first three interviews, the interviewer documented their observations in a reflexive journal and then discussed them with the principal investigator (PI) and other research assistants on the project. The PI Research Ethics Board (REB) was also informed about the suspicions and a meeting between REB and the team was organized to discuss possible strategies to prevent potential scams in the next interviews. Some solutions were discussed (e.g., removing the monetary incentive or assessing participant’s IP address) but those were judged not suitable for the project. For example, although several research shows that monetary incentives impact substantially deceptive behaviors in quantitative research (Bauermeister et al., 2012), close to six times more (Bowen et al., 2008), compensation was considered important by the team (and therefore kept) given that gender diverse people are known to face an important risk of research fatigue (Bauer et al., 2019). The team has also thought or removing the mention of monetary compensation on the poster, but it had already been widely circulated.
Finally, the team and REB felt that requiring participant to turn their camera on could act as a sufficient deterrent to scammers. First, potential participants were informed via email that the research was paused due to a large number of responses. Then, amendment was submitted to the REB and a message was sent to potential participants to inform them of the change. Only 6 persons answered back, with 4 who completed the interview and two who did not show up at the interview.
Five further interviews were conducted after. However, the change to the protocol did not protect against the emergence of new suspicions. As such, the interviewing process became tedious and demanding for the interviewer who grew more apprehensive and worried about potential scams. The interviewer verbalized the challenges of remaining aware of possible authenticity issues and at the same time not assuming deceptive behaviors from participants and making them comfortable to disclose sensitive information. The poor sound quality coupled to technical challenges and difficulties encountered in understanding some local accents complicated the process. Some interviews were also significantly shorter than others.
As the stress, frustration, and self-doubt increased for the interviewer, and after two interviews were canceled after being scheduled (participants not showing up), the PI decided to end the recruitment altogether after a total of 28 interviews completed (including the first and the second wave), some of which were potentially from scammers.
Dealing With the Potential Scams
Faced with potential scam interviews and lacking clear guidelines from the REB or available literature, the team opted to create their own process. Initially, all interviews were transcribed verbatim, irrespective of concerns about their authenticity. Subsequently, the interview transcriptions were meticulously reviewed for accuracy by the original interviewer, who revisited the audio recordings and made notes on potential discrepancies and inconsistencies in participants’ discourse.
Another member of the team independently listened to the interviews and examined the verbatim transcriptions while noting any inconsistencies. Their perspective and discussion about the interviews confirmed the suspicions. This initial phase led to the identification of five interview scripts marked by varying degrees of inaccuracies, as well as three others that were free from such concerns.
The eight scripts were classified in three categories (a) major incoherence (b) suspicion about possible scams and (c) no doubts about authenticity. Analyzing the interviews and creating levels of suspicions to categorize participants is helpful in making decision post interviews (Lawlor et al., 2021; Roehl & Harland, 2022).
Next, to limit potential bias in assessing interviews for possible scams, a third research assistant working on another prong of the study and not involved in youth interviews was asked to review unmarked copies of all 8 transcriptions, and make a note of any inconsistencies, contradictions or incoherence. They were also asked to document their observations in a journal, and to classify the transcription according to three same given categories.
Next, the PI conducted a comprehensive review of observations and annotated interview transcriptions provided by the 3 research assistants to evaluate the coherence between their assessments. Interviews that exhibited consistent “major incoherence” were identified as ‘scam’ and excluded from the analysis. Two interviews were identified with varying levels of suspicion by two staff members but not by the third. The PI made the decision to retain these interviews after careful consideration of the risks associated with analyzing potentially inauthentic content versus the risks of discarding genuine interviews that raised suspicion due to differences in expression or cultural nuances, as it will be elaborated on later in the article.
Within a week of completing the process, the decision-making process and criteria as well as the rationale for retaining or discarding interviews were thoroughly explained to the entire team during a meeting. Additionally, a written account of the process, along with a note specifying the three interview numbers that were excluded, was submitted to the REB for documentation.
Engaging in Reflective Practice to Better Manage Future Possible Scam in Research
Reflecting on our project and the process, we implemented to detect fraudulent interviews has enabled us to identify a set of potential scam indicators. This section presents 12 such indicators that arose from our experiences. We have organized them chronologically, according to when they were identified, and we have reviewed them in the context of recent literature. Additionally, we explore our experiences in light of prior research and the strategies proposed to prevent potential scams.
Indicators During the Recruitment Process
Large Volume of Emails Received in a Short Period of Time
The first indicator appeared shortly after launching the recruitment campaign, in the context of our first communication with potential participants. We saw a large volume of emails from potential participants arriving in the project mailbox within a very short period of time. The volume was much larger than during the previous round of recruitment despite recruiting within a relatively small population. This indication was also found recently in the literature published on scams in qualitative internet research (Pellicano et al., 2023; Ridge et al., 2023).
Timing of Email Receipt
We observed that a significant number of emails arrived in our mailbox simultaneously or within a few minutes of each other. This discovery was made during our investigation into potential scams when we began scrutinizing emails, many of which had initially been flagged as spam. Additionally, upon reviewing the timing of when we received the majority of these emails, we noticed that a significant portion arrived during nighttime hours, which suggested that some participants might be contacting us from different time zones. Since our team was conducting international recruitment, this wasn’t initially considered suspicious. However, during interviews, some participants in potential scam interviews claimed to be in a similar time zone than us, which didn’t align with our expectations based on the emails’ timestamps. This experience aligns with that of another research team (Pellicano et al., 2023).
Similar Email Address Format
In our project, we noticed that a large bulk of emails were sent from common providers like Gmail or Yahoo, and were often presenting according to a similar format such as “namesurnamenumber” followed by the domain (i.e.,
Similarity in the Message Contents
We received several messages with very brief content, using similar wording, such as “Hi, I would love to participate in your study,” “Hi, I am interested in the study,” “Hi, I want to take part in the study,” or “Hello, I am interested in participating.” Furthermore, most of these were sent without subject lines and provided minimal, if any, self-description. This observation again aligns with recently published literature (Pellicano et al., 2023; Ridge et al., 2023). Retrospectively, we are now able to confirm that emails from all the scam interviews in our project used that sort of brief content, compared to messages from other participants not identified as suspicious.
Rapidity of Response
We noted that potential participants responded faster than what we would usually expect after we responded to their initial intention to participate. Some even asked for their interview to be scheduled the following day. Pellicano et al.(2023) also noticed that dishonest participants responded quickly.
Scam Indicators During the Interviews
Refusal to Turn the Camera on
Some participants that we eventually identified as part of a scam refused to turn the camera on, an aspect identified in three other journal articles (Pellicano et al., 2023; Ridge et al., 2023; Roehl & Harland, 2022). That said, considering the sensitivity of the topic, not wanting to turn the camera on may be requested for genuine reasons. Confidentiality concerns may be particularly salient for some groups like detrans population as they may be easily recognizable, similar to participants from the trans and nonbinary population (Bauer et al., 2019). Indeed, people from gender diversity are also particularly vulnerable to privacy concerns, as some information that seem anonymized for other populations might not be for them, especially in small geographic areas or when the person has particularities (Bauer et al., 2019). Detrans people are even more at risk of being recognized as they represent a smaller population. The right to privacy and to confidentiality are very important to uphold in research to ensuring trust (Government of Canada, 2023) and lack of privacy and confidentiality are known common barriers to participation in research with trans and gender diverse persons (Asquith et al., 2021). After reflection, the team has decided to request to turn the camera on. After submitting an ethics amendment, we informed all forthcoming participants that camera would need to be on to proceed with the interview and let them the option to decline to participate. However, we want to stress that even after implementing the change, we continued to have doubts about some in camera interviews that were eventually discarded.
Poor Sound Quality, Technical Problems and Other Sound Cues
In our project, we came across instances of peculiar background noise. For example, in some interviews, we could hear noise of clucking or of motorbike engines, despite the interviews taking place in the middle of winter at the participant’s location. Pellicano et al. (2023) have already suggested that poor sound quality or technical problem could serve as a potential indicator of scams. Although it could have been explained as coming from background TV, these aspects, mixed in with other aspects, raised substantial doubts for the interviewer.
Interview duration can also sometimes provide clues of scams, especially in case of shorter interviews (Pellicano et al., 2023; Roehl & Harland, 2022). In insight, we now realize that several suspicious participants asked if the interview were ending soon, after 45 min to 1h. After comparing those with the other interview transcripts, we now realize that scam interviews were substantially shorter (up to 7 times shorter) than the ones that have been assessed as authentic.
Brevity of Responses
In comparison to the initial wave of interviews, scammers’ responses were notably brief and vague, often lacking content details. While this was recognized as a red flag at the time, the team hesitated to take action due to concerns about inadvertently introducing a form of cultural discrimination. Many participants in the second wave disclosed their immigrant status. Historically, specific groups, such as ethnocultural minorities, have experienced injustices and exclusion from research (Government of Canada, 2023), underscoring the significance of acknowledging diverse lived experiences within various sociocultural contexts. This highlights the need to challenge potential assumptions, especially in international studies (Henrickson et al., 2020), that responses should adhere to a specific narrative.
To respect participants’ dignity, researchers should acknowledge them as experts in their own lives and value their experiences (Henrickson et al., 2020). Before questioning the genuineness of an interview based on the details provided, it is important to consider other factors, such as whether the study aligned with their expectations or was relevant to their experiences (Henrickson et al., 2020). Individuals may exhibit variations in their communication styles (e.g., speaking more or less than usual) within a particular cultural or socioeconomic context. It is also crucial to respect participants’ preferred terminology instead of expecting them to use specific terms. Specifically in the case of research on detransition, not respecting a participant’s specific way of describing experiences has been identified as a significant issue that may deter future research participation (MacKinnon et al., 2023). In our project, the lack of details in interviews was not considered, as these elements could possibly be explained by cultural differences, comfort levels in discussing oneself, language barriers, or issues related to gender and sexual orientation, especially when answering questions in a second language.
That said, participants should provide answers that demonstrate a basic comprehension of the topic (Roehl & Harland, 2022). While the absence of specific details doesn’t automatically imply deceptive behavior (Ridge et al., 2023), our research has revealed that scammers consistently offered vague responses, lacked concrete examples, and appeared preoccupied or distracted during the interviews. These are indicators consistent with findings from other studies that included scam interviews (Pellicano et al., 2023; Ridge et al., 2023; Roehl & Harland, 2022).
Incoherence and Inconsistencies
In our study, inconsistencies were primarily related to the ages, sex assigned at birth, names, and locations mentioned by the participants. As the interviews progressed, we noticed that participants sometimes contradicted themselves when discussing specific information. For example, they might have stated one sex assignment at birth at one point and then mentioned something different. Inconsistencies like these have been observed in other research involving potential scammers, such as a participant declaring themselves as autistic and later mentioning that they are rather a parent of an autistic child (Pellicano et al., 2023; Roehl & Harland, 2022).
Asking similar questions multiple times during an interview can be an effective strategy to identify inconsistencies in the participant’s narrative. Researchers can also verify if references to times and events are logical (Roehl & Harland, 2022), a strategy already employed in surveys (Nosek et al., 2002). Interviewers may also pose challenging questions for individuals not well versed in the subject matter (Roehl & Harland, 2022). In our study, the interviewer often used this strategy by requesting more details and double-checking information when inconsistencies or unclear elements arose during the interview. However, this required the interviewer to be well prepared in advance, possess an in-depth understanding of the topic, and conduct semi-structured or unstructured interviews that allow for additional prompts, as structured interviews do not permit this flexibility. Excessive prompting may also diminish the participants’ sense of safety or come across as confrontational, especially when interviewing vulnerable populations (Flicker, 2004; Roehl & Harland, 2022). Although it is possible to simultaneously create a safe space while being assertive and transparent when seeking clarification (for concrete examples, see Flicker, 2004; Roehl & Harland, 2022), it’s crucial to strike a balance (Flicker, 2004).
The literature indicates that a post-interview review of verbatim transcripts to pinpoint possible inconsistencies or contradictions in participants’ statements, along with potentially coding these inconsistent responses, can be a valuable practice (Roehl & Harland, 2022). However, despite the effectiveness of this step in identifying some scammers, it may not be possible to identify all of them using this technique. In our study, we still have suspicions about one interview that did not reveal such inconsistencies or incoherence.
Scammers Refer to Similar Content
Two participants referred to themselves using the same name (either pre, or post-transition). Shared name was not identified in previously published literature on the topic, but previous research has discussed that participants may have similar voices, stories, mannerisms or share sociodemographic characteristics (Pellicano et al., 2023; Ridge et al., 2023). A study also mentioned having had a participant on zoom with the same name written as someone else that was scheduled a few days later (Roehl & Harland, 2022), a situation which also happened in our study and raised additional suspicions.
Strong Financial Interests
Suspicious participants displayed a particular interest in the compensation provided for their participation in the interviews. Some even asked assertively about payment at the end of the interview. Several of them sent follow-up emails inquiring when they could expect to receive their compensation. It remains unclear whether the amount of the monetary incentive affects the prevalence of deceptive behaviors (Chandler et al., 2020). Ethical guidelines for conducting research with gender minorities highly recommend offering compensation as a recognition of participants’ time and expertise (Bauer et al., 2019; Henrickson et al., 2020). Thus, offering monetary compensation in the context of our research appeared relevant.
However, some have suggested that research teams should consider possible alternatives to monetary compensation, such as implementing a lottery, or only mentioning monetary compensation at the end of the consent form rather than in the initial invitation to participate (Pellicano et al., 2023; Ridge et al., 2023). Teitcher and colleagues (2015), on the other hand, question the ethics of not disclosing financial compensation to deter deceptive behaviors. They emphasize that participants have the right to be informed and suggest raising awareness about the harmful effects of scams while potentially reducing the monetary incentive (Teitcher et al., 2015). In our study, a strong interest in monetary compensation was considered a potential indicator of fraud, although this aspect did not appear as clearly in other studies.
Dealing Ethically With Scam Interviews in Qualitative Research
Our experience in handling fraudulent interviews has led us to identify 12 key indicators to consider during the recruitment and data collection process. However, as discussed earlier, it’s not always straightforward to pinpoint fraudulent interviews based solely on these indicators, as other factors can come into play during recruitment and interviews.
On the one hand, managing potentially fraudulent interviews can be a source of stress for the team responsible for handling these situations. The decision of whether to retain or discard research materials can be anxiety-inducing and may affect the team’s ability to make a clear choice. As highlighted by Nowell et al. (2017:1), ensuring methodological precision, consistency, and thoroughness in data analysis is crucial for qualitative research to be deemed trustworthy. Discarding one or more qualitative interviews may raise questions regarding methodological rigor and the overall trustworthiness of the research.
When preparing research papers, disclosing the removal of interviews can also lead to concerns from reviewers regarding the trustworthiness of the research process, especially if the provided explanation lacks sufficient detail. This is often a challenge, considering the limited word count allowed in scientific publishing.
On the other hand, keeping and analyzing data from distrusted sources, such as scam interviews, can also give rise to issues of trustworthiness. Despite the presence of potential scam indicators, arriving at a definitive decision regarding which interviews to retain or discard presents a complex challenge. This complexity can lead to the unintended exclusion of genuine interviews or the retention of fraudulent ones. Pellicano and colleagues (2023) also emphasized the difficulty of balancing data validity optimization without compromising trust with participants and research accessibility, all while avoiding the potential stereotyping of certain groups based on assumptions about their likelihood to engage in deceptive practices.
In research involving vulnerable populations, such as detransitioning youth, the primary focus should be on upholding their trust in the research process, as a lack of trust has been shown to create barriers to participation (Adams et al., 2017; Asquith et al., 2021). Historically, gender and sexual minority groups have faced pathologization in research, and some projects have breached ethical standards (Martin & Meezan, 2003; Reicherzer et al., 2013). For example, mistrust in research can significantly hinder the participation of transgender women in studies, particularly in HIV research (Reisner et al., 2020). Detransitioning youth often already perceive medical professionals as untrustworthy, causing them to disengage from healthcare systems (MacKinnon, Kia, et al., 2022b). Therefore, it is crucial to address trust issues and maintain a strong trust relationship with this population. In this context, it underscores the need to carefully consider how to strike a balance between, on one side, excluding potential scams to maximize research validity, which, in turn, ensures more accurate conclusions and better recommendations that can enhance the population’s trust and, on the other side, affording participants the benefit of the doubt to reduce the risk of mistakenly excluding eligible participants, which could erode trust within the population. Making that sort of decision therefore requires the researcher to carefully examine the risks as well as the benefits of removing potential scam interviews.
The principle of justice, which involves ensuring equitable access to research for marginalized or vulnerable groups (Bauer et al., 2019), should be central in managing potential scam interviews. It helps prevent unjust exclusion of interviews. Researchers must strive to balance data integrity with the respect for participants’ dignity, their right to autonomy and confidentiality, and the need to minimize the research burden while maintaining trust, especially with vulnerable or stigmatized populations. In each case and at each stage, the severity of potential harm, as well as the likelihood of its occurrence, should be assessed (Government of Canada, 2023). Prior to commencing the research, during the project design phase and when applying for ethical approval, researchers must take measures to minimize the risk of potential scams. This can be achieved by considering factors such as the impact of providing compensation and the compensation type (Pellicano et al., 2023; Ridge et al., 2023), whether to disclose compensation in the participation invitation (Teitcher et al., 2015), or by requiring participants to always keep their camera on, a strategy we applied in the project to prevent future potential scam.
In some cases, despite the strategies implemented to prevent potential scams, researchers may encounter situations where they begin to doubt the authenticity of conducted interviews. Consequently, researchers may need to deliberate and decide on whether to retain or discard certain interviews, similar to our experience in this project. In the final section of this paper, we present a method (Legault, 2003) for engaging in this decision-making process to ensure that both the process and the outcomes maintain the trustworthiness of the research. During this process, researcher should meticulously record any steps taken, and ethical board should be kept informed as much as possible. 1. Identify the issue and establish the initial standpoint. The initial stage involves elucidating one’s instinctive feelings about the situation. This preliminary step should be carried out individually. It holds significance as it aids in unearthing potential assumptions in the situation and helps mitigate decision biases. To accomplish this, the researcher needs to pinpoint their immediate judgments regarding the situation, unveil their intuitions about the interviews, and understand the reasons behind them. Researchers should document what they believe is transpiring, elucidate the basis for their suspicions, and identify which interviews are likely fraudulent. 2. Review suspicious interviews and identify ethical concerns. For each suspicious interview, researcher systematically identify the aspects or dimension of the interview that cause concerns. In our project, this was accomplished by asking other research assistants to read the interviews and identify, individually, specific fraud indicators such as incoherence or inconsistencies in discourses (see Table 1 below). We recommend that this process is conducted at the same time as listening to the audio of the interview to also identify sound clues that may be impossible to identify by only reading the script. Research assistants also rate each interview as i. Very likely fraudulent; ii. Some concerns about fraud; ii. No concern about fraud. 3. Making an ethical decision through a rational resolution of the situation. Once each research assistant has completed this task, the researcher compares each research assistant’s assessment (i, ii, or iii), examines the specific fraud indicators and their consistency between each assessment, and make a decision based on this information. For example, if all research assistants assess one interview as ““i. very likely fraudulent”” and identify the same fraud indicators, the researcher flags the interview for potential discard. If research assistants rate the interviews differently or identify different fraud indicators or aspects, leading to inconsistencies in the assessments, the researcher sets the interview aside. The researcher reviews the interviews designated for discarding once more and reflects on issues that could potentially affect the trustworthiness of the project, both from the participant’s and the scientific community’s perspectives. The researcher evaluates their decision in light of ethical research guidelines, particularly those relevant to the specific population under study. The same process is applied to the interviews that were set aside, with careful documentation of the reasons behind each decision. 4. Establishing genuine dialogue with the individuals involved. Researchers and research assistants convene to review the process and outcomes of the interviews that were assessed. The researcher presents the facts and ethical principles guiding the decisions to either (a) discard or (b) retain the interviews. With the team, they discuss and scrutinize the decisions and any lingering doubts despite the systematic process. Key points from this discussion should be documented and later compared to the spontaneous decisions (step (1) to ensure that the process was consistently followed by all team members. For instance, if everyone arrives at the same response (e.g., A) without new information, it may indicate that assumptions were unduly influencing the decision-making process, prompting a reexamination of the process. This step should allow ample time for the team to reflect and reach a consensus, addressing any remaining doubts. The final decision should be submitted to the ethics board and formally documented in the research file. Indicators of Scams.
Conclusion
Ethical issues in research often bring about stress and lingering uncertainties. When confronted with a potential scam situation and scant literature to guide decision-making, teams had to critically reflect on the situation and developed rigorously structured process to ensure that research maintains its trustworthiness. Looking back, many of the 12 scam indicators we identified during our interview review process have since been published, reinforcing similar findings in other projects. Our article, which describes these indicators and the process we undertook, contributes to knowledge on research methods by adding to the scarcity of literature on this topic, by validating indicators of fraudulent interviews and proposing a systematic approach to address these issues.
Although it is impossible to guarantee the absolute rightness of every decision, any ethical decision-making process should document each step and provide clarity for accountability to the various stakeholders involved, including participants, the population covered by the study, and the scientific community. This article has proposed a systematic way to do so.
Further research is necessary to gain a deeper understanding of this phenomenon in qualitative research, particularly when working with vulnerable or marginalized populations. This underscores the importance of researchers and ethics boards to be better informed about such situations and prepared to respond efficiently. It emphasizes the need to develop strategies for detecting and preventing scams in future research and the importance of examining potential indicators in more detail while enhancing support for research teams in these situations.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Social Sciences and Humanity Research Council, funding number: 435-2020-0632.
