A study Comparing Text-Based WhatsApp and Face-to-Face Interviews to Understand Early School Dropout

Abstract

The majority of adolescents communicate via text-based messaging, particularly through WhatsApp, a widely used free communication application. Written content on WhatsApp has the methodological potential to provide rich qualitative interview data. This study compares data collected using text-based WhatsApp versus face-to-face interview techniques. Semi-structured interviews were conducted with a sample of early school leavers in South Africa, using WhatsApp (n = 9) and face-to-face (n = 27) followed by a focus group discussion with interviewers. Mann–Whitney U and chi-squared were used to assess associations. WhatsApp text-based interviews took significantly longer to complete but were comparable to face-to-face on the number of themes generated. Rapport, measured as the number of statements from the interviewer aimed at creating a sense of affinity, comfort, and distress reduction, differed between interview conditions. It may be methodologically appropriate for researchers to offer participants a choice of a preferred method of interviewing or consider pragmatic blended approaches of face-to-face and WhatsApp.

Keywords

interviews WhatsApp adolescents qualitative

Introduction

In the last decade, internet-mediated communication platforms such as Skype, WhatsApp, and Zoom have been identified as useful qualitative data collection tools (Archibald et al., 2019; Gibson, 2022; Kaufmann & Peil, 2020; Krouwel et al., 2019; Reñosa et al., 2021). The sudden emergence of the COVID-19 pandemic and the restrictions on face-to-face engagement have increased technology use and have forced researchers globally to consider switching to online data collection methods (Chen et al., 2020; Dodds & Hess, 2020). The digital world particularly plays an important role in an adolescent’s social development (Gibbs et al., 2020) as they are increasingly using digital spaces for education, entertainment, communication, and connection, representing one in three internet users globally (Stoilova et al., 2021). Given the increased use of technology for communication and interaction among adolescents, adolescent health research may particularly benefit from online data collection methods.

While there is a wide range of digital communication methods, including video and audio communication, text-based communication makes up a large part of adolescent social interaction (Ehrenreich et al., 2020). Technology offers researchers the opportunity to engage with adolescents using creative ways to elicit self-expression and communication on platforms that they are already comfortable and familiar with. It allows access to diverse groups of adolescents, gaining the voices of historically underrepresented or hard-to-reach adolescent populations (DeJonckheere et al., 2017; Elgar et al., 2015). Online data collection also has the advantage of being less dependent on scheduling restrictions and geographic location (Jowett et al., 2011; Shapka et al., 2016). It is convenient, saves costs, and allows for automatic and accurate storing of raw data (Janghorban et al., 2014; Jowett et al., 2011; Kazmer & Xie, 2008).

One promising method for collecting data online with adolescents in low- to middle-income countries is the use of WhatsApp. WhatsApp is a free smartphone and computer-mediated instant messenger application (App), with two billion users worldwide in 2021 (Iqbal, 2021). The widespread use of smartphones and internet penetration in Sub-Saharan Africa has made WhatsApp the most downloaded App in Africa, with, for instance, 52% of the South African population using it (Steup, 2019). WhatsApp allows users to send real-time text, locations, images, voice recordings, documents, and videos. Compared to other instant messaging Apps, prior registration is not required which is more favourable if internet bandwidth is poor (Church & De Oliveira, 2013). A review of the literature has shown that WhatsApp has been used in many ways including dissemination of surveys through sharing an online survey link (Chen et al., 2020; de Gruchy et al., 2021; Kamel Boulos et al., 2016) in clinical practice to communicate with patients (Kamel Boulos et al., 2016; Mars & Scott, 2016), to support interventions (Cheung et al., 2020; Durmaz et al., 2019; Patel et al., 2018; Pereira et al., 2020) and research projects (Jailobaev et al., 2021), or as an educational tool (Coleman & O’Connor, 2019; Kamel Boulos et al., 2016). Relative to the number of WhatsApp users in low- and middle-income countries, to date, the use of WhatsApp as a data collection tool for qualitative interviews has been limited (Kaufmann & Peil, 2020; Reñosa et al., 2021; Singer et al., 2020).

Face-to-face interviewing has been the golden standard for decades in qualitative research, with published guidelines on best practices (Opdenakker, 2006). However, in a digitalized world, online interviewing methods may also be a plausible tool for gaining insights into the realities of research participants. In a qualitative interview, text-based communication affords adolescents more control of the conversation, and flexibility in responding, and the absence of verbal cues may potentially decrease fear of social judgment (Lee, 2007). The lack of face-to-face contact in online discussions allows participants to speak more freely, especially around sensitive topics (Singer et al., 2020) and to exit discussions with less explanation. The distance created online between researchers and participants potentially limits social judgment and power differentials tend to recede to the background, which can be problematic in face-to-face interactions (Cleary & Walter, 2011; Voida et al., 2004).

Nonverbal cues (head nods, smiles, eye contact, etc.) in face-to-face communication provide speakers and listeners with a richness of meaning and information and are important aspects of qualitative interviewing. These non-verbal elements are generally absent in online discourse and some researchers raised concerns about the flow of conversation and the difficulty in reading or responding to emotions expressed in text from the participant (Opdenakker, 2006; Reñosa et al., 2021). However, studies have also shown that with time and experience, the same richness of face-to-face communication could occur over a virtual text-based medium (Gajadhar & Green, 2003; Jowett et al., 2011; Kazmer & Xie, 2008; Kumari & Ganagwar, 2018). Users have found ways to increase the richness of online communication that is comparable to face-to-face encounters through the creation and use of emoticons, pictograms, and internet slang abbreviations such as “LOL” (Laughing out Loud) (Hancock et al., 2007). These evolving modes of communication have become a distinct style of non-verbal communication in the digital world that is widely understood and an accepted form of communicating emotion or non-verbal elements. Furthermore, while it is possible for an individual to assume a false identity and share false information in an online text-based interview, a longitudinal study validating self-report data with actual behaviour and external data found that the risk of adolescents assuming a false identity and sharing false information tends to be small (Stieger & Göritz, 2006). Given the lack of consensus in the literature and the relatively limited research conducted to date, this study explores the feasibility of text-based interviewing among adolescents.

The few studies that have examined this issue globally have used quantitative methods (measuring interview duration, word count, and themes) to assess the quality of online and face-to-face interview methods. These studies compared telephone (Irvine, 2011), video calling (Krouwel et al., 2019), and computer-mediated instant messenger interviews (such as Microsoft Instant Messenger) (Shapka et al., 2016) to face-to-face interviews. Despite reported variations in the duration and number of words produced in the online condition, results showed that data quality was unaffected by the mode of data collection (online vs. face-to-face) with few differences in the number, depth, and type of themes discussed (Irvine, 2011; Krouwel et al., 2019; Shapka et al., 2016). To our knowledge, no studies have systematically compared face-to-face interviewing techniques to text-based interviewing using WhatsApp among adolescents.

The main objective of this study was to compare the feasibility and acceptability of interviews delivered via WhatsApp to interviews delivered face-to-face, in a sample of adolescents. This study takes a pragmatic approach (Kaushik & Walsh, 2019), which is a strategy that quantifies interview quality based on established best practices (Flick, 2013), but also consider the lived experiences of participants and researchers on the different methods of interviewing. This study, therefore, examined the following two research questions:

What were the differences between the text-based WhatsApp and verbal face-to-face interview conditions with respect to the themes, flow, focus and depth of the interview, rapport (interviewers creating a sense of affinity, comfort, and distress reduction), and interview duration?

What were the opinions, experiences, and reflections of the interviewers and participants using WhatsApp and face-to-face interviewing methods?

Methods

Study Participants

The interviews were conducted among early school leavers in South Africa. Early school leavers are considered a hidden population, with no existing register or database of adolescents once they drop out. Participants were recruited from four urban communities in the Western Cape province in South Africa, due to the high percentage of early school leavers in this region (Desai et al., 2019). Early school leavers were defined as individuals aged between 13 and 20 years who had not completed their schooling or obtained their national senior certificate, and who were not currently attending school. Individuals who were enrolled in college or vocational training were excluded from the study.

Participants were recruited and tracked using respondent-driven sampling (RDS) (Heckathorn, 2002). Using the RDS method, the initial sample or “seeds” of early school leavers were obtained through various channels such as community youth groups, referrals, and by approaching people and then were confirmed to have met the eligibility criteria in the selected communities. Our goal was to obtain four seeds in each interview condition. Simple randomization was used to assign the seeds to either the face-to-face (white in Figure 1) or WhatsApp (grey in Figure 1) interview conditions. Each seed was required to identify up to two early school leavers to participate in the interview condition that they themselves had been allocated to. These participants formed the “first wave” of sampling and were themselves asked to identify and refer a further two more early school leavers. Up to two waves of recruitment were conducted for each interview condition (Figure 1). Participant eligibility was further confirmed through contacts that the potential participant provided. Participants who expressed their inability to participate in the WhatsApp or face-to-face interview that they were initially allocated to were excluded from the study. Each respondent who completed the interview received a monetary reimbursement in the face-to-face condition and mobile data in the WhatsApp condition. Also, for every successful recruit, the recruiter received an additional monetary incentive in the face-to-face condition and mobile data in the WhatsApp condition.

Figure 1.

Respondent-driven sampling for out-of-school youth—a schematic representation of four seeds.

Three interviewers were selected from a pool of applicants based on research experience. Given that the Western Cape is characterized by diverse cultures and languages, the study aimed to gain a diverse representation of interviewers based on demographics and languages spoken in the communities where the study participants were recruited. Up to three interviewers were selected to recruit, conduct face-to-face and WhatsApp interviews, and participate in a focus group discussion at the end of data collection. Interviewer 1 described herself as White, had the most prior experience in qualitative interview training and conducted the interviews primarily in English. Interviewer 2 identified herself as Colored, had moderate experience in conducting interviews and conducted the interviews in Afrikaans and/or English. Interviewer 3 identified as Black African, had less interview experience compared to the other interviewers and conducted the interviews in either English or Xhosa. Race was classified according to the South African Department of Labor designated categories (1 = black African, 2 = Coloured, 3 = Indian, 4 = White, 5 = Other) (Stats SA, 2011).

Data Collection Tools

The interviewers used a semi-structured discussion guide to conduct a one-on-one in-depth interview and each interviewer conducted a mix of face-to-face and WhatsApp interviews. Participants were given the option to have the interview conducted in either English, Xhosa, or Afrikaans, which are the predominantly spoken languages in the region.

The discussion guide covered questions related to participant demographics (race, age, gender), the reasons for dropping out of school, social influences, opinions and experiences of health behaviours that placed them at risk, and their opinions and reflections of the interview. A complete discussion of these findings from the conducted interviews has previously been published (Desai et al., 2020). The semi-structured discussion guide was designed in English and translated into two official South African languages, namely Xhosa and Afrikaans. The researchers used anchor questions to introduce different topics during the interview or to clarify comments made by the participants. The content and format of the interview were identical for both interview conditions. A different semi-structured interview guide was used to guide the focus group discussion among the three interviewers which took place once all interviews had been conducted. Interviewers were asked to describe their educational qualifications, experiences, and opinions on recruiting participants via RDS and administrating the WhatsApp and face-to-face interviews.

Procedures

The face-to-face interviews were conducted verbally and at locations mutually agreed upon by the interviewer and participant, that is, in a private and confidential space at the participant’s home or at a public library, and were audio recorded. The WhatsApp interviews took place at an agreed time between the interviewer and participant. A dedicated WhatsApp account for this study was used by the researchers and participants were contacted using their personal WhatsApp accounts. The participants in the WhatsApp interview had the option of conducting the interview in a single session or over a few days. Interviewers initially introduced themselves over a voice note message. Thereafter, participants and interviewers only used text in the WhatsApp condition and were encouraged to use emoticons.

Once the interview was over, participants were requested to delete the interview conversation on WhatsApp. The WhatsApp conversation log was converted into an interview transcript. Once the chat was downloaded and saved on a password-protected computer, the chat was deleted from the App by the interviewer. The audio recordings of the face-to-face interviews were transcribed verbatim. To check for consistency and correct translation, the transcripts for all interviews were translated into English from Afrikaans and Xhosa, and then back-translated into the original languages by two independent researchers proficient in English and Afrikaans or Xhosa.

A total of 39 participants were interviewed; however, three participants were excluded due to the following reasons: in the face-to-face condition, one participant had a language barrier. In the WhatsApp condition, one participant had a prior relationship with their allocated interviewer, and one participant did not complete the interview.

After the data collection had been completed, all three interviewers participated in an in-depth focus group discussion. The focus group was facilitated by a neutral researcher that was not part of this study but had sufficient research and interviewing experience. The focus group took place face-to-face at a research office. Consent was obtained from the interviewers for the interview to be audio recorded and to maintain confidentiality. The focus group discussion was approximately 1 hour and transcribed verbatim by the researcher who facilitated the focus group.

Analysis

The authors adopt a hybrid approach of inductive and deductive coding (Fox, 2008). The interview transcripts from both interview conditions (1 = WhatsApp, 2 = face-to-face) were coded according to dimensions indicative of content, flow, focus, and depth of the interview process (Shapka et al., 2016). These dimensions include: (1) interview duration, calculated as the total number of hours, (2) the total number of words spoken by the interviewer and participant, (3) the number of interviewer probes, calculated as the total number of active invitations for comment, clarification of a statement or question, question re-phrases, follow-up questions by the interviewer specific to the interview guide questions, (4) rapport, calculated as the total number of statements from the interviewer aimed toward creating a sense of affinity, comfort, and distress reduction, (5) the number of off-topic statements made by the interviewer, (6) the number of interview questions answered by the participant, and (7) the number of themes that emerged from each interview condition relating to the reasons for leaving school (social interactions, vocational aspirations, family factors and poor academic performance). In addition, the number of emoticons used by participants and interviewers was also counted in the WhatsApp interviews. For standardization, the interview started from the beginning of the first interview question until the end of the response to the last interview question.

The first and second authors (RD and AM) constructed the variables from the interview transcripts. Using Atlas Ti version 8 (ATLAS.ti Scientific Software Development GmbH), a deductive approach was used to code the interviews according to predefined codes indicative of the flow, focus, and depth of the interview process (Burnard, 1991). The first and second authors blindly coded four transcripts and then compared and discussed the coding and any discrepancies found. The last author served as a third independent reviewer of the codes to reach a consensus on the codebook. A final codebook was developed, and the remaining transcripts were coded. The counts of each code were analyzed quantitatively using SPSS (IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) by RD and LM. A description of the sample for each interview condition was analyzed. Mann–Whitney U and Fishers’ exact chi-squared tests were conducted to identify associations and differences in the interview process that emerged from each of the interview conditions.

Guided by the interview questions from the individual participant interviews and the interviewer focus group, a qualitative categorical analysis approach using Atlas Ti (Burnard, 1991) was performed. Following several readings of the transcripts, data were initially coded and collated between RD and AM. Subsequently, connections between the codes were examined to identify emerging themes. Themes were then reviewed and refined. Extracts were selected, and the results of the analyses were organized and presented in this manuscript.

Ethics and Consent

Ethics approval was obtained from the Human Sciences Research Council (Protocol number: REC 2/23/08/17). Adolescents who are out of school, may be out of school without their caregiver’s knowledge or have unstable family support structures and therefore might have been unwilling to participate if they had to divulge the nature of the research to their parents or caregivers to obtain parental permission. Given the low risk of the study on respondents, obtaining independent consent from the adolescent participants themselves was feasible and increased participation. Therefore, in line with the South African National Department of Health Ethics guidelines (2015) section 3.2.2.4., permission for independent consent for minors was obtained (Department of Health, 2015). Once eligibility for participation was established, full written informed consent was obtained from each participant. The interviews in both conditions were not anonymous as the researchers could identify the participants using their WhatsApp account names, but the interviews were confidential and all identifying information was not included in the manuscript.

Results

Demographics and Characteristics of Participants and Interviewers

In the WhatsApp condition, on average each seed recruited 0.5 participants, and the first wave of participants on average recruited 1.5 participants. In the face-to-face condition, on average, each seed successfully recruited two participants and the first wave of participants recruited an average of 1.9 participants. As shown in Table 1, approximately half the participants were female (52.8%) and the majority described themselves as Colored (61.1%). The mean participant age was 18.34 years.

Table 1.

Demographics and Characteristics of the Participants and Interviewers.

	Total		WhatsApp		Face-to-face		p Value
	%/Mean (SD)	n	%/Mean (SD)	n	%/Mean (SD)	n	p Value
Participants
Age	18.34 (1.23)	32	18 (1.82)	8	18 (1.12)	24	.944
Gender							.451
Male	47.2	17	33.3	3	51.9	14
Female	52.8	19	66.7	6	48.1	13
Race							.014*
Coloured	61.1	22	22.2	2	74.1	20
black African	38.9	14	77.8	7	25.9	7
Interviewer experience
Low	38.9	14	77.8	7	25.9	7	.021*
Medium	41.7	15	11.1	1	51.9	14
High	19.4	7	11.1	1	22.2	6

p Value is significant at the .05 level.

Comparison of Interview Conditions on Dimensions of Interview Quality

Mann–Whitney U and Fisher’s exact chi-squared tests were conducted to identify differences and associations between the interview conditions on key characteristics indicative of the content, flow, focus, and depth of the interview process. As seen in Table 1, the participant’s race (χ² (1) = 7.636, p = .014) and interviewer’s experience (χ² = 7.094, p = .029) were significantly associated with interview groups. As seen in Table 2, on average, WhatsApp interviews took significantly more time (U = 0, p < 0.001), but face-to-face interviews were longer in terms of the number of words stated by the interviewer (U = 50.5; p = .008). Interviewers also appeared to engage in different ways across the two conditions, with face-to-face conversations having significantly more probes (U = 2.5; p < .001) and WhatsApp interviews having significantly more off-topic statements (U = 64.5; p = .0015). Only one theme (social interactions) was significantly different between the WhatsApp and face-to-face interview conditions (U = 72; p = .038).

Table 2.

Comparison of Interview Conditions on Content, Flow, Focus, and Depth of the Interview Process.

Interview process	Total		WhatsApp		Face-to-face
Interview process	Mean (SD)	n	Mean (SD)	n	Mean (SD)	n	p Value
Participant words	2,081.31 (1,331.41)	36	1,370.11 (291.2)	9	2,318.37(1,458.32)	27	.218
Interviewer words	1,494.56 (539)	36	1,107(330.73)	9	1,623.48(537.01)	27	.008*
Interviewer questions	0.76 (0.19)	36	0.75 (0.16)	9	0.77(0.21)	27	.430
Probes	45.11 (29.76)	36	16.89(7.5)	9	54.52(28.4)	27	<.001*
Rapport statements	6.64 (6.59)	36	10.89 (12.65)	9	5.67(5.12)	27	.410
Off-topic statements	1.42 (2.53)	36	1.44(0.82)	9	1.41(2.3)	27	.015*
Interview duration	3.06 (5.73)	35	13.24(9.64)	8	0.52 (0.21)	27	<.001*
Themes
Poor academic performance	0.31 (0.47)	36	0.56(0.53)	9	0.3(0.67)	27	.096
Social interactions	0.41 (0.64)	36	0	9	0.56(0.8)	27	.038*
Family factors	0.56 (0.88)	36	0.34(0.71)	9	0.63(0.93)	27	.392
Vocational aspirations	0.08 (0.28)	36	0.11(0.33)	9	0.07(0.27)	27	>.99
Emoji’s
Participant			19 (22.57)	11
Interviewer			23.44 (27.61)	14

p Value is significant at the .05 level.

Opinions and Experiences of the Participants and Interviewers

Extracts of quotations from the face-to-face and WhatsApp interviews on the opinions, experiences, and reflections of conducting the interviews of the three interviewers and the participants are presented in Table 3. For each interview condition, the quotations are categorized into the following emergent themes from the categorical content analysis: recruitment, rapport, duration, questions, conveniences, interview preferences, and recommendations.

Table 3.

Quotations Extracted From Participants and Interviewers From WhatsApp and Face-to-Face Interview Conditions.

Themes	WhatsApp quotes			Face-to-face quotes
Themes	Participants	N	Interviewers	Participants	N	Interviewers
Recruitment			. . .they kind of leave you and will tell you they will find somebody and then the weeks pass and then you are still busy with one interview (Int 1).			I would say, being like familiar with the community where we recruited, myself and [name] where we were introduced. I was familiar with the community, uhmm so obviously I know there’s a lot of school dropouts there and if you know one person, they link you to a whole lot of others and that made it easy for me (Int 2).
Rapport	I feel very confident talking to you because I don’t see you and don’t know you. I am very positive you won’t tell anyone about me, even if you do but you don’t know me (Int 3, ID 28).	4	Int 1: So, I was referred to another person by his best friend and he was not willing to really talk to me on WhatsApp, or chat with me because he didn’t really know who I was. He wanted pictures of me (laughing) you know and I have to kind of, you know. But uhm I managed to send him a voice note just like, he needed some huge connection. There is nothing, there couldn’t be anyway Int 2: And you can put a profile picture of anybody and say, this is you.	To me you did well, you gave me more time to think and answer after. You didn’t rush me. You understand me, you didn’t judge me. Instead, you explained everything well (Int3, ID 33).	4	But at the beginning of the interview, they feel shy, uhmm but in the middle of the interview, they feel comfortable and give you more information and even share a story that you didn’t ask (Int 3) So here, looking the person in the eye is really picking up the ques, which helps a little bit and for our understanding, did they not understand the question maybe or simply did not want to answer or is bored. At some point, you can pick up on that or I don’t know, maybe crack a joke, or do something else to kind of motivate them (Int 2). . . .the fact that I could speak Afrikaans as fluently as I can speak English, so a lot of them speak Afrikaans and I could, you know, come down to their level of understanding, explaining everything (Int 2).
Duration	That was a long interview and it was not the hours you told me. I was tired and I am a person who gets annoyed easily. I don’t like to type non-stop and I am a very busy person (Int3, ID28)	6	. . .when I interviewed one person, he just wouldn’t, well he kept the appointment the first days, but later on he was like, ok he is gonna call me back to answer the question in an hour. So, really I was busy with 1 person for almost 2 weeks because he will answer a question today at 9 and then tomorrow at 3 in the afternoon. So, that’s why I’m still busy with 1 interview for instance (Int 1).
Questions	That last question you asked, I didn’t like it (Int 3, ID 22)	2	They have time to read the question, if they don’t understand the question, then they will either ask something or quickly you can rephrase it because you have time, so basically they had time read and to understand it and they will give you a proper answer. Uhmm and it is also easier cause personally for me then I could work on the side and the answer you can forward it and then give two ticks. (Int 1)	When you asked me how many cigarettes I smoke per day or per week because I won’t easily tell people that I smoke. . . (Int 2 ID36)	2	On face-to-face, they say they prefer WhatsApp, so they are shy (Int 3)
Conveniences	. . . I am a shy person and I would not be able to answer all your questions (face-to-face). As I said I’m always busy. (Int 3, ID23)	4	The WhatsApp was fine. There were no transcriptions and that was perfect, the only advantage (Int 1)			I feel that they are more informative because they are in their comfort zone in their homes, so they feel more able to answer, they are familiar with their environment so they will talk more. Obviously, if they were in an unfamiliar environment they would… like to bring them to the office, then they would have held back a lot more information. So I think the fact that we actually put ourselves in their situation to go to their comfort zone, made them open up more. (Int 2)
Interview preference	Int 3: Do you think it would have been better if we had the interview in person? P: Nope Int 3: Why not? P: I think would have been more difficult for me, having to respond to your questions about my life, facially. (Int 3, ID35)	4		Int 2: Is there anything that you feel made it easy for you to do this interview in person? P: The questions yes. If I had to write it, it would have been more difficult. Int 2: So the questions and the fact that it was a verbal interview made it easy? P: Yes. (Int 2, ID 7) I3: would you prefer doing this interview over WhatsApp or face-to-face? P: Both because I don’t see any difference. The information I give you now I would give you even on WhatsApp. (Int 3, ID39)	2
Recommendations	To improve yourself don’t take long on WhatsApp (Int 2, ID30)	4	We change the questions to sound more different than what they are on paper, to not be as formal (Int 2)
	Video calls (Int 1, ID20)	1	Even if it is R5 a day or an hour or let’s say, you give them 10 questions now and tell them you have a few more questions to go and then you will get R5, and then you might be done. (Int 1)
	Maybe we give answers with a voice note (Int 3, ID6)	1	Int 3: When they are tired of typing, they send you a voice note M: So, do you think voice notes would have done better?* Int 3: some would Int 2: Maybe yes, a bit
	We can continue using WhatsApp or have the interview telephonically (Int 3, ID35)	1

Note. ID refers to the participant’s identity number. N = number of participants that stated the quote.

M refers to the Moderator in the focus group discussion with interviewers.

Recruitment

The recruitment numbers using the RDS method was lower in the WhatsApp condition (n = 9) compared to the face-to-face condition (n = 27). RDS recruitment was considered convenient in the face-to-face setting as participants found it easy to recruit in their neighbourhood and conduct the interview on the same day. Participants honoured their appointments in the face-to-face condition as it was convenient for them to attend the interview in their own communities. In contrast, the number of invitations sent through text on WhatsApp was higher compared to the Face-to-face condition but recruitment in the WhatsApp condition was lower, took longer and required several follow-ups.

Seeds recruiting were also familiar and well-known within the community that they were recruiting from which may have contributed to participant’s willingness to participate in face-to-face interviews. According to the interviewers, the monetary reimbursements offered in the face-to-face condition were more valuable than the mobile data reimbursement offered in the WhatsApp condition, which they attributed to the low recruitment numbers.

Rapport

Participants in the face-to-face condition (n = 4) felt that they could relate more easily to the interviewers from a similar sociodemographic background as well as speak in a language that was comfortable to them. Participants, particularly in the face-to-face interview, enjoyed the interview and learned something about themselves. Participants in the WhatsApp interview condition did not have any face-to-face contact with the interviewer, which they claimed initially made them feel suspicious and distrustful. Given the absence of visual cues in the WhatsApp condition, one participant requested videos of the interviewer during the interview. At the end of the interview, some participants (n = 4) in the WhatsApp condition reported that the distance and the increased anonymity between researcher and participant made power differentials less visible, eventually leading to confidence in the interviewer and the interview process itself. There was trust that the interview remained confidential and that they could not be identified due to a lack of face-to-face contact.

All interviewers in both the WhatsApp and face-to-face conditions found difficulty in building trust and rapport at the beginning of the interview. In the face-to-face condition, a participant and the interviewers commented on the importance of the interviewer’s skills in building rapport, which included effective communication skills, reading body language, the use of probes, and the ability to reserve judgment. These strategies used by all interviewers in the face-to-face condition allowed participants to feel more comfortable with the interview, have trust in the interviewer and disclose sensitive information.

Duration

When participants were asked what they disliked about the interview, most participants did not find any difficulties in the face-to-face interview but those in the WhatsApp interview condition (n = 6) found that the interview was time consuming, uninteresting, and tedious. Participants also reported that they were multitasking and sometimes struggled with slow internet bandwidth.

The three interviewers shared the same sentiment and felt that a single WhatsApp interview was time consuming as participants would chat at their convenience, which would be over several days and sometimes during odd hours of the night. The flow of questions was interrupted resulting in long breaks between interview questions. Following up on participants and motivating them to continue an interview was difficult and resulted in repeating questions, further increasing the duration of the interview.

Questions

Most participants were satisfied with the questions asked in both interview conditions; however, some considered the questions sensitive and found it difficult to disclose information (n = 4). From the interviewers’ perspective, the WhatsApp condition was advantageous in that it allowed participants time to read the questions and think of their responses, possibly resulting in more in-depth responses.

Conveniences

Participants (n = 4) stated that the WhatsApp condition was convenient as it accommodated participants who preferred not to speak face-to-face and felt more comfortable communicating online at their convenience. The main convenience of the WhatsApp condition for interviewers was the lack of transcription involved as the chat served as an interview transcript. Interviewers (n = 2) also felt that conducting the face-to-face interview in the privacy of the participant’s own home or environment may play a role in the depth of responses obtained from participants due to the comfort and familiarity of their own environment, as compared to conducting the interviews in a foreign space.

Interview preferences

When face-to-face participants were asked if they were willing to do the interview over WhatsApp, there were mixed responses. One participant in the face-to-face condition expressed a preference for face-to-face interviewing over writing or texting. One participant also felt that information shared on WhatsApp is not private, confidential, or secure. The interview experience would not be genuine and they would be uninterested. Other participants (n = 4) preferred the WhatsApp condition due to increased anonymity between researchers and participants, allowing participants to talk more freely about sensitive topics without fear of judgment. Two participants did not have a smartphone and would require permission from their parents to use their phones. Among those in the WhatsApp condition, one participant claimed that they would be comfortable with both interview conditions. Some participants (n = 3) in the WhatsApp group would have preferred to have done their interview face-to-face or telephonically to decrease the duration of the interview.

Although transcribing the face-to-face interviews was time consuming, all interviewers in this study still preferred conducting the interviews face-to-face as it was time efficient for the participant, a technique that they were familiar with, and that they could easily make use of rapport-building strategies to gain in-depth responses.

Recommendations

Recommendations by participants (n = 4) and interviewers were mainly for improvements to be made to the WhatsApp interview condition. Participants (n = 3) and interviewers recommended the use of alternative features in WhatsApp such as video or telephonic calls and voice notes to minimize the time taken to type a response. However, this would increase the time spent on transcribing by the interviewers.

Interviewers in the WhatsApp condition found that the questions were more appropriately designed for a face-to-face interview but needed to be tailored for use via WhatsApp, possibly by using language such as abbreviations and slang which are commonly used in text messaging. To increase motivation and participation in the WhatsApp interview, the interviewers suggested that the reimbursement offered could work differently to the face-to-face conditions, with participants receiving monetary reimbursements for every few questions answered in the WhatsApp interview rather than receiving their reimbursement at the end of the interview.

Discussion

The COVID-19 pandemic has brought an urgent need for remote data collection methods for research. This study examined the differences in qualitative data obtained from a group of adolescent early school leavers when using two different methods, namely, text-based WhatsApp, and traditional face-to-face interviews. Given that the interviews consisted of different communication modes (verbal and text), it was found that text-based WhatsApp and face-to-face interviews each presented with unique opportunities, as well as challenges.

Rapport was effectively developed in both interview conditions using techniques appropriate for the interview mode. The number of rapport-building statements used in the WhatsApp and face-to-face interview conditions was equivalent; however, significantly more probes were used by the interviewers in the face-to-face condition and significantly more off-topic statements initiated by the interviewer were present in the WhatsApp condition. Previous studies found that building rapport took longer over instant messaging with an increased number of rapport-building statements used by the interviewer compared to face-to-face interviews (Jowett et al., 2011; Kazmer & Xie, 2008; Shapka et al., 2016).

In this study, participants and interviewers made use of emoticons and acronyms (such as “lol”) to express non-verbal cues more explicitly. The number of off-topic statements initiated by the interviewer may also have contributed to greater rapport. In keeping with previous studies, we also found that the anonymity and distance between the interviewer and participant in the WhatsApp condition obscured power differentials, enabling rapport building (Cleary & Walter, 2011; Shapka et al., 2016). In the WhatsApp condition, rapport still benefitted from the interviewer introducing themselves over a voice recording as well as the interviewer’s skill in expressing emotion over text through the use of internet vernaculars such as punctuation, emoticons, and internet abbreviations and keeping participants engaged over text using off-topic conversations. The use of voice, video, and text offer participants and researchers many modes of communication that they feel most comfortable with. In the face-to-face condition, rapport still benefitted from conventional interviewer skills including attentiveness, flexibility, social skills, and effective communication.

The qualitative findings show that participants in the face-to-face condition felt that they could relate more easily to the interviewers from a similar sociodemographic background. Given that each interviewer had varying experience levels in conducting qualitative research and that participant race was not equally distributed between the interviewers, it is important to acknowledge possible interviewer and participant effects. A qualitative interview is more than just a measure of the frequency of a conversational interaction between two people (Gibson, 2022). Rather, it is the social interaction and relationship between the interviewer and interviewee that affects the process and outcome of the interview. It is pertinent to therefore acknowledge that the quality of an interview is shaped by variables such verbal and non-verbal messages, skills and knowledge of the interviewee, culture and demographics such as race, which also transpire during the interview process and affects the data produced (Frances et al., 2009; Potter & Hepburn, 2005). Future studies should consider the interviewer and participant characteristics and experiences of conducting the WhatsApp and face-to-face interviews, to understand how the different methods of interviewing shape the research encounter and the data produced.

The advantages of the text-based WhatsApp interview included time saved from the automatic transcript generated from the chat, participants could respond at their own convenience and reflect on their responses, and it accommodated participants who preferred not to talk about sensitive information in person. With the structure of the interview guide identical in both interview conditions, the number and types of themes were almost equivalent. This finding suggests that despite the differences in the number of words produced and the duration of the interview, the information obtained was similar across the interview conditions. This finding is consistent with previous studies that compared instant messaging to face-to-face interviews (Hinchcliffe & Gavin, 2009; Shapka et al., 2016; Singer et al., 2020).

Given the novelty of WhatsApp interviewing, this method presented challenges. While significant progress may have been made to extend internet access to under-served areas prior to the COVID-19 pandemic, socio-economic barriers to accessing the internet include the high cost of mobile data, the price of devices, and poor signal (Desai & Burton, 2022). Researchers should keep these digital inequalities in mind and select an approach that is suitable for the sociodemographic context in which the research is taking place. Having friends or family recruit into the WhatsApp condition yielded low recruitment. This could be attributed to the nature of the reimbursement being mobile data compared to the monetary incentive offered in the face-to-face interview for recruiting participants. Those recruited may have also not had access to a smartphone and this may have led to them not participating. A previous study that used Zoom to conduct qualitative interviews used phone calls and social media platforms to successfully recruit participants (Reñosa et al., 2021). Once participants were recruited in this study, they were still sceptical of the WhatsApp interview upon first contact with the interviewer. This challenge was somewhat mitigated by interviewers introducing themselves over a voice note, explaining the nature of the study and ensuring that the interview was secure and confidential. Furthermore, both participants and interviewers found the online interview long, and tiring and participants said that they began to lose interest. Participants and researchers recognized that a way to mitigate the challenges encountered in the WhatsApp condition would be to use alternative multimedia features available on WhatsApp such as phone, voice, and video. These features have been validated in other studies (Kaufmann & Peil, 2020; Reñosa et al., 2021; Singer et al., 2020) with relatively modest differences in data quality and participant satisfaction between face-to-face and online modes. However, incorporating these features would require further transcription time from the researcher.

This study is not without its limitations. The authors acknowledge that face-to-face nonverbal communication contains many complexities such as body language and tone that are unique to this form of communication. WhatsApp also contains unique non-verbal cues such as the use of emoticons, non-standard punctuation, text formatting, and letter repetition which mimics the tone of voice (Hancock et al., 2007). The authors unfortunately could not analyze the full range of non-verbal cues present in each of these interview conditions. An in-depth semantic analysis of non-verbal communication in the different interview conditions can be explored in future analysis to account for the complexities of nonverbal communication during interviews. The sample size of the study is within the recommended range for a qualitative study but does not hold adequate power to conduct complex quantitative analysis. However, the sample size was sufficient to identify numerous significant descriptive differences between the interview conditions which can further be explored in future studies with larger sample sizes. The relatively small sample size in the WhatsApp condition compared to the face-to-face condition should also warrant future studies to test this methodology on a larger scale. This study focused on adolescent early school leavers and their reason for leaving school. Education level may have played a role in the depth of participant’s responses and preference to conduct interviews verbally or over written communication. Although previous studies have found that adolescents are comfortable with communicating in online environments, future work is needed to test the compatibility of WhatsApp interviewing among other adolescent and adult populations. We acknowledge that sections of data may be included in multiple themes with some overlap between themes. However, the researchers considered how each theme fitted into the overall story within the entire data set. Furthermore, due to adolescent early school leavers in this study being a hard-to-reach population and the limited resources such as funding and time constraints, transcripts were not returned to the participants for comment and/or correction due to difficulty in contacting them again.

Conclusions

This study adds to the literature that is empirically comparing online text-based and face-to-face interviewing methods. This study contributes to a growing body of knowledge in the qualitative community on using remote data collection tools amid an ongoing pandemic. Remote data collection is not a replacement for face-to-face interviews but can be a highly beneficial complement to this approach. Using WhatsApp to collect qualitative data among adolescents bodes well for researchers because the number of themes generated and the rapport-building process is almost equivalent to that obtained face-to-face. Although the length of time taken in the WhatsApp condition was a concern, it is recommended that the WhatsApp interview protocol be adapted to include, for example, an open-ended question sent to participants once a week over approximately a month, which will shorten the interview time, increase interest, and allow an analysis of the responses to that question in real-time. Internet vernaculars and other multimedia features such as voice or video could also be incorporated into the WhatsApp interview. Participants should also be given a choice in how they wish to be interviewed, which could possibly have a positive impact on response rates and interest in the study.

Footnotes

Acknowledgements

We want to acknowledge all the research assistants and adolescents without whom this study would not have been possible.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

Ethics approval was obtained from the Human Sciences Research Council (Protocol number: REC 2/23/08/17).

ORCID iDs

Rachana Desai

Robert Ruiter

Author Biographies

Rachana Desai is a Post-doctoral Fellow in the Centre of Excellence in Human Development at the University of the Witwatersrand.

Ansuyah Magan is an Honorary Research Fellow at the Developmental Pathways for Health Research Unit, University of the Witwatersrand.

Innocent Maposa is a Senior Lecturer at the School of Public Health at the University of the Witwatersrand.

Robert Ruiter is a Professor of Health and Social psychology in the Department of Work and Social Psychology at Maastricht University.

Tamsen Rochat is a Wellcome Trust Intermediate Fellow in Public Health and Tropical Medicine in the Centre of Excellence in Human Development, and she is an Associate Professor in the School of Medicine, Faculty of Health Sciences at the University of the Witwatersrand.

Liesbeth Mercken is an Associate Professor in the Department of Health Psychology at Open University in Heerlen.

References

Archibald

M. M.

Ambagtsheer

R. C.

Casey

M. G.

Lawless

(2019). Using zoom videoconferencing for qualitative data collection: Perceptions and experiences of researchers and participants. International Journal of Qualitative Methods, 18, 1–8. https://doi.org/10.1177/1609406919874596

Burnard

(1991). A method of analysing interview transcripts in qualitative research. Nurse Education Today, 11(6), 461–466.

Chen

Tossyeh

Arnous

Saleh

El Hassan

Saade

Miller

K. E.

(2020). Phone-based data collection in a refugee community under COVID-19 lockdown. The Lancet Psychiatry, 7(6), e31. https://doi.org/10.1016/S2215-0366(20)30189-9

Cheung

Y. T. D.

Chan

C. H. H.

K. S.

Fok

W. Y. P.

Conway

Wong

C. K. H.

W. H. C.

Wang

M. P.

Lam

T. H.

(2020). Effectiveness of WhatsApp online group discussion for smoking relapse prevention: Protocol for a pragmatic randomized controlled trial. Addiction, 115(9), 1777–1785.

Church

De Oliveira

(2013, August 27–30). What’s up with WhatsApp? Comparing mobile instant messaging behaviors with traditional SMS [Paper presentation]. Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services, Munich, Germany.

Cleary

Walter

(2011). Is E-mail communication a feasible method to interview young people with mental health problems? Journal of Child and Adolescent Psychiatric Nursing, 24(3), 150–152.

Coleman

O’Connor

(2019). The role of WhatsApp® in medical education; A scoping review and instructional design model. BMC Medical Education, 19(1), 1–13.

de Gruchy

Vearey

Opiti

Mlotshwa

Manji

Hanefeld

. (2021). Research on the move: Exploring WhatsApp as a tool for understanding the intersections between migration, mobility, health and gender in South Africa. Global Health, 17(1), 71.

DeJonckheere

Nichols

L. P.

Moniz

M. H.

Sonneville

K. R.

Vydiswaran

V. V.

Zhao

Guetterman

T. C.

Chang

(2017). MyVoice national text message survey of youth aged 14 to 24 years: Study protocol. JMIR Research Protocols, 6(12), e8502.

10.

Department of Health. (2015). Ethics in health research: Principles, processes and structures. https://www.sun.ac.za/english/faculty/healthsciences/rdsd/Documents/Ethics/DoH%202015%20Ethics%20in%20Health%20Research%20-%20Principles,%20Processes%20and%20Structures%202nd%20Ed.pdf

11.

Desai

Burton

(2022). Child and adolescent mental health and the digital world: A double-edged sword. In K. S. L. L. e. Tomlinson M (Ed.), South African child gauge 2021/2022 (pp. 113–121). Children’s Institute, University of Cape Town.

12.

Desai

Magan

Ruiter

Reddy

Mercken

(2020). Social network determinants of alcohol and tobacco use: A qualitative study among out of school youth in South Africa. PLoS One, 15(10), e0240690. https://doi.org/10.1371/journal.pone.0240690

13.

Desai

Ruiter

Schepers

Reddy

Mercken

(2019). Reasons for leaving school and alcohol use among out of school youth in South Africa. Health Psychology Bulletin, 3(1), 48–57.

14.

Dodds

Hess

A. C.

(2020). Adapting research methodology during COVID-19: Lessons for transformative service research. Journal of Service Management, 32, 203–217.

15.

Durmaz

Ergin

Durusoy

Hassoy

Caliskan

Okyay

(2019). WhatsApp embedded in routine service delivery for smoking cessation: Effects on abstinence rates in a randomized controlled study. BMC Public Health, 19(1), 1–12.

16.

Ehrenreich

S. E.

Beron

K. J.

Burnell

Meter

D. J.

Underwood

M. K.

(2020). How adolescents use text messaging through their high school years. Journal of Research on Adolescence, 30(2), 521–540.

17.

Elgar

F. J.

Pförtner

T.-K.

Moor

De Clercq

Stevens

G. W. J. M.

Currie

(2015). Socioeconomic inequalities in adolescent health 2002-2010: A time-series analysis of 34 countries participating in the Health Behaviour in School-aged Children study. The Lancet, 385(9982), 2088–2095. https://doi.org/10.1016/S0140-6736(14)61460-4

18.

Flick

(2013). The SAGE handbook of qualitative data analysis. SAGE.

19.

Fox

N. J.

(2008). Post-positivism. In Given

L. M.

(Ed.), The SAGE encyclopedia of qualitative research methods (Vol. 2, pp. 659–664). SAGE.

20.

Frances

Coughlan

Cronin

(2009). Interviewing in qualitative research. International Journal of Therapy and Rehabilitation, 16, 309–314. https://doi.org/10.12968/ijtr.2009.16.6.42433

21.

Gajadhar

Green

(2003). An analysis of nonverbal communication in an online chat group (Working Paper). The Open Polytechnic of New Zealand.

22.

Gibbs

Kornbluh

Marinkovic

Bell

Ozer

E. J.

(2020). Using technology to scale up youth-led participatory action research: A systematic review. Journal of Adolescent Health, 67(2), S14–S23. https://doi.org/10.1016/j.jadohealth.2019.10.019

23.

Gibson

(2022). Bridging the digital divide: Reflections on using WhatsApp instant messenger interviews in youth research. Qualitative Research in Psychology, 19(3), 611–631.

24.

Hancock

J. T.

Landrigan

Silver

(2007, April 28–May 3). Expressing emotion in text-based communication [Paper Presentation]. Proceedings of the SIGCHI conference on human factors in computing systems, San Jose, CA, United States.

25.

Heckathorn

D. D.

(2002). Respondent-driven sampling II: Deriving valid population estimates from chain-referral samples of hidden populations. Social Problems, 49(1), 11–34.

26.

Hinchcliffe

Gavin

(2009). Social and virtual networks: Evaluating synchronous online interviewing using instant messenger. The Qualitative Report, 14(2), 318–340.

27.

Iqbal

(2021). WhatsApp revenue and usage statistics (2020). https://www.businessofapps.com/data/whatsapp-statistics/

28.

Irvine

(2011). Duration, dominance and depth in telephone and face-to-face interviews: A comparative exploration. International Journal of Qualitative Methods, 10(3), 202–220.

29.

Jailobaev

Jailobaeva

Baialieva

Asilbekova

(2021). WhatsApp groups in social research: New opportunities for fieldwork communication and management. Bulletin of Sociological Methodology, 149(1), 60–82.

30.

Janghorban

Roudsari

R. L.

Taghipour

(2014). Skype interviewing: The new generation of online synchronous interview in qualitative research. International Journal of Qualitative Studies on Health and Well-Being, 9(1), 24152.

31.

Jowett

Peel

Shaw

(2011). Online interviewing in psychology: Reflections on the process. Qualitative Research in Psychology, 8(4), 354–369.

32.

Kamel Boulos

M. N.

Giustini

D. M.

Wheeler

. (2016). Instagram and WhatsApp in health and healthcare: An overview. Future Internet, 8(3), 37.

33.

Kaufmann

Peil

(2020). The mobile instant messaging interview (MIMI): Using WhatsApp to enhance self-reporting and explore media usage in situ. Mobile Media & Communication, 8(2), 229–246.

34.

Kaushik

Walsh

C. A.

(2019). Pragmatism as a research paradigm and its implications for social work research. Social Sciences, 8(9), 255.

35.

Kazmer

M. M.

Xie

(2008). Qualitative interviewing in Internet studies: Playing with the media, playing with the method. Information, Community and Society, 11(2), 257–278.

36.

Krouwel

Jolly

Greenfield

(2019). Comparing Skype (video calling) and in-person qualitative interview modes in a study of people with irritable bowel syndrome: An exploratory comparative analysis. BMC Medical Research Methodology, 19, 1–9. https://doi.org/10.1186/s12874-019-0867-9

37.

Kumari

Ganagwar

(2018). A critical study of digital nonverbal communication in interpersonal and group communication: In context of social media. International Journal of Communication and Media Studies, 8(4), 1–12.

38.

Lee

C. K.-M.

(2007). Affordances and text-making practices in online instant messaging. Written Communication, 24(3), 223–249.

39.

Mars

Scott

(2016). WhatsApp in clinical practice: A literature review. Studies in Health Technology and Informatics, 231, 82–90.

40.

Opdenakker

(2006). Advantages and disadvantages of four interview techniques in qualitative research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 7(4), https://doi.org/10.17169/fqs-7.4.175

41.

Patel

S. J.

Subbiah

Jones

Muigai

Rothschild

C. W.

Omwodo

Ogolla

Kimenju

Pearson

Meadows

Nour

N. M.

(2018). Providing support to pregnant women and new mothers through moderated WhatsApp groups: A feasibility study. Mhealth, 4, 14. https://doi.org/10.21037/mhealth.2018.04.05

42.

Pereira

A. A. C.

Destro

J. R.

Picinin Bernuci

Garcia

L. F.

Rodrigues Lucena

T. F.

(2020). Effects of a WhatsApp-delivered education intervention to enhance breast cancer knowledge in women: Mixed-methods study. JMIR mHealth and uHealth, 8(7), e17430. https://doi.org/10.2196/17430

43.

Potter

Hepburn

(2005). Qualitative interviews in psychology: Problems and possibilities. Qualitative Research in Psychology, 2(4), 281–307.

44.

Reñosa

M. D. C.

Mwamba

Meghani

West

N. S.

Hariyani

Ddaaki

Sharma

Beres

L. K.

McMahon

(2021). Selfie consents, remote rapport, and Zoom debriefings: Collecting qualitative data amid a pandemic in four resource-constrained settings. BMJ Global Health, 6(1), e004193.

45.

Shapka

J. D.

Domene

J. F.

Khan

Yang

L. M.

(2016). Online versus in-person interviews with adolescents: An exploration of data equivalence. Computers in Human Behavior, 58, 361–367.

46.

Singer

Walsh

C. M.

Gondwe

Reynolds

Lawrence

Kasiya

(2020). WhatsApp as a medium to collect qualitative data among adolescents: Lessons learned and considerations for future use. Gates Open Research, 4(130), 130.

47.

Steup

(2019). WhatsApp in Africa: Statistics & business potential. https://www.messengerpeople.com/whatsapp-in-africa/

48.

Stieger

Göritz

A. S.

(2006). Using instant messaging for internet-based interviews. CyberPsychology & Behavior, 9(5), 552–559. https://doi.org/10.1089/cpb.2006.9.552

49.

Stoilova

Livingstone

Khazbak

(2021). Investigating risks and opportunities for children in a digital world: A rapid review of the evidence on children’s internet use and outcomes (Innocenti Discussion Papers Papers(indipa1183)). UNICEF.

50.

Voida

Mynatt

E. D.

Erickson

Kellogg

W. A.

(2004, April 24–29). Interviewing over instant messaging [Paper presentation]. CHI’04 extended abstracts on Human factors in computing systems, Vienna, Austria.