Sage Journals: Discover world-class research

Abstract

During the COVID 19 pandemic, Online Asynchronous Focus Groups (OAFG) through WhatsApp were conducted to explore women’s experiences in the context of Congenital Syphilis prevention in Colombia. This paper discusses issues raised by the OAFGs (not least in relation to face-to-face focus groups). After a review of the literature on online and offline focus groups, there is a consideration of some key features of our OAFGs. In particular, we note how silence, presence, attention, continuity and multiplicity manifested in our OAFGs. We suggest that rather than regarding OAFGs as inferior to the ‘gold standard’ of face-to-face focus groups, our OAFGs raise important questions about our assumptions about focus group methodology. For instance, what counts as participant engagement, what comprises ‘useful’ social data, and what constitute the boundaries of a focus group all emerge as critical issues. We go on to reflect on some of the implications of these issues for the fruitfulness of OAFG methods.

Keywords

Online asynchronous focus groups silence continuity data participant engagement congenital syphilis Colombia WhatsApp

Introduction

This paper concerns the use of online asynchronous focus groups (OAFGs). OAFGs were adopted as the preferred method in response to the limitations on mobility imposed by the Colombian government in response to the COVID-19 pandemic. But it was also a method that seemed best suited to the study of women’s experience of Congenital Syphilis prevention in Colombia not least because, prima facie, this method facilitates access to hard-to-reach groups, it can accommodate participants’ everyday routines and availability, and it affords an element of confidentiality for the discussion of potentially very sensitive medical, cultural and personal concerns.

However, within the specificities of the actual case study of women’s experience of Congenital Syphilis prevention in particular regions of Colombia, other issues become apparent. For example, silence – or the lack of contribution within the OAFG – is generative of several meanings that can ambiguate the research in terms of interpretations of engagement, taboos, and matters of care. Moreover, the affordances of the chosen platform (WhatsApp) for our OAFGs means that the messaging produced by the women can be highly complex and variegated in ways that nuance and indeed partially transform the very character of the OAFG. The case study we present allows us to explore how the platform, along with the specific circumstances of Colombia and Congenital Syphilis prevention, can shape the ways in which such features as presence and attention, silence, the introduction of ‘extraneous’ messages, and the diversity of communications affect the character and even integrity of our OAFGs. Indeed, we suggest that our case study has broader implications for OAFG methodology in general.

In what follows, we present a brief outline of the typical characteristics of online asynchronous focus groups as depicted in the methodological literature, and especially as contrasted against face-to-face focus group practice. We then consider how this method was adapted to take into account the specificities of our research sites and participants. Firstly, we note how OAFGs were particularly applicable to our case study for a number of reasons that included the pandemic, participant accessibility and network coverage. Secondly, we detail the design, implementation and modifications of our OAFGs and lay out the various rationales for these. We then report on the range of responses we received from our participants, focussing especially on how these diverged from response patterns we would expect in face-to-face focus groups. Concretely, we look, for instance, at how presence and attention were conveyed, how participants managed to be simultaneously inside and outside of the focus group, how enacting the private and the public was negotiated and how (dis)continuity of exchanges was derived. We end with reflections on a number of questions raised by our use of OAFGs in our specific case study, specifically about what counts as participant engagement, what comprises ‘useful’ social data and what constitute the boundaries of a focus group.

On online asynchronous focus groups

Traditionally, the main assets of focus groups are that they enable groups of people to respond to the researchers’ questions while also responding to each others’ contributions. Participants interact, answer and comment on, the moderator´s questions as well as one another’s responses. As such, focus groups allow researchers to see how ideas appear and are challenged through the interaction amongst participants (Lobe and Morgan, 2021; Morgan and Hoffman, 2018).

Potential advantages of face-to-face focus groups include: they facilitate voice from under-represented groups; they are a forum for talk about sensitive topics that might be difficult during an interview; and they enable support amongst participants who can thus express views and opinions that resist social pressures (Gordon et al., 2021; Williams et al., 2012; Liamputtong, 2015; Kitzinger, 1994; Reisner et al., 2018). Focus group methodology has become widely used in health care research (Flynn et al., 2018; Williams et al., 2012; Reisner et al., 2018; Ranieri et al., 2019) because it allows for the understanding of practices and interactions and thereby grounds theory in the actual language and experience of the participants – whether patients or medical professionals (Smithson, 2008). Of course, the focus group itself does not yield ‘naturally derived’ data and so analysis of the interaction of such artificially created groups and situations is at least as important as the content of what is created (e.g. Silverman, 2007).

Online focus groups have become more ubiquitous over the last few decades and particularly during the current pandemic. Whether mediated by telephones, online, platforms, emails or Apps, online focus groups have been adapted so that they broadly accord with the usual parameters of face-to-face focus groups (in their organization, sequencing and structure, number of participants, the moderator´s role, etc.). It is not surprising that some authors define online focus groups as similar to face-to-face focus groups. As Lobe and Morgan (2021) put it, an online focus group is a ‘communication event’ mediated through a computer that mimics what happens in an offline focus group. Others (Woodyatt et al., 2016; MacNamara et al., 2021) state that online focus groups are adaptations of in-person focus groups and can overcome some of the limitations that face-to-face focus groups represent in terms of recruitment, costs, confidentiality and anonymity.

Accordingly, most of the discussions surrounding online focus groups have revolved around how to adapt face-to-face focus groups to an online setting and how to obtain high quality data (Abrams et al., 2015). Results have been mixed however (Zwaanswijk and Van Dulmen, 2014; Woodyatt et al., 2016; Brüggen and Willems, 2009; Nicholas et al., 2010). On the one hand, some studies indicate disadvantages in terms of fewer words and interactions, technical difficulties with the technology used and lack of non-verbal communication in online focus groups. On the other hand, other studies highlight the facilitation of discussion of such sensitive topics as abortion (MacNamara, 2020), body image amongst transgender and gender diverse adults (Gordon et al., 2021) and cancer in children (Tates et al., 2009). At base, these comparisons reflect the perception that face-to-face focus groups are the ‘gold standard’, reinforce the dichotomy of offline/online, and neglect considerable variety in form and content amongst online focus groups (which can be text-based, video-based, platform-based, email-based, synchronous or asynchronous).

Now, much recent work on online focus groups has been primarily oriented toward text-based communication, not least as conducted on platforms dedicated to the research (Hallam, 2021; Gordon et al., 2021; MacNamara et al., 2021; Ranieri et al., 2019; Biedermann, 2018). Typically, it is noted that not all potential participants can access the platform, that there is a need for literacy skills to use the platform, that group dynamics, particularly if synchronous, can be challenging for participants and moderators to follow (Stewart and Williams, 2005; Forrestal et al., 2015; Fox et al., 2007). One proposed solution is to include several moderators for online synchronous focus groups to facilitate turn-taking. Once again, we can note that this ‘problem’ and its ‘solution’ reflect the priority accorded face-to-face focus groups.

Other researchers have incorporated video into the online focus groups to obtain data on non-verbal communication and to ‘mimic’ face-to-face focus groups (Lobe and Morgan, 2021; Flynn et al., 2018; Abrams et al., 2015). However, this detracts from some of the advantages that online focus groups afford, particularly if the focus is on sensitive topics. Recently, some scholars have conducted online focus groups through WhatsApp (Colom, 2021; Chen and Neo, 2019; Veloso, 2020). Potential benefits include: access to diverse and geographically dispersed populations; the ease of use for participants for whom WhatsApp already features as part of their daily activities; and the use of emojis and other signifiers to convey non-verbal or affective information. However, this research methodology is still relatively new, and there remains a need for further development (Chen and Neo, 2019; Veloso, 2020), particularly with regard to taking into account how participant communications are shaped by their (non-on-line) specific contexts and circumstances (Colom, 2021; Chen and Neo, 2019).

Most online focus groups conducted through WhatsApp are asynchronous (Colom, 2021; Veloso, 2020; Chen and Neo, 2019). As participants do not contribute simultaneously, that is, in quick ‘conversational’ succession, asynchronicity poses several challenges. Difficulties relating to such a reduced ‘spontaneity’ (Brüggen and Willems, 2009; Nicholas et al., 2010) include: shorter answers with fewer word counts (Chen and Neo, 2019); uneven flow during the interactions due to their lag (Veloso, 2020); and more unfocussed exchanges that do not always address the relevant research question (Brüggen and Willems, 2009). Once more, these shortcomings are set against the ‘gold standard’ of face-to-face focus groups.

It will come as no surprise that in the context of discussions about the performativity of method (e.g. Law, 2004; Lury and Wakeford, 2012), to compare and contrast face-to-face focus groups with online focus groups misses the point somewhat. Every social scientific method affects the type of data that is possible not least by othering some sources, and Othering certain pre-suppositions about the nature of social processes (Law, 2004). Face-to-face focus groups generate one sort of data, on-line focus groups another sort. And we should not be tempted by the idea that in our contemporary social world, face-to-face is somehow more authentic, or a better index of social reality. Echoing Marres (2015), we might say that digital data, insofar as they are the main way by which some people communicate given the impositions of distance and the pandemic, illuminate contemporary social relations. However, assuming such digital data also leave ‘their traces in the very form, content and character of’ (Marres, 2015: 671) face-to-face interaction, they are worth examining in their own right for the insights they furnish.

With all this in mind, the following two sections will describe the online asynchronous focus groups conducted in the specific circumstances of Congenital Syphilis prevention in particular areas of Colombia.

Adapting focus groups to the field sites: OAFGs

The research referenced here examines Congenital Syphilis prevention in three multicultural sites of study in western Colombia (Quibdó, Manizales and Riosucio). The purpose of the research was to explore variations in women’s experience of Congenital Syphilis prevention – variations that might be attributable to such factors as differences in regional health care systems, inequalities based on ethnic background and geographical isolation, and the specificities of local cultural practices. The suite of methods includes online methodologies such as document analysis, interviews, asynchronous focus groups and diaries. This research aims to contribute to improving care in these prenatal settings while acknowledging more-than-‘expert’ ontologies (Mol, 2002).

The initial stages of research coincided with the beginning of the COVID-19 pandemic, and it became clear that it would not be possible to conduct an ethnography in a primary care setting, as originally planned. Online methodologies were thus a necessity. This was reinforced because pregnant women were considered to be at higher risk from the outset of the pandemic (Ministerio De Salud y Protección Social, 2020), thereby limiting face-to-face prenatal care consultations as well as the possibility of relatives and friends joining the women in the hospitals or prenatal care settings.

In adapting to these circumstances, the research turned to on-line methods, particularly WhatsApp in order to reach out to target ‘hard-to-reach’ groups who might otherwise not have participated in the research. As scholars have suggested, the notion of ‘hard to reach’ is contested (Sokol and Fisher, 2016; Charlot et al., 2021; Islam et al., 2021), nevertheless as this app is in common usage, it facilitates access to groups who might typically be hampered by limited access to other less familiar platforms and technologies. In the present context, WhatsApp was chosen because it would – ideally – better include women from comparatively disadvantaged backgrounds, with limited experience with mobile technologies, who were living in relatively geographically isolated communities (also see below). More generally, WhatsApp was used to conduct interviews, asynchronous focus groups and diaries with different stakeholders (health care workers, health care administrators, pregnant women, partners and relatives).

In light of the characteristics of online focus groups mentioned above, and the access afforded by WhatsApp, WhatsApp online focus groups were conducted with pregnant women or women who had babies in the preceding 4 months in order to document their various experiences surrounding pregnancy, prenatal care, birth and motherhood. More specifically, online asynchronous focus groups (OAFGs) were chosen because they had the additional advantage of allowing participants to share their experiences at their own pace at home or even while at work.

To clarify further, amongst the advantages of using WhatsApp is the growing affordability of smartphones and internet access, which allows this messaging app to be incorporated into daily activities. Besides being ‘free’ of charge, it is quick and easy to use, allows for the sharing of not only text but also images, links, videos, voice recordings, emojis, stickers, etc. (Jailobaev et al., 2021). Further, there is no need to create an account, there is more immediacy in the exchange of messages (in contrast to emails), and it is possible to see if someone is online and if the messages have been received and read (Jailobaev et al., 2021). It has also been highlighted that WhatsApp can facilitate more informal conversations. Even for work-related matters, it creates a friendly environment, and allows people who share similar interests to gather digitally (Jailobaev et al., 2021). In addition, WhatsApp’s functionality means that multimodal data (e.g. Kress, 2010; Dicks, 2014) can be generated that, in addition to text, can include emojis, photos, voice and song. This enriches the data and evokes a wider range of meanings and affects.

However, the use of WhatsApp for research also implicates pitfalls. For instance, around sample selection and recruitment, not all potential participants have access to a smartphone or use WhatsApp daily (Sugie, 2018). Indeed, it is not a minor issue to conduct online research in Colombia which mainly uses WhatsApp. On this score, according to the National Department of Statistics in Colombia (DANE), only 52.7% of households have Internet access, and 72% have mobile phones. These percentages are uneven in most regions and particularly low in rural areas (DANE, 2019). By contrast, during the present pandemic, the Colombian government made efforts to improve access to digital services as more people had to rely on the use of the Internet and mobile phones. The number of users on social media, messaging apps and YouTube has grown considerably over the course of the pandemic. The number of mobile phone subscribers has increased from 66.28 million in 2019 to 69.4 million during the first few months of 2021 (Holst, 2021). Further, the diversity of multimodal data provided through WhatsApp also complicates the analysis, especially where some forms of communication do not seem to relate to the matter at hand, and where communication is fragmented. However, as we discuss below this also allows for a critical reflection on researcherly expectations about what constitutes data, and, indeed, a focus group.

The present OAFG study: Details

Participants had access to a smartphone and could use them through free Wifi, mobile top-ups or monthly plans. During the pandemic, pregnant women had to secure access to mobile phones because many health services or health services information was delivered by mobile calls, emails or WhatsApp. In spite of improvements, it was not possible to reach pregnant women in more isolated rural areas through this methodology since they do not have access to mobile phones, and in any case the signal coverage is poor. Further, due to COVID restrictions, these women could not travel to villages to access internet or data services.

Participants were recruited through snowballing. Referrals were offered by health care workers, including those attached to the association of traditional midwives in Chocó (ASOREDIPARCHOCÓ), Secretaries of Health (some had connections with a project for pregnant women in Lloró), and Corporación Sagrada Familia (NGO in Manizales). In addition, women who had already been interviewed were asked for contacts.

All potential participants received an infographic about the project and the OAFG. This laid out the nature of the research into experiences of pregnancy in relation to the prevention of Congenital Syphilis. It was presumed that some participants would relate more readily to the infographic, and this would allay any concerns (e.g. about any stigma surrounding congenital syphilis, which might have further contributed to their ‘hard-to-reach’ status). Those interested were contacted by Ana Lucia Estrada Jaramillo (henceforth ALEJ) who provided more details, the information sheet and the informed consent.

Prior to arranging OAFGs, any queries were answered and participants were provided with informed consent forms which were either filled in, signed and sent through WhatsApp, or participants replied to each consent-related question in WhatsApp through a text or audio message.

Three OAFGs were run between April and July 2021. Various suggestions have been made about the ideal length of an OAFG, including 5 days (Chen and Neo, 2019), 4 days (Gordon et al., 2021), 14 days (Hallam, 2021) and 9 weeks average (Williams 2009 in Williams et al., 2012). In the present case, a period of 3 days for the online focus group was set up. The aim was to ensure that the groups did not lose momentum, that withdrawals were limited, and that the likelihood of becoming a ‘silent’ group due to ‘tiredness’ or exhaustion was kept to a minimum (MacNamara et al., 2021; Gordon et al., 2021; Hallam, 2021). A more compressed period such as Colom’s (2021) 24 h was discounted due to constraints related to availability, work, parenting and family responsibilities and data signal: 3 days were opted for to allow more time for interaction.

Several of the OAFG participants were living in Manizales and Riosucio (Caldas), while others were based in Quibdó or Lloró (Chocó). Caldas and Chocó are ‘departamentos’ (similar to states or counties) in Western Colombia. Seven participants who live in smaller cities or villages such as Riosucio or Lloró had to travel to Manizales or Quibdó to receive prenatal care or special health care services. Quibdó and Lloró are mainly composed of Afro descendants, Riosucio has a large indigenous population, and Manizales has mainly a ‘mixed’ background. In Riosucio, there are at least two health care service providers that incorporate indigenous traditional medicine with ‘western’ medicine, while in Quibdó and Lloró, traditional midwives (afro descendants and indigenous) are consulted alongside ‘western’ medicine. In Manizales, ‘western’ medicine is the prevalent healing system. All pregnant women consulted several healing systems consecutively or simultaneously (Torri and Laplante, 2013), including religious healing system (Catholic and Christian – Evangelic) and ‘Curanderismo’¹.

Focus group 1 included women ages from 24 to 38 years living in Manizales and Riosucio. One self-identified as indigenous and five as ‘mestizas’ (‘mix’ background), one of whom was an internally displaced person because of armed conflict. Regarding healing systems, all of them attended to prenatal care (‘western’ medicine), three also used ‘Curanderismo’, two used religious system and five also accessed online resources (Apps, Youtube - Tik Tok videos, web pages) to look for information and advice regarding their pregnancy, birth, and babies’ first months. Only one was working while pregnant, and two were working after having their babies². For four of the participants, it was their first baby. For three, their pregnancy was categorized as ‘high risk’³ due to hypothyroidism, preeclampsia, gestational diabetes and anaemia.

Focus group 2 consists of women from 20 to 28 years living in Manizales, Lloró and Quibdó. Two self-identified as Afro descendants, and five ‘mestizas’, two of whom were undocumented migrants from Venezuela. All of them attended prenatal care (‘western’ medicine), two also consulted traditional midwives, three used ‘Curanderismo’, three used the religious system and four used online resources. Only one was working while pregnant, and two were working after having their babies. For five, their pregnancy was categorized as ‘high risk’ due to preeclampsia and anaemia.

Focus group 3 includes women aged 19 to 30 years living in Quibdó and Lloró, all (6) self-identified as Afrodescendants. At the time, one was pregnant, and it was her first baby. All of them attended prenatal care (“western” medicine), five also used ‘Curanderismo’, three consulted with traditional midwives, four used the religious system and two online resources. None of them referred to their pregnancy as ‘high risk’.

Each group was set up on WhatsApp as a chat group with up to seven participants. Participants were added into their chat group 1 day before the starting date and were informed that that it would be run for 3 days starting the next day at 8:30 am and finishing 3 days later at 18:00. Ground rules included the requirement that all participants were respectful of other women´s experiences, although it was acceptable that people could make comments on others’ statements and even disagree. Sharing personal pictures, taking screenshots from the WhatsApp group and/or sharing information with other people outside of the group was prohibited. Participants were encouraged to contact ALEJ directly if they had questions or wanted to address any difficulties, stress or discomfort. To secure anonymity, all participants were asked not to have profile pictures of themselves or their families on WhatsApp: they were provided with an alternative name such as ‘Participant #’. They were also reminded that they were free to exit the group if they did not want to participate without further explanation. There were 19 participants in total.

In terms of organizing OAFG contact, 2 days before starting the first focus group, for participants who wanted it, they received a new Sim Card and a data top-up paid for by the research project. Mobile service providers delivered the new Sim Cards to each participant’s home while the top-ups were paid online. In the main, participants preferred to use their own Sim Cards as they did not have dual sim cards devices, and it was vital for them to be reached on their mobile phone numbers. In these cases, new Sim Cards were not provided, although data top-ups were given to all participants. Further, some participants in the second and third groups lived in more isolated rural areas, which might have resulted in logistical difficulties for Sim Cards provision.

The first day began with a reminder of the purpose of the group and the rules of participation. In the case of the first group, the plan was to have twelve topic questions. Some of these questions were messaged together – six and seven for group 1 (though in groups 2 and 3, the questions were messaged separately). The aim here was to allow participants to focus on a topic, limiting the text length for each question and potentially having more responses. The format was one or two questions every 2 hours without posting questions during lunchtime (12:00–14:00) and after 18:00. ALEJ commented on and replied to interactions during these hours and made themselves available until 20:00. For each day, there was an introduction in which participants were greeted, thanked and provided with the questions. During the second and third days, participants were reminded to comment on what others had said previously and also on unanswered questions even if they were from a previous day. Questions addressed experiences of pregnancy, birth and motherhood, sex during pregnancy and support from health care workers, partners and relatives.

After the experience with the first focus group, some OAFG procedures were altered. It was decided to reduce the total number of questions to eight and to continue posting each question every 2 hours. The hope was that with fewer questions, there would be more time for participants to comment. The rhythm of questions every 2 hours was chosen because it was consistent with what has been acknowledged as a typical practice amongst smartphone users: they ‘pay attention to incoming messages in a continuous way’ (Kaufmann and Peil, 2020).

As the literature shows, one of the problems for asynchronous focus groups is the (lack of) ‘conversational’ flow and continuity of interaction among participants (Gordon et al., 2021; Hallam, 2021; Veloso, 2020). In response, more questions or comments by the moderator were included to clarify aspects of participants’ comments and to encourage responses to what others have shared. More emojis and comments were entered in response to participant entries as an attempt to convey a sense that participants had been ‘heard’ and to indicate active interaction.

At the end of each OAFG, participants were thanked and reminded to delete the last messages, exit the group and delete it. Also, information regarding their participation bonus (£5 voucher) was provided. Before exiting the group, several participants expressed their thanks for the opportunity to interact with other women, to learn from one another, and to be heard. As one participant (14 OAFG 3 voice message) put it: ‘Well thank you, I participant whatever number say goodbye because I learned from other moms and well if there is another occasion I’ll be attentive’ (“Bueno muchas gracias también, me despido la participante cualquier número que sea porque de las otras mamás también se aprendió mucho y bueno de todas maneras si en otra ocasión se puede estaré atenta”). Participants also highlighted the importance of doing research that potentially yields ‘real’ changes beyond the research project itself, and said they would be open to participating in future focus groups if necessary.

Some reflections on our OAFGs

In attending to the specific differences and similarities between face-to-face focus groups and our OAFGs, we can begin by summarising some key aspects of the former. In face-to-face focus groups, moderator(s) and participants gather in a space organized to allow a ‘fluid’ and ‘continuous’ interaction in which discussion proceeds in response to a series of questions asked by the moderator or to comments made by co-participants. Such groups take generally between 1 or 2 hours and involve a maximum of around twelve or fourteen participants, and they are usually audio or video recorded for further analysis. Sometimes besides a moderator, another researcher or observer is present in order to take field notes. Activities and objects are occasionally introduced into the space of offline focus groups, and their purpose is to function as ice breakers or to facilitate discussion. Only rarely are mobile phones or other digital devices brought into face-to-face focus groups and included in the group dynamic. Their use is restricted for several reasons, including to ensure anonymity, to reduce distractions, and to reproduce a ‘natural’ setting where people meet to talk about something in a way that mimics what they ‘normally’ do. In face-to-face focus groups, there is a clear indication of who is ‘in’ the focus groups, of what is public (versus private) and where the boundary lies between anonymous and non-anonymous.

By contrast, in OAFGs many of these features are different in ways that reflect the affordances of the platform and the ways in which it is typically used. However, additionally, these differences are nuanced by the specific circumstances of the OAFGs: in our case, this means pregnant women or recent mothers from relatively impoverished parts of Colombia who have been ranked as being at higher risk of Congenital Syphilis. A number of points of difference emerged that we reflect on below. These can be broken down along the following initial (intersecting) lines: Presence and Attention; Silence and Meaning; In/Out and Public/Private; and Continuity and Expression.

Presence and attention

Being conducted through WhatsApp, it is not always clear who is ‘present’ – there is a less determinate membership at any one time. As an audience, there is variability in who is paying attention. Even where there are more or less explicit indications of attention (say, an emoji being sent in response to a post), the ‘quality’ of attention is not necessarily clear (is this just a ‘superficial’ marker?) At base, the usual facial and postural markers of attention (problematically performative though these might be) are absent. In the three OAFGs conducted, all moderator messages were received and read, and participants’ voice messages were listened to. During the online asynchronous focus groups, although participants did not comment on all of the questions, there were 601 messages including deleted messages (group one 222, group two 238, group three 141), 64 of which were direct or indirect replies to previous comments from another participant. Of course, there could always have been more comments and responses and one of the issues the analyst faces is how to characterise those who do not contribute so much, or even what counts as a ‘good’ number of responses.

Usually, in online forums, as in other online platforms, participants are considered direct or indirect recipients of messages (Meredith et al., 2021). Amongst the various users, there are so-called ‘lurkers’ (MacNamara et al., 2021) who are unknown people who read messages or posts but tend not to interact, particularly on large sites (Meredith et al., 2021). Cautiously, the term ‘lurker’ or perhaps, better still, ‘onlooker’ can be applied within the OAFG setting, insofar as there are participants who do not participate in every topic, who it is assumed (rightly or wrongly) do watch other views being expressed but do not put forward their own. However, this might be less a personal characteristic than a cultural or practical one. In both offline and online focus groups, there are moments of increased participation where more participants intervene in (or are reticent with regard to) some topic or other. In the OAFGs reported here, there was more frequent participation during the first day (groups 1 and 3) but longer messages with more characters during the third day (groups 1 and 3)⁴. More participants contributed to topics concerning the expectations placed on pregnant women and new mothers, the caring received from health care workers, and advice surrounding pregnancy and birth. By contrast, contributions about sex during pregnancy were less frequent and generated more ‘silence’: indeed, only two participants in each group commented.

In sum, in OAFGs there are several ways in which presence and attention are rendered more opaque by the platform, in this case by WhatsApp. This is further complicated when more ‘taboo’ subjects such as ‘sex during pregnancy’ are raised. Does silence signal indifference or embarrassment? The signs of these that might be picked up in face-to-face focus groups are absent.

Silence and meaning

In both offline or online synchronous focus groups, concern on the part of the moderator (or even other participants) regarding participant silence is not uncommon. Silence might be interpreted as an absence of interaction, as the loss of meaning production, or as some incapacity on the part of the moderator. In an online asynchronous group, participants tend to comment to each other less, and there might be long periods of silence in which it is not easy to see engagement (Chen and Neo, 2019). Silence in asynchronous groups does not necessarily indicate a lack of interaction, particularly if we consider delays in interaction in which reading and replying to messages occurs at participants’ own pace. After all, it is important to recall that silence is a feature of interaction present in research and non-research contexts (Smithson, 2008).

Nevertheless, silence can also indicate discomfort among group participants (Myers 1998 in Smithson, 2008), an indicator of a sensitive topic or even a taboo: silence, by its very appearance, can indeed index the social discomfort that attaches to statements or comments or questions shared within the group. This was the case with the question about sex during pregnancy in our OAFGs. That question resulted in fewer messages and comments during the period following the focus group. In this respect silence can also implicate ‘stigma’ regarding specific topics, particularly where knowing ‘too much’ about a topic is problematic; that is to say, participants may be feel inhibited because of the possibility of being seen to be too knowledgeable by virtue of engaging in disfavoured practices (see Kitzinger, 1994, for similar observations in relation to HIV/AIDS).

However, silence is something worth paying attention to, especially with regard to what it might say about engagement. It has been noted that with insufficient attention to, and monitoring of, ‘silent’ participants in asynchronous focus groups, these participants might quickly drop out (Stewart and Shamdasani, 2017). However, in the context of our OAFG participants it would be preferable to see silence as a reflection of a more fluid agency whereby the participant can decide when and how long they participate. Although this might confuse the practical matter of engagement from a researcher’s perspective, it nevertheless allows us to glimpse what is relevant to participants’ engagement, interaction and the perceived importance attributed to the topic at hand. More generally, it allows us to interrogate what our assumptions are about engagement in OAFGs. In particular, can we detach the idea of engagement from data-production? People might still be engaged with the research matter of concern but not necessarily directing that engagement toward the researchers. This suggests that the engagement extends beyond the boundaries of the research encounter (OAFG) or even research project or programme. This further implies that the topics and issues raised in OAFGs might continue to resonate with participants off-line, and that perhaps other methods need to be developed to access such resonances (e.g. follow-up on-line interviews or questionnaires).

Relatedly, of course, participants can decide to withdraw from the OAFG without providing an explanation – something that is not necessarily so easy in the context of in-person focus groups (Tates et al., 2009). Such a withdrawal can be rather more blurred in a focus group conducted on WhatsApp. So, while in asynchronous focus groups non-participation can also be understood as withdrawal, a note of caution needs to be sounded because what counts as participation is not altogether clear. For instance, in the case with one of our focus group participants, while she contributed to the group during the second day of the asynchronous focus group, she participated only twice. In fact, she did leave after that day, but (as she informed ALEJ during recruitment) she had to return to work from maternity leave because of a tight work schedule. As this case illustrates, silence and withdrawal can signify several things.

Such a situation might be common to others in offline focus groups who have stopped contributing and commenting. But should we understand this as a withdrawal? A number of questions suggest themselves here. Does this rather suggest that we should consider asynchronicity as a fluid, non-linear temporal sequence regarding participation? Moreover, in partial reiteration of the foregoing, does perceiving this situation as a lack of engagement and a withdrawal not reveal our researcherly assumptions regarding the ‘gold standard’ and what is expected from research participants? In addition, does the possibility of multimodal interaction on WhatsApp (sharing links, videos, voice messages, songs, memes, emojis, stickers and gifs) suggest that other connective communicative routes can be created along with different, more inchoate ‘worlds’ (more on this below) in which withdrawal is part of a pattern of presence and absence?

In/out and public/private

Needless to say, the participants in our OAFGs were entangled in many relations and engaged in many practices. For example, judging by the background sounds that appeared on their voice messages, participants contributed while at work, while engaged in childcare or housework, or while watching TV. However, while in face-to-face focus groups talk amongst participants can be witnessed, in OAFGs it is not always possible to know if participants also read what others said and to what extent they are engaged in cross-participant discussion. In this respect, one can see that these participants might be outwith the focus group, conducting discussions out of reach of the researcher. Or rather, one might even say that it is the researcher who is ‘kept out’ in a version of Law’s (1994) adage in which, to paraphrase, ‘wherever the action is happening, the researcher is unlikely to be’. Of course, this can happen with face-to-face focus groups when discussion amongst participants continues after the end of the focus group. However, as a discussion that runs parallel to the main interactions within an OAFG, the ‘integrity’ of the focus groups seems more compromised.

This ‘integrity’ seems further eroded when the ‘private’ entanglements of participants make an appearance. As noted, the private runs in parallel to the public-ity of being in an OAFG, whereas in face-to-face focus groups the private is, as it were, ‘suspended’ for the duration of the focus groups session (of course, this is never wholly the case as, for example, emergency messages can in one way or another reach participants). However, because of the remote and asynchronous features of an OAFG participants are at the same time ‘in’ and ‘out’ of their personal life, work and family circumstances. In essence, people are switching from public to private (and non-anonymous to anonymous) interactions for the length of the OAFG. For instance, as the transcript extract below indicates, one of our participants was replying to messages on WhatsApp with her family regarding her ill son at the hospital, apologising to other participants for not participating that morning in the focus group, and writing private messages to ALEJ (who reassured her it was possible to withdraw from the focus group if she needed to).

‘Participant 13 OAFG 3 12:02: Afternoon, apologies. I didn’t attend early

It’s just that I have my boy at the Emergency Room with a terrible flu

12:03: (Picture of the boy with an oxygen mask)

Participant 17 (voice message) 12:04: {Vallenato music} The boy you have to ask God for your boy’s recovery. What disease might have the boy?

Moderator 12:04: I am sorry. I hope your boy recovers soon. Don’t worry. Remember you can participate when you can’

Here, there was multitasking: engaging and interacting with different actors at the same time. Moving from offline to online can entail the doing of several activities simultaneously, using and navigating different types of texts (text messages, voice messages, music, apps and videos). Of course, this is part of everyday digital life (Kaufmann and Peil, 2020), and the absence of direct observation on the part of the researcher, means this can continue even within the on-going unfolding of an OAFG. Participants can thus remain both within and without the research event simultaneously.

Continuity and multiplicity

If we consider digital interaction more broadly, it is any communication in which ‘at least two speakers take alternative turns’. (Meredith et al., 2021: 7). However, given that not every message will be responded to in digital interaction, communication will not have the same qualities of continuity that face-to-face interactions putatively possess. In any case, it might be challenging to reconstruct the ‘flow’ of an online focus group interaction because of discontinuities. Or rather, perhaps we should consider such ‘staccato’ interactions in their own right insofar as they can still illuminate women’s divergent experience of Colombian health care systems and Congenital Syphilis by setting these within an interactional context that is discontinuous, dispersed (involves side communications) and heterogeneous (entails multiple tasks).

Of course, ‘… any message posted on a digital platform is “designedly interactional”’(Meredith and Potter, 2014 in Meredith et al., 2021: 7) insofar as contributions are designed with the recipient in mind. In the present case participants were speaking to other women who had gone through, or were going through, the pregnancy experience, and to the researcher, someone who was interested in knowing their experiences. However, it is not always straightforward in knowing if participants had read what others had said, and to what extent they were engaged with the discussion. Expression of views does not always signal a particular form of engagement, or any engagement per se. Now, it has been alleged that in virtual focus groups, there is not much interaction (Liamputtong, 2015; Hesse-Biber and Griffin, 2013), particularly if those focus groups are asynchronous, when there are delays in comments of a couple of hours or even days. As noted above, this does not mean that there is a lower grade of data being generated, but rather that what counts as data has shifted. It is possible to find continuities across asynchronous interactions by, for example, following an extended ‘story’ that can sometimes be a drawn-out ‘argumentative interaction’ (Kitzinger, 1994). To be sure, amongst our OAFGs, there was storytelling surrounding risk, about being a ‘primeriza’⁵, about health care workers’ explanations about pregnancy and birth processes. Also, there was storytelling regarding sickness or illnesses during pregnancy, weight (diets, losing and gaining weight) and how to treat and care adequately for pregnant women and babies.

However, the key point is that running alongside such, albeit circuitous, narrative linearities, is a nexus of communications that embodies text but also voice, videos, images, links, emojis, songs and background ‘noise’ that flip in and out of public and private. Here is a brief transcribed example:

Participant 11 OAFG 2 8:37:

https://www.youtube.com/watch?v=jcH4hL0LVn8 (Song: “Yo te esperaba” {I was waiting for you} by Alejandra Guzman)

Good morning

Moderator 8:40: I did not know that song. Tell us about your experience

Pariticipant 11 OAFG 2 8:41:

Joyful and I thought how time passed by and I want to get to know my baby it was a beautiful experience

To navigate such a variety of presences and absences (including silences and discontinuities), reflects not just the usual mess of doing research (Law, 2004) but also is potentially indicative of the specific complexities and heterogeneities of our participants’ lives. These ‘data fragments’, for want of a better term, cannot be analytically grasped in any straightforward way. At the very least, they provide a sort of cacophonic background against which narratives are crafted and play out. However, they also raise the possibility of rethinking what counts as an insightful reading.

Concluding remarks: OAFGs – within and beyond research?

In this paper, we have discussed features of Online Asynchronous Focus Groups in the context of our particular project, namely, to examine the experiences of pregnant women and recent mothers of Congenital Syphilis prevention in Colombia. OAFG as a method was chosen in response to the restrictions imposed on mobility during the pandemic because some of our participants were hard to reach, and for the reason that the asynchronicity afforded by the use of WhatsApp could accommodate the womens’ busy schedules and fragmented availability. In addition, it was hoped that OAFGs, insofar as they facilitated confidentiality, could provide a relatively safe forum for sharing accounts of medical, cultural and personal concerns that were potentially hugely sensitive. With these qualities of OAFGs in mind, we provided an initial comparison between OAFGs and face-to-face focus groups, as well as tracing some of the differences between other types of OAFGs (e.g. those where only text was used). In contrast, our OAFG format also allowed for a variety of significations that included emojis and verbal recordings.

In our reflections on the sort of data we collected, as well as its ‘absence’, we pointed to a number of readings that situated that data as a sort of panoply that although in some ways inchoate, nevertheless permitted a complex and variegated engagement. Thus, we noted how the presence of participants and the quality of their attention were difficult to judge through WhatsApp, especially in relation to taboo topics such as sex during pregnancy. As a corollary, we discussed the issue of silence and how this presumes a link between participation and data generation. The women might still be engaged, but that engagement finds other ways and means of expression (and might well require supplementary methods if it is to be ‘captured’). As such we also considered how the women might be carrying on on-line conversations amongst themselves in parallel to the OAFG. This suggested a world outside of the OAFG even if it is a world partially informed by the OAFG topic of Congenital Syphilis prevention. We noted that this complex, variegated world is occasionally glimpsed in interviews or the background noises that attend vocal recordings or through the use of songs or emojis. What this implies is an erosion of the borders of the OAFG as the OAFG becomes affected by ‘data’ that are tangential to, or obscure within, the parameters of the OAFG. At the very least, one implication is that silence is not a void. Finally, all these variegated points of engagement suggest a reconsideration of what counts as data: if we have collected ‘data fragments’, as we tentatively call them above, these can no doubt resource a narratively linear account, but at the same time they can remind us about the empirical cacophony that lies behind such an account (Law, 2004; Serres, 1995). Indeed, we would argue that useful insights can be grounded in non-linear narrative forms. For example, the irregular jumping across disparate themes, issues, concerns and so on might echo the complex and fragmentary conditions of health care which our participants face in their particular localities.

Our reflections on our particular OAFGs raise some broader implications – or at least questions – for the deployment of on-line focus groups that are asynchronous. Firstly, OAFGs that employ WhatsApp (or similar platforms such as Signal) and which thus allow for the use of various forms of signification (sharing links, videos, voice messages, songs, memes, emojis, stickers and gifs) and alternative routes of communication (cross-communications amongst participants), generate the possibility of a much richer sense of the participants. Indeed, to the extent that they introduce different affects (through emojis, gifs, etc.) and identities (through songs and stickers, etc.), the persona of ‘participant’ seems rather limited. What alternative formulations of ‘participant’ might begin to approach this richness? Further, the topics to which the participants are supposed to address themselves might be rather compromised in that the various significations that are co-presented with textual answers can reinforce, detract from, nuance, and even side-line those topics. Even translating the voice message into text and reproducing the emojis, some meaning is lost in the translation. What can we as researchers derive from this panoply of ‘data fragments’? How can we bring order to it? Or better still, how might we find modes of analysis and writing that can some encompass the heterogeneity of messages?

Might we find a way of seeing this heterogeneity as expressive of the sociomaterial heterogeneities that are part and parcel by participants’ everyday lives? In light of this, we might also ask what these modes of messaging imply for the research method itself. The diversity of messaging can, as it were, ‘re-enact’ or ‘reconfigure’ the OAFG. For certain focus group memberships, such multiple messaging entails a ‘stepping out’ of the OAFG, and even of the research project per se, to, for example, direct care of one sort or another to other members of the OAFG. Sometimes this might be in the form of discrete advice, sometimes as a sharing of experiences, and sometimes as a moment of solidarity (as in the case where participants shared their bad experiences of prenatal care, birth and breastfeeding). In this way, the OAFG as a research event becomes something other – a forum for mutual support or the building of a sense of solidarity (see Michael, 2012). What we are witness to as researchers is thus an appropriation of our research tool which, by virtue of its various technical capacities, has facilitated both a partial breakdown in the aims and continuity of the OAFG and of the research project itself. In other words, there can be a re-appropriation of the OAFG by the participants themselves as a different sort of event. Often, researchers by virtue of their focus on their core research questions might miss such a shift (which can admittedly be very subtle). Given such a re-appropriation, how might researchers sensitise themselves to the usurpation of their OAFG research event so that it is no longer simply a research event but has become something other in which alternative and parallel relations are enacted, other identities are introduced and other worlds are imagined?

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by Ministry of Science and Technology of Colombia (MinCiencias); Beca Doctorados Exterior Convocatoria 860, 2019.

ORCID iDs

Ana Estrada-Jaramillo

Mike Michael

Hannah Farrimond

Notes

Author biographies

Ana Estrada-Jaramillo is a PhD Candidate in Anthropology at the University of Exeter, UK. Her research focuses on Congenital Syphilis prevention in Colombia by employing online methodologies. The study intends to contribute to how to provide good care in a prenatal care setting while acknowledging more-than-‘expert’ and ‘scientific’ ontologies. More broadly, she is interested in disease prevention, Anthropology and Sociology of Health, STS, public understanding of Science and Technology, applied research and methods.

Mike Michael is a Professor in the Department of Sociology, Philosophy and Anthropology at the University of Exeter. His research interests touch on the relation of everyday life to technoscience, the role of culture in biomedicine and biotechnology, and the interplay of design and social scientific perspectives.

Hannah Farrimond is a Senior Lecturer in Medical Sociology at the Centre for the Study of Life Sciences (EGENIS), University of Exeter, UK. Her research focuses on the psycho-social meanings of legal addictions such as smoking and vaping, alcohol, and pharmaceutical use. In particular, she is interested in stigma, how it changes over time, and the role of public health in perpetuating or challenging it.

References

Abrams

Wang

Jin Song

, et al. (2015) Data richness trade-offs between face-to-face, online audiovisual, and online text-only focus groups. Social Science Computer Review 33(1): 80–96. DOI: 10.1177/0894439313519733

Biedermann

(2018) The use of Facebook for virtual asynchronous focus groups in qualitative researchContemporary Nurse 54(1): 26–34. DOI: 10.1080/10376178.2017.1386072

Brüggen

Willems

(2009) A critical comparison of offline focus groups, online focus groups and e-Delphi. International Journal of Market Research 51(3): 363–381. DOI: 10.2501/S1470785309200608

Charlot

Carolan

Gawuga

, et al. (2021) ‘Patient powered research: an approach to building capacity for a hardly reached patient population to engage in cancer research’, Research Involvement and Engagement. BioMed Central 7(1): 1–9. DOI: 10.1186/s40900-021-00317-7

Chen

Neo

(2019) Texting the waters: an assessment of focus groups conducted via the WhatsApp smartphone messaging application. Methodological Innovations 12(3): 205979911988427. DOI: 10.1177/2059799119884276

Colom

(2021) Using WhatsApp for focus group discussions: ecological validity, inclusion and deliberation. Qualitative Research 22: 1–16. DOI: 10.1177/1468794120986074

DANE (2019) Boletín Técnico Indicadores básicos de tenencia y uso de Tecnologías.

Dicks

(2014) Action, experience, communication: three methodological paradigms for researching multimodal and multisensory settings. Qualitative Research 14(6): 656–674.

Flynn

Albrecht

Scott

(2018) Two approaches to focus group data collection for qualitative health research: maximizing resources and data quality. International Journal of Qualitative Methods 17(1): 1–9. DOI: 10.1177/1609406917750781

10.

Forrestal

D’Angelo

Vogel

(2015) Considerations for and lessons learned from online, synchronous focus groups. Survey Practice 8(3): 1–8. DOI: 10.29115/sp-2015-0015

11.

Fox

Morris

Rumsey

(2007) Doing synchronous online focus groups with young people: methodological reflections. Qualitative Health Research 17(4): 539–547. DOI: 10.1177/1049732306298754

12.

Garzón Chirivi

(2015) Aproximación a un estado del arte sobre prácticas de medicina tradicional y popular en Hispanoamérica. Folios 41: 157–168. DOI: 10.17227/01234870.41folios157.168

13.

Gordon

Calzo

Eiduson

, et al. (2021) Asynchronous online focus groups for health research: case study and lessons learned. International Journal of Qualitative Methods 20: 1–13. DOI: 10.1177/1609406921990489

14.

Hallam

(2021) Moving on from trials and errors: a discussion on the use of a forum as an online focus group in qualitative research. International Journal of Social Research Methodology: 1–11. DOI: 10.1080/13645579.2021.1888402

15.

Hesse-Biber

Griffin

(2013) Internet-mediated technologies and mixed methods research: problems and prospects. Journal of Mixed Methods Research 7(1): 43–61. DOI: 10.1177/1558689812451791

16.

Holst

(2021) Colombia: Number of Mobile Phone Subscribers 2016-2021. Available at: https://www-statista-com.uoelibrary.idm.oclc.org/statistics/622707/mobile-connections-in-colombia/

17.

Islam

Joseph

Chaudry

, et al. (2021) ‘“We are not hard to reach, but we may find it hard to trust”. Involving and engaging “seldom listened to” community voices in clinical translational health research: a social innovation approach’, Research Involvement and Engagement. Research Involvement and Engagement 7(1): 1–15. DOI: 10.1186/s40900-021-00292-z

18.

Jailobaev

Jailobaeva

Baialieva

, et al. (2021) WhatsApp groups in social research: new opportunities for fieldwork communication and management. BMS Bulletin of Sociological Methodology/Bulletin de Methodologie Sociologique 149(1): 60–82. DOI: 10.1177/0759106320978337

19.

Kaufmann

Peil

(2020) The mobile instant messaging interview (MIMI): using WhatsApp to enhance self-reporting and explore media usage in situ. Mobile Media and Communication 8(2): 229–246. DOI: 10.1177/2050157919852392

20.

Kitzinger

(1994) The methodology of focus groups: the importance of interaction between research participants. Sociology of Health & Illness 16(1): 103–121. DOI: 10.1111/1467-9566.ep11347023

21.

Kress

(2010) Multimodality. A Social Semiotic Approach to Contemporary Communication. New York: Routledge.

22.

Law

(1994) Organizing Modernity. Oxford: Blackwell.

23.

Law

(2004) After Method: Mess in Social Science Research. London: Routledge.

24.

Liamputtong

(2015) Focus group methodology: introduction and history. In: Focus Group Methodology: Principles and Practice. Sage, 1–14. DOI: 10.4135/9781473957657.n1

25.

Lobe

Morgan

(2021) Assessing the effectiveness of video-based interviewing: a systematic comparison of video-conferencing based dyadic interviews and focus groups. International Journal of Social Research MethodologyRoutledge 24(3): 301–312. DOI: 10.1080/13645579.2020.1785763

26.

Lury

Wakeford

(2012) Inventive Methods. The Happening of the Social. London: Routledge.

27.

MacNamara

Mackle

Devlin Trew

, et al. (2021) Reflecting on asynchronous internet mediated focus groups for researching culturally sensitive issues. International Journal of Social Research Methodology 24(5): 553–565. DOI: 10.1080/13645579.2020.1857969

28.

Marres

(2015) Why map issues? On controversy analysis as a digital method. Science Technology and Human Values 40(5): 655–686. DOI: 10.1177/0162243915574602

29.

Meredith

Giles

Stommel

(2021) Introduction: the microanalysis of digital interaction. In: Meredith

Giles

Stommel

(eds) Analysing Digital Interaction. Cham: Palgrave Macmillan, 1–22.

30.

Michael

(2012). Toward an Idiotic Methodology: De-signing the Object of Sociology. The Sociological Review, 60, S1, 166-183.

31.

Ministerio de Salud y Protección Social (2020) Lineamientos provisionales para la atención en salud de las gestantes, recién nacidos y para la lactancia materna, en el contexto de la pandemia de COVID-19 en Colombia. Minsalud.

32.

Mol

(2002) The Body Multiple: Ontology in Medical Practice. Durham, NC: Duke University Press.

33.

Morgan

Hoffman

(2018) A system for coding the interaction in focus groups and dyadic interviews. Qualitative Report 23(3): 519–531. DOI: 10.46743/2160-3715/2018.2733

34.

Nicholas

Lach

King

, et al. (2010) Contrasting internet and face-to-face focus groups for children with chronic health conditions: outcomes and participant experiences. International Journal of Qualitative Methods 9(1): 105–121. DOI: 10.1177/160940691000900102

35.

Ranieri

Kennedy

Walmsley

, et al. (2019) Rare but heard: using asynchronous virtual focus groups, interviews and roundtable discussions to create a personalised psychological intervention for primary sclerosing cholangitis: a protocol. BMJ Open 9(10): 1–5. DOI: 10.1136/bmjopen-2019-031417

36.

Reisner

Randazzo

Hughto

JMW

, et al. (2018) Sensitive health topics with underserved patient populations: methodological considerations for online focus group discussions. Qualitative Health Research 28(10): 1658–1673. DOI: 10.1177/1049732317705355

37.

Serres

(1995) Genesis. Ann Arbor: Michigan University Press.

38.

Silverman

(2007) A Very Short, Fairly Interesting, and Reasonably Cheap Book about Qualitative Research. London: Sage.

39.

Smithson

(2008) Using focus groups in social research. In: The Sage Handbook of Social Research Methods. London: Sage, 356–371.

40.

Sokol

Fisher

(2016) Peer support for the hardly reached: a systematic review. American Journal of Public Health 106(7): e1–e8. DOI: 10.2105/AJPH.2016.303180

41.

Stewart

Shamdasani

(2017) Online focus groups. Journal of Advertising 46(1): 48–60. DOI: 10.1080/00913367.2016.1252288

42.

Stewart

Williams

(2005) Researching online populations: the use of online focus groups for social research. Qualitative Research 5(4): 395–416. DOI: 10.1177/1468794105056916

43.

Sugie

(2018) Utilizing smartphones to study disadvantaged and hard-to-reach groups. Sociological Methods and Research 47(3): 458–491. DOI: 10.1177/0049124115626176

44.

Tates

Zwaanswijk

Otten

, et al. (2009) Online focus groups as a tool to collect data in hard-to-include populations: examples from paediatric oncology. BMC Medical Research Methodology 9(1): 1–8. DOI: 10.1186/1471-2288-9-15

45.

Torri

Laplante

(2013) Traditional medicine and biomedicine among mapuche communities in temuco, chile: new forms of medical pluralism in health care delivery. Anthropologica 55(2): 413–423.

46.

Veloso

(2020) WhatsApp como ferramenta para a organização de grupos focais online na pesquisa em Educação: Um relato de experiência. São Carlos, Ago: Congresso Internacional de Educação e Tecnologias. Available at: https://cietenped.ufscar.br/submissao/index.php/2020/article/view/964.

47.

Williams

Clausen

Robertson

, et al. (2012) Methodological reflections on the use of asynchronous online focus groups in health research. International Journal of Qualitative Methods 11(4): 368–383. DOI: 10.1177/160940691201100405

48.

Woodyatt

Finneran

Stephenson

(2016) In-Person versus online focus group discussions: a comparative analysis of data quality. Qualitative Health Research 26(6): 741–749. DOI: 10.1177/1049732316631510

49.