Conducting member checking within a qualitative case study on health-related behaviours in a large European city: Appraising interpretations and co-constructing findings

Abstract

Although member checking is a well-established strategy for appraising credibility, there is a lack of research reporting procedures and outcomes when using this strategy. In recent years, scholars have implemented new member checking strategies along with several epistemological stances. In this work, researchers conducted member checking in three neighbourhoods with different socioeconomic status in Madrid, Spain. Attendance at member checking meetings was lower than expected. Member checking meetings were conducted in public halls within each neighbourhood and lasted approximately 2 hours. During the first hour, researchers introduced the aim of the meetings and shared summarized research findings with attendees using a slideshow. During the second hour, attendees had the opportunity to comment on any aspect of the research. Researchers used grounded theory coding strategies and a constructivist approach informed the analysis of the transcribed member checking meetings. One core category emerged, and it was named ‘Co-constructing findings and side effects of the interaction between researchers and participants in member checking’. Member checking attendees contributed to co-constructing findings by means of underscore and disagreement. It is possible that these meetings might have caused side effects such as unintended comparisons between neighbourhoods and negative health-related behaviours. Attendees wondered how the findings could improve their neighbourhoods. The meetings allowed interaction between participants and researchers and resulted in appraising and co-constructing qualitative research findings. More research is needed to advance knowledge about member checking.

Keywords

case study credibility grounded theory member checking qualitative research

Introduction

In qualitative research, researchers typically act as both data collectors and analysts. Because of this, debates about rigour in qualitative research focus on researcher bias (Birt et al., 2016; Miles and Huberman, 1994). Rigour has been considered the key element to judge the quality of any research using either a qualitative or quantitative approach (Lincoln and Guba, 1985; Sandelowski, 1986). Rigour or trustworthiness in qualitative research can be defined as the truth-value of data (Lincoln and Guba, 1985).

Lincoln and Guba (1985) described credibility as the extent to which the research findings are accurate (in quantitative terms, as the extent of internal validity). They developed several strategies to increase credibility in qualitative research: triangulation, prolonged engagement, peer-debriefing, negative case analysis and member checking. Today, rigour is an increasingly central focus when appraising qualitative research, and scientific journals suggest using any of the aforementioned strategies to meet their quality standards. Furthermore, some of them are often included in quality checklists, which are aimed at guiding experts when appraising the rigour of qualitative studies (e.g. O’Brien et al., 2014; Tong et al., 2007).

The strategies developed by Lincoln and Guba (1985) have been extensively implemented in qualitative research, but in recent years reflexivity has also been considered an essential part of credibility. Using reflexivity in qualitative inquiry means that researchers must be aware of how their perspective, background, and personal assumptions could influence the research process (Patnaik, 2013). We consider that use of member checking in qualitative case studies is related to the definition of reflexivity. Furthermore, reflexivity is considered a key aspect in constructivist Grounded Theory (GT) (Charmaz, 2014, 2017) as it is essential when co-constructing realities and to preserve participants’ voices, which are also the aims when conducting qualitative case studies and member checking.

The present work was prompted by the member checking that was conducted during the final stages of the data collection process in a large type 2 qualitative case study with different units of analysis (e.g. neighbourhoods) (Yin, 2014) on health-related behaviours. Conducting member checking is recommended in qualitative case studies, evaluation research and community-based research (Creswell, 2005; Mertens, 2018; Yin, 2014). Qualitative case studies must show precise interpretations of data and in this type of studies, transactional validity is a desirable aim (Doyle, 2007). Conducting member checking in case studies provides a great opportunity to appraise data interpretations (e.g. abstractions, theoretical interpretations) (Torrance, 2012).

Background

Lincoln and Guba (1985) described member checking as a strategy that involves returning the research outcomes to participants, which allows them to confirm or deny the accuracy of the data interpretations made by researchers. Member checking is a long-standing and well-established strategy in qualitative inquiry (Guba and Lincoln, 1989; Lincoln and Guba, 1985), but scholars rarely report on details about how the strategy is used, the outcomes, ethical implications and any changes resulting from its use. The lack of reporting on the process and outcomes when conducting member checking underlies some of the criticisms of this strategy (Birt et al., 2016; Morse, 2015). Furthermore, there is little consensus about member checking in the intense debate on rigour and credibility in qualitative research (Morse, 2015; Morse et al., 2002; Rolfe, 2006; Sandelowski and Barroso, 2002; Silverman, 2017).

Although this strategy has been criticized due to the positivist roots of rigour strategies, conducting member checking does not necessarily imply that researchers must hold positivist assumptions (Thomas, 2017). Far from the positivist roots of the strategy, scholars have begun using member checking to ensure that experiences and voices of participants are adequately represented and not merely evaluated (Barbour, 2018; Mertens, 2015). Previous research has recommended using member checking to establish equitable relationships between researchers and study participants, and to give voice to participants (Bradshaw, 2001; Brear, 2019).

In recent years, scholars have been conducting member checking under different epistemological stances. The most common member checking strategy consists in researchers sending participants written drafts containing either a full report on analysed data or raw data in the form of transcriptions (Hermann et al., 2012; Pazokian et al., 2014). This strategy is like the original member checking described by Lincoln and Guba (rooted in positivist assumptions), but it is often conducted from an objectivist epistemological stance as it is often used to determine the accuracy of findings and transcriptions (Carlson, 2010; Mabry, 1998; Wainwright et al., 2010). Once the participants have reviewed either the full findings report or transcriptions, they send their feedback to researchers, who may or may not change data interpretations according to the feedback.

Another frequently used strategy consists in conducting follow-up interviews with participants. This strategy examines the accuracy of data interpretations using selected verbatims, interpretations made by researchers, and a full findings report (Buchbinder, 2011; Koelsch, 2013). In the same line, member checking can also consist in conducting follow-up focus groups (FGs). Participants in follow-up FGs could be those who took part in the research (Klinger, 2005; Stringer, 2003) or people who did not participate but belong to the studied group (Harvey, 2015). In recent years, scholars have been conducting new member checking strategies such as synthesized member checking (Birt et al., 2016; Harvey, 2015), dialogic member checking (Brear, 2019), pamphlets (Caretta, 2016) and question-answer validity procedures (Roller and Lavrakas, 2015).

The above shows that since the first member checking was described by Lincoln and Guba (1985), scholars have developed a wide variety of member checking strategies underpinned by different epistemological stances. However, while rigour is increasingly demanded in qualitative research there is a lack of reported details when member checking is conducted. Critical revisions of member checking are much needed and research reporting on the process and outcomes of this strategy seems warranted (Birt et al., 2016; Morse, 2015; Thomas, 2017).

Objective of the study

In the present study, we describe our member checking process, and we analyse its results to illuminate future research conducting similar qualitative case studies on health-related behaviours. The aim of this research was to explore the outcomes from conducting member checking in a qualitative case study on health-related behaviours in a large European city like Madrid (Spain).

Methodology of the study

In the following section, we provide detailed information about the qualitative case study within which this research is nested and about the methodology of this work. The objective of the qualitative case study was to explore health-related behaviours such as diet, physical activity, alcohol consumption and smoking in three different socioeconomic status (SES) neighbourhoods (González-Salgado et al., 2020, Rivera-Navarro et al., 2021a; 2021b) in a large European city like Madrid (Spain).

Study area

Researchers selected three distinctive SES neighbourhoods within the city of Madrid: San Diego (low SES), El Pilar (medium SES) and Nueva España (high SES). The neighbourhoods were selected using seven socioeconomic indicators representing the demographic and socioeconomic structure of the neighbourhoods. Researchers used Z-scores and performed unweighted linear additions to build an index for each neighbourhood. A low value in this index represented neighbourhoods with low SES, while a higher value represented neighbourhoods with high SES. Among the 128 neighbourhoods in the city of Madrid, researchers selected San Diego (low SES), El Pilar (medium SES) and Nueva España (high SES). Details of the selection criteria have been explained in detail elsewhere (Rivera Navarro et al., 2019). The low SES neighbourhood has a high population density and is characterized by a large percentage of low-skilled and low-income immigrant residents, a well-established Romany community, high unemployment rates and a high percentage of low-skilled workers and part-time workers. The main characteristic of the medium SES neighbourhood is that it has two distinct areas: (1) an area where most of the residents are blue-collar workers and immigrants; and (2) an area where most of the residents are liberal professionals and where fewer immigrants live. In the high SES neighbourhood, most of the residents have high-incomes and are high-skilled workers.

Data collection techniques and tools

In the qualitative case study in which this research was carried out, researchers used semi-structured interviews (SSIs) and FGs as data collection techniques. Researchers conducted 37 SSIs, six with key informants and 31 with residents, and 29 FGs across the three neighbourhoods. Researchers conducted 12 SSIs in the low and medium SES neighbourhood, and 13 in the high SES neighbourhood. Afterwards, researchers conducted 14 FGs in the low SES neighbourhoods, 11 in the medium SES neighbourhood, and 4 in the high SES neighbourhood. While the SSIs allowed us to explore issues from individual perspectives and without discourse restrictions, the FGs allowed us to gather collective discourses (Taylor and Blake, 2015). SSIs lasted 60 minutes and were first conducted between 2016 and 2018 in different settings across the three neighbourhoods studied: from a cultural centre and a church in the low SES neighbourhood, to quiet cafés in the other two neighbourhoods. FGs lasted approximately 90 minutes and they were conducted between 2018 and 2019. FGs were conducted in rented co-working spaces from each neighbourhood. FGs participants ranged from a minimum of 5 to a maximum of 8. Study participants were contacted the day prior to data collection to provide them details about the meetings and to address any possible doubt on the data collection techniques. Researchers scheduled SSIs and FGs based on the availability of participants. Researchers designed a guide for both data collection techniques, which included questions about the four main health-related behaviours studied: diet, physical activity, alcohol consumption and smoking. Both SSIs and FGs started with a prompting question asking for a brief description of the neighbourhood, which worked as an icebreaker question.

Sampling

In the qualitative case study where this research is nested, a sociological research company used networking techniques and advertisements placed in social services, public services, and health centres to recruit study participants. Researchers conducted purposeful sampling (Schreier, 2018) aimed at selecting information-rich cases that would contribute to the objective of the qualitative case study. Researchers conducted the purposeful sampling in two steps: (1) reputational sampling (Patton, 2014): in this step six key informants were recruited and interviewed (one health centre director and one school director per neighbourhood) because their job position provided them with broad knowledge (Taylor and Blake, 2015); (2) maximum variation sampling (Patton, 2014): was conducted to purposefully select participants considering a wide range of sociodemographic characteristics (e.g. sex, age, academic level, labour status, country of origin, living arrangement, smoking status, alcohol consumption) related with the objective of the qualitative case study. Only residents aged between 40 and 85 years old were recruited because they would be more likely to be embedded in their neighbourhoods and they might be within an age range more likely to develop cardiovascular disease. Details of the sociodemographic characteristics of study participants have been explained in detail elsewhere (Rivera Navarro et al., 2019). The number of participants recruited per neighbourhood was 101 participants from the low SES neighbourhood, 74 participants from the medium SES neighbourhood, and 38 participants from the high SES neighbourhood.

Between October and November 2019 once the first analyses were finished, researchers conducted one member checking meeting per neighbourhood. Before the member checking meetings, study participants were contacted and provided further information and details about the meetings. The professional sociological research company in charge of recruiting participants placed advertisements with information about the member checking meetings in public services within the studied neighbourhoods. Researchers did not restrict access to the member checking meetings to other residents who did not participate in the data collection process.

Member checking setting and member checking attendees

Member checking meetings were conducted in public halls located in each neighbourhood. Meetings lasted approximately 2 hours and they were divided in two parts of 1 hour. During the first hour, researchers introduced themselves to attendees, then explained in detail the aim of the meetings and how attendees would help researchers to improve and evaluate study results, and finally they shared summarized study findings with attendees using a slideshow. The findings were presented in a descriptive way, avoiding theoretical abstractions resulting from qualitative analysis. Each of the member checking meeting focussed on the neighbourhood where results were presented, but also contained summarized findings from the other two neighbourhoods. Researchers started the presentation of the findings by introducing qualitative data collection techniques (SSIs and FGs) and briefly describing how findings were obtained from transcriptions (i.e. coding process). The presentation of the results was divided into the four health-related behaviours studied in the qualitative project: diet, physical activity, alcohol consumption and smoking.

During the second part of the meetings, attendees had the opportunity to speak freely for 1 hour, using microphones to comment on any aspect of the research. Member checking meetings were held in Spanish, audio recorded and transcribed by a qualified audio-typist. Verbatim translations from Spanish to English were slightly modified to maintain intended meaning, following recommendations for transcultural translations (Biering-Sørensen et al., 2011). Researchers took notes during the member checking meetings to underpin analyses. All research team members equally contributed to conduct these meetings and to present the summarized research findings to attendees. During the second part of the member checking meetings, researchers also took turns to answer attendees’ questions about any aspect of the research project (i.e. findings, data collection). Following a constructivist approach, the final results of the four health-related behaviours studied were co-constructed with member checking attendees.

Data analysis

The entire dataset of the qualitative project (37 SSIs and 29 FGs) was simultaneously analysed by half of the research team using GT coding strategies (Charmaz, 2014; Corbin and Strauss, 2015), while the other half of the team used interpretative phenomenological analysis (Cuthbertson et al., 2020).

Although two analytical methods were used in the analysis of the dataset, only GT coding strategies such as open coding, axial coding, and selective coding (Charmaz, 2014; Corbin and Strauss, 2015) were used to analyse transcribed member checking meetings. After analysing the transcription of the first member checking meeting conducted in the medium SES neighbourhood, researchers carefully prepared the next meetings regarding the initial results. Researchers used grounded theory coding strategies (Charmaz, 2014; Corbin and Strauss, 2015) because of their flexibility and the importance given to social actors in microsocial contexts. The present research was outlined using a constructivist approach, as the participants’ social realities were constructed using data gathered from FGs and SSIs and co-constructed throughout member checking meetings. The constructivist approach was also important because during peer-debriefings researchers had to reflect on how their different theoretical perspectives contributed to shaping the member checking process and the results. The analytical process was managed using ATLAS.ti-8 software.

Credibility and validity of data

The project in which this research was carried out was outlined as a type 2 qualitative case study (Yin, 2014) where accurate interpretations of the experiences and perceptions of participants are needed. To ensure credibility in the qualitative case study, researchers have used different techniques. As we previously pointed out, researchers used two different methods of qualitative data analysis such as grounded theory and interpretative phenomenological analysis. These two qualitative analytical methods were triangulated because of the nature of the research (i.e. giving importance to the context, practices, behaviours, and social processes), and because using only one method would be insufficient to reveal the richness of the data from SSIs and FGs (Morse, 1991; Simons et al., 2008; Wilson and Hutchinson, 1991). Due to the length of the data collection (from 2016 to 2019), researchers could have achieved prolonged engagement. Prolonged engagement enhanced the credibility of the findings and allowed researchers to understand the participants’ surrounding realities (i.e. neighbourhoods, city) (Houghton et al., 2013).

As we were aware of the positivist roots of the member checking strategy (Lincoln and Guba, 1985), we decided to reflexively shift from positivist towards a constructivist approach. In peer-debriefings during data analysis (i.e. FGs and SSIs), and before and after conducting member checking meetings we reflected on co-constructing findings and the process of co-construction. During peer-debriefing, findings were presented to other co-authors who enriched the analysis with critical comments influenced by their different theoretical perspectives (public health and social science) and their expertise in qualitative analysis (Flick, 2018, 2019). We also reflect on avoiding comparisons that might lead to stigmatization while co-constructing findings through interaction between researchers and the member checking attendees.

Researchers aimed to generate both rich description of the process and outcomes of the member checking (i.e. participants’ experiences and perception of interpretations made by researchers), and conceptual knowledge (i.e. accuracy of data interpretations, pros, and cons of conducting member checking) that might inform future research.

Researchers believe that allowing other residents that did not participate in the research process to participate in the member checking meetings would be helpful to enhance transactional validity, which is a desirable aim in case studies. Furthermore, the other residents could hold valuable ontological knowledge about their surrounding reality, and they would not be influenced by having participated in the previous data collection process conducted between 2016 and 2019 (i.e. they would not be influenced by unequal power relations between researchers and study participants).

Ethical issues

Study participants signed an informed consent document that guaranteed their anonymity and confidentiality. In the informed consent they were also informed about the member checking meetings that would be conducted once the data collection process was completed. Alcalá University’s Bioethics Committee approved the study protocol (CEI/HU/2017/18).

Considering the number of study participants, the attendance at the member checking meetings was substantially lower than expected by researchers in the low and medium SES neighbourhoods. At the beginning of the member checking process, researchers were optimistic regarding involvement in these meetings due to the interest showed by most of study participants in follow-up phone calls (i.e. learning about the research findings and participating in these meetings). Fourteen study participants and five residents attended the first member checking meeting conducted in the medium SES neighbourhood. Twenty-nine study participants and three residents attended the second member checking meeting conducted in the high SES neighbourhood. Finally, 28 study participants and two residents attended the member checking meeting conducted in the low SES neighbourhood.

Researchers avoided comparing one neighbourhood to another, in order to not stigmatize residents living in the least affluent neighbourhood. Furthermore, they also avoided stigmatizing health-related behaviours such as drinking alcohol or smoking by not comparing smokers and drinkers with non-drinkers and non-smokers.

As highlighted by Buchbinder (2011), participants in member checking meetings would just agree with data interpretations due to existing power dynamics between participants and researchers. However, in the present work researchers considered that the way they conducted the member checking (i.e. giving the attendees the opportunity to actively take part in the research process and to preserve their voices in the final findings) diminished the effect of the power dynamics in qualitative research. Researchers adopted a naïve position in member checking meetings when asking participants for their opinion about data interpretations and did not judge them. Furthermore, researchers considered the member checking attendees to be experts on their surrounding reality because they hold valuable ontological knowledge, which could help improve how researchers interpreted data from FGs and SSIs.

Findings

One core category emerged from the data, and it was named ‘Co-constructing findings and side effects of the interaction between researchers and participants in member checking’. Three main categories emerged and underpinned the core category: (1) appraising interpretations by means of underscore and disagreement; (2) participants internalized comparisons between neighbourhoods as a side effect of co-constructing results; (3) limits in the applicability of qualitative research findings.

Appraising interpretations by means of underscore and disagreement

Study participants and member checking attendees were able to appraise the accuracy of interpretations made from the data gathered in SSIs and FGs throughout the member checking meetings. They also contributed to co-constructing the final findings of the qualitative case study where this research is nested throughout interaction with researchers during the member checking meetings. Interactions in member checking meetings were coded and classified in two categories: underscore and disagreement. Finally, the present subcategory emerged from the latter two preliminary categories. While underscoring summarized findings contributed to strengthening interpretations made from the data, disagreement contributed to enriching, complementing, and co-constructing study findings. Finally, researchers combined the latter categories and in the present subcategory.

Attendees at the member checking meetings conducted in the medium and low SES neighbourhoods underscored most of the same summarized findings regarding two of the health-related behaviours studied: physical activity and smoking. As showed in the verbatims below, interpretations made by researchers could be in line with the social realities of study participants. Furthermore, attendees also showed gratitude for conducting the member checking and showing them the results obtained in the research.

I just wanted to thank you for conducting the study in this neighbourhood and because the way you presented the things that happen in this neighbourhood was friendly and we were able to understand it. Besides, you said things that really happen here, like the fact that children aren’t allowed to play football where they used to. It’s very sad how things have changed for children the last few years. [Member checking meeting conducted in the medium SES neighbourhood]

I agree with your results regarding how non-smokers confronted smokers in some places. At first, I thought that smoking at bus stops was not allowed but I think that we all must learn about the smoke-free law instead of confronting smokers. [Member checking meeting conducted in the low SES neighbourhood]

Most of the disagreements with the study findings emerged in the member checking meeting conducted in the high SES neighbourhood. Some were coded as structural disagreements because they referred to the data collection carried out in the qualitative project. Member checking attendees in this neighbourhood wondered why there were fewer FGs in their neighbourhood than in the other two neighbourhoods. Researchers explained in the member checking meeting that they conducted fewer FGs in the high SES neighbourhood than in the other two neighbourhoods because the social profile of the residents was more homogeneous (mainly high-income residents and highly skilled workers), and they believed discourse saturation was reached sooner.

I didn’t participate in the groups (talking about FGs), but I didn’t hear about them. I wondered why only four groups were held here, while in the other neighbourhood there were more. [Member checking meeting conducted in the high SES neighbourhood]

Other disagreements were coded as thematic because they referred to the four health-related behaviours studied. Researchers found thematic disagreement in the high SES neighbourhood and in the low SES neighbourhood. Member checking attendees in the high SES neighbourhood showed disagreement with results regarding three of the health-related behaviours studied: physical activity, diet, and smoking. Regarding physical activity, the member checking attendees reported that researchers did not mention that the neighbourhood did not have bicycle path. Regarding diet, the member checking attendees claimed that researchers did not mention community gardens, which were perceived as a way to do physical activity (e.g. vegetable gardening) and to strengthen social bonds among older residents. Regarding smoking, the member checking attendees commented on the lack of public ashtrays, which in turn resulted in litter in the form of cigarette butts.

-There is one thing that you put here that I don’t agree with, about bike riding. I bike ride a lot but it’s very difficult to use a bike in this neighbourhood because there are no cycling paths, so it’s dangerous.

-You also didn’t mention community gardens. There are two of them very near here and they were a place where you can socialize and take part in physical activity.

-Regarding cigarette butts, I think that you cannot force smokers to use ashtrays when there are not enough ashtrays in the streets. [Member checking meeting conducted in the high SES neighbourhood]

Meanwhile, attendees to the member checking meeting conducted in the low SES neighbourhood considered that our results were not able to show the most recent changes in the neighbourhood.

Since you interviewed me there has been some changes in the neighbourhood: the entrances to the parks and the basketball courts were improved. You didn’t mention those changes. [Member checking meeting conducted in the Low SES neighbourhood]

Researchers theorized about why some topics did not come up in the SSIs or FGs: (1) these topics were not important enough to be mentioned by study participants at the time researchers conducted data collection; (2) study participants did not know about it; and (3) questions in the scripts were not clearly written or asked by researchers. Otherwise, phenomena that did not emerged in FGs nor SSIs but did emerge in the member checking meeting could indicate that the discourse was not fully saturated in this neighbourhood, and therefore it would have been better to conduct more FGs in the high SES neighbourhood. However, due to a lack of budget, researchers could not afford to conduct new FGs in the high SES neighbourhood. Another possible explanation could be that changes in the neighbourhood were implemented after conducting the SSIs and FGs.

Participants internalizing comparisons between neighbourhoods as a side effect of co-constructing results

Researchers carefully prepared each member checking meetings by analysing transcriptions of the previous member checking conducted. They avoided comparing the SES of the neighbourhoods and negative health-related behaviours such as smoking and drinking alcohol. They avoided such comparisons to prevent misinterpretations and internalizations of negative findings, which in turn could increase the stigmatization of socioeconomically disadvantaged residents and residents who had negative health-related behaviours (e.g. smoking, drinking alcohol). However, after presenting summarized results in member checking meetings several unintended side effects emerged. Researchers coded and classified unintended side effects as unintended comparisons, intended comparisons, and stigmatization. Although only unintended comparisons and stigmatization can be considered as negative side effects, intended comparisons were important regarding the aim of the member checking strategy.

In the member checking meeting conducted in the medium SES neighbourhood there were unintended comparisons between residents in the neighbourhoods. Comparisons were based on negative perceptions of those residents who according to study participants had negative health-related behaviours, which sometimes were Latin American residents.

Sure, a lot of young people are making noise and screaming like they are being murdered (figuratively) and they don’t let people sleep. Which is okay at 11 or 12, but it’s still happening at 2 in the morning. But they point at us as immigrants, the “immigrants”, but not all the Latino residents are there making noise. [Member checking meeting conducted in the Medium SES neighbourhood]

In the high SES neighbourhood researchers found intended comparisons as member checking attendees asked for comparisons between results obtained in all three studied neighbourhoods. Researchers believe that these comparisons might be triggered because they would like to be considered as the healthiest neighbourhood in the study. Furthermore, this neighbourhood was the wealthiest of the three in the study.

I just wanted to know if there are differences between neighbourhoods. For instance, I wonder if differences in manners are noteworthy or whatever (talking about health-related behaviours). [Member checking meeting conducted in the High SES neighbourhood]

Finally, in the member checking meeting conducted in the low SES neighbourhood, stigmatization of smokers emerged. A smoker who attended tried to explain the conflict between smokers and non-smokers in outdoor spaces where smoking is allowed. The smoker considered that he/she was a considerate smoker as he/she repeated what participants said in FGs about the conflict between smokers and non-smokers at bus stops. The fact that the participant tried to explain his/her perspective might be because researchers did not do enough to avoid the stigmatization of negative health-related behaviours when presenting summarized study findings in the member checking meeting conducted in this neighbourhood. Furthermore, sometimes it could be difficult to deal with population groups such as smokers who were already feeling stigmatization.

I would like to say that I am smoker and I said so in the meeting (he/she was talking about FGs) were I participated in: “I really like that smoking was banned in bars. Yes, people have said things to me when I was smoking at bus stops. I respect them and I just move away”. [Member checking meeting conducted in the Low SES neighbourhood]

Limits in the applicability of qualitative research findings

Researchers found that some attendees wanted to know if the study findings could help improve their surroundings (i.e. neighbourhoods, city). The expectancy of the member checking attendees regarding improvements varied between neighbourhoods. In the high SES neighbourhood, some attendees wondered how study findings could improve not only their neighbourhood, but the whole city of Madrid.

So what are your conclusions? Because in the end that’s what will influence people who live in the city (Madrid). [Member checking meeting conducted in the high SES neighbourhood]

The member checking attendees from the high and low SES neighbourhoods asked for concrete results, which seemed to be more in line with quantitative rather than qualitative inquiry. This might be because they did not properly understand the introduction about qualitative inquiry that researchers made at the beginning of the member checking meeting, or because they did not properly understand the summarized study findings. Researchers believe that this could be because findings were presented as descriptive summaries.

I am interested in your opinion (talking to the researchers), because you conduct this study and you must know all the neighbourhoods in detail. [Member checking meeting conducted in the high SES neighbourhood]

I wonder how healthy our neighbourhood is. [Member checking meeting conducted in the low SES neighbourhood]

The member checking attendees from the low SES neighbourhood also complained about how policymakers have ignored them in the past. This complaint did not emerge in the member checking meeting conducted in the other neighbourhoods. Following the perception of being ignored by policymakers, the member checking attendees from this neighbourhood also showed mistrust on how study findings could contribute to improving their health status and their neighbourhood.

-It would be interesting to know if this study is worth anything to us because I still don’t know if you (researchers) would take these results to the city council and if they would improve anything. . . I’m not sure. [Member checking meeting conducted in the low SES neighbourhood]

In the medium SES neighbourhood, the member checking attendees also wondered how study findings could improve their neighbourhood. During member checking meeting, attendees became aware of some negative health-related behaviours that might be worsening their collective health. In turn they would try to change those behaviours to improve their health status. At this point, member checking in this neighbourhood seemed to be more in line with transformational rather than transactional validity.

-And now what? What are you going to do with the results? -Maybe the results could be used to organize cultural activities in the park, which is an open and free space. Someone could do something about the people who are in the squares. Organizing something so that we can learn to respect ourselves, raising the awareness of the drinkers and pot smokers. . . [Member checking meeting conducted in the Medium SES neighbourhood]

Discussion

This work described the process and outcomes resulting from conducting member checking as part of a type 2 qualitative case study conducted in three different SES neighbourhoods. In a case study, it is essential for the contextual reality of the participants to be faithfully represented. In this study, a constructivist approach and reflexivity played key roles in obtaining an accurate vision of the neighbourhoods studied. Attendees at member checking meetings appraised research findings by means of underscore and disagreement and by interacting with researchers during the meetings. Member checking also contributed to learning about discourse saturation regarding data collection in the qualitative project in which this research was carried out. During our member checking meetings questions were raised about how the findings could improve the surroundings of the participants (e.g. their neighbourhoods, their city). Member checking might also be considered a useful tool for the final stage in data collection in qualitative case studies with different units of analysis such as neighbourhoods. Finally, including the study participants in this process could also be seen as a way to show gratitude to them for having dedicated time to the data collection process, since published scientific journals are not usually read by people outside academia. However, it should also be kept in mind that member checking could lead to some unintended side effects that should be reflexively addressed by the research team before and after conducting this strategy.

Reflexivity played a key role during the member checking process and the construction of the final findings. It is important to highlight that these qualitative findings were co-constructed through disagreement and criticism rather than endorsement. However, endorsement is also useful when appraising data interpretations because in qualitative case studies researchers must ensure that they are correctly interpreting the surrounding reality, experiences, and perceptions of the participants. Giving attendees from the medium SES neighbourhood the opportunity to participate in constructing and appraising qualitative research findings contributed in a certain way to empowering them to change their surrounding reality. Conducting member checking also might lead to questions about the data collection process, as occurred in the high SES neighbourhood where attendees commented on phenomena that did not emerge in SSIs or FGs. Although researchers reflexively planned member checking meetings avoiding comparisons between neighbourhoods or negative health-related behaviours, some attendees wanted to know the results based on such comparisons and other attendees internalized findings regarding negative health-related behaviours. Member checking raised doubts among attendees who wondered how findings could improve their surrounding reality (i.e. neighbourhood, city), especially in the least affluent neighbourhood where they also expressed mistrust of policymakers.

In the following paragraphs, the main criticisms of member checking are discussed and reflexively compared with our own experience and outcomes. One criticism is that member checking relies excessively on how participants are engaged in the research process, and sometimes researchers can overestimate the interest that participants have in the research project (Carlson, 2010). This might explain why the attendance at the member checking meetings conducted in the low and medium SES neighbourhoods was low compared to the number of study participants recruited during the data collection process. Participants in member checking might not be able to recognize their experiences or even themselves in abstract theoretical interpretations made by researchers (Morse, 2015). Although we considered that the latter criticism might be the most recurrent and strongest, in the present research the member checking attendees were able to recognize themselves in data interpretations presented as summarized study findings. Several voices criticized the member checking strategy because there is no independent or completely reliable access to reality (Green and Thorogood, 2004; Hammersley, 1992). Although we totally agree with interpretivist approaches that recognized the existence of multiple realities, within a constructivist framework an objective reality is constructed and in the present work each of the neighbourhoods had its own reality.

In recent years, member checking has been shown to achieve holistic validity, comprised by both transactional and transformational validity, which are not necessarily opposed aims (Cho and Trent, 2006; Koelsch, 2013). In line with holistic validity, other authors have stated that when research is aimed at understanding participants’ experiences and empowering them, they will be able to recognize themselves and their experiences in data interpretations (Birt et al., 2016). Although the aim of our research was more in line with transactional validity, we cannot ignore that holistic validity is a more comprehensive validity strategy, and member checking attendees at the medium SES neighbourhood did display some empowerment to change negative health-related behaviour within their neighbourhood to improve collective health status. While member checking attendees in the low SES neighbourhood mistrusted the implementation of changes in their neighbourhood after conducting the research, attendees in the high SES neighbourhood had high expectations about such changes. Researchers conducting member checking in future research should be very clear explaining if their results could contribute to improve the realities of the participants.

Our experience conducting member checking highlighted the following issues that should be considered in future research. Memory recall (Kvale, 2006), time between data collection (Morse et al., 2002), and recent changes in the environment could affect the member checking process. Based in our own experience (e.g. community gardens in the high SES neighbourhood), we recommend conduct member checking as soon as possible after finishing each data collection method. Reducing the time between data collection and member checking could help to improve the member checking results. More research is needed on how member checking process affects participants. Our results showed that participate in member checking process affected attendees. Researchers conducting future research must reflexively address all the possible ways in which member checking can affect attendees before and after participating in the member checking process. Researchers might not be paying enough attention to how participants experienced the process and to how they feel after participating in member checking, which may result in disempowerment of participants because they internalize the data interpretations (Hallet, 2013). As Bradshaw (2001) recommended, member checking must be approached with caution, due to the power relations that exist in every research process. Based in our own experience, and although we carefully considered during peer-debriefings how the member checking attendees could feel during and after participating in the meetings, there were some unintended side effects such as the internalization of negative findings.

We consider that conducting one member checking meeting per neighbourhood is a strength of our study. We also feel that allowing other residents who had not participated in the data collection process to take part in the member checking meetings is a strength, as they contributed to evaluating and co-constructing the final findings without being influenced by prior participation in the SSIs and FGs. Allowing member checking to be open meetings where all the attendees can freely speak, discuss, and critique research findings and the research process is another strong point of our approach, as study participants became an active part in the research process and not merely research subjects.

We consider the low attendance at the member checking meeting in the low and medium SES neighbourhoods to be a limitation, especially considering the high number of participants recruited in the fieldwork of the qualitative case study where this research is nested. Member checking resulted in unintended side effects, such as comparisons between different SES neighbourhoods and internalization of interpretations of negative health-related behaviours. We believe that comparisons between neighbourhoods could be potentially harmful to disadvantaged population groups and those who had negative health-related behaviours because they might suffer double stigmatization (e.g. being a smoker and living in a disadvantaged SES neighbourhood). Researchers must deal with power relations when conducting member checking, especially if they are studying disadvantaged population groups. Furthermore, member checking also raised questions among attendees regarding if the findings would be used to improve their surrounding reality. Member checking was useful to understand that discourse saturation may not have been reached in the high SES neighbourhood and that conducting more FGs during the fieldwork of the qualitative case study would have been desirable.

Conclusions

Member checking can be useful in evaluating data interpretations in qualitative case studies studying health-related behaviours in urban contexts. Presenting summarized findings in member checking meetings in this research has shown the key role that participants might have in appraising and co-constructing qualitative research findings through interaction with researchers. More research is needed to advance knowledge about the unintended effects that member checking might have on study participants and the usefulness of member checking to strengthen qualitative health research.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a predoctoral fellowship (reference BES-2017-079911) granted by the Spanish Ministry of Science and Innovation [Ministerio de Ciencia e Innovación], by a “Margarita Salas” postdoctoral fellowship granted by the Spanish Ministry of Universities [Ministerio de Universidades] and funded by the Next Generation European Union funds, and by the Spanish Ministry of Economy and Competitiveness (reference CSO2016-77257-P) [Ministerio de Economía y Competitividad] funded project Urban environment and health: qualitative approach in the Heart Healthy Hoods study.

ORCID iD

Ignacio de Loyola González-Salgado

Author biographies

Ignacio de Loyola González-Salgado is a postdoctoral research fellow at the University of Salamanca and a visiting scholar at the Urban Health Collaborative, Drexel University. His main fields of research are health-related behaviors, urban health, and qualitative methods.

Jesús Rivera-Navarro is a professor at the Sociology and Communication Department in the University of Salamanca. His main fields of research are health-related quality of life in older adults and caregivers, maltreatment of people with dementia, and health inequalities.

Marta Gutiérrez-Sastre is an associate profesor of Sociology at the Sociology and Communication Department in the University of Salamanca. Her main fields of research are health, social, and gender inequalities.

Paloma Conde is a predoctoral researcher at the University of Alcalá. Her main fields of research are urban health, qualitative methods, and food environment.

Manuel Franco is an adjunct professor at the Johns Hopkins University and an associate professor of Epidemiology at the University of Alcalá. His main fields of research are epidemiology, cardiovascular disease, and food environment.

References

Barbour

(2018) Quality of data collection. In: Flick

(ed.) The SAGE Handbook of Qualitative Data Collection. London: SAGE Publications, pp.217–230.

Biering-Sørensen

Alexander

Burns

, et al. (2011) Recommendations for translation and reliability testing of international spinal cord injury data sets. Spinal Cord 49(3): 357–360.

Birt

Scott

Cavers

, et al. (2016) Member checking: A tool to enhance trustworthiness or merely a nod to validation? Qualitative Health Research 26(13): 1802–1811.

Bradshaw

(2001) Contracts and member checks in qualitative research in human geography: Reason for caution? Area 33(2): 202–211.

Brear

(2019) Process and outcomes of a recursive, dialogic member checking approach: A project ethnography. Qualitative Health Research 29(7): 944–957.

Buchbinder

(2011) Beyond checking. Qualitative Social Work: Research and Practice 10(1): 106–122.

Caretta

(2016) Member checking: A feminist participatory analysis of the use of preliminary results pamphlets in cross-cultural, cross-language research. Qualitative Research 16(3): 305–318.

Carlson

(2010) Avoiding traps in member checking. The Qualitative Report 15(5): 1102–1113

Charmaz

(2014) Constructing Grounded Theory, 2nd edn. London: SAGE.

10.

Charmaz

(2017) The power of constructivist grounded theory for critical inquiry. Qualitative Inquiry 23(1): 33–45.

11.

Cho

Trent

(2006) Validity in qualitative research revisited. Qualitative Research 6(3): 319–340.

12.

Corbin

Strauss

(2015) Basics of Qualitative Research, 4th edn. Thousand Oaks, CA: SAGE.

13.

Creswell

(2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 2nd edn. London: Pearson.

14.

Cuthbertson

Robb

Blair

(2020) Theory and application of research principles and philosophical underpinning for a study utilising interpretative phenomenological analysis. Radiography 26(2): e94–e102.

15.

Doyle

(2007) Member checking with older women: A framework for negotiating meaning. Health Care for Women International 28(10): 888–908.

16.

Flick

(2018) Triangulation in data collection. In: Flick

(ed.) The SAGE Handbook of Qualitative Data Collection. London: SAGE Publications, pp.527–545.

17.

Flick

(2019) From intuition to reflexive construction: Research design and triangulation in grounded theory research. In: Bryant

Charmaz

(eds) The SAGE Handbook of Current Developments in Grounded Theory. London: SAGE Publications, pp.125–145.

18.

González-Salgado I de

Rivera-Navarro

Sureda

, et al. (2020) Qualitative examination of the perceived effects of a comprehensive smoke-free law according to neighborhood socioeconomic status in a large city. SSM - Population Health 11: 100597.

19.

Green

Thorogood

(2004) Analysing qualitative data. In: Gilbert

(ed.) Qualitative Methods for Health Research. London: SAGE Publications, pp.173–200.

20.

Guba

Lincoln

(1989) Fourth Generation Evaluation. London: SAGE.

21.

Hallet

(2013) Dangers of member checking. In: Midgley

Danaher

Baguley

(eds) The Role of Participants in Education Research: Ethics, Epistemologies. London: Routledge, pp.29–39.

22.

Hammersley

(1992) What’s Wrong With Ethnography? The Mith of Theoretical Descriptional. London: Routledge.

23.

Harvey

(2015) Beyond member-checking: A dialogic approach to the research interview. International Journal of Research & Method in Education 38(1): 23–38.

24.

Hermann

Rogers

Ehrenthal

(2012) Women’s perceptions of centering pregnancy: A focus group study. American Journal of Maternal/Child Nursing 37(1): 19–26.

25.

Houghton

Casey

Shaw

, et al. (2013) Rigour in qualitative case-study research. Nurse Researcher 20(4): 12–17.

26.

Klinger

(2005) Occupational adaptation: Perspectives of people with traumatic brain injury. Journal of Occupational Science 12(1): 9–16.

27.

Koelsch

(2013) Reconceptualizing the Member Check Interview. International Journal of Qualitative Methods 12(1): 168–179.

28.

Kvale

(2006) Dominance through interviews and dialogues. Qualitative Inquiry 12(3): 480–500.

29.

Lincoln

Guba

(1985) Naturalistic Inquiry. London: SAGE Publications.

30.

Mabry

(1998) Case study methods. In: Walberg

Reynolds

(eds) Evaluation Research for Educational Productivity. Bingley, West Yorkshire: Emerald Publishing, pp.155–170.

31.

Mertens

(2015) Research and Evaluation in Education and Psychology, 4th edn. London: SAGE.

32.

Mertens

(2018) Ethics of qualitative data collection. In: Flick

(ed.) The SAGE Handbook of Qualitative Data Collection. London: SAGE, pp.33–48.

33.

Miles

Huberman

(1994) An Expanded Sourcebook: Qualitative Data Analysis, 2nd edn. London: SAGE.

34.

Morse

(1991) Approaches to qualitative-quantitative methodological triangulation. Nursing Research 40(2): 120–123.

35.

Morse

(2015) Critical analysis of strategies for determining rigor in qualitative inquiry. Qualitative Health Research 25(9): 1212–1222.

36.

Morse

Barrett

Mayan

, et al. (2002) Verification Strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods 1(2): 13–22.

37.

O’Brien

Harris

Beckman

, et al. (2014) Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine 89(9): 1245–1251.

38.

Patnaik

(2013) Reflexivity: Situating the researcher in qualitative research. Humanities and Social Science Studies 2(2): 98–106.

39.

Patton

(2014) Purposeful sampling and case selection: Overview of strategies and options. In: Patton

(ed.) Qualitative Research & Evaluation Methods. Integrating Theory and Practice. London: SAGE, pp.401–410.

40.

Pazokian

Zagheri Tafreshi

Rassouli

(2014) Iranian nurses’ perspectives on factors influencing medication errors. International Nursing Review 61(2): 246–254.

41.

Rivera-Navarro

Bonilla

Gullón

, et al. (2021a) Can we improve our neighbourhoods to be more physically active? Residents’ perceptions from a qualitative urban health inequalities study. Health & Place 102658.

42.

Rivera-Navarro

Conde

Díez

, et al. (2021b) Urban environment and dietary behaviours as perceived by residents living in socioeconomically diverse neighbourhoods: A qualitative study in a Mediterranean context. Appetite 157: 104983.

43.

Rivera Navarro

Franco Tejero

Conde Espejo

, et al. (2019) Understanding Urban Health Inequalities: Methods and Design of the Heart Health Hoods Qualitative Project. Gaceta Sanitaria 33(6): 517–522.

44.

Rolfe

(2006) Validity, trustworthiness, and rigour: Quality and the idea of qualitative research. Journal of Advanced Nursing 53(3): 304–310.

45.

Roller

Lavrakas

(2015) Applied Qualitative Research Design. New York, NY: The Guilford Press.

46.

Sandelowski

(1986) The problem of rigor in qualitative research. Advances in Nursing Science 8(3): 27–37.

47.

Sandelowski

Barroso

(2002) Reading qualitative studies. International Journal of Qualitative Methods 1(1): 74–108.

48.

Schreier

(2018) Sampling and generalization. In: Flick

(ed.) The SAGE Handbook of Qualitative Data Collection. Thousand Oaks, CA: SAGE, pp.84–98.

49.

Silverman

(2017) Doing Qualitative Research, 5th edn. London: SAGE.

50.

Simons

Lathlean

Squire

(2008) Shifting the focus: Sequential methods of analysis with qualitative data. Qualitative Health Research 18(1): 120–132.

51.

Stringer

(2003) Giving voice: Interpretive and qualitative analysis. In: Rutherford

(ed.) Action Research in Education. Upper Saddle River, NJ: Prentice Hall, pp.96–124.

52.

Taylor

GAJ

Blake

(2015) Key informant interviews and focus groups. In: De Chesnay

(ed.) Nursing Research Using Data Analysis. Qualitative Designs and Methods in Nursing. New York, NY: Springer, pp.153–166.

53.

Thomas

(2017) Feedback from research participants: Are member checks useful in qualitative research? Qualitative Research in Psychology 14(1): 23–41.

54.

Tong

Sainsbury

Craig

(2007) Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal Quality Health Care 19(6): 349–357.

55.

Torrance

(2012) Triangulation, respondent validation, and democratic participation in mixed methods research. Journal of Mixed Methods Research 6(2): 111–123.

56.

Wainwright

Shepard

Harman

, et al. (2010) Novice and experienced physical therapist clinicians: A comparison of how reflection is used to inform the clinical decision-making process. Physical Therapy 90(1): 75–88.

57.

Wilson

Hutchinson

(1991) Triangulation of qualitative methods: Heideggerian hermeneutics and grounded theory. Qualitative Health Research 1(2): 263–276.

58.

Yin

(2014) Case Study Research: Design and Methods, 5th edn. London: SAGE.