Abstract
Methodologists recommend member checking (MC) to increase the credibility of results from qualitative studies. In this procedure, the researcher’s categories, hypotheses and interpretations can be verified or corrected by respondents, and the latter can add information. Various methods are available for conducting this procedure. However, in addition to methodological issues, MC also raises ethical questions. In this article, we investigate which member check method (MCM) is most appropriate for a particular study from an ethical perspective. Based on a literature review about ethical issues in different MCMs and ethical principles contained in guidelines for research ethics, we develop seven decision criteria that can facilitate an ethically reflected decision for a particular MCM. Accordingly, we consider the principles of beneficence, justice, scientific quality, informed consent and avoiding harm, which are widely accepted in research ethics. In addition, we recommend that participants’ wishes and concerns regarding the MC be inquired about and considered.
Keywords
Introduction
Classical quality criteria of quantitative social research, such as validity, objectivity and reliability, cannot be easily applied to qualitative research because of its different epistemological foundations (Guba & Lincoln, 1989). This insight has caused a lively discussion on alternative quality criteria for qualitative research (Meyer, 2018). Guba and Lincoln (1989), for example, developed five criteria for judging the adequacy of qualitative inquiry: trustworthiness, credibility, transferability, dependability, and confirmability. The authors suggest several methods to increase the credibility of an investigation, one of which is a member check (Flick, 2006; Guba & Lincoln, 1989), whereby the respondents are understood as members of the group investigated. The basic idea of this procedure is that the categories, hypotheses, and interpretations of the researchers should be verified by the respondents, giving the latter the opportunity to correct any errors of interpretation but also provide additional information. Accordingly, respondents are reminded of their answers and confirm that the researcher has presented their opinions correctly (Guba & Lincoln, 1989) and the researchers have the opportunity to further develop their interpretation (Charmaz, 2010).
The literature reveals five different methods for MC, requiring varying degrees of verbal communication with participants. While returning the transcripts of the interviews (Torrance, 2012), synthesized member checking (Birt et al., 2016), and providing the research report or a draft for publication (Brewis, 2014) require little personal interaction with participants, researchers actively seek an ongoing dialogue with participants in a member check interview (Koelsch, 2013) and a member check focus group (Goldblatt et al., 2011).
The effort required to perform these procedures is considerable; therefore, the question arises why this effort might be worthwhile. One reason is that unlike quantitative researchers, qualitative researchers follow a constructivist and not a positivist paradigm (Guba & Lincoln, 1989). Erdmann et al. (2022), for example, consider their perceptions in the research process as cognitive constructions which do not reproduce reality exactly. Constructivists assume that the researcher’s perceptions probably blend out other possible perceptions and recognize that social reality is always constructed interactively (Pörksen, 2011).
In addition, the goal of qualitative research is not to explain but understand and interpret the intentions and actions of the participants under study and, thus, resembles the method of hermeneutics. Understanding is defined here as a recognition of the meaning of utterances and actions. But every utterance can also be understood differently, which might result in multiple interpretations. According to Brühl (2021), social science hermeneutics recognize an interpretation as true when it is confirmed by the respondent(s). As a result, divergent interpretations should be corrected. However, Morse (2015) argues that the participants lack any theoretical background and overview of the data, which means they cannot necessarily verify the researchers’ analysis. Participants might suggest changes in interpretation, which would then put researchers in an awkward position. For this reason, Morse (2015) advises against using MC. Other authors problematize an objective truth, because different truths from different people can exist side by side (Forbat & Henderson, 2005). Cho and Trent (2006) emphasize that if meanings are social constructions, then multiple perspectives on a subject yield multiple meanings. In this sense, MC cannot aim to verify the researcher’s interpretation by the participants but it can confirm that the findings and interpretations are meaningful to the participants, and enable researchers to refine their interpretations and enhance the way in which the researchers self-reflect on the multiple dimensions in which their research is conducted (Charmaz, 2010; Cho & Trent, 2006). Consequently, depending on the epistemological foundation of utilizing a MCP, difficulties may or may not arise when the interpretations of the participants and the researchers are contradictory or incongruent (Varpio et al., 2017).
In addition to methodological benefits and challenges, several authors (Birt et al., 2016; Goldblatt et al., 2011; Slettebo, 2020) have mentioned ethical issues of the MC procedure, which need to be considered when choosing and applying a particular method.
In order to illustrate one of these issues to readers, we start by reporting an example of a MC focus group by Erdmann et al. (2022) in which an ethical conflict arose.
Example of a Focus Group Member Check
The MCP took place as part of a bioethical research project on the potentials and limitations of precision medicine from the perspective of researchers, healthcare professionals and patients with chronic inflammatory diseases. The main goal of this project was to study the impact of evolving precision medicine on patients’ lives. Expert and patient interviews were conducted for this purpose. In one of the subprojects, 17 experts were interviewed and invited to participate in a focus group MC after their data had been analyzed. Prior to the meeting, they were provided with the results in writing in the form of interpretative theses and supporting interview quotes for each thesis.
During the meeting, one of the researchers presented the central findings of the study with an emphasis on the findings which contradict each other, in order to clarify the accuracy of the interpretations and possible misunderstandings. The experts provided additional, important information regarding these contradictions in the data during the focus group MC (Erdmann et al., 2022). However, only 4 of the 17 experts participated, which caused one researcher to doubt whether this meeting achieved the goal of the MC, namely, increasing credibility by reducing the researcher bias. As a result, the researcher reflected on the experts’ possible reasons for not participating: • Timing unsuitable (5.00–6.30 p.m.) • Too much of their own work to do • Not interested, since the results do not directly affect experts’ own research • Need for information already satisfied by written presentation of results • Experts do not want to give up confidentiality
The last possible reason highlights an ethical issue, namely, that by participating in the focus group MC, experts, although pseudonymized in the interview quotes, would have to disclose their participation in the study, which some may have wanted to avoid in order not to have to present their opinions to colleagues. The ethical conflict for the researcher was, on the one hand, to request the experts to disclose their participation in the study although confidentiality and anonymity is an important principle of research ethics, and, on the other hand, to ensure the interpretative validity of the results. The example shows that the principles of confidentiality and anonymity are difficult to implement in a focus group MC and raises the fundamental question of which MCM is appropriate for different research projects and participants, taking all the principles of research ethics into account.
Thus, the central question of this article is how different MCMs can be evaluated from a research ethical perspective. While this question is also relevant when choosing other research methods, the special point of MCP is that this method presents researchers’ interpretations, interview quotes or observation excerpts in writing and, in some MCP, also to other participants. The participants can decide what they reveal about themselves in other data collection methods, however, in a MCP, they are confronted with the experience not only described by them but also interpreted by the researchers. This confrontation can be, for example, painful or tainted with shame if it occurs in the presence of others (e.g., focus group) and can, therefore, cause harm to the participants. Consequently, we argue that several characteristics of the research project (i.e., participants, research topic, researchers, relationships, resources, methodological approach, results of the study) need to be considered for the evaluation of the appropriateness of a MCM concerning their accordance with or violation of research ethical principles during a MCP. In this article, we develop these considerations as decision criteria for the ethically reflected choice of a MCM with the purpose of making them available to other researchers for discussion, further development and application.
Methods
We first introduce research ethical principles, which we derive from research ethical guidelines and which insofar are directive for researchers. We then provide an overview of the different MCMs and highlight ethical problems that have already been discussed in the literature to support our argument and present different ethical issues that arise with the various MCMs. We obtain a comprehensive picture of the ethical challenges by utilizing this analysis. Based on the premise that ethical principles mentioned in the guidelines should be taken into account when choosing a MCM, we finally recommend considering seven decision criteria and related issues to discover which can assist researchers in making their choice in accordance with ethical principles.
Principles of Research Ethics to Inform the Choice of an MCM
Research ethics formulates basic singular principles of morally appropriate action for empirical research practice, that more or less bindingly define how the relationship between researchers and those involved in research should be structured (Hopf, 2004). Various national and international guidelines exist for research ethics as well as within various disciplines, and, in most cases, they are committed to principlism, an ethical theory that has significantly influenced research ethics, for example, the Belmont Report published in 1979. The respect for the autonomy of a person and the principles of beneficence and justice were already mentioned in this report (Office for the Human Research Protections, 2022). However, as Beauchamp and Childress (2001) have shown, principlism can be traced back to earlier ethical theories. Contemporary interpretations of the principle to respect autonomy, for example, were influenced by Kant and Mill. Kant argued that all human beings have unconditional worth and that they should not be used as a means to achieve the goals of others without taking their own goals into account. Mill maintained that an autonomous expression should not be interfered with, but should be actively strengthened. The principle of beneficence played a central role in Utilitarianism (Beauchamp & Childress, 2001), in which this action is the ethically best choice that leads to a maximization of benefit for all those affected (Mill, 1985). The principle of justice also has its foundations in ethical theories, for example, put forward by Rawls (Beauchamp & Childress, 2001). However, these theories are rarely used for an ethical evaluation in research practice, instead, research ethics guidelines are consulted, which we also do in our analysis. Since we conduct research in the health sector using qualitative methods, the research guidelines in medicine and social science are very important to our work. Following the World Health Organization (WHO) standard, we feel committed to the Council for International Organizations of Medical Sciences (CIOMS), but, simultaneously, to the guidelines applicable in our discipline due to our profession as social scientists. Therefore, we have chosen the guidelines from the CIOMS and the German Data Forum, which also represents principlism, to be appropriate for evaluating the MCMs from a research ethical perspective.
The German Data Forum proposes adhering to the principles of scientific quality, informed consent—including voluntariness—and avoiding harm—including confidentiality, and anonymization (RatSWD, 2017). We complement these with the principles of beneficence and justice, which are reflected in the CIOMS guidelines for the equitable distribution of (individual and social) benefits and burdens of research as relevant in health-related research (CIOMS), 2016). This ethical approach now serves as a framework in which we reflect on various decision criteria, which we present later in the article.
Five Methods for MC and their Ethical Challenges
Returning Interview Transcripts
The method with the minimum effort for researchers is returning verbatim transcribed interviews to participants to enable them to check the accuracy of the data collected initially. Although this method is mentioned under the heading “Respondent Validation” (Torrance, 2012) or “Member Checking” (Birt et al., 2016), it is not actually a method in which the validity of the interpretation of the data is examined (Birt et al., 2016) because no interpretation by the researchers is given to the participants. However, adding or deleting data enables participants to express their experiences, feelings or opinions better and, thus, contribute to a more accurate database for interpretation.
As far as ethical issues are concerned, various authors indicate that some participants dislike seeing their speech in written form (Birt et al., 2016), for example, they are surprised by their false starts and use of fillers (Koelsch, 2013), or are embarrassed because they realize they have been talking a lot more than they thought they had (Forbat & Henderson, 2005). Carlson (2010) illustrates in her reflection on having used return of verbatim interview transcripts that participants felt uncomfortable reading their verbal utterances, even leading to apologizing for their interview and, in one case, even dropping out of the study. Another problem, especially in health sciences, is that participants can be in a different phase of their illness when they receive the transcript. This can cause distress to them and/or their family members, especially when health has declined or transcripts remind them of previous difficult times (Birt et al., 2016). Goldblatt et al. (2011) also point out that asking for feedback on a transcript carries the risk of overburdening the participants, possibly because of the renewed confrontation with a sensitive research topic. Returning transcripts can, therefore, be harmful to participants, especially when the latter do not have the possibility of contacting the researcher. By contrast, this method can also have a positive effect, because participants have the opportunity to see their own experience from a third-person perspective and in a different way (Koelsch, 2013). The participants can use this opportunity to reflect once again on the research subject with a little more time and possibly add something to the transcript (Torrance, 2012) or delete passages that they feel represent their experiences in an uncomfortable way (Birt et al., 2016). In addition, participants are given the opportunity to flag information that they consider confidential and should not be made public (Rowlands, 2021).
Member Check Interview
In a member check interview, often conducted in the phase of drafting the manuscript (Slettebo, 2020), the participant receives meaningful sections of the research report and has the opportunity to comment on them orally. Koelsch (2013), for example, provided separate “write-ups” of the results for each participant and instructed the participants to read them and check for accuracy even before the member check interview. In the member check interview, she asked the participants whether she had portrayed their stories accurately (Koelsch, 2013). Birt et al. (2016) emphasize that participants can be empowered to remove and add to their data during the member check interview and, in this way, develop new meaning. Chase describes the interview as a “process of crafting and revising the narratives” (Chase, 2017, p. 2693) together. In this personal dialogue between the researcher and the participant, the researcher’s interpretation can be refined independently of other participants (Slettebo, 2020).
Regarding ethical issues, the member check interview can potentially be either distressing or therapeutic (Birt et al., 2016; Buchbinder, 2011; Koelsch, 2013) if, for example, participants are again confronted with their possibly traumatic experiences. Koelsch (2013) describes a member check interview in which a woman sees her sexual assault experience put into words through the interpretation of the researcher and, thereby, experiences her feelings as a natural response to this abuse experience. Buchbinder (2011) describes different experiences of a member check interview. On the one hand, she emphasizes that the interviewee might perceive the researcher as a critic, judge or therapist by presenting the interpretation, but, on the other hand, she experienced that there is no resistance from participants to hear the researcher’s interpretation. Chase (2017, p. 2700) highlights the advantages of conducting a member check interview. She remarks that the interview “works towards social justice, equity, and empowerment” and that it “facilitate[s] collaborative inquiry based on mutual respect and trust” (Chase, 2017, p. 2700).
Member Check Focus Group
The emerging draft of the research report is discussed with the participants in a member check focus group (Torrance, 2012) and people are asked for comments on the analysis (Goldblatt et al., 2011). In contrast to the member check interview, this is not done individually but in groups. The interpretations and anonymized illustrative quotes, which should ideally only be identifiable by the specific respondent, are presented to a group of study participants in a meeting (Goldblatt et al., 2011).
Regarding ethical issues, there is the possibility that participants feel exposed when they identify their own speech on the screen and this might be intensified if the participants know each other (Goldblatt et al., 2011). This could be a problem especially when using a member check focus group for participants who are colleagues as they might not wish to share their personal experiences, which might have been the case in the example above. Similar to member check interviews, a member check focus group with clients or patients can also be distressing for them (Birt et al., 2016) when being reminded of traumatic experiences (Slettebo, 2020). This distress can probably be handled better in a one-on-one interview. On the other hand, to read one’s own experiences mirrored by the experiences of others can make commonalities perceptible, generate a “feeling of collectiveness” (Slettebo, 2020, p. 5) and help participants to see they are not alone (Birt et al., 2016). This is seen as a positive effect of the work in the member check focus group and can make some people proud of their own contributions (Slettebo, 2020).
Synthesized Member Checking
The method of synthesized member checking, developed by Birt et al. (2016), is an elaborated method of a written MCP. The method was developed for research with patients, but, according to the authors, it can be transferred to other research fields. It consists of five steps: 1. Researchers prepare a synthesized summary from emerging themes, with illustrative interview quotes from different participants which represent the themes. 2. Before mailing this summary to the participants, researchers check whether the participants are eligible to receive the report; for example, if the participant is a patient, by consulting a nurse who can assess their health status. 3. Participants receive two copies of the report, one of which they may keep, and they can make comments on the other one. A stamped addressed envelope is provided to return the commented report. Researchers describe what the participants are expected to do (read, comment, check whether the results reflect their experiences) in a cover letter. Researchers’ contact details are also provided. 4. This entails recording the level of engagement by gender, age and disease in order to judge the response rate. 5. In the last step, the new findings are integrated into the existing results (Birt et al., 2016).
Regarding ethical issues, Birt et al. (2016) stress the importance of giving participants the opportunity to contact researchers after receiving the report in order to avoid feeling isolated or overwhelmed. Birt et al. (2016) and Harper and Cole (2012) point to the possibility that when participants read comparable experiences of other respondents in the report, they might perceive commonalities that they consider beneficial, similar to group therapy. These advantages are not contrasted with any disadvantages by Birt al. (2016).
Providing the Research Report or a Draft of the Publication
Several authors mention the provision of the research report or a draft of the publication to participants as another method (Brewis, 2014; Slettebo, 2020; Torrance, 2012), with the aim of gathering new or additional information or comments from the participants (Brewis, 2014).
Regarding ethical issues, Torrance (2012) argues in favor of this method when he mentions the point that participants can feel that their confidences have been betrayed if they do not have the chance to see what will be published about them. Brewis (2014) also used this method because of the risk that the report may not adequately represent respondents’ experiences, particularly if the report is written a long time after data collection, since life experiences may have changed in the meantime.
As a result of the literature review, we can state that, from an ethical point of view, all methods involve both opportunities and risks. They can be both harmful and beneficial, which underlines our argument that researchers have to decide on the basis of the characteristics of each research project and research ethical principles which MCM is ethically appropriate. Adequate decision criteria, which we develop later in this article, are required for this decision to be made. Only synthesized member checking seems to have considerably minimized risks. However, from our point of view, even this method can be disadvantageous, since it is a written method without personal contact with and between the participants. In the case described above, for example, it was an advantage to bring the participants together in a member check focus group to discuss contradicting results from different perspectives and refine the results in this way. This benefit would not be possible with a synthesized member check. In addition, we see the disadvantage with synthesized member checking that the participants can be overwhelmed with the written material and the written feedback. Therefore, synthesized member checking should not be used without thorough ethical reflection.
Proposed Decision Criteria for the Ethically Reflected Choice of a MCM
Decision Criteria and Issues to Consider.
Participants
In order to comply with the principles of informed consent, beneficence and avoiding harm, researchers should first think about which participants were involved in the qualitative study and which resources (physically, mentally, emotionally, economically, temporally) they have. Illness—both chronic and terminal—affects whether people are able to take part in an MCP (Birt et al., 2016), but socioeconomic and time resources have an impact as well. It can also be the case that the participants belong to a group that is actually or has historically been disadvantaged and for whom participation in a MCP could be particularly empowering. The slogan “Nothing about us without us” reflects the need of marginalized groups to be involved in not only political decisions that affect them but also research (Shannon et al., 2021). It might be particularly beneficial for some participants to meet others in a focus group as they feel strengthened by sharing similar experiences (Slettebo, 2020), whereby other participants might prefer a one-to-one member check interview or giving written feedback, which the MCM of returning the transcripts of the interviews, synthesized member checking, or providing the research report or draft for publication enable. Incorporating the perspectives of the participants to research through an MCP complies with the principle of justice in all five MCM. Researchers should, however, bear in mind the potential for paternalistic decision-making, because they may think they know best what is good for participants. We propose a two-stage procedure that complies with the principle of informed consent to ensure voluntary and informed participation in the MCP. Firstly, information about possible MCMs and their opportunities and risks must be provided at the same time as the information about participation in the study and informed consent is given. We suggest asking about and documenting the concerns and wishes of the study participants, so that they can be considered when deciding which MCM is best suited for the respective study and the participants included in it. The principles of beneficence and avoiding harm play a decisive role in this decision based on the wishes and concerns raised by the study participants. Secondly, informed consent for participation in the MCP must be obtained before the MCM selected is carried out.
Nature of the Research Subject and the Research Results
The nature of the research subject and the research results are also significant for the decision. Some research subjects carry the risk that somebody could feel distressed (Birt et al., 2016), re-traumatized (Slettebo, 2020), stigmatized or ashamed by the findings presented. One of the authors, for example, in one of our current research projects talked to patients suffering from inflammatory bowel disease about their lives with the disease. In this project, the participants also report on shameful situations that some of them would not like to have presented in a member check focus group. Therefore, choosing a member check focus group would not be an ethically appropriate choice as it does not conform to the principles of beneficence and avoiding harm. In these cases, a member check interview in a trustworthy, protective atmosphere with the opportunity for the researcher to support the participant individually and show empathy would fit better. On the other hand, very similar research findings would speak in favor of the member check focus group, as it could create a sense of collectiveness among participants (Slettebo, 2020), and show them that they were not alone in their experiences (Birt et al., 2016), which could be a benefit for them. If, as in the example mentioned at the beginning, the research results involve expert knowledge which is difficult to interpret or where apparent contradictions in the data require further clarification, the scientific quality of the results can be enhanced by a personal meeting between the interviewees and the researchers in a member check focus group. In these cases, personal contact can help to improve the interpretation of the data and, thus, the ethical principle of scientific quality can be complied with. Researchers should weigh these different dimensions carefully and consider how they can reconcile such a decision with their professional attitudes and experiences.
Researcher(s)
These considerations lead us to the question of which competences the researcher actually has to bring along for the implementation of a MCM, and that these are crucial for the choice of the method. Similar to other focus groups, moderation and conflict skills are required (Jenkinson et al., 2019) for a member check focus group, especially, as in our case, when participants have different opinions regarding the same issue. If traumatic and distressing experiences are discussed, the researcher must have empathy skills, a basic skill for every groupwork (Jenkinson et al., 2019), to avoid harm to those involved. In any type of MC, the researcher must have the ability to cope with criticism and be willing to incorporate it into the data analysis and dissemination to uphold to the principles of justice and scientific quality. The skills of each researcher within the research group should determine who takes which task in the MCP (Buchbinder, 2011).
Relationship between Participants
When planning a member check focus group, the relationship of the participants to each other must also be considered to do justice to the principles of beneficence and avoiding harm. Whether participants are competitors, friends or people who have a dyadic relationship influence what they chose to share in a member check focus group. Some people may not feel able to speak openly, because they are concerned about anonymity and the confidentiality of the data at such an event. On the other hand, people who are friends, colleagues or relatives particularly may not be concerned about confidentiality, since they already share a lot. If the relationship among participants is difficult to assess, a member check interview with individual participants, a synthesized member check or providing the research report or a draft of the publication seems to be more preferable. In any case, for a member check focus group, the assessment about the confidentiality of the participants themselves should also be obtained and considered in order to ensure informed consent and avoiding harm. In one of our research projects on experiences of coercion and psychological pressure in psychiatry (Potthof et al., 2022) the participants (service users and relatives of service users) partly knew each other via self-help group before participating. The participants (service users and relatives of service users) partly knew each other via a self-help group before participating in one of our research projects on experiences of coercion and psychological pressure in psychiatry. During the one-to-one interviews one of the authors conducted, some participants said that they had exchanged their very personal experiences with psychiatry and mental illness in the self-help group and that this has been very helpful to them. In this case, a member check focus group would have been ethically appropriate since the confidentiality problem did not exist and the likelihood of benefit to the participants of not being alone with a particular experience appeared high (Birt et al., 2016; Slettebo, 2020). However, there were also service users and relatives in the same research project with very conflicting attitudes towards coercive measures in psychiatry and those did not know each other. Bringing these participants together in a member check focus group might have harmed service users who were themselves subject to coercive measures. Consequently, a member check interview would have been more appropriate in this case as it could have empowered them (Birt et al., 2016) by facilitating a collaborative process of revising the analysis between researcher and participant (Chase, 2017).
Relationship between Participants and Researcher(s)
The relationship between the participants and the researcher(s) also plays a significant role in these considerations and the principles of beneficence and avoiding harm must be taken into account. If the researcher has a close relationship with the participants, such as a colleague or friend, the participants might not want to talk openly about every research topic. The participants, for example, were other qualitative health researchers in one of our research projects on research ethics application procedures. When choosing whom to ask to take part in an interview, we took into account that they were not direct colleagues with whom we had already worked, in order to make sure they could talk about their ethics applications to research ethics committees without us knowing the other people involved—other researchers and ethics committee members. If we had known the other people involved, we might have put the interviewees in an ethically difficult situation as they would have had to decide on the confidentiality of the people involved. At the same time, a certain closeness between the researcher and the participants can be a prerequisite for speaking about sensitive topics, for instance, about professional mistakes or incompetence that they would probably only reveal to someone who can understand regarding their professional situation. In the research project mentioned, it was accordingly important for the consent to participate that the participants already knew us indirectly through other colleagues or conferences. It was particularly important to the participants that we maintained confidentiality towards the scientific community. Therefore, a member check focus group had not been ethically appropriate for these participants. Providing the research report or a draft of the publication would be an appropriate MCM to allow these participants to check the confidentiality of the citations selected for publication (Brewis, 2014; Torrance, 2012).
However, the relationship between researcher(s) and participant(s) can also be asymmetrical, when the researcher has an advantage in knowledge on the topic over the participant(s), or the participants are dependent on the researcher, for example, if they are the latter’s patients. In these constellations, the consideration of the voluntariness of research participation, which is contained in the principle of informed consent, is of particular importance to the researcher. Asymmetric relationships might be evident in a member check focus group when the researcher presents the results in front of the participants, which is physically a more unequal situation than in an individual, one-to-one conversation between the researcher and the participant. At the same time, one-to-one situations can also appear asymmetrical for participants, since they sit alone opposite the interviewer and not—like in a focus group—together with other participants. In this case, researchers should invest in trust-building measures in order to make participants feel comfortable. An asymmetric relationship, especially if not noticed by the researcher, can discourage participants from an active and critical discussion, which would then not do justice to the principles of beneficence and scientific quality.
A good option for reducing the asymmetry between researcher and participant(s) could be the synthesized member checking and providing the research report or a draft of the publication, since the researcher is not present when the participant receives the results and comments on them.
Resources
Another important decision criterion are the resources available to the researchers for the MCP that ensure their implementation and, thus, do justice to the principles of scientific quality and justice. This refers to human, space and time resources, which are usually only available to a limited extent, especially if the research is funded by third-party donors. High personnel and possibly also travel costs are incurred particularly for the member check interview and focus group, which must be weighed against the scientific and personal benefits. The member check focus group in our example at the beginning of this article could be conducted in a resource-efficient manner by using video conferencing software, which, at least, saved the experts travel time. But this option does not work with every research project and all research participants. Written MCPs are likely to require fewer resources, but, at the same time, hold scientific promises and can also have personal benefits for participants. The human resources necessary for incorporating the results of the MC into the analysis and dissemination must be considered for each MCM.
It is also important that the researchers, before choosing an MCM, consider the resources of the participant(s) to ensure that they can participate. It should be noted that certain resources can be compensated and others cannot. Travel costs for participants can be covered or avoided, but not their time and emotional effort.
Methodological Approach
Including the results of the MC in the analysis and dissemination leads us to the last criterion that we propose for the choice of the MCM, the methodological approach. The methodological approach of the research must match the MCM. The editing and return of interview transcripts, contradicts (sequence) analysis methods, which are interested in analyzing "how" exactly something was said, for example, in interpretative phenomenological analysis (Smith & Nizza, 2022) or objective hermeneutics (Maiwald, 2005). By contrast, when working with qualitative content analysis (Kuckartz, 2014), this does not cause a problem.
Furthermore, the question arises regarding which results are valid if we consider the time dimension between data collection and the MCP. Following a constructivist approach and coding paradigm allows one to consider revisions by participants at different points in time because it is based on the belief that social reality is constructed interactively and multiple realities exist (Forbat & Henderson, 2005; Goldblatt et al., 2011). Accordingly, the question is not which perspective of the participants is true—that at the time of data collection or that at the time of the MCP—but different perspectives on a research topic at different times complement each other. From this point of view, the other four MCMs also fit well with a constructivist approach and promise added scientific value, which is important taking into account the ethical principle of scientific quality.
Discussion
As we have argued, from our point of view it is necessary to consider ethical implications in addition to methodological considerations when deciding on a particular MCM. We have shown that there are ethical considerations for individual MCM in the literature, but no systematic compilation of ethical considerations that includes all MCMs and provides decision criteria for research practice. In this paper, we developed these decision criteria based on the consideration of several characteristics of the research project (participants, research topic, researchers, relationships, resources, methodological approach, results of the study) and research ethical principles for the ethically reflected choice of a MCM.
A crucial issue with every MCM is how ambiguity in the interpretation between participants and researchers can and should be dealt with ethically. If there is ambiguity in the interpretation, this polyphony of understanding should be utilized for a further comprehension of the phenomenon under research and be presented transparently when the results are disseminated (Goldblatt et al., 2011). This process of joint knowledge production is known from participatory research that includes MC, however, not as a singular event but as a continuous process (Bergold & Thomas, 2012; von Unger, 2012). Transparent presentations of a jointly produced interpretation that combines various MCMs with a participatory research approach are, for example, the works of Chase (2017), Doyle (2007) and Harvey (2015). Chase (2017) used the MCMs of providing the research report with member check interviews and a member check focus group in an iterative process of participation in her research with young teenage mothers regarding their schooling experiences. Harvey (2015) describes in detail the joint production of knowledge in her study with international UK university students on their motivation for learning English. After each interview, she opened a dialogue with her study participants by providing her analysis in writing and giving everyone the opportunity to respond.
In our view, the transparency of knowledge production, including differences of interpretation and negotiating those between researchers and participants, is most important because it lives up to the principles of justice, beneficence and avoiding harm for participants and, simultaneously, does justice to the principle of scientific quality as it matches the quality criteria of intersubjective traceability and communicative validation (Hopf, 2004). However, if there is a risk of harm from a researcher’s interpretation for participants or the group to which they are related, publication of this interpretation should be considered well.
Comparing the five MCMs presented, in our view, the return of interview transcripts alone does not sufficiently do justice to the principles of scientific quality, beneficence and justice. The method offers only a limited opportunity for the participants to check and expand jointly the researchers’ interpretation, especially in the case of verbatim transcriptions, as these are "raw data." However, as Forbat and Henderson (2005) have shown, the choice of a particular type of transcription is itself a decision regarding the way the participants are re-presented in a research report or publications. Researchers, for example, have to decide whether grammatical errors, repetitions, word breaks and intermediate sounds such as “Uhs” are corrected or not. This raises the ethical question of how participants can or should be represented by researchers while respecting the principle of avoiding harm.
Carlson (2010) and Forbat and Henderson (2005) suggest informing participants upfront about the aim of the MC and what is expected of them, and what a verbatim transcript or an interview quote looks like. Therefore, if participants know in advance that verbatim transcripts or interview quotes will contain their pure, not refined language and this is common in qualitative research, then harm to participants can be avoided. In addition, they advise discussing with the participants which transcript excerpts in which form of transcription are intended for scientific publication and how the participants feel about this; whether they want researchers to eliminate grammatical errors and involuntary vocalization (Rowlands, 2021). Accordingly, the method of returning transcripts offers, at least, the possibility of increased participation in research and reducing the power imbalance between researchers and participants (Rowlands, 2021). However, this potential beneficence for participants as the main reason for choosing this method is, from our point of view, not sufficient to counterbalance the method’s disadvantages, especially since the other MCMs offer these benefits as well.
As shown in this article, MCP is not only a method to increase the credibility of the results but can also have a benefit for the participants. There will also be research projects in which one will resort to other procedures, such as “peer debriefing” or “disconfirming evidence,” to ensure the quality of the study (Lub, 2015). We are thinking here, for example, of studies in which the interpretation requires psychological or psychiatric expertise that is difficult for the participants to follow or may even provoke resistance. Peer-debriefing to have the professional interpretation assessed by the expertise of colleagues seems to be well suited in such cases. However, one should keep in mind that the different procedures designed to enhance the quality of a study correspond to different perspectives of the people involved. The MCP incorporates the perspective of the research participants, the search for “disconfirming evidence” is a task of the researcher alone, and the peer-debriefing brings the perspective of a disinterested peer into the analysis (Lub, 2015). Comparing these three procedures, only the MCP allows the researcher to give something back to the participants being interviewed and, thus, also serves the ideal of reciprocity in research relationships (von Vacano, 2019).
Conclusion
Member checking was introduced by Guba and Lincoln (1989) under the methodological premise of improving the credibility of qualitative research. In addition to the methodological advantages or disadvantages of MC, which have already been broadly discussed in the literature, we have shown that it is important for researchers to think about the ethical implications of MC for the research participants. The particular MCM must be carefully selected, as we propose, considering the seven decision criteria developed and the principles of research ethics. In order to meet ethical principles sufficiently, an MC must go beyond a purely technical step in a study and, instead, be an ongoing process of a joint production of knowledge between researchers and participants, similar to what participatory approaches envisage. Based on our analysis, we also come to the conclusion that the use of a suitable MCM makes sense in terms of research ethics against the background of the benefit for research participants. However, in order to ensure that the choice of an MCP meets the wishes and concerns of the participants, we also recommend, if possible, asking participants about their preferences regarding a MCP and to include these preferences in the decision-making process. This type of shared decision-making involving the researcher’s decision criteria and the participants’ wishes and concerns should be evaluated in future empirical studies.
Footnotes
Acknowledgments
We would like to thank Christin Hempeler, Matthé Scholten and the anonymous reviever for their constructive criticism on an earlier draft of this paper.
Declaration of Conflicting Interests
The authors declare that this scientific work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We acknowledge the open access funding enabled by a grant from the German Federal Ministry of Education and Research (no. 01GP2191). The funding was granted for the organization and conduction of the international spring school “Research Ethics in Qualitative Health Research” held at Ruhr University Bochum in March 2022 by SP. This article was based on a presentation given by AE at the Spring School. Further development was also supported by funding from the German Research Foundation (DFG) (no. F381 434).
