“Shoot! Can We Restart the Interview?”: Lessons From Practicing “Uncomfortable Reflexivity”

Abstract

Failure is a typical experience in research, but it is largely taboo in published studies. In recent years, however, we can observe a small yet growing body of literature on failure in qualitative research to address this gap. In this article, I contribute my experiences of failed interviews in a mixed-methods study in Germany to this body of literature and highlight some aspects of failure that have not yet received enough attention. First, in my example, it was not only one interview or a few interviews that failed; rather, it seemed that the whole study failed in design due to particular methodical decisions. Second, failed research presents an intellectual challenge, but it also produces emotional and social trouble because failed research might be attributed to a failed researcher. This may be one reason failure is so damaging for one’s well-being and so difficult to share. Nevertheless, practicing some form of “uncomfortable reflexivity” (Pillow, 2003) via qualitative, close analysis helped me navigate the research process, gain methodical insights and substantive results. Third, I share lessons that might be useful for other researchers: reading literature on failure, the search for a safe and supportive space, and analyzing failure as closely and early as possible.

Introduction

Failure is a typical experience in research, but it is largely taboo in published studies. This has negative consequences, especially for novice researchers, who are not yet aware of the normality of unexpected twists and turns in the process of research, and assume personal responsibility for failures (Delamont & Atkinson, 2001).

In recent years, however, a small yet growing body of literature in qualitative research not only creates visibility of failure as a normal part of research, but also establishes a “counter-narrative” (Nairn et al., 2005) of failure by showing the insights that can be gained from working with these data (e.g., Eckert & Cichecki, 2020; Gregory, 2019; Nairn et al., 2005; Prior, 2014; see also Roulston, 2011a, 2011b).

Sousa and Clark’s “call to share more” (2019) experiences with failure is an important step toward making counter-narratives more numerous, visible, and accessible.

I wish to contribute to their call by presenting my own experiences with failed interviews. These experiences might represent an extreme case of failure: Rather than only one or a few interviews failing, as in most published accounts of failure, in principle, the whole study failed in design. In this paper, I discuss the challenges this created and then present how I practiced a form of “uncomfortable reflexivity” (Pillow, 2003) to work with these interviews. In my conclusion, I discuss the lessons I learned to deal with failure in a way that could benefit others.

Context: A Mixed-Method Study on Subjective Insecurity

The context of my experience with failure is a mixed-method study on subjective insecurity (2010–2013, n = 405, see Blinkert et al., 2015) that I joined several months after its start. Although basic decisions concerning the research design and interview guideline had already been made, I had the chance to contribute my feedback and suggestions. I could also use the data for my PhD thesis.

The point of departure for this study was well-received theories that claim that contemporary Western societies can best be understood as societies of insecurity and fear. Bauman (1999, p. 5), for example, writes that

[t]he most sinister and painful of contemporary troubles can be best collected under the rubric of Unsicherheit—the German term which blends together experiences which need three English terms—uncertainty, insecurity and unsafety—to be conveyed.

Against this background, our project team (including senior and junior researchers) asked the empirical question, “What do people fear in their everyday life?” As Germans, our main analytical concept was (Un-)Sicherheit, which we also used as the central term in our interview guideline (here translated as “(in-)security”). While Bauman appreciates this German word as an umbrella term, we valued the different dimensions of Unsicherheit that Bauman recalls and used them as an analytical lens.

The central methodological idea of our mixed-methods study was to explore a new methodical avenue that overcomes problems of existing approaches. Within the study design, qualitative research performed different functions.

The first is enhancing the validity of the data generated. Since surveys with predetermined risks overestimate respondents’ risk perceptions (e.g., Gaskell et al., 2017), we chose a more open-ended approach to data generation. In the first part of the interview, we followed the idea of openness toward all kinds of fears that our interviewees might have and invited them to talk about what they feared, or, to be more precise, what “threatened their personal insecurity,” without providing any topics. We used qualitative content analysis in order to quantify the open-ended answers and analyzed the whole sample quantitatively in order to provide insights into the numerical relevance of particular fears.

The second function was to interpret selected aspects of the statistical results based on a reconstructive analysis of a subsample of the interviews.

In all, 405 interviews with an average duration of approximately 100 minutes were conducted across four sites in Germany in 2011. The interviewees were randomly selected via the respective population registers and recruited via letter. They provided informed consent (written) and were assured that their anonymity would be protected.

The reconstructive strand of analysis was one of my tasks, and it started with confusing results. According to the quantitative analysis of the first part of the interview, fears assigned to the categories of crime, health issues and accidents ranked at the top. All of these fears fall into Bauman’s dimension of unsafety, whereas the dimension of insecurity is, among the totality of fears, of minor relevance, and the dimension of uncertainty is completely irrelevant. These results are in stark contrast to those of other studies about fears in Germany (e.g., R+V-Infocenter, 2019). I faced the question of how to interpret this phenomenon.

Problem: Questionable Data, Yet A Good Researcher?

My first qualitative interview analyses and early feedback from other scholars gave me the impression that the study had some serious methodical problems. One main point of criticism was that the questions were phrased in a way that did not adequately address the research topic.

The fact itself that I used questionable data as the basis for my doctoral thesis took a toll on me, because I did not know if I could answer my research question of what people fear. This intellectual challenge came along with a second, more emotional and social challenge that is usually not discussed: Does failed research mean a failed researcher or can one still produce a good PhD thesis, be a good researcher, and be perceived as such a person? In his “late outing” of the difficulties he had when finishing his PhD, Schröer (2011, p. 96, own translation) calls this challenge “securing social connectivity” to the scientific community to which he wished to belong and describes his emotional trouble as ranging from fear of failure to sleep problems.

Against this background, it is difficult to share experiences of failure. How could one avoid being perceived as a failed researcher? As a consequence, I chose to talk about the methodical details of my study only in “safe spaces.”

Methodology: Practicing “Uncomfortable Reflexivity”

How, then, should I deal with the failure? Nairn et al. (2005) have characterized their analysis of a “failed” interview with Pillow’s (2003) notion of “uncomfortable reflexivity.” In my reading, reflexivity means to acknowledge the researcher’s presence and influence in all stages of the research process and to practice self-reflection. This becomes uncomfortable when “your choices, presence, or influence contributed conceivably to an adverse or undesirable research process or outcomes” (Sousa & Clark, 2019, p. 1). I found three ways of practicing this kind of “uncomfortable reflexivity” (Pillow, 2003) particularly helpful.

First, I discovered the emerging debate on “failed” or difficult interviews (especially Nairn et al., 2005; Prior, 2014; Roulston, 2011a, 2011b). This small body of literature re-interprets failure as insightful, which had an important psychological function for me.

Second, this literature provided me with an epistemological and methodological perspective that allowed me to view failed interviews in particular and the interview in general in a different light. In the mixed-methods study, we implicitly followed a “neo-positivist conception of interviewing” (Roulston, 2011a, p. 79), according to which “you ask, they answer and then you know” (Hollway, 2005, p. 312). In my dissertation, I decided to adopt a largely “constructionist perspective,” according to which “interviews are co-constructed by both interviewer and interviewee” (Roulston, 2011a, p. 80), who both play active roles in the process of data generation. Thus, even if one is not interested in studying interaction, it is crucial to understand how the data were produced in order to “make adequately grounded claims” from the interviewee’s talk (Potter & Hepburn, 2012, p. 556), in my case, about their fears.

Potter and Hepburn (2012) have shown in detail which interactional features researchers should consider when analyzing qualitative data and reporting their results. These include the category the interviewees are recruited under and the task understanding (e.g., the topic of the interview) they are offered, since these issues are “potentially consequential for the research outcomes” (p. 556). This also applies to the “footings” (Goffman), that is, the speaking positions of the interviewees (p. 563 f.).

This constructionist perspective has proven useful for understanding problematic interactions: In contrast to neo-positivist interview epistemologies, they do not restrict the analysis to the product of the interview (the interviewee’s talk), but allow the process of the data generation to be reconstructed since they take “the co-construction of interview data as a topic of examination” (Roulston, 2011a, p. 78). Consequently, “researchers may examine the variety of actions that take place in interview contexts and what the outcomes are for the data generated” (p. 93).

Third, the constructionist perspective on interviewing comprises methodical recommendations about how (problematic) interactions can be studied by drawing on conversation analysis. Thematic or content analysis that focuses on the “what” is insufficient; it is also important to analyze the “how” via close analysis of the transcript. For example, “How are questions formulated by the interviewer? How do interviewees orient to the interviewer’s questions? What actions are accomplished in the talk?” (Roulston, 2011a, p. 82).

Since we used qualitative content analysis in the mixed-methods project in order to quantify the open-ended answers, we could not see the interactional generation of the data. In my PhD thesis, I re-transcribed the interviews by adding interactional information and chose a micro-analytical approach for data analysis: the “integrative approach” (Kruse, 2015). This approach is inspired by narrative analysis and conversation analysis and considers what is said, how it is said, and what is not said. Using theoretical sampling, I finally analyzed a subsample of 39 of the 405 interviews.

Nature of Failure: Failure in Design

In my close analysis, I found two methodical dimensions that were crucial for the study’s failure to adequately address the research topic, a particular question wording and rapport. The interplay of both creates failure-conducive conditions ¹ that make topics of unsafety (e.g., crime) more tellable and those of insecurity and uncertainty less tellable. As a consequence, we failed in the study design to realize the idea of openness toward all kind of fears that our interviewees might have.

Wording: Security as a Narrow Notion

Regarding the wording, I came to realize that the interviewees understood the analytically broad notion of (Un-)Sicherheit, here (in-)security, and thus the interview topic typically and at least initially in a narrow sense. This is illustrated by the following interview excerpt with an interviewee that I call Nicole (N). ² The interviewer (I) introduces the exploration of relevant topics, as suggested by the interview guideline:

I:

What is it like, talking about your situation, your experience as a whole. Are there any topics that really threaten your personal security, any dangers or risks that cross your mind?

N:

(2) Security in what sense? Well, I do, I—are you asking: am I afraid of someone just showing up and robbing me?

I:

Yes!

N:

Things like that?

I:

Yes, topics like that; yes.

As we see by Nicole’s delayed response and her wish for specification, the notion of security is not clear for her. Therefore, she herself proposes an example, which the interviewer unambiguously approves by saying, “Yes!” This is how assaults become an interview topic. Later, Nicole adds the fear of mugging, fear of being concerned by someone running amok, and fear of terrorism.

In the course of the interview, Nicole repeatedly returns to the question of what exactly we as researchers mean by the term security, with similar responses from the interviewer, who confirms her suggestions. It is only in the second part of the interview, when the interviewer introduces the topic of economic crises, that the misunderstanding is revealed:

I:

How much do you see yourself threatened in your personal security by (.) economic crises?

N:

That’s the question now, how is security defined? Is it financial security, just like social security?

I:

So personal security would in principle include everything.

N:

Ah:, OKAY OH::

I:

As you see it for yourSELF; what is—what is personal security for myself;

N:

Shoot! Can we restart the interview? ((laughs))

Unlike before, the interviewer not only approves Nicole’s understanding; more importantly, he opens up the definition by highlighting the inclusive character of the term. Furthermore, he gives Nicole the power of defining the term security. This seems new and surprising to her, as the emphazised and elongated particle “OH::” indicates. By using the word “shoot,” she expresses her displeasure, since she realizes after two hours of interviewing that she could have discussed topics other than the fear of crime, etc. Thus, shortly before the interview ends, the misunderstanding is revealed.

In other interviews, similar interactions occurred, for example, when interviewees asked questions such as “so security, is it about physical security or what?” Usually, security was at least initially associated with inner security and physical security, and not in the broad sense we intended it to be understood—which has also been observed in another study (Kohner & Kovanic, 2016). This makes topics such as crime, health issues, and accidents preferred topics to answer the interview questions. Given the broad spectrum of what we meant by (Un-)Sicherheit, this is a rather narrow understanding that covers only the dimension of unsafety (see also Eckert, 2016, 2019).

Rapport: Impersonal Interview

The second failure-conducive condition concerns the rapport between researchers/interviewers and interviewees. Our study design promoted a rapport that I call the “impersonal interview.” One aspect concerns the way the interviewees were addressed before and during the interview.

In the recruitment letter and the interview introduction, we informed them that our research interest was to determine “what ideas the population has about security” and to answer the questions “How secure do Germans feel? What makes the population feel insecure?” Thus, the interviewees were addressed as part of the ethnic and/or national collective of “Germans” and as representative citizens. Consequently, in terms of “footing,” some interviewees did not speak as individuals and did not share what was important for them in their personal lives, but talked about “normal fears” anyone could have (e.g., crime). The interviewee Valerie, for instance, considered the interview topic to be a state-related and “political” one and excluded her worry about her child’s well-being—a “personal issue,” as she says—from her interview talk.

Consequences: “Reluctant Respondent” and “Reluctant Interviewer” as Success-Conducive Conditions

Given these insights, I then wondered if it was still possible to identify success-conducive conditions in the sense that interviewees talked about what they personally fear in the broad sense of Unsicherheit.

My analyses showed that these successful interviews were conducted with a “reluctant respondent” (Adler & Adler, 2001) and/or by a “reluctant interviewer.” They overcame the failure in design because they chose a different wording and rapport.

Concerning the wording, “fear” seems to be open to the three dimensions of Unsicherheit that Bauman (1999, p. 5) mentions. This becomes particularly clear when there is a shift in the wording during the interview. The interviewee Lara, for example, answers the question about what threatens her personal security by talking about safety issues. When the wording shifts from threat to fear, she adds the fear about her professional future as a young mother which extends the interview topic to the dimension of security.

Regarding rapport, “reluctant interviewers” addressed the interviewees as individual subjects, for example, by rephrasing the questions using personal and every-day life connotations such as, “how secure do you feel PERsonally, in everyday life?” or by stressing the subjective relevance as a central concern: “we find important what is important to you.” In these “personal interviews,” interviewees were invited to speak for themselves and not for a collective.

I used these insights in two ways.

First, my published dissertation (Eckert, 2019) includes one chapter about these methodical reflections in order to promote the so far underdeveloped discussion about methods in my research field. My hope is that this helps researchers in future studies learn from the positive and negative sides of our study, that is, use open-ended interviews for researching fear, but avoid the mistakes we made. Furthermore, my analyses contribute to assessing the diverging results that previous studies on insecurity and fear have produced.

Second, in my further sampling strategy, I privileged interviews that satisfied the success-conducive conditions in order to gain substantive results. I could thus give an answer to the question of what people fear and successfully complete my thesis. This success-in-failure along with a reflexive stance allowed me to speak about my experiences beyond my “safe spaces.”

Conclusion: Lessons Learned for Dealing with Failure

As my story shows, even an extreme case of failure offers the chance to practice self-reflection and generate new knowledge. Finally, I want to discuss some lessons I learned for addressing failure in a way that could benefit others (see also Eckert & Cichecki, 2020).

One lesson is to read literature on failure in order to re-approach difficult data with a different mindset. In their book How to be a Happy Academic, Clark and Sousa (2018) propose a “growth mindset” that allows viewing failure as a site of learning. Additionally, I believe that each paradigm in qualitative research has its own counter-narratives. I provide just two examples. First, using a constructionist lens, problematic interviews can be re-interpreted as interesting “topics for analysis” (Roulston, 2011a; see also Prior, 2014). Grounded theorists may find it useful to refer to Dewey’s (1938) conception of research as pragmatic problem-solving, which makes problems the beginning of new insights.

Another lesson is to search for a safe and supportive space of peer researchers to reduce the “heavy weight” a lone researcher carries (Gregory, 2019, p. 7). Sharing stories of failure can have the psychological effect of normalizing failure (Delamont & Atkinson, 2001). Additionally, peer researchers can offer different perspectives that help to reflect how one’s preconceptions have shaped the research process. In my case, it was my qualitative data analysis group who made me aware that there were some problems with approaching and addressing the interview partners. I later identified these issues as elements of the “impersonal interview.”

A third lesson is to analyze failure as closely and as early as possible so that all subsequent research steps can be informed by the methodical insights it offers, for example, by adapting the research instrument, interview guideline, or sampling strategy (see also Roulston, 2011b). This works best within a research process that allows for flexibility and for enough time to analyze and learn from failures. In the case of the mixed-methods study, we did conduct a pretest (n = 21), and in retrospect, these interviews could have taught us that something was wrong. However, we focused our analysis of the pretest on more meso-level issues (e.g., reducing the number of questions or changing the order of questions) and did not conduct micro-analyses to determine how the interviewees understood the interview task. Accordingly, we conducted 405 further interviews with limited validity. This suggests that even if micro-analyses are not the overall strategy for data analysis, it can be useful to dedicate some time to analyzing the first interviews or pretest interviews in this way. It could even be helpful to analyze only the interview beginnings this closely because that is where the interview topic and interview roles (rapport) are negotiated for the first time, as the first excerpt from the interview with Nicole illustrates.