“Digging Deeper?”: Insights From a Novice Researcher

Abstract

This article reflects on lessons learned from a qualitative research project. Due to my quantitative academic upbringing, I missed the opportunity to dig deeper into the qualitative research material. The article critically reflects the steps of constructing an interview guide, conducting interviews, performing a qualitative content analysis and interpreting the results. Ideas on overcoming these barriers are presented. The article concludes with a positive outlook in the form of recommendations for other novice researchers in qualitative research as well as an appeal to experienced researchers to accompany us on this path.

Introduction

Participating in an international research project over the course of two years, I wanted to learn as much as possible. I was quite confident about my strengths as a researcher and knew the core competencies I would add to the team. I had not, however, reflected on my weaknesses, never having worked with qualitative data before. Through my studies I had developed as a positivist researcher (without questioning this) and I had not anticipated that this project would become a milestone in my career and make me question (quantitative) research.

The project design utilized a mixed methods approach with qualitative interviews, following the quantitative data collection. Being honest, I didn’t spend much time reflecting on this second data collection phase. As often the case in the academic world, I failed to value qualitative research. I anticipated that the interviews would simply (if at all) substantiate the results from the first (quantitative) data collection. Nevertheless, I took the responsibility of conducting the interviews very seriously and decided I would at least try to gain something out of this qualitative data collection. So, I began reading up on qualitative research and got completely lost. I read articles on different qualitative methods both in my specific field of research (e.g., Hoeber & Shaw, 2017; Smith & Sparkes, 2016) and outside (e.g., Leavy, 2014), discussions and presentation thereof (e.g., Bekker & Clark, 2018; Ellis et al., 2008; Tracy, 2010), critical thinking and epistemologies (Zakus et al., 2007) and so on. I became fascinated by narrative writing (Cowan & Taylor, 2016; Stride et al., 2017) and personal applications of research (e.g., Gannon et al., 2018) as well as the creativity with which researchers presented their findings (e.g., Hartung et al., 2017). I tried going back further, diving into the discourse about qualitative research rather than just its methods. I read about epistemological and ontological approaches to science, repeatedly stumbling over passionate discussions on paradigm conflicts. I soon concluded that I was ill prepared for the task at hand (and should have stopped or at least interrupted the process at this point to better prepare myself for the challenges ahead). Parallel to this research, the transcription of the interviews was nearly finished. This took much more time than expected but it was an interesting and new process for me. By now I had 1 month left to analyze the data, interpret the findings and write the report. In quantitative research, this would not have been a problem. When you know what you want to know, the data interpretation is a “fully automated touch-of-a-button analysis” (Clark & Sousa, 2018, p. 2). The first week of interpreting qualitative data made me change the way I think about research drastically. I thought I knew what I was doing, but I was so caught up in my quantitative paradigm, I couldn’t dig deeper into the material. I was still making lists and quantifying results in my head. I had learned how to work with MAXQDA specifically for this task, only to realize that my very structured and focused way of working with quantitative data was of no help whatsoever for this qualitative analysis. The 126 pages of single-spaced transcribed text of a total interview time of 550 minutes led to 67 specific codes and 927 coded segments. The different tools of MAXQDA enticed me to visualize my data within word clouds or matrices, all the time counting the number of times a word, thought or phenomenon was mentioned in the interview. I soon had pages of quantified qualitative data analysis only to realize that this was not at all what I was looking for. I had to start over. It didn’t help that I was still stuck in my positivist epistemology.

The goal of this article is to reflect on the process of doing qualitative research for the first time and on how my quantitative academic “upbringing” limited my abilities to supplement our quantitative findings with a second qualitative study. Due to this, I missed the opportunity to dig deep. These reflections may help other novice qualitative researchers. I conclude with a positive outlook on lessons learned and promote the leap into qualitative research.

The Task

The research project included nine countries and examined the labor market situation within my field (sport management) from an employer perspective. A colleague, a research assistant and my professor, made up the German research team. The project was designed to follow a mixed methods approach, integrating the results of the first (quantitative) data analysis into the construction of the second (qualitative) data collection. More specifically, the aim was to develop a semi-structured interview guide for the qualitative data collection with the goal to better understand the results of the first data collection and dig deep (a method I would later learn to be known as a sequential explanatory design (Ivankova et al., 2006)).We invested a lot of time into the first data analysis (quantitative), which included a questionnaire sent to a purposefully selected group of relevant experts.

The draft for the interview-guide was constructed in a joint project meeting. The results of the quantitative analysis were considered and new research goals set. The draft was later refined by team members and sent to the partners for adaptation of specific thematic blocks. Depending on individual results of the quantitative data analysis, some questions were adapted. Each partner then had the task of (1) carrying out a minimum of eight interviews and (2) analyzing the findings.

Concerning the first task, I conducted 12 interviews over the course of 3 months. The interviews lasted between 34 and 58 minutes and were audio-recorded. The audio files were then transcribed verbatim by research assistants according to a specific transcription guideline based on Kuckartz (2010). I then proofread the interview transcripts and anonymized these.

In a second step, I performed a qualitative content analysis, following Mayring (2015), of the 12 interview transcripts. I read these repeatedly and coded segments using MAXQDA Analytics Pro 2018 according to theory-based categories (deductive) and new categories that emerged from transcribed interview materials (inductive). Next, I began to describe and interpret the results. The data analysis resulted in 67 specific codes and 927 coded segments. The number of coded segments ranged from 62 to 105 across the 12 interview transcripts (average of 77 coded segments per interview transcript). I considered asking a second researcher to also categorize and code the transcripts but dismissed this thought primarily due to time pressure. In order to illustrate the results and their interpretation concerning the coded segments, I extracted exemplary quotes from the interview transcripts. Since the interviews had been held in the German language, a professional translator translated these to English. I aligned our results to previous studies and the results of our previous quantitative analysis. Used to the quality criteria of quantitative research, I wanted to emphasize the quality (in the sense of validity and reliability) of my results. Digging further into research, I found two articles by Sparkes and Smith (2009) and Smith and McGannon (2018) that discussed quality and rigor within qualitative research in my field of research. Reading these articles and others, I realized that my entire approach to the qualitative study and analysis was misled.

I learned about the “critical friend” method to assure rigor and focused on a reflective approach (Wolcott, 1994). The role of the critical friend is “not to ‘agree’ or achieve consensus, but rather to encourage reflexivity by challenging each other’s construction of knowledge” (Cowan & Taylor, 2016, p. 508). As Smith and McGannon (2018, p. 13) put it: “The role [of a critical friend] is to provide a theoretical sounding board to encourage reflection upon, and exploration of, multiple and alternative explanations and interpretations as these emerged in relation to the data and writing.” Critical friends engage in dialogue about interpretive possibilities and requires researchers to make their thought processes explicit. My research colleagues were quite bewildered upon being confronted with this method and my request to critically discuss and reflect the results/interpretation. I am very grateful that they were open for this method, which resulted in long and partially strenuous discussion rounds.

Looking back, I realize that I conducted a large part of the qualitative data collection and analysis as though it were a quantitative study. The next section reflects the process and highlights issues that may have contributed to our results (or lack thereof).

Reflection

Several problems and challenges arose before, during and after the data collection. The first problems arose in the construction of the interview-guide. As I had no prior experience and knowledge about this process, I trusted the other (more experienced) researchers in the project. My ignorance did not stop me from jumping at the opportunity to create a first draft of the interview guide when asked by the project managers. Having proven our abilities with the analysis of quantitative data as well as research management in the first project phase, the research team trusted my colleague and myself to propose a first draft. At the time, we considered ourselves capable of doing so. We constructed the draft based on the results of the previous, quantitative analysis and on collegial advice. No focused research questions based on the previous results existed, which resulted in a structured interview very similar to the questionnaire. By way of example, while the quantitative questionnaire asked participants to reflect on the importance of specific competencies on a 5-point Likert scale, we now asked the interviewees whether the resulting top five competencies were of particular importance for their organization (confirmative) and in which specific situations employees would need these. In a few days we shared our draft with our colleagues. The result of our work was (1) way too long, (2) way too rigid and (3) inclusive of far too many topics. Looking back I regret that the other researchers did not intervene or improve the structure of the interview guide. There could have been many reasons why the team did not intervene and suggest revisions to our interview guide including lack of experience, lack of time or interest, or not wanting to offend us.

The next problem arose in the data collection phase. The individual research teams of the partner countries were responsible for the execution and analysis of the interviews. No guidelines existed on what to take into consideration. I was responsible for our interviews and highly motivated to get it right as I wanted to include results in my PhD thesis. After a pre-test in the form of a practice interview with a befriended professor, I felt I was ready for the real thing. Two colleagues assisted me in the data collection phase by carrying out one and five interview(s) respectively. The remaining six interviews were my responsibility. We carried out the 12 interviews one after the other, sticking strictly to the interview guide. No discussion, reflection, possible adaptation or similar took place during the data collection phase. I believed at the time this was the correct way of approaching the situation, this being based on my belief in objectivity and my modernist epistemology. As mentioned before, the interview-guide was all but well-suited for profound qualitative research. And though the content of the interviews was clearly shallow I did not, at the time, second guess the approach.

I realized that I had failed the goal of this study during the analysis of the data. As mentioned before, I tried to approach the analysis systematically, strictly following the chosen methodological approach. I was unable to escape my quantitative mentality and repeatedly caught myself quantifying the results, working toward generalizations and objective results. The process was quite frustrating as the small sample of 12 interviews did not offer much room for deriving interpretations on a holistic level. Though I produced pages full of word clouds, correlation matrices and tables of codes, I found it extremely difficult to select exemplary quotes (wanting to find affirmative statements from all interview partners of a specific category in order to generalize for that sub-sample). Concluding, the results of my analysis were (as harsh as it may sound) simply boring. The point of qualitative research and digging deep was completely missed. Though the results confirm the previous study, they hardly added anything new. Part of this can be explained by the previous mistakes (construction of interviews, rigidity in the data collection). I strongly believe that a researcher with proper training or experience in qualitative research may have been able to dig deeper into the material and transcriptions and find things that did not fit into my analytical analysis at the time. If I were to repeat the (time consuming) process of data analysis on this same material today, I would be more successful. Reflecting on this under the premise that theory free knowledge does not exist makes me question positivist knowledge.

Conclusion

Concluding, I spent a lot of time, brain power and energy on a project that failed its goal. At the same time, I transformed into a researcher with new ideas, a broader horizon and high motivation to continue my path. I learned about different epistemologies, ontologies and the meaning of phenomenology. For the first time in my (admittedly short) academic career, I learned to differentiate between ways of constructing knowledge. This will have a major impact on my future research and academic journey. Being an athlete, I see this failure as a challenge to overcome and look ahead for future projects. Most importantly, I reflect on the lessons learned (Dweck, 2008) and what I can do differently next time.

Generally, preparation is fundamental for the success of a qualitative research project. I would discourage novice researchers to jump into qualitative research the way I did. Preparation should begin within post-graduate study programs and include philosophical texts, such as Gadamer (1976/1992), Ricoeur (1981), Josselson (2004) and Orange (2011). It is advantageous to understand hermeneutics (e.g., Packer, 2010) and read up on different intellectual movements such as post-structuralism (e.g., Foucault, 1970). At a minimum, researchers must understand the meaning of different epistemologies and be able to position their own research therein. Reflect whether research questions align with the method chosen. If you are uncertain about this, request feedback from colleagues and peers (again and again if need be). If you are caught up in a quantitative paradigm, distance yourself from it (Smith & McGannon, 2018; Tracy, 2010).

In constructing an appropriate tool for investigation, focus on the specific research question(s) and keep the interview guide short and simple. Remember, you want to dig deep and be flexible. The interview guide is just that—a guide. It does not bind you to specific schemata. Respond to specific statements of interviewees. This is where things get interesting! Disruptions need to be recognized and take precedence. If possible, reflect about the results and the interview guide after each interview. Being in close contact with other interviewers and discussing results can be beneficial. Adapt the interview-guide if and when necessary.

In the data analysis phase, time is vital. The “fully automated touch-of-a-button analysis” (Clark & Sousa, 2018) is not an option. Qualitative research relies on extensive reflection and reflexivity (Finlay, 2003). And in reflecting this, I know I still have a long way ahead of myself in reading up on and mastering qualitative research. Figure 1 summarizes these lessons as recommendations to my peers.

Figure 1.

Recommendations for researchers beginning with qualitative research.

Do I regret the time put forth for this study? Absolutely not! These are my personal learning outcomes and I am a better researcher today thanks to this research failure. Nevertheless, the process could have been much easier and maybe also more effective, and the results would have been more meaningful with more experience and preparation. I invested 270 hours (= 33.5 working days) with project meetings, acquisition of interview partners, implementation and transcription of interviews, analysis and coding of transcribed material, interpretation of coded material and presentation of results. The results of the project do not justify this expenditure. My growth as a researcher, on the other hand, was worth this investment.

Finally, I want to appeal to researchers with experience in qualitative research: help us! Specifically, study programs of the Humanities and Social Sciences need to integrate courses and texts on qualitative research. Currently, qualitative research is often being negated by lecturers, researchers and journal editors alike. The few qualitative researchers who ask the detailed “why” and the “how” find themselves in a defensive mode against researchers who focus on empirical studies with large samples and standardized survey instruments. But not all questions can be answered with these survey instruments. Both academic and outside world need and profit from qualitative research and hence, early stage researchers who are willing to commit and invest in this time consuming and complex research paradigm. To promote a more balanced research field, graduates should aim to have experience in at least one quantitative and one qualitative research program after finishing their studies in higher education.