Qualitative methods III: On different ways of describing our work

Abstract

In two previous reviews, we examined how human geographers currently report on projects involving their preferred qualitative methods – interviews and ethnographic observation. This final review steps back from specific techniques to evaluate some of the broader presentational conventions that typify this work. What can be inferred from where these geographers discuss data collection in their papers? Why do they develop new methods and what do they say about fieldwork failures? How often do they reflect on the provisional status of their findings? And what are the implications of how they define their purpose in working with qualitative material?

I Introduction

This is the third in a series of three reviews that take the temperature of qualitative research in human geography through a focus on prevailing modes of practice and presentation. Drawing on 200 papers that were taken to represent this field, and which were detailed more fully in the first report, we have sought to identify and evaluate some of the conventions that currently define this work. In doing so, we have introduced figures such as the ‘invisible interviewer’ (Hitchings and Latham, 2019a) whose disappearance was connected to how, for those using our most popular method, certain aspects of the experience of conducting research are now infrequently discussed, and the ‘elusive ethnographer’ (Hitchings and Latham, 2019b) whose ghostly presence in our papers partly bore witness to how practitioners were assumed to have quietly overcome any problems in the field. Stepping back from specific techniques, this final review draws on the full sample of papers to examine a number of broader writing conventions. These include where and how methods are presented, how novel strategies and fieldwork failures are discussed, what is said about how others might take our findings forward, and the phrases that we use to convey our essential aim in writing papers that consider the results of qualitative research. We end our reviews by reflecting on why some conventions may be taking hold and why, as a community of scholars who collectively define this field, we might want to encourage others.

II Following the methods section

We began our third review by looking for where methodological summaries were found in our sample of papers, effectively ‘following the methods section’ by pinpointing where it was found within each of them. In line with the inductive aims of many qualitative researchers, we then used this exercise to build a typology of current practices. After excluding those ‘methods papers’ whose focus was more squarely on technique, we identified four types.

First, we saw the ‘obvious methods section’ in which there was a dedicated collection of paragraphs under a title that straightforwardly referenced ‘methods’ in some way. Second, there was the ‘augmented methods section’. This was similar to the ‘obvious methods section’, but it also included other considerations, and it often came with a more evocative title that served to position it as more engaging than an unadorned account of data collection. Commonly it included extra contextual detail, and the result was often that discussion of the practicalities of fieldwork ended up being squeezed into one or two paragraphs. Third, we identified a set of papers with a ‘mobile methods section’. These sections were trickier to track down since doing so required us to scan the whole paper in order to find the relevant discussion. Sometimes it was tucked away under an unrelated heading. Most often it was found at the end of the introduction. Occasionally we eventually unearthed it from the footnotes. Fourth, there was a final group of papers that we took to have a ‘missing methods section’. Within them, not more than a few sentences on data collection were provided. In these cases, we were left with a sense that the authors thought that their methods should be mentioned in passing, but that the details of how they carried out their projects were either so obvious or so uninteresting that they were not worth wasting precious words on. Figure 1 shows the frequency of each type in our sample.

Figure 1.

Typology of ‘methods sections’.

What does this say about how these geographers relate to their methods? Over half of the papers in our sample were committed to presenting the reader with a dedicated fieldwork overview so they are better placed to evaluate the authority of the analysis. But it is also true that over a third have either sidestepped the methods section entirely or bundled it into another chunk of writing. We can only speculate on whether this arrangement was built into these papers from the start or whether it is an outcome of peer reviewing processes that have encouraged authors to prioritise the discussion of other matters. But at the very least this pattern suggests that qualitative human geographers currently have an ambivalent relationship with method. There is no consensus on how much and where data collection strategies should be discussed, and an eagerness to get past such prosaic matters pushed method to the margins of a good number of papers in our sample.

III Arguments for innovation and forbidden failures

As discussed in our first report, it is common to see ‘progress’ in qualitative human geography as something that comes by adding to the ‘toolkit’ of methods that we collectively share. Viewed in this way, the best qualitative research is defined by a focus on – or openness to – methodological innovation. But is frequency of innovation really a good proxy for disciplinary health? Another interpretation might come from recognising broader cultures of research evaluation in which demonstrating ourselves to be methodologically ‘original’ or ‘innovative’ can sometimes seem to be at least as important as the actual effectiveness of any such innovations (Travers, 2009). Adopting this kind of questioning stance on what novel approaches really mean, next we turned to what those who in some way presented themselves as methodological innovators (either quite quietly or in a more strident manner) said about why they wanted to try something comparatively new.

We identified 40 papers that met this criterion. We should therefore start by noting that the majority of us are, in fact, drawing on the benefits of established techniques without feeling any need to innovate. Within that 40, however, innovation was most often underpinned by the belief that new concepts encourage new techniques. A focus on embodiment, for example, can be taken to demand new approaches to how environments are inhabited. Examples of those who ran with such an argument include Finlay and Bowman (2017), Adams-Hutcheson (2019) and Palmgren (2018), who all explored talk on the move, and Paterson and Glass (2018), who reflect on videoing ‘bodies-in-place’. Interest in ‘non-human’ actors also continues to inspire innovation. For example, Kelsey et al. (2019) developed an auto-ethnographic strategy for studying the material practices of food provisioning and Evans and Adams’ (2018) work on Kenyan elephants combined animal tracking and more conventional forms of interviewing.

Then there is the idea that new technologies present new methodological opportunities. For example, Von Benzon (2019) drew on online blogs in her home-schooling study to examine what should be considered ‘public’, and Matulis and Moyer (2018) used Twitter archives to study the practical production of ‘counter publics’. Holton and Harmer (2019) thoughtfully discuss how they worked with smart phones, and Garrett and Anderson (2018) argue for a ‘critical drone methodology’ that carefully considers the societal spread of these devices. Finally, we saw innovation coming from those interested in what is usually called either ‘participatory action research’ or ‘action research’. Within such approaches, innovation is often regarded as a natural by-product of collaboration in support of those being studied. For example, working with green building practitioners, Preller et al. (2017) discuss the effectiveness of the ‘world cafe method’ – a technique involving a series of informal group discussions – as a means of co-producing knowledge. Hayes-Conroy (2018) used participatory mapping and co-organised a symposium, along with interviews and observation, in her study of Columbian anti-violence movements. And Brown and Tucker (2017) tell us about how they set up an anonymous phone line to record the testimonies of Peruvian women who had been subjected to unconsented sterilisation.

Whilst there is good reason to be excited about the various strategies that are being developed here, we also noted how infrequent it was to speak openly about projects not going to plan within all three camps. We might see this as being beside the point. If the exercise is about keeping pace with theoretical developments, responding to new technological opportunities, or delivering on an ethical commitment to helping communities, how could we fail – we have already done our job by being willing to try something different? But, if we want to learn collectively from our experiences of doing research, we might have expected the innovators to be most likely to talk of practical teething problems and the amendments they would make next time. Perhaps this is less the case for the participatory action researchers who are determinedly open to the uniqueness of any given research situation. But it seemed to us that there was a missed opportunity here for a more straightforward reflection on how we respond to new theories and technologies in terms of which research strategies are proving more or less effective. New methods can’t always work.

Such reflections are, however, currently taboo in qualitative human geography research. In one of only two papers in our sample to talk explicitly about fieldwork ‘failures’, Harrowell et al. (2018) note how often any hint of failure is masked by the ‘lexicon we use to camouflage our mistakes’. We have to agree with them. In undertaking this review, we found some useful passing discussion of ‘challenges’ that were eventually overcome (getting individuals to talk in Preller et al., 2017, to secure the interest of potential participants in Hacking and Flynn, 2018, or to encourage people to recollect their past motivations in Sindre, 2018). But our colleagues almost never risk presenting the reader with cautionary tales of things going wrong. One paper that came close was by Palmgren (2018), who bravely mentions how some of the teenage girls in her study were unwilling to embark on a walking interview. In so doing, we felt she was moving towards an important point about how human geographers should not presume that all social groups will be bursting with enthusiasm and insight when presented with a technique that is still in development in our discipline (see Finlay and Bowman, 2017). Yet her conclusion pulls back from this. We wonder whether this was partly about how we are socialised into particular ways of reporting on our studies, and how, within that, experiments with innovative methods are expected to be presented as successful. The other paper that spoke frankly of the authors’ experience with unsuccessful methods was by Schoenberger and Beban (2018). For them, a truer understanding of everyday life in Cambodia only became possible once they reframed fieldwork ‘failure’ as a learning opportunity instead of a shameful experience that should really be downplayed.

If more of us were willing to think and talk in this way, we’d be in a stronger position to refine our methods and do better work. In the spirit of that idea, we admit that we too have glossed over some of our own fieldwork failures. For example, we would both be reticent about being too open with our brief when asking people to write diaries for us in the future. Though we stopped short of saying this in any of the associated published pieces, we have both found that doing so can lead to such idiosyncratic entries that it becomes almost impossible to extract any kind of overall conclusion from the exercise. Our reviews leave us thinking that a more candid way of reporting on how well different methods work for us could really benefit qualitative human geography, and we applaud the few researchers found in our sample who have been chipping away at the current convention in which any mention of fieldwork failure is forbidden in our papers (see also Caretta and Jokinen, 2017, on the importance of such an exchange in human geography).

IV Is further research really needed?

One of the pieces of advice that many geographers give to their students is that they should remember to discuss how ‘further research is needed’ when they come towards the end of writing their project reports for us. The point is to encourage them to see their efforts as part of a broader conversation about how researchers work towards a fuller understanding of the empirical topic or context at hand. We offer them this advice because we know that it can be tempting to leap directly from the findings of an individual study to more definitive statements that do not always hold water. But do we practise what we preach in our own papers?

In our sample, the answer was not a great deal. We found that fewer than one in five papers spoke of how future projects might build on the presented findings. Of those that did, in roughly half the cases the argument was that this should involve the same method in other contexts. Alkhalili (2017), for example, encourages us to explore whether ‘enclosure from below’ works in a similar way elsewhere to how it was observed to happen in their Palestinian case. Kelsey et al. (2019) say more social groups should be looked at in terms of how they do budget shopping. Leahy et al. (2018) argue that studying how others, in addition to those from the organisations with whom they spoke, are responding to hostile housing policies in the UK would be a useful undertaking. In these last two instances, however, the willingness to discuss results in this way was partly based on the researchers’ intention to do this work themselves since both papers were reporting on initial project findings. Graf (2018), Schurr and Militz (2018) and Ruwanpura (2018) also point to other groups whose potentially different perspectives would provide a more rounded appreciation of what they found in their own projects (different generations of ethnic diaspora; others facing the challenge of combining artistic and academic work; those who occupy another set of positions in the garment factory workforce). Similarly, Faulconbridge and Muzio (2015) tell us that we should examine other cases in order to know whether the results of their study hold true elsewhere. So do Hacking and Flynn (2018) and Sindre (2018).

This is in line with what we might expect from qualitative researchers whose necessarily small sample sizes make them especially cautious about generalisation. Perhaps this is particularly so for geographers primed to expect variation not just between groups but also across space. But it was still relatively rare to see papers recognise the provisional status of their findings in this way. What was also perhaps surprising, given that many geographers are well versed in the benefits of mixed-method research, was how infrequently we found papers discussing how other researchers might respond by taking a different approach to their own. Kalafsky and Duggan (2016) speak of the value of further studies with larger sample sizes in generalising from their research on Nova Scotian exporters. Siebers (2017) makes a similar suggestion with reference to their work on international firms in China. These authors were perhaps especially mindful of how business audiences might prefer quantitative evidence. Finally, amongst the few researchers in our sample who do not turn to the recognised complementarity of quantitative and qualitative methods (Philip, 1998), Schurr and Militz (2018) speak of how longitudinal research could provide a fuller sense of how commodification has infiltrated the world of international surrogacy. We also have Qian and Kong (2018) who were presumably alive to the prevalence of interviewing in human geography research when they acknowledge how their discourse analysis of faith schooling in Hong Kong might be supplemented by our most popular approach.

But these papers were relatively exceptional. To talk about passing the baton to others more skilled in alternative techniques perhaps strays uncomfortably close to an admission of our own shortcomings: better to stick with the positive and avoid the matter of whether and how further research might be needed. We will return to why presenting our papers as steps on an incremental journey towards understanding particular sites and processes might currently feel like a professional risk. For the moment, however, it is worth noting that, if qualitative geographers really want to take the idea of combining methods seriously, they might speak more openly about the comparative limits of their own studies.

V Exploring a case, making an argument, advancing a theory

We ended our review process with a final further typology. This focused on how authors saw the purpose of their data collection activities because we think that all of the above findings are connected to that. We observed geographers asking their empirical material to do a great variety of things – from ‘providing accounts’ (Bunnell et al., 2018) and solving ‘empirical puzzles’ (İşleyen, 2018), to ‘shedding light’ on an issue (Loyola-Hernández, 2018) or telling stories of lived experience (Lahiri-Dutt and Chowdhury, 2018; Kölbel, 2018). The results of qualitative studies were also used to ‘teach us’ about particular contexts (Gordillo, 2018) or to ‘complicate’ the pre-existing understandings that we may have (Naylor, 2018). We saw a great diversity of phrases being used by these geographers to capture the purpose of this work (and we would like to study how some have become fashionable and others have fallen out of favour). Nevertheless, from this diversity we eventually developed a profile characterised by three (sometimes overlapping) objectives (see Figure 2).

Figure 2.

The frequency with which particular purposes were claimed in our papers. The total number exceeds 100 because some papers claimed more than one purpose.

Firstly, there were what might be called the ‘empirical papers’. These concentrated on presenting detailed data-driven accounts of a phenomenon. Within these, there was often an emphasis on describing ‘a case’ or ‘a case study’, and this was generally justified in terms of either exploring an unresearched area or extending or adding to existing empirical work on a particular real-world phenomenon. Thirty-seven per cent of papers were, at least in part, explicitly seeking to do this. Secondly, we saw a group of papers that saw themselves as focused on ‘making an argument’. The notion of making an argument implies that the point of our papers is to convince the reader. In some cases, this might be that a particular topic, or empirical focus, is important. In a small number of instances, this involved making the case for qualitative research in areas where such approaches have been absent (for example, Medby, 2018). Twenty-six per cent used their qualitative data in this way.

Thirdly, there were papers whose primary focus was on considering the value of a chosen theoretical perspective or framework. This was the most frequent way in which qualitative data were used. Half the papers in our sample used their qualitative data in this way. That is a lot. In these cases, empirical material often had a relatively subservient role. If the point is to advance a conceptual argument, qualitative data is recast by these researchers as the resource on which the author ‘draws’ to ‘demonstrate’ or ‘illustrate’ the potential of such a move. Doing so can often end up backgrounding empirical nuance since what really matters is the audacity and alacrity of the theoretical argumentation. The popularity of this third purpose helps us to understand why the ‘methods section’ can sometimes end up in the footnotes and why specific ‘further research’ projects were relatively infrequently discussed. The former can feel like an unwelcome distraction from the building momentum of the conceptual argument in papers such as these. Regarding the latter, if we see our work as part of an empirical journey towards a collective understanding of identified places and processes, we would naturally speak of next steps. But, if the aim is to showcase the potential of a conceptual approach, that can be done with a single case alone.

The phrases that we use to describe our purpose make the discipline that we are. So, do we, as individual geographers, want to see ourselves as ‘arguers’ whose objective is to convince the reader irrespective of any doubts we may be harbouring about our arguments? Do we so often want to see ourselves as ‘theoreticians’ for whom the empirical world is sometimes little more than a springboard for our conceptual work and sometimes a place to be unproblematically plundered for examples that demonstrate the applicability of the theories on which we have already settled? Turning to other candidate caricatures, in view of the reflexive character of much qualitative research, we might have expected to see more humble ‘social scientists’ who, within their papers, go through the stories of their studies before turning to the various claims and interpretations that can reasonably be associated with them.

VI Different forms of positive academic impact

Our third and final review has examined current conventions of presenting qualitative human geography research in journal papers. Unlike most of the reviews that preceded our own set of three, which more directly delivered on their allocated task of identifying and evaluating new developments in this field, we took a different approach in order to question some of the unwritten rules that inform our practice. This review has gone some way towards deepening our understanding of the limited degree to which relevant geographers speak of their practical fieldwork experiences that was noted in the previous two. Dwelling on such matters would position us quite differently to how we now often feel obliged to present ourselves.

Why is this when we know that many of us are privately willing to share tips and ideas and to speak frankly and fully about the implications of our data collection experiences? Though we would need to examine how geographical writing practices have changed over time to be sure, we think this may reflect a current climate in which the pressure is on to hammer home how every paper that we write is somehow ground-breaking. When the quality of our research is increasingly evaluated in terms of how individual papers force us to rethink important societal issues or make apparently landmark arguments about the value of conceptual approaches, publicly lingering over our variously successful attempts at undertaking effective fieldwork doesn’t sit so well with such ambitions. Speaking of the merit of further research can also seem like the last thing we should be doing when we are encouraged to signal our own academic significance at every turn. And discussing our failures can feel like professional suicide when pursuing the prestige and personal advancement that come from papers that impress in the right kind of way. In qualitative human geography, these pressures may be further compounded by longstanding anxieties about how, were we to open up the black box of our data collection too fully, we run the risk of our empirical material being immediately decried as anecdotal evidence instead of a sufficiently sturdy platform for the weighty arguments that we now often balance on top. In this climate we should be wary of our journals becoming the venue for a strange pageant of stylised pieces instead of a genuine exchange about what can be learnt from our studies.

Could this be our version of how, in other fields, some researchers feel compelled to present only positive results because anything less now risks rejection? And, if it is, could we push back against these pressures to nurture other writing conventions that more explicitly and publicly value a more candid discussion of method – ones in which we routinely share more of the trials and tribulations, along with the excitement and the adventure, of doing qualitative studies? Doing so may require some effort when our reviews suggest we currently tend to either gloss over our methods when we adopt a more commonplace approach or feel we should emphasise our successes when we have invested in the idea of doing something demonstrably different. Then there are the space constraints of journals when we know that qualitative papers can eat up words as we present sequences of quotations or vignettes to establish our authority and illustrate our findings. But perhaps we should cut back a little on those and say more about how we came by them in the first place. The result could be a healthier ongoing conversation about the realities of research. In terms of taking some steps in this direction, we see a number of interesting methods issues that would benefit from more collective consideration amongst qualitative human geographers. Experiences with specific interviewing approaches, sampling strategies and their implications, how exactly we go about ethnographic observation: these all jump out as prime candidates for a fuller discussion in this field. It is odd that we aren’t already talking more about them given how fundamental they are to so much of our work.

So, we end by highlighting the many possible routes to positive academic impact and how, whilst different papers have different purposes, talking fully, honestly and openly about our attempts at collecting and evaluating data remains an important, but sometimes overlooked, way of ensuring that communities of researchers really flourish. Writing these reports has been a nerdishly enjoyable exercise and we finish feeling proud of the determination with which many of our peers have gone about applying their chosen methods. But we must admit that it has not always been easy to come away with a strong sense of what exactly they did in terms of their practical fieldwork and why. And we are concerned about how the entrenchment of certain writing conventions may be encouraging us to speak less about what happened in our studies and the implications of that. Shying away from these matters means that our colleagues are less able to build on our efforts by refining our approaches or organising the most effective further research. It also means that outside audiences – if one of our hopes is for others to consult our papers – may remain less than convinced by the arguments that we make within them.