Integrating virtual reality in qualitative research methods: Making a case for the VR-assisted interview

Abstract

Virtual reality (VR) is still very much a niche technology despite its increasing popularity since recent years. VR has now reached a point where it can offer photorealistic experiences, while also being consumer-friendly and affordable. However, so far only a very limited amount of software has been developed for the specific purpose of conducting (social science) research. In this article, we illustrate that integrating virtual reality to good effect in social science research does not necessarily require specialized hardware or software, an abundance of expertise regarding VR-technology or even a large budget. We do this by discussing our use of a method we have come to call ‘VR-assisted interviews’: conducting a (semi-structured) interview while respondents are confronted with a virtual environment viewed via a VR-headset. This method allows respondents to focus on what they are seeing and experiencing, instead of having them worry about how to operate a device and navigate an interface they are using for the first time. ‘VR-assisted interviews’ are very user-friendly for respondents but also limits options for interactiveness. We believe this method can be a valuable alternative, both because of methodological and practical considerations, for more complex applications of VR-technology in social science research.

Introduction

Despite already existing for decades (the first head-mounted display dates back as far as 1968), virtual reality (VR) is still very much an upcoming technology. Because several large tech-companies have applied themselves to VR in recent years (most notably Google, HTC, Microsoft, Samsung and Facebook), VR has reached a point where it can offer photorealistic experiences, while also being consumer-friendly and affordable. The development of VR for large consumer audiences has also spurred software-developers to start designing applications to be used on those devices. Because of this, using VR in social science research has never been more self-evident than now: until very recently, a researcher who wanted to use VR in a study would have been obliged to develop their own software and possibly even hardware, which would of course be very complicated and costly. Still, while the last years have seen considerable advances in VR consumer-software, there has been quite limited adoption in wider society and a very limited amount of software has been developed for the specific purpose of conducting research. The challenge for researchers now is choosing from a wide range of products and services (most of which are not tailored to research purposes) and familiarizing oneself with a field of technology that, despite its long history, is still very much in development.

In this research note, we aim to illustrate that integrating VR to good effect in social science research does not necessarily require specialized hardware or software, an abundance of expertise on VR-technology or even a large budget. We will do this by discussing our use of a method we have come to call ‘VR-assisted interviews’: conducting a (semi-structured) interview while respondents are confronted with a virtual environment via a VR-headset. Since the term ‘virtual reality’ has been used to describe many different technologies, practices and techniques within social scientific literature, we would first like disentangle the concept somewhat in the next paragraph before expanding on our own research project.

Virtual reality: a catch-all term

The first uses of VR in social sciences go as far back as the late eighties. From that time until now, VR has mostly been used used in the fields of (social) psychology and communication studies. A few notable researchers that have devoted themselves, not just to using VR in their own research, but also to encouraging others to make use of these technologies are Frank Biocca, Jack Loomis and Jim Blascovich. More recently, VR has also become more popular in the field of criminology (Van Gelder et al., 2017). It has been used to research a plethora of different topics and subjects, ranging from anxiety and phobias, non-verbal behaviour, spatial cognition, media effects, juvenile delinquency, interpersonal aggression and has ‘also been explored as a tool for cognitive behavioral therapy’ (Fox et al., 2009: 100). It is clear that the possibility of evoking real-world sensations and designing virtual environments that respondents can experience has spoken to the imaginations of many researchers in the past three decades, although it should also be noted that its use is practically absent from certain disciplines within social science such as political science and sociology. This is of course linked to the subject matter of these disciplines, VR lends itself more readily to the interests of communication science or (social) psychology. That being said, it can also be a very useful tool for sociologists when studying certain subjects, as we will illustrate.

Fox, Arena and Bailenson outlined the three primary ways in which VR has been incorporated in the social sciences in 2009. Although more than 10 years have passed since then, the classification still holds true when looking at more recent research in social science that utilizes VR:

first, Virtual Environments have been studied by social scientists as objects in and of themselves. What is the human experience like within a VE that is similar to or different from the experiences in the physical world? [. . .] Secondly, VEs have been created with the intention of application outside of the laboratory in order to achieve real world goals. For example, surgical VEs have been developed to familiarize doctors with new medical procedures. Finally, VEs have been used as a method to study social scientific phenomena, enabling the replication and extension of real world experiments in a more controlled environment and also helping researchers create stimuli that may be too costly or impractical to achieve in the real world. (Fox, 2009: 98)

It is important to add that there are different ways to ‘design’ the virtual environments that respondents will experience. One way is to design and generate an environment from scratch using a computer programme. Another way is to ‘capture’ images of real-world surroundings using special 360° cameras. A hybrid technique is also possible, where 360° images are captured and then overlayed with computer-generated elements. The term ‘VR’ in social scientific research predominantly refers to viewing or interacting with an environment that is completely computer generated through a head-mounted display (Pan and Hamilton, 2018). There are, however, also studies where respondents experience computer generated environments in a two-dimensional format, of which ‘netnographies’, ‘a specialized form of ethnographic research that has been adapted to the unique contingencies of various types of computer-mediated social interaction’ (Kozinets, 2012: 39), are a good example. This type of research tends to treat virtual environments predominantly as ‘objects in and of themselves’ (Fox et al., 2009: 98), as described by Fox, Arena and Bailenson above.

In addition, there are also different ways in which the virtual environment can function with regards to the collection of data. Collecting data can happen both within the virtual environment itself and outside of it. In the first case, when data is collected within the virtual environment, respondents will have an interactive, often very ‘gamelike’ experience while participating to the research: using controls, they will be able navigate a virtual environment, perform tasks, make choices, be prompted questions, and so on, with data on their behaviour being recorded throughout. A very impressive application of this concept is the software ‘AVA360VR’ (McIlvenny, 2018). While we are certain that software of this kind is of great worth to behavioural and social science research, we would like to underscore that gathering data within the virtual environment itself is not a prerequisite for a successful utilization of VR-technology in a research design. Integrating virtual environments in more ‘traditional’ research methods can prove very valuable: in our research, we simulated a real-world setting using photorealistic 360° images (instead of computer generated environments) that can be viewed through a head-mounted display and collected data on how respondents react to it by conducting an interview during and after their experience. This choice was made both out of practical and methodological considerations, which we will discuss later on. First, it is important to provide some background information regarding our research.

The study: problem and theory

In 2018, we conducted a large scale user survey in 105 public libraries in Flanders and Brussels (Belgium). It quickly became apparent that a considerable number of libraries were less inviting for users of lower educational achievement. We also found that users of lower educational achievement are visiting public libraries increasingly less frequent in recent years in Flanders. Research that tries to understand or explain why certain libraries are visited less frequently by people with a lower socio-economic status, tends to focus on very tangible, practical and economical characteristics of public libraries such as the composition of the collection, membership fees or the amount of computers that are available to visitors (Glorieux et al., 2007). While these types of analyses are valuable, they fail to capture the cultural, often more implicit, dimension of public library participation. A threshold we found to be very relevant but that has received little to no attention, is the architectural style of the building that a public library is housed in. We identified a number of libraries where the ‘invitingness’ and/or ‘attractiveness’ of the library is evaluated very differently by visitors of lower educational achievement compared to visitors of higher educational achievement. We wanted to corroborate our findings and understand what it was that made these specific buildings and/or interiors designs less inviting for certain segments of the visiting public.

Research on cultural preferences and participation is a well-established field within cultural sociology. How frequently one visits certain cultural institutions or how many books one reads every year are questions that lend themselves perfectly to a written questionnaire or a structured interview. The subject of aesthetic preferences, however, is often regarded more as a domain of psychology and even philosophy, but also in those fields, the difficulties in measuring aesthetic preferences are often underlined. Augustin and Wagemans (2012) rightly see a ‘relative lack of standardised measurement instruments’ while also identifying problems ‘of a more theoretical nature: What is aesthetics, aesthetic experience, aesthetic impressions, and what is it that we are studying – beauty, pleasingness, etc.? Definitions of aesthetics-related terms differ widely between authors, and terminology is far from systematic’ (p. 456).

Practical challenges and the added value of VR

Bourdieu’s (1968, 1991) concepts of habitus and cultural capital provide a very interesting framework for social scientists to study class-based dispositions to aesthetics, but choosing architecture as the subject of a study regarding aesthetic preferences (and how these preferences interact with education level and class-membership), is far from self-evident. One could choose to follow Bourdieu’s example (as he did with museums in ‘Distinction’) and invite respondents to visit a set of libraries and afterwards ask a series of questions in a semi-structured interview, but considering first that we want to compare reactions to (at least five) different libraries (that were found to be experienced and evaluated very differently through our exploratory analyses) and second that these libraries are not located closely to one another (a car-ride between the two libraries that were identified through our exploratory analyses and that are furthest apart would take approximately 2 hours), this is not a feasible option. There are also other practical challenges to take into consideration. Say we would organize such visits and take groups of 5 to 10 people on a day-long field trip across Flemish libraries: it is not only highly unlikely that many respondents would be willing to devote so much of their time, but the circumstances (weather conditions, amount of people in every library, type of visitors in every library, etc.) could change drastically from one visit to the next. In short: any kind of experimental control would be almost impossible to achieve.

Therefore, it would be preferable if respondents could experience the buildings through an (audio)visual medium. Using photography or video solves the practical problem of having to take groups of people on day-long field-trips, makes it possible to evaluate several buildings in a relatively short amount of time and allows for more experimental control since every respondent would see the same images or video. But using photography or video confronts us with another problem. Architecture is distinctly experienced in a three-dimensional way: the immersive nature of buildings and spaces greatly shapes our experience of them. In that regard, film and photography, as two-dimensional mediums, fall wholly short of evoking the experience one has when standing before, or entering, a building. VR offers the perfect solution: it can facilitate very immersive, three-dimensional experiences, while also possessing all the advantages of more traditional media.

For those who are unfamiliar with how 360° images can create a realistic virtual experience of an environment or a space, we would like to refer to the figures that are featured in the Supplemental Appendix, where we also provide a more detailed account of some practical and technical aspects of our research design. We hope to provide some tangibility on what it means to use consumer level VR-technology in the practice of social science research. We believe sharing this kind of information is essential, since it will most likely be technical and practical thresholds that will prevent many researchers from adopting VR-technology in their research. McIlvenny (2018) rightly underscores that integrating VR into research methodology ‘will be a steep learning curve for new generations of scholars’. It is also rather telling that Jewitt et al. (2016) found ‘a wider range of methodological innovation’ (p. 6) among researchers in artistic research fields than in social sciences. By the nature of their expertise, researchers in the field of arts will feel much more comfortable in experimenting with new mediums and technologies. Therefore, to whom it may concern, we go into more detail on working with consumer-oriented VR-technology in the Supplemental Appendix.

We would also like to add here that, while it is certainly true that a sense of scale and space is especially relevant when researching the effect a specific style of architecture has on its visitors, we are of the opinion that most research that aims to simulate a real-life situation or experience with visual media could benefit greatly from using 360° video that can be viewed with VR-goggles. At the very least, since its application has become much more self-evident in recent years, it should be considered as an option. After all, using two-dimensional video to simulate real-world experiences has serious drawbacks, as McIlvenny (2020) also underlines: ‘media scholars have critiqued the dominant use of (2D) video recordings because of, for instance [. . .] the limitations of video to “capture” reality, (and) the screen essentialism and planocentrism of flat video’ (p. 3). Using VR-technology solves several of these ‘scenographic’ problems, not in the least those that pose themselves with regard to the ‘praxeology of camerawork’, ‘the ways in which camera technique shapes what can be “seen” in the resulting recordings’ (McIlvenny, 2020: 5). Still, while 360° video largely solves the problem of ‘framing’, we need to acknowledge that other challenges still exist: any ‘capturing’ of a natural or social setting will always involve making creative and practical choices that require deliberation: the colour temperature of daylight, for instance, changes throughout the day and different light circumstances are known to evoke different feelings (Park et al., 2013).

Making a choice: collecting data within or outside of the virtual environment

While it was clear from the beginning that we would have to film ‘real-world’ environments (libraries that were found to be experienced differently according to education level), not computer generated ones and that these video’s would be viewed using VR-goggles, a choice still had to be made with regard to how we would collect data. Our first option was to develop our own software (‘AVA360VR’ was still in development at this point in time) that would have likened the experience to that of a videogame: among other things, respondents would be able to choose in what order they would visit the different buildings and spaces and how much time they would spend in them. The software would also have a built-in question-and-answer module: upon spending some time in a certain space, questions would pop up that could be answered using a handheld controller. Respondents would be able to, for example, choose from a list adjectives they feel best describes the space they are viewing, as well as allow them to give a score for (certain aspects of) that building or interior. Answers would be immediately translated to a spreadsheet, ready to be imported in statistical software. The viability of this concept was explored with a developer that had experience designing apps for VR. Neither the fact that we would essentially be developing our own VR-game that used photorealistic environments, nor the integration of a Q&A-module or the translation of respondents’ answers to a spreadsheet would apparently pose any real problems: several development platforms, most notably Unity, allow for all of those functionalities.

However, it was quickly decided that we would not pursue this option, both out of practical and methodological concerns, the biggest of which was the user-friendliness of the whole experience. As mentioned in the introduction, VR is far from a new technology, but it is still very much a niche product despite its increasing popularity. Familiarizing oneself with how VR-environments and VR-applications work takes time, even for very tech-savvy users. Another very important consideration was the fact that a more qualitative methodology was much more in line with the subject of aesthetic preferences. In that sense, offering a list of predetermined adjectives to describe a building or space, for example, creates a danger of steering the experience and evaluation of those spaces. We therefore decided to opt instead for an approach that would require no competence in dealing with VR-applications on part of the respondents: they would simply have to put on VR-goggles and edited clips (showing the different libraries) would automatically start and stop (being triggered remotely by the interviewer). The application ‘VR-Sync’ provides exactly this functionality: the interviewer can ‘push’ a selected video to the VR-goggles the respondent is wearing and then ask questions regarding the virtual environment they are in. During and after viewing, respondents would answer questions that let them share their thoughts on what they are seeing, while everything they said was being recorded. This method allows respondents to focus on what they are seeing and experiencing, instead of having them worry about how to operate an application they are using for the first time. It’s also worth mentioning that the user-friendliness of this method also extends itself to the researchers: as far as the actual analysis of the interviews is concerned, for example, there is hardly any difference here with how one would transcribe or analyse any other series of semi-structured interviews.

It should be noted that, since there is no truly ‘interactive component’ to this experience, we consider what our respondents experienced to be a variation, rather than a real ‘example’, of what McIlvenny (2020) describes as ‘inhabiting video’: ‘exploring complex spatial video and audio recordings of a single scene’ (p. 7). Respondents could comment on what they were experiencing and were asked questions, but they could not intervene in or change their environments. Our research is also not an example of respondents inhabiting a ‘digital body’: ‘corporeal fleshy bodies interacting with digital technology but also [. . .] corporeal bodies (that) are momentarily digitally re-presented’ (Jewitt et al., 2016: 1). There is definitely a kind of ‘corporality’ to the experience though, since a ‘first person’ viewpoint at eye level is maintained throughout the VR experience. As a technique, VR-assisted interviews put their own spin on ‘interactiveness’ and ‘corporality’, one that we would argue fits how we experience architecture in the real world: when we view a building for the first time, we generally do not start changing the environment to any considerable degree (nor could we), but we might tell the person standing next to us if we like it or not.

Experiences with VR-assisted interviews in the field and quality of representation

In the end, we were able to conduct a total of 73 VR-assisted interviews in which each respondent ‘visited’ five real-world libraries, albeit virtually. We initially started recruiting respondents by visiting social organizations in different cities in Flanders (Antwerp, Ghent and Leuven). A little over half of all respondents who had no higher education were found through these organizations. Next, we visited a college and a marketing bureau (both situated in Antwerp). This was done to ensure we had a sufficient amount of younger and higher educated respondents. Students and employees were (randomly) asked if they were willing to participate (in person). For the remaining interviews, we turned to five public libraries which had participated to the large scale user survey, asking them whether we could interview respondents who had filled out a questionnaire for their library and had made themselves eligible for follow-up research. To be clear, the libraries we contacted with this question, were not the libraries that were shown through VR; users were not asked to evaluate the building of the library they visit most frequently. We sent out invitations via email, asking potential respondents whether they could come to their own local library for an interview on certain predetermined days. The good thing here was that we had a lot of information regarding their profile, which allowed us to make sure that we had a sufficient number of individuals with lower education levels across different age groups here as well.

On average, these interviews lasted 22 minutes, with the shortest being completed in 13 minutes and the longest in 34 minutes. The VR-assisted interview was accompanied by a short written questionnaire in which respondents were asked to answer a series of socio-demographic questions. When looking at the distribution of basic background variables (see Tables a through c in the Supplemental Appendix), we see that we were able to reach quite a balanced composition of respondents with regard to education-level and age, but we did interview considerably more women (44) than men (27) (two respondents did not answer the question regarding their gender). In addition to this written questionnaire, we kept a log in which we noted several things about how each respondent reacted to the experience, more specifically:

• The respondent’s general attitude towards the research;

• His or her familiarity with virtual reality;

• How smooth the interview went (both in terms of how easy it was for the respondent to participate to the research and any technical or other difficulties that came up);

• Any information pertaining to the respondent that we felt might be relevant but would not show up in the questionnaire.

This allowed us to make an overview of how these 73 respondents dealt with our rather unusual research design. First and perhaps most importantly, it should be noted that for 43 of our 73 respondents, this was their first time experiencing VR in any way, shape or form. Another 24 respondents had limited experience with VR, ranging from experiencing it once in a museum to riding a rollercoaster that featured VR. Only six respondents had extensive experience using VR, meaning they themselves (or someone in their family) actually owned a pair of VR-goggles and used it frequently. Only one background-variable seems to be a good indicator for predicting whether someone will have had previous experience with VR or not. Perhaps unsurprisingly, this is the age of a respondent. The number of respondents who have no experience whatsoever with VR steadily rises with age (see Table d in the Supplemental Appendix). Other than that, it seems the familiarity with VR is quite evenly distributed when looking at gender, education level and even income categories.

We considered it quite remarkable that only one respondent seemed particularly impressed when experiencing VR-technology for the first time. Furthermore, only a very limited number of people made a comment of any kind regarding the technology when seeing a virtual environment for the first time. Some respondents were unsure at first whether they could actually look around the environments in a way that resembled real life and were only mildly surprised when they realized they could. Almost all respondents immediately and unreflectively started looking around and evaluating the surroundings, buildings and interiors without giving any notice at all to the fact that they were experiencing something through VR-technology. We believe this is is one of the best indicators that VR in the way we used it (photo-realistic, 360° videos that are triggered by the interviewer) has the potential to invoke a real-world experience. Furthermore, it is also noteworthy that the majority of interviews were conducted without any sort of difficulties or problems whatsoever. For the sake of completeness, and because the circumstances in which we encountered some of these problems might be relevant for other researchers who are considering using VR in one their studies, we discuss them in more detail in the Supplemental Appendix.

Of course, the fact that problems (technical or otherwise) were mostly absent, is no measure for the actual ‘quality of representation’. The most important thing, after all, is whether this technique gives a faithful representation of the buildings and spaces in question. This question was also addressed during the interviews. Every time we showed a library to a respondent, we asked if they recognized the building and whether they had visited it before. If they answered ‘yes’ to both questions, they were asked whether or not the building was presented in a faithful way and if we had been able to successfully capture the look and feel of the building. There were 15 instances in which a library both was recognized and had been visited by a respondent before. Nine out of those 15 respondents said unequivocally that the VR-environment was a faithful representation of the building in question. Four respondents said it was a somewhat faithful representation of the building in question and one respondent felt that it did not represent the library in question in an accurate way. When asked what exactly made the representation suboptimal or problematic, most of the remarks had to do with the fact that the images failed to convey all the different functions and aspects of the library (it is probably not a coincidence that all these more reserved responses pertained to the largest library we showed), stating that the shown video’s gave an incomplete impression of the library (e.g. respondents mentioned that we did not show the coffeebar inside the library). While this is of course a valid remark, it should be noted that it does not pertain to the aesthetic or spatial qualities of the building. In other words: the space did not seem a lot bigger, smaller or more or less attractive than in real life.

Considering how easy participation was for even the most VR-inexperienced respondents and that the data gathered was perfectly suited for analysis, we conclude that opting for VR-assisted interviews (instead of a more advanced, ‘gamelike’ method) was the right choice.

Conclusion

In many ways, we have made an argument here for a very ‘modest’ application of VR-technology in social science research, one that stays well within the bounds of what VR-technology currently has to offer. More than trying to reinvent the wheel, we feel we have ‘resituated’ a method, ‘that is using a method in a new context but within the method’s original discipline’ (Jewitt et al., 2016: 6). This does not lessen the relevance of our application of VR-technology, since ‘resituating’ ‘can elaborate a method by pushing its limits, bring new perspectives and agendas to bear on a method, and (support) experimentation’ (Jewitt et al., 2016: 6). ‘Elaborating’ the semi-structured interview by confronting respondents with lifelike, immersive environments has, at the very least, proved of great worth in tackling our own research questions. Therefore, we would argue that there is no real ‘wrong’ way to implement VR in social science research, as long as its practical and methodological limitations are recognized and taken into consideration. We consider our research an apt example of how an experimental approach to a subject can be (relatively) modest from a technical or technological standpoint, but still provide real added value. That being said, we are also convinced that ‘gamelike’ VR research designs will open up a world of possibilities and we expect its toolbox to expand in significant ways as VR-technology advances: perhaps soon it will be possible to measure biometric data such as a respondent’s pulse or pupil dilation upon experiencing a specific event in an integrated way for example.

Finally, we want to underscore that complex research designs can have their drawbacks, especially if the technology they use has not seen widespread adoption in society: if your wish is to conduct a considerable amount of VR-assisted interviews at this point in time, expect to spend more time on one respondent, or more respondents to not be able to finish, as the user-experience becomes increasingly complex. This will likely change in the future as VR becomes more common, but at the moment is still very important to take into consideration. Another important limitation connected to the fact that VR is still a relatively niche product, is that this also severely limits the possibility for large-scale data-collection. Similar to online written questionnaires 20 years ago, you can’t expect to launch a VR-research online now and expect a diverse and representative sample of society to participate. This is why we felt it important to share our experience in integrating VR-technology in a qualitative research method that proved easy to use for both researchers and respondents of different ages and education levels.

Supplemental Material

sj-pdf-1-mio-10.1177_20597991211030778 – Supplemental material for Integrating virtual reality in qualitative research methods: Making a case for the VR-assisted interview

Supplemental material, sj-pdf-1-mio-10.1177_20597991211030778 for Integrating virtual reality in qualitative research methods: Making a case for the VR-assisted interview by Dennis Mathysen and Ignace Glorieux in Methodological Innovations