Sage Journals: Discover world-class research

Abstract

There has been limited adoption of Immersive Journalism (IJ) by the audience; simultaneously, the audience’s perspective is rarely considered in the production and research of IJ. At this point, however, it is crucial to incorporate an audience perspective to identify potentially unintended effects of IJ and improve on the innovation of IJ. This study investigates the audience’s experience and evaluation of IJ by qualitatively analyzing their thoughts after viewing two IJ cases. Our results indicate that the audience may pick up on intended effects, such as a sense of presence and an intense emotional experience, but some also express unease towards these effects. Furthermore, the audience struggles to comprehend this study’s two immersive journalistic cases as part of the journalistic genre. These findings provide insight into the gap between the initial hype and the current reality of IJ and provide the basis for propositions for future IJ productions.

Keywords

Audience studies innovation journalism research methods: qualitative technology

A radical audience perspective of the experience of immersive journalism

Much scholarly interest has focused on Immersive Journalism (IJ) as one of the most notable innovations in journalistic production during the last decades (Doyle et al., 2016). Despite a decade of experimentation with the use of IJ, the way to relay journalistic information to the audience via immersive technologies does not seem the success story it was initially thought to be. After an initial rush, the number of new productions of IJ is steadily declining during the last couple of years (Paíno-Ambrosio and Rodríguez-Fidalgo, 2020; Sirkkunen et al., 2020). Fittingly, studies indicate that the audience does not extensively use this form of journalism (Wang et al., 2018), partly due to VR goggles’ relatively poor market penetration (Mochizuki, 2023). Moreover, research suggests that not all users appreciate the different dimensions of IJ productions (Greber et al., 2023a). Thus, IJ seems stuck in the first stage of media evolution: “improvements on the old medium” (Stöber, 2004: 503). Yet, we may help close the gaps between improving on an old medium and the next stage in media evolution, namely the “emergence of a new medium” (Stöber, 2004: 503), by examining what works and does not work about IJ.

In its current form, IJ was first introduced by de la Peña et al. (2010) in their article on using Virtual Reality (VR) in journalism. In it, the first accounts of audience experiences were presented: participants reported having felt “present” at the depicted Guantanamo Bay prison and felt as if they embodied the crouched body of the prisoner whose avatar played the central role. Soon, IJ was adopted by journalists and producers due to its apparent emotionally engaging capabilities (Goutier et al., 2021). Following this, a growing number of studies immediately focused on the possible effects of IJ on an individual’s sense of presence (de la Peña et al., 2010), emotional responses (e.g. Greber et al., 2023b), and empathy (e.g. Schutte and Stilinovic, 2017) - using a range of research methods such as experiments or qualitative interviews.

While the study of these effects is highly relevant, only few studies have systematically investigated what de la Peña initially describes, namely the experience users have while watching IJ productions with different characteristics (for audience studies, see Godulla et al., 2021; Jones, 2017; Nielsen and Sheets, 2019; for user experience see: Shin and Biocca, 2018; Vàzquez-Herrero and Sirkkunen, 2022). This is surprising, given the “audience turn” in journalism research during recent years, which is aimed at understanding not just top-down assumed effects but also at exploring bottom-up perceptions of audiences to improve a fragile trust relationship between journalism and its audience (Swart et al., 2022). This becomes even more important, as studies show that IJ audiences, their personal context and intention strongly inform the experience of IJ (Shin and Biocca, 2018; Shin, 2019).

Thus, to close the gap to the second stage of innovation--the “emergence of a new medium”--we need to understand not only the motivations of journalists producing IJ, but, even more so, how the audience experiences IJ. In light of the relative shortage of the audience perspective within current research and the importance of comprehending the audience‘s evaluation of their experiences to produce valuable immersive products, it is essential to consider how the audience experiences and evaluates IJ. To take on an audience-focused perspective, we qualitatively analyze participants’ thought-listing responses after experiencing one of two IJ cases.

Immersive journalism: An effect-driven innovation?

IJ developed against the backdrop of the emotional turn in journalism, which aims to strengthen the audience’s affective relationship and understanding of a news event (Beckett and Deuze, 2016). Through using inclusive technologies, the audience can experience so-called “presence” (Cummings and Bailenson, 2016), witness an event unfolding (Nyre and Vindenes, 2021), and understand an event both cognitively and emotionally (Bujić and Hamari, 2020).

The current literature often conceptualizes IJ productions to vary along three interrelated dimensions (Baía-Reis and Coelho, 2018; de Bruin et al., 2022; Pavlik, 2019). First, the level of inclusion is the degree to which physical reality is replaced by technology (Cummings and Bailenson, 2016; de Bruin et al., 2022; Milgram and Kishino, 1994). For instance, 360° videos seen on a Smartphone are less inclusive than a VR piece seen through a Head Mounted Display (HMD). Second, interaction possibilities refer to different ways of controlling and modifying one’s environment and are an inherent affordance of inclusive technologies (Steuer, 1992). Interaction possibilities are, for example, the possibility of looking around or choosing storylines. Third, immersive narrative structures typically give the audience a more active perspective by creating a first-person narrative and including avatars to achieve a sense of embodiment (Baía-Reis and Coelho, 2018; de Bruin et al., 2022). In addition, IJ is often produced based on emotional content to elicit emotional reactions and connections (Goutier et al., 2021; Uskali and Ikonen, 2020).

While most studies on IJ have focused on its presumably strong impact, these strong experiential effects differ depending on which dimension of IJ is assessed. Most assumptions of strong influence are based on the experience of a sense of presence as a result of inclusive technology, specifically when the individual is fully surrounded by an experience, allowing only minimal distractions (Cummings and Bailenson, 2016). Users of inclusive IJ feel as if they were transported to a different place (Sundar et al., 2017). An increased sense of presence tends to have substantial effects, for instance, on empathy (Barreda-Angeles et al., 2020), enjoyment, subjective involvement, distant suffering (Van Damme et al., 2019), information-seeking (Slater et al., 2018), and attitude change (Nikolaou et al., 2022). Interaction possibilities in IJ can elicit a sense of agency, that is, a feeling of control over one’s actions (Jicol et al., 2021). A sense of agency can lead to more empathy (Schutte and Stilinovic, 2017) but also worse memory if combined with highly emotional outcomes (Bujić et al., 2023). Immersive narratives in the form of first-person narrative perspective and embodiment can increase the sense of presence (Slater et al., 2018) and elicit stronger emotional reactions (Greber et al., 2023b). Overall, users tend to evaluate IJ differently depending on which dimensions an IJ experience affords (Greber et al., 2023a). However, the dimensions of IJ and the user’s context and personal factors, such as their intentions, influence their experience of IJ (Shin and Biocca, 2018; Shin, 2019). Consequently, it is crucial to consider these dimensions, their influence on the audience experience, and the audience’s own perception to evaluate IJ (Greber et al., 2023a; Shin, 2019).

While news companies worldwide, such as the New York Times, The Guardian, Süddeutsche Zeitung, Euronews, or Al Jazeera, initially were excited to start producing IJ—often fueled by initial tech sponsorship contracts, which enabled the integration of immersive technologies (Sirkkunen et al., 2020) -- there is less interest today due to effortful productions (Goutier et al., 2021), limited audience engagement (Wang et al., 2018), and limited audience access to the required technology (Mochizuki, 2023). Simultaneously, productions of IJ have not reached their potential in including immersive narratives, such as first-person narratives (de Bruin et al., 2022), and meaningful interaction possibilities (Palmer, 2020).

The importance of user experiences and user perceptions

IJ is an innovation connected to the audience turn in journalism studies. Contrarily, most empirical studies on IJ focus either on its effects, normative issues (Bujić and Hamari, 2020), or the motivations and production of IJ (Goutier et al., 2021). However, little is known from an audience perspective; while we may know how IJ influences individuals, we do not know why and how these effects take hold of audience members and how they may be taking back control of their experience. Taking on an audience perspective potentially sheds new light on this phenomenon (Swart et al., 2022).

Firstly, we argue that we know too little about how the audience perceives the experience of IJ, particularly when it comes to the different dimensions of IJ. Media effects and perceptions of these effects are not the same (McLeod et al., 2017). It is fundamental to understand media formats not solely in terms of their effects but also the audience’s perception of these media effects (Lecheler, 2020; see: Vàzquez-Herrero and Sirkkunen, 2022). This is important, as both the technological properties and the audience’s perception of those properties are relevant to the experience of IJ (Shin and Biocca, 2018). Additionally, we know that the user’s experience and evaluation are related to the multidimensionality of IJ (Greber et al., 2023a, 2023b). As an example, IJ is a form of news often produced to evoke emotions, with immersive narratives being the main driver behind evoking valence emotions, while interactivity and inclusion increase the intensity of an experience (Greber et al., 2023b); however, we do not know how the audience experiences and evaluates IJ’s emotionality. Indeed, scholars call for carefully considering the emotionality of IJ for the audience, not to overwhelm them (Pavlik, 2019: 68). Similarly, while we know that the audience experiences a stronger sense of presence and agency in interactive and inclusive IJ formats (Jicol et al., 2021) we know little about how the audience evaluates these effects. Again, feeling present and involved depends not only on the media product but also on the audience’s intentions and perceptions (Shin, 2019). Users tend to evaluate the experience of presence as attractive (Vàzquez-Herrero and Sirkkunen, 2022). Beyond that, the picture remains murky.

Secondly, IJ is at a point in its development where understanding its audience‘s experience is crucial. Innovation in journalism does not follow a linear growth process; hence, a temporal slow-down in its growth does not invalidate IJ (Wagemans and Witschge, 2019). Media innovations follow a three-stage process: invention, innovation, and diffusion (Stöber, 2004). In the invention stage, the old medium is improved to create a new medium; in the innovation stage, a new medium enhances communication to fulfill a new function. In the diffusion stage, innovations become widely accessible, making the innovation part of the audience’s regular media diet (Stöber, 2004: 487). As the technology of VR has not evolved as fast as assumed, and audience adaption has been slower than expected, VR has not yet reached the diffusion stage (Mochizuki, 2023). Similarly, after one decade of experimentation with VR in IJ, the audience has not picked up IJ as hoped (Wang et al., 2018). We argue that IJ is at the beginning of the second stage: through experimentation, trial, and error, journalists are testing how to tell journalistic stories in IJ; however, as can be seen by the declining use (Sirkunnen et al., 2020) they might not have succeeded yet. Therefore, we need to understand the audience’s experience to provide further guidance on implementing -or not- IJ. This becomes pressing in the current climate of investments in the metaverse by Meta, Microsoft, and South Korea (Keane, 2022), potentially increasing the relevance of VR and, consequently, IJ for media agencies.

Research indicates a disparity between journalists’ intentions for IJ and audience preferences (Jarvinen, 2020). While communication experts seem enthusiastic about the potential of IJ, particularly when it comes to authenticity, users seem concerned about the possibility of manipulation due to the video-game-like character of an experience. Additionally, issues such as over-emotionalization and self-consciousness while wearing VR goggles are mentioned (Nielsen and Sheets, 2019). Further, while individuals may report increased involvement, they sometimes feel like they are invading someone’s personal space or missing essential scenes (Jones, 2017). Even as users positively recognize inclusion (Greber et al., 2023a), presence and emotional experience (Godulla et al., 2021) and the novelty of the VR technology (Vázquez-Herrero and Sirkkunen, 2022), they complain about the utility and usability of IJ apps (Godulla et al., 2021), unrealistic camera angles, unclear positionality of subtitles (Vázquez-Herrero and Sirkkunen, 2022). Further, an unfamiliar audience struggles with complicated interaction possibilities (Jarvinen, 2020). However, also journalists producing IJ seem to struggle to combine the emotionally engaging capabilities of immersive technologies and journalistic norms of objectivity (Goutier et al., 2021).

Our limited understanding of the audience’s experience and evaluation of IJ and its multidimensionality is problematic as unintended effects of IJ can be overlooked; this becomes relevant considering that effects and the subjective perception of those effects are distinct (McLeod et al., 2017). Comprehending issues like low engagement with New York Times’ 360-degree videos (Wang et al., 2018) and gathering insights for future immersive productions rely on understanding the audiences’ experience and their evaluation of IJ. Therefore, we ask:

RQ1

How does the audience experience the different dimensions (inclusion, immersive narratives, interactivity, emotions) of immersive journalism?

RQ2

How does the audience evaluate immersive journalism?

Method

We used the thought-listing technique (Cacioppo et al., 1997) with two immersive cases to answer our research questions. The data collection was part of a nation-wide extensive project, including four experiments.¹

Thought-listing

We were interested in how the audience reflected on IJ productions. Thus, a thought-listing technique, in which participants list thoughts they had during an IJ experience, was used to explore the audience’s responses (Cacioppo et al., 1997). The method is used as a protocol analysis to investigate thoughts, judgments, or cognitive processes and is suitable for cases with few “predetermined ideas” (Cacioppo et al., 1997: 929). The data for Kiya was collected in July and August 2019 in a museum, the data for The confused man was collected between April and June 2019 in a music venue, a museum, and a University of Applied Sciences, aiming at a diverse sample. Participants were recruited on-site to participate in a study about Immersive Journalism in exchange for a 10€ gift card. Ethical advice was provided by the Department of Health Studies at the University of Applied Sciences Utrecht.

After recruitment, participants gave their informed consent, in which participants were informed about the procedure, risks, and topic but not the thought-listing task. Then, participants filled out an online questionnaire about socio-demographics, after which they were shown an IJ story using a Samsung Gear Headset. After being exposed to one of two IJ productions, participants were asked: “We would also like to know what you were thinking of while viewing the story. Can you write down all the thoughts you had when viewing the story?.” Below the introduction text, 12 open boxes were provided to insert thoughts. Participants were instructed to fill in as many thoughts as they liked. 54 participants saw Kiya. Of those participants, 50% were male, 44.6% female, and on average 36.67 years old (SD = 15.62). Their highest education is university education (44.1%), 26.5% completed higher general education, 17.6% secondary general education, and 11.8% secondary vocational education. 58 participants saw The confused man, with 53.6% male, 44.6% female, with an average age of 36.41 years (SD = 16.51). 28.6% received a university education, 26.8% higher vocational education, 17.9% higher general education, 14.3% secondary vocational education, 7% general secondary education, 3.6% primary school and 1.8% primary vocational education.

Case studies

The two IJ productions chosen for this study, as illustrated in Figure 1, represent cases closely linked to the early definition of IJ (de la Peña et al., 2010), which means the recreation of an event through CGI and inclusive technologies to let the audience witness an event as if they were there. Both cases are produced by established media companies, address a subject unlikely for participants to experience in everyday life, and which producers felt to be difficult to explain in text or video, therefore representing exemplars for IJ. However, they differ in their immersive forms (interactive elements, sophistication of CGI), and the context in which they were produced, with Kiya representing an early version of IJ, and The confused man representing an example of an immersive story during its peak (Paíno-Ambrosio and Rodríguez-Fidalgo, 2020). Thus, following Flyvbjerg (2006), the two cases represent paradigmatic cases.

The first case study focuses on Kiya, produced in 2015 by Emblematic Group, which is owned by de la Peña – called the “godmother” of IJ - and the American Al Jazeera. Kiya follows the framework de la Peña et al. (2010) established in their paper. In Kiya, viewers witness the recreation of a domestic violence attack in the USA. A man threatens and shoots his spouse. The story uses CGI and the real audio of the 911 call set off by the victim’s sisters, who witnessed the incident. The audience follows the sisters as they call the police, try to de-escalate the situation, greet the police, and then witness the murder of their sister. The audience takes on an observer role in the story and does not see but hears the murder by hearing the gunshot. Participants experience Kiya through an inclusive HMD providing an omnidirectional view. Kiya is a form of IJ that focuses on re-creating a scene as it happened (Sirkkunen et al., 2020) and lets the audience witness a scene that would be largely inaccessible in real-life.

The second case study uses the production The confused man (‘de verwarde man’) about the experience of psychosis by the Dutch broadcasting company KRO-NCRV produced in 2017. The production uses simple, drawn, stick-like animations because it is a powerful way to visualize “what goes on in someone’s head when experiencing a psychosis” (Goutier et al., 2021: 1657). In the experience, the audience takes on the protagonist’s perspective and embodies his avatar. While watching the news, a clip about a young refugee boy triggers a psychosis. Using colorful animations, the production depicts the protagonist’s detachment from reality. The story ends with the protagonist’s girlfriend calling the police, who put him in a cell to restore his sense of reality. The audience takes on an embodied, first-person perspective, interacts with the environment – such as grabbing a smartphone to start the storyline -, and experiences a high level of inclusion by using an HMD and headphones.

Figure 1.

Still from the confused man by KRO-NCRV (left); still from Kiya by the emblematic group and Al Jazeera (right).

For Kiya, 345 thoughts of 54 individuals were analyzed; for The confused man, 253 thoughts of 58 individuals were analyzed.

Qualitative content analysis

Participants’ answers are analyzed with a qualitative content analysis based on Mayring (2015). As basic analysis unit, we define individual thoughts. However, if one thought contains two lines of thought, this thought might include two units. Based on the content analytical model (Lagerberg, 1975; cf from Mayring, 2015), the information we wish to derive from the content analysis is the recipient. The study aims to identify the (1) experience and (2) evaluations of the recipients towards IJ.

A combination of a deductive and inductive approach, with various iterations, is applied. As IJ has been the subject of several studies and conceptualization approaches (de Bruin et al., 2022; Sánchez Laws, 2019), building an initial category system is based on previous studies (Mayring, 2015). We used the experience and evaluation of participants as guiding categories for the category system. The following sub-categories were included concerning the experience of the different dimensions of IJ: sense of being there, sense of embodiment (Tham et al., 2018), and emotional valence of thoughts (Uskali and Ikonen, 2020), and the evaluation of IJ: references to the technology itself, such as coolness, innovation, and malfunctioning (Sundar and Limperos, 2013), fear of missing out (Aitamurto et al., 2021) and motion sickness (Pan and Hamilton, 2018).

Subsequently, the first author analyzed a random sample of 10% of thoughts of each case using the deductive codebook. Codes that did not fit the pre-defined codebook were formed. Codes generated based on this inductive approach are testing the equipment, previous expectations, thoughts about not experiencing empathy or emotions, and tension.

After revisiting the data based on the initial deductive–inductive category building, the first author analyzed the entire dataset. Then, a second author reviewed the complete coding. The second author flagged ambiguous cases, which then were individually discussed by the two coders and a third author, and a decision as to their final coding into (sub)categories was made. In this process, inductively built codes were reorganized to fit the main categories of experience and evaluation; codes that emerged inductively but did not fit these categories were excluded from the analysis.

As a result, 202 thoughts from 72 individuals over the two cases (The confused man, N = 33; Kiya, N = 39) fit the main category of ‘experience.’ In this category, we further reorganized the experiential codes into four sub-categories matching the experience of the four dimensions of IJ: the level of inclusion, the immersive narrative, interaction possibilities, and emotions. The level of inclusion includes the codes being there and realism; immersive narrative includes the code embodiment; interaction includes the code sense of control; and emotions includes codes on emotions and empathy. Concerning IJ’s evaluation, 255 thoughts form 62 individuals were included (The confused man, N = 26; Kiya, N = 36). In this category, we included the sub-categories quality evaluation of the narrative, production, and technology; the evaluation of modality; enjoyment, confusion, motion sickness, and fear of missing out. 13 participants (The confused man, N = 7; Kiya, N = 6) did not express a thought that was relevant for coding it in one of the two categories.

Results

RQ1: How does the audience experience the different dimensions of immersive journalism?

Inclusion

Participants’ thoughts after both VR experiences indicate that they seem to experience a sense of presence; simultaneously, it seems that some of them are uncomfortable with it.

As expected based on the inclusive technology used, participants’ thoughts indicate that they feel as if they are transported to a place. For instance, one participant in Kiya states: “I am totally in the situation, and I have to remember that this is not real.” Other participants’ comments refer to de la Peña’s concept of “reaction as if real” (2010): participants initially react as if the experience happened to them in real life: “I tend to lean back when I saw a gun” (Kiya).

Additionally, when implicit expectations about how ‘reality’ should appear are not confirmed in the virtual environment – the experience doesn’t seem plausible - the sense of presence can be interrupted (Slater, 2009). For instance, in Kiya a participant notices that their position in the virtual reality does not correspond to their position in reality – and therefore, the sense of presence is interrupted: “I’m not standing on the ground but a bit above it, it feels like I see the kitchen from above, in this way I’m not a part of it.”

Participants in both productions reflect on a sense of unease when feeling presence. In Kiya, some participants express discomfort and reluctance towards the emotionally intense experience and the feeling of presence. Through the technological setup of HMDs, the audience of the two IJ productions is forced to be audio-visually surrounded by a story (Goutier et al., 2021). While the audience are still aware that the experience is mediated (Hartmann and Hofer, 2022), it is difficult to escape this level of inclusion. Moreover, participants can close their eyes or look away to avoid visual immersion in intense moments, but they cannot ignore the audio without adjusting the volume or removing the HMD. In Kiya, the audio was a real police phone call, capturing the fatal gunsho’s sound and the victim’s sister’s emotional response. One participant expresses discomfort with being an involuntary witness to such an event: “After the shots were fired, I had to deal with the sound recordings of the crying women.”

Participants felt uneasy with the sense of presence in The confused man, albeit for a different reason: it simply confused them. The production intentionally aimed to evoke confusion, as reflected throughout the audience’s experience. Related to the sense of presence, participants were confused about why, where, and how they were transported to this place. One participant states, for instance: “Where did I end up.” Overall, it seems that the feeling of presence is strong, so much so that users might feel uncomfortable with it.

Narrative

In both productions, the audience’s thoughts imply a feeling of being involved in the storyline as a character rather than a bystander.

In The confused man, the production provides the audience with a virtual avatar, which participants recognize as their own. For instance: “Wow I can even see my drawn fake legs.” While participants’ role in Kiya was not explicitly defined, they still experienced the story as if they had assumed a role. For example, some participants felt their role was to help the characters and de-escalate the situation. One participant describes: “It felt as if I was good friends with the characters, and therefore, I was willing to take the risk by intervening.” One participant stated this also led to confusion: “I don’t get why I am part of the story.”

Interactivity

Allowing users to interact with a virtual environment tends to elicit a sense of having control and agency in the virtual environment (Jicol et al., 2021). Participants’ thoughts indicate a delicate balance between a sense of control and disillusionment with it. While some participants’ thoughts indicate that they experienced a sense of control, this sense of control was closely related to a sense of powerlessness and guilt. Specifically, when participants refer to a sense of control, they often state that they felt they could have intervened to help the characters, for instance, in the assault depicted in Kiya. However, despite having a sense of control, some participants decided against taking action. For instance, one participant states: “I was looking for a good moment to take the gun from the man and tackle him” (Kiya).

However, as soon as participants decide to intervene, they realize that their sense of control is an illusion, as the virtual environment does not allow them to act and influence the storyline in Kiya. Consequently, participants report a feeling of powerlessness. One participant states: “I felt extremely guilty and weak that I could not stop the disaster” (Kiya).

Emotions

Lastly, we were interested in how the audience described their feelings of empathy and emotions after having experienced the two IJ cases. Emotional reactions to IJ can be twofold. Emotions can be reactions towards the content or the experience, but emotions could also be an affective evaluation of an experience (e.g. Oliver and Raney, 2011).

Thoughts indicate experiences of intense negative emotions in both productions. Participants seeing Kiya stated that the experience was heavy and intense and to have felt sad, angry, nervous, and helpless. In addition, participants state having felt empathy to the point of personal distress (Davis, 1983), as expressed by one participant after experiencing Kiya: “I felt tears coming to me when I saw the reaction of the women to the gunshot.” Participants in Kiya also indicate having been overwhelmed by witnessing such a distressing event. After seeing The confused man, participants stated that the story was intense, too emotional, exaggerated, leaving them nervous, restless, scared, and confused. However, at the end of The confused man, participants commented that they also felt curious, surprised, and relieved, particularly at the relatively happy end of The confused man.

In Kiya, participants express being overwhelmed and even detached, which some participants ascribe to the uncanny, video-game-like CGI graphics of Kiya, and the change of the scenes between inside and outside the house. One participant states: “It felt more like a video game than a real situation. This caused a sense of detachment.”

Participants also empathized with the story’s characters, particularly in The confused man; some actively tried to take on the protagonist’s perspective. The experience of empathy seems to relate to the VR animation being based on a real-life story. One participant explicitly points out: “O, it’s a real story that makes me experience it more intensely and with more empathy” (The confused man). Similarly, in Kiya, the realness of the audio was often referred to as making the experience more emotionally intense: “Empathic because of the audio as that was ‘real’ recording.”

RQ2: How does the audience evaluate immersive journalism?

Quality

Quality evaluations diverged heavily between the two cases: Kiya was generally evaluated negatively, while in The confused man, quality evaluations were relatively balanced.

In Kiya, most thoughts were concerned with a lack of quality of the production, mainly due to the poor audio quality of the original phone call and the computer-game-like animation. Consequently, participants also express hesitation when thinking about Kiya as a journalistic production. One participant stated in relation to the animation: “it looks like walking dead.” The quality of The confused man is evaluated as slightly more negative than positive. Thoughts concerning poor quality refer to various issues, such as animation or a dull color range. When highlighting positive aspects, comments almost unanimously focus on the illustrations and the graphics: “Cool drawings and music!.”

Concerning the narrative, participants in both cases expressed confusion about the story and would prefer to receive additional and easily understandable background information about domestic violence and psychosis. For example, one participant in Kiya states, “I need more background information.” In the case of The confused man, the lack of meta-information is intended by the production to convey the feeling of confusion during a psychosis. Some evaluated Kiya as biased, with one participant commenting: “is this an information campaign or is this journalistic analysis.”

Concerning the modality, participants stress the novelty, coolness, and excitement about the VR technology, such as “Technology is amazing.” Negative evaluations of the modality are mainly concerned with its relationship to reality, with one participant stating, “What a ridiculous way to depict reality.”

Understanding - “what’s going on?”

As previously stated, in both cases, participants are confused about what is happening around them and why. While this confusion was part of the intended effect of The confused man (Goutier et al., 2021) it was not so for Kiya. Remarkably though, participants indicate in both cases that the lack of background information about their roles and other figures was the reason for the confusion.

Participants experiencing Kiya express confusion and lack of understanding that is unintended by the production. Participants ask four out of the five W-questions. For instance, participants wondered, “Who is who,” not understanding the character’s relationship with each other, or “I don’t understand why the people in the house are arguing.” These thoughts can be found throughout the timeline of the experience their thoughts refer to. It seems that solely letting the audience witness this event without clarifying accessibly the who, where, what, and why leaves participants speculating about these questions throughout the experience. In The confused man, participants wonder about the bizarre thoughts of the main character. Apart from this, some participants were confused about their role in the production and why and how to interact with some devices to continue the story. Moreover, participants struggle with this case as a journalistic production, as indicated by this participant: “Surprisedness that the goal was to give more knowledge about delusions, while I didn’t find it very clarifying.” Additionally, mainly for Kiya, the 360-degree experience left some participants wondering whether they were missing out on essential details on the virtual backside of the story.

Discussion

One decade of (research on) IJ has shown that IJ has not yet caught up to the potential provided by its technology (de Bruin et al., 2022; Palmer, 2020; Wang et al., 2018). The results of this study indicate that this might partially be due to the audience not appreciating experiencing IJ. Based on a thought-listing analysis of two IJ experiences, this paper contributes to the literature in three ways. First, IJ users might not appreciate the intended effects. Second, presence and contextual information must be plausible and clear for the audience to focus on the experience. Third, the audience struggles to comprehend these two immersive productions within the journalistic genre.

First, our findings show that participants might recognize intended effects, like a sense of being there, a sense of agency, emotions, and affective empathy. Still, they might not appreciate their experience of these intense effects. Our results suggest that participants in both cases felt overwhelmed or resistant to being dragged into a – potentially uncomfortable – story. Some participants expressed strong emotional distress after watching Kiya. The intended effects of a sense of being there and a higher emotionality are thought to catch the audience’s attention (Shin and Ognzanova, 2022). De la Peña herself discusses in interviews the strong emotional impact as a strength of her work (e.g. Melcher, 2022). However, from an audience perspective, to continue using IJ, they also need to perceive it as worthwhile (Schrøder, 2015). This becomes pressing when the audience turns away from the news because it becomes too negative, emotional, and overwhelming (Villi et al., 2022). Our results similarly show that some individuals might not appreciate the strong emotional effects of IJ. Journalistic values such as creating an engaging and emotional experience to connect to the audience (e.g. Beckett and Deuze, 2016) in contrast to not harming the audience’s emotional well-being might be at odds here.

There are several ways to mitigate causing harm to the audience. One approach discussed in the context of IJ is using de-immersion techniques that reduce the sense of unmediated witnessing, such as journalists’ voice-over and superimposing text over the 360° experiences (Gonzalez and Serra, 2022). Moreover, one could warn the audience before witnessing potentially disturbing scenes in IJ or employ a more constructive approach to telling immersive stories. Constructive journalism aims to include positive emotions, solutions, and restorative narratives to give the readership a more optimistic outlook and a sense of self-efficacy (McIntyre, 2015). Recently, IJ started to embrace a shift from negativity to more civic-based human interest stories, which is not covered by our two cases (Wu, 2022). Productions and studies on the impact of IJ should be carefully drafted with the audience evaluation of these effects, and the audience’s well-being, in mind (Lecheler, 2020).

Second, our results highlight that while some participants picked up on a sense of presence, many were confused and wondered, “what’s going on”? The sense of presence is an important aspect of an IJ experience, which users repeatedly highlight as interesting (e.g. Greber et al., 2023a; Godulla et al., 2021), and decisions such as the positionality of cameras, high-quality visualizations, and audio, should be made with attention to this (Godulla et al., 2021). This is because the audience needs to have a plausible (Slater, 2009) experience to follow the story; otherwise, individuals will focus on the discrepancies between the perspectives of their lived and virtual reality. Similarly, our analysis indicates that the audience tries to resolve unanswered questions if crucial information is not explicitly communicated. Figuring out “what’s going on?” during the experience potentially draws their attention from the experience. Specifically, in Kiya, using real-life audio as the basic structure of the witnessing experience meant not having the classic pyramid-style narrative structure of a journalistic experience. As a result, various (unintended) unclarities arose among the audience. Ultimately, it seems that the audience wants to understand the facts before they can experience facts (e.g., de la Peña et al., 2010).

Based on this, we suggest future productions – in addition to creating a plausible IJ experience – focus on storytelling. While storytelling in IJ does not (need to) adhere to the typical pyramid structure, specific storytelling setups explicitly focusing on the particularities of immersive environments might help to convey relevant information (e.g., Domínguez, 2017; Paino-Ambrosio and Rodriguez-Fidalgo, 2020). In light of findings that the audience remembers facts worse after an immersive experience (e.g., Bujić et al., 2023), and the explicit wish for more background information by our participants, it could further be argued that IJ should be used as supplementary material to a comprehensive reporting (e.g. Wu, 2022).

Third, the audience struggles to comprehend the two immersive journalistic cases as journalism. While some participants explicitly questioned the productions’ journalistic nature, for example due to their intense emotionality, many thoughts implicitly indicate the same. For instance, the audience grapples with the lack of information explicitly provided, possibly due to their expectations of journalism to inform rather than present a witnessing account. Participants also seem to struggle with their role in IJ, as indicated by their response to interactivity ranging between wanting to act and (unsuccessfully) trying to act. If the production were a game – as could be expected by the game-like animations – audiences might anticipate the ability to act and explore. However, this expectation might be violated in a journalistic production that only has the feel of a game because journalists tend to limit interaction possibilities in IJ to retain narrative control (Goutier et al., 2021; Mabrook, 2021). The audience’s struggle extends to the use of animations that depict a reality “as if” rather than “as is” (see Aitamurto, 2019). Using animations, this form of IJ is shifting from journalism using visuals as evidence to using visuals as a recreation of visual evidence. Consequently, journalists’ challenges in combining immersive, emotional technologies with a journalistic approach to convey information (Goutier et al., 2021) is echoed in our user responses to IJ.

This indicates a blurry delineation between IJ and other journalistic and non-journalistic genres. Future studies must attempt to situate IJ in a growing literature of the emotional turn (Beckett and Deuze, 2016) and digital journalistic innovation, thinking about how distinct it is from other forms of journalism, such as digital journalism, multimedia journalism, newsgames, or different genres of storytelling, such as documentaries or games. And as IJ is often defined based on its technological characteristics, the motivations to produce IJ are closely tied to these technological characteristics. However, is a journalistic story that elicits a sense of presence without using inclusive technology still IJ? Moreover, if supposedly beneficial outcomes are not appreciated by the audience, which added benefits does IJ provide beyond this?

This study comes with limitations. Analyzing participants’ thoughts is complex due to varying lengths and ambiguous reference points. Participants were asked to write down all their thoughts during the IJ experience, leading to a broad interpretation of what was asked of the audience. Future studies using the thought-listing technique should be aware of this and narrow questions down to the focus of interest (e.g., thoughts about the quality/storyline/evaluation of the IJ production). Additionally, we suggest complementing this descriptive analysis with thinking-aloud protocols in connection with in-depth interviews and experimental designs of IJ users to fully capture the audience’s perception. Lastly, this study only includes the audience responses to two cases from the early (2015) and peak (2017) years of IJ production (Paíno-Ambrosio and Rodríguez-Fidalgo, 2020). Future studies should investigate audience responses to a broader, more diverse set of IJ productions.

These shortcomings notwithstanding, the analysis of users’ thoughts explains why they might not be attracted to this form of IJ: not everyone appreciates its emotional intensity, low production quality distracts, little context information confuses and the audience struggles to evaluate IJ as a form of journalism.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Taskforce for Applied Research SIA in the Netherlands under Raak Publiek, No. RAAK.PUB04.004; and by Osterreichische Nationalbank Jubilaumsfonds, under grant No. 18403. Österreichischen Akademie der Wissenschaften; 26419.

ORCID iDs

Hannah Greber

Sophie Lecheler

Kiki de Bruin

Note

Author biographies

Hannah Greber is a Ph.D. student, funded by the ÖAW-DOC grant of the Austrian Academy of Sciences, at the University of Vienna and part of the Political Communication Research Group. Her research explores the implications of emerging technologies and novel forms of reporting the news on citizens. In her Ph.D. project, she investigates the impact of Immersive Journalism on the audience, as well as their evaluation of this type of journalism.

Sophie Lecheler is professor of Political Communication at the University of Vienna, Austria. Her research focuses on political journalism, digitalisation and politics, emotions, and experimental research methods.

Loes Aaldering is an assistant professor in Comparative Politics at the Department of Political Science and Public Administration of the VU Amsterdam. Her research mainly focuses on issues related to gender and politics, political leadership, political representation and electoral behavior.

Yael de Haan is professor of the research group Quality Journalism in Digital Transition at the Hogeschool Utrecht and professor by special appointment of Local Public Broadcasting at the University of Groningen. She leads a research group that investigates the role of journalism in people’s lives and society.

Sanne Kruikemeier is currently a Professor in Digital Media and Society at the Strategic Communication Group of Wageningen University & Research (WUR). Her research focuses on the consequences and implications of online communication for individuals and society.

Nele Goutier (MA) works as a researcher for the University of Applied Sciences Utrecht, where she studies the possibilities and effects of technological innovations such as immersive storytelling and Artificial Intelligence. In the Journalism School in Utrecht she teaches research methods and writing skills.

Kiki de Bruin is a PhD candidate at the research group Quality Journalism in Digital Transition (HU University of Applied Sciences, Utrecht) and the Strategic Communication Group (Wageningen University & Research). She studies News Avoidance and its implications for the informed citizenry and news media in democratic societies.

References

Aitamurto

(2019) Normative paradoxes in 360° journalism: contested accuracy and objectivity. New Media & Society 21(1): 3–19.

Aitamurto

Won

Sakshuwong

, et al. (2021) From FOMO to JOMO: examining the fear and joy of missing out and presence in a 360 video viewing experience. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 8–13 May 2021, 1–14.

Baía-Reis

Coelho

(2018) Virtual reality and journalism. Digital Journalism 6(8): 1090–1100.

Barreda-Angeles

Aleix-Guillaume

Pereda-Baños

(2020) An ‘‘empathy machine’’ or a ‘‘just-for-the-fun-of-it’’ machine? Effects of immersion in nonfiction 360-video stories on empathy and enjoyment. Cyberpsychology, Behavior, and Social Networking 23(10): 683–688.

Beckett

Deuze

(2016) On the role of emotion in the future of journalism. Social Media + Society 2(3): 205630511666239.

Bujić

Hamari

(2020) Immersive journalism: extant corpus and future agenda. GamiFIN Conference, 1–3 April 2020: 136–145.

Bujić

Salminen

Hamari

(2023) Effects of immersive media on emotion and memory: An experiment comparing article, 360-video, and virtual reality. International Journal of Human-Computer Studies 179: 103118.

Cacioppo

Von Hippel

Ernst

(1997) Mapping cognitive structures and processes through verbal content: the thought-listing technique. Journal of Consulting and Clinical Psychology 65(6): 928–940.

Cummings

Bailenson

(2016) How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology 19(2): 272–309.

10.

Davis

(1983) Measuring individual differences in empathy: evidence for a multidimensional approach. Journal of Personality and Social Psychology 44(1): 113–126.

11.

de Bruin

de Haan

Kruikemeier

Lecheler

Goutier

(2022) A first-person promise? A content-analysis of immersive journalistic productions. Journalism 23(2): 479–498. doi: 10.1177/14648849209220, In press.

12.

de la Peña

Weil

Llobera

, et al. (2010) Immersive journalism: immersive virtual reality for the first-person experience of news. Presence: Teleoperators and Virtual Environments 19(4): 291–301.

13.

Domínguez

(2017) Going beyond the classic news narrative convention: The background to and challenges of immersion in journalism. Frontiers in digital humanities 4. doi: 10.3389/fdigh.2017.00010, In press.

14.

Doyle

Gelman

Gill

(2016) Viewing the Future? Virtual Reality in Journalism. Miami, FL: Knight Foundation. Retrieved from http://www.nordicom.gu.se/sites/default/files/publikationer-helapdf/freedom_of_expression_and_media.pdf

15.

Flyvbjerg

(2006) Five misunderstandings about case-study research. Qualitative Inquiry 12(2): 219–245.

16.

Godulla

Planer

Wolf

, et al. (2021) An immersive journey through flawed technology: users’ perceptions of VR in journalism. Journalism and Media 2(3): 454–468.

17.

González

Serra

(2022) Between immersion and deimmersion. Adaptation to 360° technology in the New York Times daily. Journalism Practice 360: 1–21.

18.

Goutier

de Haan

de Bruin

Lecheler

Kruikemeier

(2021) From “Cool Observer” to “Emotional Participant”: The Practice of Immersive Journalism. Journalism Studies 22(12): 1648–1664. doi: 10.1080/1461670X.2021.1956364, In press.

19.

Greber

Aaldering

Lecheler

(2023a) The Worthwhileness of Immersive Journalism—Taking on an Audience Perspective. Journalism Practice 1–19. doi: 10.1080/17512786.2023.2177711, In press.

20.

Greber

Lecheler

Aaldering

De Haan

Kruikemeier

De Bruin

(2023b) Feeling the News? The Differential Effects of Immersive Journalism on Emotional Response. Digital Journalism 11(1): 39–60. doi: 10.1080/21670811.2022.2155205, In this issue.

21.

Hartmann

Hofer

(2022) I know it is not real (and that matters): media awareness vs. presence in a parallel processing account of the VR experience. Frontiers in Virtual Reality 3: 694048.

22.

Jarvinen

Aki

(2020) The Immersive Audience Journey. Retrieved from: https://audienceofthefuture.live/wp-content/uploads/2020/07/Audience-of-the-Future_The-Immersive-Journey-Report_July-2020.pdf

23.

Jicol

Wan

Hin

, et al. (2021) Effects of emotion and agency on presence in virtual reality. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI’21), Yokohama, Japan, 8–13 May 2021, 1–13.

24.

Jones

(2017) Disrupting the narrative: immersive journalism in virtual reality. Journal of Media Practice 18(2–3): 171–185.

25.

Keane

(2022) South Korea Is Betting on the Metaverse – and it Could Provide a Blueprint for Others. Englewood Cliffs, NJ: CNBC. https://www.cnbc.com/2022/05/30/south-koreas-investment-in-the-metaverse-could-provide-a-blueprint.html

26.

Lecheler

(2020) The emotional turn in journalism needs to be about audience perceptions: Commentary-virtual special issue on the emotional turn. Digital journalism 8(2): 287–291. doi: 10.1080/17512786.2023.2177711, In this issue.

27.

Mabrook

(2021) Between journalist authorship and user agency: Exploring the concept of objectivity in VR journalism. Journalism Studies 22(2): 209–224. doi: 10.1080/1461670X.2020.1813619, In press.

28.

Mayring

(2015). Techniken Qualitativer Inhaltsanalyse. Beltz, Ukraine and Weinheim, Germany: Qualitative Inhaltsanalyse - Grundlagen und Techniken. pp. 50–114.

29.

McIntyre

(2015) Constructive journalism: the effects of positive emotions and solution information in news stories. PhD Thesis, University of North Carolina at Chapel Hill Graduate School, Chapel Hill, NC.

30.

McLeod

Wise

Perryman

(2017) Thinking about the media: a review of theory and research on media perceptions, media effects perceptions, and their consequences. Review of Communication Research 5: 35–83.

31.

Melcher

(2022) Nonny de la Pena. The Future of Journalism. https://futureofstorytelling.org/story/nonny-de-la-pena-ep-64

32.

Milgram

Kishino

(1994) A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems 12(12): 1321–1329.

33.

Mochizuki

(2023) Sony PSVR2 Headset off to Slow Start as Metaverse Push Sputters. New York, NY: Bloomberg. https://www.bloomberg.com/news/articles/2023-03-30/sony-psvr2-headset-off-to-slow-start-as-metaverse-push-sputters?leadSource=uverify_wall#xj4y7vzkg

34.

Nielsen

Sheets

(2019) Virtual hype meets reality: users’ perception of immersive journalism. Journalism 22(10): 2637–2653.

35.

Nikolaou

Schwabe

Boomgaarden

(2022) Changing social attitudes with virtual reality: a systematic review and meta-analysis. Annals of the International Communication Association 46(1): 30–61.

36.

Nyre

Vindenes

(2021) Immersive journalism as witnessing. In: Uskali

Gynnild

Jones

, et al. (eds) Immersive Journalism as Storytelling: Ethics, Production and Design. New York, NY: Routledge, pp. 176–187.

37.

Oliver

Raney

(2011) Entertainment as pleasurable and meaningful: identifying hedonic and eudaimonic motivations for entertainment consumption. Journal of Communication 61(5): 984–1004.

38.

Paíno-Ambrosio

Rodríguez-Fidalgo

(2020) Past, present, and future of virtual reality: analysis of its technological variables and definitions. Culture & History Digital Journal 9(1): e010.

39.

Palmer

(2020) Breaking free from the frame: international human rights and the New York times’ 360-degree video journalism. Digital Journalism 8(3): 386–403.

40.

Pan

Hamilton

AFDC

(2018) Why and how to use virtual reality to study human social interaction: the challenges of exploring a new research landscape. British Journal of Psychology 109: 395–417.

41.

Pavlik

(2019) Journalism in the Age of Virtual Reality: How Experiential Media Are Transforming News. New York, NY: Columbia University Press.

42.

Sánchez Laws

Ana Luisa

(2019) Conceptualising immersive journalism. Routledge, In press.

43.

Schrøder

(2015) News media old and new: fluctuating audiences, news repertoires, and locations of consumption. Journalism Studies 16(1): 60–78.

44.

Schutte

Stilinović

(2017) Facilitating empathy through virtual reality. Motivation and Emotion 41(6): 708–712.

45.

Shin

(2019) How does immersion work in augmented reality games? A user-centric view of immersion and engagement. Information, Communication & Society 22(9): 1212–1229.

46.

Shin

Biocca

(2018) Exploring immersive experience in journalism. New Media & Society 20(8): 2800–2823.

47.

Shin

Ognyanova

(2022) Social media metrics in the digital marketplace of attention: does journalistic capital matter for social media capital? Digital Journalism 10(4): 579–598.

48.

Sirkkunen

Vazquez-Herrero

Uskali

, et al. (2020) Exploring the immersive journalism landscape. In: Uskali

Gynnild

Jones

, et al. (eds) Immersive Journalism as Storytelling: Ethics, Production and Design. New York, NY: Routledge, pp. 13–22.

49.

Slater

(2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society of London - Series B: Biological Sciences 364(1535): 3549–3557.

50.

Slater

Navarro

Valenzuela

, et al. (2018) Virtually being lenin enhances presence and engagement in a scene from the Russian revolution. Frontiers in Robotics and AI 5(91): 91.

51.

Steuer

(1992) Defining virtual reality: dimensions determining telepresence. Journal of Communication 42: 73–93.

52.

Stöber

(2004) What media evolution is: a theoretical approach to the history of new media. European Journal of Communication 19(4): 483–505.

53.

Sundar

Limperos

(2013) Uses and grats 2.0: new gratifications for new media. Journal of Broadcasting & Electronic Media 57(4): 504–525.

54.

Sundar

Kang

Oprean

(2017) Being there in the midst of the story: how immersive journalism affects our perceptions and cognitions. Cyberpsychology, Behavior, and Social Networking 20(11): 672–682.

55.

Swart

Groot Kormelink

Costera Meijer

, et al. (2022) Advancing a radical audience turn in journalism. Fundamental dilemmas for journalism studies. Digital Journalism 10(1): 8–22.

56.

Tham

Duin

Gee

, et al. (2018) Understanding virtual reality: presence, embodiment, and professional practice. IEEE Transactions on Professional Communication 61(2): 178–195.

57.

Uskali

Ikonen

(2020) The impact of emotions in immersive journalism. In: Uskali

Gynnild

Jones

, et al. (eds) Immersive Journalism as Storytelling: Ethics, Production and Design. New York, NY: Routledge, pp. 49–59.

58.

Van Damme

All

De Marez

, et al. (2019) 360° video journalism: experimental study on the effect of immersion on news experience and distant suffering. Journalism Studies 20(14): 2053–2076.

59.

Vázquez-Herrero

Sirkkunen

(2022) Back to Fukushima: perceptions and effects of an immersive journalism story. El Profesional de la Información 31(1): e310108.

60.

Villi

Aharoni

Tenenboim-Weinblatt

, et al. (2022) Taking a break from news: a five-nation study of news avoidance in the digital era. Digital Journalism 10(1): 148–164.

61.

Wagemans

Witschge

(2019) Examining innovation as process: action research in journalism studies. Convergence: The International Journal of Research Into New Media Technologies 25(2): 209–224.

62.

Wang

Suh

(2018) The effects of 360-degree VR videos on audience engagement: evidence from the New York Times. In: International conference on HCI 21 in business, government, and organizations, Orlando, NV, 15 July 2018, 217–235.

63.

(2022) Journalism’s immersive shift: uncovering immersive journalism’s adherence to traditional news values, norms, routines and roles. Journalism Practice 1–16. doi: 10.1080/17512786.2022.2133779

Uncovering the audience perspective: A qualitative analysis of experiences and evaluations of two immersive journalism productions in the Netherlands

Abstract

Keywords

A radical audience perspective of the experience of immersive journalism

Immersive journalism: An effect-driven innovation?

The importance of user experiences and user perceptions

Method

Thought-listing

Case studies

Qualitative content analysis

Results

RQ1: How does the audience experience the different dimensions of immersive journalism?

Inclusion

Narrative

Interactivity

Emotions

RQ2: How does the audience evaluate immersive journalism?

Quality

Understanding - “what’s going on?”

Discussion

Footnotes

Declaration of conflicting interests

Funding

ORCID iDs

Note

Author biographies

References