How do we speak about algorithms and algorithmic media futures? Using vignettes and scenarios in a citizen council on data-driven media personalisation

Abstract

‘New’ media and algorithmic rules underlying many emerging technologies present particular challenges in fieldwork, because the opacity of their design, and, sometimes, their real or perceived status as ‘not quite here yet’ – makes speaking about these challenging in the field. In this article, we use insights from a three-stage citizens council investigating citizens’ views on developments in data-driven media personalisation to reflect on the potentials of using future-orientated vignettes and scenarios in data collection on user experiences, expectations and the ethics of algorithms. We present the possibilities and potentials of using vignettes as part of a data collection approach in user-centric algorithm studies which invites users’ contextual experiences of algorithms but also enables more normative reflections on what good looks like in contemporary datafied societies.

‘New’ media and algorithmic rules underlying emerging technologies present particular challenges in fieldwork. The opacity of their design, and, sometimes, their real or perceived status as ‘not quite here yet’ – makes speaking about these challenging in the field. In this article, we suggest that there is promise and potential in using vignettes and scenarios from fictionalised accounts of the uses of emerging and new technologies, drawing upon data from a three-wave citizens’ council on data-driven media personalisation. We situate our article within the methodological approaches seen in scholarship in user-centric algorithm studies (Hargittai et al., 2020; Siles, 2023; Swart, 2021) and design futuring within Human Computer Interaction (HCI; Dunne and Raby, 2013; Lindley and Coulton, 2015). We outline the empirical case study of embedding vignettes within our citizens councils. We argue, first, that vignettes and scenarios (c.f. Gray et al, 2017; Rizvi, 2019) help make ‘new’ technologies and often abstract algorithms more concrete, thereby drawing out lived experiences of the social dynamics of new media. Second, we suggest that vignettes and scenarios, by centring unknown others in the narrative, help draw out users’ normative reflections on what good looks like in contemporary datafied societies.

Our contribution speaks to distinct challenges in user-centric research aiming to draw out people’s lived experiences of algorithmic media in everyday life. First, researching expectations, experiences and ethics of algorithmic systems is difficult when data and algorithms are opaque and their workings and impacts are not perceptible or easily understandable. Second, if we want technologies to be designed in ways that take seriously people’s views, preferences and concerns, we cannot wait for these systems to be in everyday use. We need methods and techniques to support dialogue and deliberation early on, when technologies and systems are not fully formed and when people may not have encountered or experienced them. Using speculative vignettes or scenarios − that are grounded in real-world developments and plausible dilemmas − enables us, we suggest, to make potential futures more tangible in the present. This, in turn, opens up spaces for critical reflection and discussion about what should happen; it provides a common, relatable reference point for debate in order to inform ethical development practices.

Discussing algorithms in the field

Research approaching new media technologies from the lived experiences of ‘users’ focus on bottom-up empirical examination of people’s understandings, awareness and experience of algorithms (Bishop, 2019; Siles et al., 2022 among others). Scholarship on ‘users’ draws upon broader histories of attention to audience analysis, for instance, within media and communication studies (cf. Livingstone, 2019; Ytre-Arne and Das, 2019; Das and Graefer, 2017), where terminologies – such as audiences, users, consumers, citizens and publics – have been varyingly used to describe those who engage with technologies. In ‘user-centric’ algorithm studies, most existing research reflects long-standing priorities seen within audience studies, to contextualise users’ diverse contexts and always contextual negotiations of technology (cf. Siles, 2023), and this is also how we approach users in this article. However, unlike media texts, speaking about algorithms presents scholars with something of a challenge because of the nature of algorithms both as recursive (Dogruel, 2021) and as significantly under the bonnet, in terms of the opacity of their workings in the eyes of lay users. Scholars are often split, for instance, on whether the term algorithm should emerge in their engagements with users, tending to focus on the pros and cons of specifically inviting talk about algorithms, as opposed to identifying alternative techniques to interrogate the embeddedness of algorithms in the everyday lives of users. Key issues within methodologies adopted for user-centric approaches to studying algorithms often relate to levelling the knowledge and power differentials between participants and participant/researcher, and creating space for discussions around the elusive concept of algorithms and their workings.

For example, in Kennedy et al.’s (2021) focus-group research examining the data practices of media users, the methodological benefit of their approach is evidence, in enabling a focus on the issue of inequality and exploring diverse feelings, attitudes and beliefs through interactions between focus-group members. Any effective use of group-based methods in discussing algorithms (such as ours, in this article), which involve users speaking with other users is hinged on ensuring homogeneity to some degree within the group (Krueger and Casey, 2009), since uneven technical knowledge of algorithms and how they work can result in uneven contributions as some individuals ‘may be intimidated and quieted by more knowledgeable participants’ (Hargittai et al., 2020: 767). Hargittai et al. (2020) sought to address these potentially uneven power dynamics, recognising the opacity of algorithms to even researchers. They make the methodological decision of having free flowing, conversational interviews with users, not specifically testing algorithm awareness but instead conversing about online processes that involve algorithms, across various domains of life. Their approach works towards participants feeling comfortable about any perceived lack of technical knowledge (Hargittai et al., 2020), reducing the knowledge and power differentials between researcher and participant. Similarly, Karizat et al. (2021) and Swart (2021) make the methodological decision not to use the terminology of algorithms, unless users themselves raise or use such terminology, placing the real and perceived power of invoking technical terms and calling upon technical prowess in participants’ hands. The unevenness in technical knowledge across participants is also visible in Jhaver et al.’s (2018) study of Airbnb hosts and their interactions with the site’s search algorithms, which highlighted different degrees of technical knowledge among participants.

The scholarship within user-centric algorithm studies draws on a broader range of research methods such as surveys, in some cases triangulating quantitative survey data with qualitative data from interviews and open-ended questions. For example, Cotter and Reisdorf’s (2020) surveys examining algorithmic knowledge highlight the role of socio-economic disparities in algorithmic knowledge within the context of online search. While a survey approach enables the researchers to measure algorithmic awareness against various demographic factors, as ever, other qualitative approaches might capture nuances behind these processes of knowledge accumulation and explain the social dynamics of such inequalities. Ytre-Arne and Moe’s (2021) study addresses this to some extent, by drawing on open-ended questions in surveys to collect qualitative data about algorithmic knowledge among participants, by specifically asking them how familiar they were with the use of algorithms. By combining surveys with scenarios, Büchi et al.’s (2023) work points to the potential for surveys to provide in-depth qualitative data when scenarios are used towards the shaping of open-ended, in-depth questions. In particular, Büchi et al. (2023) navigated participants through Facebook’s ‘Your Interests’ and ‘Your Categories’ sections in the Facebook app and desktop versions using screenshots, providing participants with six open text fields to gather their reactions, narratives and imaginaries to Facebook’s algorithmic profiling. They arrive at nuances in participants’ perceptions of algorithms as confining, practical, reductive, intangible and exploitative. Methodological scope to address the ambiguous and ambivalent emotions and thoughts around algorithms is extended through triangulation in Espinoza-Roja et al.’s (2023) work through a mixed methods approach involving a survey and semi-structured interviews to examine the relationship between algorithmic awareness and use of multiple platforms. Triangulating multiple methods enables them to measure users’ awareness of recommendation algorithms through a range of scales and further explore these measures through the narratives of users about their lived experiences of algorithms, leading to their conclusion that users refer to algorithms as a single, monolithic entity called ‘the algorithm’.

What seems evident is that the potential to elicit in-depth responses relating to algorithmic systems relates closely to the methods used to facilitate imaginations about algorithms, and methods are a determinant to encourage in-depth talk about everyday lived experiences around algorithms. Eslami et al.’s (2015) study explores the value of scenario-based methods towards eliciting rich narratives around feelings and behaviours generated by algorithmic knowledge. Using an application (FeedVis), Eslami et al. (2015) visualise algorithmic outputs based on participants’ network and News Feed stories, presenting this information back to participants during interviews to demonstrate how algorithms curate their News Feed (see also Rader and Gray, 2015). Such an approach not only helps interviewees to understand and visualise the impact of algorithmic curation, but makes the impact more personal by tying it back to their Facebook accounts and inviting them to adjust the algorithm to control the stories shown to them. The active involvement of Eslami et al.’s (2015) participants in shaping algorithms attributes agency to them, and may contribute to levelling the uneven power dynamics between researchers and participants as they are invited to share their feelings, opinions and understandings of the workings of algorithms. Similarly, Kapsch (2022) engaged participants in the shaping of their communicative agency through the participatory method of ‘self-reflexive ethnographic analysis’ where participants tracked their media use and reflected on their media engagement and the role of algorithms in shaping these instances. Others employ creative techniques to facilitate participants’ visualisation of the elusive and abstract concept of algorithms and their workings. For example, Lee (2018) makes use of a scenario-based method to draw out participants’ views on algorithmic management in a survey, outlining the usefulness of scenarios in simulating real-life experiences and eliciting responses that align with reality. Lomborg and Kapsch (2020) foreground the difficulty for users as respondents to understand algorithms as abstract categories. They address this in the course of fieldwork using printed prompts, including logos of algorithmic services and systems or screenshots of algorithms at work. The effective use of novel approaches and tools to investigate participants’ algorithmic knowledge highlights the need to explore and integrate creative methodologies, such as those used in disciplines that engage with user-centric algorithm studies. This might ensure purposeful engagement with users, being grounded on concrete, practical awareness and knowledge of algorithms, and redresses the participant/researcher power imbalance. In what follows, we suggest that using long-standing approaches such as focus groups, surveys and interviews are enormously useful to make sense of users’ experience of systems that already exist, such as search engines, social media, maps, video, news and product sites, evident in the choice of platforms investigated in the above studies. Yet, with emerging and new technologies that are yet to be adopted, we suggest that combining long-standing methodologies such as these with creative approaches, which help users imagine or visualise what such technologies encompass and how they interact with users, might prove to be useful.

Our approach draws on the field of design futuring in HCI, developed to explore emerging socio-technical futures critically. Design fiction and speculative design methods (see Dunne and Raby, 2013) have been used to investigate attitudes, practices, norms, values and socio-technical configurations relating to existing, emerging and alternative socio-technical worlds. Typically, these methods involve depicting artefacts or ‘speculative’ scenarios (sometimes mimicking products) that are provocatively designed or used in potentially problematic ways to provoke critical reflection and discussion. Design fiction has emerged as a design-based practice that seeks to explore and criticise possible futures by creating speculative and often provocative scenarios that are narrated through design-based artefacts (Lindley and Coulton, 2015). Rather than constructing prototypes, we rely on visual artefacts supporting speculative scenarios by designing an illustrative booklet with scenarios to encourage reflection on the ethics, politics and power relating to emerging media personalisation technologies. As well as stimulating public debate, critical and speculative design-based approaches have also been deployed to help developers and designers think through such issues. Recently, they have been used in qualitative research studies to learn about people’s responses and reactions to scenarios (Elsden et al., 2017; Wong et al., 2017). Given the pressing need to involve people in conversations about algorithmic developments that may affect them in the future and where these systems are not fully defined or deployed, speculative techniques such as future-orientated vignettes or scenarios offer tangible means to engage people meaningfully in critical reflection and discussion.

Methodology

We speak from a project that developed a participatory process for ethical discovery in relation to developments in new and emergent approaches to media personalisation and flexible media (see OFCOM, 2021 on object based media) – where users’ and audiences’ preferences might be used to personalise media content, departing from traditional linear media. The broader intent behind the project was that its findings can guide and inform ethics-related processes and broader efforts to enact Value Sensitive Design (VSD; see Friedman et al., 2017) throughout the engineering design and development lifecycle. We refer to this participatory process as a citizens council (Zubizarreta, 2014), since it seeks to involve members of the public in ethical design decisions and is inspired by the aims and methods of related participatory work with citizens assemblies, citizens juries and citizens councils.

The study took the form of a three-stage series of workshops in Guildford, Woking and Manchester in England, culminating in a final council that brought all the groups together to determine their priorities and recommendations in relation to data-driven media personalisation. The technology area that we have investigated is media personalisation, sitting alongside a major research programme focused on media technologies for personalised experiences led by the University of Surrey in partnership with BBC. We aimed to recruit our citizens to reflect the wider public in relation to gender, age, location, ethnicity and household income. We used an Expression of Interest form distributed through online public community groups in Manchester, Guildford and Woking, inviting people to register interest. We then selected a sample that broadly represented the national population and user interests (Abelson et al., 2007). We completed satisfactorily, the University of Surrey’s ethical assessment questionnaire before embarking on the fieldwork. We provided a participant information sheet, and offered adequate opportunities to ask questions and clear doubts before data collection. We also ensured that participants were fairly remunerated for their time, service and incurred expenses through a voucher per session attended. Each workshop in Manchester, Guildford and Woking had between four and eight participants in each round, and all participants came together for the final workshop which was conducted jointly online.

The vignettes

In the design of our citizen council workshops, we wanted to draw on the best aspects of existing sociological research methods (Chadborn et al., 2019; Livingstone and Lunt, 1994; Löhr et al., 2020). We used Round 1 to explore uses of and attitudes towards personalisation with a particular focus on the use of personal data in algorithmic systems. This involved drawing out users’ understandings of personalisation and what they understood as personalisation. The exploration provided an indication of starting digital literacies, their experiences of times when things recommended or personalised felt right/wrong, and offered a key opportunity to capture anecdotes. In Round 2, we first engaged participants in a recapitulation of the main topics and talking points from their previous workshop in Round 1, a month earlier. Thereafter, discussion was gently guided alongside an illustrative booklet. This supporting booklet was devised to provide connections between contemporary cutting-edge technologies and potential user benefits through their use in media personalisation. The analysis of the technologies and identification of user benefit was driven through association of this study with a parallel engineering research programme in partnership with the BBC on media personalisation – the AI4ME project. Much of the automation required to personalise media content at scale relies increasingly on artificial intelligence (AI), making this technology an appropriate juncture to investigate emerging types of personalisation for the future.

To take the notions of media personalisation beyond the realm of technologists, we designed the booklet first to introduce fundamental ideas of flexible media and then to focus on a set of user stories – the vignettes – together with a brief top-level lay explanation of an underpinning technology and a relatable illustration of how the media might be adapted using well-known types of BBC content. For the purposes of this article, we define vignettes to indicate hypothetical stories and situations involving fictitious individuals who developed themes and topics central to the research design and topic at hand. Vignettes do not need to be fictitious and as research shows can draw upon both real-life scenarios and collected data. The vignettes we developed illustrate situations where media personalisation may be encountered, centred around three broad themes that are associated with enabling technologies that use layers, segments and user profile data. These themes emerged through analysis and ethnographic study of previous technology trials mostly within BBC Research and Development (Cieciura et al., 2023), which were used to inform the engineering research project, in parallel to this study, that sought to develop media personalisation technology. Our vignettes are therefore grounded in real-world developments of personalised technologies and systems, to focus discussion on potential benefits, impacts and ethical issues or challenges.

In each of the three cases, our vignettes were prefaced by depictions of mainstream content being adapted to make a more personalised experience. These included the switching of language and presenter within a weather forecast, exploring the introduction of chapter points within a Saturday evening football round up or dance reality show, and the extension of personal data recording interactions with a nature watch series. Drawing on long-standing traditions of using vignettes and scenarios in qualitative research and key principles of speculative design, we created a set of assets depicting these future-orientated personalisation scenarios and user stories. Their speculative character stems from the depiction of possible futures to provoke critical reflection and their plausibility from their mapping to emerging technological developments involving data-driven media personalisation. Naeini et al. (2017) used vignettes focusing on privacy expectations and preferences relating to online and connected devices, using 14 concrete scenarios, the vignettes helped concretely draw out the diverse and context dependent nature of privacy preferences. Jenkins et al. (2010) note that vignettes are not only useful for exploring people’s own motivations and decision-making behaviours but can also be particularly successful at drawing out their views and ethical frameworks on normative questions. Rooted in the exploitation of flexibilities afforded through the use of AI with object-based media, our vignettes demonstrate potential user benefits centred around three thematic explorations of enabling technologies which enable re-mixing, re-ordering and generation of content, with the intention to invite citizen reflections on these technological capabilities and the ethics behind their development. We designed our vignettes to depict speculative but plausible future personalisation scenarios through hypothetical people/situations.

Scenario 1

Considering that one of the most valid affordances of layered content is to enhance the accessibility of content, in line with BBC’s aim of universality, we devised a scenario around a senior citizen character, who represented a collection of accessibility needs such as digital literacy, sensory and physical incapacities. Through changing device and adjusting certain settings, the technology could provide an enhanced experience for her.

Scenario 2

The segmentation into chapters of media, such as the TV shows depicted within the second theme, explores the use of personalised recommendation within a produced piece of content. The vignette purported two friends discussing how this might influence a news feed, whether helpfully specific to the local area or too narrow in perspective.

Scenario 3

The third theme examined the use of a user’s personal profile data, which may then be combined with other user data to suggest content that those with similar profiles have viewed. This vignette picks up on the potential outcomes for two users, who have different amounts of viewing history and are looking to make collective choices by sharing and pooling their data.

While the provision of the illustrative vignettes certainly primes the thinking of participants, these themes are not exhaustive. Kandemir and Budd (2018) discuss their use of vignettes in a small qualitative study with students in Germany and England where they placed a vignette part way into semi-structured interviews. They found that the use of the vignette was effective in terms of students engaging with the themes and concepts at hand, drawing out varied and rich data into analytical outcomes. We were hopeful that the conversations and the directions our participants would take would bring greater specificity and consistency to participants’ considerations of what the technology might offer, afford or restrain, and allow us to examine issues across a range of themes and genres, including how they link back to the technology. Moreover, by painting these pictures with users who could be thought of as friends and family of our study’s participants, it can prompt them to extend their contemplation of issues to persons beyond themselves, incorporating social, civic and community thinking in addition to the concerns of each individual. Palaiologou in 2017 talks about the importance of using scenarios and vignettes within participatory research in terms of the dialogue it allows between researchers’ frameworks and participants’ personal meanings and interpretations. Skilling and Stylianides (2019) discuss their production of a vignette framework that identifies the main elements for construction including the conception, design and administration of vignettes in relation to educational research. They demonstrate that vignettes can be particularly effective to draw out participants’ own interpretations about specific phenomena in relation to students. Equally, by providing discussions in the preceding round and at the start of this round, and by covering a set of technologies, scope was created for participants to make connections and to broaden the discussion, as they felt appropriate. In that sense, the vignettes were not presented as prescriptive or limiting, but rather as scaffolding or a starting point for participants to launch their thoughts.

Combining vignettes within a qualitative focus-group design

Our study did not use vignettes as a stand-alone method. Rather, they formed an important dimension of, and sat within a qualitative citizen council informed by decades of focus-group research. By combining the benefits of qualitative inquiry with speculative vignettes created using expert knowledge of technical developments in the field, our scenarios were intended to facilitate citizens in critical reflection and engagement in relation to ongoing developments that could affect them in the future. We began Round 2 by explaining to participants that certain approaches to media personalisation work by creating content that can be adapted and take many forms according to the context in which it is viewed or the requirements or preferences of people. We noted that while the development of this approach involves assumptions that it has the potential to improve the way content is produced, created and viewed, these processes at the same time rely more on technology making decisions and rely on more data. To exemplify this principle, we introduced speculative scenarios – our vignettes. The first scenario related to a senior citizen who was dealing with hearing and visual impairments which made it difficult for her to enjoy her favourite programmes. The scenario explored that while this senior citizen might benefit from personalisation enlarging the text on the subtitles and changing the colour to make them stand out, or remove background noise, she finds the interfaces of the systems that make it possible significantly complex and a struggle to engage with. This scenario drew upon core themes to do with the perceived benefits and risks to personalised media: prompts primed participants to reflect on digital access, literacy and skills, expectations about how it should work and, particularly, their expectations around data, the control of data and the algorithms used to create this. The scenario was effective in drawing out their views on the levels of control over personal data, understandings of privacy and how data should be managed and the notion of responsible AI. Scenario 1 laid the foundations in Round 2 for a more nuanced discussion of the latter themes around personal data. Scenario 2 was built around the experiences of a fictitious individual who was often confused by election coverage and felt that it was not necessarily showing them what different stories mean for them and their community. This individual was presented with the possibility of more personalisation to view recommended specific sets of information but they experienced a dilemma in terms of how such algorithmic selection would limit what people see of the available information, and how it could skew how people perceive key issues and civic debates. This scenario which dealt with the pros and cons inside a fictitious citizen’s mind about recommendation systems drew out citizens’ wide-ranging perspectives on circumstances where they might accept algorithmic recommendations and contexts where algorithmic recommendations were deeply problematic in their eyes. Scenario 3, also presented a dilemma, and was a dialogue between two fictitious young people discussing group-based algorithmic recommendations and pre-selections in online music consumption. Here, one young person heavily engaged with personalisation and was convinced about the benefits offered by algorithmic interfaces knowing his preferences and style over time. The other young person only rarely engaged with personalisation and had doubts about it potentially limiting her choices and options, in addition to her concerns about the privacy and protection of her personal data. This scenario highlighted how these important differences could affect their quality of experiences of personalisation based on inputs from more than one user, as well as the challenges that could arise from having to negotiate privacy in group-based algorithmic experiences. This scenario also led to expansive conversations on hyper personalisation which laid firm foundations for the final round of the citizens council. Finally, in Round 3, we invited citizens to gather again, this time bringing together participants from Guildford, Manchester and Woking, to reflect on the way their personal data are treated within algorithmic systems, to grasp their spectrum of feelings around personal data use, ranging from unbothered to occasionally bothered, watchful, fearful, critical and pro-active.

Findings

There are numerous substantial findings from this project on user experiences of algorithms that we discuss elsewhere including users’ conditional trust in these systems (Wong et al., 2023), and an individual-to-institutional spectrum in users’ attempts to live better with and improve algorithmic systems (Wong et al., 2023). Here, we pay attention to two methodological arguments. First, we discuss the potentials of scenarios and vignettes in concretising the abstractness of algorithms in the field; and second, we discuss the possibilities opened up by scenarios and vignettes in terms of centring normative discussions about what good and healthy looks like in relation to algorithmic systems in an age of rapid datafication.

Potentials of vignettes in concretising the abstractness of new technologies and their underlying algorithms

First, the three vignettes in combination drew attention to the concreteness of technological interfaces and their underlying algorithms, rather than a description of them in the abstract. While the same outcomes could have been achieved through more technical means in terms of demonstrating how media personalisation interfaces technically function, a concrete discussion of the perceived benefits and risks of algorithmic personalisation was made possible through the dilemmas encountered by the speculative scenarios and stories in the vignettes. For example, the first vignette on a senior citizen with hearing and visual impairments helped citizens to visualise the difficulties that people with disabilities can encounter when using personalisation technologies. The character of the senior citizen in the vignette took citizens beyond the abstract idea of flexible media technology to provide them with more concrete understandings of what algorithms can do within a particular context. For example, Jessica (Manchester) drew on the example of her aging parents increasingly struggling with visual problems to ascertain the benefits of flexible media as the following:

Yeah, well, I was thinking about my family. My parents as they are getting older, that it [flexible media] had be better for them. My mom’s blind to one eye on this side, and the other is quite poor. So as she gets older and older, will become worse. Yeah, so it [flexible media] will be useful for her. (Jessica, Manchester)

Even though the character in the first vignette was not immediately relevant to many of the citizens who were younger and did not have any visual or hearing difficulties, the practical and realistic nature of the story facilitated its visualisation against the context and lived experiences of loved ones. Similarly, Corey (Manchester) described how flexible media would have been beneficial to her partner with hearing difficulties, as the following:

Yeah, hearing access would really benefit my partner. He’s deaf in one ear and partially deaf in the other. And I have got quite a sensitive hearing, it’s kind of a bad combination. [. . .] But yeah, dialogue, something that you could see because he’s always saying ‘what did they say?’ Like, if there’s background noise, he struggles to hear the content [. . .] And you can see it’s like a real thing, yeah, it’s really difficult for him. So something that could kind of increase like some levels and less others. (Corey, Manchester)

Here, Corey drew on lived experiences with her partner to ascertain the difficulties described in the vignette and propose possible provisions in terms of flexible media to alleviate challenges faced in media use due to physical limitations. Corey’s narrative seems to suggest that she does not have much technical knowledge in media personalisation technologies since her proposals did not appear technically oriented. However, it is clear that Corey is able to verbalise her perceived needs in a concrete manner which can be used to inform the design and development of media personalisation technologies. In concretising discussions within lived experiences, the use of vignettes also contributes to expanding the scope of concerns beyond those illustrated in the vignette itself. For example, while the first vignette focused on visual and hearing disabilities, it inspired citizens to discuss other forms of disabilities and emphasise the importance of increasing digital access for all. Expanding on the first vignette of a senior citizen struggling to use flexible media technologies, Jackson (Guildford) pointed to the importance of taking into considering other disabilities such as learning difficulties through the example of an encounter with her friend:

It was just an issue for a friend, he has learning difficulties. So he has his bank on his phone. And I don’t know what’s happened. I think one of the daughters disconnected the account, and he couldn’t come inside and it was so complicated to set it up again. He said to me ‘no, no, I can’t do it. I can’t do it’. So I have to do it all and to call the bank and to ask. It took me two hours. And what do I understand about, everything has to do with secret keys and set up of the systems. If it is an older person, they might just put it in a cupboard. (Jackson, Guildford)

While Jackson’s example was not specific to flexible media, her narrative motivated more in-depth discussions among other citizens about the importance of broadening access to media personalisation technologies and how this may be achieved. The use of vignettes therefore creates opportunities for citizen-led discussions which can refocus researchers back onto what users consider to be important in algorithm use.

Second, the dilemmas presented in the vignettes were successful in lending a concrete shape to otherwise abstract discussions around potential downsides and barriers. This enabled specific focus on particular downsides experienced by the fictitious individuals in the scenarios thereby opening up a space for discussing downsides relevant for relatives or friends in the vast majority of cases. For example, the dilemma of the fictitious individual confused by election coverage discussed in the second vignette opened up extensive conversations on the downsides of hyper personalisation, with several citizens relating their anxieties over ‘echo chambers’ if people were allowed to use algorithms to automate the presentation and curation of traditionally broadcast news content in particular. Chelsea (Guildford) looked beyond the context of the individual character highlighted in the vignette, displaying a third person effect in positioning the highlighted issue on the scale of the wider society, to express her concerns over a tunnel vision effect if people were provided the opportunity to customise their news consumption as such:

I am afraid that we could reduce as well, reduce some vision that you have all over the world? (Chelsea, Guildford)

The use of vignettes helps citizens to visualise and scale up the downsides of hyper personalisation, with the intensity of the problem motivating the collective brainstorming for potential solutions. Similarly, the third vignette, which positioned algorithmic curation within the context of the Glastonbury festival and the dilemma encountered by two fictitious young people in online music consumption, highlighted downsides such as the lack of accuracy and relevance of recommendations, as well as concerns over individuals being confined to specific genres due to their preference settings and hence lacking exposure to other perspectives and tastes. In instances where the narratives in vignettes coincidentally echoed well with participants’ lived experiences, insightful data relating to downsides could emerge. For example, one of our citizens who was a musician, with experience of sharing music on apps such as Spotify, was particularly inspired by the Glastonbury vignette to share perceived downsides of hyper personalisation based on their personal experience as an artist. Harper (Woking) described concerns over the loss of novel content due to low algorithmic visibility as such:

Because audiences are not just audiences, audiences are also producers, creators, content makers. So just staying with that from the perspective of the Glastonbury example or something similar. From the perspective of producers of content, you know, up and coming artist, you might not have enough historical data points saying like, like like, like, like, like enough. What does this sort of flexible system mean for that kind of content, or people who make content that is a bit more on the edges? (Harper, Woking)

This issue of content loss due to the hyper personalisation of music consumption was not something that we had anticipated nor encountered in existing literature, pointing to the usefulness of speculative vignette in drawing out new insights derived from citizens’ diverse lived experiences. We suggest that while other methods of discussing the downsides of algorithmic recommendations and algorithmic systems in terms of their design could indeed be accomplished through asking participants to recollect of a time when something happened, the scenarios themselves opened up moments where participants could immediately see themselves or someone they know in these fictionalised narratives.

Utility of vignettes in centring normative discussions about the development and deployment of new technologies and underlying algorithms

The use of vignettes bridged the gap between under-the-bonnet technological workings of and lived experiences with algorithms, enabling more down-to-earth brainstorming of how things should be in a fair and equal society. They also facilitated future-orientated conversations about what should happen that could be mapped to current developments. The three vignettes used in this study focused citizens on three key areas of possible benefits/concerns in relation to media personalisation technologies, and inspired collective discussion on what good should look like, how this ‘greater good’ can be achieved and whom they expect should be responsible for achieving this. For example, the first vignette inspired talk on digital access and how this can be facilitated to ensure that users with diverse levels of skills, digital literacies, and physical capabilities are able to best maximise the affordances of media personalisation technologies to enhance their media experience. Justin (joint online session) was adamant that

You can’t talk about personalisation for those two groups [differently abled (physical and technology)] unless you have enhanced their ability to participate and make the system more accessible. (Justin, joint online session)

Justin appears to be suggesting that unless people are able to access the very technologies which were designed to cater for their needs, there is no use in discussing what media personalisation technologies they require and how these should be designed to meet those needs. For Justin, what should happen first is ability enhancement, perhaps through skills training, to address inequalities in capabilities and hence access. Other citizens agreed with Justin’s emphasis on accessibility, albeit suggesting other ways of achieving this and attributing the responsibilities for making this happen to other stakeholders. Responsibilities were placed on developers to create easy interfaces to increase access, rather than on users to develop their digital literacy skills. Penny (online session) states that

I think there needs to be something that developers are aware that there are people out there that struggle, and it needs to be a simplified version for them, very very very simple use. (Penny, Joint online session)

What is clear is that the use of vignettes helped citizens to better verbalise their expectations around how algorithms should be used for technological developments to make life better for people with disabilities, highlighting a broad range of perceived stakeholders and their responsibilities through a bottom-up approach.

While the second vignette spearheaded a conversation around worries over how algorithmic selection might exacerbate perceived issues of what participants referred to as ‘echo chambers’ and ‘filter bubbles’, they were able to quickly grasp the intricacies of this social issue in relation to media personalisation technologies and set boundaries for the affordances of algorithms to ensure accountability to society. For example, Claire (online session) suggests that news app such as BBC currently has provisions for people to input settings on the topics they are interested in, which she considers to be useful, but states that a boundary should be drawn in terms of the level of selectiveness enabled by algorithmic selection, such that

what you shouldn’t be able to do is say I only l want to see things that are fairly favourable to my beliefs that are like purposely you know, misleading or not impartial. (Claire, Joint online session)

Citizens assign responsibility to tech designers and institutions to set limitations for algorithms to protect users from the negative impacts of ‘echo chambers’, such that they will not be faced with the dilemmas encountered by the fictitious character in vignette 2. Along the same lines, another citizen Skyler (online session) was inspired to propose a new affordance for algorithms which can resolve the dilemma faced by the character in vignette 2, adding that

There could be kind of options to find out what is happening in the rest of the world. And you know, you kind of have links to various other kinds of countries, perspectives that would lead to getting perspectives from other places where things are happening, that isn’t our own version of what is happening. Because there is so many different versions of our reality. That will be amazing, I had love that. (Skyler, Joint online session)

Vignettes embedded the technicalities of algorithmic interfaces within the lived realities of everyday use, as such enabling citizens with limited technical knowledge to become more confident in proposing potential solutions for achieving what they consider to be appropriate levels of algorithmic control people should be given in relation to news consumption.

The use of vignette three brought back issues of privacy and protection which were key concerns articulated by citizens in the first session, helping them to envision what types of data, to whom they may need to provide these and the resulting benefits of giving up their private information. Through the vignette, citizens were better able to envision the different forms of data which may fall under private data, moving beyond the more limited scope of bank details, address, age or birth dates which they envisioned in the first session. In the words of Jacelyn (online session),

The only way you control it is through legislation and regulation. I don’t think it will work perfectly until companies accept the responsibility. (Jacelyn, Online session)

Similarly, Paolo (Guildford) called for a ‘trusted watchdog’ to ensure that media institutions are ‘doing everything they promise’. Vignette 3 prompted a realistic envisioning of the dangers in relation to data loss and the compromising of privacy, with these perceived threats inspiring conversations around what should happen and who should take responsibility for the safe use of technology.

Conclusion

To conclude, we present reflections on the use of vignettes and scenarios as methods used within a citizen council which sought to engage members of the public in discussion about developments in emerging media personalisation technologies.

It is important, first, to reflect on some of the challenges and limitations inherent to this work. We reiterate that we used vignettes as embedded within the long-standing approach of a focus group – our citizens council. This enabled us to build upon focus groups’ and interviews’ strengths in eliciting rich data on users’ understandings and experiences of algorithms, and combine these with scenario-based methods’ advantage of anchoring discussions around tangible and relatable situations. The embedding of one within the other, we have hoped, will provide linkages between normative discussions of emerging technologies with the public and those visions and ideas being advanced by technology and industry actors, opening up the assumptions and values that underpin them to wider comment and critique grounded in people’s current lived experiences and expectations around personalisation. This combination of approaches allowed us, we felt, the best of both worlds, in a sense, in terms of relying on the doubtless benefits of a tried and tested qualitative approach in terms of speaking about media technologies, and a more creative attempt to concretise some of the attributes of the emerging media personalisation techniques currently in development but not quite within everyday use either.

However, we recognise also that ‘to concretise’ these technologies, we have made assumptions and choices, foremost among which, is the decision to narrow down the broad spectrum and gamut of ethical questions down to a potentially reduced focus on risks, benefits and often even practicalities of use. This was not a decision lightly taken, as we knew that the resultant discussion would bypass much broader ethical debates, and focus on a narrower understanding of risks and benefits. Nonetheless, despite these limitations, we suggest that, for technologies still not quite within full public view, an understanding of public awareness of their risks and benefits adds something meaningful to a broader conversation on ethics in terms of design futures. To that end, we hope to make a modest and, by definition, partial contribution to a conversation. Here, we found that the arguably narrower, yet fairly specific foci of the vignettes and scenarios did help make emerging algorithmic systems more concrete in the present in order to draw out relevant lived experiences, social dynamics and implications of algorithmic technologies and provided opportunities for participants to engage and respond to these developments in ways that can inform both research and innovation agendas. While our conversations did indeed also act as springboards for wider ranging discussion on a more expansive level in relation to ethics, the narrower discussion on risks and benefits provided hooks to concretise and translate between normative discussions about technologies in everyday life and the technical realities and visions being actively worked upon by actors in industry. We argue that by opening up visions of possible technical futures to wider groups for comment and critique, even a partial discussion of rather specific risks and benefits offers up some potentials.

Next, we need careful reflection on our own positionalities within the team and the design of these vignettes. As a team combining sociology, engineering, communications and public service media production – we each had very different points on entry in terms of terminologies, scopes and the defining of foci in relation to vignettes. We deliberated at length within and across our diverse fields, and attempted to craft the vignettes and scenarios in order to make possible discussions of algorithmic futures while also drawing some attention to broader notions of what good might look like in datafied societies. The sequencing of the vignettes aimed to scaffold participants’ understanding of new and emerging technologies in contextual and relatable ways in order to provoke reflection on everyday experiences of algorithms while also offering them opportunities in their role as citizens to talk back to, challenge, comment on or critique visions of technical futures. We attempted to create characters with a diversity of age ranges, and we left various other socio-demographic attributes unspecified. Building vignettes and scenarios around a third person, even with unspecified demographic characteristics, we found, added an important normative dimension to the work, helping to bridge users’ own personal and deeply contextualised experiences and opinions towards questions of what broader good should look like in society for others. For example, we found that when discussing dilemmas and challenges faced by fictitious individuals, this led to more empathetic responses. While, on one hand, a diversity of participants identified with these characters or recognised family members and friends from diverse communities within these, in our conversations, we are mindful that leaving demographic attributes unspecified, or specifying them in great detail both carry methodological implications. These, we suggest, are key considerations to anyone working with scenarios and characters in similar work, going forward.

It is also very important to acknowledge the limitations of these methods. Vignettes and scenarios seeking to make tangible possible futures cannot represent all possibilities, situations or ethical tensions or issues and so will inevitably impose a frame. Our motivation – as part of our citizen council – was to create plausible and balanced scenarios able to support considered reflections and deliberations about possible algorithmic futures. But, not all research will share these aims and we recognise our positionality and our role as researchers in shaping the artefacts for these purposes. Then, we argue that vignettes and scenarios can act as useful research methods to anchor and open up discussions around possible futures by connecting everyday experiences and expectations of personalised media with imagined user experiences mobilised by actors whose visions and decisions play an important role in materially shaping their trajectories. Borrowing from lessons within foresight analysis (cf. Ytre-Arne and Das, 2019), and future studies, they provide another example of a method capable of bringing multiple stakeholders and publics together around important and topical issues and developments. Using scenario-based approaches, potentially, more direct lines of connection between people’s social experiences and expectations of algorithmic media and technology developments and design choices might begin to be drawn. Therefore, we might see scenarios as playing an important role right from the outset of the technology design process, by drawing in the views of citizens and publics (cf. Livingstone and Lunt, 2011) and ensuring their interests are concretely in the room and in the picture (c.f. Ytre-Arne and Das, 2019). Thus, we conclude by arguing that constructing contextualised, intelligible representations of emerging algorithmic systems, as people might encounter them in everyday social situations, we might help reduce barriers to achieving more meaningful public involvement in technology and innovation – which is integral to ensuring responsible innovation.