Sage Journals: Discover world-class research

Abstract

In recent years, citizen engagement in policy and research has gained considerable momentum. In the healthcare domain, patient narratives, through various mediums, have emerged as a valuable source of insight into the experiences of patients and the healthcare system. Recognizing the value of such textual data, diverse analytical methods have been developed, spanning from text mining to narrative analysis. This article presents experiments that combine computational methods, qualitative methods and citizen science for analyzing patients’ stories. In this article, we reflect on two experiments in which we combined these approaches, which we analyze through a generative lens. We distinguish three main effects of the experiments: they provide a platform for discussions as a 'site of controversy'; they act as 'mediator', fostering new connections and mutual understanding among participants; and they serve as 'tin opener', stimulating substantive discussions about methodological development and substantive healthcare matters. Narrative reduction, which occurs when rich narrative data is simplified into structured quantifiable forms, is not inherently problematic; instead, it can be meaningful when combined with qualitative methods and citizen science, emphasizing the importance of utilizing diverse methods to balance authenticity and gaining broader insights. The study highlights the significance of collaborative sense-making and meaning-making in interdisciplinary research. Engaging patients, their relatives, and professionals in the analytical process, facilitated by tools like word clouds, promotes engaged discussions that yield actionable insights. Further development of such interdisciplinary approaches holds promise for a more nuanced understanding of patient experiences, fostering epistemological pluralism, and refining healthcare practices.

Keywords

Mixed-methods citizen science interpretative analysis patient narratives text mining experimentation

Introduction

In recent years, engaging citizens in policy and research has become more and more common. Knowledge of citizens is deemed relevant for gaining insights into problems and controversies from the citizen perspective, which can help to come to more feasible and societally relevant solutions (Baibarac-Duignan and de Lange, 2021; Jemielniak and Przegalinska, 2020). Citizens’ perspectives are considered crucial in initiatives addressing pressing social issues like sustainability transformations and public health (Allen et al., 2019; Turnhout et al., 2020). The growing attention for the experience-based knowledge such actors bring into policy and research also aligns with broader societal developments, such as calls to democratize knowledge and increasing demands by citizens to be included in policy decisions (Baibarac-Duignan and de Lange, 2021; Gijsel et al., 2019).

In the healthcare domain, the involvement of citizens takes on a distinctive character due to the predominant participation of patients and their relatives in research and policy-making processes (Stewart, 2013). Consequently, citizen engagement in healthcare transcends the conventional notions of participation and inclusion, as it entails matters that intimately affect individuals’ lived experiences, making health care a deeply personal and meaningful subject for engagement. The importance of including experience-based perspectives in health care decision-making has been recognized as important for a long time (Aronson, 2020; De Graaff et al., 2019). The inclusion of such perspectives is seen as crucial to develop health care systems and practices that take into account what patients find important. Although there is broad consensus regarding the need to include patient perspectives, how to do so adequately proves to be far from straightforward. Research within health policy and Science and Technology Studies (STS) into practices of citizen engagement in science and policy demonstrates that there is a considerable disconnect between ideal and practice (Delgado et al., 2011; Filipe et al., 2017; Oldenhof and Wehrens, 2018).

The increasing importance attached to experience-based forms of knowledge is accompanied by an enlarged appreciation of the narratives of patients (van de Bovenkamp et al., 2020). Patients increasingly share their stories about living with a certain condition, including their experiences with the health care system. These narratives can play an important role in the above-mentioned endeavor to include patient experiences in decision-making. The stories shared offer a good understanding of the everyday life of a person with a certain condition, their treatments and the healthcare services received (Frank, 2013; van de Bovenkamp et al., 2020). These narratives are shared in multiple ways. For instance, many patients use online fora to discuss their lived experiences with an illness (Egher, 2023). Researchers and quality workers can also actively collect stories via qualitative methods, such as narrative interviews or ethnographic observations (e.g. Heerings et al., 2022). A readily available source of rich data is provided by patients themselves, as they write books about their condition. Books provide a large body of experiential knowledge that can be used to better understand patients’ experiences and improve healthcare based on these (van de Bovenkamp et al., 2020). Nevertheless, generating insights from such large amounts of patient-generated data can be challenging and time-consuming.

In recent years, various computational methods have been developed that have the potential to analyze such large amounts of texts. These methods are increasingly used in the social sciences to analyze various kinds of data, like social media data, documents, books and other historical sources (Grimmer and Stewart, 2013; Németh and Koltai, 2021). The promise of computational methods, such as machine learning and modeling, is that they can detect patterns in large datasets, which could help to gain more comprehensive overall insights. Different sorts of text mining techniques and models have been developed and used to establish what literary scholars refer to as 'artificial reading', where texts are analyzed computationally (Andrade and Andersen, 2020; Kestemont and Herman, 2019). Various social science scholars express optimism regarding the use of computational methods, arguing that these methods can be fruitful in social science (Antons et al., 2020; Wang, 2017; Németh and Koltai, 2021; Buyalskaya et al., 2021).

At the same time, the fields of Critical Data Studies (CDS) and STS have insisted on a more cautious approach, recognizing both the menial work that goes into producing useful data and the limitations, biases, and interpretive efforts required to contextualize and make sense of the outcome of data produced through computational methods (boyd and Crawford, 2012; Hoeyer and Wadmann, 2020; Kitchin, 2014). Scholars working in these traditions recognize that while computational methods prove to be able to identify useful patterns, they need to be complemented with other, qualitative and interpretive methods to produce meaningful knowledge (Nelson, 2020; Blok et al., 2017; Munk et al., 2022). While these two research traditions for a long time have been presented as being radically different from each other, just like the alleged dichotomy between social science and data science (Moats and Seaver, 2019; Adler-Nissen et al., 2021), research has shown that these traditions do not have to be mutually exclusive: epistemic differences can be negotiated in situ (Stevens et al., 2020) and over prolonged periods of time, new scientific disciplines like computational anthropology can emerge (Munk et al., 2022).

While these studies have shown the benefits of combining novel computational approaches with interpretive forms of qualitative research, an important aspect remains absent, especially in the context of analyzing the stories of patients. As much as qualitative researchers are crucial in interpreting the data produced by patients in stories, including patients and professionals in the research process can have additional benefits (Gijsel et al., 2019; Jemielniak and Przegalinska, 2020). Citizen science offers a way towards 'collaborative knowledge' (Jemielniak and Przegalinska, 2020), which allows for joint sense-making and meaning-making of the data used.

Although some research effort is put into finding new ways of analyzing patient experiences using either qualitative or computational methods (Dreisbach et al., 2019; Walsh et al., 2022), little attention has been devoted to ways of productively combining qualitative research with computational methods when it comes to analyzing textual data (Carlsen and Ralund, 2022) and to the best of our knowledge, there has been no such combined research conducted on datasets comprising patient narratives. Research is especially scarce when it comes to the involvement of citizens in such explorative interdisciplinary research. Therefore, the aim of this study is to explore meaningful and relevant ways for generating and mobilizing experiential knowledge from patients using an interdisciplinary approach. In this article, we reflect on our experiences with various experiments in which we analyzed patient stories using a combination of computational methods, qualitative research and citizen science.

This article proceeds as follows. First, we bring strands of literature on illness narratives, computational methods, citizen science and experimentation together. Subsequently, we outline the various experiments we conducted. We conceptualize these as generative forms of experimentation (Gomart and Hajer, 2003; Wehrens, 2019). In the results, we distinguish between three main effects produced through the experiments: the experiment as a site of controversy, the experiment as mediator and the experiment as tin opener. In the discussion section, we contemplate the complex, but meaningful, process of combining various methods to analyze patients’ stories, highlighting the need for collective interpretation and analysis.

Theoretical framework

Patient narratives

Narratives are increasingly recognized as an important means to 'tap into' the experiences of patients (Vickers, 2016). The role and nature of these patient narratives are conceptualized in different ways. Charon (2001) coined the term narrative medicine, which focuses on the narrative of the patient within the field of medicine. Narratives play an important role in different relationships within medicine, but the main one focuses on the patient–physician relationship. In this relationship, the act of listening to and respecting the narrative of the patient by physicians is seen as crucial for delivering patient-centered care. Also, within health care quality improvement initiatives, storytelling is acknowledged as useful evidence for signaling quality and safety issues and informing health service policies (Iedema, 2022; Khanbhai et al., 2021; Ziebland and Hunt, 2014).

However, patients do not only talk about their illness in a medical setting, as illness disrupts various aspects of one's life. Illness not only disturbs one's health status, but also affects aspects like one's personal relations and employment (Jones and Pietilä, 2020). Frank (1995) already pointed towards the importance of patient stories in his book The Wounded Storyteller. This work provides us with a broader understanding of the value and variety of patient stories. Patient stories go far beyond the story the patient tells in the consulting room, as people share their experiences about living with a condition (Frank, 2013). Stories thus provide insights into experiences with illness beyond the biomedical view (Vickers, 2016). Moreover, the analysis of multiple patient stories offers richness in terms of internal and external diversity (van de Bovenkamp et al., 2020).

Inspired by this idea of 'the power of patient narratives', different methods are used for collecting patient stories. Examples include in-depth interviews with patients and experience-based co-design (e.g. Heerings et al., 2022). Other researchers have focused on the collection of readily available patient stories. When it comes to these stories most studies have focused on collecting and analyzing social media and Web 2.0 data (Walsh et al., 2022; Khanbhai et al., 2022; Jung, 2023). In our study, we focus on a more traditional type of data: books written by patients about their illness, often producing rich insights into their experiences in the context of their personal life (van de Bovenkamp et al., 2020). Analyzing such material is traditionally done using narrative or thematic analysis. The aim of this type of research according to Greenhalgh and Wengraf (2008) is “to explore such things as how the individual has made sense of these events, their attitude toward them, what meanings the events hold for them, and how these feelings came to be” (244). While these qualitative methods are useful for understanding the lived experiences of patients, they limit the number of books one can analyze. Including more material can enable a more substantial understanding of shared patient experiences and can help to, among others, identify similarities and differences across experiences.

Computational methods

In recent years, the use of computational methods, such as text mining and forms of supervised and unsupervised machine learning, is seen as a promising way to analyze large amounts of texts. Computational methods are increasingly used in the social sciences to analyze different sorts of textual data. In particular, social media data is used for performing these analyses. However other data such as documents, books and other historical documents can also be used (Németh and Koltai, 2021; Grimmer and Stewart, 2013). Analyzing large amounts of textual data using only qualitative methods is infeasible, if not impossible, because of limited human resources. The promise of computational methods is the detection of patterns in large datasets which help to gain more general insights. Different sorts of text mining techniques and models are developed for such computational analysis (Kestemont and Herman, 2019; Andrade and Andersen, 2020; Németh and Koltai, 2021).

While within public discussions, especially in the context of AI, computational methods are often presented as outperforming human thinking, scholars working with text mining are more likely to argue that computers cannot replace humans when it comes to reading texts (e.g. Grimmer and Stewart (2013). Also in recent discussions, about the role of LLM (large language models) like GPT-3.5 in research, people focus on its affordances and pitfalls, thereby stating that it cannot replace humans but rather enrich their work in different ways (Thorp, 2023; van Dis et al., 2023). The way the computer 'reads' is often contrasted to the traditional manual work done by researchers (Kestemont and Herman, 2019; DiMaggio, 2015; Lin, 2012). By doing this, researchers are trying to make sense of the suitability of certain tasks for humans and computers.

One way to do this is by analyzing different 'modes of reading'. Kestemont and Herman (2019) distinguish between three modes of reading: close reading, surface reading and distant reading. The first two are performed by humans, and the latter is conducted computationally. On the one hand, close reading implicates a way of detailed attention for the text which makes the reader attuned to literary details. In distant reading, on the other hand, the human takes a distance from the text in exchange for including a larger corpus and often includes the incorporation of AI techniques. Within the digital humanities, Moretti (2005) has been influential in advocating its systematic strengths which would lead to broader literary insights. Between these two extremes lies the concept of surface reading, where the focus is placed solely on the text itself, avoiding any interpretations that would result in the creation of deeper meanings. While human reading by most is seen as 'true reading', stating that human interpretation can be seen as a golden standard still raises questions. DiMaggio (2015) stated that humans just like algorithms “seem to be bad at pretty much the same tasks” (3), thereby stating that human judgment also has its flaws as it comes with interpretative variation. Another approach is the combination of the different reading modes, where one holds onto traditional methods and uses new techniques for richer analyses (Kitchin, 2014).

In a similar vein, researchers have increasingly sought ways to complement qualitative and computational methods. Some have focused on complementing quantitative data with ethnographic data (Blok et al., 2017; Blok and Pedersen, 2014; Bjerre-Nielsen and Glavind, 2022; Bornakke and Due, 2018). The main aim here is to compensate for the lack of context of big data with ethnographic fieldwork. Again, others are looking for ways for combining computational methods and qualitative methods when analyzing these large number of texts (Nelson, 2020; Carlsen and Ralund, 2022; DiMaggio, 2015; Adler-Nissen et al., 2021; Munk et al., 2022). Others have instead of merging the methods, executed the methods separately on the same dataset and then compared these (Baumer et al., 2017). These endeavors show that qualitative methods and computational methods are not necessarily dichotomous and do not need to be “thrown on the bonfire of the dualisms” (Moats and Seaver, 2019: 7).

Sense-making and meaning-making

While the combination of computational methods and qualitative methods has potential for performing rich large-scale analysis, it is not yet clear how and if this can be performed with patient narratives. The combination of methods allows for different ways of sense-making and interpretation of patient narratives. Importantly, patients themselves are highly important in interpreting such work. That is why, besides qualitative methods and computational methods, the methodological approach for analyzing these books could benefit from the inclusion of the people who are closest to this dataset, namely patients and their relatives, as well as those directly affected, such as healthcare professionals.

This is in line with general ideas behind citizen science, which also consists of involving individuals who are directly affected by a research topic in the research process. Within health care, patients have normally been engaged in research under the name of 'patient engagement' and 'patient and public engagement (PPI)' (Heyen et al., 2022). It is only recently that some specific forms of participation in research by patients are referred to as citizen science. While citizen science can take many forms, citizens are most often involved in the data collection and data processing phases of the research process (Kullenberg and Kasperowski, 2016; Gijsel et al., 2019). Whereas the input of citizens has proven to be successful, it argued that it is worthwhile to engage citizens in early stages of research when a certain problem is being defined (Ciasullo et al., 2021), but also in the data analysis stage, as the engagement of citizens in research promotes sense-making and meaning-making of the data. Allen et al. (2019), for instance, show how health surveys can be combined with collaborative workshops with citizens where data was discussed and analyzed among participants. This approach counters forms of epistemic injustice (Fricker, 2007), as input from participants is taken seriously and contributed to better formulation of research and policy recommendations.

Experimentation: Representational and generative registers

To bring together patient narratives, computational methods and qualitative research approaches, we have organized various experiments. In this final section of the theoretical framework, we outline how we conceive these experiments and from which kind of underlying logic they have been conducted (Wehrens, 2019).

In philosophy of science, experiments are seen as a particular mode of knowledge production (Downey and Zuiderent-Jerak, 2017; Lury and Wakeford, 2012). The traditional understanding of experiments is that they can lead to objective knowledge about the world if they are properly designed. This representational logic assumes that an accurate and precise representation of real-world phenomena can be captured, if the experimenter has pure access to the phenomenon of study and can observe without letting his theoretical and material apparatus interfere (cf. Gomart and Hajer, 2003).

In contrast to this representational logic, several authors have argued that experiments as mode of knowledge production can also be understood from a generative logic. From this logic, experiments do not function as 'mirrors of nature' but as 'sites of emergence': specific events that allow joint exploration of new ideas and that can open up taken-for-granted assumptions (Gomart and Hajer, 2003). Building on this idea in the context of sociology, Zuiderent-Jerak (2015) teases out how such experiments can function as devices for heightening reflexivity. Rather than settling controversies by producing undisputable facts, experiments can be understood as tools for situated reflection, producing 'thickly descriptive' knowledge (Geertz, 1973) that serves as a starting point for further interpretation and reflection.

In our analysis, we build on this generative understanding of experiments to tease out how our attempts to bring together citizen science, computational methods and qualitative methods played out. We therefore do not focus on whether these experiments provided some kind of definitive knowledge about 'key characteristics' of patient experiences but zoom in instead on what these experiments set in motion. In other words, what effects do they produce among various participants and what can be learned from these specific events that bring different actors together?

Methods

Context

This study finds its basis in the research initiative 'patient sciences' where we research patients’ experiences from many directions, including experiences with their illness, challenges in daily life, and activities and coping strategies by utilizing a large collection of books written by patients or relatives (www.patientervaringsverhalen.nl). This collection, which consists of over 6000 books and continues to expand, finds its origins in an initiative developed by Coleta Platenkamp, a patient and sociologist herself (Platenkamp, 2018). Platenkamp wrote about her experiences with illness and started a foundation which was committed to collecting the stories of patients and their relatives. Ever since 2004, volunteers gathered books and stories in other formats, such as documentaries and blogs, about patient experiences with the aim of making experiential knowledge widely available.

Projects

In this study, we reflect on a range of experiments in two projects where we sought to develop a new combination of methods for analyzing patient stories. Through these methodological experiments, we sought ways to productively combine these methods to build on their respective strengths: aggregating data to produce insights into overarching patterns (computational methods), rooted in the lived experiences of patients (citizen science for eliciting experience-based knowledge) and carefully unpacked in the light of sensitizing concepts and theories (through qualitative interpretive analysis). From the outset, we envisaged two potentially useful ways of combining these methods. First, we considered using computational methods as a tool to strengthen qualitative thematic analysis of a selection of books by reading and structuring all texts and creating summaries to have an overview of characters, places and actions. Second, we envisaged using the combination of methods to facilitate co-creation sessions with experts, such as patients, relatives, and healthcare professionals, to share experiences and generate ideas on how to apply the findings in healthcare practice.

We performed these experiments in two research projects, one on the theme of psychosis and the other on the theme of dementia. Both projects had an exploratory character. Therefore, not all research steps have been predetermined, as we opted for a more iterative approach that allows for flexibility. A detailed account of the steps undertaken for each research project is presented below.

The first project took place in September–December 2021. This project had a specific focus on how people diagnosed with a psychotic disorder experience social integration and stigma. As qualitative social scientists we teamed up with other social scientists, data and software engineers, a clinical psychologist and an industrial design researcher. We met on a regular basis and, of those meetings thickly descriptive notes were taken.

For this project, we started with a dataset of 35 patient stories selected from the patient stories collection, of which five books were excluded because they were not suitable for computational text analysis (e.g. comics). To give ourselves some guidance we took the conventional data science cycle (Leonelli and Beaulieu, 2021) as our approach and added qualitative analysis and citizen involvement steps. The books were already available in digital format before the start of the project. The pre-processing was the first step which consisted of data cleaning, mark-up and the development of a repository. Simultaneously, we started with the qualitative thematic analysis of a selection of the books. Based on abductive coding, we made coding schemes in which we mapped processes of stigma and social integration, accompanied by additional terms that emerged from the data (e.g. 'coping with stigma'). A selection of codes was used as input during the exploratory data analysis (EDA). The next step was EDA followed by data visualization of the analysis. During the EDA and data visualization phase, we organized two focus groups. The first focus group was attended by seven mental health care professionals; the second focus group was attended by six authors of patient stories about psychosis.

We sought to delve into the preliminary text mining results while gaining insights into participants’ perceptions of this method—its meanings, benefits, and limitations. During the focus groups, we assessed how the emerging themes related to participants’ experiences and professional knowledge. The focus groups were semi-structured: standard aspects of the focus groups were an introduction to the research project and aims, the presentation of visualizations and reflection by participants based on three open questions (1) What surprises (in positive or negative way) you in the visualization? (2) What do you recognize in the visualization? (3) What do you miss in the visualization?

The second project took place in September–October 2021. This project focused on the theme of dementia using patient narratives. We explored different ways to analyze patient stories, first by organizing a reading club and later by organizing a text mining workshop. In this project, we initially focused more on the combination of qualitative research and citizen science, with computational methods being included only later. We started with one book on the theme of dementia selected from the patient stories collection, which formed the basis for the reading club and workshop. A reading club was organized which was attended by two researchers and seven citizens. A month after the reading club a focus group with two data and software engineers, seven researchers and five informal caregivers was organized where text mining was the theme.

As part of the EDA, a word cloud tool was built in Python. This tool generates a vast array of word clouds of the books based on input term(s) and position of the sliders (see figure 1). The descriptive statistics of the books revealed that the books contain substantial noisy data, such as articles and adverbs. For this TF-IDF (Term Frequency–Inverse Document Frequency) is processed in the tool. This metric allows for comparison of document word frequencies weighted against the corpus of books. The maximum document frequency (max_df) and maximum verb frequency (max_verb_df) set the upper limits to filter out overly common words and verbs, respectively, while the minimum document frequency (min_df) sets the lower limit for how often a word must appear to be included. Nouns, bigrams, verbs and proper nouns are further processed in the next four boost sliders, which increase the score of certain types of words, making them more prominent in the word cloud. The use of bigrams (two-word sequence of words) gives a better comprehension of the concepts used in the books. The last slider determines how many words before and after a given term the word cloud is based on, allowing you to adjust the context displayed around the input term(s). When using the word cloud tool on our dataset, the use of input term(s), bigrams and window size proved to be essential in contextualizing a certain theme. This interactive word cloud tool allows users to zoom in on certain concepts by using parameters like window size over the text to be worked with an application of the KWIC (keyword in context approach) and possibilities, in the form of sliders, for adjusting the weight of TF-IDF, bigrams, nouns, and verbs. We used this tool to create various word clouds around our chosen themes in the experiments, then conducted a brainstorming session to evaluate and select word clouds that appeared most thought-provoking for each focus group, besides having the option to use the tool on site.

Figure 1.

Word cloud tool.

Data analysis

In the analysis, we reflected on the process and outcomes by analyzing our collection of meeting notes, focus groups and workshops of the conducted experiments. The focus groups and workshops were recorded and transcribed verbatim. The transcripts were read, discussed multiple times, and coded using Atlas.ti. The analysis took an abductive approach, by basing our codes on both our literature review and the data itself. At the start, we recognized two overarching themes in the focus groups. The initial theme revolved around the participants’ perspectives on the methodological approach, encompassing reflections on the dataset and the analytical methods employed. The subsequent theme centered on the discourse pertaining to substantive healthcare matters, wherein participants shared their experiential knowledge and established connections with healthcare practices. We first identified those moments and each one of them was further categorized into subthemes, with the first theme being predominantly present.

Following the initial data analysis, the focus of analysis was refined to identify generative aspects of the experiments. The central question in this analysis focused on what the experiments evoked among the participants and us as researchers and how insights could be used for future research for combining these three methods.

Ethical considerations

Informed consent was obtained from all participants. Ethical approval was obtained for the focus group with authors of stories about psychosis. For other focus groups ethical approval was not required and we followed regular practices of informed consent. The dataset consisted of patients’ stories that were already publicly available.

Experiments: A generative reading

As outlined earlier, there are two distinct ways in which the kinds of experiments we conducted can be understood. In the results we interpret the experiments from a generative perspective, focusing on what kind of effects the experiments set into motion, how they provoked debate, opened taken-for-granted assumptions, and explored new connections and ideas (cf. Wehrens, 2019). Through this generative 'lens', we discuss three main effects as illustrated in figure 2, which we categorize as the experiment as a site of controversy, the experiment as mediator and the experiment as tin opener.

Figure 2.

Generative effects of experiments.

The experiment as a site of controversy

The experiment with the stories of people who experienced psychosis has led to multiple moments where eyebrows were raised, confusion predominated, and skeptical thoughts were shared. These moments occurred among all actors involved. During the focus groups, some moments of unease were evident among the participants. The presentation of visualizations derived from the EDA elicited varied reactions among the participants. The following vignette illustrates such sentiments.

Vignette 1¹

“Human? Way too general!”

After participants, who themselves were authors of patient stories, asked their questions about the research project on using patients’ stories on psychosis for analyzing stigma and social integration, the first word cloud of a series was shown: “Of course, this is just a simple word cloud (…)” was an immediate response from one of the participants. One participant mentioned the fact that is it interesting that the word “human” was the biggest word depicted instead of a typical medical term. Other participants did not share this sentiment and felt nothing for the word clouds as they did not bring forward certain terms that were important according to them: “From what I understand, there was a search for most common words (…) Yes, then such a word as ‘human’ comes up, but this does not seem to me to be in any way interesting….” These sentiments were explained by stating which themes participants felt were absent from the word clouds or appeared smaller than expected. Some explained this by stating that from their experience certain words in the word cloud should be bigger, as they found these more relevant to the theme the word cloud is trying to depict: “I would expect it [self-stigma] would be bigger in this cloud, I think well, that it should be, bam, very big in the visual.” Consequentially, the potential of computational analysis in comparison to human analysis was considered low: “Obviously, a computer cannot read or extract the way humans can read.” Later someone else continued: “The computer is unable to get to the core [of those stories], we can do that much better based on our insights [as humans].”

The vignette above describes a multifaceted discussion following the presentation of some word clouds. As we shared the word clouds beforehand in preparation for this specific focus group, participants did not need much time to interpret them. For this reason, the discussion started instantaneously. The skepticism shared indicates that the analysis made and the word cloud resulting from it were perceived as an unwarranted reduction of patient experiences in different ways. Other types of interactions with patients and family members, such as the reading club we organized about the topic of dementia, were perceived to be less problematic, as this format allowed participants to place a greater emphasis on analyzing the book's narrative, and 'disentangling' the one story and later by comprehending its meaning by relating to the story. According to some participants, the case of analyzing aggregated patient stories on psychosis was in direct contradiction with the essence of these stories: “I think the material you’ve chosen, patient stories, is actually trying to get away from the general: it is all about telling your own story, your own experience.” This shows how text mining was perceived as being unable to grasp the invisible red thread in the narratives: things which cannot easily be represented through text mining tools but are nevertheless visible as a common theme. The word cloud was therefore perceived to miss out on the main message of the storyteller. While participants understood the rationale behind aggregating stories, seeing their narrative in such a general way made them doubt if their main messages, as someone who experienced psychosis, came across. This feeling may have been even stronger as some of the patient stories of the authors were included in the analysis.

The authors also considered the distinction between computational logic and human interpretation. Specifically, this can be observed in the varying approaches to defining the significance of a word's size in the word cloud. Computational logic or 'distant reading' dictates that size, and thereby its relevance, is contingent upon the frequency of the word's occurrence within a selected text passage of a book. Although we applied TF-IDF to filter out common words like frequently used verbs, the word cloud is still driven by word frequency. Conversely, human interpretation deems size to be connected to the perceived significance of the term by interpreters, regardless of the actual frequency with which it is mentioned. The word 'self-stigma' was not the largest in the word cloud; it was, however, deemed as the most pertinent issue within the theme of stigma by the authors.

Feelings of confusion were also present among professionals when the word clouds were presented. Where the authors used their own experience of having dealt with psychosis and writing about those experiences to mirror the word cloud, professionals used their experience of contact with patients in their work. While they could understand why some words were visible in the word cloud, the inclusion of other words led to a kind of puzzlement. When certain words or word groups were hard to connect to their own practices this led to curiosity about the context of a certain word use: professionals became interested in understanding what certain words used by patients in their books meant. The words they could not place led to curiosity about the data used for the analysis: “I am curious about the people and their motivations to write these books. That is important to know, because I do not think that these patients are necessarily representative of the patients we see in our daily work” (Psychiatrist). The words which they could not make sense of led to confusion, which led to questioning the dataset and the way the word cloud was generated, thereby reflecting on the input words used for generating the word cloud.

Feelings of skepticism were also present when participants (authors) shared which words the word clouds were missing, as described in the vignette above. By stating that authors missed words like 'self-stigma' and 'trust', they questioned whether the word cloud produced an accurate representation of their experiences. A similar dynamic occurred in the focus group with professionals, who also shared certain words they would have expected in the word cloud, like 'fatigue'. By stating where the computer analysis falls short, in this case catching certain themes, one also shows what is deemed as important.

During the project, feelings of discomfort or unease were also present among researchers regarding decisions that had to be made about the research approach. All researchers involved entered a new territory, as we went beyond the borders of our disciplines. As social scientists, we encountered a distinct working style among data scientists in comparison to ours, which required us to adjust our regular working style. In our meetings, we occasionally struggled to align our respective approaches. For instance, data scientists insisted multiple times on the importance of selecting a larger dataset from the collection of patient stories available and finding ways to structure the books, as they are highly unsystematic. Although we acknowledged that the dataset is challenging, for us (social scientists) it was important to develop new ways of analyzing these underused patient stories which would enable interpretive work. Thus, given the diverse disciplinary backgrounds, we encountered differing perspectives on the methodological approach, which occasionally led to the need to conduct additional coordination work.

The experiment as mediator

Above we highlighted the effects of the experiment as a site of controversy—triggering different affective responses by researchers and participants, which were mostly related to the methodological approach of analyzing patient narratives. However, not only did the experiment lead to moments of controversy, but it also facilitated new connections and produced new forms of mutual understanding. Specifically, the experiments acted as a mediator in different ways: (1) between professional practice and experiential knowledge; (2) between researchers and participants and (3) between researchers. The vignette below describes moments where bridges were built between the professional practice of mental health professionals and the experiential knowledge derived from patient stories.

Vignette 2

Connecting to the language of patients

During the reflection process of the focus group with mental health care professionals, participants discuss their thoughts on the experiments with the books about experiences with psychosis. “I recognize many things from the word cloud. When it comes to future analyses, I also think that it is important to distinguish between the different phases patients go through, so for instance the acute phase and the rehabilitation phase.” According to this nurse, this distinction in phases helps to better attune to the needs of patients at a certain moment, as these needs can differ depending on the moment of the care process.

After having presented and discussed the word cloud visualizations of stigma and social integration, participants were quick to link this to their own clinical practices: “Especially if you look at our goal, which is to make sure patients don't relapse. They are admitted to the clinic during the acute phase and then we try to support them and make them better and we hope that with good support they don't relapse. However, we still see that relapses do happen. So, I can imagine if we have a better view of what is important to them even after admission that I indeed can benefit from that. These insights would be helpful for me to use when they are admitted again.”

Additionally, it was emphasized that it is valuable to consider what patients express in their books, as it may differ from what is shared during consultations. Another professional specifically saw the link with using experiential knowledge from these books for psychoeducation. “I see opportunities to use this method to find words in psychoeducation to patients. This might be a way to find words or connect to how I employ them in the clinic. This would be an interesting way of doing that, that you could say based on books by experts by experience these are themes that come up and then ask if the patient is able to relate.” By “using the words of patients,” the professional expected to improve his work practices.

During the focus groups, the interactions with the word cloud sparked certain connections among the participants. When interpreting the word cloud, professionals took a moment to reflect on what patients and their relatives told them. In this sense, the word cloud triggered people to think about the words depicted. These moments allowed for reflection among professionals on their own contact with patients and their relatives. Vignette two shows the willingness of professionals to learn from the insights they gain and produce ideas on how the experiences of patients can be used in their work. This is valuable as patient experiences often raise the question of how to use them in professional practice. By stating their interest in the words of patients, professionals acknowledge that their knowledge about patients is limited in many ways, for instance because of the limited time available in the consultation room to understand the experience of the patient in a more holistic way. A word cloud functioned as tool that facilitated ideas on how the experiences can be applied in practice by patients and professionals, thereby bridging professional practice and experiential knowledge of patients.

The experiment also functioned as a mediator between the research group and other participants. We saw how participants gradually became more invested in the subject matter. From our side, we tried to facilitate this by balancing information. On the one hand, we aimed to provide a sufficiently in-depth explanation of the entire research process and all the methodological choices we made, but on the other hand, we recognized the importance of simplification to enhance understandability, even if this meant making some elements of our design less explicit. Creating moments of interaction by asking for input terms for generating word clouds helped to understand how the tool works. This also led to the fact that participants could understand the main idea of text mining and rationale behind the research approach.

The word cloud tool instigated different discussions about both methodological and substantive matters. Stated more explicitly, 'staging' the insights of patients through the word cloud tool during the experiment worked as a mediator as it enabled a more substantive debate that was deemed valuable for participants. For this debate to happen, bringing together different experts by experience created a 'sense of commonality' (Meriluoto and Kuokkanen, 2022), which allows for engaged discussion. As one of the participants (an author of one of the books) concluded at the end of the focus group: “(…) the added value only comes when people start interpreting it. That value only comes when we are in this conversation, and we use such a tool like a word cloud (…). And that we then talk about that, and what we can do with our so-called experience. In this way, we take it further because we are collectively involved in a conversation and only then do new insights arise.”

The experiments also functioned as mediator between social scientists and data scientists. Our collaboration knows certain frictions, as discussed earlier, but at the same time, we saw that our working style changed as our collaboration continued: it became more interdisciplinary. By discussing the progress of the research and by organizing meetings where we work together as researchers, we could reflect on the methodology. This style of working also led us to the development of the word cloud tool, which featured elements of both text mining and qualitative research.

The experiment as tin opener

The themes mentioned above show that these experiments allowed for a space for opening up frictions and facilitating different connections. However, these environments also created more substantive discussions. In other words, we argue that the experiment also functioned as a tin opener, creating the opportunity for participants and researchers to think more deeply about a certain topic or word. The vignette below discusses one of these moments.

Vignette 3²

Word cloud as a catalyst for deeper discussion

During the focus group in the dementia project, participants, consisting of family caregivers and researchers, were able to create input terms for generating their own word clouds. They made different suggestions and deliberated on input terms. At one moment person A (researcher) suggested the input word “diagnosis,” which the data scientist inserted in the tool, generating a relatively small word cloud. Like all other word clouds, participants inspected the words depicted, and person B (informal caregiver) stated how she found the word 'time' intriguing in this word cloud: “The word ‘time’ is intriguing to me [in this word cloud] because before people reach a diagnosis, a lot of time has often passed during which people have struggled (…). It's a difficult period of: what's really going on?” She described the role of time in relation to the diagnosis of dementia and stressed the importance of an early diagnosis. This was taken up by person C (informal caregiver) who agreed with this notion and related this to her own experiences of how an early diagnosis can help in arranging care but also in adjusting to the person with dementia as a family by preventing loneliness and using certain apps for facilitating meaningful interaction in early stages. While these and other benefits were acknowledged by all other participants, it was quickly recognized that the taboo and stigma surrounding dementia were the main hindrances. Participants explained how people with dementia fear stigma and therefore ignore their dementia symptoms or refrain from telling others about their disease, as they are afraid of being treated differently. Person D (informal caregiver) specified this by saying that persons with dementia are often belittled in conversations. Person E (informal caregiver) agreed and added that in his experience the diagnosis is a very emotional process which can take a long time. Participant B concluded how this stigma takes away useful and precious time from patients with dementia and their families which could be used to arrange good care, but also for families to not delay certain activities with their relative with dementia.

The moment described above shows how the word cloud tool instigated interaction among the participants. Participants provided input words and consequently saw 'their' word cloud depicted instantaneously. In the vignette above, the word 'time' was recognized in the generated word cloud, which catalyzed a discussion about the importance of time when it comes to the diagnosis of dementia. This shows that the words that participants picked from the word cloud were the ones they could relate to their own experience. They could give examples from their own experiences and share their view on the points raised by other participants. Furthermore, it is interesting how the theme of 'time' was analyzed in-depth by participants, by linking the difficulties of an early diagnosis with stigma and consequently linking to how this impacted their lives. Another link was made between changes over time with regard to the image of dementia: “It has very much to do with that old image of grandparents [with dementia] who used to know nothing anymore. And while in our population today it [dementia] is much more prevalent (…) I’m also a grandmother, and if I had dementia now, I would certainly be a very different person with dementia than my grandmother with dementia.” This is because she expects that aspects like an early diagnosis and being able to use your time well would change the disease trajectory. Also remarkable is the fact that all informal caregivers who were present engaged in the discussion on temporality and dementia.

Not only did the word cloud tool instigate discussions on a substantive level, like in the above-mentioned example, but it also showed that the experiments can work as a 'tin opener' that can bring to the forefront various aspects of experiential knowledge of patients and informal caregivers. While the critique on computational text analysis was present, particularly among authors, all participants did not dismiss text mining for analyzing patient narratives in general. Rather, the discussion continued about the value of patient stories and the methodological approach to analyze these. For instance, participants who authored books proposed ideas for structuring the dataset of patient stories, with one emphasizing the importance of distinguishing different stages of an experience, such as identifying which parts of a psychosis narrative were written before, during, or after a psychotic episode. The suggestion states the importance of adding certain metadata to the datasets, or, from a qualitative research perspective, points towards the recognition of understanding the context in which books are written.

Discussion

The task of combining different methods for analyzing patient narratives proved to be a complex process that requires thorough evaluation and analysis. The experiments conducted showed us what the combination of methods can potentially bring, thereby looking beyond the representational logic of experiments, by observing their generative logic. Table 1 provides an overview of the key differences between both logics. In the remainder of the discussion, we situate our experiments in ongoing scientific conversations on the challenges and opportunities of the different methods and its combinations, thereby focusing in particular on the issue of meaningful reductions.

Table 1.

Comparison of representational logic of experiments and generative logic of experiments.

	Representational logic of experiments	Generative logic of experiments
Function	'Mirrors of nature': Experiments represent an accurate representation of real-world phenomena	'Sites of emergence': Experiments as a tool for situated reflection and fostering creativity
Type of knowledge	Objective of knowledge	'Thickly descriptive' knowledge
Analysis	Analysis focused on predefined variables and specific parameters with an emphasis on quantitative methods	Analysis allows for broader exploration, often including qualitative insights and emergent patterns
Measurement techniques	Relies on standardized and validated methods, often using quantitative tools to ensure precision and accuracy	Can incorporate both traditional and innovative methods, allowing for flexible and adaptive measurement approaches
Role of researcher	The researcher maintains strict controls, designs robust experiments, and ensures replicability by reducing bias	The researcher encourages experimental flexibility, explores new possibilities, and embraces unexpected outcomes

Narrative reduction

Participatory research inherently aims to capture diverse perspectives on a given topic, fostering an environment where disagreement and varying viewpoints are expected (Stewart, 2013). Within STS, socio-technical controversies serve as spaces where different actors with varied knowledge, values, and experiences come together and engage in conflicts (Baibarac-Duignan and de Lange, 2021). These controversies provide opportunities for diverse perspectives to be expressed and examined. The use of computational methods for written patients’ stories prompted several critical reactions in our research. Concerns arose regarding the potential loss the rich and complex character of qualitative data when computational methods are employed. This process where the multifaceted and rich nature of narratives turn into more structured quantifiable forms as summarizations is what we refer to as narrative reduction. This was the case in our study because, participants felt that key to the value of patient stories is the rich context they provide for understanding the patients’ perspectives. These concerns stemmed from inquiries regarding the material itself, given its inherently individual nature, and the appropriate methodologies for analyzing it. Reductionistic effects of computational analysis have been flagged earlier as problematic (Kitchin, 2014). The work of Carlsen and Ralund (2022) demonstrates how Latent Dirichlet Allocation (LDA), one of the most used forms of unsupervised methods, might not always be suitable for discovering relevant topics in large textual datasets, thereby arguing for a computer-assisted workflow with human interpretation guiding the process. This suggests an important role for researchers to recognize narrative reductions and mitigate them, for instance, by using qualitative interpretative analysis in early stages of the research process.

Narrative reduction is of course happening in all manners of analysis of rich stories such as the ones we are interested in. While such reductions may be inherent in the research process, our analysis shows that these reductions are not exclusively problematic, instead, they show that they can be meaningful when combined with other methods. This integration makes meaningful reductions possible. This does ask for working between two extremes. One side believes in preserving the unique authenticity of individual stories by performing close reading of a small selection of patients’ stories, while the other side data optimists advocate for collecting large amounts of data for machine learning to gain broader understanding. Similar discussions about scaling up arise in the context of citizen science, where tensions exist between techno-centric and people-centric approaches. For instance, crowdsourcing, which holds a techno-centric approach, can compromise the active involvement of citizens in knowledge co-production, thereby questioning the people-centric aspects of citizen science (Ciasullo et al., 2021).

By stating that there is something 'in the middle', this work builds up on earlier work of combinations of qualitative and computational research (e.g. Blok and Pedersen, 2014). But how does one navigate the middle between these two extremes? What are important aspects to consider in such interdisciplinary endeavors? How does sense-making and meaning-making happen in such efforts?

Importance of collaborative sense-making and meaning-making

The experiments have shown the importance of having a dedicated phase of collaborative sense-making and interpretation when it comes to analyzing patients’ stories, by including patients, their relatives, and professionals in the analytical process. The use of a word cloud tool played a significant role in eliciting collective interpretation. It proved to work as a mediator and a tin opener in fostering insightful discussions and connections.

During the focus groups, the interactions with the word cloud sparked certain connections among the participants. This tool triggered reflection among professionals, prompting them to contemplate the words depicted and consider the perspectives of patients and their relatives. Such moments led to discussions on how these experiences could be integrated into their professional practice. This is interesting as including patients’ perspectives in healthcare is something that is recognized as being of great importance, however, practice shows that there is still need for ways to promote this in practice (Smith et al., 2023). The word cloud tool thus served as a mediator between different participants, fostering a sense of commonality and facilitating engaged discussions which generated some useful entry points for health care improvement.

The experiment's function as a tin opener is evident in the discussions it generated. Participants’ input words led to the immediate visualization of personalized word clouds, sparking discussions related to their own experiences. The use of the word cloud tool not only encouraged substantive discussions but also provided a platform for participants to delve into various aspects of experiential knowledge. By reflecting on the methodological approach, participants proposed ideas for refining the research process. So, both mediator and tin opener effect showed that the experiments can function in a way that experiential knowledge becomes central in the discussion.

Not all computational methods turned out to be equally useful when it comes to facilitating this collaborative sense-making. Word clouds, despite their narrative reduction, turned out to be a good first step to start a discussion about the use of text mining for analyzing patient stories and at the same time make place for valuing experiential knowledge of participants. The field of digital studies offers inspiration for ongoing experiments, for instance, through the concept of critical analytics, which encourages the use of alternative metrics to assess modes of engagement rather than relying solely on mainstream metrics in analyzing social media data (Rogers, 2017; Madsen and Munk, 2019). This points to the value of experimenting with different forms of computational methods for analyzing patient stories. Although our future goal is to work with larger datasets of patients’ stories, starting small was essential to facilitate participatory research and enable collaboration. For instance, the insights of the focus group opened up new ideas for adding relevant metadata to the dataset, which we can use for current and future research projects. That is why including citizens in these early stages of research is important, for instance during EDA, also because it allows for sense-making and meaning-making processes. Based on this, organizing data sprints (Munk et al., 2019), with multidisciplinary research teams and citizens, seems to be promising for better facilitating meaning-making processes of patients’ stories.

Meaningful reductions

These initial experiments have not only paved the way for tapping into patient narratives to glean valuable insights into healthcare practices but have also shed new light on the application of computational methods in the context of written patient stories. While the reductions these methods create could be critically interpreted as a way to 'break' narratives, resulting in a loss of their intricate depth due to simplification, they hold the potential to be reassembled through the integration of various approaches from qualitative research and citizen science. Such epistemological framing contributes to combining different methods for analyzing and interpreting patients’ perspectives within social science (Kitchin, 2014). A collective interpretation process, marked by the active involvement of individuals and the establishment of a conducive environment for collaborative sense-making and meaning-making processes, proved to be key in such interdisciplinary endeavors. Hence, it is important to facilitate moments where underutilized knowledge, like experiential knowledge of patients can thrive, thereby promoting epistemological pluralism (Miller et al., 2008). In this way, our research contributes to the call in STS 'remake' participation (Chilver and Matthew, 2020), by intervening in participatory methods that aim to build toward more reflexivity and responsibility.

However, translating the lessons derived from this analysis into tangible enhancements for healthcare practices necessitates further research. More research in the forms of experiments for bridging computational methods, qualitative methods and citizen science should be conducted to learn more about how narratives can be analyzed and how this process can be used for meaning-making of qualitative data, like patients’ stories. The interdisciplinary nature of these approaches, both in prior research and within the scope of this study, serves as a double-edged sword, simultaneously presenting challenges and catalyzing innovation. While it is important to emphasize the challenges and pitfalls associated with different analytical frameworks in diverse epistemic cultures (Knorr Cetina, 1999), it is equally important to consider the generative aspects that these experiments foster and reflect on the meaning-making processes. Given that reductions are inherent in such processes, it becomes crucial to ask the question: what constitutes meaningful reductions?

Footnotes

Acknowledgments

We thank Jiwon Jung and Lea Jabbarian for their collaboration in the Open Mind project. We also thank Erica Witkamp, Janne Papma, Jeanne Brooijmans, Sandra Rijnen, Judith Rietjens and Iris Wallenburg for their collaboration in the dementia project. Special thanks go to Peter van Huisstede and Jasper op de Coul for collaborating in both projects. We thank our respondents for their participation. We are also grateful to the authors of the books included for sharing their story. Special thanks to Iris Wallenburg for her valuable feedback on the early drafts of this paper. We also want to thank our Department of Health Care Governance for their feedback during one of our seminars.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the OPEN MIND Call Convergence 2021.

ORCID iDs

Nada Akrouh

Rik Wehrens

Hester van de Bovenkamp

Notes

References

Adler-Nissen

Eggeling

Wangen

(2021) Machine anthropology: A view from international relations. Big Data & Society 8(2): 20539517211063690.

Allen

Lees

Cohen

, et al. (2019) Collaborative workshops for community meaning-making and data analyses: How focus groups strengthen data by enhancing understanding and promoting use. International Journal of Environmental Research and Public Health 16(18): 3352.

Andrade

Andersen

(2020) Digital story grammar: A quantitative methodology for narrative analysis. International Journal of Social Research Methodology 23(4): 405–421.

Antons

Grünwald

Cichy

, et al. (2020) The application of text mining methods in innovation research: Current state, evolution patterns, and development priorities. R&D Management 50(3): 329–351.

Aronson

(2020) Story as evidence, evidence as story. JAMA 323(17): 1694–1695.

Baibarac-Duignan

de Lange

(2021) Controversing the datafied smart city: Conceptualising a ‘making-controversial’ approach to civic engagement. Big Data & Society 8(2): 20539517211025557.

Baumer

Mimno

Guha

, et al. (2017) Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence? Journal of the Association for Information Science and Technology 68(6): 1397–1410.

Bjerre-Nielsen

Glavind

(2022) Ethnographic data in the age of big data: How to compare and combine. Big Data & Society 9(1): 20539517211069893.

Blok

Carlsen

Jørgensen

, et al. (2017) Stitching together the heterogeneous party: A complementary social data science experiment. Big Data & Society 4(2): 2053951717736337.

10.

Blok

Pedersen

(2014) Complementary social science? Quali-quantitative experiments in a big data world. Big Data & Society 1(2): 2053951714543908.

11.

Bornakke

Due

(2018) Big–thick blending: A method for mixing analytical insights from big and thick data sources. Big Data & Society 5(1): 2053951718765026.

12.

boyd

Crawford

(2012) Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication & Society 15(5): 662–679.

13.

Buyalskaya

Gallo

Camerer

(2021) The golden age of social science. Proceedings of the National Academy of Sciences 118(5): e2002923118.

14.

Carlsen

Ralund

(2022) Computational grounded theory revisited: From computer-led to computer-assisted text analysis. Big Data & Society 9(1): 20539517221080146.

15.

Charon

(2001) Narrative medicine: A model for empathy, reflection, profession, and trust. JAMA 286(15): 1897–1902.

16.

Chilver

Matthew

(2020) Remaking participation in science and democracy. Science, Technology, & Human Values 45(3): 347–380.

17.

Ciasullo

Carli

Lim

, et al. (2021) An open innovation approach to co-produce scientific knowledge: An examination of citizen science in the healthcare ecosystem. European Journal of Innovation Management 25(6): 365–392.

18.

De Graaff

Stoopendaal

Leistikow

(2019) Transforming clients into experts-by-experience: A pilot in client participation in Dutch long-term elderly care homes inspectorate supervision. Health Policy 123(3): 275–280.

19.

Delgado

Lein Kjølberg

Wickson

(2011) Public engagement coming of age: From theory to practice in STS encounters with nanotechnology. Public Understanding of Science 20(6): 826–845.

20.

DiMaggio

(2015) Adapting computational text analysis to social science (and vice versa). Big Data & Society 2(2): 2053951715602908.

21.

Downey

Zuiderent-Jerak

(2017) Making and doing: Engagement and reflexive learning in STS. In: Miller ES-D

Felt

Fouche

(eds) The Handbook of Science and Technology Studies, Fourth Edition. Cambridge: MIT Press, 223–253.

22.

Dreisbach

Koleck

Bourne

, et al. (2019) A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data. International Journal of Medical Informatics 125: 37–46.

23.

Egher

(2023) Digital Healthcare and Expertise: Mental Health and New Knowledge Practices. Singapore: Palgrave Macmillan.

24.

Filipe

Renedo

Marston

(2017) The co-production of what? Knowledge, values, and social relations in health care. PLoS Biology 15(5): e2001403.

25.

Frank

Arthur W

(1995) The Wounded Storyteller: Body, Illness, and Ethics. Chicago: University of Chicago Press.

26.

Frank

(2013) The Wounded Storyteller: Body, Illness and Ethics. Chicago: University of Chicago Press.

27.

Fricker

(2007) Epistemic Injustice: Power and the Ethics of Knowing. Oxford: Oxford University Press.

28.

Geertz

(1973) The Interpretation of Cultures: Selected Essays: Selected Essays. New York: Basic Books.

29.

Gijsel

Huyse

Van Hoyweghen

(2019) Citizen Science. Hoe burgers de wetenschap uitdagen.

30.

Gomart

Hajer

(2003) Is that politics? For an inquiry into forms in contemporary politics in Joerges, B. and Nowotny, H. (eds): Social Studies of Science and Technology: Looking Back, Ahead. Dordrecht: Kluwer Academic Publishers, 33–61.

31.

Greenhalgh

Wengraf

(2008) Collecting stories: Is it research? Is it good research? Preliminary guidance based on a Delphi study. Medical Education 42(3): 242–247.

32.

Grimmer

Stewart

(2013) Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis 21(3): 267–297.

33.

Heerings

van de Bovenkamp

Cardol

, et al. (2022) Ask us! Adjusting experience-based codesign to be responsive to people with intellectual disabilities, serious mental illness or older persons receiving support with independent living. Health Expectations 25(5): 2246–2254.

34.

Heyen

Gardecki

Eidt-Koch

, et al. (2022) Patient Science: Citizen Science involving chronically ill people as co-researchers. Journal of Participatory Research Methods 3(1).

35.

Hoeyer

Wadmann

(2020) ‘Meaningless work’: How the datafication of health reconfigures knowledge about work and erodes professional judgement. Economy and Society 49(3): 433–454.

36.

Iedema

(2022) The problem with… using stories as a source of evidence and learning. BMJ Quality & Safety 31(3): 234–237.

37.

Jemielniak

Przegalinska

(2020) Collaborative society.

38.

Jones

Pietilä

(2020) Personal perspectives on patient and public involvement–stories about becoming and being an expert by experience. Sociology of Health & Illness 42(4): 809–824.

39.

Jung

(2023) Developing Data-enabled Design in the Field of Digital Health .

40.

Kestemont

Herman

(2019) Kunnen machines (literatuur) lezen? Tijdschrift Voor Nederlandse Taal-en Letterkunde 135(4): 316–328.

41.

Khanbhai

Anyadi

Symons

, et al. (2021) Applying natural language processing and machine learning techniques to patient experience feedback: A systematic review. BMJ Health & Care Informatics 28(1): e100262.

42.

Khanbhai

Warren

Symons

, et al. (2022) Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care. International Journal of Medical Informatics 157: 104642.

43.

Kitchin

(2014) Big data, new epistemologies and paradigm shifts. Big Data & Society 1(1): 2053951714528481.

44.

Knorr Cetina

(1999) Epistemic Cultures: How the Sciences Make Knowledge. Cambridge: Harvard University Press.

45.

Kullenberg

Kasperowski

(2016) What is citizen science?–A scientometric meta-analysis. PloS one 11(1): e0147152.

46.

Leonelli

Beaulieu

(2021) Data and Society: A Critical Introduction. California: Sage.

47.

Lin

Y-w

(2012) Transdisciplinarity and digital humanities: Lessons learned from developing text-mining tools for textual analysis, in Berry, D.M. (ed.) Understanding Digital Humanities. Basingstoke: Palgrave Macmillan: 295–314.

48.

Lury

Wakeford

(2012) Inventive Methods: The Happening of the Social. New York: Routledge.

49.

Madsen

Munk

(2019) Experiments with a data-public: Moving digital methods into critical proximity with political practice. Big Data & Society 6(1): 2053951718825357.

50.

Meriluoto

Kuokkanen

(2022) How to make sense of citizen expertise in participatory projects? Current Sociology 70(7): 974–993.

51.

Miller

Baird

Littlefield

, et al. (2008) Epistemological pluralism: Reorganizing interdisciplinary research. Ecology and Society 13(2): 46.

52.

Moats

Seaver

(2019) “You social scientists love mind games”: Experimenting in the “divide” between data science and critical algorithm studies. Big Data & Society 6(1): 2053951719833404.

53.

Moretti

(2005) Graphs, Maps, Trees: Abstract Models for a Literary History. London: Verso.

54.

Munk

Olesen

Jacomy

(2022) The thick machine: Anthropological AI between explanation and explication. Big Data & Society 9(1): 20539517211069891.

55.

Munk

Tommaso

Meunier

(2019) Data sprints: A collaborative format in digital controversy mapping. In: Vertesi, J. and Ribes, D. (eds) Digital STS: A field guide for science & technology studies. Princeton: Princeton University Press, 472–496.

56.

Nelson

(2020) Computational grounded theory: A methodological framework. Sociological Methods & Research 49(1): 3–42.

57.

Németh

Koltai

(2021) The potential of automated text analytics in social knowledge building. In: Pathways Between Social Science and Computational Social Science: Theories, Methods, and Interpretations. Cham: Springer, 49–70.

58.

Oldenhof

Wehrens

(2018) Who is ‘in’ and who is ‘out’? Participation of older persons in health research and the interplay between capital, habitus and field. Critical Public Health 28(3): 1–13.

59.

Platenkamp

(2018) van Stichting Coleta’s Chronische Circus met patiëntervaringsverhalen naar Patiëntwetenschappen.

60.

Rogers

(2017) Otherwise engaged: Social media from vanity metrics to critical analytics. International Journal of Communication 11: 1–23.

61.

Smith

Villar

Wendel

(2023) Narrative-based learning for person-centred healthcare: The caring stories learning framework. Medical Humanities 49(4): 583–592. medhum-2022-012530.

62.

Stevens

Wehrens

de Bont

(2020) Epistemic virtues and data-driven dreams: On sameness and difference in the epistemic cultures of data science and psychiatry. Social Science & Medicine 258: 113116.

63.

Stewart

(2013) What is the point of citizen participation in health care? Journal of Health Services Research & Policy 18(2): 124–126.

64.

Thorp

(2023) ChatGPT is Fun, but not an Author. Science, 379: 313.

65.

Turnhout

Metze

Wyborn

, et al. (2020) The politics of co-production: Participation, power, and transformation. Current Opinion in Environmental Sustainability 42: 15–21.

66.

van de Bovenkamp

Platenkamp

Bal

(2020) Understanding patient experiences: The powerful source of written patient stories. Health Expectations: An International Journal of Public Participation in Health Care and Health Policy 23(3): 717.

67.

van Dis

Bollen

Zuidema

, et al. (2023) ChatGPT: Five priorities for research. Nature 614(7947): 224–226.

68.

Vickers

(2016) Illness narratives. In: Smyth

(ed) A History of English Autobiography. Cambridge: Cambridge University Press, 388–401.

69.

Walsh

Dwumfour

Cave

, et al. (2022) Spontaneously generated online patient experience data-how and why is it being used in health research: An umbrella scoping review. BMC Medical Research Methodology 22(1): 1–41.

70.

Wang

(2017) Education policy research in the big data era: Methodological frontiers, misconceptions, and challenges. Education Policy Analysis Archives 25: 94–94.

71.

Wehrens

(2019) Experimentation in the sociology of science: Representational and generative registers in the imitation game. Studies in History and Philosophy of Science Part A 76: 76–85.

72.

Ziebland

Hunt

(2014) Using secondary analysis of qualitative data of patient experiences of health care to inform health services research and policy. Journal of Health Services Research & Policy 19(3): 177–182.

73.

Zuiderent-Jerak

(2015) Situated Intervention: Sociological Experiments in Health Care. Cambridge & London: MIT Press.

Engaging citizens in experiments with computational analysis of patient stories: From unwarranted reductions to meaningful insights

Abstract

Keywords

Introduction

Theoretical framework

Patient narratives

Computational methods

Sense-making and meaning-making

Experimentation: Representational and generative registers

Methods

Context

Projects

Data analysis

Ethical considerations

Experiments: A generative reading

The experiment as a site of controversy

The experiment as mediator

The experiment as tin opener

Discussion

Narrative reduction

Importance of collaborative sense-making and meaning-making

Meaningful reductions

Footnotes

Acknowledgments

Declaration of conflicting interests

Funding

ORCID iDs

Notes

References