‘Using humanity to change systems’ – understanding the work of online feedback moderation: A case study of Care Opinion Scotland

Abstract

Objective

To gain a deeper understanding of online patient feedback moderation through the organisation of Care Opinion in Scotland.

Methods

An ethnographic study, initially using in-person participant observations, switching to remote methods due to the pandemic. This involved the use of remote observations and interviews. Interviews were carried out with the whole Scottish team (n = 7).

Results

Our results identify three major themes of work found in online patient feedback moderation. The first is process work, where moderators make decisions on how to edit and publish stories. The second is emotional labour from working with healthcare experiences and with NHS staff. The third is the brokering/mediation role of Care Opinion, where they must manage the relationships between authors, subscribing healthcare providers and Scottish Government. Our results also capture that these different themes are not independent and can at times influence the others.

Conclusion

Our results build on previous literature on Care Opinion and provide novel insights into the emotional and brokering/mediation work they undertake. Care Opinion holds a unique position, where they must balance the interests of the key stakeholders. Care Opinion holds the power to amplify authors’ voices but the power to make changes to services lies with NHS staff and services. Online moderation work is complex, and moderators require support to carry out their work especially given the emotional impact. Further research is planned to understand how patient stories are used by NHS Scotland, and the emotional labour involved with stories, from both the author and NHS staff perspective.

Introduction

Online patient feedback is increasingly used in healthcare. Many patients use and read online feedback provided on healthcare experiences, and many healthcare organisations offer patients online feedback options. ¹ There are several different platforms available to patients to provide feedback. However, not all of these platforms are equally regarded by healthcare organisations. Often organisations limit their attention to sources which are sanctioned, solicited and sought by the organisation. ² Social media platforms and blogs may be used by patients to give feedback, but these are often disregarded.

Healthcare professionals have expressed concern over unmoderated feedback being used as they are concerned about offensive comments or potential harm caused. ³ One feedback platform which addresses this issue is Care Opinion (CO). CO staff moderate the feedback stories which are submitted, to ensure they are safe to post and offer help to those in need.^4,5 Alongside working on stories, CO offers support to the National Health Service (NHS) subscribers with the feedback received. Previous research has shown that working with qualitative feedback, such as patient stories, can be challenging for NHS staff. ⁶ Patient feedback can be provided in large quantities ⁷ and may not be seen as a priority. ⁸ Therefore, having CO staff who are actively engaging with NHS staff about patient stories may help to reduce barriers on working with stories and make dealing with feedback a higher priority.

In England CO started as an unsolicited independent platform, funded by voluntary subscription by some healthcare providers but open to all to use at a basic level without subscription. This remains the model in England, sanctioned and used actively by some organisations but not others. In Scotland, the government endorsed the CO platform through awarding it a national contract for online patient feedback. This means that all subscriptions for NHS boards were funded by central Government and although most use the platform according to the most recent report, one still does not. ⁹ This adds an additional dimension for CO staff, whereby in Scotland there are three relationships to manage: to those who post stories (authors), to the healthcare organisations they work with (subscribers) and to the policy makers (awarding the contract to CO).

There have been several studies to date examining how CO is used and responded to (or not) by healthcare staff. Baines et al. identified a framework of key elements to enhance the quality of responses ¹⁰ but often ‘transparent, conversational responses’ are the least commonly found on the platform. ¹¹ Although much research has focussed on NHS England, Locock et al. used the response framework ¹⁰ to analyse the responses of one NHS Scotland board. ¹² The analysis shows that the responses performed well against the framework but there was room for improvement. In addition to analysing the quality of responses, they found that some NHS Staff were fearful of patient anonymity and reported feeling exposed on the CO platform. ¹² In addition to responding, organisations can record if a story has been used to make change but practice in doing so is highly variable.^10,12 However, it is possible that not all changes in care could be ascribed to a single story; change may also come from using feedback for learning such as being used to educate future healthcare professionals. ¹⁴ This research gives an important insight into how CO feedback is valued, responded to, and used by the healthcare service and staff.

Before the feedback reaches the subscribing healthcare providers, it is moderated by the CO team. There has been research into CO moderation by Ziewitz ⁴ and Petrakaki and Hutton, ⁵ which has demonstrated that there is considerable time and energy put into the moderation of stories. Ziewitz ⁴ undertook an ethnographic study as an embedded CO moderator in England, examining the moderation process and following stories through the journey of moderation to the NHS services. This captured the behind-the-scenes labour involved in moderation of feedback, from the careful considerations of editing and the discussion of posts to the care involved. This work details how moderation does not just end at story publication and the process is more complicated than just editing stories. Petrakaki and Hutton ⁵ investigated the role of moderation through interviewing CO staff. Their findings showed the level of care demonstrated through this moderation process, including the duty of care that moderators express for the story authors. They detail how the moderation process expresses care to story authors through support such as signposting, as well as to healthcare professionals by providing help, such as advance notification for critical stories. These studies provide important early insights into the CO moderation process but to date, much of the research has not investigated the model in Scotland which involves Government support.

There is an important body of research on online moderation in wider settings, often commercial platforms and discussion forums, notably Gillespie's Custodians of the Internet. ¹⁵ Gillespie examines the hidden but often powerful role moderators exercise, in determining which content is allowed or rejected. Ruckenstein and Turunen ¹⁶ argue that in most settings the sheer volume of activity ‘forces moderators to operate like machines’ (p. 1026), becoming more algorithmic and less person-centred. They argue for a rehumanising approach, drawing on Mol's logic of care, ¹⁷ which contrasts the consumerist model of patient autonomy and choice with a more relational and negotiated partnership of care. Similarly, Seering et al. ¹⁸ note that the predominant focus on moderation process work, tasks, duties and rule enforcement neglects the social roles and values moderators describe. Squirrell ¹⁹ explores the role of moderators in ‘mediating trust and establishing a paradigm for constructive discourse’ (p. 1910), and how they nudge, shape and negotiate in a ‘difficult balancing act’ (p. 1911) between enforcing rules but not discouraging users.

Our study builds on the previous research into moderation, both within CO and more broadly, to gain a better understanding of the moderation process, the emotional labour involved and the ‘difficult balancing act’ ¹⁹ (p. 1911) in Care Opinion Scotland between subscribing healthcare providers, government and authors.

Methods

This study was carried out as part of a wider PhD project. The researcher (EB) aimed to carry out 12 weeks of in-person ethnographic field work as a participant observer within the CO Scotland offices. Ziewitz ⁴ demonstrated that ethnographic research in this context is a viable research method and can produce important results, deepening the understanding of online feedback moderation. The fieldwork was to be carried out from March 2020 but after the first week of observations at the office, Covid-19 lockdown hit the UK and the research was conducted remotely. This resulted in the researcher carrying out remote interviews, remote observations through videoconferencing technology with individual participants (CO staff) and joining wider team meetings with the Sheffield office in Central England and observing online team group chats. This resulted in 12 h of in-person fieldwork being carried out in week 1 and 23.5 h of remote observations were carried out over a 9-week period. There were eight participants interviewed during our study, holding different roles within the CO team. Interviews were audio-recorded and transcribed. Observation notes were handwritten and typed up into electronic fieldnotes later the same day. The researcher also kept a reflective diary during the data collection and analysis.

Our initial interview topic guides were informed by the work of Petrakaki and Hutton ⁵ on CO and the initial conversations with the previous CO Scotland director. We included questions on aspects of emotional labour and care, but let the interviews be led by the participants’ experiences, allowing them to discuss aspects that they felt were important. Data analysis was carried out through a process of immersion, establishing potential themes and discussing them across the supervisory team. Selected quotations were presented by the researcher to the PhD stakeholder advisory group to discuss and reflect on our initial ideas. This group has representatives from NHS Scotland, CO, Scottish Government, and a Patient Partner. We used this process with the transcripts initially to identify a coding framework, which was then also applied to the field notes, using NVivo 12 software. These codes were derived inductively from repeated reading of the transcripts among the team with particular emphasis on different forms of labour involved in moderation. As coding progressed, we focussed also on questions of power but, after team reflection and discussion, this then evolved into a broader concept of brokerage and mediation, reflecting the four-way relationship described by participants between CO, story posters, boards and Scottish Government.

Results

Our results detail different forms of work involved in moderation and are presented thematically as process work, emotional labour and the brokerage/mediation role of CO.

The process work of Care Opinion

In this section, we discuss the process work of CO, from the moderation of stories and responses, to working with subscribing healthcare organisations. Moderation in this context is the review of online patient feedback stories, to ensure they are suitable to be made publicly available. CO moderators are tasked with reviewing and editing the stories submitted to the CO website before they are published. There are several steps involved in this moderation process, which can occur in different orders depending on the moderator. The moderation process is outlined in the moderation policy which guides staff in how they carry out their work and decision making. The stories from Scotland, England and Northern Ireland are combined into one queue, and our participants would work on stories based on English or Scottish services. This happened before the pandemic hit, but there seemed to be stronger relationships built across teams during homeworking. There are minor differences between Scotland and England story processes due to differences in the feedback landscapes and organisations. Figure 1 provides a simple summary of the moderation process, with Table 1 providing explanations of each stage. The moderation process itself may be complex depending on the story and may involve additional steps not discussed here. Although many of the moderation steps are straightforward, it will become more evident that others are, at times, more complex and required more personal judgement and opinion.

Figure 1.

A summary of moderation process, from first checking the queue to the story publication. The green box contains all story editing stages. The dashed line boxes demonstrate support options.

Table 1.

CO processes and terminology explained.

Trigger words	Words of concern that may require urgent action to protect vulnerable persons, e.g., die, kill
Story tagging	Each story is also provided with metadata called ‘tags’. These tags are words or phrases which can be used to highlight certain elements of a story, which can then be used on the website for searching or reporting functions. Public story tags were generally used for capturing the story content and feelings, as well as the positives and negatives of the experience. CO also used private tags, which allowed them to tag stories and then use the tag as a way of searching later. Story authors can also add their own tags to the story before submission.
Check queue	Moderators check the story queue for trigger words. There is also a response queue, where some responses from staff and authors are moderated.
Select a story	Moderators select a story from the queue to work on – this may be one requiring urgent safeguarding or not.
Editing story text	Moderators edit the story text to ensure patients and staff are not identifiable. Some moderators also edit grammar or spelling to improve readability.
Check author information	The moderation process also involves checking the information provided by the story authors. This involves the moderator checking the email address provided, the IP address, postcodes, and the history of previous stories (if any). This information helped moderators to better understand the story context and the author's position. For example, they could check that the author was not posting multiple stories of the same care experience, whether the author was posting about a service local to them, whether the story had been sent by a patient or staff, and to double check that the username was anonymous.
Service linkage (also known as service tagging)	Services are linked (or tagged) to stories to ensure that they were able to respond to the patient, and CO aims to get the story as close to the front line as they can. Authors could link the service involved when they submitted their story, however authors can link the wrong service, not have included all services or used a different name for a service, which was not the official one listed. The moderators used the information provided to identify the correct services in the ‘service tree’.
Adding notes	Moderators may add notes about changes made to the story before publication. For some stories with little editing, this isn't required
Deciding title	Story titles may be changed during the moderation. This is discussed further in the main text.
Criticality rating	Each story is given a criticality rating, this helps to guide CO actions on how to handle the story. This is discussed further in the main text.
Publication	Once moderation process is completed, the story is published and becomes live on the website. Author and NHS staff are sent email notification.
Safeguarding action/seek help	First line moderators may ask for second line moderation support for stories which need safeguarding, raise legal or whistleblowing issues or potentially have a higher criticality rating.
Discuss with colleagues	Moderator may ask for opinions or discuss story with their colleagues to help their decision making.
Check moderation policy	Moderators can access moderation policy to help guide their decision making when moderating stories.
Service Tree	This service tree is a digital map of the services available from healthcare providers and are kept updated by working with the subscriber staff.

The moderators generally started by reading through and editing their selected story if required. The editing process involved various changes, depending on the story and the way it was written. Stories may need editing to protect the identity of staff and patient or ensure the story reflected the author's experience and not include speculation such as on staff motivations (e.g. staff couldn't be bothered). CO staff felt that the editing process needed to be carefully administered to retain the voice of the author as much as possible, but it was necessary to reduce the risk to both patients and staff. During moderation training, it was suggested that if a patient's identity were revealed then this could potentially impact the care they received and staff being identifiable could have resulted in legal action. It was apparent during the observations that whether the story was perceived as positive or negative could impact how much editing occurred. For example, moderators were not needing to ensure anonymity if the story was praising staff.

Moderators also could edit other elements, such as story tags. From our observations moderators tended to edit tags and use established ones. These tags were selected based on the content of the story and tags could allow for story searches on particular topics, e.g. communication, vaccinations. Some moderators talked about having their ‘favourite words’ for story tags, which they added when appropriate. With negative stories, some moderators discussed trying to add at least some positive tags so that staff could see some positives highlighted in the story. Moderators also reviewed the story title that the author had written. During our observations, the moderators often changed this so that it was a quotation from the story, rather than having the name of the service used. This allowed the title to still be in the words of the author but selected by the moderator who was editing the story.

Before the stories were published, they were given a criticality rating. This is a score from 0–5 which guided their own internal decision-making processes on how to handle the story. For example, higher criticality ratings could require the CO team to contact subscribers ahead of the story publication, so that staff were aware of it and respond quickly. The moderation policy provided an outline of what each score entailed so that moderators could use this for help. From our time observing, moderators tended to discuss the stories and potential scores before deciding as they felt their own biases could influence their decisions. Moderators told us that the majority of stories were completely positive or had minor criticality, and that there was very rarely a highly critical story. When these did appear, the stories required second line moderation to confirm the score and agree the actions to be taken. During our research, it was explained that the subscribers were able to see the criticality scores of stories and were given reports on the criticality scores. However, they did not necessarily know the decision making behind each score. Despite this, we were told of some healthcare subscriber staff challenging the criticality scores given, as although they did not know why it was given, our participants felt that the staff wanted to have lower scores.

Depending on the changes carried out during the moderation process, the moderators may have needed to contact the authors to inform them. They also got in touch with authors for safeguarding if this was raised as a concern during moderation. From our time spent with the team, changing the username seemed to be the most common reason to get in touch with authors. The stories which were identified as needing second line moderation or safeguarding, were sent to a second line moderator. Second line moderation involved senior moderators making decisions on the level of safeguarding and actions required. We didn't observe this process directly during our time spent with the team but from talking to first line moderators, this seemed to mirror their own experience, as they explained that they would only then get involved with that story again if they were asked to make contact by the second line moderators.

Moderators also reviewed and edited some responses to stories. Our participants explained that the website published most responses automatically, but moderation was required for new healthcare staff responders on behalf of their organisation and the responses of story authors. These responses went into a separate moderation queue to the stories and moderators told us that they were mindful to check this, as some staff had their response times monitored. It was explained that the responses were generally moderated for the same considerations as when moderating stories, but the members of staff responding could have their full name displayed on the website as a representative responding on behalf of their organisation.

Alongside moderation, CO staff needed to work with subscribing healthcare providers to ensure that they and their staff were supported with, and embraced using, CO stories. Our research showed that CO staff generally worked with identified contacts in subscriber organisations who took lead responsibility. Each NHS subscriber had a named member of CO staff as their main point of contact and support. Our participants felt that, the CO staff worked closely with their subscriber staff to build trust and positive relationships. Examples of this involved warning if a critical story was coming out shortly or providing positive feedback when appropriate. CO have also been mindful of the additional pressures on subscribers due to Covid-19. They explained that they tried to reduce this by giving them a ‘heads up’ where they could and draw attention to positive stories of care to be shared in an attempt to boost staff morale. At times, these choices seem to take the action as a form of care but also demonstrate the influence CO holds over the feedback landscape.

But I like to have a look at stories if I think they’re maybe not… rated highly critically on our scale but are a little bit critical or time sensitive or maybe topical as well. Um, as you can imagine we get a lot of Covid-19 stories… coming in just now, so if there was something that was particularly critical around that, then I might… give them a sort of heads up about that, we wouldn't delay publishing at all, um but it's just to make them aware… It's just to support them through responding and making sure the person will equally get sort of quick and thorough response as well. (Interview)

The above explains the processes of working with subscribers but also touches upon the role that emotions and brokerage or mediation work can play, which is further discussed below.

Emotional labour

The nature of the healthcare experience stories shared on CO means that the work of moderators is not simply following a process. It involves emotional investment, starting with reading and working on stories, which can be powerful personal narratives. Staff discussed how working on these stories can bring up emotions, but our participants were aware of managing these emotions.

…it can happen, where you just have sort of one critical story after the next, and they don't have to be hugely critical… you find if you are moderating a lot of them, it's good to have a wee break from that and move on to something else… it does have an emotional impact on you. (Interview)

Although the critical stories could be emotionally draining, participants felt that the positive ones also helped to bolster spirits, and the staff enjoyed sharing these positive stories with their teams. We observed participants showing a great deal of care and happiness when they read or shared stories which talked about a positive experience of care. It was seen that these positive stories helped to counteract the emotional drain felt by the more critical ones.

… [Moderator B] told me later that they often also share the happy ones too – the “heartwarming ones” which helps to cheer them up. (Fieldnote, In-person Observation)

This type of resilience through positive emotions was also seen in other forms. Our research suggested that the moderators identify with the CO values and core beliefs. The participants expressed care in the success of the organisation, they wanted to see it do well and felt like it had an important mission. One participant explained that, at times, reflecting on these values could be helpful in dealing with conflict.

I think there are conflicts. But at the end of the day…we have… a mission and we have a set of values and it might sought a bit twee but to be quite honest with you if it does get a bit conflict ridden… then I go back to those and just say, like “Right, what are we here for? What are we trying to do? What's our job? What's not our job?” …that's what keeps me sane really in a way…  (Interview)

Our participants acknowledged the role that their own experiences could have in biasing story moderation. These previous experiences may have come from roles they have had before CO, or from their own experiences as a patient or carer. The moderator was able to decide if they felt they needed support on these. They could choose whether they asked other staff to check their work or they could choose not to moderate stories on certain topics at all. Whichever choice they made was reliant on the wider team offering support and our participants felt that the organisation aimed to foster a supportive atmosphere. This work atmosphere reflected the core values at the heart of CO and our participants reported feeling well supported as part of this organisation.

…everybody comes with their own background, baggage, prejudices as well and I do encourage people to get in touch with that… I do encourage team members to at the very least tell me what sort of things presses the buttons for them… so if you’ve had circumstances in your life that are either… that are upsetting, um they’re maybe in a story that you get, “I’m not saying that you can't handle that story or, but I just want you to be aware that it might be a struggle or that you mind have underlying prejudices that you’re not aware of, …so if a story has a visceral impact on you then ask for help, just ask for a second opinion. I’m not saying you can't do it, I’m not saying you can't moderate it, but just hand it up”, and people have done that, and I think that's really important, “It's not saying that you can't do your job, it's not saying that you’re weak, not saying any of those things, but just saying that you’re taking responsibility… and also demonstrating a commitment that our values and… our principles… (Interview)

As well as the emotions within the CO team, our participants expressed care for the authors and subscribers that they work with. During our research, participants demonstrated what they felt was a duty of care for authors and how concerns may be handled through the safeguarding policy. We observed that the safeguarding response to a story very much depended on the individual stories – some authors could be signposted to relevant resources whereas others were thought to need much more support. During our research, it was discussed that there is a need to carefully balance the response to ensure that trust and the complex relationships between CO, authors and subscriber organisations were not damaged, while ensuring that authors were cared for.

I am very aware of the fact that sometimes we have stories that come in that are needing action then and there, so they’re very time sensitive, um, or they could be vulnerable. They could be people who are needing help then and there, so I’m conscious to do that (Interview)

Aside from the care for authors, participants also demonstrated that they felt emotionally invested in the journey of stories. They wanted authors to get a response to their story from a subscriber and feel heard. They expressed a sense of satisfaction when authors received what they judged to be a ‘good’ response from a subscriber, as they felt authors would be disappointed from receiving a poor one or none at all. A good response would be considered a personalised response which comes from a member of staff, showing that the author has been listened to and responded to meaningfully.

CO staff also demonstrated care through the relationships they built with subscriber staff. They expressed that they wanted subscriber staff to do well, and at some points felt protective of them. These relationships meant that staff knew areas of concern for boards and were mindful of this when working with them. They did not want to share bad news with ‘their boards’ but enjoyed being able to share positive feedback with them. They understood how this positive feedback could make a difference to staff, especially during the challenging times such as the Covid-19 pandemic.

I would say I’ve got some Boards that I feel… like they’re my babies… I know that sounds stupid, but I get quite protective of them, so if a critical story comes out, I’m like, “Oh, I don't want to contact them and tell them that this critical story's come in”… But obviously we do, and you just have to deal with it. (Interview)

Some of our participants also mentioned that they had backgrounds from healthcare settings. This could also increase their awareness of subscriber challenges and make them empathetic to subscribers receiving criticism. In the wider sense, our participants felt that building these relationships with boards was an important step to ensure they could improve things for their authors. Although at times balancing the relationships with subscribers and story moderation may seem conflicting, our participants felt that it ultimately provided the opportunity to improve patient care in the longer term.

But ultimately… what we’re doing with the subscribers is quite transactional… we are teaching you how to use this model… you’re asking us for things – we’re giving them to you…. we have something we’re showing you how to use it…. you are asking us to make changes based on your contract… and, I think that… the motivation… behind the stories is very different. So, I think they’re two very different animals. Eh, they’re moving towards the same thing, they do… link together. But it's… how we’re…using humanity to change systems… how we’re using stories to make… make positive changes in the future …and what sits in the middle of that is the Care Opinion site and what sits at this side of it, is the subscribers and what sits at this side of it are the people telling the stories… they are two different sides of the same coin, um, but… they’re different. They are very different. (Interview)

When interviewed, participants identified emotions of NHS staff they also must deal with. Fear of feedback was identified as a common and long-standing issue across the NHS. It was suggested that this was linked to the NHS staff perceiving patient feedback as a loss of power. Participants felt that this fear of feedback was undoubtedly a barrier that they faced against the NHS fully embracing CO and learning from the feedback that they were given. Some of our participants expressed frustration at this and wanted to see this improved. This limitation was a source of emotional labour in itself, and despite CO staff offering support as a way of overcoming these fears, they rely on the NHS to take the first steps.

I think what's astonishing… over very many years, is how hard it seems to be… for health service staff to see the potential benefits of this kind of open online feedback… I think perhaps because of defensiveness, people are sort of overwhelmed by a sense, of perhaps fear, or defensiveness or that somehow power is being taken away from them… it seems to block people from seeing… all the positive impacts that can result from this kind of thing. (Interview)

As mentioned earlier, taking time to read and understand authors’ points of view and reasons for posting is an important part of moderation. However, spending time thinking about these stories, the experiences of the author and how to best reflect the author's wishes is part of the emotional labour of moderation. Our participants discussed the importance of making sure that patients were heard, and several reasons why individuals might choose to share their story. Positive stories may be a way of saying thank you to staff, which they suggested can be difficult to do through other methods of feedback. They felt that sharing negative experiences can be motivated by anger or wanting to make sure this does not happen to others. Our research also found that our participants felt it may be that authors use posting on CO as a way of processing, and coming to terms with, their healthcare experience. It could be that authors use sharing feedback to bring an end to their experience rather than seeking further interaction.

…And she went on and she shared her story, and she got a couple of really lovely responses from a couple of the services that had, that had been involved, and what she said to me was, it actually allowed her to breathe and round off the experience, because she felt like she hadn't… she hadn't rounded it off. It felt unfinished. (Interview)

During our observations and interviews, it was clear that some authors use CO as a way to seek help. These were individuals who may not know where to turn for health problems or to have their experience heard. CO aimed to support them as best as they can with their safeguarding policy. However, it was suggested by our participants that during the Covid-19 pandemic, there had been an increase in the number of people seeking guidance or support through CO, as services were reduced or closed.

…I ask [the moderator] if [they] gets them often… normally they “get a couple a week, but now it's more one every couple of weeks as the story volumes have gone down” with the pandemic. “But there are more people looking for general guidance on where to get support or advice as they couldn't get a GP appointment and they didn't tell them where else to go. (Fieldnote, Remote Observation)

From the daily moderation work and relationships with subscribers, to the emotions of authors writing stories, emotions clearly underlie the work of CO. There can also be positive emotions in this work which can help to counteract some of the emotional burdens. The team are aware of the impact that these emotions can have and offer support to each other when needed. They support subscribers to overcome the emotion of fear, building relationships which can often make them close and protective of the staff. They balance these complexities by understanding that this helps to achieve a better result for the authors in the longer term.

The brokering/mediation role of Care Opinion

We were struck by participants’ accounts of having to juggle relationships in a number of directions. We use the idea of brokerage or mediation to understand this process, not in its more limited sense of conflict resolution but in a broader sense of negotiating between different perspectives and interests.

CO holds a unique position within the feedback landscape in Scotland. They won the Scottish Government contract for online patient feedback for NHS Scotland. Our participants discussed how this was an important moment which was believed to validate the CO platform and reduce some of the barriers seen for subscribers, such as finance.

…the explicit support of the then Cabinet Secretary… for Patient Opinion for doing online feedback, for taking it seriously and trying to do it system-wide. I think that was an important point at that moment, having that level of political support… for doing this, and seeing the possibilities of that, both for… empowering citizens to be able to give their feedback in a safe and simple way like that. But also… for services, in terms of being able to hear directly from patients and act on that feedback and show how they’re acting on that feedback. (Interview)

Once CO had the support of the Scottish Government, they needed to carefully balance potentially differing viewpoints and their relationships with the Scottish Government, their subscribers and the authors that want to share their story, which may pull them in different directions. In balancing potentially competing viewpoints, CO are able to exercise a degree of power in shaping how stories are received. Our research showed that they have a role in amplifying voices, so that they can be heard. Our participants felt that at times these stories may struggle to find their way through other feedback systems, and CO offers them a voice and a chance to be heard. Indeed, being a boundary spanner and having a relationship with their subscribers may help this. Their knowledge of subscribers and their language can allow moderators to translate story elements so it can be more palatable to subscribers. Our observations of moderation showed the addition of NHS terminology as tags may be used but these words may not be commonly used by the authors who write these experiences.

[The moderator] moves onto story tags next. [The moderator] picks out words from the story and adds them in as tags – [The moderator] adds patient-centred and says “they, Care Opinion and services, like it if you tagged to patient-centred when it appears in stories.” [The moderator] says it's because it's an “NHS buzz word (Fieldnote, Remote Observation)

Although CO holds a great deal of influence in some regards, in other ways they are limited and rely on others for the impact they want to see. Ultimately, CO wants feedback and narratives to help the NHS improve care. However, their ‘change made’ or ‘change planned’ buttons seemed to be a weak mechanism for capturing all the changes which can occur from CO stories. Participants talked about how identifying a change can be challenging for subscribers, often with them thinking it needs to make a big difference to merit this description. Some also talked about how some subtler changes such as culture, staff morale and identifying priorities, could come from reading patient stories.

…there's also learning deeper than that. Because that orderly has been told that… she's done something good, maybe the other orderlies around about her went, “Oh, maybe, I should do that, as well.” Um so, it's this… institutional change in practice that we sometimes don't see. We sometimes don't see on the website. It's still happening. It's still changing that culture, but … we don't always see the minutia of that. And I actually think that those positive changes are really important, and we have to find some way… of capturing them, and we have to encourage Boards to think… about that…sometimes small level of… change that… needs to happen. (Interview)

Some also explained that even though some services do not use the website mechanisms for noting changes, they could see service improvements through changes in stories, or through discussion in meetings with staff.

It's very much when you’re having a chat with people and you’re talking about how things are going and all of that kind of stuff… it's just getting somebody to think differently, but that takes a chat. That takes… an interaction to have a chat to get them to do it and they’ll go, “Oh, do you know, I’ll go on and put that on. You’re right”, and they’ll go and change it. (Interview)

However, it was felt by some that the focus on using feedback for improvement was still primarily on the more negative experiences and positive feedback was not as often used.

…so people don't seem to make the changes as much, because they’re like, “Oh, it's not as critical as a four or a five, so we don't need to make they changes right now.” Um, or, “This might just be one person (Interview)

Although CO can use its position and influence to support the subscribers to make changes from feedback, it does appear that the decision of what changes are made, as well as the how and why, is down to the subscribers’ decision, rather than the service users. Therefore, the traditional power structure where healthcare organisations make the decisions, and decide what action is taken still remains.

…frustration that there is not the impact that I would like to see… the next stage of the journey I thought was, “Right, how can we demonstrate impact, the impact that these stories are having?” …way back in the dim and distant I did a little exercise of this board I was working with… this board in Scotland… I took… maybe 30 of their stories, bearing in mind I’m not clinical… I just read through them and just made little notes at the side, I think maybe ten/12 of them, I made little notes as to how I thought that a service could have considered a change as a result of those stories. So 30% of those 30 stories… I think with my non-clinical hat on were right for something different to happen… and we’re nowhere near 30% of the stories creating or change or impact. The impact just a different aspect, I think differently about impact, your positive stories can have a positive impact on, um, you know morale and… encourage people, encourage staff… makes you feel positive about the contribution you’re making so that's all good and that is an impact. But I guess originally, um and if… I’m being honest I would like to see more changes arise, concrete changes … (Interview)

Organisations becoming supportive of and embracing online feedback is a long and slow process. Our participants explained that CO wants organisations to be ready to welcome online feedback and committed to implementing it. However, how much time and support are needed for this to happen will be influenced by the position of staff who support it initially. Participants felt that even once boards are set up as subscribers, the CO lead in each board can have an impact on how the wider board engages. Participants reported that having staff in a position of power who champion CO can make a big difference to whether the wider organisation will embrace CO specifically and feedback culture more generally. However, our participants also reported ‘glimmers of hope’ for some of those organisations which were less engaged.

So… if you get an operational lead now… and when I say an operational lead that's the person that's responsible for Care Opinion… within that organisation. If you get an operational lead that is really open to feedback, is open to including their staff in it… in terms of reading stories, but also in terms of responding, is trusting… they are far more successful than if you get an operational lead who is a bit closed to the feedback and who is defensive… and they might be defensive for the right reasons. You know, getting critical feedback is hard… and quite often, I think they think that's all we’re going to get…. but we do have some phenomenal operational leads and they are great: they’re very controlled; they’re very planned; they’re very open in terms of people… being included as responders, and their Boards seem to do well… in terms of volumes… stats and all sorts of stuff… and the ones… they push back a bit, their Boards tend to not be as successful. We get… glimmers of hope in those Boards, though, because what happens is, somebody in a ward somewhere… cottons onto Care Opinion and they’re the ones that go, “Well, that's brilliant. How do we get some of that?”. And then they get onto the subscription and they kind of… in that little pocket… they generate a whole load of stories and we suddenly see that when we see the stories coming through, so we’re like, “Oh, somebody's switched on in the maternity section of (board name removed) or wherever it happens to be.” And you see these stories. You know, there's somebody somewhere doing something…. Because you can see that happening… so we just have to hope that in those Boards or Partnerships where maybe somebody is not quite as open about it, that those little glimmers kind of push the movement along from a different direction, you know. (Interview)

Regardless of how long the subscribers had been involved with CO, we observed that participants were mindful of how they interacted, with a considered approach with subscriber organisations. Our research found that the participants sought to use a form of soft power, to communicate effectively and persuade others without overstepping or causing resistance. An example below demonstrates the careful communication from a member of staff during our observations.

…[They] was able to command the room by demonstrating professional enthusiasm, and knowledge through using examples of other subscribers. [They] offered support to the subscriber staff throughout the meeting, which was reinforced through open and encouraging body languages. [Their] character was animated through large hand gestures, faces and the use of humour to relax the staff, but in times where they needed to be persuasive then the energy was more focussed. There were also instances where they went from a supportive service, into more of a business mode… offering to be flexible as they are quite “relaxed” as a way of overcoming obstacles that the subscribers suggested. [They were} also reflective of the fact there is tensions between the “Care Opinion vision” (where frontline staff receive the stories as feedback) versus “corporate responsibilities” – managing and dealing with stories. [They] also discusses that there will be different cultures in different subscribers so they have different ways of managing responses but there is also the opportunity to change some of these cultures. This was punctuated with different subscriber examples, demonstrating [their] excellent knowledge of the CO systems and how things work in different areas. [They] reminded the subscriber staff that (the subscription) “it belongs to you now” giving them ownership of the service, emphasising again their supporting role. (Fieldnote, In-person observation)

Our participants felt that it was important for subscribers to be ready for CO, and not forced into using it. If subscribers were ‘organisationally ready’ then it may help to build trust and improve engagement with feedback. Participants also felt that CO wanted to lead by example, ensuring that they too were open, transparent and provided feedback where it was needed. At times, there was a need to encourage subscribers to do better. One example may be where a subscriber's response was not suitable, but even in this instance they offer training for subscribers to help.

I think it is important that we set the same standards for ourselves that we expect from others. So… if we reject somebody's story then we need to email that person and tell them exactly why we’re rejecting it… If we think that our provider could do better in terms of a response then we need to use all our feedback skills, all our skills that we ask them to employ when they’re feeding back, to, to explain to them how their response could be a bit different… also it's important that we… encourage in this time you know, I’m very keen on encouraging health and care staff, but we’ve been doing that for a long time in Care Opinion, you know, and to have that kind of positive and encouraging outlook doesn't come naturally to everybody. (Interview)

Discussion

Our findings illuminate the different kinds of labour involved in moderating online feedback. The every-day moderation work carried with it the demands of editorial choices and responsibility for author safety and vulnerability. This in turn contributed to the emotional labour of reading distressing stories and trying to make sure patient voices were heard. At the same time, moderation involved a different set of emotional responses, towards the providers of healthcare receiving the stories. Underpinning both the process and emotional labour of moderation was the work of brokerage or mediation, negotiation and diplomacy between the different stakeholders. Although the emotional work of CO has been touched upon by Petrakaki and Hutton, ⁵ we bring additional insights into the depth of the emotional labour and their boundary spanning role. In addition, to our knowledge this is one of the first records of the impact of the pandemic on online patient feedback in the UK. We capture that at the start of the pandemic, there was an increase in the number of people seeking help through this platform, suggested as a response to services being closed or changed. Finally, this research illustrates how Government support has had an impact on online patient feedback and healthcare in Scotland, which we believe makes this research a novel contribution to the literature.

Our key theme of moderation as mediation builds on the work by Ziewitz ⁴ who carried out a moderator role within CO offices. Ziewitz ⁴ highlighted the careful editing process involved and that authors retain control over their own narratives, within the rules set by CO, which were then converted into quantitative measures by the hospital that received them. This speaks to the power of CO and subscribers over how narratives are created, used and who has ownership over them. The work of Petrakaki and Hutton ⁵ also highlighted how certain areas of the process could impact healthcare staff decision making. They identified criticality scores as a measure which can impact healthcare services decision-making and queried whether story tags on CO are intended for healthcare providers or patients. Our research found similar concerns, with the moderators taking time to understand the meaning behind the stories and retain the true voice as far as possible. However, they also tried to highlight stories to subscribers using NHS terminology or ensuring that some positive tags were included. These changes might be considered small, and they are not discussed with the authors of the story before posting. This translation may well be strengthening the use of corporate subscriber language as opposed to encouraging a shift towards author or patient language being the norm. Indeed, when it comes to the traditional power hierarchies, our findings suggest that subscribers were still in control of how patient narratives were used (or not) in their organisations, and that perhaps the true labour involved from the authors and moderators might not be understood. In Baines et al. ¹⁴ NHS staff talk about the feedback provided as being a ‘gift’ to learn from, this suggests that these experiences are given freely and without expectations on the recipients. Although some may choose to give feedback without expectations, our participants shared a variety of reasons why patient feedback may be shared and considering the stories as a gift may risk removing the true intention behind the narrative.

Our results capture a unique insight into moderation for a social enterprise with paid moderators, within a company who focusses on pre-publication moderation. This platform serves as a conduit for amplifying unheard voices to care services, such as NHS Scotland, and staff. At the same time, our findings also speak to the wider literature on moderation. Seering et al. ¹⁸ explore the wider range of metaphors used to describe moderators, including custodians, mediators, and networkers. Their metaphor of the gardener is particularly illuminating. The Gardener, they argue,

….nurtures and “tends” to the community, both in “planting seeds” of conversations and interactions and [….] “pull(ing) weeds”—problematic content and disruptive users—in order to make space for more positive interactions. (p.9)

This extends far beyond process work into more emotional and relational territory. In the case of CO, arguably this gardening role goes beyond the platform itself into the organisation's nurturing of relationships with subscribing healthcare providers, preparing the ground through training and encouragement, and with government funders. This four-way mediation between story authors, CO, healthcare providers and government, is a unique form of moderation-as-mediation.

The tension between the care for authors, subscribers and need to demonstrate impact from the Scottish Government may at times be hard to balance. Our participants discussed how the ultimate focus was on giving patients a voice, which guided them and supported them when dealing with the potentially conflicting viewpoints of subscribers and authors. However, to give authors the chance to be heard and responded to requires subscriber investment and support, potentially even more so when committing to making changes from feedback. So, although Scottish Government supports CO as a platform and has reduced the financial barrier for NHS Scotland, the choice to empower patient voices for change is still held within the NHS boards. Therefore, CO needs to have healthcare subscribers onside to work, and this may at times influence decisions made such as using corporate languages or the processes involved.

Our study highlights the emotional work involved within our participants’ roles and work, adding to the findings of Petrakaki and Hutton ⁵ who described the moderation of CO as a form of care and how that also can extend to the team, supporting each other with topics which are challenging. Our results found similar but added that alongside the ‘duty of care’ for authors there is an emotional investment into the story's journey and outcome. Our results also demonstrated that the team can have a much deeper connection with subscribers, there is clear care for the individuals that they work with. This additional burden is not unexpected, as roles which require forms of care ²⁰ and service improvement ²¹ are often linked to emotional labour. Our research captured the emotional labour from one perspective, our work did not capture the impact on those who write these personal narratives, nor the NHS staff who respond to them. Online moderation is also shown to have considerable emotional labour outside of the UK setting, even when not linked to healthcare. As mentioned earlier, commercial moderation and viewing unpleasant posts can be emotionally draining. ¹⁶ Other work by Dosono and Semaan ²² investigated the emotional labour involved for Asian American and Pacific Islanders moderators on Reddit, and Steiger et al. ²³ reported on the emotional and psychological impact of social media moderation and being exposed to harmful content as part of their role. Steiger et al. ²³ also capture elements important for moderator wellbeing, reflected in our own findings, such as taking breaks from moderating harmful content and having a strong support network. Although our research has focussed on Scotland, it adds to a growing body of international work on the emotional labour of online moderation.

In particular, we note a connection to Ruckenstein and Turunen's challenge to rehumanise moderation. ¹⁶ As the quotation in our title indicates, Care Opinion Scotland moderators see their role very firmly rooted in humanity and Mol's logic of care, ¹⁷ not just in their dealings with story authors and healthcare staff, but also in their wider goal of using the platform to improve the quality of patient care and patient experience across the UK NHS. As the volume of stories expands, the challenge will be to ensure this human quality can be sustained.

Strengths and limitations

This research is the first to give an account of moderation in the context of online patient feedback in NHS Scotland (provided through CO in Scotland). Although our number of participants seems small, this covered the whole of CO in Scotland which is a strength of our research. In the wider context, this research adds to the existing literature on online patient feedback moderation work and provides a novel contribution, especially in terms of the emotional and brokering/mediation work involved with the moderation of online patient feedback. The limitations of our research come mainly from the effects of the lockdowns during the Covid-19 pandemic. Our methods for this research had to be adapted quickly due to the pandemic lockdowns. That resulted in the lead author's ethnographic fieldwork being moved to remote methods only after the first week. Although the observation time was less than originally intended, we found that the observations were more focused sessions. Originally, the lead author hoped to spend time moderating stories as part of the CO team, however due to the situation this was limited to receiving training and shadowing other moderators. Certain elements were not able to be captured such as one-to-one staff discussions. Despite these limitations, our methods mirrored the experience of the CO team and their ways of communicating while working at home, providing a unique perspective during a time of transition.

Conclusion

We have reported on the work of moderators at CO Scotland, illustrating how their work involves different forms of labour. Considerable emotional labour was involved alongside meticulous process tasks, building on previous literature. Our participants expressed genuine care for the authors and subscribers in their roles as they worked with emotionally demanding patient feedback stories. As an organisation, CO exercises a degree of power over the editing and presentation of stories, but ultimately, their power to enact change from feedback is limited, as this lies solely with subscribing NHS organisations – reverting to the traditional power hierarchies seen between patient and healthcare organisation. Our research particularly draws attention to the four-way mediation role of CO, as a novel contribution to the literature. The unique position of CO requires the exercise of soft power and persuasion to balance the sometimes conflicting interests of Scottish Government, subscribing healthcare providers, and authors – which may add to the emotional labour discussed earlier. Our findings reflect and extend what is known about the role of moderators in a range of other online platforms, and reiterate the importance of providing suitable support for moderators to ensure they can carry out their work well. Our participants felt that there was a supportive atmosphere in the organisation which helped them when working with feedback stories. Further research is planned as part of this PhD into the emotional labour of authors who write these stories and the subscribers who receive them. This will further our understanding of the emotional impact in personal narratives of healthcare experiences and of how patient experiences are used from the NHS staff perspective in Scotland.