The more stories, the more positive?

It’s a common misconception that engaging with a public feedback platform encourages more negative feedback. Publishing these healthcare stories amounts to what might colloquially be called ‘airing an organisation’s dirty laundry’. Yet in our experience, and as we often tell services, most of the stories shared on Care Opinion are purely positive. Although, despite our reassurances, there is something quite different about seeing this illustrated before you.

Recently, I had the pleasure of discussing this phenomenon with Tim Norwood, Data and Measurement Advisor for Healthcare Improvement Scotland, who confirmed through data analysis what we already knew anecdotally to be true: that most feedback shared Care Opinion is positive.

Why Care Opinion

Tim’s role is to help teams within his organisation to understand what data says about care delivered in Scotland, and inform their work to improve care. Healthcare Improvement Scotland wanted to hear about the experiences of people using services, and at the time there weren’t really any sources of data that allowed them to do this other than national surveys, which had their limitations. Of these surveys, Tim said:

“They gave us some understanding of experience but only for the questions asked and with no context. So we learned about Care Opinion and the fact that it was open for anyone to see, and that they were very willing for us to use the stories that were published there and encouraged it.”

Tim, alongside Shaun Maher, who works in the Scottish Government, shared an interest in finding ways to summarise the large volume of feedback on Care Opinion, allowing local healthcare teams to view this feedback at a higher level.

The reason for using stories on the platform was partly their availability - it was a large dataset in one place, with the stories public and visible - but Tim also explained that the stories represented a range of experiences that weren’t limited to the perspective of the patient:

“The experience survey was directly to the patient. So obviously with cancer care, you can miss a lot of people because of the nature of the illness. But in Care Opinion, you can have carers or family members also post about the experience as they saw it through their eyes.”  

The dashboard

To demonstrate this high-level and more encompassing view of the data, Tim has an example version of the dashboard that he and Shaun created, which can be accessed via this link: https://timnorwood.github.io/careopinionstoryboard/output/storyboard_example.html

The dashboard draws on the tags used in stories, which are sometimes added by the authors themselves and also by Care Opinion moderators. Tim explained to me that the purpose of the dashboard was to provide a way into exploring the stories:

“The focus here is really about trying to offer a way for people to understand a big body of stories. There was just too many. If you're a manager or even a ward nurse on a busy ward and you're getting lots of feedback, it's hard to keep up to date with that. So how do you see the whole picture?”

And seeing this whole picture is exactly what the dashboard allows. You can see an aggregate of all stories about a health service and then move through individual areas -  such as emergency departments, maternity and surgical services - to see the patterns that emerge and the overall positivity ratings.

The more stories, the more positive? 

This positivity rating is perhaps one of the most intriguing measures in the dashboard, and one that demonstrates something we already know: when people are given a public, transparent and relational space for narrative feedback, most of the stories they share are positive.

To demonstrate this, Tim calculated the overall positivity of stories by comparing the number of positive tags used across stories with the number of negative tags during a particular month. This makes it possible to see whether some areas receive more positive feedback than others, as well as how the positivity of feedback changes over time.

When viewing the graph (see image above), you can see month by month how the positivity of stories has changed over time. What is particularly interesting is the longer-term pattern: the positivity rating steadily rises before beginning to stabilise.

So what changed? Did people’s experiences of care suddenly become dramatically more positive or was something else happening?

Tim explained that, in fact, this change corresponded with more stories being submitted to Care Opinion:

"As more and more people submit stories to Care Opinion, we're reaching a truer reflection of the wider experience. I think earlier on, when Care Opinion wasn't as well known in Scotland, people who are maybe more frustrated with things saw it was a place to vent a little bit. So we saw in the early stages a much lower positivity rating. But then it generally increases over time and starts to stabilise. I think we're at that point now in Scotland. Over the last couple of years the data for most areas has almost stabilised to around about 80%, which is brilliant. But I think that's genuinely really what people are experiencing.”

The fewer stories you have, the more variation you tend to seein positivity ratings. In contrast, as story numbers increase, Tim has observed across most services that the positivity rating tends to rise and then stabilise.

One could argue that this is because stories on Care Opinion don’t capture the full breadth of experiences that people might feel more comfortable expressing through a survey. Maybe the public nature of the platform discourages some criticism, with patients concerned about potential repercussions from providing honest feedback.

But Tim pointed to another piece of work that challenges this possibility. Last year, they undertook a review within NHS Greater Glasgow and Clyde in which they wanted to capture a range of perspectives. They used both survey data and Care Opinion stories, and found that the issues raised through Care Opinion closely matched those identified through the surveys:

“The themes that we found in this analysis - such as waiting, communication, attitude, those things being criticisms of the emergency service - across Scotland, those matched what they found in this survey. So again we saw that correlation with a different approach to measuring experience.”

It certainly dawned on me that maybe this is a side of Care Opinion that is too often misunderstood. We hear the fears of health services about engaging in transparent feedback and we can certainly understand them. But actually, when faced with a relational way of providing feedback dialogically through and with the health service, the results show that, overall, the more stories you receive, the higher the positivity rating.

And this isn’t simply because of the public nature of Care Opinion. In fact, as Tim mentioned, the themes that arise through traditional mechanisms such as surveys are often those that are also prominently discussed in stories. The two mechanisms align.

Through utilising dashboards such as the one Tim and Shaun have created, this reality becomes much harder to deny. The majority of patients value their healthcare system. The majority of patients want to talk about their positive stories - their life-saving treatment or the impeccable bedside manner. These small moments that are not forgotten by the individual, but often go unspoken outside the circle of those involved.

It may be that by reminding ourselves of this truth - that stories on Care Opinion are mainly positive - we can begin to turn the tide on these misunderstandings. We can begin to listen more to these wondrous stories of the ward clerk hero or the irreplaceable midwife. We can encourage these stories to fill our healthcare vocabulary, giving greater voice to the praise that is so often outweighed by the negative. Perhaps this is the change we can make, this is the healthcare system we can create. If not for the sake of changing perceptions, then for the sake of the many positive stories that might otherwise go unheard.