What lessons can be learnt from the Friends and Family Test?

Now that the dust has settled on the publication of the first set of Friends and Family Test results, I wonder what lessons are being learnt.

NHS England should, of course, be congratulated for showing an unequivocal commitment to hear the voices of patients. Too often, their views have been ignored or muffled. A commitment to making the views of patients available and transparent should be applauded. However, the debate and discussion over the last week has shown that, as is often the case, a good idea poorly implemented can lead to unintended consequences.

Two things strike me about the first set of results of the Friends and Family Test.

First, the decision not to trim the data and exclude information based on low response rates is misjudged. Data on public opinion is very difficult to interpret and we need large volumes to know it is reliable. To find that some wards have been labelled as 'failing' based on as few as three responses in a month or that an A&E department is held up as best practice based on 16 responses (probably less than the footfall in an hour!) is a concern.

Furthermore, not only are low response rates unreliable, there is also a risk of data being skewed by lobby groups. Just as we have seen the public being mobilised to get particular songs to number one (or not as the case may be), there is a risk that such fragile data could be too easily influenced by groups with particular agenda to promote. More needs to be done to increase response rates before data can be published.

The second issue is that there isn’t a consistent system for gathering feedback. Trusts have been able to choose the approach. In some cases, people are being asked to complete and return feedback cards before they leave the ward or unit. In others, Trusts are using text messaging and automated phone systems to get feedback post discharge. I am guessing that anyone who has said 'yes, lovely thanks' when asked by a waiter if they were satisfied with what was, in fact, a second rate experience will know which of these two methods is going to give the fairest reflection!

Of course, we all know that, when we see data that we don’t like, the easy response is to say the data are wrong. Of course, endless debates about measurement are always a good way of distracting people from the job of improvement and I would be the first to say 'get the data out there and the quality will improve'. However, for this to happen, the data need to be robust, reliable and used in the context of improvement, not judgement.

The Health Foundation has done much to promote effective measurement of quality in healthcare. Recent reports such as Lining Up and The Measurement and Monitoring of Safety provide some important lessons: data are more open to subjective interpretation than we often recognise; relying on any single measure is naive; qualitative data is currently under utilised.

There is a risk that, unless used sensibly, the Friends and Family Test could fall into all these traps and may not provide a sufficiently rounded assessment of experience. Our work on person-centred care highlights the importance of seeing the experience of care as including the extent to which people are involved in decisions about their care and supported to self manage. Unless the short comings of the current approach are addressed, there is a real risk that NHS England simply creates an industry in which providers seek to get the 'right' answers rather than genuinely seeking to learn from the experience of their patients.

Jo is Director of Strategy at the Health Foundation, www.twitter.com/JoBibbyTHF