What gets measured gets done. Or does it?

'What gets measured gets done' – a well-established mantra, and with good reason. But I’m not entirely comfortable with the implication that results always neatly follow from applying focus and pressure to priority problems. 

Yes, to an extent you get positive results from identifying clear, measurable issues and tracking them. Thinking of how provider dashboards have evolved, it's heartening to think of the additional attention given at senior levels to infections, falls and other quality issues that might not have had the same profile as financial indicators in the pre-Mid Staffs era.

Sometimes, however, data closes down or narrows debate, especially aggregate information on single issues. It can also provide a false assurance that leads to things not getting done: as long as indicators are above a certain target level then attention (and sometimes resources) can be redeployed elsewhere.

This becomes problematic if the markers you are using (such as MRSA bacteraemia rates) are taken as completely capturing something that is much broader (like safety), or if your target level doesn't reflect what is really optimal for key stakeholders or the system as a whole. For example, providers and those who regulate them might come to see green RAG ratings in the safety section of a dashboard as a reflection that their service, as a whole, is safe.

A couple of weeks ago I had the pleasure of being on the judging panel for the NHS Wales Awards safety category. The diversity of the three finalists provides an inspiring range of different approaches to both safety and measurement. It brought home to me that beneath the simple mantra 'What gets measured gets done' lies a slightly more complex reality. Measurement is not just descriptive but productive. The choices leaders make about what to measure and how to go about it have a big impact on what they do and the success they achieve. These choices will depend on what they are doing and who they want to engage. 

The first project we visited was the Welsh Ambulance Service, where we met with a great team who had engaged over many years with the deaf and hard of hearing community to improve access to health care. Their specific safety intervention was a medical information card that addressed communication challenges in emergency situations. Importantly, the team used qualitative patient stories not just to co-design the solution or make the case, but also to open up discussion and relationships that had broader benefits.

The second team, at Cwm Taf Health Board, presented us with a room of run charts and other analysis to galvanise improvement on many fronts as part of a large-scale patient flow programme. For example, they demonstrated a correlation between 4 hour target breaches and mortality rates. Describing A&E as the pressure valve that indicated much wider flow challenges across the health board and beyond, made the case for improvement in a way that connected well with clinicians.

Stories and hard data here were being used as part of everyday work; embedding improvement analysis and a whole system view into how people understand and address the specific operational challenges they faced.

The final project was at Cardiff and Vale University Health Board. We heard from a high-powered, innovative team who have introduced cardiopulmonary exercise testing in advance of major surgery and are able to demonstrate that they can now much more accurately risk stratify patients in order to:

• have much more accurate conversations with patients and surgeons about treatment options
• optimise patients prior to surgery
• make better informed judgements about peri-operative resource requirements such as the need for a critical care bed.

They focused on linking data systems to get the sort of robust mortality and morbidity analysis needed to gradually convince surgeons of the benefits of the new approach. In the meantime, they also did the capacity and associated financial analysis needed to convince senior managers that investment in the kit and clinic space significantly reducted length of stay. 

So, some great examples of teams using data in different ways as part of a proactive approach to improving safety, reminding us that the power of data goes well beyond the measures reported to boards and external bodies. I’ll be running a session with Jane Carthey at this year's NHS Confed conference on proactive approaches to measuring safety which will allow us to explore this territory further. Jane will be drawing on some of the seminal research she did with Charles Vincent and Susan Burnett on a broader approach to measuring and monitoring safety. Do come along if you’re attending the conference, or for some examples of how to apply proactive approaches, download our new practical guide.

And some questions to ask yourselves and your organisation: does your safety dashboard illuminate the breadth of safety issues you are concerned about? Is data being used in ways that engage those it needs to, helping steer improvements that make a real and sustained difference to the quality and safety of care? Do the targets set and data used provide appropriate stretch and inspiration to improve? I'd love to hear about examples from your work of where this is the case – do let me know in the comments below.

There's one key piece of data I won't be sharing now. If you want to know which of the fantastic finalists won the safety award in Wales, you'll have to wait until the awards ceremony in July…

Penny is an Assistant Diretor at the Health Foundation, www.twitter.com/PennyPereira1