Defining the metrics of success for ICSs

As integrated care systems (ICSs) continue to dominate discourse surrounding health and care, Gabriel Blaazer presents insight from last year’s ICS Roadshow where system leaders and key stakeholders sought to define what ‘good’ looks like for ICSs.

– By Gabriel Blaazer –

Key insights:

  • Cultural change should be considered a key metric by which to judge the success of integrated care.
  • Service integration and balanced stakeholder representation should not be considered merely ‘nice to haves’. The extent to which they occur will be vital to the success of ICSs and their ability to address their core priorities.
  • ICSs should be given more localised autonomy to allow national targets to reflect local priorities.
  • The establishment of unified, interoperable data infrastructure in each ICS will be critical to developing a holistic whole-system view. ICSs are at varying stages of digital maturity and will find it difficult to even measure their progress without interoperable data infrastructure in place.
  • Delivery of ICS goals will be impossible without a sustainable, sufficiently-paid workforce.
  • The government’s delayed workforce plan should explore how to achieve pay parity for the independent care sector and promote cooperation between NHS and independent care bodies.
  • ICSs should explore creative avenues to increase recruitment pools for all healthcare-related roles. This could include working with national educational bodies to produce more generalised curricula to attract more young people to careers in health and care.

A new focus for healthcare

The core purpose of integrated care systems (ICSs) is bringing partner organisations together to:

  • improve outcomes in population health and healthcare;
  • tackle inequalities in outcomes, experience and access;
  • enhance productivity and value for money;
  • help the NHS support broader social and economic development.

While ICSs have been designed to enable a new model of healthcare delivery, less attention has been given publicly to the measurements and metrics according to which they should be judged. Making meaningful action on these four purposes will require new frameworks to judge success, and new metrics to assess their progress.

It should be noted that ICSs have no shortage of centrally derived targets and metrics upon which they are already being judged. In Bristol, the Interim Head of Business Intelligence for NHS South, Central and West (SCW), David Boothroyd, told the audience that ICSs have already been “given lots of benchmarking materials – there are national priorities, the NHS Long Term Plan, system oversight frameworks, local priorities, Core20PLUS5. These are all going to come and hit you from NHS England and above, no matter what. Then, you’ve got to really look at what it is you want to do as an ICS and that’s how you decide what good looks like.”

These comments presaged a running theme throughout the Roadshow, which was the tension inherent in managing operational pressures and ‘keeping the lights on’, while also addressing the core purposes of ICSs around improving population health, reducing health inequalities and supporting social and economic development, all while enhancing productivity and operational efficiency – doing more with less.

James Banham, a Director at Deloitte, told the audience in Leeds that ICSs will need to strike a balance between “the operational pressures, politics and public opinion around elective recovery so what does success look like in that context? If we end up just measuring ICSs by the same old metrics that have been around for many years, is that going to disengage people? How we stop ourselves from doing that is absolutely critical.”

ICSs should not consider the tension between short-term pressures and long-term goals as an ‘either/or’. Does success for an ICS really look like having the ability to create the capacity, to create the right culture, to do be able to do both of those things at once? To be able to handle the operational pressures, as well as looking at long-term transformation?

Implementing cultural change

If ICSs are to take meaningful action on the factors which influence health inequalities and population health, it will require the input from a wide spectrum of stakeholders. This must include, naturally, the NHS and local authorities, but also social care, the voluntary, charity and social enterprise (VCSE) sector, relevant private sector stakeholders, public health experts, service users and input from the education, unemployment and justice systems too. Such broad representation recognises the role that each of these have on the health of citizens.

However, numerous contributors noted that this would be a substantial departure from the NHS-dominated models that previously existed. Ongoing cultural differences between local government and NHS leadership continue to hold back integration.

The steps that West Yorkshire ICB has taken to include a broad spectrum of perspectives are allowing it to deliver service integration according to Elizabeth Bradbury, Director of the Advancing Quality Alliance (AQUA). She related to the Leeds audience how the involvement of more than a thousand local quality improvement partners is helping to “change the dialogue and the decisions that are being made. These are local people who are passionate about local care and want to work with the professionals on everything from strategy, policy development and deployment, on intensive improvement projects and board appointments. Hopefully, we’re edging towards more integrated care because of their input.”

“By working together, the relationship between mental health and housing has been understood.” Frances O’Callaghan, Chief Executive, North Central London ICS

So, while broad representation can sound like a ‘nice-to-have’, it is also central for ICSs to fulfil their core purposes, particularly around health inequality. ICSs should therefore be judged according to the structures they put in place to facilitate and enable this cultural shift. While these structures can include the statutory inclusion of provider collaboratives and other bodies on integrated care partnerships (ICPs), there are other elements that can help to instil the necessary cultural shift required for ICSs to succeed.

This cultural shift must also be evident at the very top level of ICSs and the ICP – which will ideally involve representatives from a variety of public services – is an ideal body through which to demonstrate the value of collaborative approaches to improving public health.

The Chief Executive of North Central London ICS, Frances O’Callaghan, shared how different services talking to each other has led to an improved understanding of the relationship between inadequate housing and poor mental health – something which may appear obvious but is difficult to quantify in isolation. She said, “we know from work that was done in Islington that if you have an acute mental health admission and you go back to a rubbish housing situation, your chances of returning to the acute bed are very, very high.

“By working together, the relationship between mental health and housing has been understood and so if we don’t address housing in North Central London with our local authority colleagues, we know we will see the demand for acute beds go up, and readmission in itself.”

System approaches to finance

ICSs came to the end of their first financial year as statutory organisations in significant deficit, but progress has been made in establishing system-level approaches to financial management. To advance on this localised progress, ICSs need greater autonomy to align with public services beyond health and care.

In many systems, the introduction of ICSs has brought commissioners and providers closer together. But as quickly as old tensions have been eased, new ones have emerged – specifically the cultural differences between the NHS and local government. This has in turn hindered efforts from ICSs to create truly system-level approaches to financial decision making, and local authorities have understandably been hesitant to pool budgets with ICSs which are still perceived to be ‘NHS first’ organisations.

Communication and a desire to work together for the common good can only take ICSs so far, and more formalised local autonomy for ICSs will enable each system to reflect their unique local authority landscape. Therefore, it is imperative that the government accepts all recommendations of the Hewitt Review to empower ICSs to succeed in their localities.

“The leadership that we’ve seen in health and care over a long time… have been naturally competitive people.” Andy Kinner, former Chief Information Officer, NHS England

Meanwhile, NHS and local government organisations within ICSs must also be encouraged to relinquish a degree of control over budgets and work towards a system -approach to finances. This shift would constitute evidence of collaborative leadership driving a consistent mission across a geography, and evidence of genuine trust between colleagues working together to achieve their goals.

Former NHS Chief Information Officer, Andy Kinnear, said: “since the early nineties, the NHS has been driven by a purchaser/provider split mentality in the health service that has created an overly competitive environment.” He argued that much of this culture stems from the “the leadership that we’ve seen in health and care over a long time, who have been naturally competitive people.”

Until now, this way of working has proven prohibitive to the creation of joined up, truly integrated services. David Boothroyd, the Interim Head of Business Intelligence at NHS South, Central and West, told the Bristol Roadshow about his prior experience working on alliance contracts in Croydon; “It brought together local authorities, trusts, community providers, AGE UK and a mental health trust. We developed a contract to work together, doing exactly what ICSs are trying to do now,” he said.

“One of the key barriers to working together was around money,” Boothroyd said. “One of the reasons it took so long to get the contact signed was the ability to move money with the resources, because people protected their own organisation. They wouldn’t want the money to follow the resource.”

In her recently published independent review into integrated care systems, former Health Secretary, Patricia Hewitt advocates for government and ICSs to take a system-based approach to funding. The Review argues that DHSC, the Department for Levelling Up, Housing and Communities and NHS England should “align budget and grant allocations for local government and the NHS, so that systems can more cohesively plan their local priorities over a longer period of time.” Hewitt also recommends that NHS England give ICSs more flexibility to make allocations for services within system boundaries, and that this should be reflected in NHS payment schemes.

The ability to rapidly allocate budgets or small funding pots, in concert with a real-time, system-wide view of supply and demand, will allow ICSs to allocate resource far more effectively, to the benefit of patients, the workforce and the wider system.

The role of CQC

If ICSs are to be judged according to new metrics, it follows that the role of its regulator, the CQC, should evolve with it. As mentioned in PPP’s Ensuring ICSs represent a partnership of equals, a CQC inspector in the Birmingham audience remarked that the regulator “will certainly want to see evidence” of interaction between social care providers and the ICS. While something of a ‘soft’ metric, it will be important for the CQC to assess the extent of cultural change within ICSs and the degree to which a collaborative ethos drives action.

With Hewitt recommending that the CQC takes an enhanced role on system evaluation, this should include assessments of cultural change and system integration within individual ICSs. While the Review says this should not take the form of a single rating for an entire ICS, Roadshow participants agreed that assessments on cultural change and system integration should certainly be among the CQC’s considerations.

Data and interoperability: “We can’t improve what we can’t measure.”

Across all iterations of this panel, there loomed an elephant in the room; it may be possible to say which metrics ICSs should measure, but they first need the infrastructure in place to do so. This section does not seek to repeat the insights of PPP’s Practical steps to improve data sharing in integrated care systems, but it bears repeating that integrated care itself cannot exist without system-wide, interoperable data platforms in place.

David Boothroyd explained that “if you can put an operational management system in place across your entire ICS then you can measure everything you do at any point in time. That data then leads to strategic data, which you can then analyse and produce intelligence. But you have to put an infrastructure in place first.”

Rather than performance management, establishing an integrated data infrastructure across an ICS should be seen as a means of achieving a cycle of continual improvement in which outcomes can be measured and analysed, and processes optimised through evidence-based insights.

The healthcare applications of digital technology are too numerous to mention, but the aggregation of multi-source data into a single source of truth will allow ICSs to manage their key metrics far more effectively.

A prime example of this would be hospital discharge; many ‘failed discharges’ occur when patients are discharged into community settings in which they do not have sufficient independence or support and end up being readmitted to hospital within a short period of time. Often, this is due primary care data not being available to secondary care clinicians, who must decide whether a patient is fit for discharge on the basis of very limited information. This is most likely to occur where there are barriers to comprehension, e.g., language barriers, but patients should rightly expect that an integrated care system be aware of their basic needs and circumstances.

Delivering workforce sustainability

The recent and widespread industrial action from those in the health and care workforce are indicative of the instability facing the sectors. Stagnating pay and worsening conditions are driving many from the workforce, and Patricia Hewitt’s Review joins the chorus calling on the government to deliver a long-term, properly funded workforce plan.

Quite simply, no reform of healthcare is possible without a dedicated and cared for workforce. In the absence of a long-term workforce plan, and if the NHS Pay Review Body is unable to recommend higher wages, improving recruitment, retention and working conditions must be a focus of ICSs.

“We won’t be able to provide the health care that we will need for ourselves as patients and families if we don’t invest in an amazing workforce and we need to do more to support them.” Helen Hughes, Chief Executive, Patient Safety Learning

Helen Hughes, Chief Executive of Patient Safety Learning, explained how unsustainable the current workforce environment is; “70 per cent of our spend is on people but we’ve got a massively ‘leaky bucket’ at the moment. People are unhappy, and pay is part of it, but not all of it. It is incredibly exhausting working in health and care. And going to abroad to recruit is not a sustainable answer.”

With the pool of available workers shrinking, ICSs must look differently at how they attract and recruit potential workers – this examination should go beyond doctors and nurses. Frances O’Callaghan observed that “fundamentally, the undergraduate curriculum for medical education needs to change. People with five stars at A-level are brilliant and we definitely need them, but we need to be getting more people outside of the traditional professions of medicine and nursing.”

This is not to say that only those with five A-stars can be nurses or doctors, but rather, reflects the fact that new methods of engagement with potential workers are needed. For example, ICSs could work with local educational institutions to create new, more generalised medical curricula, which are reflective of local need, are likely to attract young people but do not limit them to the traditional professions.

Pay disparities between staff in health and social care are emblematic of the fact that they are not viewed as part of the same system and are not given equal standing. A truly integrated system would recognise that the wider system care cannot function without social care, and while there may be realisation of this fact, it is yet to be reflected through policy.

According to analysis from the King’s Fund, pay growth for care workers, while rising slightly, has stagnated over the past decade, and many care workers now would be better paid in entry-level posts in retail.ii Pay progression for more experienced care workers is also slower than for new entrants to the social care workforce, with those carrying several years’ experience earning just 7p per hour more than those with less than one year’s experience – down from an average of 29p per hour in 2012. They attribute these factors, and more, to difficulties in recruiting and retaining staff for the care sector.

Elizabeth Bradbury reiterated the damaging effect of this dynamic; “we recently had in the North West hundreds of jobs being advertised in care homes and hundreds being advertised in the NHS for care workers. The NHS pays more, so of course people chose the higher paying jobs. This destabilised the who local economy, because the local care sector could not recruit local people to work.”

This disparity is not unique to the North West – indeed, it is common across all ICSs and stakeholders across the health and care system agree that it needs urgent action. Community Integrated Care, one of the largest social care charities in Britain, released their Unfair to Care report in December 2022, which contains recommendations for government to ease the crisis in the social care workforce. The report argues that “social care must arrive at complete parity with its NHS counterparts”, and calls for an expert-led, cross-party workforce review to deliver a funded social care workforce strategy to explore how to achieve this.

The patient’s perspective

As ICSs mature and progress, they are expected to transition towards person-centred care. The idea of creating, from the perspective of patients, a single healthcare system, means that citizens should be able to access the appropriate services no matter which ‘front door’ of the NHS they enter. This will make care more accessible, or in Patricia Hewitt’s words, “bring the NHS to its patients”.

In Manchester, Dr Jeff Farrar, Chair of the Bristol, North Somerset and South Gloucestershire (BNSSG) ICS, stated, “I’m an absolute massive advocate for one public service, because our focus has to be on what our citizens need and what our patients need.”

Chris Davies, Associate Director of Business Intelligence at BNSSG ICS, reflected that when joining the NHS “relatively late in my career, I was very surprised to find that the NHS wasn’t just one organisation. One of the hopeful results of ICSs is that pathways start to feel more joined up and they manage to fit seamlessly with what is happening in a patient’s life.”

“I think success has to actually reflect listening, as well as collaborating with communities.” Andy Kinnear, former Chief Information Officer, NHS England

For service provision in an ICS to reflect the needs of their populations, however, patients must be engaged and listened to, and ICSs must have mechanisms to do so. How each ICS does this will be up to them but giving a seat to patient representative organisations on ICPs should be a first consideration.

In Bristol, Transformation Director for Health Improvements and Inequalities for NHS SCW, Andrew Fenton, argued that “I go beyond it being for the ICSs or systems to determine themselves what the measures for success are. I think that’s something to be done with, and through, local communities and the wider partnership.” Deloitte Partner, Catherine Skilton, confirmed that ICSs should seek to “learn from other industries, like the consumer industry, which is really good at measuring its users’ experience. The kind of engagement where we have to make choices about where we spend our resources has to happen in collaboration with the public, not just with finance professionals.”

The King’s Fund, among others, have long advocated the idea that judging the success of integrated care will require actively listening to, and welcoming participation from, communities and service users who are, after all, those who fund the health service and those whom the healthcare system serves. This was reflected upon by Andy Kinnear, who stated, “I think success has to actually reflect listening, as well as collaborating with communities.”

A rising tide lifts all boats

In seeking to define the metrics for success for ICSs, each iteration of this session came heavily caveated. Leaders from NHS organisations, care providers, VCSEs and the private sector were near unanimous in the belief that none of the metrics discussed could be achieved without the basic building blocks in place: a universal ethos of collaboration above competition, a strong digital foundation, a well-paid and sustained workforce, and patient involvement (both in decision making and assessing ICSs themselves). These are as essential to delivering integrated care as any conceivable metric.

The absence of a ‘’one size fits all’ metric for integration should not be seen as a failure, but rather acknowledgement of the fact that assessing the extent to which a system is collaborating and is integrated are inherently complex tasks. Ultimately, the principles of subsidiaries and local autonomy which underpin integrated care necessitate that these systems should be measured and assessed in locality.

In Bristol, Andy Kinnear summarised what many contributors had expressed across all five roadshows; “before we get into what success actually looks like, I think we’ve got to have a reality check as to where health and social care is right now. We are – hopefully – coming out of the worst decade this space has ever seen. Many people feel like just getting up and getting through the day feels like success right now. Anything beyond that should be considered a real achievement.”

David Boothroyd decried what he referred to as the “whack-a-mole approach”, where problems are viewed in isolation rather than as symptomatic of failings in the wider system. The fact remains that whether or not the different parts of the healthcare ecosystem are integrated, they are intrinsically connected, and outcomes in one area will invariably impact the outcomes in another, taking the interplay between hospital discharge and inadequate social care provision as a prime example.

While remaining mindful of their wider obligations, ICSs should also focus on creating the conditions for success, as outlined in this report.

Scroll to Top