What outcome measure for an Accountable Care System – how will we know its working

Third go at this one. Th other two are referenced

This one attempts to distill the essence

We’re all on this path.

I keep getting ask d what outcome measures we should use……..

Hers my take 
This isn’t a paper about clinical performance measures. This short paper sets out some recommendations for how might we know a system is achieving its intended effect.

1)​Process measures Indicators

  • %of budget envelope that is subject to outcome based payment
  • % of physician salary at risk for qual outcomes
  • % of contracts that are subject to up and down side risk sharing
  • % of budget that is capitation
  • % of budget that is spent on hospital vs out of hospital.
  • £ / capita on district nursing.
  • What is the trajectory of GP v hospital v mental health expenditure.


2)​Efficiency measures

  • OP procedure rate (efficiency)
  • Day case rate (efficiency) – aim on cutting down on bed use. We may already fare well in this respect.
  • % of visits (OP and GP) conducted by phone as opposed to face to face
  • % of referrals that are e consultations.

3)​Cost measures –

cost / patient month – needs more sophisticated data than we currently have, and needs to be done in segmented pops 

4)​obviously – need measures of patient experience and outcomes in this mix

Good, validated Patient-provided measures of health, well-being, and functioning are available and sometimes routinely collected.

 measures do the degree to which care for complex pts addresses their goals. 

5)​Clinical indicators

  • • Obviously it would be expected that any ACO / ACS set out a suite of clinical indicators.
  • • Need a single set of xxx indicators. Size of xxx should be determined. Should be based on NHSOF, ASCOF and PHOF Mostly these are very fit for the purpose expected of them.
  • • CMS in USA provides a comprehensive set of real and proxy outcome and quality metrics for accountable care organisations. These can readily be adopted. There are no surprises in this. The foucs of such sets of indicators is VERY much on the “system” and NOT “doctor”. In an ACO context the “system” is the organisation, for an Accountable Care System we still need to focus on system level indicators. (see my blog referenced)
  • • Its worth saying that in the realm of clinical measures there’s a creep towards outcomes relevant to older people and service use (the measurable vs the important) and lack of attention to population risk indicators.
  • • Should expect a balanced suite of measures focused on quality, experience, outcome (population health) and cost. Not just service utilisation.
  • • KEY clinical indicators suggested as: The few summary indicators that are often used are patient satisfaction (quality), OP procedure rate (efficiency), 3d r readmission, non elective rate. Id also suggest something about long stay (DTOC and resultant decompensation)




Others include

  • • Unscheduled in-patient readmissions within 30 days of discharge for selected case mix groups – stroke, COPD, heart failure, cardiac, pneumonia, diabetes, gastrointestinal, asthma, mental health and addictions
  • • Repeat unscheduled ED use within 30 days for any reason (may focus on low acuity – minor, no imaging, no follow up) – actually, emerging evidence it’s a poor indicator unless the system can focus on low acuity.
  • • Percentage of hospital patients who know important discharge aspects, for example, danger signals to watch for after going home, medication-related information, when to resume usual activities, whom to call if they need help
  • • Percentage of patients with complex high care needs identified who are targeted/receiving appropriate care (e.g., intensive case management [in development

These are hospital centric… some of the issue is about 1) what’s measured vs what’s important, data availability and 2) complex system measures of quality, experience and outcome that apply to all organisations in a system  


6)​A consistent set of quality measures covering:

Effectiveness – the extent to which system delivers in terms of patient outcomes (e.g. people feeling treated with dignity and respect), and strategic outcomes (e.g. reduced readmissions, less people going into long-term restrictive care etc)

Economy – the extent to which the system is achieving value for money and delivering the required services on budget, on time and within other resource constraints

Efficiency – whether the overall system is working optimally – e.g. length of stay, number of negative-value days (red days) in each setting, unused capacity in services etc…

7)practice level and cultural indicators of shift to personalisation and population risk management

  • • Lower-risk patients were offered disease management programmes that involved proactive management of care using guidelines with prompts to clinicians and
  • • Patients, decision-support systems for patients and clinicians, patient education and selfcare, electronic disease registries that identify affected patients and record details of
  • • Motivational Interviewing is a “norm” in terms of style of consultation
  • • A practice looking after a population has a clear plan re the risk profile of its population of patients with specific conditions. Can demonstrate it is making active use of risk profiling
  • • Visible commitment to seeing service users as part of the healthcare team
  • • Visible commitment to neighbourhood level partnerships with community groups
  • • visible commitment to including specialist nurses as part of community teams

It does boil down to knowing:-

  • • What outcomes and other things you want to achieve. Being really this. You’ve got to think hard, there are no shortcuts.
  • • what data is collected, readily (and sometimes less readily, but needs a bit of work) available
  • • What raw data can be turned into indicators. Knowing – in detail, real detail, how the datasets are derived from raw data, and what can / can’t be concluded.
  • • and the ability to backtrack all of the above, and backtrack clinical indiator sets.


Outcome measures for an ACOhttps://gregfellpublichealth.wordpress.com/2016/11/23/outcome-measures-for-aco/

My first effort http://www.yhahsn.org.uk/wp-content/uploads/2016/04/5.-GF-ACO-Performance-Metrics.pdf

  • This goes though how this is done in the states, especially the CMS data set.

Kings Fund –


IHI papers are also useful


Like all folk that like to be evidence based and robust I like to get critique from others 

I asked a colleague who’s views I respect immensely

His take one the above was as follows
 Critique 1

I’m afraid I’m not a big fan of metrics bucket lists!


With system metrics especially, it’s much more meaningful to bring a systems approach to the problem and be clear on causal linkages e.g. attached (which includes the AS&R example – note the strategy mapping in particular).


In terms of outcomes, we should also ask from our service users / public standpoint what they would expect to see? So we need to include some PREMS (eg Picker Institute Qs) and PROMS (eg ICECAP-A / ICECAP-O).


Clinical measures would be at the service delivery outputs level I think (rather than at the outcome level), and I think a key part of that would be access – primary care access measures in particular. We need to secure access to data for that.


Some measure of the number of new care plans and care plan reviews would be important primary care clinical process measures.


Re strategic outcomes – a key thing is the ‘smaller hospitals’ outcome (eg number of acute beds used – this is not the same as acute bed-days – you can reduce bed-days with no change in the size of hospital by simply reducing LoS).


An output on the road to the smaller hospitals outcome (and to counter the trend in reduced acute bed-days leading to increased spells and therefore increased costs) is an aim of eliminating zero and short LoS admissions. So we need to keep a tally of those and recognise (contrary to current thinking) that more is not good, fewer is good.


Readmissions (readmission from IC and IC beds in particular) and reablement failures would be important output measures.


Need simplified measures of (unwarranted) variation in some of the above eg GP neighbourhoods.


I’ll stop there. Strongly think we need to take a systematic run at this.

And the thoughts of a second reviewer

Critique 2
Covers a range of pertinent domains around which we can organise things.

As always there are more indicators available than are included in any specific dictionary

It’s the choice of the dictionary developer to include x,y or z which is always unknown. 

Would be good to have rationale as to what the selection process was 

As with all of these things, it boils down to detailed knowledge of what raw data is / isn’t collected, the robustness and data q issues associated with it, what level of granularity, how it can / can’t be used, what the smallprint says, what the caveats are and the questions you’re trying to answer with the dictionary

Like one of my mighty clever staff members, I’m not a fan of bucket list of indicator sets….that haven’t been bespoke designed for a purpose, with logic model that sets that out.

I’m a fan of understanding the raw data; having a specific job, question or system in mind; and design of indicator set in that context. 

There’s something also about micro, meso and macro.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s