Mortality outliers programme

What is a mortality outlier and what is the outliers programme?

We use the term ‘outlier’ to describe a service that lies outside the expected range of performance. One example of where we use this is ourmortality outliers programme.Our process involves analysing data that suggests concerning trends in the death rate for specific conditions or operations.

This data is different from other data on death rates, such as Hospital Standardised Mortality Rates (HSMR). It looks at changes over time, rather than giving a snapshot of a moment in time.It also looks at patients with a specific disease rather than all patients admitted to the hospital.Our data is therefore more specific and moretimely than HSMR.

The trends are calculated using a statistical technique known as statistical process control (SPC).This measures where there has been an increase in death rate which is greater than could be explained confidently by random variation over time. These measures may be indicative of problems in the quality of care (although as we explain below there are several other explanations) so they are of interest to us as a regulator, and the reasons for the outlier need to be understood.A concerning trend is not, of itself, evidence of poor quality.

Graphs such as that in figure 1 (from the Dr Foster Unit at Imperial College London) show the development of the concerning trend over time.When a fixed threshold (marked by the dark line at point “7” on this graph) is crossed, the variation in death rate is greater than we can confidently explain by random variation.The trend line is reset once an alert has been raised.

Figure 1 – example of a mortality outlier alert

Source: Dr Foster Unit at ImperialCollegeLondon

Where do we get our information from?

All of our outliers are calculated usingpatient-level data from hospitals which become part of a national system called hospital episode statistics (HES). Some outliers are calculated by ImperialCollege and provided to us, while others we calculate ourselves.

What do we do when we get an outlier alert?

Our process can be broken down into four different stages:

Stage 1:Analytic consideration, to see if the alert can be explained by a data anomaly

Stage 2:Work with our regional teams to gather additional intelligence

Stage 3:Central decision making panel involving clinical and data experts

Stage 4:Further consideration with the Trust(s) about the alert (where required)

In many cases we decide to proceed no further than stage three in the process, because, for example,our analysis shows quite clearly that the alert is the result of a statistical anomaly. If that is the case we will close the case after the clinical decision panel and write to let all parties know. In other cases, we may not have enough other intelligence to safely make the decision so we ask the trust involved (copying in the strategic health authority or Monitor and the local primary care trust) for more information.

Who do we tell about the alert and when?

Throughout the process we let local organisations involved in the alert know about what we are doing, and what we know. As well as the relevant NHS trust we also alert the strategic health authorityor Monitor and the host primary care trust.

Our specific communicationsare as follows:

1.We contact the trust advising them that we have received an alert and that we are starting the analytic phase of our enquiry.

2.We contact the trust informing them of the outcome of the clinical decision panel, advising them either that the case has been closed or requesting further information.

3.If we have asked for further information, on receiving that information we will respond, either continuing the enquiry or closing the case.

Does an alert mean that you have judged services to be poor?

No. Alerts can be caused by at least one of three reasons:

  • The trust is treating more complicated cases than the norm, which cannot be adequately reflected completely in the recording of individual diagnoses.
  • Problems with data quality and coding meaning that the patients who died were incorrectly assigned to a specific diagnosis.
  • Poor quality of care.

What should a trust do if it receives an alert?

An alert is a potentially important warning about problems, which needs to be taken seriously. When writing to NHS trusts we will set out a number of questions which we believe will help to identify whether there is a problem or not. From our point of view the importance of the outlier programme is to stimulate local enquiry and, if required, improvement activity. One recommended response would be for the trustto undertake urgent consideration of quality of care (for example through a case note review).

What do we do when we find poor quality care?

When we find poor quality care we will intervene.The nature of this intervention will depend upon the precise nature of the issue.In the past, interventions have ranged from agreement and monitoring of action plans to address the identified issue, to full-scale investigation. The Care Quality Commission has a wider range of enforcement powers available to it than its predecessor bodies and will use these to ensure improvements in quality.In addition to regulatory powers, we will work with other local NHS bodies, such as strategic health authorities, Monitor and primary care trusts to make sure that other available tools, including performance management and commissioning, are used to address shortcomings in quality.

Will you make information about individual alerts publicly available?

Yes, once we have closed cases, we will make the results of the enquiry publicly available on a quarterly basis. We will not make alerts publicly available until this point, as we would be providing partial and potentially

very misleading information before we know what caused the alert that

we received.