Procedures of Limited Clinical Effectiveness Monitoring Report – A How To Guide

Purpose

The purpose of the PoLCE monitoring reports was to provide a real-time benchmarked view of the numbers and costs of Procedures of Limited Clinical Effectiveness (PoLCE) carried out in London. It was intended that they would help stimulate local discussion, contributing to a reduction in levels of variation.

Methodology

The information behind the reports was extracted using SUS PbR Mart, post reconciliation point, finished consultant episode data for the period from 1st April 2009. Procedures of Limited Clinical Effectiveness were identified occurring as either the spell primary procedure or core HRG code for Varicose Veins. A full list of codes is provided in appendix 1.

The monitoring report was designed to be compatible with the DFI benchmarking report Commissioning of Clinically Effective Healthcare Interventions, updated report 30 July 2010 in terms of the procedures, codes and criteria included in the SQL procedures. Some inconsistency remained with regard to total annual expenditure due to methodological differences. The CSL analysis takes into account the HRG complexity and includes a MFF adjustment, which we hope provides a more accurate reflection. Cancelled procedures were excluded from the monitoring report as the tariff for 2010/11 was set to zero in SUS as the handling of these was subject to local negotiation.

The name of the fields that were behind the monitoring report can be found in the “Data_Fields” tab of appendix 1. All of these fields should be extracted from SUS, or calculated from the output of SUS. For example, the “Age_Group” field was calculated in excel following the extraction of the data.

To create the monitoring reports as CSL, the data was extracted from SUS to a pipe-delimited (i.e. a text file where the boundary between the separate fields was noted with the | character) .rpt file. From here, this data was imported into excel. Thus a “flat” table was created, with over 230,000 rows. From here, a pivot table was created by selecting all of the data and clicking on the “insert” tab of the excel ribbon, and then clicking on “pivot table”. By clicking on “OK” on the box that popped up, a blank pivot table was created in a new tab of the excel file. From here it is relatively simple to manipulate the fields (e.g. adding or removing the different fields to the rows and columns of the pivot table) to get the precise output wanted. The “PBR_Spell_Identifier” field was used to get total number of spells, and the “Value_Total” field was used to get the overall cost (these values are established by dropping these fields to the “∑ Value” section when creating the pivot table. The “PBR_Spell_Identifier” field should be “count” while the “Value_Total” field should be “sum”. To change these options click on the black arrow next to the field name in the “∑ Value” box and then click on “value field settings”).

Once the pivot table has been successfully created, it can be cut and pasted in a separate excel document or tab to create a “static” table. From there, the table can be made to look better by using any one of the many formatting techniques available in Microsoft Excel. For example, in the monitoring reports produced by CSL, these techniques included adding a title to the table, merging cells, adding a border to the cells, colouring the cells to specify which sector the particular line of data came from, aligning the data centrally within the cell and formatting the content of the cell. From this data, a chart can then be produced to visualise the data in the table. In the monitoring report produced by CSL, this was typically done by producing a line chart, which allowed for the monitoring of trends by month. Again, these charts were formatted to improve appearance. Techniques included adding a title, labels to both axes, moving the key to the bottom of the chart, adding and modifying the appearance of gridlines, and modifying the appearance of the lines making up the chart.

N.B. The pivot table function in excel is extremely flexible, so any combination of the variables can be used to create any number of tables.

Adding some context to the data

Although useful, just producing overall numbers of admissions and total cost can be a bit misleading. For example, looking at the overall spend and overall number of admissions by cluster, Inner North East London performs the best. However, Inner North East London is made up of only 3 PCTs, so this clusters’ population is lower than the other clusters in London. Therefore, to provide some context to the data, the monitoring reports have incorporated some information about populations. They have done this in two ways. The first (and most crude) way is to look at the number of admissions and overall spend for the whole of the population. The statistic used is number of admissions/spend per 100,000 population. It is calculated simply by dividing either the number of admissions or total spend by the total population for the geographical area you are looking at and multiplying the result by 100,000. The populations used are the mid-year estimates (MYE) published by the Office for National Statistics (ONS) which can be found at websites such as www.nchod.nhs.uk.

Although a useful starting point, this crude rate does not take into account the age or sex breakdown of a population. Some procedures may be more prevalent among older people, or may be gender specific. Therefore, areas with a relatively young population or with a higher than average proportion of males or females may have a low admission rate per 100,000 total population, even though the admission rate among the population most at risk is high. A way to get round this is to use a directly standardised rate.

A directly standardised rate is calculated by taking the age-specific rates from your local population, and applying them to a chosen standard population. When calculating directly standardised rates, it is common practice split the population into quinary (5-year) age-groups. The exceptions are the under 1s, those aged 1-4 and those over the age of 85. The standard population used is the European standard population (details of which can be found in the supporting documentation). Also included in the supporting documentation is a template which provides a quick and easy way to calculate these rates by simply adding in the observed number of local events. The templates also automatically calculate confidence intervals (using the standard APHO method). Also, the template gives an indication of whether the calculated rates are significantly different from England or London.

N.B. due to the availability of SUS data by each local organisation, and data disclosure issues, it may require collaborative working across the sectors to get the full picture for London, and then only for those procedures with relatively high activity.

Caveats

Each cluster has its own thresholds and processes for defining and managing PoLCE (e.g. cap, some approval criteria, no approval criteria) etc so there will be an inbuilt difference in activity between clusters. The monitoring report indicates total expenditure as opposed to total potential savings. When activity has taken place it is not possibleto determine from the data presented whether patients had met local criteria or gone through the appropriate local approval processes. Expenditure has not been adjusted to take into account any contractual agreements that might allow PCTs to claim back expenditure against certain procedures.

Disclosure control

In order to protect patient confidentiality, numbers ≤5 have been suppressed.

Appendices / supporting documentation

Appendix 1: SUS fields and full list of codes

Appendices 2-3: Extracts from the report that CSL commissioned from Dr Foster Intelligence indicating which PCTs and which providers are statistically significant outliers.

Appendix 4: European standard population

Appendix 5: Template for calculation of standardised rates

Appendices 6-7: Example monitoring reports produced by CSL