Describing the quality of New Zealand’s health and disability services:

Developing our Health Quality and Safety Indicators

Engagement and feedback document

Published in July 2012 by the Health Quality & Safety Commission, PO Box 25496, Wellington 6146

This document is available on the Health Quality & Safety Commission website:

For information on this report please contact

Page 1 of 39


Contents

Executive summary

1.Introduction

1.1Purpose

1.2How can I provide feedback?

2.Why develop quality and safety indicators for New Zealand?

2.1What do we mean by indicators?

2.2Measurement and indicators as part of the Commission’s toolkit

3.Definition of the indicator framework

3.1What do we seek to achieve?

3.1.1Our goal

3.1.2Our objectives

3.2What will the indicators address?

3.2.1What level of measurement?

3.2.2What services are within the scope?

3.2.3Which dimensions of quality are of interest?

3.3Who are the audiences?

4.Identifying the initial indicator set

4.1Overview of approach

4.1.1Project structure

4.1.2Process

4.2Summary list of proposed indicators and measures

4.3Balance of indicators across the scope of coverage

5.Quality and safety indicators – initial results

5.1Cancellation of elective surgery by the hospital

5.2Amenable Mortality

5.3Ambulatory sensitive hospitalisation

5.4Occupied bed-days for people aged 75+ admitted 2 or more times per year

5.5Day case surgery turns into unplanned overnight stay

5.6Hospital unplanned and unexpected readmission

5.7Eligible population up to date with cervical screening

5.8Age appropriate vaccinations for 2 year olds

5.9Healthcare cost per capita (US$ Purchasing Power Parity per capita) and

5.10Health care expenditure as a proportion of GDP

6.Further indicator development

Appendices

Appendix 1 – Expert Advisory Group

Page 1 of 39

Executive summary

This report presents the first set of health quality and safety indicators to be published by the New Zealand Health Quality & Safety Commission (the Commission) and seeks feedback from interested parties on the overall set. Publishing a set of health quality and safety indicators is a core part of the Commission’s work and is required under the New Zealand Public Health & Disability Amendment Act 2010, where the Commission is charged with:

  • providing advice to the Minister of Health on how quality and safety in health and disability support services may be improved
  • leading and coordinating improvements in quality and safety in health care
  • identifying key health and safety indicators (such as events resulting in injury or death) to inform and monitor improvements in quality and safety
  • reporting publicly on quality and safety, including performance against national indicators
  • sharing knowledge about and advocating for quality and safety.

In developing these indicators the Commission has built upon existing quality and safety initiatives throughout the sector, and involved measurement experts and stakeholders in the process of selection and definition. The Commission has also sought to select indicators across a wide range of dimensions of health care delivery and consumer experience of health care. This report both presents the work that has gone on towards developing 17 proposed indicators, presents findings where available, and asks for feedback and suggestions on how this indicator set could be further developed and strengthened.

Structure of Report

Section 1 provides details on how interested parties can provide feedback, listing the questions on which the Commission is most interested in receiving feedback. Sections 2 and 3 focus on explaining the international context in which health quality and safety indicators have been developed and how they fit with other work currently being undertaken by the Commission. These sections also provide the particular rationale and aims of indicator development in the New Zealand context, and describe possible audiences and their likely interest in the measures. Section 4 sets out the process taken by the Commission to develop and publish the first set of indicators.

Indicators have been categorised according to:

  • how ready they are for publication.This depends upon the robustness of the indicator and whether it is already in use or under development and availability of data. Indicators are categorised as either fast-track, under development or placeholder
  • whether they are a system level indicator or a contributory measure. System level indicators are headline indicators that provide a balanced picture of the status of the quality and safety of health and disability support services in New Zealand. Contributory measures focus on reporting on outcomes of health care delivery for defined patient populations, specific quality activities or services delivered
  • the overall focus (whether on safety, patient experience, effectiveness, access, efficiency or on equity).

Section 5 brings the discussion of indicators to life, through presenting initial findings where these are available.

Page 1 of 39

1.Introduction

1.1Purpose

The New Zealand Health Quality Safety Commission(the Commission) was established in 2010 to lead and co-ordinate work on monitoring and improving the quality and safety of health and disability support services to ensure all New Zealanders receive the best health and disability care within our available resources.

As part of its work on measurement and evaluation, the Commission is required under legislation to develop and regularly publish a set of indicators to drive improvement of the quality and safety of health and disability support services provided within New Zealand.It seeks to do this in a way that complements and builds on existing initiatives and learns from and involves stakeholders and key experts in the field of quality measurement.

This document is the Commission’s first publication of the proposed set of health quality and safety indicators (‘HQSIs’ or the ‘indicators’).

In this report we:

•explain why we are developing the indicators and what we seek to achieve

•set out our thinking behind the creation of the draft set of indicators and recount the process taken to develop it

•provide details for 17 proposed indicators, including 12 ‘fast-track’ indicators (where there is an existing defined and tested indicator)

•publish preliminary results for those indicators where data is readily available

•identify the other areas where we believe indicators are required, that will be developed in the future

•publish available indicators

•engage the sector and stimulate debate to encourage feedback on our proposed approach and set of indicators.

1.2How can I provide feedback?

We are interested in your views on our initial indicator set, how these indicators could be used, and our overall approach to developing the full indicator set in the longer term. Your feedback will help shape this process and help ensure indicator information will be applied to improve the quality and safety of New Zealand’s health services.

We are specifically interested in your answers to the following questions.

  1. Do you agree with the stated purpose of the indicators?
  2. Is the range of topics covered in the scope of the indicator set wide enough?
  3. What, if any, other services or quality dimensions would you include?
  4. What, if any, barriers do you envisage inreporting quality and safety indicator information?
  5. How could the reporting of the quality and safety indicators be facilitated or supported?
  6. What use is the information generated by the initial set of indicatorsto you? What impact will it have on your activities?
  7. Which, if any, of the indicators in the initial set are most useful? And why?
  8. Which, if any, of the indicators in the initial set do you see as problematic? Can you explain why?
  9. Are there any other indicators you think should be added to the initial set? If so, which ones and why?
  10. Are there any indicators or topics that you think should be included in future sets of indicators? If so, what are they and why?
  11. Are you interested in helping the Commission develop quality and safety indicator set? If you are, please provide details of the perspective or expertise you could contribute and your preferred contact details.

2.Why develop quality and safety indicators for New Zealand?

2.1What do we mean by indicators?

Over recent decades, there has been a growing focus in the healthcare systems of developed countries on measuring health care quality through national and local performance reporting. A recent King’s Fund report[1] contained the following quote:

‘We can only be sure to improve what we can actually measure.’

There are a range of definitions and understandings of what is meant by various terms used in the field of quality improvement, measurement and evaluation. We have adopted the definition from the NHS Institute for Innovation and Improvement (2008)[2]to describe our understanding of the term, as outlined below:

‘An indicator is a summary measure that aims to describe in a few numbers as much detail as possible about a system, to help understand, compare, predict, improve, and innovate.’

2.2Measurement and indicators as part of the Commission’s toolkit

International literature provides 20 years of evidence that measuring the quality of healthcare and communicating the results in a variety of ways and settings is a powerful way to stimulate improvement in healthcare. However, the literature also suggests that maximising this effect requires a clear understanding of the purpose of the measurement which in turn should influence the precise nature of the measurement.

In the New Zealand context, this prompts a number of questions.

  • Are we interested in the average performance of the overall system, or in the variations occurring between different areas or institutions?
  • Are we seeking to use measures to make an absolute judgement or to ask valuable questions?
  • Are we seeking primarily to provide accountability or to stimulate improvement?
  • Does this information relate to populations, or to institutions or to individuals?

Measurement and evaluation is a critical part of the Commission’s work, underpinning and forming the basis of some specific work programmes, as outlined below.

  • Atlas of Healthcare Variation: Concentrating on individual conditions and clinical groups, the Atlas highlights variation in order to stimulate discussion about difference in practice and the improvement actions required to eliminate these where unwarranted.
  • Quality and safety markers: These markers are a mix of process and outcome measures focused on reducing harm from in-patient falls, hospital-acquired infections, surgery and medication. They do this through the setting of expected levels of improvement, public reporting of progress against these thresholds and links to accountability mechanisms.
  • Measurement and evaluation of improvement programmes: The Commission supports a range of specific quality and safety improvement programmes (such as the Medication Safety programme that aims to reduce harm from medication errors and increase the efficiency and safety of medication management systems in the sector). The Commission approaches the design and establishment of such programmes with a clear view of the desired outcomes and the approach to measuring change.
  • Health quality and safety indicators: These are a small set of summary indicators which provide the public and the health and disability sector with a clear picture of the quality and safety of health and disability services in New Zealand, including changes over time.

3.Definition of the indicator framework

3.1What do we seek to achieve?

3.1.1Our goal

The over-arching goal of reporting against a set of quality and safety indicators is to provide robust information to support achievement and measure progress against delivery of the outcomes articulated in the New Zealand Triple Aim framework, namely:

•improved quality, safety and experience of care

•improved health and equity for all populations

•best value from public health system resources.

3.1.2Our objectives

Developing a set of credible, robust and reliable quality and safety indicators will help us achieve the following objectives:

•provide the public and the health and disability sector with a clear picture of the quality and safety of health and disability services in New Zealand, including changes over time

•inform quality improvement activities of service providers by providing information to support learning and peer review in clinical settings

•support the identification of key quality and safety issues and prioritisation of improvements to the quality and safety of health and disability support services.

3.2What will the indicators address?

In this section we describe the scope of services and quality domains that the indicators will cover. (Exhibit 1 illustrates these dimensions and shows how they all fit together.)

Overtime there are likely to be changes to the set, as definitions for existing indicators are refined, as new indicators are added (reflecting priorities identified by the sector or determined through the Commission’s work programme) and as others are ’retired‘ as they become less relevant.

The first set of indicators published will be unable to provide coverage across the entire scope of health care. We are proposing a pragmatic approach, starting with a relatively small set of indicators to get the process rolling and to test the framework. It is likely the number of indicators will increase a little as coverage becomes more comprehensive.

3.2.1What level of measurement?

In designing the framework for the indicator set, our key sector stakeholders from the expert advisory group strongly advocated for a two-tier approach.

•System-level indicators –a small set of headline indicators that provide a balanced picture of the status of the quality and safety of health and disability support services in New Zealand.

•Contributory measures– that focus on the delivery of healthcare services reporting on outcomes for defined patient populations, specific quality activities or services delivered.

3.2.2What services are within the scope?

The scope of the indicator set will potentially cover services throughout the patient’s journey, provided across the entire health and disability sector, including:

  • public, private and non-government organisation (NGO) providers
  • primary care, hospital, aged care, mental health and disability support sectors.

It is not essential that all indicators and contributory measures can be applied across every type of service provider. However, as we develop the quality and safety indicators we will seek to ensure a balanced spread of indicators that reflect all service areas within the defined scope. (We recognise our first proposed set of indicators has a bias towards hospital care, although some of the measures can act as proxies for the effective coordination of care across the whole system.)

3.2.3Which dimensions of quality are of interest?

The three outcome areas of the New Zealand Triple Aim provide the foundations for the indicator framework. These reflect a broad interpretation of ‘quality’ across the system, population and patient. As such, the set of indicators selected should represent a balanced spread relating to all contributing dimensions of quality including safety, patient experience, effectiveness, access, efficiency and equity.

3.3Who are the audiences?

A range of potential audiences will have different interests and requirements from the information provided, as outlined in Table 1 below.

Table 1: Audience interests and requirements

Some key audiences / Inform quality improvement activities to support learning in clinical settings / Provide a picture of quality and safety of health services / Identify priorities for improving quality and safety / Specific interests and requirements
Members of the public /  / Clear, concise, accessible information that tells a story.
Clinicians & other service providers /  /  /  / Primary interest likely to be in measures directly relevant to clinical/service activities. Need to win ‘hearts and minds’ by ensuring information is robust and relevant.
Private service providers, eg, nursing homes & NGOs /  /  / 
District health boards /  /  /  / Interests from range of perspectives/different parts of organisation.
Ministry of Health and Ministers /  /  / Alignment/improvement against priorities.

Page 1 of 39

Exhibit 1– The HQSI framework

Page 1 of 39

4.Identifying the initial indicator set

4.1Overview of approach

4.1.1Project structure

The Health Quality and Safety Indicator project (the HQSI project) was established to oversee the development and implementation of the initial set of indicators.

A Project Steering Group from the Commission has been working closely with an External Advisory Group (EAG) made up from key stakeholders and sector experts. (Membership of the EAG is listed at Appendix 1.)

4.1.2Process

Guiding principles

When developing the initial indicatorset, the Commission wanted to build on existing work in the field of quality and safety measurement both in New Zealand and internationally and to minimise the reporting burden on the sector. This means wherever feasible and relevant we sought to:

•useindicators that have already been developed, defined and tested

•use existing data collections

•alignindicators to other parts of the Commission work programme

•select some international measures to enable comparability with other countries.

These principles underpinned the following step by step process.

Stage One: Review of Institute for Healthcare Improvement (IHI) Whole System Measures

The IHI Whole System Measures[3] were developed as a balanced set of metrics to measure the overall quality of a health system. This approach is strongly aligned to the purpose envisaged for the New Zealand HQSI indicators.

As we developed our initial set of indicators, we reviewed the IHI measures as a ‘start-point’ and used them as a ‘straw man’ to test coverage of our key areas of interest and relevance in a New Zealand context. This resulted in several IHI measures being adopted and adapted for the first indicator set.

Other measures were identified as useful for the development of future indicators, but require further work on identifying and assessing relevant sources of data.

Stage Two: Identifying existingquality and safety indicators

The project team alsoundertook extensive research to identify and assess existing indicators in terms of fit with the HQSI purpose from both New Zealand and overseas indicator frameworks.

A number of indicators fell within this scope, including those from the:

•New ZealandDistrict Health Board Hospital Quality and Productivity programme

•New ZealandPrimary Health Organisation (PHO) performance programme