URBIS TEAM RESPONSIBLE FOR THIS REPORT:
Directors / Alison WallaceDr Linda Kurti
Associate Directors / Tomas Lopata
Duncan Rintoul
Senior Consultant / Zoe Cox
Sub Consultants / Murray Benton, INCA Consultancy
Acting Professor Margaret Kelaher, Centre for Health Policy, Programs and Economics, Melbourne School of Population Health, University of Melbourne
Expert Advisors / Professor Ross Bailie and Dr Yin Paradies, Menzies School of Health Research
Group Support / Lynda Jones, Alison Rees, Jillian Yeomans
Job Code / KAJ44109
xdisclaimerx
TABLE OF CONTENTSAppendix AICDP Monitoring and Evaluation Reference Group Members
Appendix BList of Stakeholders Consulted
Appendix CDoHA Divisions Responsible for Managing ICDP Measures
Appendix DMatrix Mapping Sources of Data (Package-wide and Measure-specific) Reported Against ICDP Results as Specified in the Package-wide Program Logic
Appendix EData Sources
Appendix FData limitations
Appendix GTerminology
Appendix HKey Indicators with Data Sources
ICDP Monitoring and Evaluation Framework Volume 3appendices
Appendix AICDP Monitoring and Evaluation Reference Group Members
ICDP Monitoring and Evaluation Reference Group Members
Name / OrganisationMs Alison Killen / Assistant Secretary, Better Health Care Branch Office for Aboriginal and Torres Strait Islander Health
Dr Ron Borland / Cancer Council Victoria
Dr Alex Brown / Baker IDI Heart and Diabetes Institute (Melbourne)
Professor Sandra Eades / Baker IDI Heart and Diabetes Institute (Melbourne)
Dr Lauren Cordwell / Royal Australian College of General Practitioners
Dr John Daniels / Redfern Aboriginal Medical Service
Dr Sophie Couzos / National Aboriginal Community Controlled Health Organisation (NACCHO)
Ms Dea Delaney Thiele / NACCHO
Mr Justin Mohamed / NACCHO
Dr Carmel Nelson / Kimberley Aboriginal Medical Services Council
Ms Indrani Peris-Caldwell / Australian Institute for Health and Welfare
Professor Lisa Jackson Pulver / Muru Marri Indigenous Health Unit, University NSW
Ms Leanne Wells / Australian General Practice Network
Professor Kerin O’Dea / Sansom Institute, University SA
Dr David Dumbrell / Department of Health and Ageing
Mr Jeff McKenzie / Department of Health and Ageing
Appendix BList of Stakeholders Consulted
Over sixty stakeholders provided feedback on various drafts of the Monitoring and Evaluation Framework, drawn from the following:
- The ICDP Monitoring and Evaluation Framework Reference Group
- NACCHO
- NACCHO Affiliates
- AGPN
- Australian Primary Care Collaboratives
- State/Territory Departments of Health
- DoHA personnel.
Appendix CDoHA Divisions Responsible for Managing ICDP Measures
ICDP Measure Management Responsibility
Measure / DoHA Division Management ResponsibilityA1National Action to Reduce Indigenous Smoking Rates /
- Business Group Division
- Mental Health and Chronic Disease Division
A2Helping Indigenous Australians Reduce their Risk of Chronic Disease /
- Mental Health and Chronic Disease Division
A3Local Indigenous Community Campaigns to Promote Better Health /
- Business Group Division
B1Subsidising PBS Medicine Co-payments /
- Pharmaceutical Benefits Division
B2Higher Utilisation Costs for MBS and PBS /
- Medical Benefits Division
B3Supporting Primary Care Providers to Coordinate Chronic Disease Management /
- Primary and Ambulatory Care Division
B4Improving Indigenous Participation in Health Care through Chronic Disease Self Management /
- Mental Health and Chronic Disease Division
B5Increasing Access to Specialist and Multidisciplinary Team Care /
- Primary and Ambulatory Care Division
B6Monitoring and Evaluation /
- OATSIH
C1Workforce Support, Education and Training /
- Health Workforce Division
C2Expanding the Outreach and Service Capacity of Indigenous Health Organisations /
- OATSIH
C3Engaging Divisions of General Practice to Improve Indigenous Access to Mainstream Primary Care /
- Primary and Ambulatory Care Division
C4Attracting More People to Work in Indigenous Health /
- Business Group Division
C5Clinical Practice and Decision Support Guidelines /
- OATSIH
Appendix DMatrix Mapping Sources of Data (Package-wide and Measure-specific) Reported Against ICDP Results as Specified in the Package-wide Program Logic
ICDP Monitoring and Evaluation Framework Volume 3 / Page 1appendices
Appendix EMatrix Mapping Sources of Data (Package-wide and Measure-specific) Reported Against ICDP Results as specified in the Package-wide Program Logic
MEDIUM TERM RESULTSResults / ICDP / A1 / A2 / A3 / B1 / B3A / B3B / B4 / B5A / B5
B / C1 / C2 / C3 / C4 / C5
Chronic disease risk factors / There is a reduction in the incidence of preventable chronic disease risk factors among Indigenous Australians. / X / X / X / X
Smoking rates among Indigenous Australians are reduced. / X / X / X / X
More Indigenous Australians with or at risk of chronic disease adopt healthy lifestyle choices relating to smoking, nutrition and exercise. / X / X / X / X
Chronic disease manage-ment and follow up care / ICDP-funded health care services deliver a comprehensive and coordinated approach to chronic disease management, including increased and earlier access to primary health care, specialist and allied health services, affordable care and medicines. / X / X / X / X / X / X / X / X
More Indigenous Australians with or at risk of chronic disease actively participate in their own healthcare. / X / X / X / X / X / X / X / X / X / X
Health outcomes are improved among Indigenous Australians with or at risk of chronic disease who participate in ICDP measures. / X
Workforce expansion and support / More health care providers are equipped to assist Indigenous Australians with or at risk of chronic disease to make healthy lifestyle choices and to manage their condition. / X / X / X / X / X / X / X / X / X / X / X / X
More health care providers are accessed by and provide quality care to Indigenous Australians with or at risk of chronic disease. / X / X / X / X / X / X / X / X / X / X / X
There is an increase in the workforce providing primary health care and other health services to Indigenous Australians. / X / X / X / X / X / X / X / X / X / X / X / X / X
EARLY RESULTS
Results / ICDP / A1 / A2 / A3 / B1 / B3A / B3B / B4 / B5A / B5
B / C1 / C2 / C3 / C4 / C5
Chronic disease risk factors / Resources for designing and delivering health promotion campaigns are accessible, effective and evidence-based / X / X / X
Indigenous Australians who have had contact with the ICDP have a better knowledge and understanding of the impact of preventable chronic disease risk factors on their wellbeing. / X / X / X / X / X / X
Indigenous Australians who have had contact with the ICDP are more aware of and utilise the expanded range of health services and supports available to them to adopt healthy lifestyle choicesand reduce smoking. / X / X / X / X / X / X / X / X / X / X / X
Indigenous Australians who have had contact with the ICDP make positive decisions about their health and lifestyle. / X / X / X / X / X
Chronic disease manage- ment and follow-up care / ICDP-funded health system supports, incentives and subsidies are operating to facilitate the provision of quality primary health care for Indigenous Australians with chronic disease. / X / X / X / X / X / X / X / X / X
Financial and other barriers to accessing health care and medicines are reduced. / X / X / X / X / X / X / X
Health services funded under the ICDP demonstrate cultural awareness and commitment. / X / X / X / X / X / X
Care coordination within ICDP-funded services is improved for Indigenous Australians with or at risk of chronic disease. / X / X / X / X / X / X / X / X
Indigenous Australians with chronic disease or associated risk factors have more services and supports available to help them manage their condition. / X / X / X / X / X / X / X / X / X / X
The number of Indigenous Australians with or at risk of chronic disease who access primary health care services is increased. / X / X / X / X / X / X / X / X / X
Indigenous Australians in contact with ICDP measures value the enhanced services. / X / X / X / X / X / X / X / X
The number of Indigenous Australians with or at risk of chronic disease who access specialist and multi-disciplinary follow-up care is increased. / X / X / X / X
Workforce expansion and support / Health care providers demonstrate increased knowledge and improved practice in relation to the prevention, early identification and management of chronic disease for Indigenous Australians. / X / X / X / X / X / X / X / X / X
The ICDP workforce is retained and developed within funded services. / X / X / X / X / X / X / X / X / X / X
Marketing, training and recruitment strategies are successful in encouraging more people to work in primary health care and other services available to Indigenous Australians. / X / X / X / X / X / X / X / X / X
OUTPUTS (YEAR 1 AND ONGOING)
Results / ICDP / A1 / A2 / A3 / B1 / B3A / B3B / B4 / B5A / B5
B / C1 / C2 / C3 / C4 / C5
Workforce expansion and support / The workforce required to implements the ICDP is recruited, oriented and trained. / X / X / X / X / X / X / X / X / X / X / X / X / X / X
Package measures are efficiently implemented in accordance with agreed guidelines and timelines / X / X / X / X / X / X / X / X / X / X / X / X / X / X
Monitoring and reporting requirements are met / X / X / X / X / X / X / X / X / X / X / X / X / X / X
Internal and external stakeholders, sentinel sites and service providers consider the implementation of the Package to be addressing identified needs, and enhancing the existing service system / X / X / X / X / X / X / X / X / X / X / X / X / X / X
ICDP Monitoring and Evaluation Framework Volume 3 / Page 1
appendices
Appendix FData Sources
Defining data sources terminology in the Monitoring and Evaluation Framework
Terminology / DescriptorAboriginal and Torres Strait Islander Health Performance Framework (HPF) / Published biennially by the AIHW incorporating a wide range of chronic disease performance indicators collected through various national and other databases/surveys (more information on HPF data sources and technical notes are included in this Appendix)
AGPN / As a key stakeholder in a number of ICDP measures, the AGPN will participate in a number of monitoring and evaluation activities
APCC / Program data from over 500 participating medical practices including clinical and service enhancement measurements
BEACH / Collects information about clinical activities in general practice in Australia. The BEACH database includes over 1,200,000 GP-patient encounter records. Annually BEACH analyses data from 100 consecutive consultations from each of the randomly selected sample of 1,000 GPs
CDSM register / Number of patients enrolled at the practice level in the Chronic Disease Management Program
Clinical data systems and/or quality improvement systems / Systematic collection and reporting of data relating to clinical performance and clinical outcomes
Community members consultation / Face to face interviews or focus groups with members of local Indigenous communities to assess knowledge, awareness and impact of ICDP measures on local individuals, families, and communities
Community-level survey / Quantitative survey tool used in A1 evaluation to assess impact of health promotion and social marketing campaign activity, to be determined when measure evaluations are developed
Consultation with … / Face to face or telephone interviews, focus groups, email or other direct interaction with stakeholders (key stakeholders are identified in each measure)
GPET / Data regarding postgraduate vocational training activity including the number of services requesting a general practice registrar length of time before these vacancies are filled and the number of placements which are completed
Health workforce survey / One or more surveys of the ICDP workforce to be developed by the evaluator to assess impact of ICDP measures on health care professionals
Healthy for Life (H4L) / Program data set within OATSIH, including data from participating Indigenous and other health services within a web-based reporting system
MBS / Data held by Medicare Australia regarding uptake of clinical services provided under the Medical Benefits Schedule
MSOAP / Program data held by DoHA including the number of patients per location and in total, number of visits and the types of participating organisations
National Aboriginal and Torres Strait Islander Health Survey (NATSIHS) / A survey of a representative sample of Aboriginal and Torres Strait Islander Australians, conducted at six-yearly intervals, which covers a range of health and well-being issues
NT AHKPI / Northern Territory has developed a set of core indicators based on a range of clinical and social measurements in participating Indigenous community health services
OATSIH Services Reporting (OSR) / OATSIH service reporting tool for annual reporting by ACCHSs
Organisational survey / One or more surveys to be developed by the evaluator for Indigenous health services and general practices funded under the ICDP to assess the impact of ICDP measures at a service level
PBS / Data held by DoHA regarding utilisation of pharmaceuticals subsidised under the Pharmaceutical Benefits Scheme
PIP/PIPIHI / Data held by DoHA regarding the Practice Incentives Program, including registrations and uptake of the ICDP PIP Indigenous Health Incentive and process measures
Program documentation / The range of qualitative and quantitative information held by program managers within DoHA including information regarding design and implementation of measures, process data, and some outcome measures
QAIHC / QAIHC has developed a set of core indicators based on a range of clinical and social measurements in participating Indigenous community health services
USOAP / Program reporting data held by DoHA including the number and type of Program occasions of care, extent of Program participation by specialists and allied health professionals, and reported success factors in the implementation of the Program
ICDP Monitoring and Evaluation Framework Volume 3 / Page 1
appendices
Sources of data utilised in the ICDP Monitoring and Evaluation Frameworks (Package-wide and individual measures)
Data type/category / Data source / ICDP / A1 / A2 / A3 / B1 / B3a / B3b / B4 / B5a / B5b / C1 / C2 / C3 / C4 / C5National Performance Indicators as Reported in the HPF Publication / x / x / x / x / x / x
Administrative/system data sets / PIP / x / x / x / x
PIP IHI / x / x / x
MBS / x / x / x
PBS / x / x
APCC / x / x / x
NT AH KPI / x / x
Healthy for Life / x / x
QAIHC / x / x
BEACH / x
OSR / x / x / x
USOAP / x / x
MSOAP / x / x
NATSIHS / x
GPET / x
ICDP Program documentation / x / x / x / x / x / x / x / x / x / x / x / x / x / x / x
ICDP Program-specific evaluations / x / x / x / x
Surveys for ICDP evaluation / ICDP health workforce / x / x / x / x / x / x / x / x / x / x
Organisations funded by the ICDP / x / x / x / x
Website users (C5) / x
Longitudinal survey of health promotion program participants / x
In-depth stakeholder consultations conducted for the ICDP evaluation (one-on-one interviews and focus groups; stakeholders described in each framework) / ICDP stakeholders at national, State/Territory, regional and local level / x / x / x / x / x / x / x / x / x / x / x / x / x / x / x
Surveys and statistics produced by associated providers (e.g. RTOs, Quitline) / x / x / x / x / x
Clinical data systems and/or quality improvement systems / x / x / x
Other / Communications audit / x
ICDP Monitoring and Evaluation Framework Volume 3 / Page 1
appendices
Aboriginal and Torres Strait Islander Health Performance Framework data sources
Following the advice of Department of Health and Ageing, Urbis understands that the Australian Institute of Health and Welfare’s Aboriginal and Torres Strait Islander Health Performance Framework (HPF) is the agreed standard for monitoring changes in the health of Indigenous Australians. The HPF consists of 70 measures across three tiers - health status and outcomes, determinants of health and health systems performance.
Of the HPF’s 70 performance indicators. Urbis has identified 24 which are of particular relevance to the monitoring and evaluation of the ICDP. See Appendix G for Technical Notes on these data sources.
List of data sources utilised in HPF indicators as referenced in ICDP Monitoring and Evaluation Framework
Number / Name / Data SourcesHPF #1.02 / Top reasons for hospitalisation /
- AIHW National Hospital Morbidity Database
HPF #1.05 / Circulatory disease /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- Bettering the Evaluation and Care of Health Survey (BEACH)
- AIHW National Hospital Morbidity database
HPF #1.07 / High blood pressure /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- Bettering the Evaluation and Care of Health Survey (BEACH)
- AIHW National Hospital Morbidity database
HPF #1.08 / Diabetes /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- Bettering the Evaluation and Care of Health Survey (BEACH)
- AIHW National Hospital Morbidity database
HPF #1.09 / End stage renal disease /
- Australia and New Zealand Dialysis and Transplant Registry (ANZDATA)
- AIHW National Hospital Morbidity Database
- AIHW National Mortality Database
HPF #1.15 / Perceived health status /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- National Aboriginal and Torres Strait Islander Social Survey (NATSISS)
HPF #1.16 / Social and emotional wellbeing /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- National Aboriginal and Torres Strait Islander Social Survey (NATSISS)
- Western Australian Aboriginal Child Health Survey
- Bettering the Evaluation and Care of Health Survey (BEACH)
- AIHW National Hospital Morbidity database
- AIHW National Mortality Database
- AIHW National Community Mental Health Care Database
- AIHW National Residential Mental Health Care Database
HPF #1.17 / Life expectancy at birth /
- 2006 Census Population and Housing
- AIHW National Mortality Database
HPF #1.22 / All causes age-standardised death /
- AIHW National Mortality Database
HPF #1.23 / Leading causes of mortality /
- AIHW National Mortality Database
HPF #2.03 / Environmental tobacco smoke /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
HPF #2.18 / Tobacco use /
- 2004-05 National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- 2004 National Drug Strategy Household Survey
HPF #2.19 / Tobacco smoking during pregnancy /
- AIHW National Perinatal Data Collection
HPF #2.22 / Level of physical activity /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
HPF #2.23 / Dietary behaviour /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
HPF #2.26 / Prevalence of overweight and obesity /
- 2004-05 National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
HPF #3.03 / Early detection and early treatment /
- Medicare Database
- AIHW BreastScreen Australia Database
- National Bowel Cancer Screening Register
- 2004-05 National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- AIHW National Mortality Database
- Service Activity Reporting (SAR)
HPF #3.04 / Chronic disease management /
- Service Activity Reporting (SAR)
- Healthy for Life data collections
- Divisions of General Practice National Performance Indicators
HPF# 3.05 / Differential access to key hospital procedures /
- AIHW National Hospital Morbidity Database
HPF #3.06 / Ambulatory care sensitive hospital admissions /
- AIHW National Hospital Morbidity Database
HPF #3.10 / Aboriginal and Torres Strait Islander people in the health workforce /
- 2006 ABS Census of Population and Housing
HPF #3.12 / Access to services by types of service compared with need /
- National Aboriginal and Torres Strait Islander Health Survey (NATSIHS)
- National Aboriginal and Torres Strait Islander Social Survey (NATSISS)
- Community Housing and Infrastructure Needs Survey (CHINS)
- 2006 ABS Census of Population and Housing
- AIHW National Hospital Morbidity Database
- General Practitioner data
- Medicare Database
- AIHW Labour Force surveys
- Service Activity Report (SAR)
- Drug and Alcohol Service Reporting (DASR)
- Health expenditure data
- Palliative care data
HPF #3.18 / Aboriginal and Torres Strait Islander peoples training for health-related disciplines /
- Australian Government Department of Education, Science and Training’s Higher Education Student Statistics Collection
- National Centre for Vocational education Research Collection
HPF #3.20 / Recruitment and retention of clinical management staff (including GPs) /
- Service Activity Reporting (SAR)
- Rural Workforce Agency National Minimum Data
- GP Data
- AIHW Labour Force surveys
Appendix GData limitations