State of Iowa Performance Measures Guide
PERFORMANCE
MEASURES
GUIDE
State of Iowa
2005
1Acknowledgements
State of Iowa Performance Measures Guide
TABLE OF CONTENTS
ACKNOWLEDGEMENTS...... ii
AGA IMPLEMENTATION TASKS...... iii
INTRODUCTION...... I - 1
MEASUREMENT CHAPTERS
CHAPTER 1 - Measurement Concepts...... 1 - 1
CHAPTER 2 - Overview of the Measurement Process...... 2 - 1
CHAPTER 3 – Identifying, Locating, and Using Data...... 3 - 1
CHAPTER 4 – Communicating Data...... 4 - 1
ACCOUNTABLE GOVERNMENT COMPONENT CHAPTERS
CHAPTER 5 - Strategic Planning...... 5 - 1
CHAPTER 6 - Performance Planning...... 6 - 1
CHAPTER 7 - Iowa Excellence...... 7 - 1
CHAPTER 8 - Service Contracting...... 8 - 1
PERFORMANCE MEASUREMENT VOCABULARY...... V – 1
CONTACTS AND RESOURCES...... R – 1
1Table of Contents
State of Iowa Performance Measures Guide
ACKNOWLEDGEMENTS
The Performance Measures Guide is the result of the hard work and dedication of the following individuals and agencies:
Performance Measures GroupAgency
Jeff AndersonDepartment of Human Services
Greg AnlikerDepartment of Elder Affairs
Phyllis BloodDepartment of Human Rights
Mike CoveyouDepartment of Public Safety
Bernie HoyerDepartment of Natural Resources
Linda MillerDepartment of Education
Jerry NorthwickDepartment of Revenue
Mary Noss ReavelyDepartment of Management
David PutzDepartment of Transportation
Ron RobinsonLegislative Services Agency
Nickie WhitakerDepartment of Management
Doug WulfLegislative Service Agency
Bev ZylstraDepartment of Inspections and Appeals
1Acknowledgements
State of Iowa Performance Measures Guide
AGA Calendar, Calendar 2005
Tasks / Components / Responsible Party /Due Date
/ Due ToDevelop a 3-5 Year Enterprise Strategic Plan /
- Assessment
- Vision
- Mission
- Goals with measures
- Strategies
December 31 - final / Department of Management
Develop a 3-5 Year Agency Strategic Plan /
- Assessment
- Vision
- Mission and Core Functions
- Goals with Measures
- Strategies
December 31-final
(3-5 year cycle with annual updates if warranted) / Department of Management
Develop an Annual Agency Performance Plan /
- Core Functions
- Outcomes
- Performance Measures (outcome) and Targets
- Services, Products and/or Activities
- Performance Measures and Targets
- Strategies/Recommended Actions
August - enter performance information into I/3 / Department of Management
Develop an Annual Agency
Performance Report /
- Introduction
- Agency Overview
- Strategic Plan Results
- Performance Plan Results
1Acknowledgements
State of Iowa Performance Measures Guide
INTRODUCTION
Under the Accountable Government Act (AGA), Iowa is implementing a comprehensive and integrated performance governance system. The purpose of this system is to produce results valued by Iowans and do so in the most effective and efficient way we can.
There are five basic elements in our governance system. First, we plan what results should we achieve and how will we achieve them. Then, we use plans to allocate resources. With our plans as guides and resources as fuel, we produce results for Iowans. To know what we’re accomplishing and to guide improvement efforts, we measure and analyze our results. Finally, we communicate results and other important messages to internal and external audiences.
Performance measurement is not a new activity in Iowa. We are fortunate that we can build on a strong record of measuring the productivity and activity of state programs. During the 1990s, state agencies marked progress through a Progress Review system. That system did a good job of providing month-to-month management information by tracking expenditures, program outputs and some outcome measures for state programs. But input measures (number of employees and size of the budget) and output measures (volume of workload) did not paint a complete picture of state government’s successes and challenges, and it did little to gauge the impact government funded efforts have on Iowans. The questions “What is government doing to make life better, and is it working?” were not being answered. In answer to the demand for greater government accountability, Iowa state government began to report quarterly on a variety of indicators and goals related to six key policy areas: education, new economy, health, safe communities, environment, and accountable government.
With the AGA, there is increased emphasis on performance measures and the integration of performance measures into the basic elements of the governance system. Performance measures are a common thread, supporting both planning and budgeting processes. This guidebook was developed to aid your efforts to develop performance measures that are of value to your agency.
The first Chapter, Measurement Concepts, will help you:
- understand the types of measures used in Iowa state government;
- use performance measures as a tool; and
- avoid misuse of measures.
The Overview of the Measurement Process in Chapter 2 will help you:
- become familiar with one process that can be used to develop measures; and
- gain an understanding of the typical development of systems of measures.
In Chapter 3, Identifying, Locating, and Using Data, you will learn to:
- identify appropriate data for your measures;
- locate the data you have identified; and
- present the data once you have collected it.
Communicating Data is explained in Chapter 4 where you will learn how to:
- communicate data effectively; and
- select audiences and methods for communication.
The last four chapters will help you understand how to identify which performance measures are appropriate for the various components of the governance system. You will learn:
- which performance measures to use in strategic plans;
- which performance measures to use in performance plans;
- how your measures fit into the Iowa Excellence Self-assessment report; and
- which measures should be used in a service contract.
The Performance Measures Guidebook is a work in progress. This edition was designed to provide you with the basic information you will need to implement the measurement component of the AGA. We anticipate refining and adding to the fundamentals found in this edition. Your comments and suggestions will be welcome input into our improvement process.
I-1Introduction
State of Iowa Performance Measures Guide
CHAPTER 1
Measurement Concepts
INTRODUCTION
Why should we use performance measures? We use performance measures to make better, more informed decisions in managing our agencies. Measures are a tool used by managers as a means to run the agency. Just as saws, hammers and levels are a carpenter’s tools; performance measures are tools in a manager’s toolbox. Consequently, performance measures serve as a means to an end, not an end in itself.
When used effectively, measures help provide a powerful means of focus within the agency. When leadership is focused on reaching goals set for the agency, measurement is a means to assure the leaders the agency is on course to reach those goals. When leadership checks on the measures, the agency will pay attention to the measures. Consequently, “what gets measured gets done.” This helps to assure accountability in reaching agency or enterprise goals.
Family of Measures
Any master carpenter can tell you how to properly use a tool. A carpenter may also warn you to avoid misuse of a tool. The carpenter can also tell you when to use the tool to do the job. Like any tool, a manager can use, misuse, or not use performance measures. This chapter is designed to help you use performance measurement as a tool. You will also be given advice on how to avoid misuse of measures.
There are five specific types of measures that have been identified, defined and will be applied throughout Iowa state government. These types of measures are
(1) Input(2) Output(3) Efficiency(4) Quality(5) Outcome
The following pages provide a definition, description and examples of each of these five types of measures.
TYPES OF PERFORMANCE MEASURES
INPUT Measures
Input measures are used to monitor the amount of resources being used to develop, maintain, or deliver a product, activity or service.
OUTPUT Measures
Output measures are used to monitor “how much” was produced or provided. They provide a number indicating how many items,
referrals, actions, products, etc. were involved.
Summary of Input and Output Measures
Inputs and outputs have sometimes been called “activity” or “process” measures.
Input and output measures typically consist of a single numeric value for a given activity. See the examples below:
-76 clients served (output)
-10,236 documents processed (output)
-$149,367 of operating expenditures (input)
-153 full time employees (input)
-257 pavement miles resurfaced (output)
-3,791 licenses issued (output)
-1,860 hours worked (input)
-317 arrests (output)
-$ 6,852 of equipment costs (input)
EFFICIENCY Measures
Efficiency measures are used to monitor the relationship between the amount produced and the resources used. This means that efficiency measures are created by comparing
input and output, and are often expressed as a fraction or ratio.
The comparison of input to output (resources used / number produced) is often used to create unit cost. Measures generated by this method might look like:
-Cost per license issued- Cost per client served
-Cost per employee taught- Cost per document
-Cost per lane-mile paved
The comparison of output to input (number produced / resources used) is often used to create measures of productivity. The input considered for productivity measures is often time-related. Measures generated by this method might look like:
-Licenses processed per employee-hour
-Units produced per week
-Students taught per instructor
-Cases resolved per agent
-Calls handled per hour
QUALITY Measures
Quality measures are used to determine if we are meeting the expectations of our customers. These expectations can take many forms, including: timeliness, accuracy, meeting regulatory
requirements, courtesy, and meeting customer needs. The expectations can be identified as a result of internal or external feedback.
The comparison of outputs is often used to create measures of quality. It may be important to identify certain aspects (aspects / total outputs) about the services, products or activities produced by an organization that are important to its customers. This comparison of specific outputs to total outputs is used to create measures of accuracy, timeliness and to determine the extent regulatory requirements are met.
Evaluation of customer feedback data is used to build quality measures.
OUTCOME Measures
Outcome measures are used to determine the extent to which a core function, goal, activity, product, or service has impacted its intended audience. These measures are usually built around the specific purpose or result the function, goal,
service, product, or activity is intended to deliver or fulfill. An outcome measure should show us if we are achieving our mission or goals.
Summary of Efficiency, Quality and Outcome Measures
- Efficiency, quality and outcome measures consist of two or more variables.
- Efficiency, quality and outcome measures show a relationship or make a comparison. It is typical for these measures to be expressed as proportions, percentages, rates or ratios.
An outcome measure can express a result. It can be compared to actual outputs or to set targets. It can be compared to past trends or measures. An outcome could be compared to agency data with corresponding data from another agency.
WHAT IS A FAMILY OF PERFORMANCE MEASURES?
It is quite common to use a group of measures to gauge the organization’s efforts in providing services, generating products, or conducting activities. When evaluating core functions, it is normal to have a number of input and output measures. It is also normal to develop a set of efficiency, quality and outcome measures for each of your core functions. If you were to develop an inventory of your measures, and classify them by the five types of measures, you should find you have a family of measures.
Taken together, a family of measures should help the organization understand what is happening in regards to its services, products, and activities. These performance measures help the organization measure its processes. This set of measures can help the organization gauge its efficiency, determine progress in reaching desired results, and determines how well it is meeting the needs of its customers.
Has your organization inventoried its performance measures? Does your agency have all five types of measures represented for each of your core functions? If not, there may be some opportunities to improve your use of measurement tools.
MEASURE SELECTION
This guide contains a large variety of terms and phrases associated with measurement and the Accountable Government Act. The one clear message is that measurement will play a critical role in monitoring agency progress and performance. Even with the support provided by this guide and the guides created for the various components of the Accountable Government Act, the question most often asked is, “What kind of measure should I use?”
Ultimately, the selection of a measure depends on too many factors to offer the “perfect formula” for selecting measures. The following general guidelines are offered to assist in the selection of the types of measures that should be used. Guides developed for specific pieces of the Accountable Government Act should be used to fine-tune the measure selection process.
Levels of State Government
Measurement is important at all levels of government. From the individual work activities of employees to efforts at the statewide enterprise level, information is needed to monitor and improve the function of government.
At the very top, efforts are constantly underway to shape and monitor the direction and
policies that guide state government’s efforts to determine and meet the needs of Iowans. Measures used to track progress in these broad policy areas are called indicators. These policy areas are most likely associated with
Governor- or legislature-initiated efforts and may be components of the enterprise strategic plan. State agencies may contribute information for monitoring these policy areas, but according to this definition, measures developed for use by individual state agencies should not be called indicators.
Below the enterprise level, agencies are responsible for developing and reporting on aspects of both their strategic and performance plans. The results-based nature of monitoring the goals associated with these plans aligns with the use of outcome measures. The guides developed for each type of planning provide specifics on reporting requirements.
To provide the information needed to both support the monitoring and reporting of planning efforts and deliver input to decision making in the day-to-day operations of state government, agencies rely on performance measures. Discussed else where in this guide, these measures (input, output, efficiency, quality, and outcome) are used to provide feedback on the services, products, activities, processes and programs an agency must engage in.
The figure on the next page shows the types of measures often associated with various levels of state government.
Reporting FocusTypes of Measures
Statewide Policy AreasIndicators
Strategic PlansOutcomes
Performance PlansQuality & Efficiency
Day-to-DayQuality & Efficiency
OperationsInputs & Outputs
Measuring Work Performance
At the lower levels of government, there is a very real need to have useful information for making decisions. The work flow is often thought to involve three components: resources, work, and products.
The results-based focus present in government today provides for the addition of the desired purpose or outcome addressed by the work. The figure below aligns this new “model” of work with the types of measures usually associated with each component.
Work Flow
Component
Type(s) ofInputEfficiencyOutputOutcome
MeasureQuality
As stated earlier, there is no “magic wand” that allows for the picking of the perfect measure. The ability to develop good performance measures is something that improves with knowledge and experience.
QUALITY OF PERFORMANCE MEASURES
Evaluating the quality of performance measures is an important step in the process. As can be seen from the various types of measures available for use, it may be possible to generate many more measures than are necessary for an intended use. Being able to identify which measures to use from a larger group of potential measures can help focus measurement efforts.
One approach to weeding out undesirable measures is to create a set of rules or criteria to use. Based on the situation, criteria can be selected to evaluate the appropriateness of a set of measures. For the most part, criteria fall into two categories: technical and functional.
Technical Criteria
Validity: Does the measure accurately measure what it is supposed to?
Reliability: Does the measure produce consistently accurate data?
Validity and reliability are extremely important issues to maintain public credibility with the state’s performance measurement system. You need to ensure that the measures selected actually measure what they are intended to measure, do so in a manner that produces the same measurement year after year, and produces the measurement accurately.
An example of invalid measure would be measuring the graduation rate of a local high school as the percentage of seniors who attend the graduation ceremony rather than the percentage of seniors who received their high school diploma.
A good working definition of reliability is systems that produce controlled and verifiable data. An example of reliability would be multiple persons conducting a telephone survey coding open-ended responses in the same manner using standard criteria so similar responses are coded exactly the same way. Another example would be the hanging chads problem in Florida. Obviously, in this example, the voting results were not reliable since different polling places treated the hanging chads differently.
Development of a data dictionary (See Chapter 3 for the requirements of a data dictionary) will greatly assist agencies in ensuring validity and reliability by documenting exactly how the measure is defined and how it is calculated. If another person or agency cannot reproduce the same measure and measure values using your data dictionary, the measure may not be valid or reliable. When performance audits are performed, the agency performing the audit will test whether your measures are indeed valid and reliable.
Functional Criteria
Comparable: Is the measure defined, collected and calculated in such a way that it