PWGSC Performance Monitoring and Reporting Standard

PWGSC Performance Monitoring and Reporting Standard

EDRM # 1129286

National Project Management System

1

PWGSC Performance Monitoring and Reporting Standard

Table of Contents

1. Introduction

2. Project Performance Performance Monitoring and Reporting Standard

2.1 Overview

2.2 Context

2.3 Purpose of This Document

2.4 Audience

2.5 Model

2.6 Process

2.7 Standard Components

Annex A - Performance Management Tools

1. Dashboard

2. Status Reports

3. Independent Review / Monthly Process Delivery Management

5. Performance Management Tools Output

6. Definitions of Key Performance Indicators

6.1 Mandatory Performance Performance Monitoring and Reporting Standard KPI Definitions (Tier-1)

6.2 Examples of Selected Additional Project Management Standard KPI Definitions (Tier-2)

6.3 Notes on Sub-KPIs and Definitions

Annex B –Status Reports to TBS – Standardized Content/Look and Feel

Annex C – Monitoring and Control Function

1. Introduction

A Performance Monitoring and Reporting Standard is used to quantify the efficiency and effectiveness of an organization’s actions. An effective Standard acquires, sorts, analyzes, interprets and disseminates significant data to enable management to make informed decisions and to take necessary actions. A Standard includes sufficient processes, measures and infrastructure to enable a project to produce the right data and disseminate it to those with a need to know.

A project or an organization with a “value” mindset starts by defining the Key Performance Indicators which they require. Key Performance Indicators (KPIs) represent a set of measures focusing on those aspects of project performance that are the most critical for the current and future success. KPIs encompass the various drivers that link the project business case objectives to accountabilities within the project and to the outcomes that are expected.

The value of an effective Performance Monitoring and Reporting Standard is that it enables the project to obtain Senior Management Buy-In as the project competes for the financial and human resources required for executing the organization’s mandate. KPIs link the original project Business Case objectives to Measurable and Ongoing Project Value.

The starting point is to focus on a solid business case justification that clearly articulates a plan for delivering project success and therefore project business value. A successful business case will be the determining factor for getting the project approved and obtaining the required funding, resources, and time commitment from the appropriate governance body. The project needs to identify and execute against business value drivers to show which financial drivers, business processes, and operational metrics will be used to drive the project results. From the point of identifying the project value drivers, key project performance metrics are established as a foundation for measuring project results.

2. Project Performance Monitoring and Reporting Standard

2.1 Overview

Outcome Management is Treasury Board’s (TB) recommended strategic approach to performance management. Effective Outcome Management ensures that “initiatives are designed around departmental and government outcomes and that the intended outcomes are achieved. As Project Management deals with the processes necessary to deliver a capability and/or product within a pre-established time frame and budget, Outcome Management focuses on the processes needed to realize the benefits of each initiative”.

The Project Performance Monitoring and Reporting Standard is intended to span the project lifecycle in order to capture the project performance components consistently. The Standard deals with performance from the early identification stage (business case), through the delivery stage and concludes with project closeout. Governance and Stakeholder Management are also essential components to the effectiveness and success of the Standard. The outputs of the Standard provide management with the means to monitor, measure and control progress towards achieving the projects intended outcomes. These outputs are provided through the governance structure defined in NPMS and the Project Charter and to the stakeholders defined in the Communications Action Plan.

The five key components of the proposed NPMS Project Performance Management Model are as follows:

  • The Intended Outcomes (derived from the Business Case)
  • KPIs
  • Performance Tracking
  • Governance
  • Stakeholder Management

2.2 Context

The most effective Project Performance models view each project as part of larger organizational, project portfolio and investment strategy frameworks. An effective performance tool provides more granular and less granular views or lenses, depending on the needs of the audience.

2.3 Purpose of This Document

The intended purpose of this document is to provide a high-level Standard for performance management and reporting in PWGSC.

It is intended to form the basis of a more detailed future document. The current document identifies components and standard KPIs to be used by projects across the Department and to external agencies for reporting purposes. See Annex B – Treasury Board Status Reports.

2.4 Audience

This document is designed to be used by managers responsible for implementing project management policies, procedures, roadmaps, tools and templates within Public Works Branches as well as by individuals providing oversight at the Departmental level.

2.5 Model

2.6 Process

The performance management process begins by determining the project’s intended outcomes / results. The team then defines specific outputs that will deliver the intended outcomes / results of the project. Value drivers, extracted from the baseline, are used as inputs to develop key performance indicators. These measure progress towards targets which are aligned with the defined outputs and consequently the achievement of the intended outcomes / results.

Value Drivers include financial drivers, business processes, and operational metrics. Formative and EndState improvements to these value drivers are what justifies the Business Case and what is used to drive the project results.

(Note: some of the intended outcomes will not be measureable until the project is in full, steady-state operation).

In PWGSC, four key project performance indicators are standard: on time, on budget, on scope and overall project status. Additional selected KPIs may be reported on as determined by Branch practice or based on the project’s complexity, risk and purpose (see Annex A).

The project team uses reporting tools to produce information that clearly illustrates the current and forecast status of the KPIs. The team also provides additional information with respect to milestones and deliverables. The project communicates this information to the project governance structure through the channels described in the communications plan and project governance to key stakeholders and to decision makers. In return, through the communications and reporting channels, the KPIs generate feedback that the project team uses to refine the strategies, plans and the actions required to deliver the project and the intended results. The process is refined and updated in an iterative fashion as required during the project lifecycle.

2.7Standard Components

The key components are described in more detail as follows:

2.6.1 Intended Outcomes (from the Business Case)

The Business Case is a description of the reasons for the project and the justifications for undertaking the project based on an analysis of the costs, risks and intended outcomes.The Business Case identifies the benefits the intended solution is expected to bring to the organization. These benefits can be short-term or long-term, direct or strategic. The Business Case outlines and often quantifies these benefits to be tracked and managed during the project effort, once approved.

The Business Case represents what was promised at the outset. Changes to this baseline must be approved in accordance with the NPMS governance, and with the detailed project governance and change management processes defined in the Project Charter and Project Management Plan.

2.6.2 NPMS Key Performance Indicators

In reporting to Senior Strategic Management regarding the NPMS system, the following Key Performance Indicators will be used:

NPM Design – Is the NPMS Design adequate and does it meet the standard for the Treasury Policy on the Management of Projects

NPMS Application – How many projects are following NPMS?

Project Performance – What percentage of projects following NPMS are on track? And what percentage of projects not following NPMS are on track?

2.6.3 Project Key Performance Indicators (KPIs)

Projects follow a defined process to identify which KPIs are required in addition to those that are mandatory. KPIs are created using:

  • Input from: Value Drivers
  • KPIs are based on Project Management Knowledge Areas
  • Cover short and long term, direct and strategic benefits

There is a Standard Set of KPIs common to all projects across the Department. These are referred to as tier-1 core indicators (e.g. Scope, Cost, Schedule, Overall Status). Additional indicators may be provided as Tier 2 indicators depending on Branch practice or the intended purpose of the product or result of the project.

Mandatory NPMS Project KPIs – Colour-coded Indicators
Measure Project Performance
  • Time
  • Budget
  • Scope (with relation to time and budget)
  • Overall Status

Selected Additional Project Management KPIs (Tier 2)
Provided depending on Project Complexity, Risk and Purpose
Normally reported in text; not as colour-coded indicators
Maturity measures
  • Client Satisfaction
  • Other Knowledge Areas such Procurement, Risk etc

Please see Annex A – Performance Management Tools for sample KPIs and indicator definitions

2.6.4 Performance Tracking

Branches provide KPI data to decision makers and stakeholders. This occurs at the NPMS control points and via project specific governance processes.

Reports are provided to stakeholders with indicators that measure performance against the common mandatory KPIs as well as against the specific additional indicators defined for the project based on its Outcomes Management needs. Currently, each Branch uses its own reporting tool; in future, a common on-line tool may be adopted. This will require further analysis and development. PMAC may be a forum for initial requirements gathering.

2.6.5 Governance

The Performance Monitoring and Reporting Standard Governance aligns to the approved NPMS control points. The Standard is in turn:

  • Integrated with the specific project governance structure, roles and responsibilities defined in the Project Plan and Project Charter or the OGD Agreement.
  • The detailed governance process defines how the project reports performance to governance bodies, including tool, format, and frequency.
  • Project governance includes an identified escalation model
2.6.6 Stakeholder Management

Projects need to maintain a continuous connection with the stakeholders in order to inform them as to how the project is doing. The key is to recognize that different stakeholders may have different perspectives and different interests. Stakeholder Management integrates/ align these perspectives and interests with the effort to achieve the intended outcomes and benefits.

Details Developed in Appropriate Knowledge Areas

Details regarding performance management processes and tools will be incorporated into relevant NPMS Business Projects-IT-Enabled knowledge areas, including the quality management, risk management, and communications management areas.

Annex A - Performance Management Tools

The following are suggested options for Performance Management Tools:

1. Dashboard

  • Standard Enterprise tool – common feel and look for all branches/sector: facilitates a consistent view of projects, their progress and performance; information can be easily rolled up to get a higher level view (branch-level, departmental-level).
  • Project-style dashboard: a status report with pre-defined elements; higher level information, focusing on specific KPIs, and has colour coded status for the indicators to report performance, highlight challenges, and seek assistance.
  • Defined Frequency
  • Defined Access Controls

2. Status Reports

  • Standard set of indicators and items
  • Defined recipient list and frequency

3. Independent Review / Monthly Process Delivery Management

The NPMS Performance Monitoring and Reporting Standard includes independent review components. Internally, in the Real Property Branch, independent project review is provided by the Project Review Advisory Committee (PRAC) and by the National Project Review Advisory Committee (NPRAC); in ITSB, independent review is provided through the Monthly Process Delivery Management Process, which in turn is linked to the ITSB Project Oversight Committee. Externally, Independent Review is also a key component of Treasury Board’s project review and oversight process. TBS will provide oversight to projects exceeding the Department’s delegated authority and to other projects as and when required by TBS. In support of TB’s review process, Departmental Projects will be reviewed by independent analysts certified by TB. These reviews may occur at any of the seven gates defined by TBS, depending on the proven project management capacity of the NPMS stream:

  • Strategic and Concept
  • Approach
  • Business Case and Readiness
  • Project Charter and PMP
  • Detailed Plan
  • Construction/Deployment
  • Post-Implementation

A full Independent Review / Monthly Process Delivery Management process is resource intensive, as it needs to be done at regular checkpoints. Consequently, third-party review is not price justified for all “Lite” Business Projects-IT-Enabled.

The key points on Monthly Process Delivery Management:

  • Investigative Performance Monitoring function, based on best practices
  • May complement other performance reporting tools/mechanisms by performing a validation/verification role
  • Performed by an independent party – may follow the general guidelines for the Treasury Board’s Independent Review Program
  • Leverages Key Performance Indicators to gauge project progress and assess its performance
  • Considered a Preventative function – performed on a monthly basis to identify potential areas for improvement and record the corresponding action plans
  • Continuously monitors project Lessons Learned
  • Helpful for the organization to identify project inter-dependencies and getting a portfolio view of strategically aligned projects
  • Positioned as a constructive tool to help the projects overcome their challenges and meet their goals vs. being seen as an additional administrative burden
  • Requires compliance to ensure consistent implementation of project management practices
  • Produces a report that is presented to an executive-level governance committee, so appropriate action may be taken to help the project resolve its challenges

5. Performance Management Tools Output

Outputs from the project management Standard will include as a minimum:

  • A comprehensive picture of the status of the projects in the portfolio
  • Recommended courses of action
  • Escalation of issues
  • Lessons Learned
  • Input into the Continuous Improvement process
  • Acknowledgement by senior management

6. Definitions of Key Performance Indicators

Projects are assessed against several Key Performance Indicators (KPIs) representing key project management knowledge areas, used to establish the final benchmarks for measurement and reporting.

6.1 Mandatory Performance Monitoring and Reporting Standard KPI Definitions (Tier-1)

6.1.1 On Time

This refers to the timely completion of project based on NPMS phases and control point deliverables. The baseline schedule is as defined in the Project Plan and is the schedule last approved (ex: PPA/EPA OGD Agreement). Performance is based on any variances from the last approved schedule in terms of the Critical Path.

  • Green: Within baseline last approved schedule
  • Yellow: Behind baseline schedule – No major Impact on project
  • (Does not affect project Critical Path)
  • Red: Behind baseline schedule – Major Impact on project
  • (Does affect any of the critical path, scope objectives, budget objectives, or operational requirements).
6.1.2 On Budget

The evaluation is based on the overall project budget. A project is considered to be “On Budget” if the final expenditure is within the approved contingency at the identified phase control point. For OGD projects the overall budget is the total amount in the signed Agreement (MOU, LOI, SA etc).

  • Green: On Budget (Forecasted expenditure follows the budget as planned).
  • Yellow: Forecasted total project expenditure is not exceeding the approved funding but requires the use of Risk allowance.
  • Red: Forecasted total project expenditure exceed approved Budget

N.B. For OGD Project:

PPA = Amount for project initiation in Agreement with OGD (MOU, LOI, SA etc).

EPA = Amount for project implementation in Agreement with OGD (MOU, LOI, SA etc).

6.1.3 On Scope

Evaluation is based on the overall project scope as established at the most recent approval. Deviation from scope reflects approved changes as well as failure to meet the requirement meet the baselined scope (scope statement in the Project Charter).

  • Green: No change or impact on project. (Change within the budget & baseline schedule).
  • Yellow: Minor change - little impact on the project. (Change impacts budget or schedule but is still within contingency fund and does not affect the Critical Path Schedule)
  • Red: Important change - significant impact on the project. (Major effect on budget & Contingency Fund and/or the Critical Path Schedule is at Risk).
6.1.4 Overall Project Status

The overall project status is rated as either Green, Yellow, or Red, based upon the evaluations of the project indicators, of “On time/ On Budget and On Scope” as follows:

  • Green: Project assessed as being “Green” for each of the On Time, On Budget and On Scope indicators.
  • Yellow: Project assessed as being “Yellow” if at least one of the On Time, On Budget and On Scope indicators are yellow, and remaining indicators are either green or yellow.
  • Red: Project assessed as being “Red” if at least one of the On Time, On Budget and On Scope indicators is Red.

6.2 Examples of Selected Additional Project Management Standard KPI Definitions (Tier-2)

6.2.1 Risk:

Risk is uncertainty. A risk performance indicator measures the current ability of the project to manage risks and issues that may prevent it from reaching its objectives. This may cover risks not related to Time, Budget, or Scope; in other words, dependency risks; funding risks; political risks; technical risks and so on.