CCA Review Team Manual:

A Guide for Reviewers and Team Leaders

June 2015 edition

Canadian Centre for Accreditation | Centre canadien de l’agrément

500A – 970 Lawrence Avenue West, Toronto (ON) M6A 3B6

416-239-2448 1-844-239-2448

|

1

Table of Contents

Introduction

The CCA Accreditation Program

CCA’s Approach to Accreditation

About This Manual

For More Information

The Reviewer Role

Who Is Involved in a CCA Review?

The CCA Review Team

CCA Staff

A Reviewer’s Responsibilities

A Team Leader’s Additional Responsibilities

Workload

What We Expect from CCA Reviewers

What Reviewers Can Expect from CCA

A Reviewer’s Guide to the Process

Overview

Planning and Preparation

Things to Remember

Before the Site Visit: Divide the Work

Things to Remember

Before the Site Visit: Assess Pre-Site Evidence on One’s Own

A Note on Strengths and Areas for Development

Things to Remember

Before the Site Visit: Make Team Decisions and Prepare the Visit

Team Decision Making

Site Visit Schedule Preparation

Things to Remember

The Site Visit

Interviews

A Word on the Chief Executive Interview/Meeting

On-Site File Review

Client Journey

The Verbal Wrap Up

Things to Remember

The Preliminary Report and End of Review

The Preliminary Report

Final Tasks

Things to Remember

Team Leader Guide

Introduction

Before the Site Visit: Team Work and Decision Making

Key Deliverables

Checklist

Before the Site Visit: Schedule and Prepare the Site Visit

Key Deliverables

Checklist

During the Site Visit

Key Deliverables

Checklist

After the Site Visit

Key Deliverables

Checklist

1

Introduction

The CCA Accreditation Program

The Canadian Centre for Accreditation (CCA) is a national non-profit bilingual accreditation body serving a wide range of community-based health and social service organizations. CCA’s accreditation program supports organizations to continually improve the quality and efficiency of the programs and services they provide.

CCA’s Approach to Accreditation

CCA offers a four-year accreditation that supports an organization’s quality management.

CCA looks at the whole organization, with an accreditation program that is modular. The accreditation program combines a set of Organizational Standards covering areas such as governance, organizational planning and performance, leadership and risk management, with program and service modules relevant to different services.

CCA standards reflect foundations of excellence at community-based health and social service organizations. These foundations include:

  • A focus on people and communities served, respecting their dignity and diversity
  • A strength-based approach to service delivery, recognizing the social and physical environments that impact people’s well-being and health
  • A commitment to equity and to actively addressing barriers to service
  • A drive to improve outcomes for people and communities served
  • Integration of continuous learning, improvement and innovation
  • Decision making that is evidence-based
  • Safe services and work settings
  • Efficient use of resources to achieve outcomes
  • Social and ethical responsibility
  • Accountability and transparency

Evidence that the organization has met CCA standards is gathered throughorganizational documents and narratives presented ahead of time; responses to CCA surveys of community partners and other stakeholders; and review team observations, interviews, and document, file and/or processreviews as part of a site visit.

Reviews are conducted by senior staff and board members recruited from partner organizations participating in CCA accreditation and trained as reviewers by CCA. Teams typically consist of a team leader and at least one other reviewer -- with some teams as large as five. The size and characteristics of the organization determines the size of the team and the length of the site visit.

About This Manual

This manual is designed to be thecentral referencefor CCA reviewers.

The term “reviewers” includes team leaders and youth reviewers. Where something pertains only to these specialized roles, the manual specifies it.

This manual provides:

  • An introduction to a CCA reviewer’s role and responsibilities
  • A reviewer’s guide to the process, organized chronologically according to a reviewer’s tasks at each stage of the review
  • A team leader’s guide to the additional deliverables and tasks that are part of leading a team

The manual references additional essential tools and resources made available to reviewers in the Reviewer’s Corner of GoCCA, the password-protected section of the CCA Web site at

For More Information

Canadian Centre for Accreditation | Centre canadien de l’agrément

500A – 970 Lawrence Avenue West, Toronto (ON) M6A 3B6

416-239-2448 1-844-239-2448

|

The Reviewer Role

Who Is Involved in a CCA Review?

The CCA Review Team

CCA reviews are conducted by reviewers recruited from organizations that participate in CCA accreditation. They areeither staff, members of the board or governing body, or volunteers that are trained by CCA.

Review teams consist of a team leader and at least one other reviewer -- with some teams as large as five. The team is adjusted based on the size and characteristics of the organization. Review teams are made up of individuals from a mix of community-based sectors, with the team leader generally having direct experience in the sector of the organization being reviewed.

Team leaders are reviewers that have had experience participating in several reviews, and have been selected and trained by CCA to lead teams.

CCA Staff

A CCA Accreditation Manager is assigned to each review. The manager, the CCA Accreditations Coordinator and CCA’s Office Managerprovide support to the review team and the organization during the review process.

A Reviewer’s Responsibilities

A reviewerisexpected to:

  • Participate actively throughout the process
  • Assess pre-site evidence (documents, narratives, surveys)before the site visit
  • Makeindividual assessments of whether the organization meets indicators using the GoCCA Web Tool
  • Participate inmaking team decisionson all indicators
  • Lead interviews and other site visit activities
  • Prepare and deliver the Verbal Wrap Up on site
  • Debrief with team members and support team decision making throughout

See A Reviewer’s Guide to the Processbelow for more on the tasks at each stage of a review.

A Team Leader’s Additional Responsibilities

The team leader assumesresponsibility for leading the team’s work between two and three months before the site visit. This responsibility endson delivery of the draft Preliminary Report to the CCA Accreditation Manager.

In addition to carrying the same responsibilities as other reviewers, the team leadermanages the work of the review. This involves scheduling meetings to keep track of the work of the team, helping to orient team members to their roles,and mentoring less experienced team members.

More specifically, a team leader is expected to:

  • Provide leadership to the review team throughout the process
  • Liaise with the CCA Accreditation Manager
  • Lead the assignment of standards/indicators to team members for their individual assessments prior to the site visit (teammembers should expect to takeresponsibility for specific on-site interviews and observations linked to these standards/indicators, and these aspects of the Verbal Wrap Up)
  • Facilitate the reaching of a team decision on all indicators in the review
  • Help to orient team members to their roles and mentor less experienced reviewers as required
  • Plan for the site visit in consultation with the organization and according to CCA requirements, and in particular to develop a detailed site visit schedule
  • Oversee the successful preparation of the team and completion of all site visit activities – including the Verbal Wrap Up
  • Complete tasks and writing up of findings for production of thewritten Preliminary Report

Workload

A CCA review represents a substantial workload.

The team leader spendsbetween 25 and 30 hours pre-site visit coordinating various aspects of the review in consultation with the CCA Accreditation Manager, the organization, and the other reviewers. This includes time reviewing pre-site evidence.

The other reviewers on the team need between 20 and 25hours before the site visit to review pre-site documentation, assess indicators and record assessments and comments in the GoCCA Web Tool.

During the site visit, the team should expect busy, full days taken up with interviewing individuals and groups, making observations, reviewing files, checking on-site documents and visiting the organization’s site or sites. Throughout the process reviewers are expected to use the GoCCA Web Tool to continue to document their individual decisions and keep notes that will facilitate team decision making. Time between interviews and other activities as well as late afternoon and early evening hours are set aside for team debriefings, team decision making and preparation of the Verbal Wrap Up.

During the site visit or immediately after, the team leader may need between two and five hours to review assessment and edit findings so the CCA manager can prepare the written Preliminary Report.

What We Expect from CCA Reviewers

CCA expects reviewers to demonstrate:

  • Respect for the confidentiality and privacy of the organization and its clients
  • Experience and knowledge that is well-grounded in the realities of organization being reviewed
  • Curiosity and willingness to learn
  • Understanding of not-for-profit governanceand a client-centred community-based approach to program delivery
  • Good written and verbal communication skills
  • Interviewing skills
  • Familiarity and ease with using computers and the Internet
  • The ability to analyze and synthesize information
  • The capacity for making sound judgments
  • Teamwork skills
  • Good time management, flexibility, and problem-solving skills
  • Integrity and trustworthiness
  • Support for CCA including a willingness to sharewith CCA insights and observations to contribute to the ongoing improvement of the CCA program

CCA issues each reviewer with a username and password that provide access to the thepassword-protected GoCCA section of the CCA Web site and to the GoCCA Online Accreditation Tool to complete their tasks on the review to which they are assigned.

Reviewersmust have access to their own or their organization’s portable computer (PC or Mac) throughout the review period, beginning two months before the site visit. The computer must have:

  • Capability to connect to the Internet
  • A Web browser updated to a recent version
  • Microsoft Word, Excel and PowerPoint (version 2003 or later)
  • Adobe Acrobat Reader
  • Capability to download and install software (for example, GoToMeeting and GoToWebinar for trainings and online meetings)

What Reviewers Can Expect from CCA

CCA is committed to providing reviewers with:

  • Orientation to the structure and content of the CCA accreditationprogram
  • Training to carry out duties as a reviewer
  • Logisticalassistance in preparing for the review
  • Support at all phases of the review, including consultation on interpreting CCA’s standards and requirements
  • A readiness to accept reviewers’ feedback and to use it to improve the process
  • Appreciation and recognition for their participation in the CCA accreditation program

1

A Reviewer’s Guide to the Process

Overview

This is the timeline of a review from the perspective of a reviewer.

Planning and preparation starts about six months before the organization’s site visit, when we recruit the team. Between two and three months before the site visit is when the work of the team really begins in earnest. By the end of the site visit, the level of involvement tapers off.

1

Planning and Preparation

Planning and preparationfor a review starts about six months before the organization’s site visit, when we recruit reviewers onto a team. CCA builds the review team (including a designated team leader) based on factors such as sector-related experience and knowledge, CCA-related experience, availability, and the mix of skills and backgrounds needed on the team.

By email, we then do a conflict of interest check with team members and with the organization. CCA defines a conflict of interest as “any obligation or interest that materially affects or interfereswith the judgment and/or objectivity of a review team member in the performance of his/herduties for the organization being reviewed.”A past or current working relationship in and of itself does not necessarily constitute a conflict. Reviewers are encouraged to ask CCA if in doubt. If there is a conflict of interest declared either by the organization or a team member, the CCA selects and assigns another reviewer.

Once conflict of interest clears, we connect team members with one another and with the review’s designated CCA Accreditation Manager, and sendinstructions on booking travel and accommodations.

Consult the Travel and Expense Policy for Site Visits located in the Reviewer’s Corner section of GoCCA for instructions on booking travel and accommodations, and claiming expenses.

Things to Remember

  • If you have not logged in to GoCCA in a while, test your username and password now to make sure you can get in.
  • Look at the calendar. Plan to book off at least 20 hours to complete your pre-site tasks starting two months before the site visit.
  • Make sure the information CCA has on file for you is up to date. Check preferred contact information, biography, and dietary and other special requirements.

Before the Site Visit: Divide the Work

An organization’s pre-site evidence in the GoCCA Web Tool is ready for the review team between two and three months before the site visit.

The immediate focus of the review team’s work is in assessing all the indicators that are dependent on pre-site documents, narratives and CCA’s surveys of the organization’s stakeholders. A reviewer first workson his/her own in the GoCCA Web Tool to assess a set of assigned pre-site indicators.

The indicators are assigned during a team kickoff phone or Web meeting called by the team leader before or around that two-month point. Each member of the team (including the team leader) is assigned indicators based on factors including experience, workload balance and the goal of having two sets of eyes on the indicators. A reviewer’s assigned indicators become areas on which the reviewer will focus throughout the review. For example, if a reviewer was responsible for assessing human resource indicators, s/he should also expect to take the lead on those questions in on-site staff interviews and in the Verbal Wrap Up.

During the meeting, the team also reviews travel plans and other practical arrangements, goes through a refresher on how to enter decisions and comments in the GoCCA Web Tool, divides up the work, and sets dates for follow-up check ins and meetings.

Consult the Team Kickoff Meeting Agenda located in the Reviewer’s Corner section of GoCCA.

It is also at the two-month point that the team leader begins working with the first draft of the site visit schedule provided by the CCA Accreditation Manager.

See the Team Leader Guide below for details.

Things to Remember

What you do now and how you do it contributes to the final product and will make the process easier. So, let’s understand the end game.

  • By the end of the site visit, there must be a team assessment of Met or Not Met on each and every indicator.
  • The site visit closes with a Verbal Wrap Up in which all team members participate to present the preliminary results of the review. It has a set format that includes highlighting for each module particular strengths, any areas where there are unmet standards that will have to be addressed for accreditation, and other areas that are opportunities for further quality improvement.
  • After the site visit, a Preliminary Accreditation Report is prepared. It has a module-by-module structure consistent with the Verbal Wrap Up.

Refer to the Preliminary Report Template and the Verbal Wrap Up Template located in the Reviewer’s Corner section of GoCCA.

Before the Site Visit: AssessPre-Site Evidenceon One’s Own

At this stage, a reviewer’s job is to make an individual assessment on all his/her assigned indicators that aredependent on pre-site documents, narratives and CCA’s surveys of the organization’s stakeholders.

The reviewer logsa decision of Met, Not Met or Don’t Know for each of these indicators in the GoCCA Web Tool, adding comments and questions on each in the text box provided. (The default is No Decision; leave No Decision only if no assessment of the indicator has taken place at all yet.) A brief note on why the indicator is assessed as Met, Not Met or Don’t Know is essential to the team debriefing and decision-making process. Each reviewer’s decisions and notes on an indicator are visible to the rest of the team in the GoCCA Web Tool.