Evaluation Plan

FY2012-13

Tool/Measure

/

Purpose

/

When

/

Who Completes (or Retrieves)

/

Where to Access

/

Reports to Use

/

Implementation Fidelity – Completed with the help of your school wide facilitator (SWF)

Self-Assessment Survey (SAS) and School Safety Survey (SSS) / Identify status, needs and change over time / Prior to initial 3 day SWPBS training, then Annually / All building level staff complete and review; Anonymous data are retrieved by PA PBS evaluators via secure website / www.pbisassessment.org / Individual Item Scores: Current status (in place, partially in place, and not in place) and improvement priority (high, medium, and low) percentages for each item within each system.
Team Implementation Checklist (TIC) / Implementation Status and Technical Assistance Needed / For schools in Universal Phase I and II: October, December, March / Team completes checklist and comes to consensus. SWF enters on PBIS Assessment. / www.pbisassessment.org / Implementation by Feature: This report shows the percentage implemented and partially implementedfor each feature.
Benchmarks of Quality (BoQ) / Implementation status and change over time / Annually / Building Coach and Team Members. District Coach submits final BoQ to SWF who then enters on PBS Surveys website; PA PBS evaluators retrieve via secure website / www.pbisassessment.org / Elements Report: Percent implemented for each BoQ element in a bar chart and accompanying data table.
School-wide Evaluation Tool (SET) (by appointment after achieving a 70 or higher on the BoQ) / Assess and evaluate the critical features of school-wide positive behavior support / Minimally every 3-5 years based on need for additional data and/or recognition purposes / Trained external evaluator completes and reviews with team; PA PBS evaluators retrieve via secure website
(by appointment after achieving a 70 or higher on the BoQ) / www.pbisassessment.org / SET Summary Score Report: Used (a) to determine annual goals for SWPBS (b) to evaluate on-going efforts (c) design and revise procedures as needed, (d) to compare annual accomplishments

Outcomes – data gathered in summer for program evaluator with help from your SWF and school point of contact

School Safety Survey (SSS)

/

Identify safety status, needs and change over time

/

Annually

/

All building level staff complete and review; Anonymous data are retrieved by PA PBS evaluators via secure website

/ www.pbisassessment.org /

Office Discipline Referrals (ODR)

/ Evaluate impact on rates of problem behavior / Monthly (by team), Quarterly (shared with staff), Annually shared with PAPBS evaluators / Enter in SWIS or other data system; If schools use SWIS, their annual SWIS data are automatically accessed by PAPBS evaluators via secure website /

www.swis.org ; or School district and building level database

/

“Big Five Reports”:

·  Avg Referrals per Day per Month

·  Referrals by Problem Behavior

·  Referrals by Location

·  Referrals by Time

·  Referrals by Student

Pennsylvania School System of Assessment (PSSA) / Evaluate impact on academic performance / Annually / PAPBS evaluators retrieve via public access / http://paayp.emetric.net/
Pennsylvania Value Added Assessment System (PVAAS) / Evaluate impact on annual academic growth / Annually / PAPBS evaluators retrieve via public access / https://pvaas.sas.com/evaas/welcome.jsf
Safe Schools / Evaluate impact on violence and weapons possession on school property / Annually / PAPBS evaluators retrieve via public access / www.safeschools.state.pa.us
School Profile
·  Total Students
·  Student attendance rate
·  Staff attendance rate
·  LRE per 3 reporting categories
·  Referrals for special education eligibility determination
·  Newly identified special education identifications
·  OSS/ISS
·  Out-of-school placements for students
·  Out-of-school placements for students with ED
·  Collaborating MH agency
·  Number of students with IEPs / Used to identify changes over time with implementation / Annually / School/District Point of Contact (POC) submit to PAPBS evaluators / www.papbs.org


Data Collection Schedule

Measure / Pre / Phase One / Phase Two / Phase Three
Fall / Winter / Spring / Summer / Fall / Winter / Spring / Summer / Fall / Winter / Spring / Summer
SAS/SSS1 / X / X / X / X
TIC1 / X / X / X / X / X / X
BoQ1 / X / X / X
SET2
SSS1 / X / X / X / X
ODR4 / X / X / X
PSSA3 / X / X / X / X
PVAAS3 / X / X / X / X
Safe Schools3 / X / X / X / X
School Profile4 / X / X / X / X

Notes regarding PAPBS evaluators’ access to data. 1 = data completed via PBISAssessment website and automatically obtained by PAPBS evaluators via PBSEval.org secure website; 2 = will be administered minimally once every three to five years, or more frequently based on need for additional data and/or recognition purposes; 3 = data obtained by PA PBS evaluators via public access; 4 = school must supply to PAPBS evaluators


Data Measures

·  SAS = Effective Behavior Support: Self-Assessment Survey (SAS; Sugai, Horner, & Todd, 2003, 2009)

·  TIC = Team Implementation Checklist (TIC; Sugai, Horrner, Lewis-Palmer, Rossetto Kickey, 2001, 2011)

·  BoQ = School-wide Benchmarks of Quality (BoQ; Kincaid, Childs, & George, 2010)

·  SET = School-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2005)

·  SSS = School Safety Survey (SSS; Sprague, Colvin, & Irvin, 2002)

·  ODR = Collected via SWIS (School-Wide Information System; Educational and Community Supports, 2010)

·  PSSA = Pennsylvania School System of Assessment

·  PVAAS = Pennsylvania Value Added Assessment System

·  Safe Schools = Pennsylvania Safe Schools Online Database (https://www.safeschools.state.pa.us/)

·  School Profile = Data specifically requested from schools including:

o  Total number of students

o  Student Attendance Rate

o  Staff Attendance Rate

o  LRE per 3 reporting categories

o  Referrals for Special Education Eligibility Determination

o  Newly Identified Special Education Identifications

o  OSS / ISS

o  Out-of-School Placements for Students

o  Out-of-School Placements for Students with EB

o  Collaborating MH agency

o  ODRs – if not using SWIS; otherwise, these data provided via SWIS

o  Number of Students with IEPs

08.23.2012 FINAL

Page 1