S3TAIR ProjectReplicationSchool Protocol for Review of Current Practice Status Page 1 of 15

S3TAIR (Supporting Successful Strategies To Achieve Improved Results) ReplicationSchoolProtocol for Review of Current Practice Status

This document provides an opportunity for S3TAIR Project Replication Schools to review the current status of the practice they will address through the Project. Based on the protocol used by Project site visit teams in determining the validity of nominated practices, the Replication School Protocol presents descriptions of elements of practices that have been determined through research to be correlated with successful implementation and maintenance of evidence based effective practices and in turn in positive outcomes for students with disabilities.

Note that the elements of the protocol are not specific to the pedagogical elements of the practice (for example, there are no descriptions of the specific elements of effective literacy instruction), but rather are focused on effective practices in IMPLEMENTATION. Effective use of this tool will by necessity be a reflective process, and will involve a team of people who are familiar with the several levels of practice components in every school setting.

As the Protocol is completed, the depth of description for current status will help drive your understanding of the relevant first and subsequent steps in implementation. Your Regional Facilitator will guide you in using this tool to explore and document your baseline status for these elements before you initiate your implementation efforts with your PartnerMentorSchool. In order to provide a baseline of practice that can easily be compared against progress toward your goal of full implementation, please use the following rubric along with descriptive comments to describe practice status:

Not Evident / Emerging / Operational / Highly Functional
1 / 2 / 3 / 4
Data COLLECTION for Analysis to Inform Practice - The district/building has:
Desired Status / Where are we now? / Comments
Developed and implemented rigorous practice for data collection, including centralized data manager or team (consider the following points in your reflection).
  • Has the school/district identified a specific set of data that need to be collected to determine if instruction or behavior management are successful?
  • Is there a clearly described set of tools or practices that are to be used to collect the specified data?
  • Is there a specified time frame for when data will be collected by educators?
  • Is there a strategy in place for gathering data collected by educators to be analyzed at classroom, grade, school, and other levels?
  • Is there an educator or group of educators who are designated as managers of the data and who have been provided training in organization, analysis, and display of data so that it can be understood by all who need to use it?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Collected data collection from multiple sources, including but not limited to state, commercially available, internal/external formative and summative assessments, diagnostics and progress monitoring (consider the following points in your reflection).
  • Is the impact of instruction or behavior management analyzed on the basis of at least three separate sources of data?
  • Are the data sources used to analyze the impact of instruction or behavior management providing information that really tells what is going on?
  • Can the data collected from different sources be grouped together for analyzing how instruction or behavior management is working across groups?
  • Can the data collected from different sources be broken down to the individual student level to be used to monitor progress and make decisions about day to day instruction?
  • Are the tools used to collect data ones that have been research tested and shown to actually test the concepts they claim to test?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Has the school/district developed and distributed written information about the process by which data should be collected? (Consider the following points in your reflection).
  • On how the data should be organized once collected?
  • Has professional development been provided to all staff who must used the strategies for data collection and analysis?
  • Is technical assistance readily available to support accurate data collection and analysis?
  • Is there a process in place to check on the consistency with which agreed-upon data are collected by the relevant educators?
  • Is there a process in place to check on the accuracy of the data being collected?
  • Is there a process in place for provision of support to correct any issues in data collection when they are identified?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Look For Check List
Database/reporting is comprehensive and accessible to all stakeholders.
Acceptable turn around time for internal formative assessments & progress monitoring.
PD (internal/external/district wide) on data analysis and decision making - for example:
  • Understanding what data represent
  • Using data to differentiate or form intervention groups,
  • Strategies for progress monitoring.
Evidence of scheduled times dedicated to data analysis for decision making by instructional team, grade level, content area, et cetera / Evidence of data analysis and resulting decisions that led to changes in practice and improved student outcomes
Written description of process used in decision making (e.g., specific data sources analyzed, strategies for identifying hypotheses and interventions from data, structures to support implementation, analysis of impact, and adjustment to interventions as needed)
Evidence that analysis of data has resulted in changes in core program or supplements, addition of service providers (TAs, Literacy Specialist, AIS), et cetera
Evidence of monitoring of decision making process, with identification of areas in need of improvement and provision of PD/TA in the areas of need
ANALYSIS of Data to Inform Decisions – The district/building:
Desired Status / Where are we now? / Comments
Provides on-going staff development of skills for use of data analysis in decision-making.(Consider the following points in your reflection).
  • Has the school/district identified the skills educators need to successfully analyze collected data? To make decisions based on the analyses?
  • Has the school/district developed written materials describing the necessary skills and providing information about how to do them?
  • Has the school/district regularly provided professional development to staff who have the responsibility for analyzing and using the analysis in making decisions based on the analyses?
  • Does the school/district have a process in place to provide as-needed training to new hires to ensure their ability to analyze data and use the results to guide decision making?
  • Does the school/district have a well-established process for providing technical assistance to educators who are struggling with data analysis and/or decision making based on analyzed data?
  • Does the school/district have a practice in place to review the accuracy of data analysis and the instructional decisions made on the basis of the analysis?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Makes data driven decision a priority by providing time and support.(Consider the following points in your reflection).
  • Has the school/district scheduled creatively to ensure that educators have time to collect data and meet in relevant teams to analyze and make decisions about instruction and/or behavior management based on the analysis?
  • Has the school/district provided necessary materials to facilitate data analysis (e.g., software and other supplies for analysis and display of results?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Analyzes data at the building & classroom level including individual and disaggregated group performance on group assessments and day-to-day classroom performance data.(Consider the following points in your reflection).
  • Are data analyzed to determine success of instruction and/or intervention:
  • At the individual student level?
  • A at the level of small instruction or intervention groups?
  • At the classroom level/At the grade level?
  • At the level of specific intervention groups (e.g., all students who are limited English proficient, or who receive special education intervention)?
  • At the school level?At the district level?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Ensures building wide and individual use of analyzed data to make decisions about instruction at classroom and system level.(Consider the following points in your reflection).
  • Is there a process in place to check and support the analysis of data and use of the analysis to make instructional decisions?
  • If issues are discovered in the consistency or accuracy of data use, is there a process to provide support for correction?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Ensures that administrators and instructional staff collaboratively consider summative data annually for decisions about programmatic changes.(Consider the following points in your reflection).
  • Is there a defined practice of bringing staff together for reviewing the analyzed data on an annual basis?
  • Does the district/school use the analysis of all the year’s data to make decisions about how successful school or system-wide programs have been?
  • Are decisions about modifying programs or practices at the system level made annually based on this analysis of the year’s data?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Look For’s
√Evidence that administrators/instructional staff can accurately describe the critical elements of the practice as validated through research, and discuss how they should be and are implemented in the school
√Documented program goals and steps in implementation of the practice (i.e. manual describing implementation, classroom charts of practice steps, checklist of steps in implementation)
√Documented structures for checking fidelity of implementation (e.g., written guidelines of what to look for, how often to look, how to evaluate what is observed to determine fidelity of implementation)
√Evidence/documentation of validation procedures used consistently across all implementers using program guidelines.
√Evidence results of fidelity observation are used to monitor and adjust practice implementation to recover from drift away from fidelity
Fidelity of Implementation – The district/building:
Desired Status / Where are we now? / Comments
Has established a protocol for monitoring implementation of the practice to ensure fidelity of implementation. (Consider the following points in your reflection).
  • Are there manuals describing implementation, classroom charts of practice steps, checklist of steps in implementation?
  • Can administrators/instructional staff can accurately describe the critical elements of the practice as validated through research, and discuss how they should be and are implemented in the school?
  • Is there a procedure in place to check how accurately the practice elements are being implemented? (monitoring for fidelity of implementation)
  • If there is a procedure for monitoring fidelity of implementation, is that procedure provided in a written document to staff?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Has a defined strategy for professional development/technical assistance for correcting drift from fidelity? (Consider the following points in your reflection).
  • Is there a procedure in place to provide corrective support to educators to correct drift away from the correct implementation of adopted practices?
  • Has the procedures been shared with staff in written form?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Has documentation of fidelity monitoring and adjustment protocol in action? (Consider the following points in your reflection).
  • Is there documentation of implementation of the procedure?
  • Is there documentation of the results of correction (e.g., data showing student outcomes)?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Context of Practice - There is evidencethat:
Desired Status / Where are we now? / Comments
The practice is delivered to students with disabilities in a general education setting with general education peers most or at all times. (Consider the following points in your reflection).
  • Has the district/school reviewed provision of services to students with disabilities to ensure that they get the benefit of the evidence based practice with their general education peers?
  • Has the district/school considered how personnel roles can be expanded to provide support to students with disabilities, allowing them to receive benefit of the evidence based practice with their general education peers?
  • Do Students with disabilities targeted by this practice:
  • Share all academic and special area classes and non-academic school activities with their general education peers
  • Are addressed in the same way as their typical peers for classroom and building behavior management and group contingency plans
  • Have equal opportunity to participate in extracurricular activities like clubs. pep rallies, field trips, sports teams, and community service
  • Do General education and special education teachers and other educators involved in implementing this practice collaborate in teaching both students with disabilities and their typical peers, including:
  • Co-planning, co-teaching, and co-evaluating learning
  • Sharing responsibilities for behavior management and implementation of FBAs and BIPs
  • Collaboration on grading
  • Collaboration on communication with parents
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Evidenceof Impact - The district/building’s data for students with disabilities show:
Desired Status / Where are we now? / Comments
Evidence from norm referenced and/or curriculum based measures that indicate students with disabilities are gaining skills at an accelerated pace that will help close the gap with typical peers. (Consider the following points in your reflection).
  • Is the district/school consistently comparing data to analyze outcomes for students with disabilities against the performance of their typical peers?
  • Has the district/school looked specifically at outcome data for students with disabilities to ensure that the evidence based practice is resulting in a strong, positive learning curve?
  • Do the data suggest that students with disabilities’ learning curve is sufficiently strong that they will close or come close to closing their gap in performance with their typical peers?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Progress toward IEP goals is measured regularly and shows successful attainment of relevant and appropriate IEP goals. (Consider the following points in your reflection).
  • Are IEP goals/objectives written so that the measures are data based and align with the general education curriculum in the are of the evidence based practice?
  • Is progress on IEP goals analyzed at least annually using outcome measures from the data used to monitor and adjust the evidence based practice?
  • Do the data show that students with disabilities are successfully attaining the IEP goals and objectives?
  • If the data show during the instructional year that IEP goals and objectives are not consistently being met, is instruction analyzed and adjusted on a regular basis during the instructional year?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Work samples that are evaluated to document growth in skill and adjust interventions. (Consider the following points in your reflection).
  • Has the school/district identified specific work samples that will document the degree of success in skill development for students with disabilities?
  • Are these work samples consistently collected and analyzed?
  • Is instruction adjusted on a day to day basis as a result of this analysis?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Clear documentation of results from progress monitoring and intervention adjustment to achieve successful outcome. (Consider the following points in your reflection).
Has the school/district maintained records of
  • The data collected for the purpose of monitoring individual and group progress?
  • Analysis of these data?
  • Decisions about adjustments to instruction or behavior management made on the basis of these data?
  • Results of modifications based on adjustments to instruction?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Soft data that support the effectiveness of the practice, such as student and parent reports of success in using skills or behaviors in out of school settings. (Consider the following points in your reflection).
  • Has the school/district considered what other kinds of information need to be considered, such as how students and parents like or value the evidence based practice, how students feel about themselves as learners, how educators feel about their skill in implementing the practice, et cetera?
  • Has the school/district adopted strategies to collect these data, consistently collected them, analyzed them, and made adjustments to practice implementation based on the analysis?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional

Look For’s Checklist
√Evidence from statewide assessment of success in meeting or making progress toward SPP targets/AYP
√Interim measures (intermediate assessment data both summative and formative) reflecting positive trajectory of improvement
√Curriculum-based measurements/progress monitoring data documenting successful outcomes/trajectory of improvement
√Comparative student work samples demonstrating proficiency or progression in skill/content area
√Impact of practice on student referral rates to special education
Evidence of Systemic Support for the Practice - The district/building:
Desired Status / Where are we now?
Has developed a constituency for the practice among families and community by actively advertising and communicating about the practice and its impacts and by providing training to ensure families can support practice as partners in their children’s education.(Consider the following points in your reflection).
Has the school/district:
  • Communicated what the practice is, why it is being implemented, and what can be expected as it is being implemented?
  • Communicated consistently and on a regular basis about how the practice is being implemented and what the outcomes are?
  • Offered family an opportunity to be partners in the implementation of the practice, either in the school or at home?
/
  • Not evident
  • Emerging
  • Operational
  • Highly Functional