Early Childhood Outcomes

Program Quality Self-Assessment Process

Overview

The attached draftearly childhood outcome system (COS) program self-assessment documents are intended to assist early childhood school district leaders in participating in the broader LEA self-assessment process that the DPI Exceptional Children Division is currently piloting (see, draft summary of LEA Self-Assessment sections). The Division’s intention is to roll-out the LEA Self-Assessment process to all LEAs during the 2015-16 school year. This process will replace the previous LEA Continuous Improvement Program Plan (CIPP). Regional EC Division consultants will provide professional development to LEA Exceptional Directors/Leaders during four regional meetings each year on the overall LEA self-assessment process.

The sections of the LEA Self-Assessment Process that directly align with the COS self-assessment process include: Core Element 4: Problem-Solving for Improvement and Core Element 6: Communication and Collaboration. Upon analyzing the child outcome data, an LEA may incorporate this process into identifying needed research-based initiatives to ensure student’s mastery of the Foundations: Early Learning and Development Standards (common core and essential standards) which addresses Core Element 5: Research-based Instruction and Practices.

The emphasis on the current draft COS program assessment is on ensuring that LEA data is of high quality and that a system for monitoring the data is in place and that appropriate follow-up actions are taken. New professional development resources have been developed and provided to conduct LEA trainings on understanding the Child Outcome System, the developmental trajectories, and explaining child outcomes to families ( For assistance with these materials or questions related to implementing your local professional development plan, please contact your regional Early Learning Network consultant (

Child Outcome Summary (COS) System: Program Quality Assurance Self- Assessment

Program Administrator Completing Review:

Date of Self-Assessment:

Question / Level of Implementation
Not Yet / In Process / Achieved / Don’t Know
  1. COS initial ratings for all children are monitored to ensure data are in place two times during a reporting period

  1. COS exit ratings for all children are monitored to ensure data are in place prior to submission at the end of reporting period

  1. Process in place to ensure that data is reported for all children who exited the program who were in the program for at least 6 months of service (less than 5% of missing data)

  1. File review for quality of supporting evidence for rating is conducted on at least 10% (sampling) of enrolled children, or a minimum of 10 files (see, File Review for Supporting Evidence tool)

  1. Data analysis for overall program is conducted at least annually (see, Data Quality Self-Assessment tool)

  1. Stakeholder process including families is conducted regularly to consider program implications of child outcome data (e.g., administrators, teachers, related service providers, and family members)

  1. Leadership team that oversees the COS program data analysis develops follow-up long-term and short-term program improvement plans

  1. Information on “Understanding the Child Outcome System” is regularly provided to families

  1. Professional development on the COS system is planned and delivered at least annually to all staff (teachers and related service providers)

Child Outcome Summary (COS) System: Data Quality Self-Assessment

Program Administrator(s) Completing Self-Assessment:

Date of Self-Assessment:

Question / Yes / No / Don’t Know
  1. COS data patterns across progress categories for each outcome are consistent with expected distribution

  1. Comparison of entry and exit ratings by rater/team shows appropriate variety of ratings ( e.g. not all children given a rating of 6/7 for all three outcomes)

  1. COS data analysis compares patterns by different populations for this LEA to identify trends (race, ethnicity, gender, disability categories)

  1. Distribution of almost all ratings at entry are centered on a lower number than exit

  1. In most cases, COS ratings increase by no more than 3 points

  1. Groups of children with more severe disabilities will have distributions with a larger percent in categories a – c

  1. Groups of children with less severe disabilities will have distributions with a larger percent in categories d and e

Recommendations for follow up:

Overall program data analysis suggests that most entry ratings are consistently too high or too low: Develop follow-up action plan (e.g. provide staff with professional development on formative assessment and age referencing of data and use of the COS decision tree)

Overall program data analysis suggests an unexpected gain/loss in performance (e.g. as evidenced by a larger than 3 point increase over the time in program): Drill down to identify root cause and/or data quality issues; develop follow-up action plan

Analysis of data by individual rater/team shows patterns that suggest need for improvement (e.g. entry ratings of three and exit ratings of five for all children in a classroom or caseload): Develop follow-up action plan

Early Child Outcome Summary Rating - File Review for Supporting Evidence

Child:

Raters:

Reviewer:

Question / Outcome I / Outcome II / Outcome III
Yes / No / Yes / No / Yes / No
Is formative or summative assessment data provided that supports the rating in this outcome?
Is the data age-referenced and compared to the child’s chronological age for this outcome?
Does the “summary of relevant results” relate specifically to this outcome?
Does the summary of relevant results include examples of the child’s every-day functioning in this outcome?
Was there evidence of family input that supported the rating for this outcome?
Was there evidence that the rating was determined with more than one person’s input for this outcome?
Can this reviewer estimate, within one point in either direction, the rating based on the information provided for this outcome?
Total

Follow-up Recommendations:

Commend raters for job well done

Have exemplar raters assist others

Review results with rater and provide information about needed areas of improvement

Require raters to attend COS training

1