Ohio Board of Regents

Adult Basic and Literacy Education

Desk Review Description

Fiscal Year 2013

June 2014

Introduction

The Adult Basic and Literacy Education Program (ABLE) under the Chancellor of the Ohio Board of Regents (Chancellor) is required by law to show progress of the State ABLE Program toward continuously improving its performance. In addition, the Chancellormust describe how the State ABLE Program evaluates annually the effectiveness of the adult education and literacy activities based on the performance measures described in the Workforce Investment Act of 1998, Title II, the Adult Education and Family Literacy Act. The Desk Review is one means of measuring local program performance.

The uses of the ABLE Desk Review include:

Accountability - meet performance goals and demonstrate state and local program achievement

Promotion of program improvement – establish a system for assisting programs in their continuous improvement

Information and advocacy for program – share key information with internal and external stakeholders

The ABLE Desk Review provides a classification for the local programs based on the following elements:

Minimum Performance Levels (MPL) for the Educational Functioning Levels (EFL) as determined by standardized assessments

MPLs for the follow-up measures including those for employment, entering postsecondary education, and/or training, obtaining a GED and increased involvement in family literacy activities

Student enrollment projections and student retention rates

Pretesting and posttesting rates of students to determine level completion

Key requirements and assurances of the FY 2013 grant

Explanation of the Report Format

The Desk Review Report consists of three forms – data form, worksheet for FY2013 and the report.

  • The data form compares key performance measures relating to enrollment, student achievement, program achievement and assessment to state performance targets.
  • The worksheet for FY 2013is a summary of the results and achievement levels. The worksheet is completed by the state ABLEregional program manager.
  • The report provides the classification of the program and comments from the state ABLE regional program manager. The comments provide feedback on areas that may need to be improved.

The overall classification on the FY 2013Desk Review indicatesthe performance status of a local program by using twomeasures. The two measuresare weighted as follows:

  • Measure 1,Student Achievement, represents 67% of the overall rating.
  • Measure 2,Administrative Requirements,represents 33% of the overall rating.

The designations for FY 2013have fourclassifications - Exemplary, Superior, Acceptable and Not Acceptable. The table below is used for determining the classifications for Measure 1, Measure2,and the overall Desk Reviewclassifications. For example, if the final percentage for Measure 1 is 79%, the rating for that measure is Acceptable. Ratings will be rounded to the nearest percent in this process.

Designation / State Indicators Met
Exemplary / 95%-100%
Superior / 85% - 94%
Acceptable / 70% -84%
Not Acceptable / Below 70%

Explanation of the Measures

Measure 1 – Student Achievement

Student Achievement has three (3) sections,A -C,focusing on factors related to student performance and the reporting of those achievements. Measure 1 receives more emphasis since it reflects the major intent of the legislation for ABLE’s state and federal funds.The elements of this measure are:

  • Completion of the Educational Functioning Levels (EFLs)
  • Placement into postsecondary education and/or training
  • Attainment of employmentand retention of employment
  • Obtainment of the GED
  • Achievement of family literacy goals
  • Other performance measures such as retention, enrollment, pretestand posttestrates

Section A:Core Indicators of Performance (CIP) - Educational Functioning Levels (EFL) Completion

In this section, programs receive points forthe student completionof the Educational Functioning Levels (EFLs).Programs are given one point for each EFL for which students’ performance met or exceeded the state’s Minimum Performance Level (MPL) targets. Points are totaled and divided by the total number of EFLs in which students were enrolled. The percentage is recorded.

Example:

EFLs Attempted / EFLs Achieved / Result for Section A
6 / 5 / 83%

Section B: Core Indicators of Performance (CIP) – Follow-up Goals

Follow-up goals looks at a program’s achievement in meeting placement in postsecondary education and/or training, attainment and retention of employment, obtainment of aGED, and achievement of the family literacy goals. Credit will be given for each indicator for which the programs met or exceeded the state’s MPLs. These indicators are only measured for students who had the indicator as a goal.

Example:

CIPs Attempted / CIPs Achieved / Results for Section B (%)
4 / 3 / 75%

Also, a program will only receive credit for achievement of indicators for which it has an approved component(s).

  • For example, if a program reports achievement in the family literacyindicators, but does not have an approved family literacy component, the programwill not receive credit for achievement of those indicators.

Section C:OtherPerformance Measures

In this section, programs are rated onfiveother student performance measures. Each element achieved is worth onepoint.

  • Enrollment - Enrollment is based on achieving 95% or more of the FY 2013 projected enrollment or exceeding the FY2012 actual enrollment – one point.
  • Retention - Retention rate of 75% or aboveis achieved. Retention refers to a student who completed a level or who was enrolled during the fourth quarter of the previous program year- one point.
  • Pretest information - Pretest data entered in ABLELink that meets or exceeds 99%of the number of students enrolled - one point.
  • Pretest and posttest- The number of students posttested equals or exceeds 65% of the number of students pretested using a standardized assessment - one point.
  • Overall completion- This measure compares the program’s overall EFL completion rate to the state’s overall FY 2013EFL completion average of 62%. To receive the point, a program must meet or exceed the state’s overall completion average.

Example:

Element / Achievement Level / Pt(s) Earned / Possible Pts
Enrollment / Met 84% of FY 2013 projected enrollment and did not meet FY 2012 actual enrollment / 0 / 1
Retention / Rate was 96% / 1 / 1
Pretest / Number tested matches 99% of the number enrolled / 1 / 1
Pretest and posttest / Program posttested 66% of the number pretested / 1 / 1
Overall Completion / The overall average for the program was 62% which does not exceed the state’s average (67%) / 0 / 1
Total points received for five (5) elements / 3 / 5*
Total points achieved__= 3 Percentage of elements met
Total elements possible 5 / 60%

*The divisor is always five (5) since we use the number of the items being examinedthat have potential point values.

Once the scoringfor each of the sections is completed, the score for Measure 1 is determined by adding the percentages for each section and calculating an average percentage for the measure.This percentage is used to find the classification for this measure.

Classification for Measure 1 Example:

Section / Total Pts for Program / Points Achieved / % of EFLs achieved / Score
Section A: EFL Completion / 6 / 5 / / 83%
Section B: CIPsFollow-up goals / 4 / 3 / / 75%
Section C: Other Performance Measures / 5 / 3 / / 60%
Average Score / = 72.6 / 73%
Rating for Measure #1
(refer to page 3) / Acceptable

Measure 2 - Administrative Requirements

Administrative requirements are determined by the extent to which the program has complied with key state/federal standards and grant requirements. These items are tracked through the State ABLE Program. Each item that meets the FY 2013requirements will receive one point.There are 7itemsto be evaluated for all programs this year. Attendance at the New ABLE Directors’ Orientation applies only to programs that have new administrators. Corrections FER applies only to programs with an approved Corrections Component. The items being tracked are listed below.

The items listed here are tracked for timeliness and completeness.
Data Certification Checklist Verification Form / ABLE Final Expenditure Report (FER)
Program Improvement Consultation Plan (PICP) / Fall Directors’ Meeting
LD Planning Guide / Technology Report
Schedule A & Schedule A InKind
New Directors’ Meeting (if applicable) / Corrections FER (if applicable)

Example:

A particular program was not required to attend the New ABLE Directors’ Orientation and does not have a Corrections Component. The program met6of the required items.The score and classification for the program would be as follows.

Requirements / 7
Met / 6
Not met / 1
Score / 86%
Classification for Measure 2 / Superior

OverallClassification

Determining the Classification for the Desk Review

To determine the overall program classification, the percentage for Measure 1 is doubled and added to the percentage for Measure 2 as indicated below. The total is then divided by three.

Overall Classification Example:

Measure / Percentage / Multiplication/Division Factor / Weighted Value
Measure 1 / 73 / 2 / 146
Measure 2 / 86 / 1 / 86
Totals / 3 / 232
Classification / Acceptable / %

Improvement Actions

If any program receives a classification of “Not Acceptable” on the FY 2013 Desk Review, the following actions should occur:

  1. Review program data frequently during the year to be aware of the accuracy and reliability of the data and makeprogram adaptations.
  2. Reflect changes in programming to address the areas of concerns in the FY 2015 grant revision(s).
  3. Address any areas needing improvement in the FY 2015Data Quality Certification Checklist.
  4. Request technical assistance as needed from the State ABLE Program Office and/or the Ohio Professional Development Network.
  5. Seek appropriate Professional Development for the local program personnel.

If program remains “Not Acceptable” for more than two consecutive years, the funding level of the program maybe affected.

If you have any questions regarding your FY 2013 Desk Review please contact your state ABLE regional program manager.

FY 2013 Desk Review Page 1 of 7June 2014