DRAFT PROGRAM EVALUATION PROCESS

Peralta Community College District

Program Evaluation Process

DRAFT Common Review Criteria for CSEP Review and Discussion

UPDATED: May 22, 2007

This document presents a program evaluation process for use in the 07-08 educational master planning process. The Committee for Strategic Educational Planning (CSEP) developed the process as part of their charge to create a common framework for the colleges to use in educational master planning in keeping with board policy 5.11, which states: “The goal is to provide accessible, high quality adult learning opportunities to meet the educational needs of the multicultural East Bay community.” CSEP used examples from other California Community College districts and input by Educational Services and college staff. This is a modification of the basic process approved by the Strategic Management Team (SMT) and reviewed by Strategic Planning and Policy Advisory Committee (SPPAC).

This document presents a set of criteria and a process that will be implemented at the college level, not in a district-wide process. CSEP developed this as a tool so that the four colleges have a consistent procedure that establishes a “level playing field” for the four colleges regarding program evaluation.

Rationale

CSEP developed this model for two primary reasons. First, it is good academic practice to have clear procedures and processes for evaluating programs at an institution-wide level to ensure the full set of offerings meets the changing needs of the community. It is central to the mission of the Peralta colleges that we regularly evaluate the relevance, quality and productivity of our offerings. The process below provides a collegial framework for evaluating relevant information and making systematic planning decisions so that the colleges evolve as the district’s communities evolve. The CSEP process establishes the implementing procedures for Board Policy 5.1.1, which describes the need for a program evaluation process.

Secondly, the primary recommendation of the Western Association of Schools and Colleges accreditation report was that that the four colleges conduct and integrated process of educational planning: “The team recommends that a district wide plan and implementation process should be created that is strategic and systematically integrates the educational, financial, physical and human resources of the district. All planning processes should be inclusive of the four colleges and the communities served by the district. The plan should include identified institutional outcomes with criteria for evaluation on a periodic basis. It is recommended that the district wide plan integrate the educational master plans and program reviews of the colleges. The chancellor should ensure that the plan and the ongoing planning processes are communicated throughout the district (Standards 3.B.1, 3.B.3, 3.C.3, 10.C.1, 10.C.6.) The CSEP process is one of the core elements of this integrated strategic educational planning process for the colleges.

Educational Master Plan Update Process

The colleges will use a continuous program improvement process to support all programs in achieving quality, relevance, and productivity. The colleges will use a common set of “horizontal” criteria to assess their program offerings.

Guiding Principles

The purpose of the program improvement process is to ensure responsiveness to our communities and deliver academically sound programs meeting the mission of California community colleges as defined in the Education Code. The guiding principles for the process are as follows:

§  Provide high quality education;

§  Ensure relevance to student and community needs;

§  Meet community expectations for sound financial management of public funding (i.e., productivity);

§  Emphasize revitalizing programs through staff development, community outreach, use of best practices, and innovative scheduling and delivery; and

§  Respect all contract provisions.

Key Terms and Committees

The following are the key terms and organizations referred to in this document

Committee for Strategic Educational Planning (CSEP) / Purpose: Develop process to be implemented at the Colleges to evaluate programs at an institutional-level to ensure relevance, quality and productivity. Ad hoc.
Membership: Academic Senate Presidents, Vice Presidents of Instruction, Vice Chancellor – Educational Services, President, CoA, Karolyn van Putten.
Strategic Planning and Policy Advisory Committee (SPPAC) / Purpose: Shared governance committee to provide advice to the Chancellor on strategic issues.
Strategic Management Team (SMT) / Purpose: Ensure implementation of strategic initiatives.
Membership: Chancellor, College Presidents, Vice Chancellors, Associate Vice Chancellor – Admissions and Records.
College Educational Planning Committees (EPCs) / Purpose: Recommend a strategic, community-needs based educational master plan. EPCs will review program evaluation results, program reviews, and environmental scan. These committees have a specific task in developing college educational master plans in Fall 2007; there may also be a periodic role for the EPCs to monitor innovations and planning trends, and to monitor the implementation of the program evaluation process.
Membership: These committees are in formation.
Program Evaluation Process (PEP) / Purpose: Conduct an evaluation of program relevance, quality and productivity as described in this document. The Program Evaluation Process (PEP) uses a college-level perspective to identify three categories: Grow / Maintain / Watch. The PEP uses a consistent set of criteria across the colleges and encourages cross-college dialog.
Accelerated Program Review (APR) / Purpose: Engage each instructional, student service, administrative, and service center unit in a self-assessment of effectiveness. The colleges’ educational planning committees will assess instructional program together with the results of the institution-level program evaluation process.
Unit Plans (UPs) / Purpose: Present a detailed plan of action for improving quality, relevance, and productivity as indicated by the APR and the PEP. All units – departments, programs, disciplines – will complete unit plans. Unit plans will be reviewed and synthesized into College Educational Master Plans.
Educational Master Plans (EMPs) / Purpose: Present an College-wide plan that is strategic and comprehensive and gives direction to all programs and services. The educational master plans (EMPs) will be based on the data and analysis in the APR and PEP and will synthesis the unit plans.

Roles and Responsibilities

§  The Vice Presidents of Instruction and Academic Senate Presidents will work together to establish college educational planning committees (EPCs).

§  These EPCs will be the primary committees to conduct college-wide educational master plan update activities.

§  They will develop recommendations for their respective presidents. District educational services will provide support and data for the Unit Plans.

OVERALL TIMELINE

Action / Draft Due Date
Initiation of Committee for Strategic Educational Planning (CSEP) / February 27, 2007
Vice Chancellor Haig and CSEP members present draft process at the Council for Instruction, Planning and Development / May 7
CSEP prepares criteria and process for Educational Master Planning review. / May 15
Vice Presidents to work with Academic Senates to establish Educational Master Planning committees. / May / August
Vice Chancellor, Vice Presidents and CSEP members to present the program review process to faculty at Flex Day / August
College Educational Master Plan committees to apply criteria and create lists of Grow / Maintain / Watch programs. / October 15
SMT and SPPAC Review for consistency. / October 18, 22
Colleges prepare draft Educational Master Plan updates. / November 30
College Collegial consultation, SPPAC and SMT review. / December 1 – February 1

Definition of “Program”

Programs are defined as:

§  Any series of courses leading to a degree, a completion or achievement certificate, or any other credential of completion or achievement.

§  All vocational programs

§  Any discipline offered at two or more colleges

§  A set of related courses not leading to a degree of certificate that students take in a sequence of increasing skill levels (“a set of related courses that students navigate”)

§  Multi-disciplinary special topics such as biotechnology.

Steps and Criteria

The Educational Master Planning committees at each college will screen all programs using the following criteria.

1. Review each program’s enrollment trends for the five preceding Fall terms and code as:

A - Growth in each consecutive year

B - Mixture of growth and decline

C - Declining enrollment in each consecutive term

2. Review FTES/FTEF ratio for the five preceding Fall terms and code as:

A - Over 17.5 for all five years

B - Mixed pattern of under and over 17.5

C - Under 17.5 for five consecutive terms

(Step 2 is not conducted for vocational programs that have mandated class size limitations.)

3. For programs with classes offered at more than one college, review classes for the last five years for class size and code as:

A – Over 15 students per class for all five years

B - Mixed pattern of over and under 15 students per class

C - Under 15 students for five consecutive terms

4. Vocational Programs Only: Review cost of programs, community needs, and presence of alternative providers in addition to Peralta. (High cost is defined as twice the average cost that is embedded in the 17.5 FTES/FTEF standard. **This needs to be quantified by Educational Services staff.) Code programs as

A - Not high cost, or high cost but also high need and insufficient alternative providers

B - High cost program with low community need and/or multiple alternative providers

5. Review each program’s trends for five terms regarding retention, persistence, student learning outcome achievement, program completion or student satisfaction. This review is to provide information on improvement regarding student success. Note any major patterns or trends.

A – Trends and levels generally positive

B - Some trends/levels positive; some declining or too low.

C - Most trends/levels declining or too low.

6. Relevance to community needs and labor market trends as documented in state and local labor market scans and community needs information.

A – Programs clearly aligned to community and labor market needs

B - Programs generally aligned, but could use some refinement or updating

C - Programs need to update or refine offerings

7. Relevance to the college’s strategic plan.

Describe the role of each program within the college’s strategic plan. This can be evaluated qualitatively but should include evaluation and if possible quantification of how important the program is to the college’s plan. Relevance can include:

§  New program under development

§  Program that is integral to the college’s overall strategy

§  Program that is essential for transfer

§  Program that serves a community niche.

§  Programs where student enrollment or success has been demonstrably affected by extraordinary external factors, such as barriers due to housing, employment, childcare etc.

Program Evaluation Process

Each college Educational Master Plan committee will implement the program evaluation process with support from the Vice Presidents of Instruction and District Educational Services. The goal of the screening process is to consider a range of factors in creating a framework for the future direction of the college’s educational programs. The EPC will quality, relevance, productivity and the overall strategic direction of the college. The EPC will also consider collaborative educational strategies across the four colleges. The end result will be a listing of the college’s programs into three categories.

Description / Process
Grow / New, existing or modified programs that the college intends to grow. / Complete a Program Improvement Plan. Review annually.
Maintain / Existing programs that will be maintained at current levels. / Complete a Program Improvement Plan. Review annually.
Watch / Revitalize / Programs that do not meet the standards set in criteria 2 and/or 4. / Prepare a unit plan; review at key milestones.

Key Considerations

Link to Community Needs and Visioning. The environmental scan and other dialogs will assist the colleges in crafting a positive vision for the future of the colleges, including emerging areas or existing needs not currently served. The process below is only one element of the Colleges’ collaborative educational master planning process.

Colleges Can Move More Rapidly. The process below is proposed as a process that all colleges will implement consistently. This will provide an “even playing field” in which all programs receive treatment. The process below regarding “watch/revitalize” program, if adopted, would be the baseline expectation that all colleges would meet. However, a college can make decisions regarding program continuance on a faster timeline through its consultation process.

Focus on Programs. The focus of the evaluation process is on programs, not on people. The faculty contract describes the procedures for identifying reassignment and retraining options for faculty whose programs are discontinued. For example, faculty can develop new FSAs in an area of growing community need.

Program Evaluation Builds on Program Review. The EPCs will review program reviews in developing Grow / Maintain / Watch-Revitalize lists as shown on page 9.

Fall 2007 / Each college will screen all programs using the criteria above. All programs complete a Program Improvement Plan (unit plan) in Fall 2007.
Grow and Maintain Programs
Programs that meet the standards set in criteria 1 through 4 follow the schedule below /
Watch / Revitalize Programs
Programs not meeting criteria 1 through 4 complete the following process. Programs at two more colleges are encouraged to collaborate with their counterpart programs at the other colleges on district-wide strategies such as consolidation, enhanced coordination and specialization.
1. Develop and Implement a Program Improvement Plan (PIP).
§  Curriculum approval
§  Marketing and outreach
§  Instructional changes / 1. Develop and implement a Program Improvement Plan (PIP) (using existing resources?).
§  Curriculum approval
§  Marketing and outreach
§  Instructional changes
Spring 2008 / 2. Continue Implementing PIP / 2. Implement PIP.
Fall 2008 / 3. Screen against all criteria and update PIP. / 3. Continue implementing PIP. Adjust as necessary.
Spring 2009 / 4. PIP Implementation. / 4. Continue implementing PIP. Adjust as necessary.
Fall 2009 / 5. Annual review and planning cycle: Screen against all criteria and update PIP. / 5. Evaluate and adjust PIP. (Remove entry-level courses from schedule if insufficient progress has been made.)
Spring 2010 / 6. PIP Implementation / 6. Continue implementing PIP. Adjust as necessary.
Fall 2010 / 7. Annual review and planning cycle: Screen against all criteria and update PIP. / 7. Make decision on Continuance. Reinstitute entry-level courses if sufficient progress has been made. Offer final courses if insufficient progress has been made.

Implementation Steps

1. Educational Services provides