Guidance for Annual Programme Evaluation of Validated and Institutional Partnership programmes
Contents
Contents 2
Guidance on Annual Programme Evaluation 3
Overview 3
Forms 2
Can more than one programme be included in an APE? 2
Management Information 2
Timetable 2
Indicative Undergraduate Timescales 2
Indicative Postgraduate Taught Timescales 3
Indicative Postgraduate research Timescales 4
Completing the form 5
Front sheet 5
Statistical data 5
Summary Evaluation 5
Student feedback 5
Action Planning 6
Appendix 1: Responsibilities 8
Guidance on Annual Programme Evaluation
The Policy on Annual Programme Evaluation (APE) sets out the purpose of APEs and the principles that guide their development and use.
This guidance should be used in conjunction with the policy and provides additional information to support the development and use of APEs. This guidance is specifically for internal and partnership provision. Guidance for validated provision can be found at: http://www.city.ac.uk/__data/assets/word_doc/0008/69254/ape_guidance_vip.doc.
Overview
APEs are designed to support reflection on the past year’s activity and to capture and monitor actions being taken to enhance programmes across the course of the present academic year.
They are the primary way in which programme teams plan and report on performance against key metrics for student satisfaction, progression and employability.
They are ‘living documents’ updated throughout the year to reflect on-going feedback, developments and new data. Programme Committees monitor progress at every meeting and Boards of Studies[1] consider APEs twice each year. APE (and particularly action planning) should be undertaken in consultation with Staff Student Liaison Committees (SSLCs)[2] and reported to the Course Board on an annual basis.
APEs should be based on evidence from a variety of sources including (but not limited to):
· data on recruitment, progression, performance, retention, graduate destinations
· Student surveys, such as, but not limited to, NSS/PTES/PRES/ Your Voice surveys*
· module evaluation
· SSLC minutes
· External Examiner feedback
· professional body reviews or audits
· revalidation
* Your Voice is relevant for some institutional partnership programmes only
APEs are also designed to capture actions being taken by Professional Services and by management colleagues within Institutions.
Forms
APE forms are available in Section 8 of the Quality Manual on the Student and Academic Services website. There are three forms covering validation and institutional partnerships[3]:
· Undergraduate programmes
· Postgraduate taught programmes
· Research programmes
Can more than one programme be included in an APE?
Some institutions/departments choose to cluster a number of programmes into a single APE. This is very helpful where a number of modules are shared or the broad provision is comparable. Advice should be sought from Student and Academic Services on how best to group programmes.
It is vital that differences across provision are adequately reflected and feedback, updates and actions may relate specifically to a particular route or cohort. The summary evaluation is also an excellent place to draw out any key differences between the groups covered. Consideration should also be given to whether some or all of the Management Information tables should be separated out so that differences in admissions, progression, destinations and the student experience can be identified and addressed.
A balance must be struck between covering a very small programme or clustering too many programmes together – both approaches present risks for undertaking meaningful evaluation and action planning.
Management Information
Staff responsible for preparing the draft of the APE should reference the technical annex.
If a programme is required to provide statistical data on an annual basis to an accrediting or professional body, and if this data covers the above areas, the programme team may be able to append that data rather than duplicating the information using the tables in the APE. Programme teams should discuss these options with Student & Academic Services at the University.
Timetable
Throughout the lifecycle of the APE on-going monitoring and development takes place via a standing item on Programme Committee and SSLC agendas. Both Committees should receive a full copy of the APE as early as possible in the APE lifecycle. Following this the Committees will focus on the progress of the action plan and development required resulting from feedback and new data including matters arising during the year.
Indicative Undergraduate Timescales
Monitoring and development of previous year’s APE via Programme Committee and SSLC. Overseen by PD and HoD. /ê
Activity / Input / Responsibility / Committee/s / TimingReview and planning
Identify broad issues to be covered in new APE / · Previous year’s APE
· Impact of strategic developments / PD, HoD / PC / May - June
Draft APE
Incorporate inputs as they become available
Identify and incorporate any development support needs for each action (insert under ‘Support needs and status’)
Where possible early consultation with students is also helpful (e.g. on internal survey results) / · Survey data
· Reflection on operation of programme/ modules, student feedback and direction of enhancement activity
· Admissions & destinations data
· Assessment Board data/ External Examiner comments and any actions arising
· Initial NSS results
· External Examiner report(s)
· Resit Assessment Board data / PD (in liaison with HoD) / BoS/PC / June -Sept
APE meeting
Meeting to discuss APE’s interaction with Institution plan / · Institution/ School plan and impact of any strategic aspects
· Resource implications / Principal, HoD, PD / Inst. Management Committee/ School Exec / As needed
Confirmation that APE is ready for receipt by Board of Studies (or PC for IP provision) / PD / BoS/PC / Sept
First stage consideration and approval of APE / Principal / BoS/ PC / Oct
Student input
Discuss and update APE / Targeted focus groups and other forms of feedback might usefully be used here in addition to SSLC / PD / SSLC, BoS/PC / First SSLC
Receipt/ approval by Course Board (receipt for Validated provision, approval for Institutional provision) / Chair of Course Board / Course Board / Nov/Dec
Monitoring and updates / · Matters arising
· Additional feedback
· Additional data / PD / PC, SSLC / Each PC & SSLC
Report to University-level Committees / Academic Services, Dean of Validation / VIP Committee / Feb
Discuss University-level issues / Dean of Validation / Senate, UET/ExCo / As needed
APE update
Actions updated/ monitored / PD, Principal, Chair of Course Board[4] / BoS/PC, Course Board / By last BoS
ê
Use APE to commence development cycle for new APE. /Indicative Postgraduate Taught Timescales
Monitoring and development of previous year’s APE via Programme Committee and SSLC. Overseen by PD and HoD. /ê
Activity / Input / Responsibility / Committee(s) / TimingReview and planning
Identify broad issues to be covered in new APE / · Previous year’s APE
· Institution and University strategic developments / PD, HoD / PC / Jul-Aug
Draft APE
Incorporate inputs as they become available.
Identify and incorporate any development support needs for each action (insert under ‘Support needs and status’).
Where possible early consultation with students is also helpful (e.g. on survey results) / · PTES survey data
· Reflection on operation of programme/ modules, student feedback and direction of enhancement activity
· Admissions & destinations data
· Assessment Board data/ External Examiner comments and any actions arising
· External Examiner report(s)
· Resit Assessment Board data / PD in liaison with HoD / BoS/ PC / Jul-Nov
APE meeting
Meeting to discuss APE’s interaction with Institution plan / · Institution/ School plan and impact of any strategic aspects
· Resource implications / Principal, HoD, PD / Inst. Manag. Committee/ School Exec / As needed
Confirmation that APE is ready for receipt by Board of Studies (or PC for IP provision) / PD / BoS/ PC / Dec - Jan
First stage consideration and approval of APE / Principal / BoS/ PC / Jan- Feb
Student input
Discuss and update APE / Targeted focus groups and other forms of feedback might usefully be used here in addition to SSLC / PD / SSLC,BoS PC / First SSLC
Receipt/ approval by Course Board (receipt for Validated provision, approval for Institutional provision) / Nov/Dec
Monitoring and updates / · Matters arising
· Additional feedback
· Additional data / PD / BoS/ PC, SSLC / Each PC & SSLC
Report to University-level Committees / Academic Services, Dean of Validation / VIP Committee / Feb/June
Discuss University-level issues / Dean of Validation / Senate, UET/ExCo / As needed
APE update
Actions updated/ monitored / PD, Principal, Chair of Course Board[5] / BoS/PC, Course Board / By last BoS
ê
Use APE to commence development cycle for new APE. /Indicative Postgraduate research Timescales
Monitoring and development of previous year’s APE via BoS and SSLC. Overseen by STR and HoD. /ê
Activity / Input / Responsibility / Committee(s) / TimingReview and planning
Identify broad issues to be covered in new APE / · Previous year’s APE
· Institution and University strategic developments / HoD / BoS / May - June
Draft APE
Incorporate inputs as they become available.
Identify and incorporate any development support needs for each action (insert under ‘Support needs and status’).
Where possible early consultation with students is also helpful (e.g. on internal survey results) / · PRES results (when available)
· Reflection on operation of programme/ modules, student feedback and direction of enhancement activity
· Admissions & destinations data
· Assessment Board data/ External Examiner comments and any actions arising[6]
· External Examiner report(s)
· Resit Assessment Board data / STR in liaison with HoD / BoS / June-Aug
APE meeting
Meeting to discuss APE’s interaction with Institution plan / · Institution/ School plan and impact of any strategic aspects
· Resource implications / Principal, HoD, STR / Inst. Manag. Committee/ School Exec / As needed
Confirmation that APE is ready for receipt by Board of Studies (or PC for IP provision) / STR / BoS / Sept
First stage consideration and approval of APE / Principal / BoS / Oct
Student input
Discuss and update APE / Targeted focus groups and other forms of feedback might usefully be used here in addition to SSLC / STR, ADE / SSLC, BoS / First SSLC
Monitoring and updates / · Matters arising
· Additional feedback
· Additional data / STR / BoS, SSLC / Each BoS & SSLC
Course Board
Receive APE / Chair of Course Board / Course Board / Nov/Dec
Report to University-level Committees / Academic Services, Dean of Validation / VIP Committee / Feb
Discuss University-level issues / Dean of Validation / Senate, UET/ExCo / As needed
APE update
Actions updated/ monitored / STR, Principal, Chair of Course Board[7] / BoS /PC, Course Board / By last BoS
ê
Use APE to commence development cycle for new APE. /Completing the form
Front sheet
Key information is given on the front page which covers responsibilities and enables tracking.
Statistical data
Statistical data
The APE is accompanied by a series of tables which are completed throughout the year as data becomes available. Programmes may also wish to add data available to them related to the specific discipline, demographic information or other areas which will help with evaluation and planning. Commentary is likely to focus on:
· trends seen year on year
· particularly weak or strong indicators as well as any contextual information that explains weak/ strong performance
· what indicators the data provides in relation to the overall strategy of the programme moving forward
· discussion of performance in relation to equivalent programmes/cohorts in the University and sector competitors
Summary Evaluation
This section is likely to provide useful context for discussion of the APE at Boards of Studies and with students. It provides an opportunity to reflect on the overall health of the programme over the past academic year, assessing the cumulative impact of enhancement activity and any relevant wider changes within the programme, discipline area, School, University and/or sector. It can also be used to reflect on future direction, relevant opportunities and challenges as well as to highlight any issues and contextual factors the programme team has not raised elsewhere on the form. Some programme teams have usefully employed a SWOT (strengths, weaknesses, opportunities, threats) analysis as part of the summary evaluation.
Supervisors (for research programmes only)
List the approved Category A and Category B research supervisors.
Student feedback
Student feedback and student views on content are core to the APE process.
The APE can also be revised throughout the year to include additional student feedback and ensuing actions.
Student views
Whenever practical involve students in the APE process as early as possible, ideally at the drafting stage.
Outline how student views have been sought and incorporated, not only as a means of assuring quality but to demonstrate to our students how much we value their input and that it is taken seriously
SSLCs[8] provide an important mechanism for discussion on actions designed to address student feedback and/or enhance student satisfaction. The first SSLC of an academic year provides an opportunity for programme staff to gain students’ views on the contents of the first iteration of the APE including proposed actions designed to enhance the student experience and address student satisfaction issues.
It is recognised that not all SSLCs will have met by the point of the Board of Studies meeting where APEs are initially considered – where this is the case state how student views will be sought and updated once students have contributed.
Updates should be provided to students via the SSLC and other appropriate mechanisms that help ensure students are aware of further actions being taken and on-going programme developments. It is good practice to ensure students are able to access relevant sections of the APE via the institution’s VLE and to provide copies of previous APEs that will also help ensure students are aware of issues that have previously been raised and addressed.
Your Voice, Our Action
Identify specific changes made to the programme following feedback from students, either through student surveys or via other channels. This section should be shared via SSLC and in any fora where it will have a positive impact on the student body. This will ensure the feedback loop is closed prior to the point at which student surveys are released in the academic year.