Contents / Page
Guidelines for Internal Subject Review / 2
Annex 1 Template for the Internal Subject Review briefing paper / 8
Annex 2Aide-mémoire for the Panel of an Internal Subject Review / 20
Annex 3Report Template for an Internal Subject Review / 36
Annex 4Generic Agenda for an ISR Planning Meeting / 42

Guidelines for Internal Subject Review

1Internal Subject Review is conducted on a periodic basis to review the management of academic standards and quality in programmes within a given subject. It begins with self-evaluation by the subject team which is followed by peer review by an informed Panel. The Panel consists of internal University staff and students and one or more external advisors drawn from outside the institution. The subject team provides its self-evaluation in a briefing paper that is supported by evidence. Panel members conduct independent desk-based research to consider and evaluate the evidence in the context of the briefing paper. The Panel formulates lines of enquiry which are provided for the subject team to provide transparency. A series of review meetings is held to enable the Panel to explore lines of enquiry with the subject team, students and employers. The meetings can be arranged to occur over full or part days, be grouped into an event or spaced apart over several weeks. The exact pattern of meetings is to be determined by the Panel and will take into account the nature of the subject and the volume of evidence to be reviewed.

2The process of Internal Subject Review begins with a planning meeting. This meeting considers the scope and timeline for the review and is arranged by a Standards and Enhancement Officer. The planning meeting is chaired by the Academic Registrar (or nominee) and attended by subject representatives and the intended Chair of the ISR Panel. A list of programmes in scope forms the basis of discussion and notes are taken by the Standards and Enhancement Officer.

3The process of Internal Subject Review is based on a briefing paper and a set of supporting documentary evidence. The briefing paper is intended to comprise a self-evaluation by the subject team with reference to the Expectations of the UK Quality Code[1]. A template for the briefing paper is provided on the Standards and Enhancement webpages about ISR (Annex 1).

4The ideal briefing paper is a concise, evidence-based, constructively critical self-evaluation of practice during the preceding two complete academic years/cycles. The briefing paper should be as short as possible and refer to evidence in extant documentation. The word length of the briefing paper will depend on the scope and complexity of the ISR, typically it might fall in the range 2500-5000 words.

5A briefing paper should identify good practice and scope for improvement as perceived by the subject team. It should show how previously identified strengths/areas for improvement have been built upon/addressed. It is important to include plans to address remaining areas for improvement. There should be an evaluation of the effectiveness of the measures taken to maintain and enhance the quality and standards of programmes. The Panel will be particularly interested in the effectiveness of evaluation and use of quantitative data and qualitative feedback in a strategy of enhancement and continuous improvement. Quantitative data might include entry qualifications; progression and completion rates; statistics on student achievement in all forms of summative assessment; degree classifications; first employment destinations. Qualitative feedback might include student feedback; staff feedback; external examiners’ reports; employers’ views on graduates they have recruited; accreditation and monitoring reports by professional or statutory bodies; previous internal subject reviews; comments from internal re-validation.

6The briefing paper should be submitted on the template which uses the following headings:

  • Brief contextual description of the subject
  • The subject’s track record in managing quality and standards
  • The setting of academic standards within the subject
  • Achievement of academic standards by students
  • The quality of learning opportunities within the subject
  • The information provided and used by the subject team
  • Enhancement of students’ learning opportunities
  • Evidence list: a numbered list of supporting evidence.

7The supporting documentary evidence should normally consist of extant documentation to show practice throughout the academic cycle. A list of requirements and suggestions is given below.

Within the briefing paper, teams should direct the Panel to the documentary evidence. This will enable the Panel to test statements made in the briefing paper and to reach sound judgements on quality and standards. It is not necessary or desirable to re-iterate in the briefing paper factual informal and/or evidence provided in the supporting documentation. The briefing paper should be submitted to the Secretary for the ISR in as an electronic Word document.

The following list provides examples of suitable materials, but is not exhaustive. The arrangements for the subject team to provide the evidence base for the Panel will be agreed during the planning meeting. The evidence should be provided in electronic format and the Panel may request that certain items be provided on paper. This can include provision of evidence through a memory stick or similar device. Where a document is cited as evidence under several sections of the briefing paper it need only be supplied once. The evidence will require clear labelling and numbering in a systematic way that is readily accessible to readers from outside the subject team.

Section 1: Brief contextual description of the subject

  1. Major changes since last ISR: outline the organisational changes and the changes to provision, staffing, resources, accommodation etc.,
  • Reports of any Validation, Standing Panels, Minor/Major Modifications
  • Related action plans showing resolution of conditions/recommendations, as relevant
  • List of staff, staff CVs, responsibilities
  • Details of accommodation and learning resources
  1. Major changes since last ISR: outline the organisational changes and the changes to provision
  • Local Academic Plan for the School, Unit or Division
  • Off Campus Division Strategic Plan etc.,
  1. Key Challenges the subject faces in 2017-18
  1. Implications of changes, challenges and strategic aims for safeguarding academic standards and the quality of students' learning opportunities.
  1. Programmes and awards
  • Provide weblinks to the pdf of the programme specification for each programme (from the programme database)
  1. PSRBs
  • Provide reports of any accreditation visits since the last ISR
  • Provide weblinks to website of any relevant PSRB(s) and frameworks or standards, as applicable
  1. Off Campus programmes
  • Provide weblinks to the pdf of the programme specification for each programme (from the programme database)
  • Provide weblinks to website of the partner(s)
  • Most recent contract(s), last 2 ARPOs, most recent Operations Manual(s) for each partner and partnerships

Section 2: The subject’s track record in managing quality and standards

  • School Quality Enhancement Plans (SQEPs) for each of the last 2 full academic years
  • Programme Quality Enhancement Plans (PQEPs) /Programme Plans covering for each of the last 2 full academic years
  • Annual Course Monitoring Statistics and Data Analysis Reports covering the previous 2 full academic years
  • Previous ISR report(s)[2]

Section 3: Setting academic standards within the subject and

Section 4: Achievement of academic standards by students

  • External Examiners Reports covering for each of the last 2 full academic years
  • Correspondence with External Examiners showing programme teams’ responses to their comments
  • A representative sample drawn from assessment items sent to the external examiner showing the full cycle of assessment including assignment brief, marking and moderation, feedback to student, overall module mark list.
  • Visits by external examiners, including in OCD provision.
  • UNIPULSE data showing comparability of outcomes across on and off campus awards.

Section 5: The quality of learning opportunities within the subject

  • The extent to which the subject addresses designated University requirements such as TIRI, LTA Strategy, the SE Strategy and enhancement themes.
  • Marketing research and evidence of consultation through industry advisory boards etc., evidence of your approach to programme design and development
  • Marketing materials –Include weblinks and leaflets
  • Programme handbooks and module guides, staff details, local evidence of CPD in relation to LTA, staff performance review and appraisal. Peer review of learning, teaching and assessment, direct observation…and outcomes
  • List of physical and human learning resources, individual and group staff development activities includingLakeside event;evidence of staff and student CPD in digital literacy, student support and enrichment activities. Dedicated support activities, e.g. "Results Day", skills activities, residential workshops /visits, guest speakers, careers talks etc., student data showing evidence of achievement for each of the last 2 full academic years and how you have used this data
  • Local student engagement activities, committee minutes showing student engagement for each of the last 2 full academic years; student surveys for each of the last 2 full academic years. Programme surveys = UPSS, NSS, PTES or PRES; responses to the surveys and evidence of action/impact; MEQs for each of the last 2 full academic years and semester 1, of the current year;
  • Assessment Board Minutes for each of the last 2 full academic years; documentation showing the assessment strategy for each programme, evidence of approval of assessment briefs/ examination papers/ internal moderation verification of marking standards/ samples of written feedback to students on coursework and examinations/ policy for return of marked work
  • External examining arrangements: correspondence with external examiners; a representative sample drawn from assessment items sent to the external examiner showing the full cycle of assessment including assignment brief, marking and moderation, feedback to student, overall module mark list. Visits by external examiners, including in OCD provision. Data showing comparability of outcomes across on and off campus awards;
  • Programme Quality Enhancement Plans (PQEPs) / Programme Plans foreach of the last 2 full academic years; Annual Course Monitoring Statistics and Data Analysis Reports (DARs); School Quality Enhancement Plans (SQEPs) covering each of the last 2 full academic years;
  • Data showing the numbers of appeals and complaints received for each of the last 2 full academic years and the outcomes. Guidance provided for students regarding early resolution.
  • Placement handbooks and related standard documentation for each of the last 2 full academic years; samples of any reports from placement providers
  • A sample of dedicated documentation showing all stages of the PGR student life cycle from application, enrolment, induction, assignment of supervisory team, progress monitoring, stage reports and completion and examining reports. Retention and achievement data for PGRs. Student survey feedback through PRES and related action taken. Evidence of the quality of the research environment. Monitoring arrangements and oversight.

Section 5: The information provided about the provision & used by the subject team

  • The information policy for approval of information to staff, students and external facing information in print and electronic form; evidence of the policy in action; samples of information provided for staff, students and external audiences.
  • The views of students about the accuracy and reliability of information that they received before and since enrolment.

Section 6: Enhancement of students’ learning opportunities

  • Retention
  • Innovative approaches to student-centred learning
  • Student employability and enterprise
  • Digital literacy of staff and students
  • Teaching excellence

Section 7: Evidence: a numbered list of supporting evidence

  • Number the items of documentary evidence sequentially using a simple numbering system [001,002,003,004…100]
  • Insert the number of each item of evidence in the relevant column in the ISR template
  • Provide a simple list of the documentary evidence in section 7 of the ISR briefing paper
  • Provide N memory sticks each containing a full set of the numbered evidence in which the file names contain the corresponding number and corresponding title of the item of evidence.

List of abbreviations

ARPOAnnual Report on Programme Operation

CPDContinuing Professional Development

CVCurriculum Vitae

DARData Analysis Report

FHEQFramework for Higher Education Qualifications

HERHigher Education Review

ISRInternal Subject Review

LTALearning, Teaching and Assessment

MEQModule Evaluation Questionnaire (Survey)

NSS National Student Survey

OCDOff Campus Division

PGRPostgraduate Research

PQEPProgramme Quality Enhancement Plan

PSRBProfessional, Statutory & Regulatory Body

PTES Postgraduate Taught Evaluation Survey

PRESPostgraduate Research Evaluation Survey

QAAThe Quality Assurance Agency for higher education

SEOStandards and Enhancement Office of the University of Bolton

UPSSUniversity Programme Student Survey

INTERNAL SUBJECT REVIEW
Policy ref: tbc
Version number / 05
Version date / 29 April 2015 updated 040216; 160217; 171017; 191017
Name of Developer/Reviewer / Academic Registrar
Policy Owner (Group/Centre/Unit) / SEO
Person responsible for implementation (postholder) / Academic Registrar
Approving committee/board / Education Committee
Date approved / February 2015
Effective from / 1 September 2015
Dissemination method e.g. website / website
Review frequency / Annual update with three yearly substantive review
Reviewing committee / Education Committee
Consultation history (individuals/group consulted and dates) / ISR Panels during 2014-15 piloted the revised approach on a developmental basis
Document history (e.g. rationale for and dates of previous amendments) / Updated to refresh previous guidelines and to align process with UKQC

1

[1] UK Quality Code for Higher Education, Quality Assurance Agency

[2] Available on request from SEO