Open Awards

Access to HE Provider Standardisation Handbook

Welcome

Welcome to the Open Awards Access to Higher Education Provider Standardisation Handbook.

This handbook aims to give you a comprehensive overview of the processes required to plan and participate in standardisation activities.

This handbook will outline what is required by Open Awards Access Providers in terms of standardisation practices and provide guidance on how to plan and conduct effective internal standardisation activities.

The intended audience for this document is:

  • Access to HE Coordinators
  • Managers (including quality managers)
  • Internal Verifiers
  • Tutor / Assessors

We hope that you find this document useful and informative but if you have any questions or

queries about any information provided here, please feel free to contact our Quality team on 0151 494 2072 or email .

Definition

Standardisation

‘In the context of assessment, this describes the process through which those involved in assessment develop a shared understanding of the standard of student achievement that will be assessed as meeting the requirements of the assessment outcomes available. (In the context of Access to HE pass and grades of merit or distinction.) Such assessment decisions cannot be made in a way that is reliable and valid unless the underpinning assessment structures (unit design, assessment design and so on) are of the highest quality, so standardisation of these aspects of the assessment infrastructure to ensure their quality is also a key component to be addressed’.

QAA October 2016

Standardisation is a vital component of any robust quality assurance system and as a condition of

centre approval all approved Open Awards Access Provider organisations agree to participate in standardisationactivities.

The aims of standardisation:

  • ensure the consistency of the award of credit to students across different assessors/moderators/modes of delivery/provider and to agree the standards to be achieved.
  • compare the assessment strategy and planned assessment tasks used across a unit/s to ensure that they:

o allow for the generation of sufficient/appropriate/authentic evidence to be produced

o give fair access to assessment and prevent any barriers to achievement and

respect for equality and diversity issues

  • ensure that the assessment tasks are written in language that is appropriate to the level and are free from bias
  • ensure consistency in the assessment judgements made by assessors on the available evidence
  • ensure that adequate constructive feedback is given students
  • ensure that adequate constructive feedback is given to assessors which demonstrates therobustness of the internal moderation of the assessment judgements
  • identify and share best practices

The purpose of standardisation is to:

  • promote and achieve consistency of assessment and moderation practice
  • ensure consistency in the application of level and credit across open awards Access
  • provision
  • identify and develop agreed points of reference by which standards can be referenced and agree processes for them to be applied and reviewed
  • ensure that a full range of assessment activities allow for the generation of appropriate
  • assessment evidence
  • inform the quality improvement cycle by identifying training and development issues
  • share and disseminate best practices

Introduction

Open Awards standardisation strategy for Access to HE provision will ensure that assessment is consistent and accurate standards are being applied and maintained across all provision.

The development of robust standardisation processes for Access to HE Provision, which can be applied both within centres and nationally,will provide the evidence of consistent practice. The process outlined in this handbook will ensure that standards arecomparable across:

  • providers (and satellite sites)
  • different types of providers(colleges / private providers)
  • different subject areas
  • modes of delivery (blended / distance learning)

…..and over time.

As a minimum requirement each provider must:

  • Plan and undertake standardisation of internally set tasks and the outcomes of internal Assessment
  • Contribute assessed material to open awards standardisation activities and events upon the request of open awards Access to HE staff.
  • Allow subject specialist staff to participate in and attend standardisation activities organised by Open Awards

Types of Standardisation Activities

1.Provider standardisation (internal standardisation)

Where there is more than one tutor / assessor delivering the same course or similar courses and making assessment decisions and recommendations for the award of credit to learners it is essential that the internal verification process includes an opportunity for the standardisation of their practice.

The internal verifier/moderator for the course(s) has the responsibility for arranging and facilitating standardisation events. Guidance for planning and conducting internal standardisation events and exercises is outlined in this handbook.

As part of compliance monitoring activities, Open Awards may review Provider standardisation activities via scheduled spot checks.

2.Regional Standardisation

Please see below a list of the regional standardisation events for this academic year:

Pathway / Date / Mode of Standardisation Activity / Location / Lead
Science Health and Engineering / Tuesday 28th February / Annual Standardisation Event / Open Awards / Katherine Wales
Humanities and Social Sciences / Monday 23rd January, end Friday 28th April / Annual Standardisation Event 28th April / Open Awards / Katherine Wales
Electronic Standardisation Event 23rd Jan – 28th April / Online / Katherine Wales
Julie Mizon
Business Management and ICT pathway / Thursday 30th March / Annual Standardisation Event / Open Awards / Katherine Wales
Creative and Performing Arts pathway / Monday 27th February / Annual Standardisation Event / Open Awards / Katherine Wales

Alongside these dates, we will also be offering regional standardisation activities during our Access forums.

The Access to HE forum Dates for 2016-17 are:

Date / Location
7th November / Manchester
Stockport College
26th January / Liverpool
23rd May / Manchester
Salford City College

All Pathways, modules and units will be subject to standardisation and review on a three year rolling programme. Please refer to the Standardisation/Unit Revision Plan (Appendix 1)

There are two ways in which you can participate in regional standardisation events:

•to contribute sample materials to the event, and

•to participate in the event by attending and contributing samples.

3.E: Standardisation

Alongside the 'Live' standardisation events, this year Open Awards will be piloting online standardisation activities with the Humanities and Social Science Pathway. Online standardisation activities will take place from January 23rd 2017 to April 28th 2017. Your centre will be provided with a Moodle login password, which will give you access to the standardisation activities, training and opportunities to share best practices.

There are two ways in which you can participate in e:standardisation activities:

•to contribute sample materials to the activities, and

•to participate in the event by participating electronically and contributing samples.

Fig One: Online Standardisation Platform

4.National Standardisation / Inter-AVA Standardisation

Samples of learners evidence of achievement used for regional standardisation will be made available for national standardisation events as appropriate.

Open Awards supports working with other awarding organisations to raise quality standards and assure that specified levels of attainment for a qualification are consistent and reliable. It is the aim of Open Awards to facilitate an inter-AVA standardisation event this year (2016-17).

Outcomes for all national and inter-AVA standardisation activities will be reported to you through the dissemination of:

  • A standardisation report
  • Exemplars/ Sharing of best practices

5.Moderator standardisation

External moderators will be required to identify and collect, during the course of moderation, evidence/examples of:

a) unit assessment plans and assignment briefs

b) assessed work graded at pass, merit and distinction (to include ‘typical’ and

borderline grades)

Assessment plans and assignment briefs will be used to allow moderators to compare assessment demands across provision. Assessed work will be used in standardisation activities. Such activities include comparison and discussion in subject ‘clusters’ during moderator training and agreeing excellent practice to disseminate across provision.

Please note that the standardisation activities for each level are related and will influence and be influenced by the outcomes of each other.

Sample Retention and Sampling Strategy

Open Awards lead and pathway moderators are responsible for collecting samples (‘clean’ samples) to be used for standardisation activities. Samples may be requested either:

  • In person, via pathway moderation visits
  • by email or telephone

The sample should support the evidence of achievement for a whole unit of assessment and should include:

  • A copy of the unit descriptor including unit code.
  • Whole assessment strategy for the unit being standardised including tasks/assignment

briefs/benchmarks. These should be mapped to the assessment criteria for the unit.

  • All of the evidence generated by a learner(s) against all of the assessment criteria for

the unit(s).

  • Clear record of the assessment decision including feedback to the learner in relation to

the achievement / non achievement of the assessment criteria/grading criteria (if applicable).

  • Internal verification records including feedback to the Assessor and the Internal Verifier’s judgments on the security of the award of credit.
  • Evidence where possible of subsequent monitoring of any actions identified by the

internal verification process.

  • Copy of completed RA1 if a Reasonable Adjustment has been made at the discretion

of the centre for any learner.

Samples will be taken from current learner evidence, logged and stored at Open Awards electronically. This is to allow Open Awards to build up a sample in order to compare standards over time.

Samples will be retained by Open Awards for the lifetime of the qualification to ensure samples are available to allow a comparison of standards over time.

You will be required to retain samples of learners work for a three year period. Providers will also be required to retain the following records for a three year period from the end of the year to which they relate:

  • Names of learners, dates of birth and contact address
  • Title and accreditation number of each qualification and unit studied
  • Name(s) of Assessor(s)
  • Assessment records, including assessment decisions and reasons for the decision
  • Names of Internal Verifier(s)
  • Full records of the internal verification process i.e.:
  • The internal verification plan;
  • Minutes of internal verification meetings;
  • The sample taken by the internal verifier, normally in the form of a spreadsheet/matrix;
  • Any resulting action points and details of how and when these have been or will be addressed.

Outcomes of Standardisation

The outcomes of standardisation activities are used to:

  • Improve the assessment practice of assessors and bring about consistencyof assessment decisions to safeguard the integrity of the award of credit and grades
  • ensure that equivalent standards and requirements for achievement apply on differentAccess to HE courses which lead to Diplomas, and across different studentcohorts
  • Improve consistency in the judgements of moderators
  • Improve the quality of feedback to assessors
  • Improve the quality of feedback to students
  • Monitor and evaluate equality and diversity issues in the design of units, qualifications, andassessment strategies and use these to inform unit and qualification review
  • Identify training needs to inform Open Awards annual training programme.
  • At a national level, influence and continuously improve unit and qualification design and anyprescribed assessment tasks within the qualification

Planning and Conducting an Internal Standardisation Event

What is this guide for?

The following is intended as a guide for the implementation of aninternal standardisation event within your Access to HE Programmes.

How many standardisation events should I plan for each year?

The frequency of internal standardisation activities depends on your internal quality assurance

processes. This will depend on factors such as the:

  • size of the centre (inc satellite sites)
  • number of tutors/assessors and internal moderators at the centre
  • number and range of Access courses/Diplomas delivered at the centre
  • modes of delivery
  • the number of new or inexperienced staff delivering the Diploma qualification

Grading

Standardisation activities should giveparticular attention to the differentiated judgements that are indicated by the award of differentgrades. The activities should provide a focus on judgements made about the use ofgrade indicators of pass, merit and distinction, as they are applied for each of the gradedescriptors, in different subject areas.

Standardisation activities must identify the respective responsibilities of the participants and outline

expected outcomes. The activities will allow participants to:

  • consider standards across a range of different types of evidence e.g. essays,

short answer questions, tutor observation sheets etc

  • confirm the validity of evidence
  • assure consistent approaches to assessment decisions
  • consider whether assessment methods are relevant and appropriate and pitched at the

right level for the learner

  • review the differentiated judgements that are indicated by the award of different grades
  • review the standards of assessor feedback
  • review IM assessment decisions and feedback to assessors
  • identify and share good practice
  • confirm standards across time by examining materials and outcomes from previous

standardisation events and activities.

Who should attend a standardisation event?

This will depend on the size of your organisation and Diplomas you offer. A facilitator must be

identified to plan and conduct the event and they should be prepared to compile a report of the

discussions, agreements and actions (See Appendix Two: Provider Standardisation Event Activity Sheet and Appendix Three: Provider Standardisation Event Feedback Form).

All tutors, assessors and internal moderators who arecurrently involved or who may be involved in the planning and delivery of theAccessDiplomas must be invited to the event(s). Feedback to tutors, assessors and internal moderatorswho cannot attend will form part of the actions following standardisation.

Unit Selection

Your standardisation strategy should include a plan which sets out which units are to be

standardised ensuring that, over time, all units and Diploma courses reviewed through this

process. This will be particular to your organisation and will of course depend on the nature ofthe courses and the overall number of units you offer. This may also depend on what has been requested by

Open Awards.

Sample Selection

Clear instructions must be agreed and given to assessors and/or internal moderators about what

should be included in the sample. There should be a clear indication, for the rationale for the sample choice which may include any of the following factors:

  • strong evidence of achievement
  • weak evidence of achievement and the reason for that judgement
  • evidence which is not sufficient to demonstrate achievement of all the assessment criteria
  • the use of an innovative assessment strategy or a new or innovative assessment method
  • an example of good and constructive feedback to a student
  • an example of good and constructive feedback from the internal moderator to the assessor
  • evidence from a student who has not achieved and why
  • any other valid reasons for the selection

The sample should include evidence assessed by each of the assessors involved and evidence

moderated by each of the internal moderators

Documents required for the meeting

  • Provider Standardisation Event Activity Sheet (Appendix 2).This will report on individual units so enough copies should be made available depending on what is to be covered. Provider Standardisation Event Feedback Form (Appendix 3).A copy should be available for each participant, Access to HE Coordinator and Quality Manager.

All documents identified above should be made available to the Pathway and Lead Moderators, during their visits.

The following documents will also be a useful source of reference for participants and are available

for download from open awards website:

  • Assessment Definitions and Level Descriptor documents
  • Access Tutor handbook containing relevant Diploma Rules of Combination
  • QAA Grading Scheme Handbook September 2013

Standard Agenda

The following is an example of a possible suitable agenda for a standardisation meeting:

Agenda:

1. Overview and expected outcomes

2. Standardisation (assessment) - consideration of the samples provided and completion ofProvider Standardisation Event Activity Sheet (Activity 1)

3. Verbal feedback – agreeing the standard/adequacy of the evidence and supporting

documentation/best practice under review

4. Standardisation (Grading) - consideration of the samples provided and completion of Provider Standardisation Event Activity Sheet (Activity 2)

5. Verbal feedback – agreeing the standard/adequacy of the evidence and supporting

documentation/best practice under review

6. Selection of sample – to be used for regional standardisation if required

7. Facilitator feedback and agreement of any actions

Conducting the event

Overview and expected outcomes

The facilitator should at the outset of the meeting ensure that participants are clear about thepurpose of the standardisation event and their role in it.

  • Samples should be provided as a vehicle to allow you to discuss and make recommendations and agree performance standards.
  • The overview should also include how the Provider Standardisation Event Activity Sheet (see Appendix 3) should becompleted and an explanation about how that information will be used in producing the Provider Standardisation Feedback Form (see Appendix 3).
  • Reference should be made to any guidance documents and how they can beused by the participants during their consideration of the samples.
  • Remind participants that the objective is not to second mark the work but to consider whether they agree with the judgements and why or why not.
  • Participants should be encouraged to challenge decisions and therefore it is perhaps useful to consider some “ground rules” or agreements before this exercise is undertaken.
  • As best practice, Open Awards would recommend the implementation of using ‘blind’standardisation exercises (i.e. where samples have been stripped of feedback and assessmentdecisions).

Consideration of Samples

Each participant should examine each sample and record their responses to the questions on the