Assessment for Learning Project

2013-2016

Contents:

Section 1: Introduction 2

Section 2: Assessment Provision – The Plymouth position 2013 3

Section 3:Recommendations 13

References 14

Pauline Kneale

Jane Collings

January 2013

Update: March 2015

Progress and activities towards achieving the recommendations have been updated (p 13 -15). Schools and programmes in the university are continuing to embed Assessment for Learning in practice. There is a review commissioned by Learning Support & Wellbeing on the impact of the reduction of the DSA (2015/16) on students with additional needs. This will have implications for assessment.

Introduction: October 2013

This report updates Plymouth University on the Assessment for Learning Project after nine months. The report includes the original material, further examples of assessment practice at Plymouth and progress against the recommendations.

Assessment for Learning 2013-2016 Project

Section 1:

Introduction:- October 2013

The AfL project has been underway for 18 months. This report updates the previous document with additional examples of practice, and progress against the Recommendations (p12).

The PedRIO ED team continue to run sessions with School and programme teams to enable appreciation of equality of access and inclusive assessment. There have been over 30 sessions to date with School and Programme groups, with all faculties represented.

There have been some excellent examples of changed practice put in place this session. Some Schoolsare recognising or increasing their use of the University Regulation that the form of retake assessment does not have to be the same as the original assessment, but must meet the stated learning outcomes. This is very welcome to students who otherwise have the expense of travelling to Plymouth for resits in August.

Further briefings and workshops can be arranged through

Or see

Introduction:- January 2013

Research with University colleagues during 2012-3 identified opportunities to evolve our assessment and feedback processes. This is a crucial NSS indicator; strength in this area would reduce our reputational risk ahead of the next QAA Institutional Audit.

The attached report Assessment Provision –The Plymouth position 2013 is the result of research with a wide range of students, professional and academic staff. Each time we considered an issue, more concerns surfaced. The report gives a flavour of the issues raised, the difficulties that students encounter, and makes a suite of recommendations.

It is clear we need to be better at supporting students with diverse needs; improving the assessment experience of all students will enhance everybody's experience. The University has an enviable reputation in supporting students with particular needs through one-to-one enablers and other specialised provision. However, as our disabled student numbers have grown we have fallen behind in providing effective and research informed assessment and feedback practices. We should prioritise inclusive assessments, and minimise our MAP (modified assessment provision) needs, so that all our students are treated as equally as possible in all aspects of their programme

Disabled students working towards different assessments miss opportunities for group support and are treated as ‘other’. The same learning, support and inclusive assessment opportunities (MAP-free assignments) would provide a common, collegiate, level playing field for all our students, would streamline many processes and be transparently fair.

There is some excellent practice in Plymouth with some programmes having a range of diverse assessmentsspaced throughout the year and there are a small number of modules which have a choice of assessmentmethods where students select the assessment methods which best meet their individual needs. Where this is already happening in the University, the student experience is exceptionally positive and MAP needs are rare. This practice should be extended to all programmes.

The research leading to this report identified benefits in changingthe pattern of level 4, year 1 teaching to help students develop their understanding of academic standards and reduce the numbers referred and resitting modules. Moving away from terminal examinations would have the additional advantage of minimising staff workloads in providing written feedback on examination scripts in June. The Plymouth University Curriculum Enrichment Project (PUCEP) is a separate but related proposal.

We must revise and where appropriate consolidate all the documentation (there are currently over 30 separate documents) relating to assessment and feedback. These will be thoroughly overhauled and made available on a single consolidated website, which meets the QAA accessibility requirements. In addition, the new Teaching, Learning and Student Support Handbook and updated Teaching and Learning website when completed will become the University resource for all assessment and feedback matters.

Pauline Kneale

Assessment Provision –The Plymouth position 2013

Section 2: Report and Recommendations

“We face a system of assessment that has been subject to slow incremental change, to compromise and to inertia...... Assessment is not sufficiently equipping students to learn. We are failing to prepare them for the rest of their lives. ” Boud & Falchikov (2009)

“Exams assess surface learning. They do not assess teamwork, leadership, creativity or lateral thinking...... it is no longer enough for graduates just to be knowledgeable.” (Race 1999)

1. Overview

Ambition – To provide all students with equal learning and assessment opportunities.

The numbersof disabled students studying at Plymouth University are increasing. The HESA disability descriptors 2012/13record 2,262 students with a declared disability at Plymouth University and UPC, over 8% of all students.

In the academic year 2011/129.7% of Plymouth University students claimed the DSA compared to the national average of 5.9%. HE institutions withhigher percentages of students in receipt of theDSA are Performing Arts or Agricultural Colleges.

In 2012/13 disabled students at Plymouth University byHESA categories included:

1302 specific learning difficulties
238 mental health condition
185 a long standing illness or health condition / 60 social/communication impairment
24 serious visual impairment
47 serious hearing impairment
60 physical impairment

In May 2013 examinations period, 1,020 disabled students at Plymouth University required modified assessment provision (MAP) involving over 10,829 hours of examinations. These MAP examinations required over 100 invigilators and exam rooms in 70 different locations. This entailsconsiderable actual and hidden costs and additionaltime commitment for academics, administrative and exam invigilation staff. The examinations office estimate that 90% of their time is taken organising modified assessment provision.

Plymouth University has some excellent practice in supporting disabled students in recruitment and their learning.This probably influences the numbers of disabled students who chose to study in Plymouth. Plymouth University was a key partner in the 2003 – 2005 HEFCE funded SPACE project which aimed to improve the assessment provision for disabled students. The SPACE project resulted in the comprehensive publication ‘Inclusive Assessment in Higher Education: A Resource for Change’ (Waterfield & West 2006 ). This publication offers guidance on innovative assessment practice which is still applicable today.

“We need parity of assessment experience through strategic change,embedded, consistent practice rather than a ‘bolt on’ or ‘ad hoc’ provision”

(Waterfield & West 2006 ).

The SPACE project identified the need to support students with disabilities and to provide satisfactory student experiencefor all using an ‘inclusive approach’model of assessment. In inclusive assessment a range of assessment opportunities are made available to all students who chose their assessment methods. Currently most programmes at Plymouth either use the ‘contingent approach’ where there is provision of special arrangementswithin existing systems or the ‘alternative approach’which offers different assessment methods as a bolt-onfor a minority of disabled students. Where staff have adopted the ‘inclusive approach’, MAPs are not needed.

The purpose of this review and the recommendations reported here is to:-

  • Evidence our current position with respect to our legal and student experience expectations
  • Identify a programme of enhancement to our policies, procedures and training that will lead to:
  • Enhanced experience of assessment for all students
  • Reduction in the stress and anxiety experienced by disabled students in the assessment process
  • Assessment practices that are more aligned with workplace practices
  • Clarity for staff on the design and delivery of fair and objective inclusive assessments
  • Cost savings from the reduction of MAP, examination adjustments and case conferences.
  1. Establishing the 2012 assessment position at Plymouth

Evidence was gathered through:

  • ScrutinisingPlymouth University data, policies,reports,assessment guidance and researching national and international assessment practices.
  • Informal individual and group interviews with 27 staff including Associate Deans, programme leaders, senior lecturers, business managers, faculty registrars, administrators, the Disability Assist team, examinations office staff, student representatives, student advice workers and students.
  1. Findings

3.1) Interviews with academic and support staff showed:

1. Alack of clarity inthe processes and procedures associated with assessment. Academic and support staff are uncertain where to locate current assessment regulations and guidance.Staff report difficulties locating information about students’ Student Support Documents (SSDs).

2. Staff were concerned about designing appropriate assessment alternatives and asked for guidance on assessment tariffs and equivalences.

3. There is inconsistent practice across the university relating to assessment.Within some programmes staff ensure the provision of inclusive assessment and as a result have few complex MAPs. In other programmes there appears to be culture of mistrust towards students who require modified assessment provision.

Staff identified specificareas of uncertainty regarding assessment including:-

  • Modified assessment provision
  • Alternative assessment
  • Inclusive assessment
  • Innovative assessment
  • Adjustments in assessment
  • Assessment equivalents
  • Choice of assessment methods
  • The assessment requirements of professional bodies & QAA subject benchmarks
/
  • Student Support Document
  • Extenuating circumstances
  • Chronic illness forms
  • The use of case conferences
  • Issues of ‘fitness to study’, ‘fitness to graduate’ and ‘fitness to practice’
  • The role of the Claro enabler
  • The use of assistive software
  • Location of regulations

3.2)Existingassessment modifications lead to unreasonable student experiences:-

A student with a MAP of 100% extra exam time andwith 25% rest breaks involves 7 ½ hours foreach 3 hour exam. Combined with the additional invigilation costs the process is untenable for both the student and the university.

A student who is unable to take five 2 hour exams due to exam stress and phobia was given ‘home papers’ with a 2 week time limit and one time constrained exam in their ‘own room’. During the exam the student experienced a panic attack and needed to be escorted home by the invigilator.

Long thin modules with 70% summative assessmentleads to exam overload in the May examination weeksfor over 1000 students. We should consider alternatives that reduce the stress on our students and spreads the workload more evenly. Introducing short modules is one option.

Guidance notes on alternative assessment in one faculty advise a 2 hour unseen Stage 4 exam assessmentmight be replaced by four 2500-word essays. This is not a comparable assessment experience.

A student who did not receive their MAPs subsequently sued the university. The Court awarded the student a sizable financial settlement. We need to be fair to all students assessing their ‘learning’ in a manner that is compatible with their disability.

3.3) Current assessment practice offersstaff challenges.

There are concerns over issues of ‘fitness to study’, ‘fitness to graduate’ and ‘fitness to practice’.

Academic staff report capacity issues. Where programmes have large cohorts of students,some staff report exams were one of the few viable assessment methods.

Where modified assessment provision is identified for a student,academic staff are required to setalternative or adjust assessments. Staff experience difficulties in identifying appropriate inclusive assessmentmethods and report it is a time consuming process.

Staff stated they are unclear of the alternativemethods they can recommend and what constitutes an appropriate assessment. Staff requested more support, a predetermined framework or principles of inclusive assessment practice.

There are staff concerns over alternative assessment, plagiarism and academic dishonesty.

Staff report that when they provide alternative forms of assessment they often have the responsibility for additional arrangements including timetabling, rooms, invigilators, amanuensis, own rooms, ICT and other adjustments. All MAP arrangements for exams and in-class tests are now organised by the examinations office.

Staff report that they can arrange MAPs with sufficient notice but there are problems of late notification of a MAP by students.

Assessment equivalence is another concern with varied practice across programmes. One school has a guidance document ‘Principles to be considered when designing alternative assessments’. This type of guidance is not consistent across the university.

Some staff are concerned at the level of ‘over assessment’ of students and the volume of summative assessment especially in students’first year of study.

Some staff want to see an increase in formative and continuous assessment which count towards the final module mark but with pass or fail as the only grades.

The development of self and peer assessment including student designed assessment criteria could offer students insight in assessment and ownership of the process and reduce staff workload.

Staff report that many students choose modules on the basis of preferred assessment methods rather than itscontent . The SPACE project (2006) researched the most popular assessment modes and reported the five preferred methods:-

1.Continuous assessment,

2.Coursework with discussion,

3.Research projects,

4.Essay assignments,

5.Multiple choice questions.

3.4) Modified Assessment Provision (MAP)

The Student Support Document (SSD)contains all the information about a student’s needs, support and assessment adjustments required throughout their programme of study. Modified Assessment Provisions (MAPs) are used for any assessment that contributes to the final mark of a module. The purpose of these arrangements is to facilitate equality of access for students who may be disadvantaged when assessed.Over recent years there are increasing student numbers requiring MAPs.

During May 2013 there were 1,020 students requiring MAPs in exams. Of these, 121 studentshad complex MAPs requiring taking examinations in their own room with an invigilator and an additional 88 students required their own room and a reader or a scribe.

In degree programmes where inclusive assessment is embedded into the course design MAPs and Case Conferences are rarely used. This is excellent practice the University can build on.

3.5) Examinations

In the Faculty of Arts and Humanities there are very few exams complex MAPs are seldom required. In other faculties there are programmes with up to 70% time controlled summative examinations with many complex MAP issues.

In-class tests and Computer Aided Assessment (CAA) are increasinglyused as a method of assessment. These methods can be time efficient and often require only simple MAPs such as extra time and ICT support.

One academic reported the logistics of organising computer based tests was problematic dueto resourcing issues; locating 300 computers at one time was not possible. However other universities with a module cohort of a 1,000 usecomputer based assessments.Software such as QuestionMark Perceptionenables students to taketheir assessments at different times over 2 days from any location, overcoming limited ICT resources and offers students instant feedback on their achievements.

Some academics delivering modules with large student cohortsinsist that exams are a necessity.Students are required to demonstrate answering questions under time constraints. There appears to bea lack of understanding of the use of inclusive assessment methods with large student cohorts.

A number of academics cited academic dishonesty as the main reason to hold exams. Electronic submission though enables checking for plagiarism. Turnitin is accepted internationally as appropriate software to detect plagiarism and provide positive assistance in inclusive assessment practice. Good assessment design can minimise plagiarism.

During the examinations period the exams office requires up to 70 rooms in a variety of locations for MAP assessments. Thesesmall group or individual room exams require over 100 invigilators. It is difficult to control the environment around many rooms on campus.In 2012 there were 20 formal complaints from these students about rooms, location, interruptions, noise and invigilators. One exam had to be terminated due to double booking. The examination office reportsthat the main MAP exam hall is a distracting, noisy environment; with students leaving at different times, asking questions, requiring comfort breaks etc.

“Unfortunately I feel the special examination arrangements put in place on this occasion assisted only in exacerbating my disability further.” Student with MAP

The continuing emphasis on traditional handwritten examinations is not aligned to the 21st century workplace where reports, research etc. are completed on electronic devices, group research, presentation and report writing are common practice. (Mogey et al 2008)

3.6) Case Conference

Some faculties are arranging over 65 Case Conferences each academic year. The process is time consuming and has considerable cost implications for the University.

Case Conferences are held to ensure action is taken to foster student achievement. Triggers for case conference include a studentexperiencing multiple difficulties,submitting claims for extenuating circumstances for the same reason on two occasions,or a student has problems across a number of modules.Students will be recommended to see Disability Assist, Counselling Service or a Student Union Advice Centre worker prior to the Case Conference being held.

The Case Conference is convened by the faculty registrar with membership that includes the student, programme manager, appropriate professional staff and support service staff. The Case Conference discusses the student difficulties and recommends future action. These actions are discussed with the external examiner and forwarded to the Award Assessment Board for approval.