Assessment Standards: A Manifesto for Change
Professor Margaret Price
Berry O’Donovan
Dr Chris Rust
Jude Carroll
OxfordBrookesUniversity
Assessment standards are at the heart of the assessment process underpinning everything from assessment strategy to design and marking. For students, assessment standards provide guidance for their learning, allow them to monitor their progress and will ultimately be used to judge their performance. Yet there has been little serious consideration of the nature of assessment standards or their effective communication between stakeholders (primarily staff and students but also employers, professional bodies and government). However amid accusations in the media about dumbing down there have been wider concerns expressed about assessment and assessment standards not least in the Burgess report which identified a range of problems, including reliability and communication of assessment standards.
In response to these concerns the Assessment Standards Knowledge exchange (ASKe), the Centre for Excellence in Teaching and Learning, has established an international forum of experts on assessment in higher education (the Weston Manor Group) to consider issues around assessment standards and make proposals about changes to policy and practice. The group is clear that learners who engage in assessment are entitled to fair and transparent assessment practices and standards. However, it is the group’s view that assessment standards reside in the practices of academic and professional communities and in the nature and level of the tasks in which learners engage, rather than being fully describable in statements of learning outcomes or marking criteria. In addition, the group believes that there is an over-emphasis on the measurement of learning, often at the expense of assessment for learning. These fundamental evidenced-based beliefs underpin the Manifesto for Change which the group is putting forward with a view to stimulating debate across the sector and, more importantly, as a step towards bringing about necessary changes in policy and practice. The rationale and evidence for the tenets of the manifesto are presented in this paper.
Tenet 1
Acknowledgement of the importance of assessment in student learning is well established in the literature
“Assessment is at the heart of the student experience” (Brown, S. & Knight, P., 1994)
“From our students’ point of view, assessment always defines the actual curriculum”(Ramsden, P.,1992)
“Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates...... If you want to change student learning then change the methods of assessment”(Brown, G. et al., 1997)
and yet there is considerable evidence that the assessment used does not set standards that are appropriate for high level complex learning that one would expect to be central to higher education.
“The types of assessment we currently use do not promote conceptual understanding and do not encourage a deep approach to learning………Our means of assessing them seems to do little to encourage them to adopt anything other than a strategic or mechanical approach to their studies.” (Newstead, 2002, p. 3)
“Conventional assessment procedures are unable to do justice to the most important outcomes of any educational process worth the name” (Raven, 1991, p. 1)
“Even when lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise” (Gibbs, 1992, p. 10)
Knight (2002) reviewed and discussed the assessment process in higher education and concluded that summative assessment processes are in “in disarray” ( p. 275). Such disarray requires the review and reappraisal of the curriculum and in particular the role of assessment. This underpins the first tenet of the manifesto:
The debate on standards needs to focus on how high standards of learning can be achieved through assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning.
Tenet 2
While acknowledging that assessment retains a role for measuring performance, it must be acknowledged that our current systems, which focus on marks and grades, are not working. Rust (2007) puts forward a number of embedded beliefs that evidence the fallibility of marking systems:
- Belief that it is possible to distinguish the quality of work to a precision of one percentage point (Elander & Hardman, 2002)
- Belief that double-marking will ensure fairness and reliability
- Belief that consistency can be achieved through conformity and simple numerical rules (e.g. level 1 essay 3,000 words, level 3 essay 5,000; or no more than two pieces of assessment per module)
- The combination of scores, which obscures the different types of learning outcome represented by the separate scores
- The distortion of marks by the type of assessment (e.g. coursework compared to examination) and the actual subject discipline/s studied (Yorke et al., 2002; Bridges et al., 2002)
- The distortion of resulting degree classifications by the application of idiosyncratic institutional rules (e.g. Armstrong et al., 1998).
Despite this there has been an emphasis on the quest for reliability in marking which has had a detrimental effect on the role of assessment standards in promoting and developing learning.
“This quest for reliability tends to skew assessment towards the assessment of simple and unambiguous achievements, and considerations of cost add to the skew away from judgements of complex learning” (Knight, 2002b, p278)
“…summative judgement itself is the problem” (Burgess, 2007, p. 8)
“Many research findings indicate a declining use of deep and contextual approaches to study as students progress through their degree programmes. (Watkins & Hattie, 1985; Kember et al., 1997; Zhang & Watkins, 2001)
“…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead, 2002, p. 2)
This evidence underpins the second tenet of the manifesto:
When it comes to the assessment of learning, we need to move beyond systems focused on marks and grades towards the valid assessment of the achievement of intended programme outcomes.
Tenet 3
The role of assessment standards in supporting assessment for learning must be supported by commonly held understandings of the standards. As Sadler (1989) points out,an indispensable condition for improvement in student learning is that “the student comes to hold a concept of quality roughly similar to that held by the teacher”(Sadler, 1989).
However, the dominant model of transparency and sharing of standards in higher education relies on explicit criteria and level descriptors which have been shown to be inadequate because many aspects of quality cannot be communicated through explicit criteria. Admittedly there are some standards that are amenable to explicit articulation. Sadler (1987) labels these as regulative and logical criteria where “standards can be defined in terms of well-defined outcomes” (Sadler, 1987, p. 70). However, he also identifies prescriptive and constitutive criteria which refer to matters of degree and “It would be difficult or impossible to guess the educational level at which they are applicable… ” (Sadler, 1987, p. 70). Such types of criteria are often interdependent and can only be assessed using holistic/professional judgement (Sadler, 2008). Meaningful understanding of standards, and therefore holistic judgements, requires both tacit and explicit knowledge (O’Donovan et al. 2004). However, tacit knowledge is experience-based and can only be revealed through the sharing of experience—socialisation processes involving observation, imitation and practice (Nonaka, 1991)— which means that most criteria are socially constructed and this requires the sharing of tacit knowledge over time (O’Donovan et al., 2004; Rust et al., 2005).
This evidence underpins the third tenet of the manifesto:
Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment.
Tenet 4
Coming to understand assessment standards requires the development of tacit knowledge which involves the sharing experience and ‘making sense of the world’ which is a social and collaborative activity (Vygotsky, 1978). For both staff and students developing an understanding of assessment standards applied to high-level complex learning can only be understood through active engagement with members of the disciplinary community that ‘holds’ those standards. We know that some processes, such as feedback, exist to support the development of understanding but passive receipt of feedback has little effect on future performance (Fritz, et al., 2000). It requires active engagement to foster in depth understanding e.g. dialogue and participatory relationships are key elements of engaging students with assessment feedback (ESwAF FDTL, 2007). Staff may more readily have the opportunity to actively engage with standards especially when required to mark work but students’ opportunities to closely engage with the community are not so obvious. However there is clear evidence that it is active involvement and participation that fosters engagement, understanding and achievement
“participation, as a way of learning, enables the student to both absorb, and be absorbed in the culture of practice” (Elwood & Klenowski, 2002, p. 246)
“The only common factor in a study of departments deemed excellent in both research and learning and teaching is high levels of student involvement”(Gibbs, 2007)
“The most significant factor in student academic success is student involvement fostered by student/staff interactions and student/student interactions”(Astin, 1997)
This evidence underpins the fourth tenet of the manifesto:
Assessment standards are socially constructed so there must be a greater emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards. It is when learners share an understanding of academic and professional standards in an atmosphere of mutual trust that learning works best.
Tenet 5
Developing an understanding of complex standards through a process of socialisation requires time. Important aspects of complex, high-level learning outcomes can only be achieved when students are allowed time to ‘come to know’ the standards in use by the community. Standards will be understood through practice, observation and imitation. We know that slowly learnt academic literacies require rehearsal and practice throughout a programme (Knight & Yorke, 2004) and students need to engage as interactive partners in a learning community, relinquishing the passive role of ‘the instructed’ within processes controlled by academic experts (Gibbs et al., 2004). To support the slow learning that leads to high-level complex learning there needs to be integrated and coherent progression based on programme outcomes. A focus on programme outcomes and standards rather than incremental outcomes provides students with a more holistic view and where there is a greater sense of the holistic programme students are likely to achieve higher standards than on more fragmented programmes (Havnes, 2007).
This evidence underpins the fifth tenet of the manifesto
Active engagement with assessment standards needs to be an integral and seamless part of course design and the learning process in order to allow students to develop their own, internalised, conceptions of standards and monitor and supervise their own learning.
Tenet 6
Assessment standards underpin the assessment process and yet they are not amenable to explicit articulation and require active participation over time to develop deep understanding. However, their significant role in both guiding learning and measuring achievement requires safeguards and mechanisms for consistency. Such safeguards could be delivered through the greater professionalization of university teachers but there has been slow progress in this regard, in particular there has been limited attention paid to professional assessment practice. In addition, changes in higher education (e.g. massification, reduced units of resource, expectations of increased productivity in staff) threaten the ‘health’ of disciplinary communities and their ability to share and exemplify professional judgement. It could be argued that in the UK the external examiner system protects and promotes the consistency of assessment standards but Newstead and Dennis(1994) show that reliance on the external examiner system to mediate standards within the system is misplaced.Consequently there must be processes established that will provide confidence in the assessment standards dependent upon professional judgements.
This evidence underpins the sixth tenet of the manifesto:
Assessment is largely dependent upon professional judgement and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities.
Summary
The Weston Manor Group believes that the current dominant logic of the explicit approach to assessment standards is inadequate and must be challenged. In addition the quest for reliability is getting in the way of learning when our main concern should be enabling students to achieve high- level, complex learning.Active involvement by staff and students in the learning community is essential to reach common understandings of assessment standards. To achieve this there will need to be a review and evaluation of the allocation of time and resources within learning teaching and assessment in higher education, but, the Weston Manor Group believes, such change is achievable.
References
Astin, A. (1997) What matters in college? Four critical years revisited.San Francisco: Jossey Bass
Brown, G., Bull, J. & Pendlebury, M. (1997) Assessing student learning in higher education.London: Routledge
Brown, S & Knight, P. (1994)Assessing Learners in Higher Education.London: Kogan Page
Burgess, R. (2007) Beyond the Honours Degree Classification: Burgess Group Final Report. Universities UK
Elwood, J. & Klenowski, V. (2002) ‘Creating communities of shared practice: the challenges of assessment use in learning and teaching’, Assessment and Evaluation in Higher Education, 27, 243-256
ESwAF FDTL (2007) Final Report. Available on line at:
Fritz, C.O., Morris, P.E., Bjork, R.A., Gelman, R. & Wickens, T.D. (2000) ‘When further learning fails: stability and change following repeated presentation of text’, British Journal of Psychology, 91, 493-511
Gibbs, G. (1992) Improving the quality of student learning.Bristol: TES
Gibbs, G. (2007) ‘Part-time effort for full-time degrees’, THES, 28th September
Gibbs, P., Angelides, P. and Michaelides, P. (2004) ‘Preliminary thoughts on a praxis of higher education teaching’,Teaching in Higher Education, 9, 183-194
Havnes, A. (2007) ‘What can feedback practices tell us about variation in grading across fields?’ Presented at the ASKe Seminar Series, OxfordBrookesUniversity, 19th September
Kember, D., et al. (1997) Case studies in improving teaching and learning from the action learning project. Action learning project,Hong Kong
Knight, P. T. (2002) ‘Summative assessment in higher education: practices in disarray’, Studies in Higher Education, 27(3), 275–286
Knight, Pand Yorke, M (2004)Learning, Curriculum and Employability in Higher Education.London: Routledge
Newstead, S. (2002) ‘Examining the examiners: why are we so bad at assessing students?’,Psychology Learning and Teaching, 2 (2), 70-75
Newstead, S.E. and Dennis, I. (1994) ‘Examiners examined: the reality of exam marking in psychology’, The Psychologist, 7, 216-19
Nonaka, I.. (1991) ‘The knowledge-creating company’, The Harvard Business Review, November-December, 96-104
O’Donovan, B., Price, M. & Rust, C. (2004) ‘Know what I mean? Enhancing student understanding of assessment standards and criteria’,Teaching in Higher Education, 9, 325-335
Ramsden, P. (1992) Learning to teach in higher education.London: Routledge
Raven, J. (1991) The tragic illusion: educational testing.New York: Trillium Press
Rust, C. (2007) ‘Towards a scholarship of assessment’,Assessment and Evaluation in Higher Education, 32(2), 229-237
Rust, C., O’Donovan, B. & Price, M. (2005) ‘A social constructivist assessment process model: how the research literature shows us this could be best practice’, Assessment and Evaluation in Higher Education, 30 (3), 231-240
Sadler, D. R. (1987) ‘Specifying and Promulgating Achievement Standards’, Oxford Review of Education, 13, 191–209
Sadler, D. R. (1989)‘Formative assessment and the design of instructional systems’,Instructional Science, 18, 119-144
Sadler, D. R. (2008) ‘Indeterminacy in the use of preset criteria for assessment and grading’, Assessment and Evaluation in Higher Education, April
Watkins, D., and Hattie, J. (1985) ‘A longitudinal study of the approaches to learning of Australian tertiary students’, Human Learning, 4, 127-41
Vygotsky, L. S. (1978) Mind in society: the development of higher psychological processes.MA: HarvardUniversity Press
Zhang, L. F. & Watkins, D. (2001)‘Cognitive development and student approaches to learning: an investigation of Perry's theory with Chinese and US university students’,Higher Education, 41, 236-261
1
Price and O’Donovan, Manifesto. 16/11/2018