1
RE-DEFINING LEARNING OUTCOMES: A CASE FOR THE ASSESSMENT OF SKILLS AND COMPETENCIES IN A LAW DEGREE
Peter Devonshire and Ian Brailsford*
1. INTRODUCTION
The assessment of undergraduate law teaching tends to focus on the formal syllabus and the effectiveness with which the core content is conveyed and understood. As a result, course and teaching evaluations provide limited answers to limited questions.Pedagogical standards and benchmarks should be driven by a simple but fundamental question: What is the purpose of a law degree? The first, and most conservative response is that it is to train students to become lawyers. The obvious rejoinder is that legal education should not be dominated by this objective when many graduates are destined to seek employment outside the legal profession.[1]Another response is that the purpose of a law degree is to train students to think like lawyers. This is more promising, at leastto the extent that it is less focused on the vocational aspects of legaleducation. This leads to the third, and it is submitted, preferred rationale, that the purpose of a law degree is to imbue students with skills and competencies unique to the discipline of law.[2] These include specific analytical skills such as legal research and writing, statutory interpretation and an understanding of judicial process. Such skills are underpinned by deductive and syllogistic reasoning associated with legal method. In addition, practical skills can be identified such as mooting, alternative dispute resolution, clinical training and community placements.[3]
The assimilation of skills and competencies within a law degree is only meaningful if standards are prescribed and measured. Assessment must therefore be integrated into the syllabus so that the attainment of certain standards of competence is a specific goal of a law degree. However, there are significant limitations to traditional assessment models, which are tethered to the formal syllabus and dominated by the needs of summative assessment.
This paper argues that a law degree should be re-conceptualised to acknowledge that the study of law is a process that requires the attainment of certain skills and competencies.This can most effectively be achieved by introducing formative assessment as a key tool in promoting professional and generic capabilities. In the first section of this paper we discuss an approach to identifying required skills and competencies. This particularly focuses on the role of graduate profiles in raising awareness of skills-based outcomes and providing a foundation for systemic change across the curriculum. In the second section we discuss the integration of formative assessment techniques into the syllabus to reinforce and assess aptitudes that underpin the discipline of law.
2. IDENTIFYING OBJECTIVES AND OUTCOMES
(a) The Case for embedding skills and competencies in a law degree
At the outset we should clear away a possible misapprehension. There is a long-standing debate as to the function of an undergraduate law degree.[4] On one view, a law degree is simply the academic backdrop for a career in the practising profession. This is certainly true to the extent that most law degrees in the Commonwealth align their compulsory courses with the accreditation requirements of the governing professional bodies such as the Law Society and the Bar Council. Beyond that, however, it is generally held that law is an academic discipline. Theory and black letter law belong in this domain while practical and skill-based aspects are left to vocational courses following graduation. It is our view that this is an outmoded and unhelpful distinction that can no longer be sustained.[5]It has been overtaken by a growing awareness that academic study is enhanced by the active development of skills and competencies.[6]
The case for embedding skills and competencies in a law degreeis compelling. The stakeholders include those who will never practise law and those who will. With respect to the former, Law Schools have long since abandoned any pretence of regulating programme admissions by reference to the employment market. Subtly and over time, law degrees have been re-branded. In addition to its traditional role, a law degree is presented as a valued form of analytical training that provides a useful pathway into other fields and careers. For such students the more specific aspects of legal practice have limited relevance, whereas an understanding of the underlying skills, techniques and intellectual structure of law is a prime incentive for taking the degree.
With regard to the latter, there is pressure from both students, as intending lawyers, and the practising profession, for a minimum skills-based content in a law degree.[7]The process is already well in train.Particularly in the last decade, significant progress has been made, spurred in part by the fact that education providers are operating in an increasingly globalised and competitive environment. Teaching standards and pedagogical objectives are under intense scrutiny as universities strive to meet the expectations ofa sophisticated body of consumers.[8]
These developments suggest that the attainment of specific skills and competencies should be a declared objective of tertiary legal education. This promotes the key principle that the study of law is essentially a process. If aspects of that process can be identified, it becomes possible to perceive these elements as learning outcomes. The ideal is to achieve a constructive alignment of training and learning.[9]There is an obvious correlation between graduate attributes and learning outcomes. The latter is the principal means of identifying core process objectives while the former provides a framework for the enquiry.
(b) Mapping graduate attributes
Increasingly, Law Schools are publishing graduate profiles, identifying not only the subtstantive knowledge that will be acquired, but also the accompanying skill sets and aptitudes that will be fostered throughout the course. This provides some insight into the range of competencies that are, or should be, imparted. For example, the University of Edinburgh LLB programme has a detailed list of graduate attributes, covering skills and abilities in research and enquiry, personal and intellectual autonomy, communication, and personal effectiveness.[10] The desired outcomes are developed incrementally, enabling students to hone a range of abilities within these categories as they progress through the degree. A common variation is that a university will create a generic graduate profile,which is adopted, perhaps with discipline-specific modifications,as a subset of the institutional model.[11] In some cases specific attributes from university or faculty profiles are selected for individual courses within the law programme.[12]
To be meaningful, graduate attributes should be mapped across the curriculum, and where appropriate, aligned with particular subjects or course clusters. Desired competencies are not confined to black letter law and include “soft skills” which mimic the social side of law, such as client interviewing, mediation and legal writing. This also includes professional ethics and a broad understanding of law in society. On paper the task seems ambitious but attainable. However, itshould not be underestimated. The embedding of graduate attributes is dependent on the commitment of course directors and their academic colleagues to undertake what is essentially an exercise in curriculum renewal.[13] The exercise is not complete until learning outcomes are matched with assessment.[14]This involvesdeveloping an assessment matrix directed to the attainment of learning outcomes.[15] The comprehensive nature of the overall exercise – and indeed the level of commitment required – is depicted in the following description of a project to integrate professional and generic training into the law programme at Queensland University of Technology:[16]
The project was structured in three stages. Stage 1 identified the graduate attributes using feedback from employers and graduates and various studies. Stage 2 identified the skills and the different competency levels of those skills that a law graduate needs in order to possess those attributes …. Stage 3 required much discussion and negotiation with all Law School staff to identify the units in the LLB curriculum in which particular skills would be integrated. Learning objectives had to be reconsidered, as did teaching strategies, assessment methods and modes of delivery. Appropriate assessment tools and reporting procedures were designed for students to track their skill development, a staff development model was created and implemented, and resources compiled for staff and students.
The project team identified four broad categories of generic and discipline specific skills which should be developed across a range of undergraduate core subjects: attitudinal skills, cognitive skills, communication skills and relational skills.[17] Some are focused on specific abilities to facilitate the study of law while others are directed to the personal qualities and attributes of a lawyer. The former includes IT literacy, document management, legal analysis and advocacy. The latter encompasses ethical values, a social justice orientation, an appreciation of diversity issues and an ability to work both independently and as part of a team.[18]
(c) Changing perspectives
It should be recognised that we are here discussing a paradigm shift from learning the contents of a traditional law syllabus to testing the development of skills inherent in legal education. Consider, first, the focus of Bloom’s “Taxonomy of Educational Objectives”, written over 50 years ago.[19]In classifying educational goals,[20]the followingcategories were identified: knowledge, comprehension, application, analysis, synthesis and evaluation. These heads are hierarchical and transitional, beginning with objectives that are readily accomplished and moving towards more complex and challenging goals.Each is divided into sub-categories.[21] Knowledge, for example, is reduced to three elements: specifics (knowledge of particular facts, terminology, and the framework of the legal system), methods (such as statutory interpretation and the application of different rules), and universals (basic principles and underlying theories).As a critique of teaching and learning, it is limited in scope.It is apparent that this taxonomy projects the content of the formal law syllabus and rationalises the components of learning from that particular perspective.The resulting model is simply an abstract representation of the intellectual elements of academic study.
Much has changed. There is increasing awareness of the wider dimensions of a law degree. And more broadly, university assessment practices have been revisedin response to educational theories informed by the concept of constructive alignment.[22] Empirical research has become widespread and the methodologies for teaching and learning are seen as a challenge for both the academy and the profession. This is evident in the Australian Law Reform Commission’s review of the Federal Civil Justice System.[23] In an influential report, the Commission emphasised the importance of a law curriculum going beyond a contents focus, towards skills and values addressing what lawyers need to be able to do, as opposed to what lawyers need to know.[24]The attainment of skills and competencies in a law degree is consistent with training programmes for law graduates seeking admission to the profession.For example, the skills taught in the Bar Professional Training Course include casework skills, written skills, interpersonal skills and legal knowledge (including professional ethics).[25] While these will necessarily have a practical bias, it is unrealistic to suggest that the attainment of such competencies is the sole preserve of vocational training and that it is inappropriate to introduce future lawyers toapplied skillsat an earlier stage.
3. FORMATIVE ASSESSMENT
(a) Curriculum integrated model
From the university’s perspective, assessment must itself be assessed. It must be acknowledged that a Law School is essentially a gatekeeper of both academic abilities and practical competencies. The former is too apparent to require elaboration. In contrast, the latter is often perceived more as an administrative duty than a pedagogical goal. This understates important functions that serve to support academic standards. For example, competencies such as writing and comprehension are assessed where English language proficiency is uncertain. In turn, support is often provided for marginal students to facilitate their continued study. Again, learning strategies are adjusted as first year students transition from “bulk” teaching to smaller modules. The linking of these states or conditions to instructional strategies is a common form of micro-adaptation.[26] But in the main, initiatives tend to be more reactive than proactive. It is submitted that if graduate attributes are embedded in the law degree curriculum, the support of skills and competencies will have come of age. This should encourage education providers to redefine their relationship with their students and inspire a proactive approach in setting and meeting positive goals.
In the previous section we argued that graduate profiles provide a conceptual structure to define the required skills and competencies for legal education. The intent is that they should be mapped across the undergraduate syllabus so that each course contributes to a collective goal. The prescribed attributes are only meaningful if they are reinforced by assessment. It is this element which transforms the students’ experience by encouraging critical reflection on their own learning. This is a transformative process “because it views learning as a process of change and development rather than knowledge acquisition”.[27]
There are of course, many variables. The assessment ofskills and competencieswithin the Law curriculum will reflect the vision of individual institutions. Ideally, it should be recognised that these elements are interdependent and complementary. It has aptly been said that “law units are not silos … knowledge and skills learnt in one unit are transferable to other law units”.[28]Moreover, skills and their assessment must be incorporated across the curriculum, both horizontally and vertically. The former denotes integration of similar levels of accomplishment across each year of the degree. The latter creates increasing expectations and demands, commensurate with a student’s advancement through the programme.The scheme is incremental and comprehensive:[29]
[G]raduate attributes need to be the subject of scaffolded development, and the foundational learning must take place in the first year of law studies so that higher level attribute attainment can be progressively developed in the following years.
The markers for these transitional phases can be conceptualised as a progression from knowledge, to practice to creative application.[30]
Discussion to this point assumes commitment to a “curriculum integrated approach” as opposed to stand-alone development, where attributes are promoted in a unit separate from the discipline.[31] The latter is pedagogically unsound[32] because, by definition, it segregates the skill from the discipline. Secondary skills derive their vitality and relevance from their primary source. Where those skills are incorporated in the degree structure, the relationship between skills and the discipline is symbiotic and each is diminished by the absence of the other.
(b) Formative and summative assessment
It has been argued that assessment must be aligned with learning outcomes. As the outcomes include skills and competencies, the focus necessarily turns to the appropriate methodology for assessment. This requires a balancing of formative and summative assessment. Formative assessment nurtures and improves learning, in particular by providing:[33]
regular feedback to students in order to stimulate learning and to provide students with information which will enable them to judge the effectiveness of their learning strategies and to make progress.
In contrast, summative assessment measures achievement. This usually takes the form of a mark or grade. A final end of year examination is summative.It fulfils the limited objective of providing a grade for the course.[34]It does not assist students to improve their performance in that paper. The most that can be said is that it may provide insight with respect tofuture courses within the programme. The development of skills and competencies istherefore more the province of formative assessment.[35] Thisdirectly facilitates learning, and from a wider perspective,supports the achievement of graduate attributes.
There is a perennial tension between formative and summative assessment because the number of assessments in either form that can realistically be administered is finite.Beyond a certain point they are burdensome to students and staff alike. In this paper it is argued that formative assessment must be given more prominence in teaching and learning. Unfortunately this runs counter to changes in course design and degree structure,which have resulted in increased summative assessment.[36]Most notable in this regard is the move towards semesterisation and teaching in modules. Both impose additional demands for summative assessment. Typically, a semester is an inclusive unit requiring examination of all its courses for a final grade. Compounding this, modularisation increases the number of papers that must be examined.[37]Itis submitted that this position must be reviewed to accommodate a greaterrole for formative assessment to promote skills and competencies.
The optimal outcome is for formative assessment to complement traditional assessment regimes, recognising that there are limitations with the latter and gaps to fill. The relationship between formative and summative assessment is a major field of pedagogical discourse, which is beyond the immediate scope of this paper. However, two general observations will be made to establish at least a threshold case for change.First, formal examinationsplace a premium on students knowing the material rather than testing the development of skills.[38]Thisinstils passive and uncritical study habits, manifested in extreme cases, by rote learning. Even with the widespread introduction ofopen book format, students tend to focus on the examiner’s expectations and predicting the questions that will be asked. To that extent preparation for examinations (whether open book or closed) is formulaic. In its bleakest form:[39]
An assessment system dominated by the needs of summative assessment can encourage mere surface learning …. The surface learner tends to be passive and unquestioning, and will memorise facts without concern for understanding.
Predictably this draws the criticism that summative assessment is counterproductive in that it fails to motivate students and is not conducive to a transformative learning experience.[40]
Secondly, the pressure of formal assessment associated with semesterisation and modular teaching encourages students to become strategic learners. There is a perception that an examination is a game played by rules. It is strictly inward reflecting. The winners are those who are most responsive to its narrowly defined prescriptions.Allied to this, the proliferation of formal assessment affects the relationship of teacher and student. The pastoral emphasis of teaching is eclipsed by the role of judging performance in course work and examinations.