Responding to Problems with Electronic Assessment: Inculcating the Ethics and Standards

Ross Ashcroft

Responding To Problems With Electronic Assessment: Inculcating The Ethics And Standards of The Legal Profession Whilst Providing Students Flexibility To Undertake Assessment

Introduction 1

Educational Theory and The Need for Informed Practice: Assessing Towards a Future of Required Knowledge and Skills 3

Social Constructionism and Its Application to the Study of Law: 4

Scaffolding: 6

The Principles of Rigorous and Fair Assessment: 7

Enforcement of Established Guidelines and the Maintenance of Evidence: 8

The Purpose of Assessment: The Bloom’s Taxonomy Alongside Types of Assessment Items 9

Summary - Curriculum Design and Assessment Design Based on Educational Theory: 11

Electronic Assessment: Is It Enough To Only Consider the Existence of a ‘Pros’ or ‘Cons’ Dichotomy? 12

The ‘Pro’ Arguments for Electronic Assessment: 12

The ‘Con’ Arguments Against Electronic Assessment: 13

Resolving the Concerns – The Existence of Middle-Ground Between the Pro- and Con- Arguments: 15

Examples from Reflection: 17

Introduction to Legal Studies: 17

Torts Law: 19

General Comments from Reflections: 22

Concluding Remarks: 24

Introduction

Legal education around the world has been undergoing significant numbers of reviews over the past century, particularly the last three decades. The re-developments revolve around both the content which should be incorporated into the legal education and the manner in which the teaching should be conducted. Part of these changes, such as shifting legal education to university environments, have been the result of economic pressures (such as a view that law schools are easy income for universities, and comparatively cheap to teach), whilst other reasons have been a result of disgruntled community or governments about lack of standards of professionals. The changes have especially in countries with a common law legal tradition, however, it is also true of countries which have civil law or mixed jurisdiction traditions. Major reports have been conducted in Australia, the United States, Hong Kong and the United Kingdom, reviewing the methods of education and making recommendations about how changes can be made to improve the education.

At the same time there have been calls to reform aspects of legal education, ubiquitous technology has been making profound changes to the way in which society conducts business, searches for information, interacts and communicates with each other.[1] Over time, the technology has even impacted on aspects of the legal professions’ and law firms’ business practices. Whilst technological changes have been occurring ever more rapidly, educational and assessment practices, including those in law, have remained significantly similar to those habits existing at the beginning of the twentieth (20th) century.[2] This is because electronic assessment (e-assessment) is viewed with caution as a tool of assessing students’ knowledge of a particular subject matter. Concerns have been raised by parts of the legal profession about e-assessment for various reasons, including perceptions that students may not be able to get the same level of inter-activity online as being physically present in a classroom (formative assessment/low stake assessment) and there being a possibility that e-assessment will lead to greater chances of cheating without invigilators present (summative assessment/high stake assessment). The concern about distance education or electronic forms of assessment is highlighted in the United States, where the American Bar Association has for a long time remained reluctant to recognise qualifying professional degrees (e.g. equivalent to JD or LLB) for the purpose of sitting the bar exams.[3] Other jurisdictions, as well as other academic disciplines, have likewise expressed concern about electronic assessment, in part due to its unconventional nature and the fact it goes against tradition. This paper will discuss how, from both a practical and educational perspective, the concerns are arguably inherently unjustified. The paper will begin with a short discussion of educational theory, starting with broad purposes of education. The discussion of education theory will also address theories in relation to the purpose of assessment, including types and levels of assessment which can be adopted to achieve the desired outcomes of the assessment. Following this, there will be a short discussion in of the pros and cons of e-assessment regimes, and whether such a dichotomy is really necessary. The final part of the paper will explore a reflection of how I have used e-assessment in Australia, and both the positive aspects I have experienced and the problems faced. The purpose, which will be enhanced through discussion at the Association of Law Teachers Conference, is to highlight the important of reflection in minimising problems and rectifying problems, especially in terms of assisting students in understanding the real world ethical issues faced by lawyers.

From the outset, it must be kept in mind that the purpose of this paper is not to resolve all problems of electronic assessment or provide an exhaustive list of types of assessment which could be modified and used electronically. Rather, the purpose is to act as a starting point for critical self-consideration.

Educational Theory and The Need for Informed Practice: Assessing Towards a Future of Required Knowledge and Skills

Obtaining a law degree is the first step towards an individual possibly beginning a life in a steeped and practical profession. Traditional legal education, at least in the common law world, has not until comparatively quite recently, occurred in academic institutions. Rather common law legal education occurred in the profession itself, such as Inns of Court or articles in law firms, which acted essentially as apprenticeships. This practice was highly criticised because of inconsistencies in quality and lack of quality control.

In recent decades, with the shift towards basic qualifications being obtained at universities and higher education institutions, lawyers have become engaged in doing something which they are not trained to do within their profession: teach. Although lawyers may have the substantive knowledge which they desire to transmit, the lack of training or interest in educational theory and practice hinders good legal education and enables the ‘business as usual’ model of assessment to continue.[4] Many of those who enter the academy draw upon their own experiences of assessment to determine what the assessment regimes will be for their pupils.[5] It could be a fear of change, or the lack of training in educational philosophies, which leads to this outcome.[6] Regardless of the reasons, it is clear changes need to be made, and the changes need to be made within a constructive dialogue between the academy and the professional bodies.

Although the traditional forms of assessment in law schools have merit for displaying certain types of knowledge or skills, there is little to suggest that new methods of assessment and education cannot be interesting, nor can it be said that modifying old forms of assessment into new forms of delivery is not without practical consequence for the real world. It is from this perspective that this paper will begin with some basic elements of adult education which can be a useful starting point in developing a teaching practice, as well as assessment methods.

Social Constructionism and Its Application to the Study of Law:

The so-called pedagogy[7] in adult education swings like a pendulum between various trends. The Russian philosopher and psychologist, Leo Vygotsky, developed a theory of learning referred to as ‘social constructionism’. Social constructionism is, like many sociological disciplines and discourses, something difficult to put a single ‘pure’ definition to, as it is described by scholars in different ways.[8] One definition of ‘social constructionism’, offered by Gale Miller and James A Holstein, is that social constructionism is a manner of analysing issues as a ‘social process’.[9] At its most basic level, this holds true of legal education. Legal education can be viewed as a process of attempting to ‘socialise’ students into a particular community – the legal community. This is done by students learning a set of ‘knowledge, skills and ethics’[10] which are peculiar to the legal profession. It is highlighted by universities gearing their qualifications to meet the requirements of entry to the legal profession. A core objective of legal education is the ability to go through the process in order to apply law in order to assist in ordering or bringing about changes in society. The reason this is a social construct is because there is a need for the awareness by students and graduates of the wider societal application of their skills and knowledge in law. This awareness is a social construct – an ability to apply the special skills and knowledge in the community,[11] which is becoming more commonly done in mediums that are not merely on a paper letterhead.

However, the definition offered above is rather limited if viewing social constructionism as merely a process. A broader understanding of social constructionism is offered by Vivian Burr. Instead of seeking a single ‘definition’ of social constructionism per se, Burr aimed at looking for common ‘elements’ of social constructionism. These elements are a useful contribution to social constructionism and for the purpose of this research paper. The elements identified by Vivien Burr include:

·  ‘a critical stance toward taken-for-granted knowledge’;

·  ‘historical and cultural specificity’;

·  ‘knowledge is sustained by social processes’;

·  ‘questioning realism’;

·  ‘historical and cultural specificity of knowledge’; and

·  ‘a focus on process’.[12]

Social constructionism is also useful due to its flexibility of application. Whilst a law degree does serve as a pathway to obtaining black letter legal knowledge, it is now widely questioned as to whether a degree should remain solely focused on ‘black letter’ learning, or solely on the traditional approaches of teaching law. Some of the research has articulated the need to have the substantive content requirements more liberal with the introduction of legal knowledge of other jurisdictions or other disciplines, whilst other research, such as that by Professors Weisbrot and McCrimmon have suggested there is need to remain flexible and innovative.[13]

What does this mean for educators of law, especially in terms of assessment regimes? The relevance is actually quite simple. In determining the assessment regime, it is important to decide:

1.  What content (skill, ethic or knowledge) am I trying to teach?

2.  What content am I thus trying to assess?

3.  What level of engagement or capacity am I trying to assess?

4.  What method of assessment am I going to chose to assess this?

Scaffolding:

Like social constructionism, Leo Vygotsky contributed the notion of ‘scaffolding’ to educational theory. Scaffolding was developed through the notion of a ‘zone of proximal development’.[14] Scaffolding is a metaphor from construction. A particularly helpful discussion of this theory can be found in the works of Van Der Stuyf.[15] According to this paper, the scaffold should be designed as nothing more than a temporary support structure, that, as a student progresses in their abilities (whether knowledge or skills), is gradually removed.[16] The scaffold should be designed in a manner which enables a student to develop ‘more sophisticated cognitive systems’.[17] In teaching, scaffolds can take many forms, including ‘models, cues, prompts, hints, partial solutions, think-aloud modelling and direct instruction’,[18] which become gradually less intrusive as the student’s cognitive skills develop.[19] The principles of the scaffolding approach has a direct correlation to the principles of assessment, whether traditional pencil-and-paper assessment or e-assessment. According to Van Der Stuyf, McKenzie identified several key principles of the scaffold, including: clarity of directions and purpose, ensuring students stay on task, clarifies expectations for assessment and feedback, points students to the important sources and attempts to reduce uncertainty and disappointment.[20]

Scaffolding is not applicable just to adult education in the area of knowledge development. The development of practical skills often takes time and repetition. This has been widely recognised, for example, in sports coaching which relies on the methodologies of scaffolding in ensuring safe practices are developed when teaching new skills, such as somersaults or aerial movements in gymnastics, diving and trampolining, or the advancement of lifting techniques in sports such as Olympic weightlifting.

Thus, the critical point to keep in mind, from the perspective of both scaffolding and social constructionism is to ensure that there is an alignment between the items of assessment and the knowledge, skills and/or ethics which one is trying to have the learner achieve. This can be done through the process of curriculum design and alignment of assessment tasks, keeping in mind both the principles of assessment and using the Blooms’ taxonomy as a useful guide towards this alignment.

The Principles of Rigorous and Fair Assessment:

Educational literature is likewise varied in the definitions it gives to the principles of assessment. However, despite the linguistic differences, there are some constant elements which arise. Griffin et al identify three key factors to keep in mind for an assessment regime that is fair, which are: 1. clarity of expectation; 2. highly visible assessment based on standards; and 3. Use assessment to communicate expectations and publish data to inform decisions.[21] A different way of explaining a well-designed assessment regime is to suggest it is based on a tripartite system of balance using three “c’s”: coherent, comprehensive, continuous.[22] Coherent ensures it is understandable and consistent across all bureaucratic measures; comprehensive refers to the requirement that all skills and types of knowledge are tested (me to introduce bloom here), and allows feedback to be given at different times; and continuous means that there are multiple, ongoing tasks, which permits the tracking of performance, both for students and as an institution, over time.[23] It has also been suggested that the validity of assessment will be enhanced with the proper development of guidelines and assessment principles being clearly articulated.[24] This is applicable to both traditional and electronic forms of assessment.[25]

It is widely acceptable that all of these factors can be assisted with constant, or at least some level of, feedback on assessment tasks.[26] But an assessment regime does not just need to be fair, it needs to be rigorous and suitable to its end goals. For law, we want lawyers who have the knowledge and skills to operate in a world where they will apply the law in a manner which complies with ethical standards and is of high quality. A good assessment regime will thus be appropriately rigorous if it includes multiple pieces of assessment,[27] and assess the performance at various cognitive levels (discussed below in the section on Blooms’ Taxonomy).