Building an evidence-base for the Civil Justice System

Civil justice system framework

and literature review

Report

Prepared by:

Dr Robyn Sheen and Dr Penny Gregory

for the

Australian Government’s Attorney-General’s Department

3 September, 2012

Civil justice system framework and literature review – Final report

______

Building an evidence base for the civil justice system
Contents
Acknowledgements
Acronyms and abbreviations
Executive summary
Key findings / i-ii
iii
iv-v
vi-vii
iix-xi
Introduction / 1
Background / 1
The project requirements / 2
Civil justice objectives / 2
Civil Justice Framework principles / 3
The classification framework 4
The nature of research / 4
Classification 1: Social stability and economic growth 6
The civil justice system / 6
The justice market / 8
The cost of justice / 11
The benefits of justice / 12
International research / 12
Findings / 15
Classification 2: Community resilience and capacity building / 16
Information and education / 16
Building dispute/conflict resolution skills in individuals and communities / 18
Findings / 18
Classification 3: Expeditious resolution of disputes / 19
Referrals to appropriate legal and non-legal services / 19
Use of ADR as a means of early resolution of disputes / 22
Settlement rates and time to settlement (including re-activation of cases) / 24
International research / 24
Findings / 25
Classification 4: Service quality, including just and fair processes / 26
Procedural fairness / 26
Practitioners / 26
Strengthening the legal profession / 28
Regulation/accreditation of practitioners/standards / 29
Service quality / 29
Client satisfaction/perceptions of fairness / 30
International research / 31
Findings / 33
Classification 5: Equity of access / 34
Affordability / 34
Barriers to access / 35
Access to informal, formal and everyday justice institutions and services / 38
Judicial case management / 40
System complexity / 40
International research / 41
Findings / 42
Classification 6: Wellbeing / 44
Impact of justice services on individual groups with particular needs 44
Effective referral to appropriate non-legal services / 50
Collaborative relationships / 50
International research / 51
Findings / 53
Classification 7: The research and evidence base / 54
Data collection / 54
Performance measurement / 56
Research and evaluation / 59
International research / 60
Findings / 61
Findings in relation to the state of civil justice research and evaluation in Australia 62
Data challenges / 62
Fostering quality research / 65
Barriers that inhibit research / 66
The availability and visibility of research / 67
Research funding / 68
Findings in relation to gaps in research / 70
Findings in relation to the quality, utility and effectiveness of the research / 71
Findings in relation to the draft objectives / 72
Detailed findings on the research attributed to each classification / 73
Appendices
A: Research findings and methodologies / 73
B; NADRAC data findings from the Resolve to Resolve report / 85
C: Collection of court statistics on ADR / 86
D: National Health Information Agreement / 89
Bibliography / 90

Acknowledgements

This project was completed within a very short period of time and we were greatly assisted by a number of people. Many thanks to all of the participants at the forum in Melbourne on 25 May who provided us with valuable feedback on the draft framework and the complexities of the task. In particular, the follow-up assistance from Elizabeth Shearer, Steve McEachern, Mary Polis, Kylie Kilgour and Danny Kingsley is much appreciated.

Dr Geoff Mulherin and Dr Suzie Forell at the NSW Law and Justice Foundation gave generously of their time and substantial knowledge, and the report has benefited from their guidance. Professor Kathy Mack from the School of Law at FlindersUniversity provided us with helpful advice and information which saved us a lot of time and effort. Professor Tania Sourdin from the Australian Centre for Court and Justice System Innovation at MonashUniversity provided important insights that focused our analysis of the evidence base. Nerida Wallace, the Director of Transformation Management Services and Dr Lawrie Moloney from the Australian Institute of Family Studies were also very helpful sources of guidance and referral.

We are extremely grateful to all of these people who share the credit but cannot be blamed for any errors in the report.

Finally, thanks also to Serena Beresford-Wylie and Julia Phillips at the Attorney-General’s Department for their cheerful guidance and support.

Robyn Sheen

Penny Gregory

ACRONYMS and ABBREVIATIONS

AATAdministrative Appeals Tribunal

ABAAmerican Bar Association

ABSAustralian Bureau of Statistics

ACCJSIAustralian Centre for Court and Justice System Innovation

ADRAlternative dispute resolution

AGDAttorney General’s Department, Australia

AIFSAustralian Institute of Family Studies

AIHWAustralian Institute of Health and Welfare

AIJAAustralasian Institute of Judicial Administration

ALRCAustralian Law Reform Commission

ASICAustralian Securities and Investments Commission

ATOAustralian Taxation Office

CALDCulturally and linguistically diverse

CAVConsumer Affairs Victoria

CCJSICentre for Court and Justice System Innovation, MonashUniversity

CCPChildren’s Cases Program of the Family Court of Australia

CDRACivil Dispute Resolution Act 2011

CLCCommunity Legal Centre

CLRPCivil Litigation Research Project

CMDCourt mandated diversion

CthCommonwealth

DF&DDepartment of FinanceDeregulation, Australia

DIISRTEDepartment of Industry, Innovation, Science, Research and Tertiary Education

DSCVDispute Settlement Centre for Victoria

FaHCSIA(Department of) Families, Housing, Community Services and Indigenous Affairs

FCAFamily Court of Australia

FDRFamily Dispute Resolution

FICSFinancial Industry Complaints Service

FMCFederal Magistrates Court

FOSFinancial Ombudsman Service

FRCFamily Relationship Centre

GAOPGrants to Australian Organisations Program

HERDCHigher Education Research Data Collection

HILDAHousehold, Income and Labour Dynamics in Australia

JPEJudicial Performance Evaluation, US

LACLegal Aid Commission

LATLess Adversarial Trial

L&JFLaw and Justice Foundation, NSW

LSACLongitudinal Study of Australian Children

LSRCLegal Services Research Centre

NADRACNational Alternative Dispute Resolution Advisory Council

NCSCNational Centre for State Courts, US

NJCNeighbourhood Justice Centre, Victoria

NSUNorfolkStateUniversity

PILCH Public Interest Law Clearing House

PIPParent Information Program, UK

ROGSReport on Government Services

RRRrural and remote regions

SBDCSmall Business Development Corporation, Western Australia

SCRGSPSteering Committee for the Review of Government Services Provision

SRLsself-represented litigants

SREDSurvey of Research and Experimental Development

UoEUniversity of Exeter

VCATVictorian Civil and Administrative Tribunal

VLRCVictorian Law Reform Commission

VSBCVictorian Small Business Commissioner

Executive Summary

The aim of this project was to scan research into the civil justice system conducted over the last five years to find empirical research that relates to the draft objectives for the civil justice system. The research has been classified under the relevant objectives through assignment to a classification framework developed in consultation with the Commonwealth Attorney-General’s Department (AGD).

Through this process of scanning the literature, the consultants were also asked to make findings in relation to the state of civil justice research in Australia, with a view to informing the planned development of an evidence base for the civil justice system.

The project was conceived as part of the “bottom-up” approach to building an evidence base for the civil justice system - assessing the utility of existing data collections. Completion of the task was greatly assisted by discussions at a Forum on Building an Evidence Base for the Civil Justice System on 25 May 2012 and by further consultations with researchers in the field.

One of the challenges associated with the project has been to classify broad-based research, often produced as part of a program evaluation, against the various classifications that flow from the draft objectives. Few research studies fit neatly within the system objectives and a close examination of many studies reveals findings against a number of the classifications. These have been dealt with to the extent possible by cross-referencing.

Detailed findings on the research have been included at the end of each classification. These brief statements encapsulate findings on the research and evidence base for the objective covered under the respective classifications. A full list of these findings are on the following pages.

The final chapter provides a discussion on the state of research and data collection, as evidenced by the literature scan, and contains a number of recommendations for moving forward (see below).

On data challenges, it is recommended:
  1. that a list of data items/concepts considered essential be developed as a starting point for clarifying what is important to measure in improving the civil justice system;
  2. that a high level case be developed and considered by all governments at a high level that demonstrates the benefits to be gained from, and seeks in principle commitment to, national consistency of data in particular areas of the civil justice system;
  3. that options for development of infrastructure to facilitate and drive the achievement of nationally consistent data across the civil justice system in Australia be explored and costed; and
  4. that the potential for development of a National Information Agreement relating to the civil justice system be explored in the context of the infrastructure options.
On fostering quality research, it is recommended:
  1. that AGD consider formulating a civil justice research strategy in collaboration with the states and territories and other key bodies, to focus attention on the highest priority areas of desirable research.
On barriers that inhibit research, it is recommended
  1. that the AGD further investigate, in collaboration with the Department of Industry, Innovation, Science, Research and Tertiary Education, why the ratings applied to civil justice research in universities do not appear to encourage such research.
On the availability and visibility of research, it is recommended:
  1. that the AGD initiate discussions with the states and territories and other relevant bodies who may have commissioned confidential evaluations or other research to determine whether the research (or abridged versions of the research) can be made publicly available; and
  2. that the AGD discuss with Australian Centre for Court and Justice System Innovation (ACCJSI) ways in which the AGD can encourage and support use of the ACCJSI Clearing House as a means of sharing research on the civil justice evidence base, including writing to states and territories and other relevant bodies inviting their participation.

The final sections of the report deal with research funding, the quality, utility and effectiveness of the scanned research as well as the gaps identified against each classification. Finally, some observations are made in relation to the draft objectives, based on the experience gained from the literature scan.

Dr Robyn Sheen

Dr Penny Gregory

3 September 2012

1

Civil justice system framework and literature review – Final report

______

Detailed findings on the research attributed to each classification

Classification 1: Social stability and economic growth
  • While there are numerous theoretical journal articles and judicial papers on the civil justice system which provide valuable insights into the various aspects of the system, and how it operates, there is very limited empirical research. The only study that was found was an opinion poll on the Constitution. Otherwise, the evidence base must rely on data in publications like the ROGS which are limited in terms of their scope and analysis, but valuable in terms of their national comparability.
  • Research relating to demand in the justice market is generally limited to services to socially and economically disadvantaged people. Much reliance is placed by researchers on the NSW Law and Justice Foundation’s 2006 Justice Made to Measure Report. Comprehensive though it is, this report targets legal needs in disadvantaged areas which leave a significant gap in the identification of legal needs in other demographics and State and Territory jurisdictions. Further, Justice Made to Measure covers both the criminal and civil spheres which place some limitations on analysis of the civil justice system in isolation. There is a major gap in the areas of empirical research on the private market and government institutions.
  • More research has been undertaken on supply in the justice market although, apart from a Commonwealth survey on Community Legal Centres, most of the work emanates from the NSW Law and Justice Foundation. Though it is broader in scope and includes some research on the supply and retention of solicitors in NSW, there are gaps in areas of empirical research in other aspects of the market in NSW and in all aspects of the private market in other jurisdictions.
  • No empirical evidence was discovered about the benefits of justice.
  • There is substantial international empirical research with coherent methodologies on civil justice although a review of research on the cost of justice showed there is very little research in that area.

Classification 2: Community resilience and capacity building
  • There is some empirical research available on how information is accessed and the difficulties associated with awareness amongst disadvantaged people of whether they have legal problems. These studies, however, cover both the criminal and civil justice systems.
  • While government websites provide substantial information and guidance which aims to increase civic awareness about the justice system and legal processes, there is no research which indicates the extent to which they are accessed or have beneficial effects.
  • There is ample research about information and awareness of ADR in Victoria but not it would appear in other jurisdictions.
  • There have been a number of surveys taken about industry/small business awareness of dispute resolution mechanisms.
  • There is limited research on dispute and conflict resolution skills in the community. The substantial work by NADRAC relates solely to Indigenous people and could usefully be replicated in the wider community.

Classification 3: Expeditious resolution of disputes
  • Empirical research over the last five years in relation to timely resolution of disputes has focused on the use of alternative dispute resolution (ADR) and family dispute resolution (FDR). In particular, some robust research has been undertaken into the use of court-related mediation and the way in which it increases the efficiency of court processes.
  • Much of the useful research has been undertaken as part of government-commissioned evaluation of programs, aimed at evaluating whether a program met its objectives, for example the KPMG evaluation of FDR in Legal Aid Commissions or the AIFS evaluation of the Family Law Act reforms or the research into mediation in the Victorian courts.
  • The introduction of new government legislation has also been a trigger for high quality research in this area. For example, the introduction of a default position of mandatory FDR as part of the 2006 family law reforms has triggered analysis, evaluation and commentary about dispute resolution processes. Similarly, government reforms in relation to the adoption of pre-action protocols have also triggered a desire to understand whether pre-action protocols and guidelines speed up the resolution of cases, or whether they add barriers and costs. While the views of many parties have been put forward and analysed (for example in a Senate Inquiry), the only empirical research in relation to pre-action protocols in Australia is currently still underway. Again, it is commissioned research – this time by the Australasian Institute of Judicial Administration.
  • While the ROGS regularly publishes clearance rates in the various courts, there has been no accompanying research that attempts to examine the timeliness issue in more detail, particularly in relation to judicial decisions.

Classification 4: Service quality, including just and fair processes
  • Very little reference to therapeutic jurisprudence was found in the literature scanned, although there is considerable work of a theoretical nature.
  • There is a significant body of research into the skills of mediation practitioners (with the introduction of the National Mediation Accreditation Standards relevant to this).
  • Researchers have called for further empirical research into the alleged tendency of some lawyers to create an adversarial approach and into the ethical infrastructure of large legal firms.
  • There would seem to be potential for further research into what constitutes ‘expertise’ in family law and into the apparent differences between practitioners with a social science and a legal training.
  • A limited amount of empirical research was found in relation to measuring how the professional skills of those involved in the civil justice system could be strengthened, or the impact of that up-skilling.
  • Findings in relation to service quality are found in almost all evaluations reviewed for this project. The large-scale evaluations commissioned by governments or other bodies use robust methodologies that rely on multi-dimensional approaches to measuring quality, including objective measures as well as measures of perception.
  • Levels of satisfaction and perceptions of fairness and their inter-relations with outcomes are concepts which have been the subject of a significant amount of empirical research – through client and practitioner surveys. The extent to which people accept early resolution outside of the courts, appears to be closely relatedto their perceptions of fairness and authority and this remains an area where further research is warranted.

Classification 5: Equity of access
  • There are several sources of data on the income levels of clients of government-funded legal assistance services. These data provide a rich source of research material for determining affordability and the impacts of low income on disadvantaged people who are in dispute.
  • One of the impacts of the cost of legal services is the increase in self-represented litigants. There has been considerable research undertaken on self-represented litigants across a range of issues, however, there has been little in recent years.
  • There is substantial research available on the groups in society who experience barriers to accessing legal assistance services; and the predominant factors which impact on equitable access.
  • Research and evaluation on video-conferencing as a response to barriers to access in rural, regional and remote communities is limited. A review of the available research has made recommendations on how video-conferencing could be used more efficaciously.
  • Barriers and access to services to people who are vulnerable or marginalised in society have been the subject of several Australian and international studies. This is an area where non-government organisations play a key part, however, there appears to be limited research or evaluations on their impact in the area of civil justice. A far greater emphasis is placed on the interface by vulnerable people with the criminal justice system.
  • Statistical evidence suggests that Judicial Case Management has been successful in enabling access to justice and improving dispute resolution outcomes pre-court. Statistics aside, there have been no authoritative studies or evaluations undertaken in Australia with review and judicial commentary relying heavily on international studies.
  • Descriptive information aside, some work has been identified on system complexity in the field of ADR. No empirical work on the broader civil justice system in Australia has been located.
  • International research on barriers and access to justice provide sound methodologies which would be useful for undertaking empirical studies in Australia.

Classification 6: Wellbeing
  • Research relating to this classification is dominated by extensive evaluation of the impact on children and families of the civil justice processes for resolving family disputes. The series of changes to the Family Law Act 1975 were intended to bring the views, feelings and experiences of children into sharper focus and considerable high quality empirical research has addressed how well this objective has been achieved.
  • There is also considerable high quality research into the way in which family violence is managed by civil justice services.
  • While there is some high quality research into the way the civil justice system impacts on Indigenous people, there is very little research into the way the system interacts with culturally and linguistically diverse clients or people with disabilities or mental health problems. Some of this research relates to client satisfaction and is covered under Classification 4.

Classification 7: The research and evidence base
  • There are many examples in the research literature where the paucity of data and data systems are highlighted, indicating the need for improved comparability of data. A useful analysis of the differences in concept and definition of the term ‘mediation’ provides a good example of the need for common agreed definitions.
  • There are a number of examples in the literature over the last five years where performance measures have been developed within services – some have been useful, others have been found unsatisfactory. All rely on comparable data being available if they are to be useful and reliable.
  • The literature provides some examples of innovative approaches to fostering research that may serve as useful models as an evidence base is being developed.

1