ESRC Funded Workshop: “The Politics of Policy Appraisal”
24-26 June 2007, New Hall, Cambridge
Co-organised by: Andrew Jordan, Claudio Radaelli and John Turnpenny
Co-funded by: the ESRC’s CSERGE and the Centre for Regulatory Governance at the University of Exeter, project on Regulatory Impact Assessment in Comparative Perspective [ESRC grant RES-000-23-1284]
Background
Policy appraisal has become increasingly popular in the UK, Europe and more generally the OECD countries. It comes in different shapes, from meta-regulation (that is, rules and standards on the process leading to the formulation of regulation) to policy analysis of various hues. It is intended to facilitate progress on a range of different issues, from sustainable development to “better regulation”. It operates ex ante, during the implementation of policy programmes, and ex post, in their evaluation. Its “names” vary, from policy evaluation to value-for-money surveys, citizens’ satisfaction with administrative performance, regulatory impact assessment, horizon scanning exercises, sustainability tests, hearings and evidence-inspired policy. Sometimes two forms of policy appraisal are inter-connected, for example in the recent EU evaluation of regulatory impact assessment, in which ex post appraisal is conducted on a tool of ex ante appraisal. Very rarely do organisations in the public sector (such as government departments and independent regulatory authorities) use several forms of policy appraisals in an integrated way. There are several publications on integrated regulatory management, but empirical research suggests that different types of appraisals tend to proliferate in the same organisation without any serious attempt to integrate the various findings.
Purpose
The ultimate aim is to consolidate the state of the art in this important area of research and identify new and promising research needs. Eventually, this information will be summarised in an edited book or journal special issue. Since we are still fairly early on in this process, and this workshop is fairly short, we are inviting a small group of experts to present, viva voce, their research and ideas for book chapters, rather than writing a full paper ‘up front’. The aim then, to distil out a series of chapter titles, rather than for individuals to present their papers and then try to fit them together after the event. In fact, each chapter could eventually involve several attendees. Hence, the workshop has been structured to encourage wide-ranging discussion and open debate, as well as identify new research collaborations.
Content
The workshop will cover the following interconnected themes on policy appraisal. Within each, we have set out potential avenues for exploration, more to stimulate debate than to set out a comprehensive list of what should be covered. Contributions would be expected to cover one or more of these four themes, but not all of the ideas and questions contained therein.
1. The aims and objectives of appraisal.[“What are the different reasons for doing policy appraisal?”]. Principal questions for this theme include: Where do different types of policy appraisals come from (in terms of ideas generated by organisations, such as the OECD or governmental departments)? What are the underlying ideas behind them (ideational analysis)? What is the main problem they are associated to (problem-definition analysis)? Why have they been adopted and for what purposes? Who sets the agenda for appraisal?
This theme will be explored via conceptual and theoretical analysis. Conceptually, we will discuss concepts that shed light on the properties of different form of appraisals. To illustrate, can broad categories such as meta-regulation and evaluation make sense of families of appraisal tools? Theoretically, we will draw on middle-range theorising in the social sciences to explain why public organisations adopt policy appraisal tools. The speakers may wish to consider agenda setting models, Europeanisation, institutional analysis, principal agent models, discourse, epistemic communities, principal-agent models and policy diffusion. An important part of the workshop is the attempt to draw lessons from comparative analysis (across time, organisations, countries, and levels of governance) about how to integrate different types of policy appraisals.
2. Theories behind policy appraisal: [“how do people think about knowledge, reality, and policy within appraisal?”]. The main questions are: What are the (often implicit) theories of the policy process embedded in policy appraisal?[1] What is the dominant notion of public administration informing appraisal tools (for example, new public management, learning organisation, etc.)? What are the theories of valuing, use, social programming, and practice that inform policy appraisals (these theories vary to a great extent even within a single type of appraisal, such as policy evaluation[2])? What are the ontologies and epistemologies on which different types of appraisal draw? What are the models of politics (including models of cabinet decision-making[3], normative ideas about the relationship between bureaucrats and politicians, etc) that lead to the design of a specific type of appraisal?
3. Appraisal in practice? [how are appraisals actually used?]. If we consider appraisal as independent variable, what is the impact on the socio-economic environment and on politics? For example, does regulatory impact analysis produce more efficient regulation (socio-economic impact) and empowers those who use economics as main lens through which regulatory problems are formulated and addressed (political impact)? In turn, both the socio-economic impact and the political impact can be broken down in short-term impact (the impact of a single evaluation study on a policy decision) and long-term impact. As shown by Carol Weiss[4] and Susan Owens et al., the long-term impacts are potentially the most important, including the impact of policy appraisals on communities of decision-makers that have performed-commissioned appraisals for a long period of time. A set of papers might investigate the use and impact of policy appraisals in particular national policy systems and the EU. Also, what are the consequences of policy appraisal – e.g. does appraisal make a difference to public opinion?
4. Future directions for policy appraisal research and practice.
This theme brings together discussion from the preceding three and aims to generate normative and lesson-drawing ideas for future types and activities surrounding policy appraisal. Is there a future for policy appraisal, or is it more appropriate to focus on other forms and mechanisms of evidence-gathering? What is the likelihood of a true ‘evidence-based policy’ regime actually bringing sustainable development and/or better regulation about? What format should appraisal take to have maximum influence? Who should carry it out, and what checks and balances should there be? What are the limits to stakeholder consultation?
Timing and length
The meeting begins in the morning of Monday 25 June (with an optional dinner on Sunday 24 June), and finishes after lunch on Tuesday 26 June.
Programme
Day 0 (Sun 24 June): Arrival and welcome
2.00 onwards:check in to accommodation
6.00 onwards:registration for workshop – desk open
7.00:Pre-dinner drinks
7.30:Dinner
Day 1 (Mon 25 June): The aims, theory, practice and future of policy appraisal
9.00:Welcome, introductions and aims of meeting
Brief welcome - by Andy Jordan
Introductions - by all
What are the politics of appraisal? - Andy Jordan
Structure and philosophy of the two days - John Turnpenny
10.00:Theme 1 - The aims and objectives of appraisal
Chair: Claudio Radaelli
(2 presentations, 15 minutes each)
Ray Pawson [University of Leeds]
Nicoletta Stame [University of Rome]
Discussion (25 minutes)
Summary of key issues by the chair (5 minutes)
11:Coffee
11.30: Theme 2 - Theories behind policy appraisal
Chair: Andy Jordan
(2 presentations, 15 minutes each)
Susan Owens [University of Cambridge]
Elliot Stern [Lancaster University]
Discussion (25 minutes)
Summary of key issues by the chair (5 minutes)
12.30:Lunch
2.30:Theme 3 - Appraisal in practice
Chair: John Turnpenny
(2 presentations, 15 minutes each)
Ian Sanderson [Head of Analytical Services, Scottish Executive]
Wil Thissen [Delft University of Technology]
Discussion (25 minutes)
Summary of key issues by the chair (5 minutes)
3.30:Tea
4.00: Theme 4 - Future directions for policy appraisal
Chair: Andy Jordan
(2 presentations, 10-15 minutes each, plus discussion)
Robert Scharrenborg[Secretariat General, European Commission]
Claudio Radaelli [University of Exeter]
Discussion (25 minutes)
Summary of key issues by the chair (5 minutes)
5.00/5.30: Finish
7.30 for 8.00:Dinner, Restaurant ’22 Chesterton Road’
Day 2 (Tue 26 June): Planning the joint publication
9.00:Drawing together the conclusions of the discussions thus far /introducing the outline of a book/special issue
Introduced by Claudio Radaelli
9.30Breakout into thematic groups - brainstorming of chapters and collaborators
10.30:Coffee
11.15:Feedback from thematic groups/agree plan of action
Chairs: Andy Jordan and Claudio Radaelli
12.30:Lunch and depart
1
[1] Robert V. Bartlett and Priya A. Kurian (1999), The Theory of Environmental Impact Assessment: Implicit Models of Policy Making, Policy and Politics, 27(4).
[2] William Shadish, Thomas D. Cook, and Laura C. Leviton (1991) Foundations of Program Evaluation: Theories of Practice, London, Sage.
[3] Robert Elgie (1997) Models of Executive Politics: A Framework for the Study of Executive Power Relations in Parliamentary and Semi-Presidential Regimes, Political Studies, XLV,217-231. Rod A. W. Rhodes and P. Dunleavy (1995) (Eds.) Prime Minister, Cabinet and Core Executive, New York, St. Martin’s Press.
[4] Weiss, C. H. (1979) ‘The many meanings of research utilization’, Public Administration Review, 35:5, 426-31. Owens, S., T. Rayner and O. Bina (2004) ‘New agendas for appraisal: reflections on theory, practice, and research’, Environment and Planning A, 36, 1943-59.