Meso-Matrical Synthesis of the Incommensurable

Meso-Matrical Synthesis of the Incommensurable#

Bojan Radej*

Abstract: Many of those currently involved in the assessment of government interventions’ contribution to the social welfare have had significant difficulties in summarizing known but sometimes contradictory facts into summary conclusions. There are incommensurable viewpoints with regard to many social realities, and they provide us with very different ‘numeraires’ and macro-views of the world which are not reducible to common denominator. In particular, there is a disagreement over assumptions about the aggregation of the assessed policy impacts (micro) into summary conclusions (macro) that inform decision-makers operating at meso level. A new method is proposed of meso-matrical impact assessment (MIA) of policy interventions to cope with social incommensurability in scale (micro-meso-macro) and scope (economic-social-human-natural). It is based on Leontief's square input-output matrix that evaluate overlaps between incommensurable sets on their margin. In MIAsecondary issues play a central role. Practical example illustrates the achievement, schematic presentation generalises the principle. Paper explains that social or political common ground is not the condition of unity and cohesion with the meso-matrical perspective in mind.

Keywords: Incommensurability, scope, scale, meso, matrix, evaluation.

JEL Code: A13,A10,F12, A19, C50, H40

# Revised 15. December 2008; submited in October 2008.

*Bojan Radej, MSc in economics; independent social researcher; Slovenian Evaluation Society; Ljubljana, Slovenia,

1. Introduction

Sustainable developmentis a norm that obliges governments to put forward policies that ‘meet the needs of the present generation without compromising the ability of future generations to meet their own’(WCED, 1987). This calls for multi-criteria assessment of policy interventionswhich significantly complicates summative evaluation. There are incommensurable viewpoints and values with regard to different social realities, such as economic, social and environmental, or local and global, which provide us with very different ‘numeraires’ and views of the world, and they are not reducible to one common denominator (Funtowicz and Ravetz, 1994, in Martinez-Alier et al., 1998).[1] For the fact that variegated forms of welfare need to be taken into account implies that ‘different principles of social primacy and legitimacy must be reckoned with and reconciled’ (Wacquant, 1997) in a multi-criteria impact assessment (IA).

Incommensurability is not accompanied only by value comparisons but also by scientific ones since its formal systems decisively differ in the propositions they take as their axioms. When confronted with multidisciplinary issues involving scientific controversy, even competent, honest and disinterested scientists may arrive at different conclusions because of systematic differences in the way they summarise available information of one and the same world (Kuhn, 1970; Mumpower et al., 1996).Kuhn (1993 in Sankey, 1998) concludes that theories and scientists are unable to truly reflect reality.When values and facts are based on incommensurable oppositions, there is no objective basis for rational choice between theories and than no neutral observation of social reality is possible. No individual person or theory is than able to fully comprehend social complexity – such as sustainable development - and its multiple meaningsare persistently disagreedcollectively. This alleges that multiplicity is one of the determining conditions not only of values and facts but also of social systems (Munda, 2004).

The current welfarist model is on the other hand unable to conceptualize and operationally apply substantivelydifferent aspirations in a public sphere. Bar-Yam(2003) thinks that governments have experienced systemic failures in addressing complex welfare issues such as large-scale and multi-scope matters. They rely on reductionist approaches, which simplify the welfare concern by dividing it into independently ministered sub-problems. The very act of simplifying by sub-divisions loses the interconnections and therefore cannot tackle overlapping concerns (Chapman, 2001). The analytical concept of reality, is for example based upon the idea that the total is equal to the sum of its parts and that the parts can be quantitatively described, is insufficient for studying complex issues. A complex phenomenon is a structure of sets, connected by ordering relations of ‘super-, sub- and co-‘, where each sub-system (itself a system) has a plurality of relations of all three sorts with other subsystems (Ravetz, 2006). Reasoning on different levels is discontinuousand transgression between them (Weaver, Rotmans, 2006) breaks linearity. For complex social considerations, where ‘sum differs from the aggregate of their parts’ (Veen, Otter, 2002), the summation procedure is far from trivial (Veen, Otter, 2002), so there is an aggregation problem (Foster, Potts, 2007) in the summative IA.For sustainable development to flourish, an approach to welfare considerations is needed that is summative in a way that is equally sensitive for all incommensurable aspects.

Existence of an aggregation problem in policy studies is probably less obvious than the vast consequences it manufactures. In policy-making even good individual policies, based on strong values and common sense, often lead to disappointing results.Inability of a government to coordinate and aggregate opposite claims can be hold responsible for generatinggrowth antagonismssince 1960’s such as income inequality (Giraud, 1996; Milanović, 2006), and happiness paradox (Easterlin, 1974) where increased material wealth is not accompanied by a subjective satisfaction; an ecological footprint (Wackernagel, Rees, 1994) shows that the wealthiest nations actually consume resources at a volume that could be sustained only if the planet Earth were two to almost six times larger. It was not that growth antagonisms took place in the absence of policy evaluation. Evaluation has increasingly obtained its role in a policy cyclesince the sixties, but it failed to address deep conflicts that accompany large-scale and multi-scope policy interventions.So ‘ill development’ took place with our informed knowledge. Thus the question how to think intelligently about the conflict nature of trade-offs between different aspects of welfare that are characterised by the incommensurable oppositions (Williams, 1972, in Martinez-Alier et al., 1998) arises again.

Arrow explained using his impossibility theorem (1951, Nobel Prize for Economics in 1972), that it is not possible to scale up from all individual preference functions to produce a social welfare or “public interest” function that satisfies desirable properties of an aggregation process (Evans et al, 2002) such as non-dictatorship. The theorem has challenged the presumption thatan aggregation of individuals’ wants such as in Condorcet’s majority voting rule or Bentham’s ‘the largest utility for all’can simultaneously meet individual and collective expectations. It further demonstrated that there is a discontinuity between rationality at the individual and group level (Evans et al., 2002) that mirrors the tension between private and public in the governance of the social. Coleman (1986 in Åberg, 2000) maintains that this micro-to-macro link, also referred to as social causation (Sawyer, 2003) is a controversial and most poorly developed part of sociological theory.

Associated with these meta-theoretical concerns, there is an apparent paradigm crisisinIA(Virtanen, Uusikylä, 2004; Hertin et al, 2007). IA aims to suggest more consistent policy interventions (Hertin et al, 2007). However, evidence shows that policy response to evaluation recommendations is very limited (Picciotto, 1999). Existing IA methods are designed for the appraisal of homogeneous interventions or ‘projects’ with only limited diversity of impacts (Elbers et al., 2007; Rotmans, 2002), while governments work with heterogeneous issues. IA conventionally forwards two-valued or binary logic which is focused on only two scopes of the assessed phenomena - cause and effect -, such as economy’s impact on environment. But government operates in multi-value context, where causes and effects are not straightforward. The problem is also that majority of IA models assess processes only at one scale level, micro or macro. This all results in a ‘evaluation deficit’ – the situation in which most assessments provide the kind of information that does not inform policy-makers whether a global objectives can be met – consequently, evaluations remain under-utilized (Stame, 2004) or even get mis-used.

Summation is the Achilles heel of the evaluation effort (Scriven, 1994). It arises from a disagreement over assumptions about the aggregation of numerous policy impacts when assessment phenomenon is multiple in scope (economic, social…) and scale (micro, macro). As elaborated by Carlsson (2000), different evaluators assessing the same policy interventions may come to opposite conclusions solely because of differences in their assumptions about the summation of the assessed impacts across incommensurable scopes and scales.We often think of them as coupled because of the most common ways in which we encounter them (Bar-Yam, 2004):consider observing a system through a camera that has a zoom lens. For a fixed aperture camera, the use of a zoom couples scope and resolution (scale)in the image it provides. “As we zoom in on the image we see a smaller part of the world at a progressively greater resolution. This leads to a particular relationship of observations of parts and wholes, suggesting that when observing details of the system, the whole is not being observed. We must allow a decoupling of scope and resolution, so that the system as a whole can be considered at differing resolutions as well as part by part. For this purpose scale can be considered as related to the focus of a camera—a blurry image is a larger scale image—whereas scope is related to the aperture size and choice of direction of observation” (Bar-Yam, 2004).

For evaluator this means that what one sees is always predefined in scope and scale of his or her observation.Recognition of this is central for the methodology of meso-matrical impact assessment (MIA) that is proposed here. It builds on previous observations that to summarise the trade-offs between intrinsicsustainable values, evaluator needs to derive conclusionsacross different scalesand scopesat which appraisal takes place (Weaver, Rotmans, 2006; Dopfer et al, 2004; Dopfer, 2006; Easterling, Kok, 2002). As Chapman (2001) elaborated: the theory of complex systems offers an alternative to conventional IA strategy for studying large-scale issues: instead of going down to the elementary level as in the standard reductionist model, the system theory justifies the opposite, namely going up a level of abstractionwhich establishes a multi-scale view.

The aggregation problem in IA that apply the impact matrix method has been first addressed by Luna Leopold et al. (1971). They proposed a detailed expert-based impact matrix at the micro-level from which the macro aspect remains absent. It is concerned with only two scopes – economic and ecological – and assesses the possible impact of the former on the latter (second chapter). Recently Ekins and Medhurst (2003) proposed a method for IA in the complete multi-scope perspective. They developed a reduced version of Leopold matrix, but with the scope dimension expandedon the four scopes of sustainability (Leopold-Ekins-Medhurst impact matrix – LEM); they further allowed for aggregation of all impacts, represented in rows of their matrix on each particular scope – by column of the LEM. An analogous approach is conventionally applied in various standard IA procedures, such as the strategic IA (2001/42/EC), territorial IA (ESPON – 3.2, 2006), and ex-ante IA of the contribution of the EU structural funds to sustainability of regional development (GHK et al, 2002). Nevertheless, LEM’s summation approach is inappropriate, as the effects of individual policy measures on each appraised scope are not homogenous (Rotmans, 2006), and therefore not commensurable. So impacts could be aggregated by column in LEM only by source and area of impact, i.e. partially. This reorganises LEM into square matrix of Leontief (1951) with equal number of rows and columns (third chapter). We learn that social complexity does not prohibit aggregation in IA when one takes apart weak from strong incommensurability. This permits two step summation procedure in IA - the second step is synthesis with the correlation of intersections between different scopesthat are obtained from the Leontief matrix. Both summative steps are illustrated with a practical example. The regional development programme for the Slovenian region of Pomurje for 2007-2013 (RDPP) is first assessed ex-ante with LEM and then with the MIA. Their summary results are found diverging. The second part of the paper studies reasons for this incompatibility. To accomplish this task a complete multiple-scale perspective is introduced into the IA (fourth chapter). When scale and scope aspects of IA are combined it becomes obvious that inconsistency in standard IA arises from inappropriate interpretation of assessment results across scope and scale. MIA offers new understanding on how to integrate policy concerns without imposing uniform norms. The implications for the governance of the social close up the paper. Work continues.

2. Summation in standard approaches

The first impact matrix generationmethodologies for the ex-ante assessment of large scale policy interventions such as the one elaborated by geologist Luna Leopold et al. (1971; for review see Munn, 1979) explicitly reject the summation of multifarious policy impact into aggregate indicator. They were concerned with the impact of economic policy measures on the components of the environment. Their impact matrix lists the 100 most important economic policy actions horizontally and 88 environmental fields of impact vertically. The intersection between these two scopes creates a detailed matrix with 8800 fields – each further divided into four subfields that characterise each impact by its size (strong, medium, small), direction (positive, negative), probability and risk factor (critical or not). In this way, all possible impacts are presented in sufficient detail to enable informed decision. In the finest analytical manner, Leopold aims to provide the large picture of complex policy issue with a detailed description of all its elementary parts. The first generationmethodologies claim that impacts should be presented disaggregated, leaving policy-makers with full responsibility for the synthesis conclusions in the light of a policy decision. There must be a clear demarcation in IA between the evaluator and policy-makers to ensure that value judgement rests with policy-makers and is not inappropriately shifted towards the evaluator. Refusal of aggregation is then important because it protects the evaluator from the political interference (Kunseler, 2007) that accompanies decision-making.

The rejection of summation in evaluation of large-scale policy interventions is problematic. Let us recall the impossibility theorem, which implies that the social optimum can not emerge from a reduction procedure. It is precisely the inability of policy-makers as social aggregators that calls for a policy evaluation in the first place. If evaluators fully accept their role in the policy cycle, they should not refuse summation of evaluation results! Another difficulty is that Leopold and even many contemporary authors observe incommensurable values as antagonistic, so they can not explain how opposing views work together, without which it is impossible to say anything about integrative capacity of a policy proposal. Failing to summarize impacts in the assessments is “letting the client down at exactly the moment they need you most” (Scriven, 1994).Without aggregation, an appraisal will usually not generate clear-cut conclusions but ‘information overload’ fostering a piecemeal rationality. Results that are too fragmentary and unrelated offer little value, fail to satisfy information needs at the strategic level and cannot provide an informed basis for a decision. All too often disaggregated results produce banal answers to complex and multidimensional societal problems (Virtanen, Uusikylä, 2004). Evaluation methodology hewing simply to the production of non-overlapping information tends to underplay the need for learning and substantiation of findings, legitimating the disregard of stakeholder issues (Stake, 2001), which alone is sufficient to leave an evaluation utterly exposed to a political interference.

To synthesize or not to synthesize – this is how the question is again posed for Scriven (1994). In somewhat less tragic storyline he explains that we need to distinguish between cases in which it is improper to push for a synthesis and cases in which it is improper not to. This fuzzy instruction may not be exciting for writers of tragedy, but it is very helpful for an evaluator. To see how this advice helps us, we first need to broaden the conceptual framework of social incommensurability to identify the conditions under which summation is feasible even though commensurability of welfare indicators is abandoned. Incommensurability is of two kinds: weak for individual considerations and strong for collective ones. Two arguments aim to justify the distinction below: uniqueness of individual elements of the system must to be delimited from incommensurability of social issues; under certain conditions even different aspects of social incommensurability can be traded between each other.

At the elementary or micro level the social systemdoes not entail incommensurability but uniqueness(Li Xiaorong, 2007): for example, even though individuals are unique, they can be compared because their unique formation of characteristics is obtained from the shared base of human characteristics. The result is that individuals, their statements and acts are in some characteristics similar and comparable to some other individuals, but not to all others. So similar sortsof individual uniqueness are partly commensurable andcan be locally averaged.

The aggregation of diverse impacts takes not only that the positive (or negative) effects of one particular policy measures can be summed up. More crucially it also assumes that a positive effect of one policy intervention outweighs its own or another policy’s negative effect. Is it, for example, sustainableto trade tons of greenhouse emissions for euros in EU’s scheme of tradable pollution permits market, when we know that greenhouse emissions can cause irreversible changes in the atmospheric conditions? Such a trade-off is not adequate as a general or macro principle, because economic and environmental aspects of welfare are equally important. However, one needs to reason in a multi-scale perspective, where micro is different from macro. Trade-offs between economy and environment are not accompanied by incommensurability in every single case, or at least people and their communities are not willing to treat them as such.

To incorporate this peculiarity in policy evaluation, system thresholds of sustainability – such as ecological, human and social ones – have emerged as anew assessment determinant (for a survey of literature see Muradian, 2001). System thresholds exist that should not be crossed for this would endanger the basic integrity of the system. Policy interventions that could jeopardize the thresholds should be avoided in order to circumvent (even more) antagonistic confrontation between, say, climate conditions and economic growth. So the concept of thresholds is closely linked to the concept of incommensurability. As Wiggins (1997) explains, two values are incommensurable if “there is no general way in which A and B trade-off in the whole range of situations of choice and comparison in which they figure”. Measurable social phenomena are incommensurable in IA ‘only’ beyond (or sometimes below) their threshold values. However, within safety limits, an agent either does not sense the difference between two conditions, or refuses to declare a preference for one or the other (Luce, 1956 in Munda, 2006) such as in the cases of minor damages that stay within ‘safe’ ecological standards. The practical consequence for policy evaluation is that positive and negative impacts can be aggregated (averaged, regressed, correlated, benchmarked etc.) only if they occur in the safe interval of system normality. Reliance on thresholds also simplifies evaluation because risk factors would be already incorporated in the definition of thresholds.