The Basics of ROME: What Results Matter and How Can They Be Identified

From Washington to ROME:

A Road Less Traveled by Public SectorReformers

by

Matthew Andrews1 and Anwar Shah2

Second draft, September 2001

Do not cite without authors’ permission.

Comments welcomed

1. Matthew Andrews can be contacted at

2. Anwar Shah can be contacted at

The views expressed in this paper are personal views of the authors’ alone and should not be attributed to the World Bank Group.

contents

introduction:

towards effective public sector reform in the developing world

section one: the need for ROME

chapter one:

the need for public sector reform in the developing world

/ 12

1.1. Administrative and governance weaknesses in the developing world

1.2. Chapter conclusion

chapter two:

common reform responses in the developing world

/ 20

2.1. Elements of reform response to administrative failure

2.2. Chapter conclusion

chapter three:

understanding reform experience in the developing world

/ 29

3.1. Identifying why reforms ‘fail’ in the developing world

3.2. Chapter conclusion

section two: presenting rome

chapter four:

rome: responding to reform need

/ 42

4.1. ROME: Making sense of the reform puzzle

4.2. ROME: completing the puzzle, tackling the problems

4.3. Chapter conclusion

chapter five:

participatory-decentralization in the rome model

/ 50

5.1. Providing citizens with a voice, creating demand for results

5.2. Decentralizing governments to access and respond to citizen voice

5.3. Conclusion: The citizen-government contract

Chapter six:

Results Oriented Management in the ROME model

6.1. Results oriented relationships in the administration

6.2. Providing the tools required for a results-oriented response to citizen demand

6.3. Conclusion: Facilitating results-oriented and responsive administration

110

chapter seven:

Results Oriented Evaluations in the ROME model

/ 114

7.1. Involving citizens in results-based evaluations

116

7.2. Transforming formal evaluations processes

120

7.3. Conclusion: Results oriented evaluation and incentives to govern well

133

chapter eight:

implementingrome: ensuring the road to rome is not one of wracks and ruins

136

8.1. Institutionalizing ROME

136

8.2. Implementing ROME

138

8.3. ROME, the institutional environment and reform space

141

8.4. Conclusion: Implementing ROME by increasing ‘reform space’

153

conclusion:

rome: an effective public sector reform for the developing world

155

selected references

156

introduction:

towards effective public sector reform in the developing world

Public sector reforms are pervasive in the developing world. Unfortunately the failure of public sector reform is equally pervasive in the developing world. Research by organizations like the World Bank find that many interventions fail to make lasting and effective impressions on recipient countries. Unresponsive, unaccountable, inefficient and ineffective bureaucracies seem impossible to change.

The problem with reform in the developing world

Reform options ranging from macroeconomic stabilization approaches to capacity-building initiatives and decentralization have failed to deliver on their promises of change in the last two decades. Consider evidence and perspectives on the issue:

Public expenditure adjustment is a major focus of public sector reforms in the developing world. A 1997 World Bank report of public expenditure reforms under adjustment lending examined the degree to which reforms between 1979 and 1994 actually helped to shape spending policies in developing nations (Huther, Roberts and Shah, 1997). The paper concludes that reforms based on adjustment had “small positive effects on expenditure patterns” but also points out a number of reform shortcomings. In particular, the report suggests that reforms have not significantly improved expenditure quality, or changed the government incentives and mind-set.

As with many reform observers, Merilee Grindle writes that, “Public sector organizations perform poorly in many developing countries; in some places they barely function at all” (Grindle, 1997: 481). On the subject of dominant 1990s reforms focused on improving public performance through civil service adjustments, she is equally pessimistic, quoting De Merode and Thomas that “no conclusive evidence was found of better pay and leaner staffing…leading to major product9ivity gains.”
The Medium Term Expenditure Framework (MTEF) is a promising reform in the developing world. The South African example of MTEF is seen as a particularly ‘good’ application of the reform. The South African government introduced the MTEF in 1996 as part of a general policy package emphasizing fiscal discipline, deficit reduction, and development (Walker and Mengistu, 1999). The MTEF is part of a vibrant set of reform policy and legislation. The reform is considered best practice in the development community (World Bank, 1998) largely because it has ‘led’ to reduced deficits (a key budgetary outcome), with deficits (as a percentage of total expenditure) falling from an average 15.86 between 1981 and 1993 to 11.25 between 1994 and 1999. Unfortunately, the MTEF reform tenure has also seen decreases in key expenditure types: capital spending comprised 9% of spending between 1981 and 1993, but only comprised 4% between 1994 and 1999; Fixed assets made up 2.94% of total expenditure up to 1993, but had a 1.49% share between 1994 and 1999 (South African Reserve Bank, July 2000). These results raise questions about MTEF’s ‘success’: Is the MTEF responsible for the improved fiscal discipline, or was it a factor related to economic upturn in the period? While fiscal discipline appears to have improved, other fiscal outcomes have not. Strategic spending has dropped in real terms and there is a concern critique that departments are underspending their budgets, leading to inefficiencies and low levels of budgetary responsiveness (Business Day, April 9 2001). Can a reform be successful if it improves one outcome but not others? Is the decline in capital spending a manifestation of Alesina and Perotti’s (1996) observation that medium term budgeting allows creative accounting and the postponement of ‘difficult’ projects?

Even recent reforms are problematic…

Recent solutions to the problems of governance and administration in the developing world are taken from the results and performance movement that seems to have proven effective in countries like New Zealand. ‘Results oriented management tools’ copied from such experience are seen in nations from South Africa to Malaysia to Uganda, Chile, Brazil, Indonesia, India and Mauritius. Frank discussion with development experts involved in the application of such tools, and with local reformers, indicate that these ‘tools’ are not taking shape in most settings. Reform evaluations and case studies of ‘results oriented management’ and ‘new public management’ tell similar stories. Consider evidence on the issue:

Desai and Imrie (1998) provide interesting observations of reform to the “bureaucratic and protectionist regime” of India. New public management and performance-based interventions in India date back a number of years. In these reforms, the authors find more rhetoric than results, however, especially pertaining to proposed participation in the initiative: governments remain insular even though policy and legislation requires that they open their systems to citizens. Desais and Imrie imbibe a generally negative tone, stating that: “The possibilities for enhanced democracy within the managerialist state are being circumscribed by a heightening of clientilism and co-option” and “Much of the new managerialism is….contradictory and flawed, characterized by de-democratizing tendencies and a fixation with procedural and technical processes” (1998: 645).

Problems with new public management reforms are also evident in Africa. In Malawi, a number of interventions were introduced between 1991 and 1997 (Adamolekun, Kulemeka and Laleye, 1997). These included pay reforms, the introduction of a Medium Term Expenditure Framework, and a privatization program. If one evaluates the reforms in terms of their implementation, they appear generally successful. Most of the mechanisms and management processes have indeed been implemented. If one evaluates the reforms in terms of their impact on behavior, they do not appear that successful. There is little evidence of actual change—the majority of civil servants still hold to old patterns of interaction and behavior and the old rules of organization. Further, reform elements do not always support each other (decentralization of government and centralized financial planning, for example), and although some technical process changes have been praised, old processes remain in operation.

Larbi (1998: 378) provides an example of problematic new public management interventions in the Ghanaian health sector, where managerial reforms at central government level are blamed for the failure of “decentralization and other long-term institutional reforms” because they facilitated a tightening of central controls over the key managerial tasks of spending and staffing. Where the general basis of new public management in countries like New Zealand is results based flexibility, in this situation the reform is manifest as a strangling, controlling intervention used by central authorities to expend their interests.

Integrated Development Plans (IDPs) are intended to facilitate performance-based, participatory government in South African municipalities (Africa, 1999; Otzen et al., 1999; Wallis, 2000). All municipalities are required by law to introduce such plans into their management processes as the primary driver of their operating and capital budgets. The plans are intended as the main means by which local governments develop strategic policy capacity, mobilize resources, and target such resources to strategic activities. The national Department of Provincial and Local Government has recently begun to ask how well IDP implementation is progressing—whether the reform is proving a ‘success’. While the results of in-house surveys are reported under a heading that “most municipalities are now involved in integrated development planning” (DCD,1999) the surveys actually show that the reform is not taking off as planned: as of 31 March 2000 only 46% of the 483 municipalities responding to the quarterly Project Viability survey had formally approved IDPs in place. The same percentage answered that communities are involved in updating IDPs, while fewer answered that the IDP constituted the budget driver its was intended to (Department of Provincial and Local Government, 2000). Furthermore, case studies reveal that IDPs are having no effect on participation (with the participation element sorely limited). The IDP also lacks significant effect on failures associated with administrative information flows. In essence, observers question whether the reform has had any effect on incentives, behavior or outcomes. Even where it is in place, IDP has not countered ‘business as usual’.

Is there hope for a ‘better’ approach to reform?

Given this evidence, it seems as if another reform idea is proving an unsure ‘solution’ to public sector problems in the non-industrialized world. But there is hope, evident in isolated cases of effective results-oriented reform in developing countries. From the influence civic results demand has had on bureaucratic responsiveness in Rajasthan in India, to the successes with community-based evaluations in Uganda and Malaysia, it is possible to identify experiences where the focus on public sector outcomes is indeed causing governance structures and incentives to change and outcomes to improve. Consider these cases:

The Malaysian public sector adopted local level, citizen-focused, results oriented government in the 1990s. The reform is described as having “encouraged a change in the mindset of public officials, who are now required to search for more efficient and effective methods for the delivery of public services that satisfy customers” (Chiu,1997: 175). The reform envisions the civil service and the government working closely with business and civic organizations, leading to mutually beneficial relationship (Abdul Karim, 1997). The changed mind-set has gone a long way in improving the performance of the Malaysian public sector (Mohamad,1997).

In Uganda decentralization and participatory-based performance budgeting reforms are cited as reasons why crucial sectors like education and health are seeing a turn-around and increased responsiveness(Kisubi,1996; Reinikka,1999; Brinkerhoff,2000). The turn-around is remarkable because the three biggest problems argued (by authors like Schick,1998) to limit reforms are evident: The traditional bureaucracy followed no rules and had little capacity, the service types resemble complex craft and coping products, and there is no private market from which competition and competitive forces can be felt. The results are clear, in a survey in 1998 the Ministry of Education “found major improvements in the flow of funds” (Reinikka,1999). The mix of decentralization, contract, and evaluation (enabled by improved information access) led to increased financial accountability.

Another example comes from Bolivia, where the Popular Participation and the creation of Territorial Base Organizations set up to monitor the provision of public services (community evaluation) have had a marked impact on accountability and responsiveness. “While the functioning of the system is still hampered by a lack of resources, illiteracy, difficult transport, unclear procedures and the absence of transparency and of a democratic culture, there are also indications of increased accountability” (Schneider, 1999, 526).

A final example comes from Rajasthan province in India. It points to the centrality of information in reform). It also shows that reform can emanate from within civil society. Essentially, the ‘reform’ in Rajasthan involves local communities demanding the rights to photocopy government documents and then performing informal, community-based evaluations of their governments . The community evaluation devices take a number of forms, including public hearings at which detailed accounts derived from public expenditure records and other documents are read aloud to assembled villagers. “Through this direct form of ‘social audit’” discrepancies have been identified and public officials (politicians and administrators) have been bought to account. Furthermore, there is now an active process of social negotiation regarding how government documents are structured so that they maximize the civil society window (evaluations) and facilitate government learning from civil interaction (ROM) (Jenkins and Goetz, 1999).

Learning from reform ‘success’ and charting a new reform approach, ROME

Beyond identifying these positive cases, it is also possible to identify the factors that separate reform experience in these ‘successful’ settings with the prevailing, ‘unsuccessful’ results-oriented reform applications. The dominant observation when comparing past and current ‘underachieving’ reform examples with the sub-set of successes, is that results oriented reform only appears to effectively change the incentives and focus of governments when it is sourced within representative, participatory structures at the local level. Essentially, ‘bottoms-up’, decentralized participatory structures are the basis of effective results oriented reform in developing countries. These structures provide the source of results identification (citizens communicate their demands) and the prime location for results evaluation (citizens are in the best place to identify if their demands are met or not).

This monograph builds the observation that localized participation is central to results-oriented reform in the developing world into a model of reform, Participatory, Results Oriented Management and Evaluation (ROME). The model is inherently bottoms-up, combining three elements: participatory decentralization, results-oriented management and results-oriented evaluation:

Participatory decentralization provides the foundation in which to locate results oriented management and evaluation ideas. Through this element, political and administrative entities have their incentives defined as they focus on citizens at the local level, are directed to the results that matter most (as identified by citizens), and are ideally located for capacity-enhancing, results-oriented civil society partnerships. This element counters the top-down, paternalistic model common in developing countries with bottoms-up, localized governance structures in which citizens drive, inform and capacitate their own governments.

Results-oriented management (ROM) takes place within the context of the participatory, decentralized governance system in ROME (unlike its top-down structure in countries like New Zealand). ROM provides tools like performance-based budgeting that are only effective in this setting, where citizens can hold governments accountable for their implementation and ensure their influence. ROM tools, and the implied delegation within the ROM approach, are the basis of a major public-sector mind-set change in the developing world: from a process and probity concentration in administration and management to an output and outcomes focus, in which government focuses on what it does for citizens (as measured in outputs and outcomes produced).

Results-oriented evaluation (ROE) is the final element of ROME. ROE solidifies the incentives created by results-oriented management techniques, providing an important formal and social ‘check’ on governance and an important feedback loop for public actors to improve their operations. Politicians and administrators are held accountable for the results they produce as these results are regularly evaluated—through internal and external audits as well as through community evaluations (or social audits). These evaluations create a link between the participations and results oriented management elements of the reform model and help to ensure that governments are accountable and transparent.

These elements will all be recognized by development experts and government officials in countries seeking reform solutions, who might ask: “what’s the difference between ROME and reforms already in place?” Indeed, reform elements like these have been regularly mentioned in theories like new public administration (Marini, 1971; Frederickson, 1980), new public management (Osborne and Gaebler, 1982; Barzelay, 2000) and new institutionalism (North, 1990; Ostrom, Gardner and Walker, 1993; Picciotto, 2000). They are also commonly alluded to in the development literature, with Picciotto’s recent statement standing out: “To achieve poverty reduction, we need new concepts that are holistic, results based, and participatory” (Picciotto, 2000: 361).

But ROME is different to the established reform theories and practices.

The ROME differences

ROME differs on two dimensions: (1) The way in which elements are organized in ROME is significantly different from conventional reform approaches, and (2) The participatory decentralization element is more pronounced in ROME than it is in other reform approaches, constituting the proverbial reform ‘stumbling block’. In terms of the first of these, convention in the development community emphasizes the tools of results-oriented management in centralized reform initiatives. Participation, decentralization and evaluation are politically correct ‘add-ons’ downstream, with participation generally limited to ‘professionally appropriate’ groups with pre-existing access to government, decentralization occurring only when local governments have the capacity—which is never, and evaluation attached as a concept needed ‘when there is something to evaluate’ by central authorities (limiting the change effect evaluations can embody). In contrast to this conventional approach, ROME emphasizes participatory decentralization as the foundation of results oriented management and evaluation, with the latter pair implemented jointly in response to civic demand at the local level. The model so depends on citizen participation and influence that a non-reform is considered a more viable option than a non-participatory reform.