APPLICATION OF CROSS-CASE ANALYTIC FRAMEWORK IN EVALUATING EMPOWERMENT OUTCOMES IN PROJECTS

Abstract:

This paper provides insights into empowerment outcomes of development projects by applying a cross-case analytic framework. It employs a methodology that allows for examining, identifying, and highlighting similarities and differences across projects that share a comparable profile within a specific empowerment related focus area. A key analytic theme is the emphasis on the degree to which the means of empowerment have been incorporated into the projects and how empowerment is linked to project outcomes. By analyzing within and across programs (e.g. ICT, micro enterprises etc.) we have identified the distinguishing features of projects and program categories. We have also developed linkages, in terms of lessons learned, between program areas by evaluating outcomes. Additionally, cross examination of the projects with similar profiles, but different social settings, provides insights into diverse ways of conceptualizing and operationalizing future initiatives. The diversity that we have captured in empowerment processes will help create an effective knowledge base for agencies that are working toward the goal of empowerment and poverty reduction in various institutional settings.

Introduction:

Poverty Reduction is the underlying objective in development initiatives. The approach that one adopts to tackle this multidimensional issue involves creating opportunities for empowerment, security of the poor, and community mobilization and collective action. Pro-poor strategies have been adopted in various sectors with differing thrust areas to deal with the common concern of narrowing regional and social disparities, empowering individuals and communities, and creating opportunities for the poor to move closer to the goal of eliminating poverty. The projects discussed in this document are premised on four key areas common to many development projects, which have shown significant results when deployed. They are: ICT for development, good governance through participatory budgeting and public expenditure tracking mechanisms, information disclosure as a mechanism for empowerment and creation of innovative financial services for the poor.

This document synthesizes the findings of multiple projects and highlights the relevance, importance and need for such initiatives. The objective of this endeavor is to provide a mechanism to disseminate field experiences, key issues and lessons learned The methodology is based on the application of a cross case analytical framework that scrutinizes, identifies and highlights similarities and differences across projects that share a comparable profile within a specific developmental domain. This approach captures variations in implementation strategies, context specific issues, and the degree to which the means of empowerment have been manifested and enacted. A key area for analysis is the emphasis on the degree to which the means of empowerment have been incorporated in the implementation and deployment of these initiatives. The paper also reflects on how the means of empowerment have been manifested and how they have played out in projects.

The empowerment model used in this paper is adopted from one outlined in the World Bank Source book on Empowerment and Poverty Reduction. The empowerment framework endorses the existence of four elements (means), which are integral for empowering citizens/communities[1]. They are (1) access to information, (2) participation, (3) accountability, and (4) local organizational capacity[2]. These elements are common across different developmental settings – be it social, institutional or political.

While the primary interest in this study is empowerment, the analysis also ties together the objectives of the projects, special features, impacts and lessons learned. Comparison of the lessons learned provides insights based on differences in the learning processes that can be attributed to social settings, institutional structures and organizational capacity. It will also act as a learning tool for organizations that intend to deploy similar initiatives.

Before conducting the cross-case analysis, individual case studies had been undertaken that employed the empowerment framework described above. Though individual case studies evaluated projects using the same empowerment parameters, a comparative approach helped bring out varying and distinctive features of the empowering process.

Relevance of a cross-case analytical framework:

By capturing, detail and depth, case studies are among the most effective methods of evaluating program or project performance and outcomes. Case studies evaluate the effectiveness of a project by highlighting the conditions, processes, facilitators, inhibitors and key issues. However, this approach only addresses a single project or a specific program. It does not provide a framework for comparing and contrasting the outcomes of multiple programs that share a similar profile and objective.

By deploying a cross-case analytical framework, we can evaluate the relative project performance by looking for patterns and themes in the data that are common across projects. This approach also allows us to analyze discrepancies in notable outcomes or attributes and their contributing factors.

Cross-case analysis is also an effective method to communicate the success and failures of programs with similar objectives. By analyzing the context, relevance, degree of maturity and expertise in implementation across programs we are able to provide a comparative framework for analysis.

The imperative for adapting this particular framework of analysis as opposed to one-off case studies, lies in its ability to synthesize the ideas embedded in theories, validated by practitioners and supported by policy at varying levels. Additionally, cross examination of the projects with similar profiles but different social settings will allow for diverse ways of conceptualizing and operationalizing future initiatives by distilling the lessons learned from the different contexts and examining their applicability to a given situation.

Though each of the projects inform the reader of unique experiences characterized by sectoral focus, social and cultural setting, there are certain attributes and dimensions that are comparable and offer a valuable source of lessons. The diversity demonstrated in the means of empowerment will help create an effective knowledge base for agencies that are working toward the goal of empowerment and poverty reduction in various institutional settings.

Objective:

The objectives of this initiative are to:

Identify and document the key issues across projects with similar focus

Analyze the degree to which projects have been able to contribute in empowering participants

Compare performance across projects to synthesize and glean greater meaning from the rich mosaic of discrete and disparate findings.

Demonstrate the importance of comparative analytical modeling technique in project performance analysis.

Target audience:

This paper is aimed at donor agencies because it will help in understanding and evaluating the relevance, efficacy, viability and feasibility of competing approaches. Thus, the projects that are covered will serve as exemplars and also contribute to the understanding of the social context, viability and feasibility. These factors will aid in providing a decision-making framework on which to base future decisions regarding similar projects. Non-government organizations (NGOs), which form an important group of stakeholders in implementing projects, will also benefit from this exercise. They can better estimate the effectiveness and feasibility of implementing intended programs by assessing the socio-economic, cultural context of the project area and applying the experiences of past projects.

Research Design:

Collection and analysis of background literature:

Information was collected from the following sources:

Concept papers discussing theoretical linkages, including the development and implementation of projects.

Empirical research papers on effectiveness, implementation strategies, and outcomes.

Websites of the organizations implementing the projects.

Case studies of the respective projects

Sample selection:

We developed four analytical categories: Information and Communications Technology as a tool for empowerment, Participatory Budgeting and Public Expenditure Tracking Surveys, Information Disclosure as a vehicle for empowerment, Institutional Innovations in Financial services for the poor and included cases into each of the categories based on the objectives of the programs. We grouped projects sharing similar profiles into these separate categories. Each of these categories describes a particular program that incorporates various projects. For instance, the category, Information and Communications Technology as a tool for empowerment would be a specific program area that includes projects such as Gyandoot, Grameen Phone, Kothmale Community Radio and M.S.Swaminathan Research Foundation’s Information Village Research Project.

The full categorization of projects is shown below:

  1. Information and Communications Technology as a tool for empowerment

Kothmale Community Radio: Sri Lanka (objective: to create an interface between the rural people and the internet)

Gyandoot: India (objective - improving local governance)

Grameen Phone:Bangladesh (objective – access to markets and business-related information)

M.S. Swaminathan Research Foundation’s Information Village Research Project: India (objective: to train and provide access to information to poor, rural farmers)

  1. Participatory Budgeting and Public Expenditure Tracking Surveys

Participatory Budgeting in Porte Alegre: Brazil (objective: to involve citizens from different areas and interest groups to define expenditure and investment priorities for the government)

The Women’s Budget Initiative in South Africa (objective: promoting sustainable democracy in South Africa)

The People’s Voice Program: Ukraine (objective: to promote public participation in building an open, accountable and effective government)

Public Expenditure Tracking Surveys: Uganda, Ghana, Tanzania and Honduras (objective: to track the leakage of funds)

  1. Information Disclosure as a vehicle for empowerment

The Program for Pollution Control, Evaluation and Rating in Indonesia (objective: to promote industrial compliance with pollution control regulations through community participation)

Kecamatan Development Program: Indonesia (objective: to alleviate poverty and improve governance by strengthening village capacities and increasing community participation)

Nike in Vietnam (objective: to make multinational corporations accountable to local communities)

Parivartan: countering corruption in Delhi (objective: to mitigate corruption in the public departments)

  1. Institutional Innovations in Financial services for the poor

Micro Enterprise Bank: Kosovo (objective: to provide financing for small and micro businesses)

Consultative Group to Assist the Poorest (CGAP) (objective: to provide flexible financial services to the poor)

Swayam Krishi Sangam: India (objective: to offer loans to the poor especially poor women to engage in economic activities)

Self-Employed Women’s Association (SEWA): India (objective: self-reliance of women in the unorganized sector)

The rationale for categorization of projects under specific program areas is based on the premise that such an approach will allow us to make intra-group and inter-group comparison.

Methodology:

The framework for analysis involves two forms of examination – intra-group (between projects within a particular category) and inter-group (between program categories). Tables 1 and 2 depict the procedures for this inquiry.

Table 1 is a sample of the analytic framework. It depicts the dimensions of inquiry for a particular category. In this case, we have taken Information and Communication Technology as a tool for empowerment. The table represents analytical dimensions against which projects were compared. The dimensions are: application domain/objectives, actor, action/implementation process, performance/impact, means of empowerment and lastly, issues and lessons. Application domain represents the areas/sectors that are served by these initiatives. These areas are adopted from the World Bank’s “Empowerment and Poverty Reduction Sourcebook”. These areas are – provision of basic services, improved local governance, improved national governance, pro-poor market development and access by poor people to justice. Actor refers to agencies or stakeholders that have played the role of implementers. Action or implementation process depicts the design, strategy and methods employed in the deployment of projects. Means of empowerment incorporates the four areas that are integral to empowerment. They are access to information, accountability, inclusion/participation and local organization capacity. Impact/performance highlights the positive outcomes of the projects. Issues and lessons outline the challenges, critical success factors and valuable inputs as lessons learnt which could be used as course correction measures for implementation of subsequent projects.

Analytical Dimensions / Kothmale Community Radio /
Gyandoot
/ Grameen Telecom / MSSRF Information Village Project / Learning Issues by Dimension
Application domain
Actor
Action or implementation process
Means of empowerment
Impact/performance
Issues and lessons
Learning issues by project / Overall learning issues

Table 1: analytical matrix

Each of these cells will be discussed in the body of the paper.

The thematic matrix which allows us to make a inter-group comparison is shown in Table 2. This matrix aids in grasping the pertinent convergent and divergent areas across programs. The idea behind such an analytical model is to provide the readers the different strategies used to achieve a common developmental goal – poverty reduction and empowerment.

Analytical Dimensions /

Program categories

Information and Communications Technology / Participatory Budgeting and Public Expenditure Tracking Surveys / Information Disclosure as a vehicle for empowerment / Institutional Innovations in Financial services for the poor
Application domain
Actor
Action or implementation process
Means of empowerment
Impact/performance
Issues and lessons
Learning issues by program / Overall learning issues

Table 2: Thematic matrix

Inter-program comparisons will depict similarities and differences across each program area with respect to each dimension. For instance, cross-examination of the application domain across each program category will allow us to recognize the areas that the programs target. Additionally, this approach helps us analyze what distinguishes a particular program area from another with respect to each dimension. Lastly, this methodology allows us to develop linkages between program areas by evaluating program outcomes.

I. Information and communications Technology (ICT) as a tool for Empowerment

ICT for development has been used in varying settings and sectors and with differing objectives and goals. ICTs have been shown to have an impact on poverty by providing opportunities for awareness generation through information access and by facilitating empowerment.

We have identified four cases using ICT with different objectives: a) theKothmale Community Radio: (Sri Lanka) was implemented with theobjective to create an interface between the rural community and the Internet; b) Gyandoot, in India had improving local governance as its primary goal; c) Grameen Phone in Bangladesh was implemented with the intention of providing access to markets and business-related information and d) M.S. Swaminathan Research Foundations’ (MSSRF) Information Village Research Project (IVRP) in India was deployed with the idea of training and providing access to information to rural poor farmers.

The common thread, in terms of objectives, lies in providing access to information to the rural community. While there appears that there are common traits across project implementation strategies, the way these ICT tools have been used varies across projects. For instance, the Kothmale project’s primary ICT instrument was radio and later it attempted to converge radio and the Internet. The Grameen Telecom initiative used telephone services in an effort to set up avenues for income generating activities. Gyandoot initiated intranet-based model to connect rural public. The major difference between the above mentioned initiatives and MSSRF’s Information Village Research Project is the implementation model in terms of both deployment as well as multiplicity of services offered.

Application Domain/Objectives:

The projects that have been chosen follow an integrated approach that incorporates all the five areas (application domain/objectives, actor, action/implementation process, performance/impact, means of empowerment and issues and lessons) in terms of focus. However, the primary foci for all the projects have been the provision of basic services, improved government and citizen relationship and pro-poor market development. Though the eventual expected outcome has been poverty reduction and empowerment, each project has had varying objectives with the primary one being knowledge delivery systems.

The common theme that ties these ICT-led initiatives has been rural connectivity, leveraging the potential of ICTs within a resource challenged and underserved community, and providing access to information that can facilitate the efforts for poverty reduction and empowerment. However, there are areas that are more pronounced in terms of service delivery systems that separate these initiatives into different but somewhat overlapping efforts. For instance, the MSSRF initiative has addressed a multiplicity of service areas as compared to other initiatives. Apart from information dissemination, MSSRF also incorporates a component of training that equips the participants with certain skills which opens avenues for employment.

The premise underlying the initiation of Gyandoot, a “government to citizen” activity was to tackle the problems pertaining to transaction costs associated with government services. The objective was to provide access to diverse market related and government information needs through information kiosks. Though Grameen’s objective is consistent with the other three initiatives, Grameen Telecom tends to differ in that it integrates the idea of information needs with that of creation of micro enterprise aimed at income generation for Grameen Bank members. The commonality that ties the Grameen Telecom and MSSRF in terms of their objectives is their thrust on inclusion of women as Village Phone (VP) and Knowledge Center (KC) operators respectively. All the undertakings constitute efforts for rural connectivity, serving the marginalized and disadvantaged rural population, training of volunteers, information access and empowering challenged and under-served communities.

Actor:

The actors or implementing agencies were a combination of efforts of non-profit organizations and intervention of the government. However, Gyandoot and Sri Lanka’s Kothmale Community Internet Project were largely government initiatives. MSSRF and Grammen Telecom used their own resources (though they also received external funding) to set up project offices and operations to initiate their work. Though the providers have their differences, what is common across all the four projects is community involvement and participation as major stakeholders. There is also evidence to support the view that a combination of implementing actors, styles, competencies, strategies, business models, and quality of service delivery leads to differing outcomes. This is not to say that actors determine the success of an initiative. However, the nature of the implementation agency and implementation strategies influences the outcomes of these endeavors (refer to implementation and impact section).