Very Preliminary Do Not Distribute Or Cite
VERY PRELIMINARY – DO NOT DISTRIBUTE OR CITE
The Miracle of Microfinance Revisited:
Evidence from Propensity Score Matching
Inna Cintina and Inessa Love[*]
Abstract
We provide new evidence on the effectiveness of microfinance intervention for poverty alleviation. We apply the Propensity score matching (PSM) method to data collected in a recent randomized control trial (RCT) in India by Banerjee et al. (2014). The PSM method allows us to answer an additional set of questions not answered by the original study. First, we explore the characteristics of MFI borrowers relative to two comparison groups: those without any loans and those with other types of loans, predominantly from family and friends and moneylenders. Second, we compare the impact onexpenditures of MFI borrowersrelative to these two comparison groups. We find that microfinance borrowers have higher expenditures in a number of categories, notably durables, house repairs, health, festivals and temptation goods. The differences are stronger relative to those without any loans. Our results suggest that microfinance can make a larger difference for households previously excluded from other credit sources. However, there is no significant difference in total expenditures, likely because the categories showing significant increases represent a relatively small portion of the total expenditures.
- Introduction
The impact of microfinance on poverty alleviation has become a topic of intense debate in recent academic and policy literature. Originally touted as a means for poor people to escape poverty, more recent reports suggest that the impact is likely to be small and often mixed, with such negative effects as over-indebtedness, leading to illegal organ sales and suicides in extreme cases.[1]
The key challenge of evaluating the impact of a microfinance program is to make sure any observed outcomes are due to the program itself and would not have occurred without the program. Thus, it is not sufficient to compare those with a microloan and those without it because people who obtain a microloan may be fundamentally different from those that don’t. Randomized Control Trials (RCTs) have increasingly become the preferred method of evaluation for many development economists (Duflo and Kremer, 2005). However, an important limitation of RCT in evaluating microfinance effectiveness is that researchers cannot randomly assign the recipients to receive a microfinance loan for two main reasons.First, not everyone in a randomtreatment group would want to obtain a loan, which will result in a selective take up. Second, the financial institution has to ensure the borrower’s creditworthiness and thus cannot allocate loans randomly. Both of these problems make it difficult for an RCT to evaluate the impact of microfinance on the individual level.[2]
To avoid these problems, many recent RCTsevaluate the impact of the microfinance introduction to specific geographic areas.[3]In these studies, the microfinance is offered in some areas (villages, slums, towns) but not in others. Regardless of how many people actually take up the microfinance, the outcomes are compared across areas – i.e. a treated area is compared with a non-treated one. However, such studies can only produce the intention to treat estimates (ITT), which is the average impact of making microfinance available in an area (i.e. averaged over those who take it and whose who do not), or the Local Average Treatment Effect (LATE) if the IV estimator with random assignment as an instrument for take upwas used.[4]Neither method can produce the estimate of the impact of microfinance on theindividuals or householdsthat actually take out the loans(i.e. the Average Treatment Effect on the Treated, TOT), which is of critical policy importance. This gap can be filled with the Propensity Score Matching (PSM) method and it pursued in this paper.
The PSM method creates a statisticalcontrol group of individuals without microfinance loans that has similar observable characteristics to the treated group, i.e. individuals with microfinance loans. Thus, the control group is created to be observationally equivalent to the treated group. While controlling on observables (such as education, gender, family size, living conditions, employment status and others) will reduce many of the significant differences between participants and non-participants, it cannot address the differences in unobservable characteristics, such as the entrepreneurial talent of the borrower, their time and risk preferences or their social support network. It is likely that such latent factors affect the selection of people to obtain an MFI loan and the outcomes of interest,such as poverty status. Thus, the PSM approach is not a perfect solution to identify the exact impact of microfinance on individual level. Nevertheless, multiple studies that compared performance of PSM estimators relative to experimental results have argued that PSM can produce accurate estimates if three conditions are met: first, sufficient data on observable characteristics are available to be used for matching; second, the outcomes for participants and nonparticipants are measured using the same survey instruments; and third, participants and non-participants come from the same local markets (see Heckman et al. 1997, 1998a, 1998b, Diaz and Handa, 2006). Our data satisfies all three of these requirements. Moreover, the same authors argued that the bias due to unobservables is small relative to the bias due to observables. In addition, the PSM method has been used successfully to evaluate impact of different programs in a wide variety of settings (seeRavallion 2008 for a survey). Thus, PSM appears as an appropriate method to apply in an effort to evaluate microfinance effectiveness and has an important advantage of allowing a direct comparison of borrowers to non-borrowers.
We apply the PSM method on the data collected by a recent randomized experiment by Banerjee et al. (2014). The PSM methodology we use allows us to answer an additional set of questions. Specifically, our study addresses four main questions. First, what are the main characteristics of microfinance borrowers? The knowledge of the borrowers’ characteristics is important for microfinance program targeting, especially in light of low microfinance take-up identified in many of the recent studies (see footnote 3). Second, what is the impact of microfinance on consumption and expenditures of average borrowers relative to non-borrowers? Third, how do microfinance borrowers differ from those using other types of loans? The rich dataset allows us to compare whether characteristics of those who borrow from microfinance organizations are different from those who have no other loans vs. those whoborrow from other sources (predominantly from family and friends or from money lenders). Finally, we can also identify whether the impact of microfinance depends on the comparison group we use – i.e. those without any loans vs. those with other types of loans. From the policy perspective it is important to know whether microfinance targets same or different types of households that use other sources of finance and whether it provides same or different benefits relative to other types of credit. To our knowledge such comparison has not been done in the previous literature.
We have several main results. First, we present a profile of MFI borrowers, who are more likely to be middle aged, have low education and be relatively poor (i.e. they have overcrowded living conditions, no landline, and receive government assistance). They are more likely to have prior MFI experience. The characteristics of MFI borrowers are mostly similar when compared to those without any loans vs. those with other types of loans. However, there are a few interesting differences. For example, we find that those with a health-related accident are more likely to have an MFI loan relative to those without any other loans, but are not more likely to get MFI loan among those that already have other types of credit.
Second, we find significant increase in many of the expenditure categories. In general, the comparison of impact on MFI borrowers relative to those without loans and relative to those borrowing from other sources yields many similar results: increased durable purchases, home repairs, festivals, and temptation goods. However, the magnitudes of the impact are often larger in comparison with those without any loans. In addition, health expenditures show different patterns for these two comparison groups and a clear increase is only observable in the case of comparing MFI borrowers vs. non-borrowers. However, these results should be taken with some perspective: the categories of expenditure that we find significantly increasedrepresent a relatively minor share of total expenditures. Thus, food and non-durable expenditures, which are the largest shares of the total expenditure, show no significant changes. This explains why we don’t find a significant increase in the total expenditure either.
Our results are robust to including neighborhood fixed effects (and seem mostly unaffected by these effects) and are largely robust to four different matching methods. We also find that of the four matching methods we use, the nearest neighbor with replacement offers the largest standard errors and is least consistent with other methods.
Finally, we compare the results from the PSM method to RCT results obtained on the same dataset by Banerjee et al. (2014). Two of our main results – the increase in durables and the lack of overall increase in total expenditures – are the same using both methods. This provides some validation for using the PSM method in evaluating microfinance effectiveness. However, PSM can also provide more nuanced evidence, such as comparing borrowers to different comparison groups of interest. Thus, whenever rich data exists, PSM can be a useful evaluation method with significantly lower costs relative to an RCT.
Some earlier papers used other non-experimental methods to estimate the impact of microfinance on actual borrowers, most notably Pitt and Khandker (1998) and Khandker (2006). However, these studies are still surrounded by controversy; see for example the re-evaluation by Roodman and Morduch (2014) and a response to re-evaluation by Pitt (2014). In light of this mixed evidence, our paper serves as an important addition to a scant non-experimental microfinance evaluation literature.
To the best of our knowledge only Floro and Swain (2012) have previously used the PSM method to study the impact of microcredit on individual level.[5] Their paper is also set in India, but they study the impact of bank-connected Self Help Groups (SHG) rather than loans from a specialized microfinance institution. In addition to this, our study has several other important differences. First, we compare MFI borrowers to two distinct groups of controls: those without any loans and those with loans from other sources, such as family and friends and moneylenders. Second, we have a larger and richer dataset, which allows us to use a lot more of control variables in RSM estimation and makes it much more likely that better matches can be found and bias can be minimized.[6] Third, while Floro and Swain (2012) focus primarily on an indicator of vulnerability, which they measure as the variance of consumption, and average food expenditures, we have a much wider set of outcomes, including purchases of durables, education, health expenditures, home repairs and other expenditure categories, which allows for broader focus. Fourth, we use the data that has been used in an RCT evaluation, which allows us to compare the performance of these two methods. Thus, our paper is an important extension of Floro and Swain (2012) along several dimensions.
We contribute to existing literature on evaluation of the impact of microfinance in several important ways. First, we provide evidence on the impact of microfinance on individual level, which is not possible using RCT designs, which can only produce either ITT or LATE estimates (see footnote 4). While such parameters can be of interest, in most cases the policymakers would like to know the impact of microcredit on people who actually take it up (i.e. those that obtain a loan). Second, we describe characteristics of microfinance borrowers relative to those without loans and relative to those who also borrow from other sources. This is important for program targeting and allows for better understanding of factors influencing demand for microfinance. Third, we compare the outcomes of microfinance users to those without other loans and those who borrow from other sources. This allows us to test whether microfinance has same or different benefits relative to other sources of informal credit most often used by the poor. Finally, our results corroborate some of the key findings of the RCT that produced the dataset we use, notably the increase in durable purchases and lack of increase in total expenditures,thus confirming the robustness of these results using a completely different methodology. This also has implications for the validity of PSM methodology for estimating the impact of microfinance. Thus, our paper serves as a great complement to the recent emergence of RCT papers (cited in footnote 3).
The rest of the paper is organized as follows. Section 2 discusses PSM methodology; Section 3 describes our data; Section 4 presents our results; Section 5 contains a discussion and caveats; and Section 6 concludes.
- Methodology
PSM constructs a statistical comparison group that is based on a model of the probability of participating in the treatment conditional on a set of observed characteristics X. Ravallion (2003) refers to PSM as the observational analog to an experiment: “just like an experiment, PSM equalizes the probability of participation across the population — the difference is that with PSM it is the conditional probability, conditional on the X variables.”
Suppose T is a binary variable indicating whether an individual has participated in the treatment (i.e. obtained an MFI loan). The propensity score is given by the probability of participating in a program given observed characteristics: P(X) = Pr(T = 1|X ). It is assumed that X are unaffected by the individual’s participation in the program. Ideally, the X variables are observed pre-program, and can include pre-program outcome values. Unfortunately, we do not have pre-program data. Therefore, we are careful in selecting control variables that are unlikely to be affected by the program, as we discuss below.
An important assumption for validity of PSM is conditional independence, which states that given a set of observable covariates X that are not affected by treatment, potential outcomes Y are independent of treatment assignment T.[7]This condition, which is also referred to as conditional exogeneity of placement, allows the unobserved counterfactual (i.e. the outcome of treated if they had not participated in the treatment) to be replaced by the observed outcome of the control group (see Ravalion, 2008). In other words, this condition implies that the uptake of the program is based entirely on observable characteristics, and hence the differences in outcomes between treated and controls can be attributed to the treatment. While this assumption is inherently untestable, it can be more credibly invoked if there are rich observable data on control variables (i.e. the X vector) that would allow one to control for as many of the relevant characteristics that can affect program participation,and the institutional setting in which the program takes place is well understood (see Caliendo and Kopeinig, 2008).
The important question is how well PSM method performs relative to experimental methods. Fortunately, a number of studies have established that PSM can provide fairly accurate estimates under certain conditions. Heckman et al. (1997, 1998a, 1998b)analyze performance of various matching schemes relative to experimental estimators.[8]They find that propensity score matching performs well if three conditions are met: 1) using a rich set of control variables, 2) using the same survey instrument for treated and controls and 3) comparing participants and non-participants from the same local market.
Our data satisfies all three of these conditions. First, we have a very rich set of control variables. As we describe below, the data used in this paper come from a detailed household surveys and provide ample individual and household characteristics which we use to control for observable factors affecting participation in microfinance. Specifically, we use characteristics of eligible female (i.e. aged 18-59) including her age, education, and nature of employment. In addition, we have details forthe male head of household, details on the household composition and details on the dwelling. Second, the same survey instrument was used for participants and control group. Third, participants and control group come from the same local markets. To further satisfy this requirement, we only use slums in which microfinance was introduced and compare users to non-users. Thus, we believe that our rich data and setting provide solid justification for using PSM method.
While PSM cannot control for unobservable characteristics affecting program participation, Heckman et al. (1997, 1998a, 1998b) argue that the bias coming from unobservable characteristics is small, relative to the bias coming from the incorrect use of observable characteristics (i.e. comparing units outside of the common support). More recently a meta-study by Glazerman et al. (2003) also finds that bias of non-experimental estimates was lower when the comparison group was drawn from within the evaluation itself rather than from a national dataset and locally matched to the treatment population. Another relevant study by Diaz and Handa (2006) evaluate performance of PSM relative to randomized experiment using Mexican conditional transfer program PROGRESSA. They find that in cases when the outcomes are measured using comparable surveys, the bias arising from PSM is negligible. To summarize, even though PSM cannot eliminate the bias arising from unobservable characteristics, previous research indicates that this bias is likely to be small.