Case study:
Time-limited vouchers to encourage fertilizer adoption: Are they effective?[1]
Introduction
In 2010, the Ministry of Agriculture in Naguda decided to work with an NGO called “Fertilizer for All” to pilot a new approach to increase adoption of fertilizer. Fertilizer adoption is limited in Naguda and the Ministry wants to increase fertilizer use to increase yields.In the month after harvest, the NGO staff visited1,000 farmers to offer them time-limited vouchers to adopt fertilizer in their farming plots, and convey the message below:
“Hello. My name is Marc Oyeye, and I’m fromFertilizer for All. We are an NGO that promotes the use of fertilizers in farming plots. We’d like to let you know that with this voucher, you could get fertilizer at a 50%discount.Please also note that it will expire in two weeks.”
Did the time-limited voucher increase the adoption of fertilizer? How can we find out? This case study addresses these questions by examiningthe various methods that can be used to assess the impact of a program orintervention. While the context of this case study is fertilizer use in Naguda, thequestions raised here are also valid for the assessmentof the impact of other publicprograms in developing countries.
Background
By some estimates, there are approximately 1.4 billion people living on less than $1.25 a day, many of whom are farmers. As such, identifying ways to increase agricultural incomes is crucial in meaningfully alleviating poverty. Such strategies are especially important in Naguda, where agricultural yields have been low and remained stagnant for many years. The use of fertilizer has the potential to dramatically increase yields and, if used correctly, is a highly profitable investment. However,adoption rates remain are low. If fertilizer is so profitable, why do so few farmers in Naguda use it? Is it lackof information about profitability, lack of money to purchase it, or an inability to save for the purchase?
In search of an explanation for this low fertilizer adoption rate, andespecially in search of a solution, the Minister of Agriculture hired a consultant, whoproposed the following strategy:
The Ministry of Agriculture is skeptical about this proposal. Before rolling out thestrategy in the whole country, they decide to hire an NGO, “Fertilizer for All” to run a pilot experiment to test the efficacy of time-limitedfertilizer vouchers. An impact evaluation will be built into the pilot experiment. TheMinister of Agriculture hires you to run the impact evaluation.
Please complete Question 1 before reading further.
******************************
Did the time-limited fertilizer voucher work?
In December 2010, the NGO “Fertilizer for All” obtained a list of1,000farmers in Naguda. The list of 1,000farmers was obtained from the archives of the national civilregistry of Naguda, in which all farmers are registered. The archivesalso contain data on the size of the farmer’s household, the age of the farmer,where the farmerlives (i.e., Northern Region or Southern Region), and thelevel of economic development of the farmer’s district.
In January 2011,interviewers visited all 1,000 farmers, but were only able to speakwith 403 people. That is to say, only 403 farmers were at home and offered the voucher. For each of the 1,000farmers, the volunteers noted whether thefarmers are at home for visit or not.
Finally, after 6 months, Fertilizer for All revisited the farmers and determinedwhether they had actually used the vouchers in the new farming season or not.
Fertilizer for All has agreed to share with you its data concerning the 1,000farmers involved in theirfertilizer voucher program. We are asking you to use this data to gauge the impact of the fertilizer voucher campaign on fertilizer adoption rates, i.e. its impact on the percentage of farming plots that are fertilized.
Method 1 – Difference in the proportion of fertilizer usage, between farmers that received fertilizer voucher versus those that did not receive it.
Assume that the 403farmersthat received the fertilizer voucher constitute the ‘treatment’ group and the remaining 597farmers (i.e. those that were visited but were not in their home) represent the ‘comparison’ group. If you want to determine the impact of receiving a voucher on fertilizer adoption rate, you might check to see whether those who received the voucher were more likely to fertilize their farming plots than those who did not. Table 1a below compares the proportion of farmers in the ‘treatment’ group that had their farming plots fertilized with the proportion of such farmers in the ‘comparison’ group.
Table 1a: Percentage of farmersthat had adopted fertilizer in 2011… among farmers
that got the voucher / … among farmers
that did not get the voucher / Estimated
impact
Method 1:
Simple difference / 16.5 % / 9.6 % / 6.9 pp*
Please complete Question 2 before reading further.
******************************
Method 2 – Use a multiple regression model to determine the differences between farmers that received the fertilizer voucher and those that did not.
If you believe that the farmers that received the fertilizer voucher may have inherent characteristics different from those who did not receive them, you can test the difference by using a multivariate regression, as follows:
The participant group and the comparison group are defined in the same way as in
Method 1. To estimate the impact of the program, one does a regression in which the
‘dependentvariable’ indicates whether the farmer used fertilizeror not (i.e., 0 = did not fertilize plot, 1 = fertilized plots).The ‘key explanatory variable’ is a variable indicating whether thefarmerwas offered the voucher or not (i.e., 0 = offered, 1 = was not offered). Potential differences in other characteristics of the farmers can be disentangled by adding other‘explanatory variables’ such as the size of farmer’s household, the age of farmer, size of the property etc. The coefficient of the ‘key explanatory variable’ is the estimated program impact.
Table 1b shows the estimated impact of the fertilizer voucher using the multivariate method (Method 2) compared to the estimated impact of the simple difference (Method 1)
Table 1b: Percentage of farmersthat had adopted fertilizer in 2011… among farmers
offered voucher / … among farmers not offered voucher / Estimated impact
Method 1: Simple Difference / 16.5 % / 9.6 % / 6.9 pp*
Method 2: Multiple regression / 5.1 pp*
pp=percentage points*: statistical significance = 5 %
Controls include size of farmer’s household, age of the farmer,a variable indicating the level of economic development in the farmer’s district and a variable indicating whether the farmer lives in the Northern Region.
Table 2 compares the average characteristics of the ‘treated’ groups and ‘comparison’ groups used in these two methods.
Table 2: Average characteristics of farmersFarmers that got voucher / Farmers that did not get voucher / Difference
Size of farmer’s property (Ha) / 2.8 / 1.0 / 1.8*
Average age of farmer / 35.8 / 31.0 / 4.8
Percentage of farmers that have mobile phone / 32.7 % / 32.2% / 0.5 pp
Percentage in the Northern Region / 54.7 % / 46.7 % / 8.0 pp*
Sample size / 403 / 597
Please complete questions 3 through 6 before reading further.
******************************
Method 3 – Using Panel Data
If you are still concerned about differences in characteristics between farmers that were offered the vouchers and those that were not,youmightusepaneldata,i.e.you could follow the same farmers over time.
Asitturnsout,thearchivesoftheNGO alsohaddataindicatingwhetherfarmers had adopted fertilizer in the past two years, from 2009 to 2010.Thefarmers’pastbehaviorwithregardtofertilizer adoption can be a solid predictor of their future fertilizer adoption behavior.Table3showsthepastfertilizer adoptionbehaviorfor the group of farmers that were offered the vouchers versusfarmers that were not at home during Fertilizer for All’s visit.
Table 3: Percentage of farmers who adopted fertilizer prior to 2011*… among farmers
that got voucher / … among farmers
that did not get voucher / Difference
Used fertilizer in 2011 / 16.5 % / 9.6 % / 6.9 pp
Used fertilizer in 2010 / 14.4 % / 8.6 % / 5.8 pp
Used fertilizer in 2009 / 12.7 % / 7.3 % / 5.4 pp
Difference between fertilizer usage in 2011,those in 2010 / 2.1 % / 1.0 % / 1.1 pp*
pp=percentage points*: statistical significance = 5 %
Please complete questions 7 through 8 before reading further.
******************************
Randomized Experiment
As it turns out, the 1,000 farmers were chosen at random from the archives of the national civil registry of Naguda. This is similar to the random drawing done in a clinical trial, where the treatment/drug is administered randomly so as to be received by one group of patients but not the other. The complete list includes 7,000 farmers. We can exploit this random drawing of 1,000farmers to estimate the impact of the fertilizer voucher. The idea is that the 1,000farmers that were targeted for receiving vouchers from Fertilizer for All (now referred to as the ‘treatment’ group) should be identical to the 6,000 other Nagudianfarmers (now referred to as the ‘control’ group) in termsof observable and non-observable characteristics. The only difference between the treatment and control groups is that the first group was visited by Fertilizer for All and the second was not. Table 4 compares the ‘treatment’ group and the‘control’ group on the basis of observable characteristics. Table 5 shows the estimatedimpact of the fertilizer voucher by comparing fertilizer use in the treatment group with fertilizer use in the control group.
Table 4: Characteristics of treatment and control groups‘Treatment’ group
(Visited) / ‘Control’ group
(Not visited) / Difference
Adopted fertilizer in 2009 / 9.5% / 9.2% / 0.3 pp
Adopted fertilizer in 2010 / 10.9% / 11.3% / -0.4 pp
Size of farmer’s property (Ha) / 4.7 / 5.1 / -0.4
Average age of farmer / 33.1 / 32.0 / 1.1
% of farmers with mobile phone / 32.4% / 31.2% / 1.2 pp
Percentage in the Northern Region / 49.9% / 51.4% / -1.5 pp
Size of sample / 1,000 / 6,000
pp=percentage points
*: statistical significance = 5 %
Table 5: Randomizedtreatment andcontrol groupsPercentageoffarmers that adopted fertilizer in 2011
‘Treatment’group / ‘Control’group / Impact estimate
Method4a: RandomSimple difference / 12.4 % / 12.2 % / 0.2 pp
Method4b: RandomMultiple Regression / 0.2 pp
pp=percentage points
*: statistical significance = 5 %
Please complete questions9and 10 beforereadingfurther.
******************************
Technical note: Adjustment for take-uprate
Table5showsthesimplecomparisonoftreatmentandcontrolgroups,where the treatmentgroupconsistsofallthosewere visitedby Fertilizer for Allandthecontrolgroupconsistsof allthose whowere not visited.This estimatedimpactdoesnottakeintoaccountthefactthat597individualsinthe “treatment”group werevisited but not at home, therefore werenot given vouchers.
Ifwewishtoestimate the impactofactually handing the voucher tothe farmers,rather than just“visiting” thefarmers,thenwewouldneedtoadjusttheestimateusingthemethodology of Instrumental Variables.
A possible formula formakingtheadjustment isas follows：
So: = 0.3
Conclusion
Table 6 shows the estimated impacts of the fertilizer voucher on fertilizer adoption rates using the various methods discussed in this case study.
Table 6 – Summary of estimated impacts of the fertilizer voucherMethod / Estimated impact
Method 1: Simple difference / 6.9 pp*
Method 2: Multiple regression / 5.1 pp*
Method 3: ‘Double difference’ based on panel data / 1.1 pp*
Method 4a and 4b: Randomized experiment / 0.2 pp
Method 4c: Randomized experiment adjusting for take-up rate / 0.3 pp
pp=percentage points
*: statistical significance = 5 %
As you can see, not all methods yield the same results. It is therefore critical to choose the appropriate method. The purpose of this case study was not to assess a specific fertilizer voucherprogram, but to test various assessment methods in this particular context.
In the analysis of the fertilizer voucher program, we noticed that those who received voucher were probably going to adopt fertilizer in their farming plots, but that they were also more likely to have fertilized their farming plots in previous years. Even when we accounted statistically for (known!) observable characteristics of farmers, including demographic characteristics and fertilizer use in previous years, there were still some inherent non-observabledifferences between the groups, independent of the fertilizer voucherprogram. Thus, when our non-random methods demonstrated a positive and significant impact, this result was attributable to a ‘selection bias’ (in this case, the selection of those who were at home during the visit and received the voucher) rather than to a successful fertilizer voucher program.
[1] Based on Duflo et al. “Nudging Farmers to Use Fertilizer: Theory and Experimental Evidence from Kenya.”