Outcomes Working Group
Background - Social Performance Task Force 2014 Annual Meeting
Plenary Session: Top Ten Lessons on Measuring Outcomes
(held June 2014 in Dakar; speaker was Amelia Greenberg)
In preparation for its 2014 annual meeting, the Social Performance Task Force (SPTF) reviewed reports of many microfinance institutions that have attempted or are attempting to track client-level outcomes. It also interviewed financial services providers, investors, TA providers and researchers who have experience with collecting and analyzing client-level outcomes data. This brief sets out the background, and the Top Ten lessons that emerged, leading to the creation of the Outcomes Working Group.
Background
Some financial service providers (FSPs) pursue a double bottom line, seeking to achieve financial success and to contribute to positive changes in clients lives. SPTF contends that by implementing strong social performance management (SPM) practices, including defining and tracking indicators to measure the changes in the lives of their clients (i.e., client-level "outcomes"), FSPs better position themselves actually to achieve their social goals. This is because without reliable data on how customers’ lives are affected by products/services, businesses cannot know whether their products and services have helpful effects or the reverse, much less how to improve them. On the other hand, by clearly defining and systematically tracking specific outcome indicators, such as enterprise growth, household poverty level, assets, health, and women's empowerment, FSPs can understand how they are doing, and how to do better.
Ten lessons on measuring outcomes
1. Educate external stakeholders. Many external stakeholders are unrealistic about what changes to expect over what time frame. FSPs and others should be clear on what is realistic in terms of the extent of positive outcomes and the time involved.
2. RCTs are not the whole answer. Randomized control trials (RCTs) are complex, require a large sample and a limited time frame. They may have particular uses, but appear to be too expensive for the typical FSP to undertake on its own.
3. Collect outcomes data that are credible and relevant but affordable and simple to collect. Data quality must be reliable. However, the data collection process must not overburden the institutional capacity. It should be affordable and relatively easy to execute.
4. Ask field agents for input. Field agents are often able to suggest good outcomes indicators to track. And, one Freedom from Hunger study found that answers given by field agents closely matched household survey information from clients and bank records.
5. Use multiple tools to measure changes. We can make better use of a range of available methodologies to understand whether and how an FSP's products and services are helping clients: e.g., a quantitative measurement tool such as the PPI or PAT, combined with other relevant indicators, and client focus groups or case studies to understand more deeply the process of change. You can find different tools in the SPTF Implementation Guide.
6. Self-perception can be an additional valuable source of outcomes data. Client self-perception can be a valid and inexpensive way to gain additional insights.
7. Report honestly. No institution can achieve positive outcomes for 100% of clients, and investors and donors know this. One investor reported feeling more comfortable investing in an FSP when it provided complete and accurate information on defaults, than previously when the FSP had shared only positive stories.
8. Analyze outcomes by client segment. Different types of clients can be affected in very different ways by the same product or service. Averages are not as useful as an analysis for different types of clients (‘segmented’ by e.g., gender, geography, livelihood) to understand which services work for whom and in what context. More on this in this CGAP Focus Note.
9. Use outcomes data to monitor achievement of social goals and to improve. Reporting outcomes data keeps an institution accountable to its social mission and demonstrates the extent to which the institution is achieving its social goals. Analyzing who is benefitting from the use of a financial product and who is not, and what led to these changes, gives the FSP insight into adjustments it might make to its products and services to improve client outcomes.
10. Use monitoring to increase employee satisfaction. Employee feedback shows that monitoring outcomes is important to employees because it helps them better understand clients and it motivates them to see positive change in their clients' lives.
overall
Reliable outcomes data should help providers answer the fundamental question: are clients’ lives improving? FSPs need not be afraid of finding surprising or negative results, but should strive for an accurate understanding of client conditions. This knowledge is a powerful motivator for operational and product changes that can lead to improved client outcomes.
SPTF launched the Outcomes Working Group in October 2014 to continue our learning in this area.
The working group is open to all stakeholders :practitioners, networks, TA providers, investors, researchers, subject matter specialists – academics.
Contact us to share your experience and views: