LLIN e-coupon Process Evaluation

Interview Guides

Karen Bulsara

Justice Ajaari

of

Tropical Health LLP

(November 2013)

Contents:

Interview.Quest’n.

  1. Pilot Implementation Team4-
  2. Promo Team2
  1. External Stakeholders:
  2. Ministry of Health / Education2-
  3. International Donor / PMI1-
  4. Local Government Officials2-
  1. Distribution Partner1-
  1. Technology/SMS Partner 1-
  1. Employer Program
  2. Plantation Owners2-
  3. Plantation Managers /e-coupon issuers2-
  4. Plantation Employees (recipients)30
  1. Schools Program
  2. School Principals2-
  3. School e-coupon Issuers10
  4. Schools (Parents) recipients20
  1. General Population
  2. Community Leaders3
  3. e-coupon Issuers26
  4. Recipients20
  1. Redemption Points – Outlets35
  1. Pilot Implementation Team

Purpose of Interview:

-To gain a full understanding of the LLIN e-coupon pilot program in order to inform the process evaluation

-To ascertain any implementation issues encountered and lessons learnt to inform scale-up

-To highlight any limitations or areas of concern

-To confirm the necessary systems are in place for e-coupon implementation at scale

Interview to cover:No. of Respondents - 6

Question the goals of the pilot as stated (in SoW) – they appear to be goals of the e-coupon program rather than pilot,i.e.

Goals of the Pilot:

  • Stimulate private sector investment in malaria control in endemic countries
  • Demonstrate viability of retail shops as LLIN distribution points for subsidized nets

Confirm Pilot Implementation Objectives and see if met, i.e.

Pilot Implementation objectives

  • Create an e-coupon platform with capabilities of efficiently managing subsidies from multiple donors, to a variety of target populations, with differing levels of subsidies for specific targeted populations.
  • Create demand for subsidized LLINs in Koforidua and in the selected workplaces
  • Demonstrate a cost effective public private partnership to reach targeted consumers with different levels of subsidies.

Specifically:

Process:

How are monitoring systems working – fraudulent activities being detected?

How is the mobile network uptime affecting the program?Other technology issues?

What is the response time and are systems in place to respond to market actors?

Are reports clear and scheduled to monitor progress – what is necessary for scale up?

Relationship between MEDA and technology provider (SMSGH/Peaceworks) – implication of scale-up?

Training – is this sufficient – is there on-going support provided?

Logistics / transport issues?

Pricing:

What considerations have been made to ensure equity in the project design and how will you ensure that equity measurements will be possible once the program has gone to scale?

Setting of subsidy for different targets audiences, how was this calculation made?

Demand Creation:

What demand creation efforts are being implemented?

Mass campaign (free LLIN distribution) completed at end 2012 – considered impact of this?

Private Sector sales lagging behind in LLIN distribution – considered reasons for this?

Acceptability:

How has the programme been received by the various actors?

Perceptions of ease of use of system for market actors? Levels of satisfaction ?

Recommendations to improve program:

In addition:

A focus on the questions affecting each of the stakeholders participating in the program (as contained in this guide) and the teams’ perceptions of the issues for each group?

From the team and their Program Monitoring Reports, inform the following:

  1. current functioning of the e-coupon system, identifying strengths and weaknesses
  2. recommendations for improving or streamlining the processes
  3. recommendations for key considerations for program scale-up.
  1. Program efficiency in terms of:
  2. Total numbers of LLINs distributed to identified target beneficiaries;
  3. Average cost per net delivered to a beneficiary, as well as individual cost components along the supply chain;
  4. Time and effort required on stakeholders’ part (retailers, coupon issuers, manufacturer/distributer, recipient) to participate in the program
  5. Describe challenges, bottlenecks and other hindrances from stakeholders’ perspectives that impact the smooth running of the program
  6. Program effectiveness in terms of:
  7. Proportion of selected target populations reached through this system, based on administrative data;
  8. Redemption rates of coupons issued, by target population
  9. Willingness of participants in the system (coupon distributors, LLIN retail distributors, LLIN wholesale distributor(s), donors and beneficiaries to continue participation;
  10. Recommendations for scale-up, based on problems encountered and potential solutions/changes needed.
  1. External Stakeholders:

Purpose of Interview:

-To ascertain the level of understanding and commitment to the e-coupon program, as plans are made for scale-up.

Interview to cover:No. of Respondents - 5

Intro:

Background to this pilot program Process Evaluation and potential scale-up

Operating Environment in Ghana:

Mass campaign (free LLIN distribution) completed at end 2012 – considered impact of this?

Private Sector sales lagging behind in LLIN distribution – considered reasons for this?

Potential for e-coupon program in Ghana:

Coupon Program (as implemented in Tanzania) is considered a way to boost the private sector in LLIN distribution/sales – is this accepted wisdom – can it work in Ghana?

Networks hope to use pilot to inform possible approaches for private sector involvement in LLIN distribution – expectations beyond e-coupon program?

Results of pilot to inform scale-up and to share lessons learnt for other country programs – how is the concept of scale-up viewed?

Willingness to continue with program – reasons for or against

Recommendations to improve program

  1. DistributorPartner

Purpose of Interview:

-To ascertain the level of effort and interest of this partner in the program.

Partner: Agrimat

Interview to cover:No. of Respondents - 1

Intro:

Background to this pilot program Process Evaluation and potential scale-up

Operating Environment in Ghana:

Agrimat does not operate in consumer retail markets and so is not ideally placed to develop the LLIN consumer market – own perceptions of fit for such a program?

How has mass free distribution affected the private sector market for LLINs?

The e-coupon program in Ghana:

What are the operating issues that have been encountered?Positives and Negatives?

How easy has it been to adapt operations to accommodate the e-coupon program?

Is there an interest and is there the capacity to operate at scale?

What is the implication of opening up the program to other distributor partners?

How has the “Payment system” worked?

Are sales volumes sufficient for participation?

Have demand creation efforts been satisfactory?

Is the subsidised price low enough to entice new consumers?

What are the difficulties in offering credit to retailers?

Consideration of other possible distributor partners e.g. pharma, bedding etc

What successes have come from this program for your business?

Willingness to continue in program – reasons for or against?

Recommendations to improve program?

  1. Technology Partner

Purpose of Interview:

-To ascertain the level of effort required for participation in such a system and understand any issues for scale-up

Partner: SMS Ghana / Peaceworks(Nana Kwame)

Interview to cover:No. of Respondents - 1

Intro:

Background to this pilot program Process Evaluation and potential scale-up

Operating Environment in Ghana:

Details about mobile network in Ghana and ease of operation?

Dealing with current level of mobile business interactions?

Do you have the capacity for scaling up program?

Acceptability of e-coupon use in Ghana?

Partnership Issues:

Working relationship with MEDA

Willingness to continue in program – reasons for or against

The Pilot Program:

Are there any aspects of the pilot program that have worked better than expected; or have caused more problems than expected? Implications of this for scale up?

Is there sufficient capacity amongst the population to successfully utilise the e-coupon for LLINs at issue and at redemption?

Recommendations to improve program?

Possible Technological Advances:

Launch of additional SMS short codes (beyond the 4 current providers)

Additional technological advances e.g. limiting no. of vouchers per customer; self-issue

  1. Employer Program

Purpose of Interview & Questionnaire Survey:

-To see how the employer program has been operating for all actors:

  1. Plantation Owners: (?)
  2. VREL (400 employees)
  3. Golden Exotic(2500 employees)
  4. Plantation Managers / e-coupon issuers
  5. Plantation Employees (recipients)

-To see if there are any issues for Plantation Owners in signing up to such a scheme

-To see if there is any implementation issues for coupon issuers

-To see how acceptable the program is to recipients/employees

Owner Interview to cover:No. of Respondents - 2

Intro:

Background to this pilot program Process Evaluation and potential scale-up

Benefits of e-coupon scheme:

What are the benefits to the company in being part of this scheme? Any problems?

What are the benefits of LLINs and benefits of reduced price with e-coupon for employees?

Process:

Ability to use e-technology for issuers and employees?

Are Issuers motivation levels high – understanding of the wider program goals?

Pricing:

Expected price of LLIN and price actually paid – perception of subsidy level?

Ability and willingness to pay amongst employees?

Demand Creation:

Implications of the recent mass campaigns free nets on perceived value of LLINs?

Is there sufficient demand creation to stimulate participation in the program?

Acceptability:

Acceptability of program to the company and amongst employees?

Continued Support:

Willingness to continue in program – reasons for or against

Recommendations to improve program

Questionnaires for Issuers to cover:No. of Respondents - 2

Promotion of LLINs and e-coupon Subsidy:

Can you relay benefits to employees of LLIN and benefits of reduced price with e-coupon?

Expected price of LLIN with e-coupon – perception of subsidy – is it adequate?

Are there any implications of the recent mass campaigns on perceived value of LLINs?

Process:

Acceptability of e-technology amongst employees?

Ease of system for issuers of e-coupons / use of technology?

Motivation levels of issuers – understanding of the wider program goals?

Willingness to continue in program – reasons for or against?

Recommendations to improve program?

Questionnaires for Employee Recipients to cover:No. of Respondents - 30

Current Net Purchase and Use:

How many nets that you own have been given free / bought?

Is the net actually used – perceptions of malaria risk?

Promotion of the LLIN e-coupon scheme:

How/where did you hear about the reduced price nets?

Understanding of benefits of LLIN and benefits of reduced price with e-coupon?

How do you feel about your employer participating in this program?

Process:

Perceptions of the subsidy program? Does the e-coupon make you want to buy a net?

Acceptability of e-technology?

Any issues in obtaining an e-coupon from issuer? e.g. trust, access, need for ID cards etc

Reasons for not participating? E.g. free net distribution, other?

Any issues in obtaining a subsidized LLIN from retailer? e.g. need for ID cards

Would you get your own e-coupon on your cell, if you saw it advertised or do you prefer someone you trust to do this for you?

Pricing:

Expected price of LLIN and actual price paid for LLIN?

Is the subsidy motivating enough for you to purchase an LLIN?

Reasons for not purchasing e.g. price, access, no supply, trust e-system, network problem etc

Program Improvements:

What improvements could be made that would make this work better for you?

  1. Schools Program

Purpose of Interview & Questionnaire Survey:

-To see how the school program has been operating for all actors:

  1. Secondary School Principals
  2. Pope John’s
  3. Oyoko Methodist Senior High
  4. Ghana Secondary
  5. Primary School Prinicipals
  6. Nana K Boateng
  7. Riis Presby
  8. Presby
  9. School e-coupon Issuers
  10. Schools (Parents) recipients

-To see if there are any issues for School Principals in signing up to such a scheme

-To see if there is any implementation issues for coupon issuers

-To see how acceptable the program is to recipients/parents of school children

School Principal Interview to cover:No. of Respondents - 6

Intro:

Background to this pilot program Process Evaluation and potential scale-up

Benefits of e-coupon scheme:

What are the benefits to the school in being part of this scheme? Any problems?

Understanding of benefits to children of LLIN use and benefits of reduced price with e-coupon?

Process:

Acceptability of and ability to use e-technology for issuers and children/parents?

Are Issuers motivation levels high – understanding of the wider program goals?

Pricing:

Expected price of LLIN and price actually paid – perception of subsidy level?

Ability and willingness to pay amongst parents of school children?

Demand Creation:

Implications of the recent mass campaigns free nets on perceived value of LLINs?

Is there sufficient demand creation to stimulate participation in the program?

Continued Support:

Willingness to continue in program – reasons for or against?

Recommendations to improve program?

Issuer Questionnaires to cover: No. of Respondents - 6

Promotion of LLINs and e-coupon Subsidy:

Can you relay benefits to community of LLIN and benefits of reduced price with e-coupon?

Expected price of LLIN– perception of subsidy – is it adequate?

Implications of the recent mass campaigns free nets on perceived value of LLINs?

Process:

Acceptability of e-technology amongst children and parents?

Ease of system for issuers of e-coupons / use of technology?

Motivation levels of issuers – understanding of the wider program goals?

Willingness to continue in program – reasons for or against?

Recommendations to improve program?

Focus Group for Parents to cover:No. of FGDs –2 or 4

Current Net Purchase and Use:

How many nets that you own have been given free / bought?

Is the net actually used – perceptions of malaria risk?

Promotion of the LLIN e-coupon scheme:

How/where did you hear about the reduced price nets?

Understanding of benefits of LLIN and benefits of reduced price with e-coupon?

Perceptions of the program and child’s participation in it?

How do you feel about your school participating in this program?

Perceptions of the subsidy program? Does the e-coupon make you want to buy a net?

Understanding of benefits of LLIN use for children and benefits of reduced price with e-coupon?

Process:

Any issues in obtaining an e-coupon from the school? e.g. trust, access etc

Reasons for not participating e.g. perceptions of political associations for voter appeasement, free nets?

Acceptability of e-technology?

Would you get your own e-coupon on your cell, if you saw it advertised or do you prefer someone you trust to do this for you?

Pricing:

Expected price of LLIN and actual price paid for LLIN?

Any issues in obtaining a subsidized LLIN from retailer? E.g. need for ID cards

Reasons for not purchasing? e.g. price, access, no supply, trust e-system, network problem etc

What improvements could be made?

  1. General Population

Purpose of Questionnaire Survey:

-To see how the general e-coupon program has been operating for all actors:

  1. Community Leaders[N.B. Sam Seddoh does not think this is necessary]
  2. e-coupon Issuers
  3. Clinics
  4. Hospitals
  5. MTN Mobile Credit Vendors
  6. Post Offices
  7. Community Centres
  8. Recipients

-To see if there are any issues for community leaders in recommending such a scheme

-To see if there is any implementation issues for coupon issuers of differing types

-To see how acceptable the program is to recipients/general population

[N.B. Sam Seddoh does not think Community Leaders are required]

Community Leader Interview to cover:No. of Respondents - 3

Low redemption rates and low usage of LLINs amongst population – reasons for this?

Perceptions of political associations for voter appeasement – is this happening?

Implications of the recent mass campaigns free nets on perceived value of LLINs?

Does community understand benefits of LLIN and benefits of reduced price with e-coupon?

Expected price of LLIN and price actually paid – perception of subsidy – is it adequate?

Acceptability of e-technology amongst different target groups?

Ease of system for issuers of e-coupons / use of technology?

Motivation levels of issuers – understanding of the wider program goals?

Recommendations to improve program?

Issuer Questionnaires to cover: No. of Respondents - 25 (5 of each issuer type):

Promotion of LLINs and e-coupon Subsidy:

Can you relay benefits to community of LLIN and benefits of reduced price with e-coupon?

Expected price of LLIN– perception of subsidy – is it adequate?

Process:

Acceptability of e-technology amongst different target groups?

Ease of system for issuers of e-coupons / use of technology?

Motivation levels of issuers – understanding of the wider program goals?

Implications of the recent mass campaigns free nets on perceived value of LLINs?

Willingness to continue in program – reasons for or against?

Recommendations to improve program?

Exit Interview Questionnaires for Recipients to cover:No. of Respondents - 20

Current Net Purchase and Use:

How many nets that you own have been given free / bought?

Is the net actually used – perceptions of malaria risk?

Promotion of the LLIN e-coupon scheme:

How/where did you hear about the reduced price nets?

Understanding of benefits of LLIN use and benefits of reduced price with e-coupon?

Process:

Perceptions of the subsidy program? Does the e-coupon make you want to buy a net?

Any issues in obtaining an e-coupon from issuer?e.g. trust, access, need for ID cardsetc

Reasons for not participating e.g. perceptions of political associations for voter appeasement?

Acceptability of e-technology?

Would you get your own e-coupon on your cell, if you saw it advertised or do you prefer someone you trust to do this for you?

Pricing:

Expected price of LLIN and actual price paid for LLIN?

Any issues in obtaining a subsidized LLIN from retailer? E.g. need for ID cards

Reasons for not purchasing? e.g. price, access, no supply, trust e-system, network problem etc

What improvements could be made?

  1. Redemption Points (Retail Outlets)

Purpose of Questionnaire Survey:

-To ascertain how successfully the e-coupon system has worked and the whether the private sector retailers have been stimulated to participate in the program

Type of Outlet:

  1. Pharmacy
  2. Drug Shop
  3. Supermarket
  4. High School (not a retail environment?)

Questionnaires to cover:No. of Respondents – 35 (7 of each outlet type)

Sales Rate:

Redemption rates – how many nets are you selling?

Impact of e-coupon on business and sale of LLINs?

Process:

Ease of redemption system?

How do you purchase from supplier, in what quantities, do you need credit facilities?

Use of the mobile technology for subsidy – network issues?

Location of redemption point to issue point?

Competition with other providers?