Learning Clients/Beneficiaries SatisfactionFinal 09/09/2016

Community Stakeholder’sSatisfaction with Project

Purpose of the document: good project management includes giving participants an opportunity to actively engage in the design, implementation, monitoring and evaluation of a project. A community stakeholder is a person, group or community which actively participates by receiving some benefits from a project but also contributes to the project design and learning.

This document is a monitoring and evaluation tool used to measure the level of satisfaction of community stakeholders regarding the services or activities provided through the project, their degree of participation and the changes (positive and or negative) to their lives as a result of the project.

The questions and ideas provided are useful for any project category, including livelihood and income generation projects, water and sanitation projects, education projects, health projects, etc.

This will help to provide understanding of how appropriate the project was for the community needs and supply lessons learnt that can be used for future community projects.

The approach will encourage community stakeholders in a project to give feedback on the project interventions. The project implementers should facilitate an environment – through the project and during this learning process – to build trust with the project participants.

Users of the document:Salvation Army representatives at mid-management/field level in Implementing Territories (IT). Supporting Territories (ST) can also use it, but it is designed for ITs.

Application:It is not sufficient to only ask a community ‘Are you satisfied with the project?’, because people may feel too shy to answer such a direct question honestly and it will provide limited information. It is more valuable to ask open-ended questions, rather than closed questions which can be answered with a ‘yes’ or ‘no’.

Instead, there are questions that are part of the satisfaction criteria that can provide suitable evidence to all stakeholders, including to Salvation Army leadership and the donor, that the community stakeholders are satisfied with the project (or not) in many aspects of the project.

Satisfaction criteria include:

  1. Appropriateness of the project

Ask questions about this to learn if the project is actually what the community thinks is a priority for them, and if – in their opinion – it has addressed root causes of poverty. When the project does not address the priorities of the community or contributes only in a minimal way to the circumstances in the community, then there is a high chance that the community will not be satisfied.

  1. Involvement of community stakeholders in the project

Ask questions about this to learn if community stakeholders believe they have contributed throughout the project cycle and if they have ownership of the project. If they have had no say in what the project did, how can they have ownership of the project? How were the participants selected? What efforts were made to include vulnerable groups? Were there any particular criteria for inclusion or exclusion?

  1. Information sharing

Ask questions about this to learn if information and updates about the project were regularly provided and made available to everyone. This includes communicating in various ways that are culturally sensitive, allowing people with difficulties/vulnerabilities (including children) to be included and providing a safe way for people to raise concerns. If information is not properly provided to all participants, communication is not honest and open; they may not be as involved as they would like to be.

  1. Salvation Army representatives

Ask questions about this to learn if Salvation Army representatives, including staff, volunteers and external contractors, were always qualified, transparent, respectful and behaved in a manner that was in line with relevant codes of conduct.

  1. Addressing challenges

Challenges or problems may occur in every project. It is acceptable if they happen sometimes, but it is important to learn how soon the challenge was identified and if it was resolved quickly in a manner that will mean it is not likely to happen again.

  1. Change (impact) for community stakeholders

Ask questions about this to learn what significant direct and indirect changes for the individual, family and community have occurred since the project started in their community. Were there any unexpected changes (positive or negative) in the community?

Preparation:

What to consider when planning for monitoring or evaluation activities to learn a community stakeholder’s satisfaction with the project. The information provided by the community is called ‘data’:

Decide who will collect data

  • Will it be an individual or a team?
  • Will it be field staff, projects officer, corps officer, external partner or someone else?
  • Who is a good facilitator?
  • Who is objective and does not create an unwanted power imbalance?
  • Will the gender of the data collector reflect the gender of the community participants? For example, will female data collectors be needed if the project participants are all female?

Decide where to collect data

  • Will there already be a meeting you can attend? For example, sewing class, savings group meeting, community committee meeting.
  • Do you need to organise individual interviews? If so, you need to know what day and time are suitable for the people you want to interview.
  • Do you need to organise a community meeting?

Decide which community stakeholders you will collect data from

  • Will it be every participant or just a sample?
  • How will you make a representative sample, including vulnerable groups?
  • Refer to the community stakeholder groups table below for ideas.

Decide when to collect data

  • How frequently do you conduct this data collection activity?
  • Is it more beneficial to collect this kind of data halfway through a project to see if it is going well?
  • If it were a multi-year project in many communities, would it be beneficial to collect this data in a different community each year?

Decide what the data collection method will be

  • Will it be individual interviews, group discussion or another style of tool? (For example, you may want to reach a wider group of people, using a survey instead of interviews. The questions below then need to be fewer in number and suitable for a survey).

Community Stakeholder Groups:

Please note this is a small group of examples and not a complete list.

Category of Project / Participant
Microfinance /
  • Microfinance group members
  • Family of microfinance group members
  • Microfinance operation team

Training and Education/Adult Education /
  • Students/trainees
  • Teachers/trainers
  • Parents

Agriculture and Animal Husbandry /
  • Villagers/town dwellers
  • Experts

Infrastructure /
  • Women, men and children who use the infrastructure, including people in the community with a physical disability
  • Local person/team responsible for infrastructure maintenance

Medical and Health /
  • Patients
  • Community members who receive health education training

Water and Sanitation /
  • Women, men and children who use a water point, including people in the community with a physical disability
  • Local person/team responsible for maintenance

Please note that a project will most likely have both direct/primary participants and indirect/secondary beneficiaries.

Application:

These key and specific questions can either be used in an interview or focus group discussion or put into a questionnaire or survey for project participants. There are other styles of tools you can use these questions in as well.

If the project is less than 12 months and under USD15,000 then it is enough to just ask the Key Questions. If the project is longer than 12 months and more than USD15,000, it is encouraged that both the Key and Specific Questions are asked.

Refer to IPDS tool box, where documents from the below link are to be found (among others):

Satisfaction Criteria / Key Question / Specific Questions / Indicators of Satisfaction
Appropriateness of project / How has this project addressed the priorities of the community? /
  • What were your main priorities before the project, at both individual and family levels?
  • What were the main priorities in the community prior to the project?
  • Were you asked about your or the community’s priorities before this project was implemented?
  • To what extent did the project address these priorities?
  • Did the project address any root causes of poverty?
  • How have your priorities changed since the completion of this project?
/ Project addressed the main priorities for participants
Was the timing of project activities suitable for you? /
  • At what time did the project activity take place?
  • Was the time suitable to start this activity? If answer is ‘no’ then ask why it was not a suitable time.
  • What time would have been best for you and why?
/ Project activities were at a suitable time for participants.
How were the knowledge and skills you learnt useful? How can you use them once the project has finished? /
  • What knowledge or skills have you gained from this project?
  • How have you been able to use this knowledge or skills up to now?
  • How have you shared this knowledge or skills with others?
  • Do you have access to the resources needed to continue to use this knowledge or skills once the project is finished?
  • How has using these skills or knowledge changed your life. What was the impact?
/ They expressed the belief that skills and knowledge learnt are useful. That they are able and willing to access the appropriate resources to still use the skills or knowledge after the project is completed.
Involvement of community stakeholders during the project cycle / How were you involved in assessments, project planning or starting (implementing) the project? /
  • How were you involved with the project in identifying needs and priorities?
  • How were you involved in planning what type of project would occur?
  • How were you involved in the starting of these project activities?
  • (If not directly involved) Who from the community was involved and when?
  • Were there other groups/members of the community that should have been involved in identifying needs and priorities, planning or starting projects?
/ Participants in the community were regularly involved at all stages of the project cycle.
Do you understand how the community stakeholders for the project were selected? /
  • Do you know who was on the selection panel?
  • Were there any particular selection criteria for inclusion or exclusion in this project?
  • Who set the selection criteria?
  • Were there others who were in greater need that missed out being a project participant?
/ Participant selection is fair when those most appropriate were selected.
Information
sharing / When, where and how did you hear about the project or project activities? /
  • When, where and how did you hear about the project activities?
  • Did you learn about the planning for a project before the activities started?
  • Was the information provided on time?
  • What would have been the best way of giving you information about the project?
  • Do you feel communication from The Salvation Army was honest and open?
  • What would have been helpful to have more information about?
  • What information would community stakeholders want to know from the project?
/ Information was regularly available and accessible to all people who wanted to know. More than one method of information sharing was used. Communication style was adapted for people with disabilities to be included.
Do you know how to make a complaint about the services or share concerns about the project? /
  • What is the process of reporting project concerns/ complaints?
  • How were you informed about this process?
  • If the ordinary procedures set for the complaining channels fail, how else would you be able to make a complaint?
  • Do you feel that yours or other people’s concerns were listened to and resolved?
  • Do you know who to make a complaint to if you are concerned for the safety of a child/woman/person with disabilities?
/ The Salvation Army communicated how participants could raise a complaint, and community stakeholders feel they could make a complaint without fear of repercussions.
Salvation Army representatives / Were you satisfied with the behaviour of Salvation Army staff, officers, external contractors and volunteers during theproject? /
  • How was the behaviour of Salvation Army staff, officers and volunteers during the project?
  • Were TSA representatives implementing the project activities according to what you designed/planned and agreed to?
  • Were Salvation Army officers or staff responsive to your opinions?
/ Professional, reliable and accountable behaviour was demonstrated by all Salvation Army representatives and external contractors.
Addressing challenges / How were challenges or difficulties addressed and resolved during the project? /
  • Were there any problems or challenges affecting the services/activities during the project?
  • If yes, what was the reason for them?
  • How were they resolved?
  • Did challenges continue to happen?
/ When challenges happened, that they were acknowledged and resolved so as to fix and to prevent them from happening again in future.
Change
(impact) for project participants / What are the changes that have happened for you, your family and your community since the project started? / (Direct Changes)
  • What are the changes that have happened for you since you have been involved with this project?
  • What impact has this project had on your husband/wife/children/brother/sister and other family members?
  • Were there any other unexpected changes for you?
  • Has there been any negative impact – and who for?
(Indirect Changes)
  • What unexpected changes have happened in the community since the project started?
  • Has there been any positive change?
  • Has there been any negative change?
  • Were there any other unexpected changes in the community?
/ Project has created a positive, ongoing impact for the most vulnerable community members. The community members are a safer and more empowered/resilient group due to the project. That the project did not create a negative impact.
Conclusion / What could have been improved or done differently during the project? /
  • What could have been improved or done differently during the project?
  • Is there anything else that you want to share that you think is important?