For M&E Officers of IFAD Funded Projects in Viet Nam

Training Manual

PROJECT MANAGEMENT AND M&E

For M&E officers of IFAD funded projects in Viet Nam

HANOI, 26/11/2012-30/11/2012

CONTENTS

Lecture 1: Project management

1.Defining Project Management

2.The importance of project management

3.The role of the project manager

4.Keys to successful project

5.Managing the contractor

6.Managing the project

Lecture 2:Result based management

1.What is Results-based management (RBM)

2.The purpose and benefits of RBM

3.RBM involves three major steps

4.The Results’ Chain

Lecture 3: Basic M&E concepts

1.Monitoring

2.Evaluation

3.Important terms

Lecture 4: M&E system

1.What is M&E system?

2. How to set up the M&E system

3. M&E Plan

3.1 The structure of the monitoring plan

3.2 The structure of the evaluation plan

3.3 M&E and project logic frame

3.4 Detailing M&E plan

3.5 Using the M&E Matrix for detailed planning

3.6 Indicative contents for an M&E Plan

4. Data collection methods

5. Gender and M&E

6. M&E implementation responsibilities

Lecture 5: Gender mainstreaming in development projects

1.Basic definition when conducting gender analyzing

2.Gender mainstreaming in development project

2.1What is gender mainstreaming?

2.2Why do we need gender mainstreaming?

2.3Objectives of gender mainstreaming in projects

2.4Tools and strategies for gender mainstreaming

2.5Gender analysis

2.6What to expect in a successful gender mainstreaming project

2.7Basic principles in gender mainstreaming

Lecture 6-7: Measuring and reporting first and second level of results

1.Overview

2.Why to measure and report the first and second level results?

3.Collecting information and reporting first and second level results

4.Measuring first level results

5.Measuring and reporting second level results

Lecture 8: Steps to conduct a survey

1. Step 1: Define objectives of the survey

2.Step 2: Assess the resources

3.Step 3: Determine methods

4.Step 4: Designing questionnaire, interview guide and pilot

5.Step 5: Selecting the sample

6.Step 6: Enumerators’ training and field testing of the questionnaire

7.Step 7: Logistical planning

8. Step 8: Data collection

9. Step 9: Data entry

10. Step 10: Data analysis

11. Step 11: Report writing

12. Step 12: Communication – Dissemination

Lecture 9: Questionnaire design

1.What are questionnaire?

2. The importance of questionnaire

3. Steps to design questionnaire

3.1 Study project documents

3.2 Identify the indicators, goals

3.3 Study the template questionnaire of IFAD

3.4 Develop new questions, gender questions

3.5 Structuring the questions

3.6 Questionnaire presentation

4. Principles for questionnaire design

5. Questionnaire piloting

6. Summary of steps to design the Questionnaire

Lecture 10: Sampling method

1. Definitions

1.1 Population

1.2 Sample

1.3 Sampling frame

1.4 Sample size

2. Overview sampling method

2.1 Random sampling method

2.2 Non-random sampling method

3.Sampling method in project funded by IFAD

3.1 Stage 1: Select villages

3.2 Stage 2: Select households

Lecture 11: Introducing the Annual Outcome Survey

1. What is the annual outcome survey?

2. How and when to conduct the Annual Outcome survey?

3. Steps to conduct the AOS

3.1 Step 1: Designing questionnaire, interview guide and pilot

3.2Step 2: Sampling

3.3 Step 3: Enumerators’ training and field testing of the questionnaire

3.4 Step 4: Logistical planning

3.5 Step 5: Data collection

3.6 Step 6: Data entry

3.7 Step 7: Data analysis

3.8 Step 8: Report writing

Lecture 12: RIMS and RIMS+ baseline and end-line survey

1.Introduction to RIMS and RIMS+ survey

1.1 Background on RIMS

1.2 Changes in RIMS+

2.How to conduct RIMS and RIMS+ survey

Lecture 13: Key informant interview

1.Definition of key informant interviews

2.Classification of key informant interviews

3.Steps to conduct key informant interviews

3.1 Identify the topics that need in-depth information

3.2 Identify the key informants

3.3 Develop the questions by topics

3.4 Prepare the interview guide

3.5 Pre-test and finalize the questions or interview guide

3.6 Conduct the interview

3.7 Analyze and write report

Lecture 14: Focus group discussion

1. Overview of focus group discussion

2. Steps to facilitate a focus group discussion.

2.1 Identify topics and information needed for each topic

2.2 Prepare guide for discussion

2.3Select participants

2.4 Conduct FGD

2.5 Analyze results and write report

LIST OF TABLE

Table 1: Result Chain

Table 2: Example of Result chain-IFAD’s approach

Table 3: Example of Result chain- CIDA’s approach

Table 4: Monitoring and evaluation

Table 5: M&E tasks

Table 6: Critical Reflection Schedule for a project

Table 7: Questions to guide the detailed planning of M&E conditions and capacities

Table 8: The structure of an evaluation plan

Table 9: Task needed when detailing the M&E plan based on a project appraisal report

Table 10: Indicative contents for an M&E operational Plan

Table 11: Examples of multi-purpose M&E methods

Table 12: Comparison of quantitative and qualitative methods

Table 13: Discriminate between sex and gender

Table 14: Example for 1st level results

Table 15: Example for 2nd level results

Table 16: Example for indicator about rehabilitation of infrastructure and facilities

Table 17: Example for indicators about training

Table 18: Example for indicators about group formation/strengthening

Table 19: Example of reporting first level

Table 20: Rating scale for second level result

Table 21: List of performance questions for each factor

Table 22: Framework of factors and guiding performance questions

Table 23: Advantages and disadvantages of methods of administering survey questions

Table 24: Overview of changes in RIMS+

Table 25: Information in RIMS and RIMS+ questionnaire

Table 26: Example for measuring impact of project

Table 27: Advantages and disadvantages of unstructured and semi-structured interview

LIST OF FIGURE

Figure 1: Project process

Figure 2: Project Results-Chain

Figure 3: The use of control group

LIST OF BOX

Box 1: Five key evaluation questions

Box 2: The basic performance questions per level of the objective hierarchy

Box 3: Examples of performance questions

Introduction

IFAD has been working for and with the poorest people in Vietnam, including ethnic minorities, small-scale farmers and households headed by women. At present, IFAD is working at ten provinces in Viet Nam (9 projects), and some other projects will be implemented soon. In order to assess its progress against implementation plans (monitoring), and to assess the outcomes and impact it has achieved (evaluation), the project’s monitoring and evaluation (M&E) system is extremely important. The purpose of monitoring and evaluation is to obtain accurate information that will be used by decision-makers in the project to improve and ensure proper use of resources as well as to maximize the impact achieved from these resources.

However, there is a fact that most of IFAD project management staff in Viet Nam has limited exposure to IFAD M&E system. Therefore, building and improving capacity for IFAD project’s management staff is extremely important. The management staff needs to be equipped with knowledge and practices on project management and M&E, in order to organize and manage resources so the project is completed within defined scope, quality, time, and cost constraints.

Development and Policy Research Center (DEPOCEN) coordinated with APMAS and IFAD Viet Nam to organize a training on Project management and M&E in Hanoi to equip the IFAD partners in Vietnam with knowledge and practices on M&E.

Planning and timing

The duration of training course was 5 days, from 26 to 30 November, 2012 in Hanoi. The course is hence expected to provide participants the following knowledge and skills:

•Project management

•M&E includes basic concept and methodology, how to set up M&E system, M&E plan

•Mainstream gender in development project and M&E strategy

•Measuring and report first, second and third level results

•Sampling method

•Methodology of collecting qualitative information includes key informant interview and focus group discussion

The course was designed to provide trainees with both theory and opportunities to practice. Inevery section, participants were motivated to raise questions,comments, sharing experience on specific themes as well as to engage in exercises in which they have the chance to apply all theories to solve problems. In addition, participants listened to experiences of Ninh Thuan project staff in conducting RIMS+ baseline survey and experiences of Tuyen Quang project staff in conducting annual outcome survey.

Table 1: Project management and M&E training program

Date / Time / Content
26/11/2012
(Monday) / Morning / 1. Project Management
Afternoon / 2. Results Based Management
3. M&E Basic concepts
4. M&E system
27/11/2012
(Tuesday) / Morning / 4. M&E system (cont.)
Afternoon / 5.Gender mainstreaming in development project and gender sensitization of M&E strategy
28/11/2012
(Wednesday) / Morning / 6. Measuring and reporting 1st level result
7. Measuring and reporting 2st level result
Afternoon / 8. Steps to conduct survey
9. Questionnaire design
29/11/2012
(Thursday) / Morning / 10. Sampling methodology
11. Annual outcome survey
12. Annual outcome survey –(presented by representative from Tuyen Quang)
Afternoon / 13. Measuring and reporting 3rd level result
14. RIMS+ baseline survey (presented by representative from Ninh Thuan)
15. RIMS+ end-line survey (presented by representative from Ha Tinh)
30/11/2012
(Friday) / Morning / 16. Key informant interview
Afternoon / 17.Focus group discussion

The trainees

Total of 43trainees participated in the training course, including project staffs from Cao Bang (05), Bac Kan (07), Ha Tinh (03), Quang Binh (02), Tuyen Quang (05), Ninh Thuan (03), Gia Lai (07), Dak Nong (06), Ben Tre (05).

Most participants were enthusiastic and involved in the training course with good attitude. They actively engaged in the lecture by giving questions and examples and sharing their experiences that made the lesson attractive and exciting.

After the training, all trainees were asked for an assessment on the training quality and effectiveness. Through feedback and comments of trainees on training contents and teaching strategy, it will be used by trainers to improve the quality of the next courses in the future.

The trainers

Trainers participating in delivering the training include trainers from DEPOCEN: Ms. Nguyen Thi Phuong Mai - an adviser on M&E, conducting survey, in-depth interviews and questionnaire design; Ms. Tran Thanh Thuy, Ms. Nguyen Thi Huyen and Ms. Pho Thanh Huong.Ms. Hoang Hong Hanh,a gender expert contracted with APMAS, was invited to give lecture on gender mainstreaming. Trainers’ responsibility, enthusiasm and friendliness created a favorable environment for learners’ participation.

Table 2: Trainers participating in the training

Full name / Organization / Tasks
1. Ms. Nguyen Thi Phuong Mai / DEPOCEN / - Project Management
- Results Based Management
- M&E Basic concepts
- M&E system
- Measuring and reporting first level result
- Measuring and reporting second level result
- Measuring and reporting third level result
- Annual Outcome Survey
2. Ms. Hoang Hong Hanh / Gender expert / Gender mainstreaming in development project and gender sensitization of M&E strategy
3. Mrs. Tran Thanh Thuy / DEPOCEN / -Sampling methodology
-Key informant interview
4. Ms. Nguyen Thi Huyen / DEPOCEN / -Steps to conduct survey
5. Ms. Pho Thanh Huong / DEPOCEN / - Questionnaire design
6. Mr. Le Thanh Bien / TNSP TUYEN QUANG / Annual Outcome survey in Tuyen Quang
7. Mr. Truong Truong Han / TNSP NINH THUAN / RIMS+ baseline survey in Ninh Thuan
8. Mr. Le Van Hiep / IMPP HA TINH / RIMS+ end-line survey in Ha Tinh

Lecture 1: Project management

What is a project?

A general of related policies, activities and expenditures are designed to achieve certain objectives in a given time.

PM Triple Constraints

Main challenges of project management:

 Achieve all expected goals within a certain scope (workloads and technical requirements), to ensure project benefits reach to all beneficiaries including women, men, girls and boys

 Completed in time (performance progress),

 Use in given budget (investment)

 Meet all requirements (quality) and expectations.

1.Defining Project Management

Project management is the application of knowledge, skills, tools and techniques to project activities to meet project requirements
Organizing and managing resources so the project is completed within defined scope, quality, time and cost constraints

2.The importance of project management

Why Projects Fail

Lack of senior management commitment
Inadequate project planning (budget, schedule, scope, etc.)
Absence of community participatory
New or unfamiliar technology
Lack of defined, clear, or concise requirements

Why Projects Succeed

Sound project management processes
Senior management commitment
Detailed requirements
Realistic schedule
Good stakeholder relationships
Empowered project manager
Skilled and appropriate team members with defined roles and responsibilities
Availability of funding
Taken into account gender perspective

3.The role of the project manager

Empowerment

A project manager has to be empowered. With empowerment comes responsibility. The Project Manager represents the project in a professional manner and stand ups for the project when required

Project management skills

Understand knowledge, standards, and regulations applied in the project
Understand the project environment
Understand the important of gender perspective in development project
General management skills (budgeting, scheduling)
Communication skills
Interpersonal skills

4.Keys to successful project

Keys to successful project is PLAN - LEAD – COMMUNICATE - MANAGE

PLAN

•Identify all stakeholders

•Develop the project plan beforestarting the project

•Establish communications protocols

•Define your requirements in detail – remember gender sensitive requirements

•Identify risks

•Make contingency plans

•Plan a reasonable roll-out schedule

LEAD

•Ensure strong, committed management support

•Experience in project management

•Establish clearly defined directions

•Be proactive

•Set clear performance expectations

•Ask for technical assistance

•Do not start roll-out until pilot is complete

COMMUNICATE

•Communicate objectives frequently

•Recognize different perspectives, gender perspective

•Check assumptions frequently

•Manage expectations

•Make people aware of why gender mainstreaming is necessary

•Share success and broadcast achievements

•Invite feedback

MANAGE

Essential management skills:

•Project planning, initiation, and organization

•Recruiting people, training and keeping them

•Effective project negotiation

•Accurate estimating and cost control

•Manage unreasonable spend

•Implementation and project management

•Personal and project leadership

•Managing risk and making decisions

•Effective problem management

•Performance management

•Managing the projects within the organization

•Growing and sustaining a high-performance team

A good project team can be the key to a successful project.

5.Managing the contractor

Don’t “marry” the contractor
Trust but question
Their interest is their benefits
Your interest is a working system
Nothing is free
The result is only as good as your requirements
Don’t give the contractor or developer too much control, responsibility, or authority

Provide performance feedback

Build performance reviews into the contract
Be clear and specific
Be timely

6.Managing the project

Successful project management is delivering good result (quality product) that meets the donor’s requirements (client) within time, scope, and budget.

Gradual, progressive increase in the project’s scope that is not noticed immediately
Occurs when additional requirements result in scope change and can cause cost and schedule overruns

Keeping control of the budget

Monitor project cost
When having expenses incurred, determine the cause
Change the activities of the project, reduce scope
Avoid unapproved changes to the project

Reporting mechanism: Monthly report, Quarter report, Annual report, Progress report

Project management reporting

Establish reporting requirements upfront
Include the strengths, the limitations, and the mistakes
Be brief but accurate
Management really does want to know
Including sections on Gender issue within your reports will show how gender sensitive your project is

Project Management Tips

Project Manager does not work other than manage the project
Project Manager spends 90% of time communicating
Understand the project flow from beginning to end
Identify risks and develop contingencies
Clearly outline the critical path of the project
Establish the right relationships
Establish a speedy conflict resolution process
Be flexible
Be firm

Lecture 2:Result based management

1.What is Results-based management (RBM)

Results Based Management (RBM) is a management strategy by which an organization ensures that its processes, products and services contribute to the achievement of desired results (outputs, outcomes & impacts).

RBM rests on stakeholder participation and on clearly defined accountability for results. It also requires monitoring of progress towards results and reporting on performance/feedback which is carefully reviewed and used to further improve the design or implementation of the programme/project.

RBM is a life-cycle approach to management that integrates strategy, people, resources, processes, and measurements to improve decision making, transparency, and accountability. The approach focuses on achieving outcomes, implementing performance measurement, learning, and adapting, as well as reporting performance.

RBM is:

defining realistic expected results based on appropriate analysis;
clearly identifying program beneficiaries and designing programs to meet their needs;
monitoring progress toward results and resources consumed with the use of appropriate indicators;
identifying and managing risk while bearing in mind the expected results and necessary resources;
increasing knowledge by learning lessons and integrating them into decisions; and
Reporting on the results achieved and resources involved.

Why RBM?

Traditional management: focused inputs (what have spent), activities (what have done), and outputs (what have been produced), it did not tell the progress toward solving the problems the problems remained when projects were completed
Modern management: requires looking beyond activities and outputs to focus on actual results: the changes created, and contributed to, by our project/programme.

Establishing clearly defined expected results;

Collecting information to assess progress regularly,;

Taking timely corrective action;

Can manage projects or investments to maximize achievement of development results: a sustained improvement.

Pillars in RBM

RBM is based on four main pillars:

(i)the definition of strategic goals which provide a focus for action;

(ii)the specification of expected results which contribute to these goals and align programmes, processes and resources behind them;

(iii)on-going monitoring and assessment of performance, integrating lessons learnt into future planning;