Evaluation of support for continuing professional development of health workers in Tanzania:

Terms of Reference

1. Introduction

VSO are commissioning an evaluation of work we have conducted that has supported the continuing professional development (CPD) of health workers. According to Laura Golding and Ian Grey “continuing professional development is defined as a process by which individual healthcare professionals maintain and improve standards of healthcare practice, through development of knowledge, skills, attitudes and behaviours”.[1]The evaluation will help VSO to test some of the assumptions we make as an organisation about the impact of our CPD work whilst also informing VSOs health, HIV and AIDS programming moving forward.

2. Background

VSO has been deploying skilled health workers as volunteers for over 50 years. More recently, the work of our health volunteers placed in partner health facilities in developing countries has shifted away from focusing solely on service delivery. VSO’s health, HIV and AIDS theory of change (see appendix A) states “within health facilities and district health officesVSO now supports the delivery of continuing professional development training to improve health worker and health manager skills, attitude and behaviour with a particular focus on patient friendly, gender sensitive and inclusive approaches to service delivery”. [2]

The ultimate aim of our support to CPD is to increase the number of qualified and professionally competent health workers engaged in direct health service delivery and so contribute towards addressing the human resources for health crisis that exists in many of the countries where VSO operates. VSO believes that building and maintaining an effective and well functioning workforce is one of the key pillars needed to support a successful health system. If this is achieved we believe that the health outcomes of the poorest and most vulnerable who access health services will improve.

The VSO health, HIV and AIDS theory of change highlights the interdependencies of the different strands of our work as we seek to enable poor and marginalised communities living in poverty to access and advocate for better quality maternal, child, sexual and reproductive health and HIV and AIDS services. We recognise that in addition to providing CPD directly to health workers, VSO also needs to advocate for national governments to demonstrate their commitment to health workforces through the development of effective CPD policies. Where CPD policies do exist, VSO also works to ensure that health workers are able to equitably access CPD training through supporting the development of well functioning human resourcesfor health management approaches and systems.

The range of CPD support provided by VSO volunteers varies depending on context, partner demand and individual VSO country strategies. For example, VSO’s position paper ‘Continuing Professional Development for Health Workers’[3] highlights 3 different ways of how VSO volunteers have contributed ranging from providing on–the-job training to anaesthetists in Ethiopia, to developing competency based training for health managers in Mongolia to carrying out an in-depth evaluation of a national CPD programme in Malawi. However, the majority of VSO’s work in support of CPD is generic taking place at health facility and community level as in Tanzania, hence the decision to focus on evaluating CPD support for targetedhealth interventions inTanzania. Other countries where we currently deliver CPD programmes in health include Sierra Leone, , Nepal and Zambia.

2. Scope of the evaluation

The primary focus for the evaluation is learning about CPD inputs to inform best practice and VSOs health programming moving forward. The evaluation is being commissioned as part of VSO’s Impact and Effectiveness programme which aims to increase evaluation capacity across the organisation.

Evaluation findings will serve the following key purposes:

  1. To test a set of key assumptions about our work in CPD and to provide evidence that either supports or challenges our traditionally held organisational assumptions.
  2. To inform the ongoing development of the theory of change for VSOs health, HIV and AIDS work, and contribute to a global evidence base examining the contribution volunteers make to development. This work is being led by advisors across VSOs Strategy and Programme Effectiveness Group.
  3. To feed into the Independent Progress Review that will be carried out on VSO’s Strategic Grant Agreement with DFID, due summer 2014.
  4. To strengthen the evidence for our annual review process and inform VSO’s future work inhealth, HIV and AIDS
  5. Last but not least this evaluation is part of a wider programme called the Impact and Effectiveness Programme which aims to enable VSO staff expand their experience on designing, commissioning, managing and using evaluations. This is very much embedded in the overall vision of evidence based programming and embedding learning throughout the organisation.

3. Specific context of the evaluation: Health CPD in Tanzania

The evaluation will focus on two initiatives which deliver CPD in Tanzania; No Baby Left Out and the Hospital Strengthening Project.Fieldwork for the evaluation will cover two regions:

  • The Lindi region in the south of the country where No Baby Left Out is implemented
  • The Kagera region in the north of the country where the Hospital Strengthening project is implemented with Kagondo and Rubya Hospitals being identified as sample sites for the evaluation.
  1. No Baby Left Out (NBLO) is a collaboration between the Regional Health Management Team Lindi Region, VSO, GIZ and the partner hospitals St Walburg’s (District Hospital), Sokoine Hospital (Regional Referral Hospital) and Masasi Hospital (Town Hospital).VSO has contributed to this project by providing funding and technical support in the shape of a 10 month volunteer placement (October 2012 – August 2013). The volunteer – Dr Sandra Subtil – was recruited through the VSO/RCPCH partnership.

It aims to reduce neonatal mortality by:

  • Strengthening the Newborn and Preterm Care and Treatment at the Health Facility level (Rural Dispensaries and Health Centers)
  • Ensuring centers of advanced Neonatal Care are developed (Nyangao, Lindi, Masasi hospitals)
  • Strengthening the Referral System between community and facilities

This is achieved by:

•Capacity building (on-the-job training)

•Clearly defined local responsibilities and strong partnerships

•Introduction of an IMCI related Newborn Triage Checklist (NTC)

•Provision of essential equipment for neonatal resuscitation

•Monitoring and Evaluation within an integrated system of care

There are on average 75,003 babies born each year in the Lindi region and in the region of 777 health workers employed in the regional health facilities. By the end of July 2013, the following results had been reported.

-178 trained health staff

-70 trained health facilities (almost the entire Lindi Municipal and District and parts of Masasi District are covered)

-1400 newborns have been screened using the NTC

-50 referred babies with a survival rate of 80 %

-Resuscitation equipment delivered to 92% of the health facilities (45 ventilation bags with masks and 47 penguin sucker)

-85% of the referred babies transported by ambulance

  1. The Hospital Strengthening Projectis funded through DFID unrestricted funding, and contributes to the hospital system strengthening and quality of care outcomes. The project seeks to improve the standard of medical care in hospital settings through:
  2. On the job training to improve the quality of service delivery in the laboratory and clinical areas
  3. Assisting medical officers and administration staff in the development of local protocols and implementation of the national guidelines.
  4. Improving the administration ofstaff working conditions and introducing a performance based management systemi.e. developing an incentive structure to improve staff performance

The project works across 10 hospitals in Tanzania and we have identified the following two hospitals for fieldwork:

At Kagondo Hospital over the last 12 months volunteers have been delivering on the job training to doctors, clinical officers and nurses to improve the quality of services in the hospital. In the laboratory volunteers have established operational procedures and standards, developing clinical protocols for the doctors, clinical officers and nurses. On the job training on the proper use of X-RAY and ECG machine has also taken place. VSO has contributed three international volunteers and one national volunteer to this project at Kagondo Hospital over the last year and the partnership has been in place since 2010.

The following results have been reported:

  • Improvement in technical competence of the staff
  • The improved quality of care in the hospital which has increased the number of patients using the hospital especially for laboratory services.
  • Reduction of maternal and neonatal mortality rates.
  • Improved administration practices at the hospital which has improved the way staff welfare issues and challenges are dealt with which has increased staff motivation and retention.
  • The number of people seeking HIV counselling service has increased from 50% to 84%as a result of the improved HIV services.
  • Increased knowledge and skills of staff in a range of diagnostic and reporting measures

At Rubya Hospital support has focused on administration and workforce issues but also on improving surgical management through on the job training. Another element has involved working with Rubya nursing school to improve the quality of service delivery through preservice training and CPD. VSO has contributed two international volunteers to this project at Rubya Hospital over the last year and the partnership has been in place since 2012.

The following results have been reported:

  • Preceptorship training to nurses has helped to improve delivery of care to patients
  • Improved teaching and supervision of clinical nursing skills
  • On the job training has improved staff management and accountability skills.

4. Tasks

Given the context for this study, there are fourareas for evaluative inquiry:

Task 1:Outlining the context

  • What is the national context/landscape on CPD in Tanzania – is there a national training plan, is there licensing/accreditation and if so is it mandatory?
  • What CPD is being delivered within the initiatives identified for this studyand what methods are being used to deliver the CPD?

Task 2: Gauging outcomes and impact:

What has VSOs contribution been and what has been the impact (if any) since the start of the programmes:

a)Health workers and managers within the selected partner organisations

  • Improved clinical skills resulting in better service delivery
  • Improved management of health facility services
  • Improved attitudes amongst the facility health workforce towards service users
  • Improved morale and motivation amongst health workforces within the health facility

b)Health service users of the selected partner organisations

  • Improved patient satisfaction with services received through the health facility
  • Improved health outcomes amongst service users

c)The selected health partner organisations themselves

  • Improved responsiveness to need?
  • Increased utilisation of services offered
  • Improved reputation of the health facility locally and nationally
  • Increased range of services offered

Task 3:Assessing the value of the VSO CPD model

Test the assumptions upon which the model is based:

  • That it is welcomed and valued by our partner organisations and the health workers in country themselves
  • That it directly contributes towards improving the health outcomes of service users
  • That it is sustainable in terms of longer term impact
  • That it is worthy of replication and scale up as a health systems strengthening approach.

Addressing the following key questions:

  • Is the model of placing skilled health volunteers within the selected health facilities to build the capacity of the workforce effective and worthy of replication?
  • Is there anything about the way VSO has worked with its volunteers and partner organisation that has influenced the success – or otherwise – of the model? Which types of activities/interactions are most effective and why?

Task 4:Lessons learned and recommendations

  • What does this evaluation tell us about the CPD model as part of VSO’s health programming approach?
  • Reflecting on the CPD model, what is VSOs actual or potential unique selling point / niche skill area(s) i.e. what value in our use of long and short term volunteers makes our approach different or more effective?
  • What were the elements of the CPD model that have worked well and could / should be replicated?
  • Have there been any weaknesses or failures? If so, what can VSO learn from them?
  • What are the key pre-requisites required (in contextual analysis, VSO staff capacity, selection of local partners and positioning of volunteer placements) to make real transformative shifts in the quality of service delivery and improving health outcomes for service users that must be included/addressed in any VSO programmes that include supporting CPD as an approach?

4. Methodology

The contractors are expected to develop their methodological approach in their proposal, which will be refined and finalised at the inception phase. VSO values mixed method approaches. The proposed approach should include at the minimum:desk based research, Skype consultations with relevant stakeholders and country visits to conduct primary research.

In addition, all methodologies proposed should be participatory in nature - including substantive engagement with stakeholders throughout.

VSO and the partner organisations will provide a range of documentation and monitoring data to inform the scoping process, including:

  • VSO’s position statement on CPD
  • VSO’s theory of change for health, HIV and AIDS (Appendix A)
  • VSO volunteer and partner reports
  • Annual VSO partnership review findings
  • Relevant health institution data e.g. patient attendance records, patient satisfaction surveys, health statistics such as maternal mortality/morbidity, child mortality/morbidity, STI treatment rates, HIV infection rates, family planning access rates etc.
  • Standard monitoring data
  • Evidence based case study for No Baby Left Out

5. Timing and deliverables

The following milestones have been agreed for the study:

  • The inception meeting will be held
  • DELIVERABLE 1: Inception report – to be submitted by 22 April 2014

To include a finalised methodology and workplan, with draft data collection tools. The inception report will be reviewed by VSO, revised and signed-off by 25 April 14.

  • Fieldwork should be conducted during May 2014.
  • DELIVERABLE 2: Draft final report – to be submitted by 16 June 14.

To include a 3-page executive summary, with summarised raw data provided as annexes. The draft final report will be reviewed by VSO, revised and signed-off by 30 June 2014.

For quality assurance purposes, all statements made in the report should be evidence-based, and this evidence should be available, in full and specific to that particular statement (i.e. no unorganised raw data but raw data that are specific to whichever statement is made), upon request.

  • DELIVERABLE 3: A presentation of the evaluation’s findings to be delivered to the Steering Group.

In addition, VSO will also require regular updates on project progress to be submitted to Project Manager via telephone/e-mail.

6. Reporting and accountability

The evaluator will formally report to Janet Clark, Project Manager Monitoring and Evaluation within VSOs London-based Strategy & Programme Effectiveness Group. Day to day support, facilitation and coordination will be provided by Rosalia Marandu, health programme manager in Tanzania.

A cross-functional, senior-level Steering Group will be convened to oversee the evaluation. This includes:

  • Cvetina Yocheva – Head of Impact and Accountability
  • Janet Clark – Project Manager, Monitoring and Evaluation
  • Clive Ingleby – Global Advisor, Health, HIV and AIDS
  • Jean Van Wetter – VSO Country Director, Tanzania
  • Rosalia Marandu, VSO Health Programme Manager, Tanzania
  • Jenny Snook – Healthcare Marketing Adviser

7. Expertise required

VSO invites applications from suitably qualified consultants. Specifically we require:

  • Experience in conducting and/or managing evaluation studies in Africa;
  • Experience of participatory approaches
  • Specialist knowledge of evaluation methodologies for health interventions in developing countries;
  • Strong facilitation and communication skills, with experience of working collaboratively with local teams of development professionals;
  • Language / translation skills, as appropriate for proposed methodology visits; and,
  • An understanding of the importance of CPD for health workers

8. Insurance requirements

Contractors are required to have in place professional indemnity insurance cover for £1,000,000 and public liability insurance cover for £1,000,000.

9. Budget and payment schedule

The indicative budget for this study is £25,000. Please note, proposals submitted that exceed this amount may be automatically rejected.

Payments will be made as follows, in line with receipt of key deliverables:

  • 30% upon approval of the inception report.
  • 40% upon presentation of the first draft report
  • 30% upon submission and approval of the final report

9. Proposal submission

Proposalsmust demonstrate an understanding of the tender specifications and should be clear, concise and unambiguous. Each tender submissionshould be no longer than 25 pages andmust include:

  • Approach

An understanding of the project environment and our requirements for the evaluation including a detailed description of the methods to be used in undertaking the project. They must include recommendations for the size and composition of the sample, justifying the approach and highlighting any risks.

  • Timings

A detailed timetable for carrying out the work based on the proposed approach and method. This should highlight key milestones and deadlines, including suggested meetings.

  • Staff

Alist of staff that will be involved in the project at all levels from director, project manager through tofield researchers – with a summary of their relevant experience and proposed role in the project.A full CV for each team member should be provided as an annex.

  • Budget

Abreakdown of costs, including day rates for each member with time allocations by task.The price quoted must be fixed;inclusive of all staff, travel and subsistence costs; exclusive of all duties and taxes; and expressed in GBP.[4]