M&E Draft Report

M&E DRAFT REPORT

1. Executive Summary

2.0 Background

The Goal of BESPOR is the achievement of education for all within a balanced national poverty strategy. Its purpose is to enhance the capacity of DoSE to implement the strategy in and through the educator sector. A key priority to support its implementation is strengthening monitoring and evaluation (M&E) systems and coordinating structures with DoSE. These will, in turn, support the establishment of a strong education sector programmeand facilitate the monitoring BESPOR’s progress towards expected outcomes (purpose level) as defined in the logical framework analysis (LFA).

2.1Government/Sectoral Policy

Monitoring and evaluation is guided by the draft 2004-2015 Education Policy which gives priority to basic education with a focus on increasing access, gender parity and quality of education. The policy in Section 11.5 gives the responsible for monitoring, support and supervision of teaching and learning at the school level to the Standards Quality Assurance Directorate (SQAD). Head Teachers are responsible for improving the performance output of both teachers and students with schools setting internal objectives, targets and strategies for quality improvement. Student assessment is based on learning achievement targets. This is discussed further in section 2.3 under situational analysis.

At the regional and central levels, there are no clear policies on monitoring and evaluation of administration, finance, and knowledge management. At the institutional level, section 14.2.1 states that SQAD is responsible for “monitoring compliance of the Education Policy and its associated acts and regulations with all institutions operating in The Gambia”. In reality, reviews have taken place on an adhoc and informal basis with GambiaCollegeand seldom with other institutions.

2.2 Stakeholders and Beneficiaries

A wide range of stakeholders are impacted on and/or by monitoring and evaluation processes and activities in the educator sector. These range from national institutions like WAEC, Gambia College, and DoSE with its Regional Directoratesto schools, teachers, parents,children and external providers; e.g. donors.

Table 1 provides an overall of their roles and how each stakeholder can contribute to the process and what the benefits could be derived in terms of BESPOR interventions.

Table 1: Stakeholder Analysis for M&E

Stakeholder / Role / Interest / Contribute / Receive
Senior Management Team / Policy Direction and Coordination / Accountability / Policy Directives, control and resources / A clearer picture how of how M&E works in DoSE
Coordination Committee Meetings / Policy Implementation and Problem-Solving / Accountability / Advise on how to strengthen coordination of M&E with DoSE / A clear picture of implementation issues related to M&E
Planning
Directorate/ EMIS / Control of Planning and EMIS / Accountability / Updating data for planning and tracking EFA targets / Improved EMIS system to be used as a tool for planning & monitoring at center and regional levels
Standards and Quality Assurance Directorate / Monitoring of quality of teaching and learning in schools / Improved QA monitoring systems to address EFA in the six regions / Inputs into baseline on teaching/learning conditions; Supervision of NATs, develop decentralized monitoring to the cluster level / Development of a more coordinated and coherent system for monitoring quality of teaching and learning across projects and programmes
HRD Directorate / Management of Staff Performance Appraisal data and processing information for rewards and sanctions / To institutionalize SPAS / Issues related to existing SPAS and practical ways to move forward / Establishment of a standardized system for performance assessment
Region 5 Education Directorate / Coordination of M&E activities / Improved delivery of education quality / Knowledge of situation in Region 5; lead changes to management systems, inputs into baseline studies / Coordinated M&E System across existing projects with Region 5;
better baseline data on quality of education in the region
Cluster Monitors / Monitoring Schools within designated cluster / Assessing whether schools are achieving their objectives/targets / Training needs of teachers and gaps in M&E System / Improved monitoring with agreed upon forms and reporting formats and flow of information
Schools / Monitoring Whole School Development including curriculum. resources, teachers, PTAs, etc. / How to effectively coordinate and monitor school targets / School Development Plans
Advise on what is going well and what is not at the school level and / Improved M&E Systems involving teachers and community at large
Community/
Parents / Local decision-makers / Improved education serve / Advise about the situation on the ground / Better understanding of how well their school and children are performing
Children / Pupil’s perception of what works and does not work in schools / Improved quality of education / Advise on how to improve school performance / Better classroom instruction and better school management
EFA Unit / Coordination EFA Activities in six Regions and conduct annual impact assessment and reports of progress towards targets / To use the Catalytic fund to achieve EFA Goals / Plan for EFA activities in Region 5 to avoid duplication / Increased Coordination and cooperation in Region 5 where BESPOR is piloting a whole school development model
West African Examination Council / Development and administration of National Assessment Tests for Grades 3 & 5 an d9 / Assessment of how well the education system is performing / Expertise to develop tests, do data analysis and reporting to see how well the system is performing in 2005 / Increased awareness of developing NATs that test a range of competencies and increased knowledge of the situation in the six regions
GambiaCollege / Lead a Longitudinal Study on WSD in 15 to 20 schools in Region 5 / Linking research to teaching training / Expertise to conduct school-based case studies / Ways to strengthen pre-service and in-service training
Donors / Financing and Monitoring / Influence / Advice and Resources / Goodwill and influence
National Consultants for BESPOR / Professional advice / Increase one’s professional knowledge/skills / Advice / Income

2.3 Situational Analysis

2.3.1 Monitoring Learning Achievements

The scale for gauging the effectiveness of the education system in The Gambia rests on pupils’ performance on examinations. With the phase out of the Grade 6 examination, there was an identified need to develop learning achievement targets (LATs) as benchmarks for improving the quality of education. These LATs were developed for lower basic education in 2000 and were revised in 2005 to align with the current syllabus and textbooks in use. How closely this alignment has been achieved requires further investigation. The revised materials are now being distributed to Regional Offices, which are responsible for familiarizing teachers with the LATs. At the time of this report, Regions 5 and 6 had not received the materials or training. For upper basic, LATs have not been developed although there is a plan to do this in the future.

The system for monitoring learning achievements (MLAs) is still evolving with a two prong approach that includes continuous assessment and National Achievement Tests (NATs). Continuous assessment by lower basic teachers remains problematic as many of them have do not have a good understanding of the LATs nor do they have the skills to develop performance monitoring tests (PMTs). SQAD plans to take corrective measures through in-service training of teachers to address this. For upper basic, teachers have been trained in continuous assessments and are doing this without any LATs as benchmarks.

Within the Education Policy, a National Achievement Test (NAT) should be conducted every two years using 25% of all pupils in Grades 3 and 5 to inform the system on pupils’ achievement at the lower basic education level. This has not happened. Delays are attributed to lack of resources to do this on a continuous basis.

Data on the quality of basic educationis very limited. The MLA study on Grade 4 learning achievements in 2000 indicated that less than ten percent of the children attained 70% or higher on learning achievements set for the four core subjects (English Language, Mathematics, Science and Social/Environmental Studies). Test items were based on LATs with no baseline data on what was actually being taught in the classroom. As such, many pupils were tested on items that they had not learned. This was a contributing factor to high failure rates as were the high level of difficulty of the items. The study showed also that pupils in private schools performed better than their colleagues in Mission and GovernmentSchools and that there were wide disparities across regions and between urban and rural schools. Similar findings were found in 2002 when WAEC carried out a small study of learning achievements of Grade 3 and 5 in 2002.Since then, no national assessment tests have been carried out.

It is important to note that the study of Grade 4 pupils in 1998 looked at conditions of teaching and learning that may have influenced children’s learning achievements. Findings indicated that a host of factors impacted on pupil’s performance. They ranged from the level of support from parents or caregivers, availability of resources, qualifications of teachers as well as their degree of confidence and motivation to the level of support from Head Teachers to name a few. There was unfortunately insufficient empirical data to determine which underlying factors had had a major negative impact. Also the study paid insufficient attention to gender issues. For instance, for many survey questions there was no gender disaggregation of the answers, representing a missed opportunity to build understanding of how pupils’ gender affects learning conditions and hence achievement.

One of the outcomes of the study was that monitoring, supervision and inspection processes needed to be revisited with the view to increasing the frequency of visits and providing professional support to teachers. SQAD’s recent efforts to decentralize monitoring to the cluster level with the appointment and training of Cluster Monitors in all regions is seen as a positive step towards strengthening the monitoring system. In reality, however, there are few Cluster Monitors on the ground in Region 5 where BESPOR will pilot whole school development (see section 2.3.2).

2.3.2 M&E Systems within DoSE

Within DoSE, there is no coordinated monitoring system across the Directorates at the national and regional levels. Existing M&E systems tend to operate as separate entities with little flow or sharing of information. Some of these systems are fairly well-developed (e.g. EMIS); others are in the process of being developed (SQAD’s new Participatory Monitoring System) and some are inactive (Staff Performance Appraisal System). Details follow.

EMIS

The EMIS was developed as a multi-segment database to provide management reports for planning purposes. It includes data on schools, personnel, financial and geographical information. Data across all segments is not up-to-date. Efforts to validate school data between 2002 and 2004 were plagued with problems related to poor data collection procedures and limited capacity of EMIS Staff. Efforts are underway to strengthen EMIS staff skills with the goal to complete validation of 2004 data by the end of May 2005. School data for 2005 was being collected at the time of this report. Data entry is scheduled to start in late May and take two months. If there are no major problems, data for 2005 should be ready to validate in September 2005 as a first step towards getting baseline data for BESPOR.

Data analysisremains a problem due to weak capacity of EMIS staff and high attrition rates. No in-depth data analysis has been done to support planning and monitoring since the EFA Assessment Report in 2000. What little has been produced has been done by a Peace Corp volunteer, who will leave in June, thus further aggravating the capacity problems. Furthermore, the 1993 Population Census data was used to calculate the net enrolment (NER) and will need to be updated with the 2003 Population data in order to provide a more accurate picture. To address these issues, the BESPOR Consultant for ITC is putting together a short-term plans to hire an interim consultant to support data analysis and long-term plans to build sustainability with the EMIS Unit (see Philip’s report). Such steps will enable BESPOR to support a 2005 EFA Assessment as part of its baseline studies. (see section 5.2.2 for details).

Staff Appraisal Performance System (SPAS)

In 2001, The Department of State for Education introduced a nationwide staff performance appraisal system (SPAS) to enable policy makers and managers to make well-informed decisions on how to effectively deploy and upgrade the civil service personnel. The system was mainly been used to assess the performance of teachers and head teachers, and on occasion middle and senior managers. Information was then entered to a database which has not been kept up to date since the Peace Core Volunteer left in 2003.

At the time of this report, it was not clear to what extent SPAS was in use at the school and regional levels, if at all. At the central level, SPAS information was still used for short-listing and for awards or promotion purposes. Informal reports, however, indicated that teachers were being penalized if their records were not up-to-date or if they had never been assessed. The PMO is one of the key stakeholders and has the mandate to do the system wide assessment but this has not happened.

Problems of SPAS have been related to a lack of interest and ownership by DoSE to maintain and improve the system. Assessment has been top down with Regional officers assessing head teachers and head teachers assessing teachers. There is no mechanism for bottom-up assessment of head teachers by teachers, parents and pupils. Related to this is the lack of a standardized system for assessment with competency levels for teachers, head teachers, managers to attain. Lines of reporting are unclear and feedback systems are not well developed to support the professional development of personnel.

Standards, Quality Assurance Directorate (SQAD)

SQAD is leading the drive to improve quality of teaching and learning in basic education by 2015. Based on the Ghanian model, a decentralized approach to monitoring and supervision is being developed whereby cluster monitors (CM) are appointed for each main cluster within the six regions. TORs for cluster monitors are multi-faceted and include responsibility for training needs analysis, training, monitoring resources, supervising teachers, and developing cluster action plans to name a few. This shift from the traditional role of the inspector as the authority to one of facilitation is aimed at fostering a participatory approach to monitoring where all stakeholders are working as a team to improve teaching and learning conditions in schools.

One important aspect of this new system is Participatory Performance Monitoring (PPM) where by schools and communities work together to set their own school targets for EFA, develop and implement schools plans and then monitor and assess progress towards these targets. This approach entails teachers setting performance monitoring tests (PMTs) each year and schools holding annual school performance appraisal meeting (SPAM) with community stakeholders to review the test and other results before setting new or revised targets. Community leaders and parents would be trained in school development and monitoring so that they would be able to make valuable contributions to the process as well as monitor how well their children are doing.

The system in many ways reflects a shift to a holistic model of whole school development in whichthe quality of teaching and learning must be improved if ultimately pupil achievement is to improve. This means that administration, management, curriculum, infrastructure, resources, etc should aim to positively impact upon teaching and learning and stakeholders need to adopt values, attitudes and beliefs to support this focus.

In reality, SQAD’s model is in its infancy stage with few cluster monitors on the job in Region 5, and little infrastructure and training in place to operationalise the system. At all levels, advocacy for innovation and change is still beingcarried out to build a sense of ownership among stakeholders. Lines of communication and reporting are being developed and feedback mechanisms established to allow continuing refinement and improvement as the system evolves. BESPOR is well-timed to model good practices and facilitate the development of this new monitoring system in Region 5.

2.5 Other Interventions

2.5.1 EFA Initiatives

Education For All National Action Plan 2004-2015 has been developed and funding was recently procured to the tune of $US 4 million from the Catalyic Fund to support EFA initiatives in 2005. Funding for 2006 will depend, to a large extent, on mechanisms put in place to monitor and evaluation progress against EFA Fast Track Initiative (FTI) targets that have been set by DoSE.

These M&E mechanisms are not well defined in documents reviewed except to say that PPARBD and REDs will jointly collect school data annually. SQAD will monitor learning achievements and school performance based on its proposed cluster model. HRD will continue to carry out staff appraisal schemes to inform the training and promotion prospects of teachers. The SMT and CCM will backstop and provide policy guidance and directives and that sector reviews with donors would be conducted annually. What remains unknown is how these mechanisms are working at school, cluster and regional levels, what indicators they are measuring and means of verification, who is responsible for data collection and reporting, and what mechanisms are in place to feedback to SMT and donor institutions.