Tool 7

Guidelines for Baseline Study

The traditional definition of a Baseline Study is ”an analysis describing the situation prior to a development intervention, against which progress can be assessed or comparisons made” (DAC, ”Glossary of key Terms in Evaluation and Results Based Management”).

Although also IBIS subscribes to this definition, it only describes one of the uses of a baseline study – to be able to measure progresscompared to the situation at the start of a development intervention. However, the baseline study has other merits. It serves as familiarization with the programme or project area for the programme and partner staff and tests the assumptions behind the project or programme formulation. And further, it serves as the first basis for evidence based advocacy.

1) Baseline for measuring progress

The baseline study differs from the context analysis[1]. The context analysis is part of the Country Strategy and is revisited in the ThematicProgramme Document, where also objectives and indicators of the TP are defined. The baseline study should not repeat the context analysis, but takes its point of departure in the objectives and indicators spelled out in the thematic programme document, in the concrete geographic area of operation of the TP. It analyses the factual situation on the ground and shows what the situation is when the development interventions start. Therefore, the baseline study is vital for all future monitoring and evaluation. Further, it serves to verify and qualify (and if necessary revise) the indicators on TP specific objectives as well as outcome indicators at TP and project levels.

Data collection and -analysis are the tools used in baseline studies, and the data should be quantitative as well as qualitative. (For an example of methods and tools, see Annex A).The important thing is that the data collection directly targets the objectives and indicators of the thematic programme and not pretends to be a fully-fledged socio-economic analysis. The great danger is to collect more data than it is possible to process and analyse.

The concrete elaboration of methodology, questionnaires, semi-structured interview guides, observation tools, etc. should be developed together with programme and partner staff, so that everybody understands and agrees on their content and the link to the TP or project objectives and expected outcome. Capacity building will often be important in order to secure that interview- and recording techniques are learned and agreed upon.

2) Familiarization with the programme and project area and the target group

It should be prioritised that it is the programme and partner staff who conducts the baseline study instead of for example a hired external consultant alone. It is the programme- and/or partner staff that will come back to the indicators at least once a year and their ownership to the monitoring process is important. Further, the data collection serves as a first familiarization with the programme or project area and the target group, already building up relations and giving the possibility of discussing the planned development intervention with the target group.For familiarization purposes it can be important to also include some data collection that does not directly contribute to the monitoring system, for examples see Annex B.

3) Basis for evidence based advocacy

Already in the context analysis national statistics or estimates, based on averages, will often be referred to. As the baseline study investigates the real situation on the ground it gives a specific background to future advocacy work. This goes for quantitative as well as qualitative data, i.e. the number of out-of-school children in a district (quantitative data) is in itself a good starting point for advocacy activities in an education TP. Or citizens’ perception of own rights and local government’s role and duties (qualitative data) will inform the content of capacity building for advocacy in a governance TP.

Organisational baseline study

IBIS Tool 1 “Partner Assessment and Partnership Development Plan” describes the tools and process for a partner assessment, which in fact constitutes the baseline for the development of the organisation in question.

Cost considerations

The data to be collected and analysed should be chosen keeping future monitoring in mind. The baseline study in itself might include data that are seen as important in order to understand the social processes and the complexity of the programme or project area. However, after the baseline study it is important to limit future data collection for monitoring to a few and manageable processes. It is not advisable to repeat the whole baseline study once a year. As the initial baseline study serves various purposes as shown above, it should be regarded as an investment in developing the TP further and improving implementation strategies. The ongoing or annual monitoring should be kept simple and easy and not take up more staff time than necessary.

Timing

As a rule, a baseline study should be one of the first activities after an inception phase Program Document has been approved, necessary staff hired and partnership contracts entered into. If a Thematic Program starts with a fully fledged Programme Document (i.e. for a second phase of a Thematic Program) there might also be need for a baseline study, if for example the geographical area changes or if new types of development problems are targeted.

Data collection and analysis

Alongside the development of different tools - quantitative and qualitative – (or adaptation of existing tools), the criteria for selecting whom and where to interview and observe has to be developed and be clear to all participants. After a test and perhaps revision of the tools and after the necessary capacity building of staff, the data collection should take place in a short and concentrated time span. During the data collection period it is important to strike a balance between interpreting the results and sticking to the agreed methods and tools. Some of the data will after the collection period be computerized and processed, whereas some of the interview results will need discussions and analyses by the staff that collected them, and time should be set off for this also. After computerization of (some of) the data, analysis of these should be taken up again and concluded. The baseline study should then be compared to the project and programme documents and necessary adjustments of indicators and strategies discussed and decided upon.

Tools (quantitative and qualitative, disaggregated by sex):

  • Questionnaires (for people to fill in or for interviewers to fill in (depending on literacy levels/language)
  • Interview guides, fully or semi-structured
  • Individual and (focus) group interviews
  • Various participatory methods (PRA (Participatory rural appraisal), PPA (Participatory Poverty Assessment and others)), (Robert Chambers et.al.))
  • Observation tools
  • Checklists (Example: INTRAC Praxis Guide no. 1, “Sharpening the Development Process”, p.24)

For a good example of a comprehensive IBIS education baseline study, see “EDEC Baseline Study, June 2008”that also describes the process of conducting the study. (To be found in IBIS AHS: Activities/Mozambique/EDEC/Monitoring and evaluation).

Revised January 2012

Annex A

Examples of typical elements in a baseline study for an education project

The below list is in no way exhaustive, but can give an idea about some basic data and methods/tools, which almost always will be necessary to include in a base-line study for an education project.

Subject / Examples of methods and tools
Number of matriculated boys and girls at each grade level in primary school / Existing national, provincial, district, county statistics (gender segregated)
If the above are unavailable, numbers can be obtained at each single school
Completion and retention rate of boys and girls / As above
Number of children of school age who are not in school / Existing national, provincial, district, county population census of number of boys and girls of the relevant age groups – subtracting the number of matriculated boys and girls. (Projection of data from older censuses with national population growth rate)
Number of trained and untrained teachers, male and female / Existing national, provincial, district, county statistics (gender segregated)
If the above are unavailable, numbers can be obtained at each single school
School Board (members’) perception of their role in relation to teachers and school management / Qualitative, semi-structured interviews – or
Questionnaires – or
Focus group discussions – or
a combination of these
Teachers’ and School management’s attitude to School Boards / As above
Parents’ and community members’ attitude to boys’ and girls’ education / As above
Teachers’ teaching methods / Classroom observations (structured in relation to use of different methods, gender specific)
Qualitative semi-structured interviews with teachers – or questionnaires – or focus group discussions
Existence of didactic material at schools / Interview with school management and head teachers
Observation - a “check-list” format should be elaborated and used for all schools
Civil society organisations working within education in the area / Interviews with education authorities, interviews with community leaders.
IBIS’ Toolbox Paper no. 1
Budgets for Schools (Didactic materials, teacher salaries, operational costs, etc.). Versus real life transfers from district, provincial and/or national level. / Interviews with Education authorities at local levels. (in some cases also other local government bodies) Cross check with the single schools’ management.
Etc. …

Annex B

Familiarization - examples of what should be part of a baseline study, but NOT part of the annual monitoring, unless substantial changes have occurred

The majority of IBIS’ Thematic Programmes will be within the focal areas of education and/or governance and the main part of the baseline study will concentrate on issues related to these, but a number of other issues are also of importance for the implementation of development activities in a given area.Some of such issues should be mapped and understood during the baseline study, as they will be of importance for the implementation of the project or programme. However, these factors, although important, do not need to be part of the monitoring system. Below are a couple of examples of this type of information and its relation to programme and project activity planning.

Other examples:

  • Household/family structure and division of work
  • Migration patterns/time of year/gender/age groups
  • Social and physical infrastructure

1

[1] See: Guidelines to Context Analysis, AHS/GLS/Programme methodology/Context analysis