Agricultural Policy Analysis and Institutional Reform in Africa:

A Tracer Study Evaluation of Two “Training of Trainers” Workshops

Regional Workshop on Agricultural Policy and Institutional Reform (Kenya, March 1996)

Regional Workshop on Sustainable Management of Renewable Natural Resources (Ghana, March 1997)

EDI Evaluation Studies

Number ES98-15

Evaluation Unit

Economic Development Institute

The World Bank

Washington, D.C.

Copyright © 1998

The International Bank for Reconstruction

and Development/The World Bank

1818 H Street, N.W.

Washington, D.C. 20433, U.S.A.

The World Bank enjoys copyright under protocol 2 of the Universal Copyright Convention. This material may nonetheless be copied for research, educational, or scholarly purposes only in the member countries of The World Bank. Material in this series is subject to revision. The findings, interpretations, and conclusions expressed in this document are entirely those of the author(s) and should not be attributed in any manner to The World Bank, to its affiliated organizations, or the members of its Board of Directors or the countries they represent. If this is reproduced or translated, EDI would appreciate a copy.

This report was prepared by Marie-Aline Wood and Mita Marra,

Evaluation Unit, Economic Development Institute.

CONTENTS

Executive Summary...... vii

1. Introduction and Background...... 1

Workshop Trainer-participants and Program...... 1

Networking among African Universities...... 2

Feedback from the End-of-Workshop Evaluation (EOW)...... 2

2. Evaluation Design and Objectives...... 3

3. Evaluation Constraints...... 5

4. Analysis of Results...... 7

5. Conclusions and Recommendations...... 11

Annex 1 - Questionnaire...... 13

Annex 2 - List of Trainer-participants...... 15

EXECUTIVE SUMMARY

Introduction and Context

This report presents the results of a tracer study evaluation of two regional Training-of-Trainers (TOT) Workshops conducted by the Economic Development Institute (EDI) during March 1996 and February 1997 in Naivasha, Kenya and Akosombo, Ghana respectively. The two workshops represent part of a multi-year program on Agricultural Policy Analysis and Institutional Reform carried out by EDI with twenty-three partner institutions in Africa.

The context of the program is (i) the continent-wide process of structural adjustment that governments are undertaking in response to external economic shocks and to previous domestic policies that were biased against the agricultural sector and proved economically unsustainable; and (ii) the changing role of the state from a direct provider of a wide range of agricultural services to a facilitator of increased private and civil sector provision of agricultural services.

The overall program has consisted of awareness and sensitizing seminars for senior officials as well as sector management seminars and special topic seminars on agricultural research and rural infrastructure for policy managers and analysts. Special Training-of-Trainers workshops have been organized, on the same subjects, for trainers from universities and government.

Specifically, the two workshops that are the focus of this tracer study were held in response to a recommendation that arose from a program development workshop conducted in Harare, Zimbabwe in February 1995. Trainer-participants recommended that EDI should institute a series of workshops to provide refresher training on current developments in policy and institutional analysis, to upgrade trainer-participants’ teaching skills, and to enhance networking among themselves. The topics consisted of the application of the new institutional economics to agricultural policy and institutional reform and the sustainable management of renewable natural resources.

Program Objectives

The goal of the program is to build analytical and training capacity in government and in national universities to support agricultural policy reforms that will complement macro-economic and sector adjustment programs in individual countries. The Training-of-Trainers workshops are expected to enhance the trainer-participants’ capacity to conduct workshops at the national level, building on curricula and training materials developed within the program.

In particular, the objectives of the Regional TOT Workshops are:

  • to enhance the capacity of trainers and facilitators;
  • to provide them with material for teaching and research; and
  • to create an informal network of partners interested in the application of the new institutional economics to agricultural policy.

Evaluation Objectives and Methods

The tracer evaluation has been undertaken by the Evaluation Unit of EDI (EDIES) in collaboration with the task managers. The evaluation seeks to identify the outcomes of the program on trainer-participants’ learning and teaching skills. It focuses on the didactic and methodological aspects of the training of trainers and it analyzes if partnerships and networks have taken place as a result of the workshops.

The approach chosen is to focus on four key issues, specifically:

  • did trainer-participants learn new information and best practices which they can use in their work;
  • did trainer-participants use the training materials provided;
  • to what extent have the communication skills of trainer-participants been strengthened; and
  • have inter-institutional networks among EDI’s partner universities been fostered?

The questionnaire that was sent to and completed by trainer-participants provides insights of the respondents to each of the key issues mentioned above. Government officials were not included in this study because it was intended to assess whether university-based trainers have incorporated these materials into their training activities.

The first two questions on the questionnaire seek feedback on the overall usefulness of the workshops as learning events and their capacity to enhance trainers’ and facilitators’ skills. The third question sheds light on trainer-participant use of the material provided. The fourth analyzes the format of the workshops, particularly, if it facilitated communication and collaboration among trainer-participants. The last question leaves space for comments and suggestions.

A total of twenty-eight questionnaires were mailed to trainer-participants (five of them participated in both workshops). They were selected by the managers of the program because they were considered as the target audience of the workshops. The total number of trainer-participants was forty-four (twenty-four in Kenya and twenty in Ghana), but the eleven senior government officials that participated were not included in the study. To encourage the completion of the questionnaires e-mails and follow-up letters were sent out and by January 30, 1998, of the twenty-eight questionnaires, twelve were completed and returned to the Evaluation Unit, for a response rate of forty-three percent. Although this is a lower rate than anticipated, the twelve respondents are occupationally representative of the total population of the trainer-participants who attended.

The returned questionnaires were analyzed according to their content by two evaluators in the EDI Evaluation Unit. Closed-ended responses were examined for their validity. The responses were tallied in each category for all questions. The arithmetic mean was calculated and the frequency of responses was expressed in percentage terms. Open-ended responses were content analyzed into themes that best qualified their responses.

Evaluation Findings

Results from the evaluation show an overall high approval rating by trainer-participants with several areas merging as specific strengths and weaknesses.

Respondents reported favorable judgments on topics regarding institutional economics and policy analysis. The material provided was considered useful for the analysis of agricultural policy and natural resource management. For respondents’ training purposes, the material was helpful for making presentations and for proposals of research projects. Largely, the materials were used for teaching in university (sixty-seven percent) and in consulting projects (fifty percent) for poverty reduction and environmental protection policies for national governments and international organizations. It was noted, however, that the material was not sent far enough in advance to be read beforehand.

Respondents did point out that workshops enabled them to better prepare and utilize case studies. However, they asked for more examples and case studies from contexts to which they could relate. It may be appropriate in future TOT workshops to present more African cases, with special emphasis on developments in the field. Case studies are important not only for training purposes, but also for influencing senior policy-makers and for learning about events and activities in other countries.

The respondents’ reactions to the workshop’s format were particularly favorable. Group and team work were positively rated as a means of triggering discussion and facilitating communication and collaboration. Trainer-participants stated that the workshops helped them most in strengthening their interpersonal skills. The duration of the workshops, however, was considered too short to allow the development of strong interpersonal links.

On the selection of trainer-participants by local universities, respondents indicated that they were singled out regardless of their scientific interests and questioned why some of them were invited to the Kenya workshop, but not invited to the Ghana workshop. This hindered contacts and collaboration and might have decreased the overall quality of the second workshop. It was recommended by respondents that local institutions make the selection criteria more transparent by giving priority to specific needs of professors, lecturers and researchers. The weakness of the selection process also led to another weak point: the lack of continuity. At the end of the Kenya workshop, EDI established a mailing list based upon past participation in its African policy analysis training activities and used it to circulate new analytical and teaching materials. According to the managers of the program, this attempt was only successful with the five trainer-participants that participated in both workshops. They were further involved in the organization of other EDI’s activities and are considered to be key “resource persons” with whom task managers are developing a strong partnership for the implementation of an exit strategy.

Networking among universities is an issue. Partnerships have been limited because of the high communication costs. This is of great concern for sustainability. There is, however, an opportunity to foster interactions among the institutions, individuals, and EDI by strengthening and enlarging the existing network. Greater involvement of trainer-participants in the organization of EDI’s activities is consistent with the objective of the workshops and of the World Bank. However, this would entail time and resource costs, and would need to be explicitly considered by EDI management.

1

1

INTRODUCTION AND BACKGROUND

Introduction

The Economic DevelopmentInstitute (EDI) has collaborated with a range of African universities and policy units of different governments since 1989 in training activities designed to enhance the formulation and application of appropriate public policy for the agricultural sector. Working initially with anglophone countries, the program has grown to encompass many of the francophone countries of West Africa as well. The program has consisted of (1) awareness and sensitizing seminars for senior officials, (2) sector management seminars for policy managers and analysts, (3) training of trainers workshops, (4) program and curriculum development activities for trainers from universities and governments, and (5) special topic seminars on agricultural research and rural infrastructure.

Building on the consensus that emerged at the program development workshop held in Harare, Zimbabwe, in February 1995, a number of annual TOT workshops rotating among southern, eastern, and western Africa were designed. These workshops responded to trainer-participants’ need to receive refresher training on current developments in policy analysis, upgrading teaching skills, and enhancing networking among themselves. To obtain guidance from the profession on appropriate ways of incorporating the new institutional economics into its training programs, EDI invited leading academics and practitioners in Washington, D.C., in December 1995 to develop the curriculum of the TOT workshops. The first TOT workshop was held in Naivasha, Kenya, in March 1996 on the application of the new institutional economics to agricultural policy and institutional reform. The second, on the Sustainable Management of Renewable Natural Resources was held in Akosombo, Ghana in February 1997. In March 1998, the third workshop on building capacity for training in relation to agricultural sector investment programs was held in Swaziland.

Workshop Trainer-Participants and Program

At the week-long seminar in Naivasha, Kenya, in March 1996, EDI invited English speaking trainer-participants from universities and agricultural policy units from fifteen African countries. Those who attended came from Cameroon, Côte d’Ivoire, Ethiopia, Ghana, Kenya, Malawi, Sierra Leone, South Africa, Swaziland, Tanzania, Uganda and Zambia. Of the twenty four trainer-participants, thirteen came from universities and eleven from Ministries of Agriculture. The second workshop in Akosombo, Ghana was attended by twenty trainer-participants that came form Cameroon, Côte d’Ivoire, Ghana, Kenya, Malawi, Mali, Mauritania, Nigeria, Sénégal, Sierra Leone, Swaziland, Tanzania, Uganda, Zambia and Zimbabwe. Five of the trainer-participants attended both workshops.

During the first workshop, the first three days were devoted to a discussion of the “new institutional economics.” It consisted of an overview and its application to both the substance and the process of policy and institutional reform in the agricultural sector. The fourth day was devoted to effective training practices, the preparation of training activities, the preparation and use of teaching cases, and the preparation and delivery of effective presentations. The fifth and last day was spent on discussion about the future of networking among participating institutions.

During the second seminar in Akosombo, Ghana, in February 1997, the first three days were devoted to institutional issues in relation to the sustainable management of renewable natural resources. The last two days were devoted to a field trip, to training in facilitating more effective group and team work, and to expanded networking among participating African Universities.

Networking Among African Universities

According to the final report of the Kenya workshop, trainer-participants agreed that the series of regional TOT workshops was a very valuable innovation in many ways, not only for the training which occurred, but also for the informal networking that it facilitated. At the Ghana workshop, in particular, trainer-participants recommended continuing this series of annual TOT workshops and enhancing the informal network in a number of ways, such as:

  • Putting together a list of trainer-participants of previous workshops;
  • Sharing more information on what everybody is doing in their own countries; and
  • Establishing a newsletter.

The report also shows that trainer-participants discussed the issue of case studies. They agreed that case studies were important, not only for training purposes, but also for influencing senior-policy-makers and for learning about events in other countries. Trainer-participants indicated that they would like to see more effort towards the production of African case studies which could reinforce the framework and the concepts introduced in these seminars.

Feedback from the End of Workshop Evaluation (EOW)

Respondents completed a standard evaluation for training-of-trainers activities. Overall, they rated the workshops a success. In a range of 1 to 6, most scores were between 4.6 and 5.0. For both workshops the highest scores were given for providing pertinent and useful supporting materials (5.19 and 5.21 respectively for the Ghana and Kenya workshops), maintaining interest and participation (4.95 and 5.00) and satisfying trainer-participants’ learning needs (4.90 and 5.00).

At the Ghana workshop, trainer-participants also completed a supplementary evaluation of the individual components of the workshop. Of the plenary presentations, respondents evaluated the “substance of policy and institutional reform” and “alternatives regimes for sustainable natural resource management” the highest (5.48 for the presentation, 5.33 for insights and 5.33 for analytical capacity; the rating scale ranging from 1 to 6). The case study on integrated pest management (and accompanying field trip) received the highest ratings (4.57 for presentation, 4.95 for insights and 5.52 for analytical capacity).

2

EVALUATION DESIGN AND OBJECTIVES

Evaluation Design and Objectives

This tracer evaluation seeks to identify the outcomes of the two TOT workshops on trainer-participants’ learning and teaching skills. The analysis takes into consideration both workshops because of similar format and low number of trainer-participants who responded to the tracer evaluation.

The main objectives of the workshops were:

  • to provide training in the application of the new institutional economics to agricultural policy and institutional reform for sustainable development in Sub-Saharan Africa;
  • to provide training in facilitating more effective group and team work;
  • to receive feedback from workshop trainer-participants with respect to EDI’s ongoing development of training materials in agricultural policy analysis and sustainable rural development; and
  • to discuss continued networking among African universities in agricultural policy analysis and institutional reform.

The approach chosen for the evaluation is to focus on didactic and methodological aspects of the TOT workshops. The intent is to learn how university based trainers did or did not incorporate didactic materials and team work practices into their training activities. In addition, the evaluative analysis takes into consideration if partnerships and networks have taken place as a result of the workshops.

The evaluation seeks to address the following four key issues:

  • Did trainer-participants learn new information and best practices which they can use in their work?
  • Did trainer-participants use the training materials provided?
  • To what extent have their communication skills been strengthened?
  • Have inter-institutional networks among EDI’s partner universities been fostered?

A questionnaire was developed in collaboration with the managers of the program to allow trainer-participants to rate each of the key issues mentioned above. In addition, an interview was conducted with one of the task managers who organized the workshops to clarify the most controversial points. The questionnaire is comprised of five questions. The first two areclosed-ended with a 6-point rating scale, while the remaining three are semi-structured with open-ended responses.