INSETRom PROJECT

TEACHER TRAINING EVALUATION REPORT

Barry van Driel, International Association of Intercultural Education, the Netherlands

Dragana Nikolajevic, International Association of Intercultural Education, the Netherlands

Fokion Georgiadis, International Association of Intercultural Education, the Netherlands

Executive Summary

TheINSETRom teacher training evaluation suggests that, on the whole, the project was successful in sensitizing the teachers from seven of the eight participating European countries about the Roma issues that they did know enough about before the training. Especially their sense of confidence to teach about the Roma, to teach their Roma students and work with Roma parents received a boost. Some concerns remain, however, about the ability of teachers to develop new approaches to their teaching, and to confront existing barriers. Teacher responses to the questionnaires and focus group discussions revealed their need for additional exposure and exercises that would challenge them and help them translate theory and the curriculum development methodology into useful tools that they themselves could then adapt to their own class and school environments.

According to the teachers’ responses to the pre- and post-training questionnaires, the following six specific objectives of the INSETRom Project developed, were largely met:

Objective 1:Help teachers become more aware of Roma culture, challenges and stereotypes and to redefine their role as educators that will facilitate Roma inclusion;

The evaluation suggests that the training sessions managed to sensitize the teachers about Roma issues and about the need to include Roma parents in their children’s education.

Objective 2:Assist teachers to develop their social and intercultural competencies in order to redefine their pedagogical tools for creating an intercultural school environment that will respect ethnic and cultural diversity;

The teachers did not always recognize that their role as educators also includes tailoring of their pedagogical tools to fit the needs and circumstances of their students. The teachers at all locations expressed their need for more concrete tools, materials and methods that they can directly apply in their classrooms and that would answer their day-to-day needs.

Objective 3: Improve teachers’ social and intercultural skills in order to communicate and work effectively with Roma parents;

The evaluation of the teacher training across nations suggests that it provided the teachers with the basic framework and understanding of the importance to involve and cooperate with Roma parents in order to support Roma children’s education.

Objective 4: Build a functional and efficient interface for cooperation between Roma parents and school;

The data provided to the evaluators did not provide indicators for assessment of this objective.

Objective 5: Motivate Roma parents to engage in school and become active contributors;

The evaluation tools were not designed to measure this objective.

Objective 6: Develop European partnerships.

This project has been a product of a close cooperation of teams and organizations in seven EU countries. Whereas the project teams have had the opportunities to meet and share their experiences, the actual teachers expressed the need to learn from and share their experiences with their colleagues in other countries.

In addition, the results from the teacher focus groups and post-training questionnaires suggest that working with teachers through the training modules leads to increased empathy, understanding and recognition of the values of diversity, multiperspectivity, equality and equity, inclusion, social justice, social cohesion and human rights. It leads to inclusion of both school experiences of individual teachers and the larger concerns of cultural diversity within the participating countries. Finally, the programme cultivates teachers’ positive attitudes and professional behavior.

Evaluation Report

Introduction

The problems that Roma students and their families face are very old, complex and multi-faceted. Whereas the education systems and policies in the contemporary EU member states attempt to confront a number of these challenges, it is the teachers that have a unique opportunity and role in turning social exclusion into inclusion. Teacher training programmes are seen as crucial to assist teachers in fulfilling this role. Within this framework, the Teacher-IN-SErvice-Training-for-Roma-inclusion (INSETRom) programme (134018-LLP-1-2007-1-CY-COMENIUS-CMP) aimed to assist school and Roma community partnerships through an innovative teacher training concept in order to establish an environment of collaboration and shared goals for Roma children’s education. The project was initiated in 2007 by co-operating institutions in eight European countries, namely, Austria, Cyprus, Greece, Italy, Romania, the Slovak Republic, the Netherlands and the U.K. in order to provide for development of adjustable outputs that enable implementation in different educational settings and to allow for good practices to be disseminated across Europe.

More specifically, the INSETRom project aimed to bridge the gap between Roma communities and school communities, as well as address the stereotypes that feed into this gap.The project focused not just on interventions involving Roma communities but also identified primary and secondary school teachers as the principal agents who are in a position to change educational outcomes for Roma students. Its major component aimed at training teachers:

-to adjust their perceptions, approaches and methodologies according to the needs and perspectives of multicultural societies to that they can improve their effectiveness in approaching Roma parents and involving them in the school life of their children;

-to improve their intercultural, socio-psychological and educational skills in order to enhance teachers’ awareness of Roma culture, which can in turn help them to better engage with Roma parents to become active agents in their children’s education.

The INSETRom Project developed the following six specific objectives, in addition to the previously mentioned general objectives.

  1. Help teachers become more aware of Roma culture, challenges and stereotypes and to redefine their role as educators that can facilitate Roma inclusion.
  2. Assist teachers to develop their social and intercultural competencies in order to redefine their pedagogical tool for he intercultural school environment that will respect ethnic and cultural diversity;
  3. Improve teachers’ social and intercultural skills in order to communicate and work effectively with Roma parents;
  4. Build functional and efficient interface for cooperation between Roma parents and school;
  5. Motivate Roma parents to engage in school and become active contributors;
  6. Develop European partnerships.

Some of the key strengths of INSETRom programme were:

(i) thoughtful analysis of teachers’ multiple identities to connect insight, experiences and visions of educating Roma students.

(ii) a comparative analysis and framework of ‘perceptions’ of multicultural and intercultural education among the participating countries in terms of Roma students’ education.

(iii) an apparent disconnect between ‘European’ educational policy, national educational policies and teacher pedagogy within the realm of interculturalism.

INSETRom Teacher Training Evaluation: IAIE Scope of Work

From the proposal:

“The IAIE will be responsible for the overall evaluation of the teacher training as well as the implementation of the interventions at schools (WP5-WP7), using its network of professional project evaluators. The overall evaluation will take place by two persons from the IAIE: one person who is involved in the project and one who is not involved in the project except for the evaluation component. The evaluation of the training modules will take place through distribution of research tools for evaluation to training participants and teachers through the partner institutions (who will translate the instruments and distribute the questionnaires). This will result to a short descriptive report based on (pre and post training) questionnaires given to teachers, which will be issued by IAIE regarding the teacher training in each partner’s country (January 2009) and a common report on the evaluation of the training and the evaluation of the implementation of the intervention (June 2009).”

IAIE Evaluation Outputs:

The IAIE evaluators created the following tools that were subsequently adapted for each country, translated and disseminated to the teachers:

-Pre-training questionnaire for teachers (Appendix A)

-Guidelines for teachers’ log books (Appendix B)

-Post-training questionnaire for teachers (Appendix C)

-Suggested questions for focus groups (Appendix D)

Due to the questionnaire adaptation in some countries and the variations in the quality of summaries/data provided[1], this independent evaluation is limited in scope and looks at the pre-training and post-training data in the aggregate for the project, not by country. It will offer per country or per question analysis only for the segments where data was available.

This independent evaluation is primarily based onteacher feedback given on the pre- and post-training questionnaires and on the input provided through focus groups. Teachers were recommended to write log-books as self-assessment tools. However, they were not expected to share them or submit them for project evaluation purposes.

Training of teachers:

Training Modules:

A total of 67 training modules were delivered during the Spring of 2009 to over 165 attendees at eight locations in seven participating countries (note that Italy participated with two locations: Turin and Florence). The Netherlands did not participate in the implementation phase, but was responsible for the project evaluation through the IAIE.

The specific training modules used in the various countries were developed by local trainers and the academics participating in the project, but closely followed the standard modules developed by the project team as a whole. All 67 modules delivered could be grouped into one of the following general themes:

  1. Culture and enculturation
  2. Roma history, culture, language, traditions
  3. Stereotypes and prejudices (delivered in all countries except Cyprus)
  4. Teacher-parent relations/communication (delivered at all locations except Cyprus)
  5. Intercultural/multicultural education, multicultural, multi-linguistic schools
  6. Curriculum design and adaptation (training delivered in Austria, Cyprus, Romania and the United Kingdom)

The teacher training modules did not only vary in the number of modules delivered at each location, but they were also delivered over different lengths of time and with different frequencies of trainings. One significant variation is related to the composition of attendees of these trainings. Whereas all the modules but those delivered in Florence were taught to mixed groups of teachers, teaching different subject areas, the training delivered in Florence divided the attendees into three groups (mathematics-scientific, linguistic-literary and historical-geographical) and designed the training modules to specifically target the needs of each of these three subject groups. If and when the data become available, it would be interesting to compare the outcomes of the approach used in Florence (24% of all the attendees of INSETRom teacher trainings) with the outcomes of the approaches in other locations combined.

Teachers:

Whereas the above-mentioned number of 165 attendees (the large majority women) represents the core number of teachers that went through the INSETRom teacher training program, we understand that the total number of attendees that attended the trainings was larger at some locations and for certain modules. This evaluation report will focus only on this core number of attendees at all eight locations, out of which 4% work in school management, 52% teach 5-10 year old pupils and 44% teach pupils older than 10 years of age. Only 2 % of all teachers were Roma (attending teacher training modules in Austria, Romania and Slovakia).

The average duration of teaching experience of the teachers who attended the training modules varies across the eight locations, with the group of teachers from Slovakia having the shortest teaching experience (11.67 years, and a range of 2-31 years in the teaching profession), and teachers from Austria having the longest experience (average of 23 years, with a range between 7 and 31 years working as teachers).

Almost all of the attendees teach Roma children. The percentage of pupils in their classes that are Roma varies greatly: from less than 5% (UK) to at least 25% and up to 100% (the case of teachers in Slovakia)

Teachers’ pre-training assessment

1. “How well trained do you feel you were in the past …?”

Teachers’ self-assessment of the quality of training they previously had to teach Roma and about Roma

  1. When asked about how well trained in the past they were to teach Roma children, only one teacher out of the 147 that answered the question, reported that he/she was well trained. The majority of teachers reported that they were not well trained (49%) or not at all trained (33%). The remaining percentage (17% of teachers who answered this question) reported that they were “somewhat well trained”.

% of teachers / Response
33% / Not at all well
49% / Not well
17% / Somewhat well
1% / Very well
  1. When asked how well trained they felt in the past to teach about Roma issues (history, culture etc.), 120 teachers who answered this question responded in the following way:

% of teachers / Response
47% / Not at all well
29% / Not well
22% / Somewhat well
2% / Very well

2. Teachers’ assessment of their teaching materials regarding…

 Most frequent challenges that teachers experience when teaching Roma are: attendance, Roma children’s attitude, their attention span and preparation for school, parent participation in school life of their children, socialization and overcoming stereotypes and prejudices, integration of Roma pupils into society. These answers and frequencies with which the above challenges were listed by teachers were quite uniform across the countries participating in this project.

 Teachers reported that their greatest challenges with teaching about Roma could be found in: thelack of adequate materials and in their concern that as a result of poor training and knowledge they would end up dispelling prejudice; some also noticed the issue of self-identity of Roma children.

 When asked what kind of training would help them teach Roma students many expressed interest in learning about Roma history and culture, but they also listed communication and cooperation with Roma parents, improving curricula and teaching methods, as well as finding ways to motivate Roma children.

 When asked what would help them teach about Roma culture and history, teachers listed teaching materials on Roma culture and history, expert literature, magazinesand films as their top needs.

  1. “How would you describe the relationship between Roma and other children in your classroom?”

  1. “How would you describe your relationship with the Roma-pupils’ parents?”

Teachers’ self assessment: pre and post-training

Measured on a scale of 1-4, Pre- and Post-training questions about teachers’ confidence to teach Roma children, to teach about Roma issues and to address stereotypes in the classroom indicate that on average the teachers’ confidence increased by approximately 1 point, with “1” indicating the highest level of confidence, and “4” the lowest.

Before the training, the average aggregate score of teachers’ level of confidence to teach Roma children was 2.71, and after the training the average score was 1.68. Similarly, their confidence improved for teaching about Roma issues (from 2.87 before the training, to 1.76), and when asked about handling prejudices and stereotypes in the classroom, the score went from 2.59 to 1.52.

The three graphs below illustrate the percentage of teachers that evaluated their confidence about three aspects of teaching, before and after the INSETRom training:

  1. “How confident do you feel when teaching Roma Children?”

  1. “How confident do you feel teaching about Roma issues?”

  1. “How confident do you feel about addressing any existing stereotypes/prejudices towards Roma in your classroom?”

Teachers’ evaluation of the training modules and of the training overall (post-training questionnaire)

Teachers’ Expectations

As many as 48% of teachers reported that their expectations were fully met by the provided training, and another 30% reported that their expectations were met to the “average” extent. The average score that INSETRom training received on a 1-4 scale was 1.82. The submitted summaries of teacher training evaluations showed certain variations between countries, with teachers in Cyprus being the most critical and the teachers in Greece and the United Kingdom completing the training with mostly positive impressions.

The more detailed teachers’ feedback suggests that teachers expected more concrete tools, materials and examples that they could immediately use in their classroom. The teachers in Cyprus also thought that the training content was not focused enough on the Roma specifically.

Finally, teachers from several countries had expected to share experiences and practical ideas with other representatives from educational systems participating in this project. Though a Yahoo Group for teachers had been created, this was rarely used. Language problems could have been a major cause of this.

Teachers’ Overall Impressions

The average impression of the teachers from five countries (that submitted answers for evaluation) was that the INSETRom training was stimulating and interesting. Only the teachers in Cyprus considered the training to be “average”, and “not interesting at all”.

Training Elements

Teachers were also asked to provide a more detailed assessment of the following five training aspects:

Training Aspects / Teacher Ratings (average)
Content / 1.78
Clarity / 1.64
Methodology / 1.86
Duration / 1.98
Professionalism of the trainers / 1.59

The analysis of the teachers’ feedback indicates that teachers were the least satisfied with the duration of the training. In their notes the teachers wrote about their need for continuous and regular training opportunities and some teachers called for a change in the schedule – from weekends to week-days.