Table of Contents
Introduction 1
Background to the Project 1
Current Status of CLP 3
Evaluation Objectives 4
Field Testing 5
Methodology 5
Results 7
Training Sessions 7
Classroom Use 8
Teacher Reflections 11
Analysis of Translations 13
Methodology 13
Results 14
IsiZulu 14
Afrikaans 15
IsiXhosa 15
Readability and User-friendliness 16
Methodology 16
Workshop Discussion 16
Conclusions and Recommendations 19
Appendices
Appendix A KwaZulu-Natal Field Testing Report
Appendix B Eastern Cape Field Testing Report
Appendix C Instruments
Acknowledgements
This report was prepared by Liora Hellmann, Paul Mphisa and Christine Randell.
The team would like to thank the CLP staff - Prof Doug Young, Julie Van Der Vlugt, Siyalo Qanya, Rosemary Wildsmith-Cromarty, Mary Gordon, Marc Schafer and Thami Tokwe for their assistance in the evaluation process.
Acronyms
CLP / Concept Literacy ProjectEC / Eastern Cape
FET / Further Education and Training
GET / General Education and Training
KZN / KwaZulu-Natal
MML / Maskew Miller Longman
SAIDE / South African Institute of Distance Education
UCMS / Understanding Concepts in Mathematics and Science
UCT / University of Cape Town
ZF / Zenex Foundation
Introduction
SAIDE has been commissioned by the Zenex Foundation to conduct a formative evaluation of the Further Education and Training (FET) phase of the Concept Literacy Project (CLP). The focus of the evaluation was on testing the validation processes utilized in the development of the CLP FET book and conducting field-testing of sample material in the classroom, concentrating on its usability.
The results of the review of the CLP validation process were reported on in Report 1: Moderation of the Validation Process. The validation process was comprehensive and involved selected authors and validators. It aimed to ensure the overall correctness of the text, the clarity of the concepts, the readability from the teacher’s perspective, and the applicability to the FET curriculum.
This is the second evaluation report. It examines the field testing of samples of the FET book, an analysis of a sample of translations from the book and provides insights and suggestions from a workshop on usability of the book with the CLP Team.
Each report deals with specific aspects of the development process and together they represent the overall evaluation report.
Background to the Project
The Concept Literacy Project commenced formally in October 2002, following foundational academic research and formal teaching of "Concept Literacy and Language Use" courses and supervision of research at UCT by Prof. Douglas N. Young.
The project is aimed at addressing the lack of an adequate mediation mechanism that empowers teachers to skillfully code-switch between the language of instruction and mother tongue in order to give learners access to key conceptual building blocks in Mathematics and Science education.
The underlying theoretical assumption on which the concept literacy project is based is that Second Language English learners are only able to access mathematical and scientific knowledge through an understanding of core concepts and the learning of area-specific language. The project defines concept literacy as reading, understanding and using the learning area specific words, terms and related language forms which are an integral part of knowledge formation in these learning areas [mathematics and science]. [1].
The key activities and outputs achieved by the project to date are the following:
· In 2002 the project undertook research and compilation of a database of African equivalents of concepts and terminology used in Mathematics and Science in the South African school curriculum;
· In 2003 and 2004 work began on the translation of identified concepts and terminology into Afrikaans, isiXhosa and isiZulu and the validation of the concepts and terminology. This was followed by the development of a manuscript containing Mathematics and Science concepts for the GET curriculum in English and the three African languages for submission for publication;
· In 2005 the project tested a pre-publication version of the Concept Literacy book in use by 30 Mathematics and Science educators in the Western Cape, Eastern Cape and KwaZulu Natal. The manuscript was submitted to Maskew Miller Longman (MML) for publication and the published book was launched in Cape Town on 24 November 2005. This book, entitled Understanding Concepts in Mathematics and Science – a multilingual teacher’s resource book in isiXhosa, Afrikaans, isiZulu and English (UCMS), is now identified as Volume 1 of UCMS.
After the publication of the first volume, the project embarked on a second phase in May 2006. Building on the foundation of the GET curriculum the project turned its attention to expanding to the FET curriculum. The main objectives [2] for this phase are:
· To increase the scope of content of the current CLP teachers’ resource book in order to cover key learning concepts contained in the new FET curriculum for Mathematics and Science. (This development is planned to result in the completion and publication of UCMS Volume 2, which covers the FET curriculum, which, at the time of publication of Volume 1, had not yet been introduced in South African schools.
· To support the Zenex Foundation teacher training programmes through the integration of the existing teachers’ resource book into Mathematics and Science programmes. In particular, the project will supply the ZF funded educator training programmes with the current resource book (Volume 1) and provide the ZF project trainers with skills to use the book in the training of teachers of Mathematics and Science through ‘train the trainer’ programmes.
This is a national project and the expectation is that the book, when published, is nationally available and approved for use as widely as possible. The intention is that both Volume 1 and Volume 2 should be available for interactive use across the GET-FET continuum. Volume 2 takes the content of Volume 1 as a given and it can (and should be) referred to in order to fill in any gaps.
Current Status of CLP
The process of developing the FET CLP book comprises several key stages and activities as illustrated in the table below. The project has made good progress and has almost completed the development of the book. At the time of writing this report only a few activities remain to be done. They are highlighted in the table.
Table 1: Overview of key stages and activities to develop the FET CLP book
Writing of units / Selecting authorsAuthors were selected on the basis of their familiarity with the FET curriculum. A description of the authors is given in Report 1.
Drafting units
The units in the book were drafted by relevant authors in these disciplines: mathematics, chemistry, physics, life sciences and geography.
Refining units
Authors received comments from and made adjustments and changes.
Validation of concepts / The validation process entailed employing subject experts checked for errors, possible misconceptions or even better ways of communicating the relevant concepts in their area/s of expertise. Essentially validators had to evaluate each unit in terms of the question: “Is this good and appropriate science/maths at FET level?”
As people have different perspectives, based on their experiences and world view, the negotiated understanding of content between the author and the validator became the ‘authority’ for the text.
Selecting validators
One of the criteria used for selection was that the validators needed to be located in Cape Town, as there would be a need for interaction between with the CLP team, the authors and the validators.
Comments from validators
The CLP team received comments and/or edits from the validators, and provided these to the relevant authors.
Translation of texts / Selected translators were required to submit comments on the ‘translatability’ of the texts.
The edited versions of the English texts were submitted to the Afrikaans, isiZulu and isiXhosa consultants for translation.
The units will be given to new consultants to ‘back’ translate’ into English to check their accuracy.
Editing of texts / An intensive edit of the English versions of the text was carried out.
All translations were edited.
Final editing must still be done.
Final layout and proofreading / The manuscript will be submitted to Maskew Miller Longman in late December or mid January 2007. MML will be responsible for Desktop publishing, including graphics. The manuscript will then be sent back to the CLP team for final proofreading in 2008 (Proofreading will be the sole responsibility of CLP). The book will then be ready for publication mid year 2008 and should be available to schools in 2009.
Evaluation Objectives
The main objectives of the evaluation are centred on the development of the FET Concept Literacy Book and field-testing of the materials, namely:
· To ensure that the CLP processes used for the accurate development of the book have been successful; and
· To provide information on usability to feed back into the project.
The issues to be addressed in the evaluation, as per the scope of work, are:
· Development of the FET Concept Literacy Book
o Moderation of the CLP project validation processes for ensuring the correctness and accuracy of the original English concepts and terminology use in the book and the relevance of the concepts and terminology to the new FET curriculum.
o Correctness and accuracy of a sample of the translations of concepts and terminology into isiXhosa, isiZulu and Afrikaans.
o The prospective design of the FET Concept Literacy Book in terms of its readability and user-friendliness.
· Field-testing of the FET materials
o The usability of a selection of the materials for a sample of teachers in classrooms.
o The different kinds and levels of usage of the book (e.g. where training has been provided and where no training has been provided).
Field Testing
The field testing focused on the usability of the CLP FET book. The aim was to observe the ways in which a sample of teachers used a selection of draft units from the book in the classroom. Another objective was to ascertain whether mediation in the form of training and/or guidelines would be required in order for teachers to successfully use the resource book for teaching and learning in the classroom.
Methodology
Prof. D Young and the CLP team were consulted in January to finalise the strategy to be followed in the field testing of selected FET units. The sampling strategy agreed on was to select and involve five teachers (Grade 10 or 11 teaching biology, geography, maths or science) from three schools in a province. The teachers from two schools (i.e. ten teachers) would be provided with training, whilst the teachers from one school (i.e. five teachers) would not receive training. It was imperative that none of the teachers had had exposure to the CLP GET book.
The sample was drawn from two provinces. Rosemary Wildsmith-Cromarty and Mary Gordon from the University of KwaZulu-Natal selected the teachers in KwaZulu-Natal and Marc Schafer and Thami Tokwe from Rhodes University selected the teachers in Port Alfred and Grahamstown in the Eastern Cape. The sample thus included 30 teachers in total of which 20 would receive training and 10 would not.
The planned and actual sample for both provinces for the training is shown in the table below.
Table 2: Planned and actual sample for training
Training / Planned / ActualKZN / 10 teachers from two schools / Five teachers from two schools (one maths, two biology and two science)
EC / 10 teachers from two schools / 11 teachers from three schools (five maths, five science, one geography, one combined)
The training took place in Port Alfred in the Eastern Cape on 11 September 2007 and in Pietermaritzburg in KwaZulu-Natal on 12 September 2007. The CLP team compiled a training manual, and conducted the three hour training session. SAIDE observed the training, and teachers were requested to complete a short questionnaire regarding their training experience.
Following the training, a SAIDE consultant was appointed to conduct the classroom observations. The consultant was Paul Mphisa who is a researcher at SAIDE. Paul specialises in Community and Adult Literacy work and has had extensive experience with field testing. He is a graduate of English with Applied Linguistics as one of his subjects. Paul can speak nine of the eleven official languages including Xhosa and Zulu and is a former high school educator.
The methodology for the school visits included an observation of the classroom session, an interview with the teacher after the observation, and a focus group with all the teachers (Refer to Appendix C). The consultant was given adequate training in the methodology adopted in this project for the observation of lessons during site visits to ensure that the findings would be comparable.
Three schools were visited in KwaZulu Natal on 3-5October 2007 and three schools were visited in the Eastern Cape on 10-12 October 2007.
The planned and actual sample for the school visits is presented below:
Table 3: Planned and actual sample for school visits (observations and focus groups)
Observation / Planned / Actual classes observed and teachers interviewed /KZN Training received / 6 teachers from two schools (Grade 10 or 11, range of subjects) / Two classes / Siyanda: One maths and one science class
5 teachers in focus group
(1 science, 1 geography, 3 maths)
Three classes / Nsikayethu: Three maths classes observed (two received training)
5 teachers in focus group
(3 science, 2 maths)
KZN
No training
received / 3 teachers from one school (Grade 10 or 11, range of subjects) / One class / Siyahlomula: One maths class
3 teachers in focus group
(2 maths, 1 biology/maths)
E Cape
Training
received / 6 teachers from two schools (Grade 10 or 11, range of subjects) / One class / Nomzamo: One biology class
3 teachers in focus group
(2 maths, 1 biology)
Two classes / Kuyasa: Two science classes
2 teachers interviewed
(Science)
E Cape
No training
received / 3 teachers from one school (Grade 10 or 11, range of subjects / None / Ntiska: No observation
1 teacher interviewed
Nine Classes / 19 teachers participated in focus group interviews
Results
Training Sessions
The training sessions began with a short welcome and introduction of the participants. The objectives of the workshop were clarified. The workshop proceeded with an introduction to the book, the structure and development of the book, the nature of a concept, and language matters. Emphasis was given to the use of structured code-switching in teaching and learning. The training session concluded with a group planning session, where teachers discussed how they could use the units to plan a lesson in the classroom. Table 4 gives an overview of the workshop programme.