Common Framework for a Literacy Survey Project

< Replace/Add/delete logo(s)>

Common Framework for a Literacy Survey Project

Literacy Survey

Training Guide

< Insert Date>

< Insert other relevant information on cover page of Guide>

Table of Contents

Preface

How to Use the Training Guide

Classroom Training Procedures and Techniques

Supplies Required for Training

Welcome and Introductions

Client Presentation

Home Study

Objectives of the Training Session

Objectives of the <Country Name> Literacy Survey

Overview of <Country Name> Literacy Survey

Survey Flow

Survey Sections

Material and Supplies

Step-by-Step Instructions for the Administration of the Assessment Booklets for Interviewers

Interview Steps

Exercises

In Class Exercises

Mock Interviews - Introduction

Mock Interview #1

Mock Interview #2

Mock Interview #3

Mock Interview #4

After the Interview

Conclusion and Evaluation

Appendix A: Workshop Powerpoint Presentation

Appendix B: Interviewer’s Manual

Appendix C: Supervisor’s Manual

Appendix D: Review Exercise (without answers)

Appendix E: Review Exercise (with answers)

Appendix F: Exercise to Minimize Non-Responses (without answers)

Appendix G: Exercise to Minimize Non-Responses (with answers)

Appendix H: Interviewers’ Debriefing Questionnaire

Preface

The project “Common Framework for a Literacy Survey” was executed by the Caribbean Community (CARICOM) Secretariat under funding provided by the Inter-American Development Bank (IDB) Regional Public Goods Facility and was intended to design a common approach to the measurement of literacy in countries. This common framework is built upon international methodologies and fundamentally the International Survey of Reading Skills (ISRS) that enable reliable measurement of literacy than what presently exists in the Region.

The literacy assessment is designed to measure functional literacy. In other words, it will determine an individual’s literacy level by employing a series of questions designed to demonstrate the use of their literacy skills. This involves two steps – the objective testing of an adult’s skill level and the application of a proficiency standard that defines the level of mastery achieved. The assessment measures the proficiency of respondents on three continuous literacy scales - prose, document and numeracy. In addition it will collect information on reading component skills. Component skills are thought to be the building blocks upon which the emergence of reading fluency is based. Information on the reading component skills will be collected from people at the lower end of the literacy scale only. The testing phase is preceded by a selection phase which includes the administering of a Background or Household questionnaire and post the selection of the respondent from the specific household an initial pre-assessment is undertaken through a filter test booklet to determine what type of assessment should be undertaken in the testing phase.

A consultant, Mr. Scott Murray of Canada was hired to undertake the provision of services on this project. The CARICOM Secretariat (including Regional Statistics and Human and Social Development Directorate) and the CARICOM Advisory Group on Statistics (AGS) were instrumental in the execution of the project throughout all phases. In addition, there was participation by Member States and some Associate Members relative to the technical rollout of the instruments and documents.

This training Guide is aimed at providing <country undertaking a Literacy Survey> with guidelines that can enable the effective training of interviewers and other field staff relative to the conduct of a literacy survey as recommended under the IDB-funded CARICOM project.

The Secretariat is indebted to the Consultant, Scott Murray, for his inputs and guidance as well as to the AGS and representatives from MS/AMS

How to Use the Training Guide

General Instructions

This Training Guide will help Trainers train Interviewers to conduct PAPI (Paper and Pencil Interviews) for the <Country Name> Literacy Survey.

This Guide provides verbatim text to be used when conducting training for Interviewers. In order to ensure that the context and format are understood, Trainers must review this Guide in advance of the training. This Guide must be followed closely to ensure that ALL the survey materials are covered.

Mock Interview Instructions

For the Mock Interviews

Trainees are required to answer the questionnaires based on the following:
Summary profile of hypothetical households and
the information relative to the questionnaires, booklets and exercises.

Two trainees will volunteer to act out the interview- One as the respondent, the other as the interviewer.

The trainees will read aloud the questions in the survey instruments, i.e., questionnaire, booklet, exercise. These are printed in bold.

The answer that the trainer will give are printed in regular text and preceded by a R:

Any relevant comments (information) the trainer must give will be in regular text.

Refer to the entries highlighted in red and enclosed in <brackets> ONLY if the trainees are experiencing a problem answering the questions.

Try not to offer more information in the mock interview than provided as this may result in inconsistencies or problems in the mock interview.

Classroom Training Procedures and Techniques

Instructions

Before starting your training session, meet with your co-trainers and observers (if any) to explain what you expect them to do. Make sure they know:

How you will proceed during the training;
How much involvement you want on their part;
Who you want to help the interviewers during the mock interviews; and
How you want the interviewers to be helped.

Take a few minutes before the training to prepare your material.
If the trainee asks a question on a topic that will be covered later in the training, provide a brief answer and state that you will be covering the subject later. Avoid lengthy discussions, since they can cause confusion.
Be approachable. Ensure that the trainees feel they can come to you for assistance.
If a trainee asks a question you cannot answer, ask one of the co-trainers or observers to provide the answers if you think they might know it. Otherwise, take a note of the question and provide an answer as soon as possible.
Provide encouragement and keep the training session positive. Positive reinforcement is the best teaching method. Notice and comment on effort and progress. Be aware that the volume of information that trainees are being asked to absorb can be overwhelming.
Provide stretch breaks if you feel they are needed.
Be relaxed in your delivery. When possible, don’t hesitate to get up and walk around while instructing. Take several pages of the text with you. A change of pace helps the learning process.
If the mock interview is very long, ask various trainees to take turns role playing.
If you feel that trainees have not fully understood a part of the interview use another practice case, if time permits, to demonstrate areas of weaknesses.

Supplies Required for Training

Before you begin your training session, make sure you have the following supplies.

For the trainer

PC viewer with adapter and connector
Laptop computer loaded with PPT presentations
Overhead projector

Training Agenda
Training Evaluation
Training Guide + Mock Interviews
Interviewer Home Study Package – Interviewer’s Manual, Review Questions (with answers) and Exercise to Minimize Non-response (with answers)

Administration Guide for Interviewers
Assignment Control List (ACL)
Background Questionnaire (BQ) Response Card
Carrying Bag – 2B pencils, erasers, pencil sharpeners, tape recorder, cassettes, labels for cassettes, timer
Interviewer’s Debriefing Questionnaire
Interviewer’s Manual
Introductory Letter
Notice of Visit
Numeracy Toolkit – calculator, photo, ruler
Materials Return List
PPVT Notebook – Shortened Version
Procedure’s Manual
Reading Exercise Cards (6)
Reference Card
Respondent Envelope containing:
Background Questionnaire (BQ)
Change of Address Form
Digit Span Forward – Interviewer’s Recording Sheet
Digit Span Backward – Interviewer’s Recording Sheet
Filter Booklet
Filter Scoring Sheet
Interviewer Observation Note
Locator and Main Booklet
Non-Interview Report
PPVT – Interviewer’s Recording Sheet
Respondent Envelope Checklist
Transmittal Form

For each trainee

Training Agenda
Training Evaluation
Name Card
Writing Pad/Post-it Pad
Pencil

Each trainee should also bring

Administration Guide for Interviewers
BQ Response Card
Home Study Package
Exercise to Minimize Non-Response (completed)
Interviewer’s Manual
Review Questions (completed)
Interviewer’s Debriefing Questionnaire
Introductory Letter
Notice of Visit
Materials Return Form
PPVT Notebook – Shortened Version
Reading Exercise Cards (6)
Reference Card
Respondent Envelope containing:
Background Questionnaire (BQ)
Change of Address
Digit Span Forward – Interviewer’s Recording Sheet
Digit Span Backward – Interviewer’s Recording Sheet
Filter Booklet
Filter Scoring Sheet
Interview Observation Note
Locator and Main Booklet
Non-Interview Report
PPVT – Interviewer’s Recording Sheet
Respondent Envelope Checklist
Spelling Answer Sheet
Transmittal Form

Welcome and Introductions

Good morning and welcome to the Literacy Survey Training. Over the next 5 days we will be training you to conduct the Literacy Survey.

Introduce yourself. Explain your role within the project team.

Let your co-trainer or observers introduce themselves.

Let the trainees give their name, where they are from, and if they worked on other surveys.

You can invent your own icebreaker activity. The activity should not last longer than 10 minutes.

Tell us what you enjoy reading, where you find your reading material, and why you enjoy reading that specific material, i.e. I enjoy reading the home section of the newspaper because I am always looking for a new home.

Administration

Before we get started, I’d like to mention some administration things.

Describe the location of washrooms, cafeteria and emergency exits.

Schedule

We will do our best to follow the following schedule

Start - 9:00

Break - 10:00

Lunch - 12:00

End - 4:00

General

All the information that we cover during training can be found in the Interviewer’s Manual and the Administration Guide for Interviewers.

Comments/ Questions

Are there any questions before we get started?

Training Agenda

See Appendix A for powerpoint presentation

Day 1 (Slide 2)

Introduction/Presentations/ Ice Breaker
Objectives of the Training Session
Objectives of the Literacy Survey
Survey Overview
Survey Flow
Survey Materials and Supplies
Interview Steps
Review Exercise

Day 2 (Slide 3)

Review of Day 1 Training
Mock Interviews
Contact
Background Questionnaire (BQ)
Filter and Main Booklets
Exercise to Minimize Non-Response
Review Exercise
Scoring Exercise

Day 3 (Slide4)

Review of Day 2 Training
Mock Interview
Contact
Background Questionnaire (BQ)
Filter + Locator Booklets
Reading Exercises

Exercise to Minimize Non-Response
Review Exercise
Scoring Exercise

Day 4 (Slide 5)

Review of Day 3 Training
Mock Interview
Contact
Background Questionnaire (BQ)
Filter and Locator Booklet
Reading Exercises

Exercise to Minimize Non-Response
Review Exercise
Scoring Exercise

Day 5 (Slide 6)

Review of Day 4 Training
Mock Interview
Contact
Background Questionnaire (BQ)
Filter and Locator Booklet
Reading Exercises

Exercise to Minimize Non-Response

Review Exercise
Scoring Exercise
After the Interview
Interviewer’s Debriefing Questionnaire
Conclusion and Evaluation
Administration

Question/ Comments

Are there any questions about the agenda?

Client Presentation

/ Introductions - Survey manager, clients, guest speakers, visitors.

Question/

Comments

/ Are there any questions for any of the presenters?
Thank the speaker.

Home Study

Home Study Package

Are there any questions or comments regarding the Home Study Package?

Interviewer’s Manual
Review Exercise
Exercise to Minimize Non-response

Set-up

I would like you to take out all the material you received as part of your home study, now. We will be referring to these documents throughout the training today.

Objectives of the Training Session

Objectives of Training Session

PPT 7 /

To understand the <Country Name> Literacy Survey

M … Why are we conducting the survey?
To gain information on the distribution of literacy skills and areas where interventions in literacy education would be most useful.
M … Who are we interviewing?
Usual residents aged 16 to 65.
M … Where are we conducting this survey?
Several communities – we hope to collect a good cross section of individuals in terms of age, gender and education.
M … When are we conducting the survey?
<Collection Start Date> to <Collection End Date>
M … How are we conducting the survey – what method?
Paper and Pencil Interview (PAPI) – personal interview.

To become familiar with the survey’s definitions and concepts

M … What is Literacy?
The information processing skills necessary to use printed material commonly encountered at work, at home and in the community.
M … Why do we collect Background Information from the respondent?
The Background Questionnairewasdesigned to collect data on the factors thought to be associated with reading skills.

Objectives of Training Session, continued

PPT 7 /

To become familiar with the sections of the survey

M … What are the sections of the survey?
Contact/Entry, Household Questionnaire, Background Questionnaire (BQ), Filter Booklet, Locator or Main Booklet, Reading Exercises, Exit.
M … Do we administer the Reading Exercises to all respondents?
No – only respondents who have scored 05 or less, and 06 to 15 in the Filter Booklet are administered the Reading Exercises.

To learn how to conduct interviews

M … How do we make initial contact with the respondent?
Personal interview
M … What are some of the steps in conducting an interview?
Administering booklets, scoring answers, documenting results

To learn how to minimize non-response

M … How do we minimize non-response

Know the survey

Review the Exercise to Minimize Non-Response for questions and answers, and

Review the Interviewer’s Manual – How to React to Difficult Situations

Our goal is to provide you with the information, materials, and training you will need to successfully conduct interviews.

Question/ Comments

/ Is there anything else that you would like to get out of this training session?

Objectives of the <Country Name> Literacy Survey

Objectives of <Country Name>Literacy Survey,

PPT 8, 9 / The objectives of the <Country Name>Literacy Survey are to:

Help the Government of <Country Name> to objectively diagnose the different levels of literacy among adults aged 16 to 65;

Find practical solutions and suggestions to reduce illiteracy in all it’s forms and eventually to eliminate it completely;

Document how skill influences labour market, health, educational and social success;

Profile the skills of youth aged 16 to 25. This will shed light on the quality of recent school leavers both graduates and leavers;

Provide literacy profiles for small areas for program planning and delivery;

Raise the levels of literacy of the population in order to increase the social, health, economic and living levels and to eliminate illiteracy in all its forms.

In Class Exercise 1

Paraphrase in your own words the objective of the <Country Name> Literacy Survey.
Refer to the <Country Name> Literacy Survey objectives in the Interviewer’s Manual and any other material you may have that may be helpful.
Try to keep your answer as brief as possible
Time – 10 minutes
Ask each trainee to answer aloud.

Comments/Questions

/ Are there any questions about the objectives of the survey?

Overview of <Country Name> Literacy Survey

Partnership

PPT 10 / M … Who are the partners in this survey ?

<Partners>

Use

PPT 11 / M … What are the primary uses of the data?

to assist in developing skill enhancement policies and programs, and

to identify those individuals at risk because of low literacy skills so that programmes can better targeted to improving their skills.

Users

PPT 12 / M … Who are the primary users of the data?

policymakers

to plan national programs aimed at improving literacy skills (including identifying best practices) through providing them with information about the distribution of literacy and numeracy skills which can then assist in developing skill enhancement policies and programmes;

local/community level officials

to enhance adult education and other literacy programs;

business and educational organizations

to better understand the skills of the labour force and plan programs to address skill gaps;

Social/welfare and related agencies, civil society

to assist in identifying those individuals/households at risk because of low literacy skills so that programs can be better targeted to improving their skills;

researchers and other users

to inform the public about similarities and differences of literacy of the adult populations across countries.

Target Population

PPT 13

Sample Distribution

PPT 14 / M … Who is the target population?
The target population comprises usual residents, aged 16 to 65, who are living in private housing units at the time of the survey.
M … Who is excluded?

Residents of Institutions - prisons, hospitals, residents of senior homes and hospices, etc.

Response Rate/Sample

PPT 15 / M … What is the response rate?

85% complete cases

M … Where does the sample come from?
Sample Frame to be included here

Collection Method

PPT 15 / M … What is the method of data collection of <Country Name> Literacy Survey?

Paper and Pencil Interviewing (PAPI) method.

M … How do we make initial contact?

Initial contact will be attempted by personal visit.

M … Do we allow proxy interviews?

No. The Filter, Locator, Main Booklet and Reading Exercises can not be completed with the help of another household member.

M … Can an interpreter be used for this survey?

No. The Filter, Locator, Main Booklet and Reading Exercises can not be completed with the help of an interpreter.

Continued on next page

Collection Period