Me? Write an exam?
How to write a communicative exam

JoAnn Miller, Editorial Macmillan

Balancing a Exam

  • Grammar? Vocabulary? Function?
  • Accuracy or fluency?
  • Production or recognition?
  • Subjective or objective?






1

Grading Communicative Exams Communicatively

JoAnn Miller, Editorial Macmillan,

Exam Writers’ Worksheet Example

JoAnn Miller, Macmillan Publishers,

Course: Lengua Adicional 1 Exam: 1st parcial Book: Skyline 5 .

Units covered: 1 to 2 Pages in book: p. 6 to p. 21 Total number of points: 15 .

Be sure to have the book and content analysis in front of you before you begin writing.

Sec. / Contents (subject) / F/G/V/ / Pages in bk / Format / # of Points / Each / Items
Events in past (2), famous people (4), conferences (2) (Class) / F / 8-13 / Complete the conversation / 5 / 1 / 5
Connectors (3), whoever (2) (Conversation) / G / 9, 17 / Box / 5 / 1 / 5
Two word verb collocation (conversation) / V / 18-19 / Two boxes / 5 / 0.5 / 10
Total: / 15

F: 5 G: 5 V: 5 = 15 .

Written Communicative Exams

  • Reflect course content
  • Test both accuracy and fluency
  • Include both production and recognition sections
  • Test both functions and grammar (as well as reading, writing, listening)

Fluency

  • The ability to produce language with ease
  • Have a good but not necessarily perfect command of vocabulary and grammar
  • Communicate ideas effectively
  • Produce language without causing comprehension difficulties or a breakdown in communication

Accuracy

  • Ability to produce grammatically correct sentences
  • May not include the ability to write fluently.

Recognition:

  • Student chooses the correct answer
  • Multiple choice, true-false, choose from a box
  • Requires less knowledge and analysis

Production:

  • Student has to write sentences or phrases
  • Complete the conversation
  • Requires a lot of thought

What is a function?

  • The communicative purpose of the users of the language
  • Usually expressed as gerunds: introducing, apologizing, asking directions, requesting
  • Are much more communicative and relate to fluency more than accuracy
  • If you get the function right, even if you make a grammar mistake, you can be understood.

Functional Sections

  • Complete the conversation
  • Complete the conversation. Use the words in parentheses.
  • Circle the correct answer to complete the conversation.
  • Write the conversation in the correct order.
  • Write sentences from the box.

Production vs. Recognition:
Grammar and Functions

Recognition

worth fewer points—less work

  • fill in (verb, adjective form)
  • write words from the box
  • multiple choice

Production

worth more points—more work

  • fill in (subject / verb)
  • ordering (words or conversations)
  • ordering (plus one)
  • cued sentences
  • complete the conversation

Correcting

Grammar, Reading, Vocabulary, Listening

  • In general these sections are all right or all wrong.
  • We rarely give partial credit.
  • These sections test accuracy.

Communicative sections

  • You can give partial credit
  • These sections test fluency.
  • Ask yourself if the S’s answer communicates what the S wants to say.

Examples of partial credit

Correct answer: What’s your name?

Student writes: What you name?

Correct answer: If you invited me, I’d go.

Student writes: If you invite me, I go.

Correct answer: I went to the movies yesterday.

Student writes: I go to the movies yesterday. / I go to the movies.

1

Grading Communicative Exams Communicatively

JoAnn Miller, Editorial Macmillan,

(Licenciatura Exam, Course 1, Midterm Exam. Text: Atlas 2; Actual Student Responses)

The student sees this:

IV. The clerk knows Cleopatra. Caesar asks the clerk about Cleo. Complete the conversation. Use the words in parentheses. ( 4 points, .5 each)

Clerk:Yes, I know her.

Julius:(1) ______(work) ?

Clerk:(2) ______( palace downtown).

Julius:(3) ______( do) ?

Clerk:(4) ______( help people).

Julius:(5) ______( close friend)?

Clerk:Yes, (6) ______(funny).

Julius:(7) ______( sports)?

Clerk:Yes, (8) ______( tennis ).

Answer key. Teachers use as a guide. Not the only answers possible.

IV. Functions. ( 8 points, 1 each)

1. Where does she work? 5. Does she have a close friend?

2. (She works) at the palace downtown.6. she’s funny

3. What does she do?7. Does she like sports?

4. She helps people8. she likes/plays tennis.

Grading Communicative Exams Communicatively

JoAnn Miller, Editorial Macmillan,

Student 1: Points: / 8

Clerk:Yes, I know her.

Julius:(1) Where does she works?

Clerk:(2) She does work palace downtown.

Julius:(3) Where do she does?

Clerk:(4) She does help people.

Julius:(5) Where does she close friend?

Clerk:Yes, (6) she does funny.

Julius:(7) Where does she lift sports?

Clerk:Yes, (8) she does play tennis.

Student 2: Points: / 8

Clerk:Yes, I know her.

Julius:(1) You do?

Clerk:(2) I’m work in the palace downtown.

Julius:(3) What do you do?

Clerk:(4) I’m help help people.

Julius:(5) How you close friend?

Clerk:Yes, (6) they are funny

Julius:(7) ar you play sports?

Clerk:Yes, (8) I’m play.

Student 3: Points: / 8

Clerk:Yes, I know her.

Julius:(1) Where does she work?

Clerk:(2) She work at the palace downtown.

Julius:(3) What does she does?

Clerk:(4) She helps people.

Julius:(5) Does she have a close friend?

Clerk:Yes, (6) she does. She’s very funny..

Julius:(7) Does she like sports?

Clerk:Yes, (8) she play tennis.

Student 4: Points: / 8

Clerk:Yes, I know her.

Julius:(1) What does she works?

Clerk:(2) She works in palace downtown.

Julius:(3) What does she do?

Clerk:(4) Work with help people.

Julius:(5) Are you close friend?

Clerk:Yes, (6) she is funny.

Julius:(7) Are you practice sports?

Clerk:Yes, (8) she plays tennis.

Grading Communicative Exams Communicatively

JoAnn Miller, Editorial Macmillan,

Exam Writers’ Worksheet

Course: Exam: Book: ______.

Units covered: to Pages in book: p. to p. Total number of points: 50 .

Be sure to have the book and content analysis in front of you before you begin writing.

Sec. / Contents (subject) / F/G/V/R/LC / Pages in bk / Format / # of Points / Each / Items
Total: / 50

F: G: V: R: ___ LC: = 50 .

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

How could you improve these multiple choice items?

1. California:
a. Contains the tallest mountain in the United States
b. Has an eagle on its state flag.
c. Is the second largest state in terms of area.
*d. Was the location of the Gold Rush of 1849.
2. If the pressure of a certain amount of gas is held constant, what will happen if its volume is increased?
a. The temperature of the gas will decrease.
*b. The temperature of the gas will increase.
c. The temperature of the gas will remain the same.
3. Suppose you are a mathematics professor who wants to determine whether or not your teaching of the unit on probability has had a significant effect on your students. You decide to analyze their scores from a test they took before the instruction and their scores from another exam taken after the instruction. Which of the following t-tests is appropriate to use in this situation?
*a. Dependent samples.
b. Heterogeneous samples.
c. Homogeneous samples.
d. Independent samples.
4. The term hypothesis, as used in research, as defined as:
a. A conception or proposition formed by speculation or deduction or by abstraction and generalization from facts, explaining or relating an observed set of facts, given probability by experimental evidence or by factual or conceptual analysis but not conclusively established or accepted.
b. A statement of an order or relation of phenomena that so far as is known is invariable under the given conditions, formulated on the basis of conclusive evidence or tests and universally accepted, that has been tested and proven to conform to facts.
*c. A proposition tentatively assumed in order to draw out its logical or empirical consequences and so test its accord with facts that are known or may be determined, of such a nature as to be either proved or disproved by comparison with observed facts.
5. How long does an annual plant generally live?
*a. It dies after the first year.
b. It lives for many years.
c. It lives for more than one year.
*d. It needs to be replanted each year.
6. A word used to describe a noun is called an:
*a. Adjective.
b. Conjunction.
c. Pronoun.
d. Verb.
7. Which of the following would do the most to promote the application of nuclear discoveries
to medicine?
a. Trained radioactive therapy specialists.
*b. Developing standardized techniques for treatment of patients.
c. Do not place restrictions on the use of radioactive substances.
d. If the average doctor is trained to apply radioactive treatments.
8. You have just spent ten minutes trying to teach one of your new employees how to change a typewriter ribbon. The employee is still having a great deal of difficulty changing the ribbon, even though you have always found it simple to do. At this point, you should:
a. Tell the employee to ask an experienced employee working nearby to change the ribbon in the future.
b. Tell the employee that you never found this difficult, and ask what he or she finds difficult about it.
*c. Review each of the steps you have already explained, and determine whether the employee understands them.
d. Tell the employee that you will continue teaching him or her later, because you are becoming irritable.
9. Which of the following is the best indication of high morale in a supervisor’s unit?
a. The employees are rarely required to work overtime.
*b. The employees are willing to give first priority to attaining group objectives, subordinating any personal desires they may have.
c. The supervisor enjoys staying late to plan the next day.
d. The unit gives expensive birthday presents to each other.

Fluency (Longman Dictionary of Language Teaching and Applied Linguistics)

•The ability to produce written and / or spoken language with ease

•Speak with a good but not necessarily perfect command of intonation, vocabulary and grammar

•Communicate ideas effectively

•Produce continuous speech without causing comprehension difficulties or a breakdown in communication

Accuracy(Longman Dictionary of Language Teaching and Applied Linguistics)

•Ability to produce grammatically correct sentences

•May not include the ability to speak or write fluently.

Fluency in Tasks(Dave Willis, “Accuracy, fluency and conformity” Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 50)

“Learners need opportunities to process language for communicative purposes as receivers and producers.

“These opportunities should be unfettered by the perceived need to conform to teacher expectations in terms of the production of specific language forms.”

Accuracy in Tasks

“Whenever learners are involved in communication they are concerned with accuracy…making the best use of their language systems…

“In spontaneous communication [they] have little time to reflect on the language they produce.

“ If…they are given time to prepare what they have to produce, there will be a concern for formal accuracy…”

Speaking(Peter Skehan, “Second language acquisition research and task-based instruction”, Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 22.)

•Fluency: “concerns the learner’s capacity to produce language in real time without undue pausing or hesitation.”

•Accuracy: “how well language is produced in relation to the rule system of the target language.”

Implications(Miriam Eisenstein Ebsworth, “Accuracy Vs. Fluency: Which Comes First in ESL Instruction?”, ESL Magazine. 1:2, 24-26. March/April 1998.)

“Teachers should balance issues of fluency and accuracy depending on the specific needs of learners and the resources of time and materials for instruction.”

Oral Evaluation

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

Accuracy

•5% in oral exam

•5-7% in projects

Fluency

•5% in oral exams

•5-7% in projects

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

Oral Exam

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

Accuracy

•Role play

•5 minutes+ preparation

•Perform for teacher

•1-2 minutes maximum

Fluency

•Role play with teacher and another student

•No preparation

•Cues only

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

Oral Exam Organization

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

•Students (pairs) given role cards as they enter classroom

•Told to prepare—when finish exam, can leave

•Can use any reference or ask questions

•Come to the front of the class, talk only to teacher

•Perform

•Either stay together or separate for second role-play

1

Oral Testing of Accuracy and Fluency

JoAnn Miller, Macmillan Publishers, ,

5 point scale

Accuracy

1 ---Little or no language produced.

2 --Poor vocabulary, serious mistakes in grammar, poor pronunciation.

3 --Adequate vocabulary, mistakes in grammar, adequate pronunciation.

4 --Good vocabulary, occasional errors in grammar, good pronunciation.

5 --Wide vocabulary, very few errors in grammar, very good pronunciation.

Fluency

1 ---Little or no communication.

2 ---Very hesitant and brief utterances, sometimes difficult to understand.

3 ---Communicates ideas, but hesitantly and briefly

4 ---Effective communication, but does not elaborate on response.

5 ---Easy and efficient communication. Elaborates on responses.

Oral Exams: Grading Instructions

I. Pair Roleplay. Used to grade accuracy. (5 points)

Instructions: (1) Give each pair of SS a roleplay card. Give them at least 5 minutes to prepare their talk. (2) Call pairs of Ss to the front. They perform without written notes. (3) Grade them based on the following scale:

Little or no language produced. / 1
Poor vocabulary, serious mistakes in grammar, poor pronunciation. / 2
Adequate vocabulary, mistakes in grammar, adequate pronunciation / 3
Good vocabulary, occasional errors in grammar, good pronunciation / 4
Wide vocabulary, very few errors in grammar, very good pronunciation / 5

Note: Please take S's level into account. A Course 1 student cannot produce as much language as a Course 3 student. To get 3 points, the student should be able to use structures and vocabulary taught in the course he / she just finished. However, expect errors since the student has not fully acquired the material. To get 5 points, the student may still make a few isolated errors, but will speak much above a typical student at the same level.

II. Roleplay with the teacher. Use to grade fluency. (5 points)

Instructions: (1) Choose one of the situations given below. (2) If three people are needed in the situation, keep the pair together. If not, separate Ss and grade individually. (2) S does not see situations. (3) Explain the situation to the S and perform your part of the roleplay. (4) Grade S on the following scale:

Little or no communication. / 1
Very hesitant and brief utterances, sometimes difficult to understand. / 2
Communicates ideas, but hesitantly and briefly / 3
Effective communication, but does not elaborate on response. / 4
Easy and efficient communication. Elaborates on responses. / 5

Note: Students should be graded considering their course level.

Oral Exams: Roleplays (Teacher's copy)

I. Pair Roleplay. Used to grade accuracy. (5 points) (See separate sheet. Make enough copies for all the Ss and cut them up. Put Ss in pairs. Give each S a card.)

Student A: Imagine you are at a meeting and not in an exam. All of your classmates are at the meeting too. Your partner doesn’t know anyone. Tell him who the people are. / Student B: Imagine you are at a meeting and not in an exam. All of your classmates are at the meeting too. You don’t know anyone. Ask your partner who the people are.
Student A: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments. / Student B: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments.
Student A: You are making a survey about what people think they will be able to do with telecommunications in twenty years. Ask your partner at least three questions about the topic. / Student B: Your partner is making a survey. Answer his/her questions.

II. Roleplay with the teacher. Use to grade fluency. (5 points) (Do not show these questions to the Ss. )

1. Keep students together. Ask them what they plan to do when they finish school. Then ask them to tell you the pros and cons of that job.

2. Separate students for a moment. You are going to give one student a message for the other student. For example, ask Student A to tell Student B you are going to meet him/her after class. Then have the student pass on the message. Make the messages a little bit complicated. When you finish, give Student B messages for Student A.

3. Separate students. Tell student to imagine his girlfriend / her boyfriend is angry. Ask him / her what he / she will do. Then ask a “what if” question: What if he doesn’t believe you? What if he goes out with someone else?, etc.