Questions à choix multiples – Principes généraux

General Suggestions

  1. Do not write the test in one day. Spread the work out over time. Questions demanding high-level thinking take longer to craft-professional item writers often write only 3 or 4 per day. Write one or two questions after each class, so it becomes a simple matter of assembling them into an exam. Some teachers keep a rubber-banded stack of note cards in their desk for this purpose.
  2. If students are to hand-write the letters of their chosen answers, ask them to use CAPITAL LETTERS. The handwritten, lower-case letters "a" and "d" and "c" and "e" can be difficult to distinguish when scoring.

[Top]

Writing the Stem

  1. Phrase stems as clearly as possible-confusing questions can generate wrong answers from students who do understand the material.

For example, a confusing stem like: / "According to Tuckman's model, groups develop through several stages over time. Furthermore, it contradicts Poole's activity-track model which has groups switching among several different linear sequences. Which of the following is not one of the stages identified in Tuckman's model?"
could be cleaned up to read: / "Tuckman's model of group development includes: [Select all that apply]
  1. Avoid extra language in the stem. Some think extraneous details make a question more complex. However, they most often just add to the students' reading time. This reduces the number of questions you can put on a test, therefore reducing the reliability of the test. For example, in the Tuckman question above, the information on Poole's model had nothing to do with the information sought by the question.
  2. Include any language in the stem that you would have to repeat in each answer option. For example, a stem such as "Biology is defined as the scientific study of:" keeps you from having to repeat "is the scientific study of" at the beginning of each option.

[Top]

Answer Options

  1. Avoid lifting phrases directly from text or lecture. This becomes a simple recall activity for the student. Use new language as frequently as possible.
  2. Most literature recommends writing the correct answer before writing the distracters. This makes sure you pay enough attention to formulating the one clearly correct answer.
  3. Answer options should be about the same length and parallel in grammatical structure. Too much detail or different grammatical structure can give the answer away.

For example, the specificity and grammatical structure of the first option here are dead give-aways:

The term "side effect" of a drug:

a) refers to any action of a drug in the body other than the one the doctor wanted to drug to have.
b) is the chain effect of a drug.
c) additionally benefits the drug.

  1. Limit the number of answer options. Research shows that three-choice items are about as effective as four-choice items. Four choice items are the most popular, and never give more than five alternatives.
  2. Distracters must be incorrect, but plausible. If you can, include among the distracters options that contain common errors. Students will then be motivated to listen to your explanations of why those options are incorrect.
  3. To make distracters more plausible, use words that should be familiar to students.
  4. If a recognizable key word appears in the correct answer, it should appear in some or all of the distracters as well. Don't let a verbal clue decrease the accuracy of your exam.

For example, someone with no biology background would not have to think very hard to make a correct guess on this question:

Every organism is made of cells and every cell comes from another cell. This is the:

a) Relativity Theory
b) Evolution Theory
c) Heat Theory
d) Cell Theory

  1. Help students see crucial words in the question. For example: "Which of the following is NOT an explicit norm?" Likewise, when you ask a similarly-worded question about two different things, always highlight the difference between the questions.
  2. It is often difficult to come up with 3 or 4 plausible distracters, and teachers will sometimes add some that are not plausible, or even humorous. Be careful. If it is too easy to eliminate one or two options, then the question loses much of its measurement value. If energy or time is limited and you must come up with one more distracter, consider either offering a true statement that does not answer the question and/or a jargon-ridden option that is meaningless to someone who understands the concept.
  3. Use Rarely:
  4. Extreme words like "all," "always" and "never" (generally a wrong answer).
  5. Vague words or phrases like "usually," "typically" and "may be" (generally a correct answer).
  6. "All of the above" - eliminating one distracter immediately eliminates this, too.
  7. "None of the above" - use only when the correct answer can be absolutely correct, such as in math, grammar, historical dates, geography, etc.. Do not use with negatively-stated stems, as the resulting double-negative is confusing. Studies do show that using "None of the above" does make a question more difficult, and is a better choice when the alternative is a weak distracter.

Effective Multiple Choice Questions:

Guidelines for Writing Multiple Choice Questions:

Constructing good multiple choice items requires plenty of time for writing, review, and revision. If you write a few questions after class each day when the material is fresh in your mind, the exam is more likely to reflect your teaching emphases than if you wait to write them all later. Writing questions on three-by-five index cards or in a word-processing program will allow you to re-arrange, add, or discard questions easily.

The underlying principle in constructing good multiple choice questions is simple: the questions must be asked in a way that neither rewards "test wise" students nor penalizes students whose test-taking skills are less developed.

The following guidelines will help you develop questions that measure learning rather than skill in taking tests.

Writing the Stem:

The "stem" of a multiple-choice item poses a problem or states a question. The basic rule for stem-writing is that students should be able to understand the question without reading it several times and without having to read all the options.

  1. Write the stem as a single, clearly-stated problem. Direct questions are best, but incomplete statements are sometimes necessary to avoid awkward phrasing or convoluted language.
  2. State the question as briefly as possible, avoiding wordiness and undue complexity. In higher-level questions the stem will normally be longer than in lower-level questions, but you should still be brief.
  3. State the question in positive form because students often misread negatively phrased questions. If you must write a negative stem, emphasize the negative words with underlining or all capital letters. Do not use double negatives--e.g., "Which of these is not the least important characteristic of the Soviet economy?"

Writing the Responses:

Multiple-choice questions should usually have either four or five options to make it difficult for students to guess the correct answer. The basic rules for writing responses are:

  1. Students should be able to select the right response without having to sort out complexities that have nothing to do with knowing the correct answer and
  2. They should not be able to guess the correct answer from the way the responses are written.
  3. Therefore:
  4. Write the correct answer immediately after writing the stem and make sure it is unquestionably correct. In the case of "best answer" responses, it should be the answer that authorities would agree is the best.
  5. Write the incorrect options to match the correct response in length, complexity, phrasing, and style. You can increase the believability of the incorrect options by including extraneous information and by basing the distractors on logical fallacies or common errors, but avoid using terminology that is completely unfamiliar to students.
  6. Avoid composing alternatives in which there are only microscopically fine distinctions between the answers, unless the ability to make these distinctions is a significant objective in the course.
  7. Avoid using "all of the above" or "both A & B" as responses. These options make it possible for students to guess the correct answer with only partial knowledge.
  8. Furthermore, these types of responses make it all but impossible to randomize responses.
  9. Use the option "none of the above" with extreme caution. It is only appropriate for exams in which there are absolutely correct answers, like math tests, and it should be the correct response about 25% of the time in four-option tests.
  10. Again, you cannot randomize responses when using the "none of the above" response option.
  11. Avoid giving verbal clues that give away the correct answer. These include: grammatical or syntactical errors; key words that appear only in the stem and the correct response; stating correct options in textbook language and distractors in everyday language; using absolute terms--e.g., "always, never, all," in the distractors; and using two responses that have the same meaning.

General Issues

  1. Base each question on student learning objectives, not trivial information.
  2. All questions should stand on their own.
  3. Avoid using questions that depend on knowing the answers to other questions on the test.
  4. Also, check your exam to see if information given in some items provides clues to the answers on others.
  5. Develop a "randomly generated " exam from a question pool.
  6. The question pool should contain at least three questions for each question used in the exam.
  7. So, a 15-question exam should be generated from a pool of at least 45 questions.
  8. Randomize the position of the correct responses.
  9. Placing responses in alphabetical order will usually do the job.
  10. Remember, you can not randomize responses containing the following options:
  11. All of the above.
  12. None of the above.
  13. A and B above.
  14. Etc.
C2.1 Level 1: Knowledge

At this level, one simply requires the recall of acquired knowledge. WARNING! A test at this level can easily become a "Trivial Pursuit" exercise!

Example C2.1.1

Which one of the following persons is the author of "Das Kapital"?

1. Mannheim

2. Marx

3. Weber

4. Engels

5. Michels

Note that the responses are internally consistent - they are all the names of Germans whose written work have been major contributions on social issues.

Example C2.1.2

In the area of physical science, which one of the following definitions

describes the term "polarization"?

1. The separation of electric charges by friction.

2. The ionization of atoms by high temperatures.

3. The interference of sound waves in a closed chamber.

4. The excitation of electrons by high frequency light.

5. The vibration of transverse waves in a single plane.

Simple recall of the correct definition of polarization (#5) is required. Internal consistency and plausibility are maintained in that all responses are actual physical phenomena.

Example C2.1.3

According to the microgenesis of perception concept, the threshold

of awareness consists of a hierarchy of thresholds. Which one of the

sequences shown below is correct?

1. Recognition thresholds > physiological thresholds > detection

thresholds.

2. Physiological thresholds > detection thresholds > recognition

thresholds.

3. Physiological thresholds > recognition thresholds > detection

thresholds.

4. Recognition thresholds > detection thresholds > physiological

thresholds.

In this example, nothing more is required than the recall of the order of certain pieces of related information. The correct answer is #2.

C2.2 Level 2. Comprehension

At this level, knowledge of facts, theories, procedures etc. is assumed, and one tests for understanding of this knowledge.

Example C2.2.1

Which one of the following describes what takes place in the so-called

PREPARATION stage of the creative process, as applied to the solution

of a particular problem?

1. The problem is identified and defined.

2. All available information about the

problem is collected.

3. An attempt is made to see if the proposed

solution to the problem is acceptable.

4. The person goes through some experience

leading to a general idea of how the problem

can be solved.

5. The person sets the problem aside, and gets

involved with some other unrelated activity.

In this question, the knowledge of the five stages of the creative process must be recalled (KNOWLEDGE), and one is tested for an understanding (COMPREHENSION) of the meaning of each term, in this case, "preparation".

Note that this question violates the rule that the answer and distractors should all be of about the same length. It is difficult to get around this one here, so the text is edited so that each line is about the same length.

C2.3 Level 3: Application

In order to classify a question into this group, ask yourself if prior knowledge of the background to the question is assumed to be both known and understood, and whether one is merely expected to apply this knowledge and understanding. Calculations based on known formulae are good examples of this, as shown in the example below:

Example C2.3.1

Which one of the following values approximates best to the

volume of a sphere with radius 5m?

a. 2000m³

b 1000m³

c. 500m³

d 250m³

e. 125m³

In order to answer this question, the formula 4[pi]r³ /3 must be known (recall of knowledge) and the meaning of the various symbols in the formula understood (comprehension) in order to answer this question. The correct answer is #3.

Example C2.3.2

Which one of the following memory systems does a piano-tuner

mainly use in his occupation?

1. Echoic memory.

2. Short-term memory.

3. Long-term memory.

4. Mono-auditory memory.

5. None of the above.

This is clearly a case of testing for the application of previously acquired knowledge (the various memory systems), which is also understood, as the meaning of each term must be clear before the student can decide whether it is applicable to the given situation. The correct answer is #1. Note that students may not necessarily know what a piano- tuner is or does. Watch out for cultural bias!

The next example is more difficult to classify:

Example C2.3.3

You are the sole owner and manager of a small enterprise

employing 15 workers. One of these, Alfred, (who has been

working for you for the past year and has somewhat of a

history of absenteeism), arrives late for work one Wednesday

morning, noticeably intoxicated. Which one of the following

actions is the most appropriate in the circumstances?

1. You terminate Alfred's employment on the spot, paying him

the wages still due to him.

2. You parade Alfred in front of the other workers, to teach them

all a lesson.

3. You give Alfred three weeks' wages in lieu of notice, and

sack him.

4. You wait until Alfred is sober, discuss his problem, and give

him a final written warning, should it be required.

5. You call Alfred's wife to take him home and warn her that this

must not happen again.

Note that this this question is classified as APPLICATION as in order to answer it, the relevant labour legislation should be known and understood. One could made a case for it to have a higher classification such as EVALUATION, on the grounds that one is asked to evaluate which one of the proposed actions is the best in the circumstances, or ANALYSIS, on the grounds that in order to select the most appropriate answer, one should analyse the possible outcomesof each decision. For both these levels, one would expect a greater amount of information as to Alfred's situation, the relationship between Alfred and his co-workers, union involvement in the enterprise etc., and have a more sophisticated set of distractors. Here, option #4 is clearly the best both on legal and human terms. Note that the figure of speech "on the spot" may not be understood by second- language students. Use suitable language!

C2.4 Level 4: Analysis

Example C2.4.1

"The story is told of the famous German Organic Chemist

Auguste Kékulé who was struggling with the problem of how

the six carbon atoms of benzene were linked together. He was

getting nowhere with the problem, and one day fell asleep in

front of the fireplace while he was pondering on it. He

dreamt of molecules twisting and turning around like snakes.

Suddenly, one of the snakes swallowed its own tail and

rolled around like a hoop. Kékulé woke up with a start, and

realized that his problem could be solved if the six carbon

atoms of benzene were attached to each other to form a ring.

Further work showed that this was entirely correct."

The above passage illustrates a particular phase of the

creative process. Which one is it?

1. preparation

2. incubation

3. orientation

4. illumination

5. verification

In the above example, the student is expected to know and understand the five stages of the creative process, and to apply this knowledge to an important factual example of creative thinking (the elucidation of the chemical structure of the benzene molecule). The ability to analyse the data (i.e. the given text) in terms of each of the five stages is what is being tested. The correct answer, by the way, is #4.

Example C2.4.2 (Assume the question below is asked in a philosophy test.)

Read carefully through the paragraph below, and decide which

of the options 1-5 is correct.

"The basic premise of pragmatism is that questions posed by

speculative metaphysical propositions can often be answered

by determining what the practical consequences of the

acceptance of a particular metaphysical proposition are in

this life. Practical consequences are taken as the criterion

for assessing the relevance of all statements or ideas about

truth, norm and hope."

1. The word "acceptance" should be replaced by "rejection".

2. The word "often" should be replaced by "only".

3. The word "speculative" should be replaced by "hypothetical".

4. The word "criterion" should be replaced by "measure".