Activities: Writing Questions

Activities: Writing Questions / Clicker Workshops
University of Colorado Science Education Initiative
http://colorado.edu/sei
ddddddddd

Overview

These activities give participants a chance to try writing and revising clicker questions, with feedback from peers or group leader.

Objectives

The goal is for participants to learn the features of good clicker questions, and get a chance to practice these skills where they can get formative feedback. Participants should be able to identify the cognitive level, or “depth” of a question, as well as its’ pedagogical goal.

Activities

1. Write a question based on a pedagogical goal
Participants choose a pedagogical goal from the Question Cycle handout (see below); e.g., “predict outcome.” They draft a clicker question related to their content area, including some plausible distractors, to fulfill this pedagogical goal. This activity occurs before discussing features of effective questions; this draft question is intended to serve as a “touchstone” during the rest of the workshop, as participants shop for ideas to improve it and later revise the question based on what they have learned. Then, after discussing features of good clicker questions, participants work on revising their question and share with a neighbor.
Note that this only works well in workshops focused on question-writing; otherwise instructors don’t want to take the time for this activity.
Variation #1: Instead, do this activity after a discussion of features of effective questions, and have participants then swap with a neighbor.
Variation #2: Have participants brainstorm a question in groups of 2-3, and then swap with another group to revise each others’ questions.
Variation #3: Draft a question on a topic of their choice, and then determine the pedagogical goal of that question after the fact. /

Materials

1.  Write question based on pedagogical goal: Handout, below, including Question Cycle, and Question Writing Tips. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.
2.  Write question based on learning goal: Handout, below, including Question Writing Tips. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.
3.  Revise and improve existing question: Handout, below.
4.  Rate and Swap: 5-10 mins. Handout, below, including one of Bloom’s Taxonomy handouts. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.

Time

1.  Write question based on pedagogical goal: 5-10 minutes to write, 10 mins to revise.
2.  Write question based on learning goal: 5-10 minutes to write, 10 mins to revise
3.  Revise and improve existing question: 5-10 mins
4.  Rate and Swap: 5-10 mins.
Activities: Writing Questions
ddddddddd
2. Write a question based on a content learning goal
Same as above, but participants choose a learning goal related to their content area. Here are some example learning goals in the sciences:
  Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell
  Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).
  Chemistry: Explain trends in boiling points in terms of intermolecular interactions
  Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.
  Math: Solve a system of linear equations in two variables using algebra or graphing.
3. Try to revise and improve an existing question
It can be difficult in a short workshop for participants to really engage with the process of writing and revising their own question; examining an existing question can be an easier strategy. This is challenging with a multidisciplinary audience, however. You might give each disciplinary group their own question (from the Gallery Walk, for example).
4. Rate and swap
Participants are given a handout of Bloom’s Taxonomy levels and verbs. Using the question that they wrote, they rate it on Bloom’s Taxonomy, swap with a neighbor to come to consensus on its’ Bloom’s level, and then try to write a question at a higher level of Bloom’s using the verbs on the Bloom’s Taxonomy sheet. Note: It is often difficult to identify a single Bloom’s level for a question, typically it will span two categories.
Variation: Have participants rate the Bloom’s level of a pre-written question, and attempt to “Bloomify it up.”

Write a Draft Question: Pedagogical Goal

To Do

Think about a topic you will be teaching next week (or that you recently taught). On your own, write a draft question that addresses one of the pedagogical goals from the Question Cycle. Include 3-4 plausible distractors.
This doesn’t need to be perfect – we will work on revising it later in the workshop.
If you finish, swap with your neighbor and discuss.

Notes

Write a Draft Question: Content Goal

To Do

Look at the learning goals below. On your own, write a draft question that addresses this learning goal. Include 3-4 plausible distractors.
  Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell
  Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).
  Chemistry: Explain trends in boiling points in terms of intermolecular interactions
  Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.
  Math: Solve a system of linear equations in two variables using algebra or graphing.
Then, consider the pedagogical goals from the Question Cycle: What is the pedagogical goal of this question?
This doesn’t need to be perfect – we will work on revising it later in the workshop.
If you finish, swap with your neighbor and discuss.

Notes

Rate & Swap

To Do

Use the Bloom’s Taxonomy handout (below) to rate the Bloom’s level of the question that you wrote. (You can indicate that it belongs to more than one level).
Swap your question with a neighbor. Do you agree on the Bloom’s level of your question?
Try to “Bloomify up” the level of your question – rewrite it at a higher level of Bloom’s, making use of the verbs associated with each level on the Bloom’s handout.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Tips for Writing Clicker Questions

See also the Instructor’s Guide to the Effective Use of Clickers, at http://STEMclickers.colorado.edu

  Start with existing questions where possible: See lists of question banks at http://STEMclickers.colorado.edu.

  Don’t agonize too much; after all, the perfect question doesn’t solve all problems: It’s hard to tell in advance which questions will be great, and a great question poorly facilitated can fall flat.

  Don’t make them too easy. This is a common mistake, and misleads students as to your expectations. Challenge student thinking (students prefer this!) rather than testing memorized facts. You can write multiple choice questions that test higher levels of thinking (e.g, Analyzing, Evaluating; see Bloom’s Taxonomy)!

  Use questions that emphasize reasoning or process rather than the correct answer; this is knowledge that is more generalizeable.

  Use questions that will prompt discussion. Interesting questions that students can’t answer on their own are more likely to spur productive discussion.

  Use clear wording so that students understand what they are being asked. Keep revising over time.

  Write tempting distractors using your knowledge of student difficulties. For example, look at student answers on exams or quizzes, or first give the question as an open-ended question to generate common wrong answers. Talk to other instructors who have taught the course in the past, and talk to students one-on-one in office hours.

  Use a wide variety of creative question types. You can survey your students on their experiences or beliefs, assess what students already know about a topic, ask them to reflect on their own understanding, ask students to predict an outcome, use a series of questions to break problems into parts, stimulate discussion, or use pictures or graphs in the answer choices.

  Good sources of questions:

  Questions your students ask you or that you overhear

  Common analogies you use as a teacher

  A series of connected questions to lead students through reasoning

  Interpret graphs, data, pictures, etc.

  Discussion questions where there is no one right answer

Bloom’s Taxonomy: Cognitive Levels and Verbs

(short handout #1)

Knowledge / Comprehension / Application / Analysis / Evaluation / Synthesis
know
define
memorize
list
recall
name
relate /
restate
discuss
describe
recognize
explain
identify
locate
/
translate
interpret
apply
employ
demonstrate
dramatize
practice
illustrate
operate
/
distinguish
analyze
differentiate
calculate
experiment
compare
contrast
criticize
solve
examine /
compose
plan
propose
design
assemble
construct
create
design
organize
manage /
judge
appraise
evaluate
compare
value
select
choose
assess
estimate
measure

Bloom’s Taxonomy: Cognitive Levels and Verbs

(short handout #2)

Writing Questions Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Bloom's Taxonomy “Revised”
Key Words, Model Questions, & Instructional Strategies

Bloom’s Taxonomy (1956) has stood the test of time. Recently Anderson & Krathwohl (2001) have proposed some minor changes to include the renaming and reordering of the taxonomy. This reference reflects those recommended changes.

I. REMEMBER (KNOWLEDGE)

(shallow processing: drawing out factual answers, testing recall and recognition)

Verbs for Objectives / Model Questions / Instructional Strategies
choose
describe
define
identify
label
list
locate
match
memorize
name
omit
recite
recognize
select
state
/ Who?
Where?
Which One?
What?
How?
What is the best one?
Why?
How much?
When?
What does It mean? / Highlighting
Rehearsal
Memorizing
Mnemonics

II. UNDERSTAND (COMPREHENSION)

(translating, interpreting and extrapolating)

Verbs for Objectives / Model Questions / Instructional Strategies
classify
defend
demonstrate
distinguish
explain
express
extend
give example
illustrate
indicate
interrelate
interpret
infer
judge
match
paraphrase
represent
restate
rewrite
select
show
summarize
tell
translate / State in your own words.
Which are facts?
What does this mean?
Is this the same as. . .?
Give an example.
Select the best definition.
Condense this paragraph.
What would happen if . . .?
State in one word . . .
Explain what is happening.
What part doesn't fit?
Explain what is meant.
What expectations are there?
Read the graph (table).
What are they saying?
This represents. . .
What seems to be . . .?
Is it valid that . . .?
What seems likely?
Show in a graph, table.
Which statements support . . ?
What restrictions would you add? / Key examples
Emphasize connections
Elaborate concepts
Summarize
Paraphrase
STUDENTS explain
STUDENTS state the rule
“Why does this example. . .?”
create visual representations (concept maps, outlines, flow charts organizers, analogies, pro/con grids) PRO| CON
NOTE: The faculty member can show them, but they have to do it.
Metaphors, rubrics, heuristics

III. APPLY

(Knowing when to apply; why to apply; and recognizing patterns of transfer to situations that are new, unfamiliar or have a new slant for students)

Verbs for Objectives / Model Questions / Instructional Strategies
apply
choose
dramatize
explain
generalize
judge
organize
paint
prepare
produce
select
show
sketch
solve
use / Predict what would happen if
Choose the best statements that apply
Judge the effects
What would result
Tell what would happen
Tell how, when, where, why
Tell how much change there would be
Identify the results of / Modeling
Cognitive apprenticeships
“Mindful” practice – NOT just a “routine” practice
Part and whole sequencing
Authentic situations
“Coached” practice
Case studies
Simulations
Algorithms

IV. ANALYZE (breaking down into parts, forms)

Verbs for Objectives / Model Questions / Instructional Strategies
analyze
categorize
classify
compare
differentiate
distinguish
identify
infer
point out
select
subdivide
survey / What is the function of . . .?
What's fact? Opinion?
What assumptions. . .?
What statement is relevant?
What motive is there?
Related to, extraneous to, not applicable.
What conclusions?
What does the author believe? What does the author assume?
Make a distinction.
State the point of view of . . .
What is the premise?
State the point of view of . . .
What ideas apply?
What ideas justify the conclusion?
What's the relationship between?
The least essential statements are
What's the main idea? Theme?
What inconsistencies, fallacies?
What literary form is used?
What persuasive technique?
Implicit in the statement is . . . / Models of thinking
Challenging assumptions
Retrospective analysis
Reflection through journaling
Debates
Discussions and other collaborating learning activities
Decision-making situations

V. EVALUATE (according to some set of criteria, and state why)

Verbs for Objectives / Model Questions / Instructional Strategies
appraise
judge
criticize
defend
compare / What fallacies, consistencies, inconsistencies appear?
Which is more important, moral, better, logical, valid, appropriate?
Find the errors. / Challenging assumptions
Journaling
Debates
Discussions and other collaborating learning activities
Decision-making situations

VI. CREATE (SYNTHESIS)

(combining elements into a pattern not clearly there before)

Verbs for Objectives / Model Questions / Instructional Strategies
choose
combine
compose
construct
create
design
develop
do
formulate
hypothesize
invent
make
make up
originate
organize
plan
produce
role play
tell
/ How would you test. . .?
Propose an alternative.
Solve the following.
How else would you . . .?
State a rule. / Modeling
Challenging assumptions
Reflection through journaling
Debates
Discussions and other collaborating learning activities
Design
Decision-making situations

Web References:

·  http://www.coun.uvic.ca/learn/program/hndouts/bloom.html

·  http://www.fwl.org/edtech/blooms.html

·  http://apu.edu/~bmccarty/curricula/mse592/intro/tsld006.htm

·  http://152.30.11.86/deer/Houghton/learner/think/bloomsTaxonomy.html

·  http://amath.colorado.edu/appm/courses/7400/1996Spr/bloom.html

·  http://www.stedwards.edu/cte/bloomtax.htm

·  http://quarles.unbc.edu/lsc/bloom.html