Creating Effective Questionnaires on Cultural Topics

Creating Effective Questionnaires on Cultural Topics JoAnne Zoller Wagner

Creating Effective Questionnaires on Cultural Topics

presented on May 20, 2000 at the NAME Conference in Omaha, NE

by JoAnne Zoller Wagner

Cultural questionnaires help students learn about American culture from “the source,” Americans themselves. Texts such as Face to Face by Virginia Vogel Zanger (Heinle & Heinle) include questionnaires on such topics as body language, male and female roles, proverbs, and eating habits. While published questionnaires can be very helpful, most ESL teachers will find themselves needing to design their own questionnaires for one of two reasons: topic or audience.

For example, my students were studying consumerism in America. The students had just completed an analysis of mail order catalogs. I wanted them to be able to ask Americans about how they use the catalogs. Since there was no published questionnaire on mail order catalogs, I set about writing one.

My students also visit the local retirement residence once a semester. Since we had been studying male and female roles, it occurred to me that these senior citizens (many of them in their 90s) could offer my students an excellent longitudinal perspective on male and female roles in America. The questionnaire on male and female roles in Face to Face, however, was written with younger informants in mind. An original questionnaire was needed which would fit the audience—retired senior citizens.

In writing these and other questionnaires, I realized I had developed guiding principles based on my experiences, both successful and unsuccessful, with the Face to Face questionnaires. In this article, I will be referring to those experiences, as well as to the questionnaires which I have developed myself.

First, it is necessary to establish a clear purpose for the interviews. What is your overarching research question? Once you have established this, each discrete question can be selected and organized according to your research goal. The overarching research question should arise out of class discussions and readings so that the interviews can extend or deepen students’ understanding of the topic. For example, my students were interested in the following question: “How have male and female roles changed over the course of the 20th century in America?” This overarching research question guided the selection and ordering of questions for the male and female role questionnaire.

Second, consider informant variables in writing questions. This is perhaps the most important principle. Cognitive, emotional, and cultural characteristics of the informants will determine the nature of the information which you will be able to retrieve, and also the way you should go about it. For example, when interviewing senior citizens about proverbs based on a questionnaire in Face to Face, my students found that the seniors experienced memory problems in coming up with the proverbs themselves, yet they could easily recognize a proverb they knew when they heard it. The seniors were anxious to explain the proverbs to my students, but sometimes they could not go beyond the literal level. “A stitch in time saves nine” was interpreted as applying only to clothes mending, for example.

Even informants who have no memory problems and who are able to think abstractly need a little help. In interviewing Americans about their use of mail order catalogs, for example, my students found it helpful to show the informants a list of the most common catalogs, which the informants could then check in answer to the question, “Which catalogs are you familiar with?” If the same informants had had to come up with the titles on their own, my students would have been misled into thinking most Americans only know two or three catalogs, when the average number checked on the checklist was 20+, substantially changing the cultural picture they would have received.

Emotional factors revolve around the stress level of the informant and the level of trust the informant feels in speaking to an international student. Questionnaire length will vary according to the anticipated “busyness” of the informant, in order to reduce stress. Senior citizens are more willing to spend half an hour with an international student than busy college students, staff, and faculty. It is wise to proceed from the least threatening to the more threatening questions. For example, in the mail order catalog interview, “How often do you receive mail order catalogs?” is a much less threatening question than “Have you ever ordered anything from a mail order catalog? If so, what kinds of items have you ordered?”

Cultural factors include the cultural behaviors of the informants, and the level of awareness which informants are likely to have on the topic. “Do you ever browse through a catalog without any intention of buying anything? If so, what is your motivation for looking through it?” is a question which probes the cultural behaviors of the informants. On the other hand, it is not reasonable to expect the average American to answer a question such as this one from Face to Face: “What do you communicate if you look directly at older people when they are speaking to you? Can you explain any other ‘rules’ for when it is important—or not OK—to look directly at other people?” Most Americans are not consciously aware of the “rules” of body language.

The third principle is to consider interviewer variables in writing the questionnaire. What is your students’ level of language proficiency? Perhaps they will need prompts to help them initiate and conclude the interview gracefully. They will likely need training in how to deal with potential communication breakdowns. This training was especially helpful in preparing students for the interviews at the retirement home, where faltering memories and weak voices often required new communication strategies.

What are the cultural filters through which your students will be viewing American culture? What are the gaps in their cultural knowledge? Questions should be designed according to what is likely to be surprising or new for the students. Whereas my students had viewed the catalogs as evidence of the laziness and materialism of Americans, they were surprised to learn that Americans view mail order catalogs in a more positive light, as sources of “good gift ideas” and a convenient and efficient way to shop.

Finally, it is important to allow opportunities for unanticipated findings. Including an open-ended question, “Are there any further comments you would like to make about mail order catalogs?” inspired several informants to bring up the issue of how on-line shopping might impact the use of mail order catalogs. Avoid yes/ no questions, but if you do use them, follow them up with an opportunity to explain. My students objected to a questionnaire on party manners in Face to Face which required them to choose yes or no. They wanted a “depends” option which they could then use to explain the variables.

The procedure for developing a questionnaire is straightforward, once you are aware of the principles for designing, selecting, and organizing the questions. First, brainstorm the possible subcategories of a cultural topic. For example, a broad topic such as “aging in America“ might conjure up subcategories such as care of the elderly, activities of the elderly, medical insurance issues, financial resources, social signs of respect, and the personal rewards and challenges of aging.

Then brainstorm the possible groups of informants: college students with grandparents, adults with elderly parents, the elderly at home and in retirement homes, retirement home professionals, and hospital personnel. My students are interested in what life is like inside a retirement home. The best group of informants is the residents themselves.

In order to construct a questionnaire, I would brainstorm with the students a list of possible questions, such as their daily schedule, how long they had lived there, what they liked and didn’t like about living there, and questions designed to elicit a description of their social, emotional, and intellectual life in the home. After brainstorming questions with my students, then I would return to the principles for guidance in selecting, wording, and ordering the questions in order to create the most enjoyable and productive interview possible.

Creating effective cultural questionnaires takes time, thought, and patience, but the rewards are great. Students feel more confident approaching informants when they have a set of questions to ask, and the ESL teacher can be assured that the students will return with information which will help deepen and extend students’ knowledge of American culture.