Questionnaire Design Is a Long Process That Demands Careful Attention. a Questionnaire

Abstract

Questionnaire design is a long process that demands careful attention. A questionnaire is a powerful evaluation tool and should not be taken lightly. Design begins with an understanding of the capabilities of a questionnaire and how they can help research projects.This paper establishes that questionnaires are versatile, allowing the collection of both subjective and objective data through the use of open or closed format questions. Modern computers have only made the task of collecting and extracting valuable material more efficient.However, a questionnaire is only as good as the questions it contains. There are many guidelines that must be met before you questionnaire can be considered a sound research tool.The paper also explains the manual and electronic preparation and use of the different types of questionnaires in research works in Nigeria. The later parts of the work details thoroughly reveals the comprehensive process involved in the design of questionnaire.

CONTENTS

Abstract………………………………………………………………………………….1

Table of contents…………………………………………………………………………2

1.0Introduction…………………………………………………………………………3

2.0What questionnaires can measure………………………………………………4

3.0When to use questionnaires………………………………………………………5

4.0Characteristics of a good questionnaire………………………………………………6

5.0Methods of Administration………………………………………………………7

5.1Direct contact……………………………………………………………….7

5.2Mailed questionnaire………………………………………………………7

6.0Forms of questionnaires……………………………………………………………….9

6.1Closed form……………………………………………………………….9

6.2Open form……………………………………………………………….9

6.3Pictorial form……………………………………………………………….9

7.0Questionnaire design process………………………………………………………11

7.1Defining the objectives of the survey…………………………………….11

7.2Determining the sampling group……………………………………………..11

7.3Writing the questionnaire……………………………………………..11

7.4Question sequence………………………………………………………13

7.5Impedance to the use of questionnaire…………………………………….14

8.0Conclusion………………………………………………………………………..15

References…………………………………………………………………………………16
1.0INTRODUCTION

The level of technology in Nigeria has greatly increased in recent times and the generation of lots of solutions to our pressing problems has been through the process called research. This is an activity or a process of proffering solutions to problems through a methodical collection, analysis, and interpretation of data, for promoting progress, and for enabling man to relate more effectively to his environment, to accomplish his purpose, and to resolve his conflicts. In a nutshell, research process is the method of gathering reliable information to answer questions or support an experimental hypothesis.

The process of gathering the required information or raw data is dynamic and there are different approaches to it. Various methods of data collection are:

Observation: Direct study of the environment and the behaviour of people, and events are recorded as they occur.

Interview: This involves being in direct contact with people to collect information in verbal or oral form.

Questionnaire: The use of a set of relevant questions requiring the respondent to fill and return to the researcher.

Literature review: Here, appropriate and relevant textbooks, electronic library, lecture notes and past works are consulted.

For the purpose of this term paper, we shall be considering the use of questionnaire as a research tool in architecture.

A questionnaire as invented by Sir Francis Galton is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents. It attempts to get the feelings, beliefs, experiences or activities of respondents. They also may involve formats which can be relatively structured depending on the situation.They are often designed for statistical analysis of the responses, but not in all cases.Questionnaires have advantages over some other types of surveys in that they are cheap, do not require as much effort from the questioner as verbal or telephone surveys, and often have standardized answers that make it simple to compile data. However, such standardized answers may frustrate users. Questionnaires are also sharply limited by the fact that respondents must be able to read the questions and respond to them. Thus, for some demographic groups conducting a survey by questionnaire may not be practical.

As a type of survey, questionnaires also have many of the same problems relating to question construction and wording that exist in other types of opinionpolls.

2.0WHAT QUESTIONNAIRES MEASURE

Questionnaires are quite flexible in what they can measure, however they are not equally suited to measuring all types of data. We can classify data in two ways, Subjective vs. Objective and Quantitative vs. Qualitative.

When a questionnaire is administered, the researchers control over the environment will be somewhat limited. This is why questionnaires are inexpensive to administer. This loss of control means the validity of the results is more reliant on the honesty of the respondent. Consequently, it is more difficult to claim complete objectivity with questionnaire data then with results of a tightly controlled lab test. For example, if a group of participants are asked on a questionnaire how long it took them to learn a particular function on a piece of software, it is likely that they will be biased towards themselves and answer, on average, with a lower than actual time. A more objective usability test of the same function with a similar group of participants may return a significantly higher learning time. More elaborate questionnaire design or administration may provide slightly better objective data, but the cost of such a questionnaire can be much higher and offset their economic advantage. In general, questionnaires are better suited to gathering reliable subjective measures, such as user satisfaction, of the system or interface in question.

Questions may be designed to gather either qualitative or quantitative data. By their very nature, quantitative questions are more exact then qualitative. For example, the word "easy" and "difficult" can mean radically different things to different people. Any question must be carefully crafted, but in particular questions that assess a qualitative measure must be phrased to avoid ambiguity. Qualitative questions may also require more thought on the part of the participant and may cause them to become bored with the questionnaire sooner. In general, we can say that questionnaires can measure both qualitative and quantitative data well, but that qualitative questions require more care in design, administration, and interpretation.

3.0WHEN TO USE QUESTIONNAIRES

There is no all encompassing rule for when to use a questionnaire. The choice will be made based on a variety of factors including the type of information to be gathered and the available resources for the experiment. A questionnaire should be considered in the following circumstances.

1. When resources and money are limited: A Questionnaire can be quite inexpensive to administer. Although preparation may be costly, any data collection scheme will have similar preparation expenses. The administration cost per person of a questionnaire can be as low as postage and a few photocopies. Time is also an important resource that questionnaires can maximize. If a questionnaire is self-administering, such as a e-mail questionnaire, potentially several thousand people could respond in a few days. It would be impossible to get a similar number of usability tests completed in the same short time.
2. When it is necessary to protect the privacy of the participants: Questionnaires are easy to administer confidentially. Often confidentiality is the necessary to ensure participants will respond honestly if at all. Examples of such cases would include studies that need to ask embarrassing questions about private or personal behaviour.
3. When corroborating other findings: In studies that have resources to pursue other data collection strategies, questionnaires can be a useful confirmation tools. More costly schemes may turn up interesting trends, but occasionally there will not be resources to run these other tests on large enough participant groups to make the results statistically significant. A follow-up large scale questionnaire may be necessary to corroborate these earlier results.

4.0CHARACTERISTICS OF A GOOD QUESTIONNAIRE

Desirable qualities of a good questionnaire appear to be a matter of common sense. Designing questionnaires is one of the most difficult tasks of an investigator. A pilot survey carried out prior to the actual survey almost invariably leads to modifications and improvements in the questionnaires. The following are the common characteristics of a good questionnaire:

1A good questionnaire should not be ambiguous: This means that the question must be capable of only one interpretation.

2A good questionnaire must be easily understood. Technical terms should be avoided, except where the questionnaire is meant for specialists.

3A good questionnaire must not contain words of vague meaning.

4A good questionnaire should not require that calculations be made.

5A good questionnaire should not require the respondent to decide upon classification.

6A good questionnaire should not be in such a form that the answers will be biased.

7A good questionnaire should cover the exact object of the inquiry.

8A good questionnaire should not be too long.

5.0METHODS OF ADMINISTRATION

This refers to the methods by which the respondents receive the questionnaire to be answered.

5.1Direct Contact

In this case, the researcher presents the questionnaire to the respondent(s) directly by meeting him personally. This reduces to the minimum, any act of refusal or bias responses. This is simply because he can explain the purpose and significance of the study, explain points, answer questions, and motivate respondents to answer questions fully and truthfully; but it could be very expensive and time consuming.

• Advantages and Disadvantages of The Questionnaire

The choice of the questionnaire in preference to other survey techniques is generally a matter of weighing its advantages and disadvantages against those of the interview, with which it is most nearly interchangeable.

• Advantages

1.For the same cost, many more responses to questionnaires can be obtained than can responses to interviews.

2.It requires much less skill to administer a questionnaire than an interview. They are simply mailed or handed to respondents with a minimum of explanation.

3.The impersonal nature of a questionnaire ensures some uniformity from one measurement situation to another.

4.Respondents may have greater confidence in their anonymity and thus feel freer to express their views.

5.Questionnaires place less pressure on the respondent for immediate response. This may not always be desirable.

• Disadvantages

1.Respondent motivation is difficult to assess.

2.Questionnaire cannot be used with illiterates and there could be loss of meaning in interpretation.

3.Unless a random sampling of returns is obtained, those that are completed and returned can represent biased samples.

5.2Mailed Questionnaires

With efficient and reliable postal system, questionnaires can equally reach many people in widely scattered areas quickly and at relatively low cost, but the rate of response could be so low as to introduce biases that will render the obtained data almost useless.

• Advantages

Among the major advantages of the mail questionnaire is that;

1It permits wide coverage at a minimum expense both in money and effort. It affords not only wider geographical coverage than any other technique, but it also reaches individuals who are normally difficult to contact.

2Particularly when it does not call for a signature or other means of identification, the questionnaire may, because of its greater impersonality, elicit more candid and more objective replies.

3The questionnaire also permits more considered answers. In an interview if the respondent does not have the information, he may still give an answer rather than admit his ignorance.

4The questionnaire is more adequate in situations in which the respondent has to check his information.

• Disadvantages

The questionnaire possesses the following disadvantages:

1The questionnaire does not permit the investigator to note apparent reluctance or evasiveness of his respondent, a matter which is better handled through the interview. Similarly, the questionnaire does not permit the investigator to follow through on misunderstood questions or evasive answers.

2The advantages of the questionnaire are more obvious than its disadvantages, and, consequently, it frequently appeals to the amateur who uses it for all purposes regardless of its suitability and without sufficient awareness of its semi-hidden weaknesses and limitations. The major weakness of the questionnaire is undoubtedly the problem of non-returns.

3Another major disadvantage of the questionnaire is the possibility of the misinterpretation of the questions. This danger is increased when the questions are ambiguous because of improper formulation or because of the differences in socioeconomic and cultural status.

4The validity of questionnaire data depends, in a crucial way, on the ability and the willingness of the respondent to provide the information requested.

5The questionnaire frequently does not provide the researcher with sufficient opportunity for developing interest on the part of the respondent, nor does it allow him to develop the rapport necessary to permit him to ask questions of a personal or embarrassing nature.

6.0FORMS OF QUESTIONNAIRES

There are various forms of questionnaire used in research process.

6.1Closed Form

Closed format questions usually take the form of a multiple-choice question. They are easy for the respondents to give. There is no clear consensus on the number of options that should be given in a closed format question. Obviously, there needs to be sufficient choices to fully cover the range of answers but not so many that the distinction between them becomes blurred. Usually this translates into five to ten possible answers per questions. For questions that measure a single variable or opinion, such as ease of use or liability, over a complete range (easy to difficult, like to dislike), conventional wisdom says that there should be an odd number of alternatives. This allows a neutral or no opinion response. Closed format questions offer many advantages in time and money. By restricting the answer set, it is easy to calculate percentages and other hard statistical data over the whole group or over any subgroup of participants. Modern scanners and computers make it possible to administer, tabulate, and perform preliminary analysis in a matter of days. Closed format questions also make it easier to track opinion over time by administering the same questionnaire to different but similar participant groups at regular intervals. Finally closed format questions allow the researcher to filter out useless or extreme answers that might occur in an open format question.Respondents’ answers are limited to a fixed set of responses. Most scales are closed ended. Other types of closed ended questions include:

• Dichotomous questions - The respondent answers with a “yes” or a “no”.
• Multiple-choice: The respondent has several options from which to choose.

The closed form of questionnaire consists of prepared list of specific questions and a choice of possible answers. They are easy to administer and to fill out; they keep the respondent’s mind fixed to the subject and facilitate the process of tabulation, analysis and scientific generalisations. But they do not reveal the respondent’s motives (why he answers as he does) and they do not always yield information of sufficient scope or depth; nor do they discriminate between fine shades of meaning.

6.2Open Form

No options or predefined categories are suggested in this case. The respondents supply their own answers without being constrained by a fixed set of possible responses. Examples of types of open ended questions include:

• Completely unstructured - For example, “What is your opinion of democracy?”
• Word association - Words are presented and the respondent mentions the first word that comes to mind.
• Sentence completion - Respondents complete an incomplete sentence. For example, “The most important consideration in my decision to buy a new house is . . .”
• Story completion - Respondents complete an incomplete story.
• Picture completion - Respondents fill in an empty conversation balloon.
• Thematic apperception test - Respondents explain a picture or make up a story about what they think is happening in the picture.

Open format questions are those that ask for unprompted opinions. In other words, there are no predetermined set of responses, and the participant is free to answer however he chooses. Open format questions are good for soliciting subjective data or when the range of responses is not tightly defined. An obvious advantage is that the variety of responses should be wider and more truly reflect the opinions of the respondents. This increases the likelihood of you receiving unexpected and insightful suggestions, for it is impossible to predict the full range of opinion. It is common for a questionnaire to end with and open format question asking the respondent for her unabashed ideas for changes or improvements.

6.3Pictorial Form

This form of questionnaire involves the presentation of drawings or photographs instead of written statements from which to choose answers. It is quite suitable for collecting data from children and illiterates. They have two limitations: (a) they can be used only in situations where the visual characteristics are distinguishable and easily understood. (b) They are difficult to standardize particularly when the pictures are photographs of human beings.

7.0QUESTIONNAIRE DESIGN PROCESS

Questionnaires are an inexpensive way to gather data from a potentially large number of respondents. Often they are the only feasible way to reach a number of reviewers large enough to allow statistically analysis of the results. A well-designed questionnaire that is used effectively can gather information on both the overall performance of the test system as well as information on specific components of the system. If the questionnaire includes demographic questions on the participants, they can be used to correlate performance and satisfaction with the test system among different groups of users.

It is important to remember that a questionnaire should be viewed as a multi-stage process beginning with definition of the aspects to be examined and ending with interpretation of the results. Every step needs to be designed carefully because the final results are only as good as the weakest link in the questionnaire process. Although questionnaires may be cheap to administer compared to other data collection methods, they are every bit as expensive in terms of design time and interpretation.

The steps required to design and administer a questionnaire include:

1. Defining the Objectives of the survey
2. Determining the Sampling Group
3. Writing the Questionnaire
4. Administering the Questionnaire
5. Interpretation of the Results

7.1DEFINING THE OBJECTIVES OF THE SURVEY