Questionnaire Design
Relevancy and accuracy are the two basic criteria a questionnaire must meet if it is achieve the researcher’s purposes. Design will be require making several decisions.
What should be asked?
How should each question be phrased?
In what sequence should the question rearranged?
What questionnaire layout will best serve the research objectives?
How should the questionnaire be pretested? Does the questionnaire need to be revised?
Questionnaires are an inexpensive way to gather data from a potentially large number of respondents. Often they are the only feasible way to reach a number of reviewers large enough to allow statistically analysis of the results. A well-designed questionnaire that is used effectively can gather information on both the overall performance of the test system as well as information on specific components of the system. If the questionnaire includes demographic questions on the participants, they can be used to correlate performance and satisfaction with the test system among different groups of users.
It is important to remember that a questionnaire should be viewed as a multi-stage process beginning with definition of the aspects to be examined and ending with interpretation of the results. Every step needs to be designed carefully because the final results are only as good as the weakest link in the questionnaire process. Although questionnaires may be cheap to administer compared to other data collection methods, they are every bit as expensive in terms of design time and interpretation.
The steps required to design and administer a questionnaire include:
1. Defining the Objectives of the survey
2. Determining the Sampling Group
3. Writing the Questionnaire
4. Administering the Questionnaire
5. Interpretation of the Results
This document will concentrate on how to formulate objectives and write the questionnaire. Before these steps are examined in detail, it is good to consider what questionnaires are good at measuring and when it is appropriate to use questionnaires.
What should be asked?
1. Questionnaire relevancy:
A questionnaire is relevant if no unnecessary information is collected and if the information that is needed to be solved the business problem is obtained.
2. Questionnaire accuracy:
Accuracy means the information is reliable and valid.
When to use a questionnaire?
There is no all encompassing rule for when to use a questionnaire. The choice will be made based on a variety of factors including the type of information to be gathered and the available resources for the experiment. A questionnaire should be considered in the following circumstances.
a. When resources and money are limited. A Questionnaire can be quite inexpensive to administer. Although preparation may be costly, any data collection scheme will have similar preparation expenses. The administration cost per person of a questionnaire can be as low as postage and a few photocopies. Time is also an important resource that questionnaires can maximize. If a questionnaire is self-administering, such as a e-mail questionnaire, potentially several thousand people could respond in a few days. It would be impossible to get a similar number of usability tests completed in the same short time.
b. When it is necessary to protect the privacy of the participants. Questionnaires are easy to administer confidentially. Often confidentiality is the necessary to ensure participants will respond honestly if at all. Examples of such cases would include studies that need to ask embarrassing questions about private or personal behavior.
c. When corroborating other findings. In studies that have resources to pursue other data collection strategies, questionnaires can be a useful confirmation tools. More costly schemes may turn up interesting trends, but occasionally there will not be resources to run these other tests on large enough participant groups to make the results statistically significant. A follow-up large scale questionnaire may be necessary to corroborate these earlier results.
Defining the Objectives of the Survey
Broad objective:
Main objective of the research is to find out the discriminative behavior done on the basis of gender in higher education in private universities in Dhaka city.
Specific objectives:
• To determine the students experience how they are facing the inequality in higher education.
• To identify what they need to erase the disparity.
• To develop an appropriate standard of behavior toward the victim who are discriminated.
• To asses student knowledge and opinion to come up with effective solution.
What kind of questions do we ask?
In general, there are two types of questions one will ask, open format or closed format.
What kind of questions do we ask?
In general, there are two types of questions one will ask, open format or closed format.
Open format questions are those that ask for unprompted opinions. In other words, there are no predetermined set of responses, and the participant is free to answer however he chooses. Open format questions are good for soliciting subjective data or when the range of responses is not tightly defined.
Example:
1. What things do you like most about your job?
2.What means of local banks can you think off offhand?
3.What comes to mind when you look at this advertisement?
An obvious advantage is that the variety of responses should be wider and more truly reflect the opinions of the respondents. This increases the likelihood of you receiving unexpected and insightful suggestions, for it is impossible to predict the full range of opinion. It is common for a questionnaire to end with and open format question asking the respondent for her unabashed ideas for changes or improvements.
Open format questions have several disadvantages. First, their very nature requires them to be read individually. There is no way to automatically tabulate or perform statistical analysis on them. This is obviously more costly in both time and money, and may not be practical for lower budget or time sensitive evaluations. They are also open to the influence of the reader, for no two people will interpret an answer in precisely the same way. This conflict can be eliminated by using a single reader, but a large number of responses can make this impossible. Finally, open format questions require more thought and time on the part of the respondent. Whenever more is asked of the respondent, the chance of tiring or boring the respondent increases.
Closed format questions usually take the form of a multiple-choice question. They are easy for the respondent; give. There is no clear consensus on the number of options that should be given in an closed format question.
Example:
1.Did you work overtime or at more than one job last week?
a.yes b.no
Closed format questions offer many advantages in time and money. By restricting the answer set, it is easy to calculate percentages and other hard statistical data over the whole group or over any subgroup of participants. Modern scanners and computers make it possible to administer, tabulate, and perform preliminary analysis in a matter of days. Closed format questions also make it easier to track opinion over time by administering the same questionnaire to different but similar participant groups at regular intervals. Finally closed format questions allow the researcher to filter out useless or extreme answers that might occur in an open format question.
Whether your questions are open or closed format, there are several points that must by considered when writing and interpreting questionnaires:
1. Clarity: This is probably the area that causes the greatest source of mistakes in questionnaires. Questions must be clear, succinct, and unambiguous. The goal is to eliminate the chance that the question will mean different things to different people. If the designers fails to do this, then essentially participants will be answering different questions.
To this end, it is best to phrase your questions empirically if possible and to avoid the use of necessary adjectives. For example, it asking a question about frequency, rather than supplying choices that are open to interpretation such as:
1. Very Often
2. Often
3. Sometimes
4. Rarely
5. Never
It is better to quantify the choices, such as:
6. Every Day or More
7. 2-6 Times a Week
8. About Once a Week
9. About Once a Month
10. Never
There are other more subtle aspects to consider such as language and culture. Avoid the use of colloquial or ethnic expressions that might not be equally used by all participants. Technical terms that assume a certain background should also be avoided.
2. Leading Questions: A leading question is one that forces or implies a certain type of answer. It is easy to make this mistake not in the question, but in the choice of answers. A closed format question must supply answers that not only cover the whole range of responses, but that are also equally distributed throughout the range. All answers should be equally likely. An obvious, nearly comical, example would be a question that supplied these answer choices:
1. Superb
2. Excellent
3. Great
4. Good
5. Fair
6. Not so Great
A less blatant example would be a Yes/No question that asked:
7. Is this the best CAD interface you have every used?
In this case, even if the participant loved the interface, but had an favorite that was preferred, she would be forced to answer No. Clearly, the negative response covers too wide a range of opinions. A better way would be to ask the same question but supply the following choices:
8. Totally Agree
9. Partially Agree
10. Neither Agree or Disagree
11. Partially Disagree
12. Totally Agree
Example: Many people are using dry cleaning less because of improved wash and wear clothes.
How do you feel wash and wear clothes have affected your use of dry cleaning facilities in the past four years?
A. Use less
B. No change
C. Use more
This example is also poor in the way it asks the question. It’s choice of words makes it a leading question and a good example for the next section on phrasing.
3. Loaded question:
A question that suggest a socially desirable answer or is emotionally charged.
Ex: I feel capable of handling myself in most social situation……. A. agree B. disagree
4. Phrasing: Most adjectives, verbs, and nouns in English have either a positive or negative connotation. Two words may have equivalent meaning, yet one may be a compliment and the other an insult. Consider the two words “child-like” and “childish”, which have virtually identical meaning. Child-like is an affectionate term that can be applied to both men and women, and young and old, yet no one wishes to be thought of as childish.
In the above example of “Is this the best CAD interface you have every used?” clearly “best” has strong overtones that deny the participant an objective environment to consider the interface. The signal sent the reader is that the designers surely think it is the best interface, and so should everyone else. Though this may seem like an extreme example, this kind of superlative question is common practice.
A more subtle, but no less troublesome, example can be made with verbs that have neither strong negative or positive overtones. Consider the following two questions:
1. Do you agree with the Governor’s plan to oppose increased development of wetlands?
2. Do you agree with the Governor’s plan to support curtailed development of wetlands?
They both ask the same thing, but will likely produce different data. One asks in a positive way, and the other in a negative. It is impossible to predict how the outcomes will vary, so one method to counter this is to be aware of different ways to word questions and provide a mix in your questionnaire. If the participant pool is very large, several versions may be prepared and distributed to cancel out these effects.
5. Embarrassing Questions: Embarrassing questions dealing with personal or private matters should be avoided. Your data is only as good as the trust and care that your respondents give you. If you make them feel uncomfortable, you will lose their trust. Do not ask embarrassing questions.
6. Hypothetical Questions Hypothetical are based, at best, on conjecture and, at worst, on fantasy. I simple question such as:
1. If you were governor, what would you do to stop crime?
This forces the respondent to give thought to something he may have never considered. This does not produce clear and consistent data representing real opinion. Do not ask hypothetical questions.
7. Prestige Bias: Prestige bias is the tendency for respondents to answer in a way that make them feel better. People may not lie directly, but may try to put a better light on themselves. For example, it is not uncommon for people to respond to a political opinion poll by saying they support Samaritan social programs, such as food stamps, but then go on to vote for candidates who oppose those very programs. Data from other questions, such as those that ask how long it takes to learn an interface, must be viewed with a little skepticism. People tend to say they are faster learners than they are.
There is little that can be done to prevent prestige bias. Sometimes there just is no way to phrase a question so that all the answers are noble. The best means to deal with prestige bias is to make the questionnaire as private as possible. Telephone interviews are better than person-to-person interviews, and written questionnaires mailed to participants are even better still. The farther away the critical eye of the researcher is, the more honest the answers.
7. Simple-dichotomy question:
a fixed alternative question that requires the respondent to choose one /two alternatives.
Ex: did you make any long distance calls last week ?
a. Yes b. no
8. Determinant choice question:
A type of fixed alternative question that requires a respondent to choice 1 response from among several possible alternatives.
Ex: please give us some information about your flight. In which section of the aircraft did you sit?
a. First class b. business class c. coach class
9. Frequency determination question:
Types of fixed alternative that asked for an answer about general frequency of occurrence.
How frequently do you watch the MTV tv channel?
a. Everyday
b. 5-6 times a week
c. 2-4 times a week
d. Once a week
e. Less than once a week
f. Never
10. Attitude rating scale:
Measures used to rate attitude such as likert scale, scientific differential and staple scale.
Ex: Do you satisfied with your new gym instructor?
Strongly disagree (1) Disagree (2) uncertain (3) Agree (4) Strongly agree (5)
11. Checklist question:
Type of fixed alternative question that allow the respondent to provide multiple answers to a single question.
How mach you expend for your mobile call charge?
a. 10-20 b. 50-100 c. 100-300 d. Above 500
What is the best question sequence?
Order bias: we get different results depending on whether the respondent ranks the items from highest to lowest or vice versa.
Filter question: Question in the questionnaire that screens out respondents not qualified to answer a second question.
Ex: do yo ever have a problem cashing a check in spring field….A. yes B. no
