Elementary Statistics (1+1)

2.1.10 Designing the Questionnaire

Before framing the questionnaire it is essential to set out in detail the data which we desire from the answers to questionnaire. It shall be wise if we can construct the type of tables which we would like to obtain from the enquiry. It may not always be possible to set out all the possible data we would like, in advance, since many things may be learnt in the course of enquiry and one may find that what he believed to be ideal was not in fact ideal. For this reason those who are likely to be concerned with analyzing the results should be called in at the very early stage. For example, it may not be very appropriate for a government statistician to collect some data on, say, unemployment, and then hand them over to an economist to analyse. The wise thing would have been to consult the economist first on what data were desirable.

The success of the questionnaire method of collecting information depends largely on the proper drafting of the questionnaire. Drafting questionnaire is a highly specialized job and requires a great deal of skill and experience. It is difficult to lay down any hard and fast rules to be followed in this connection. However, the following general principles may be helpful in framing a questionnaire:

1. Covering letter. The person conducting the survey must introduce himself and state the objective of the survey. It is desirable that –

(i) A short letter is enclosed. The letter should state in as few a words as possible the purpose of the survey and how the informant would tend to benefit from it.

(ii) Enclose a self-addressed stamped envelope for the respondent’s convenience in returning the questionnaire.

(iii) Assure the respondent that his answers will be kept in strictest confidence.

(iv) Promise the respondent that he will not be solicited after he fills up the questionnaire.

(v) If possible, offer special inducements (free gifts, concession coupons, etc.) to return the questionnaire.

(vi) If the respondent is interested, promise a copy of the results of the survey to him.

2. Number of questions should be small. The number of questions should be kept to the minimum. The precise number of questions to be included would naturally depend on the object and scope of the investigation. Fifteen to twenty-five may be regarded as a fair number. If a lengthy questionnaire is unavoidable, it should preferably be divided into two or more parts.

It should be noted that there is an inverse relationship between the length of a questionnaire and the rate of response to the survey. That is, the longer the questionnaire, the lower will be the rate of response, the shorter the questionnaire, the higher will be the rate of response. Therefore, each question must be clearly presented in as few a words as possible and each question should be deemed essential to the survey. In addition questions must be free from ambiguities.

3. Questions should be arranged logically. The questions must be arranged in a logical order so that a natural and spontaneous reply to each is induced. They should not skip back and forth from one topic to another. Thus, it is undesirable to ask a man how many children he has before asking whether he is married or not. Similarly, it would be illogical to ask a man his income before asking him whether he is employed or not. Thus the sequence of the questions should be considered carefully in terms of the purpose of the study and the persons who will supply the information. Questions applying identification and description of the respondent should come first followed by major information questions. If opinions are requested, such questions should usually be placed at the end of the list. Two different questions worded differently be included on the same subject to provide cross-check on important points.

4. Questions should be short and simple to understand. Unless the person being interviewed is technically trained, technical terms should be avoided. Words such as “capital” or “income” that have different meanings for different persons should not be used unless a clarification is included in the question.

5. Ambiguous questions ought to be avoided. ‘Ambiguous questions’ means different things to different people. It will not be possible to obtain comparable replies from correspondents who take a question to mean different things. For example,

Consider the following question:

Do you smoke? Yes / No

There are several ambiguities in this question. It is not clear whether the desired response pertains to cigars, cigarattes, pipes or combinations thereof. Also it is not clear whether occasional smoking or habitual smoking was the primary concern of the question. If we are interested only in current cigarette consumption, it would be better to ask:

How many cigarettes do you currently smoke each day?

Less than 5

5 to 9

10 to 14

15 to 19

20 and above

6. Personal questions should be avoided. As far as possible, questions of a personal and peculiar nature should not be asked. For example, questions about income, Sales-tax paid, etc., may not be willingly answered in writing. Where such information is essential, it should be obtained by personal interviews. Even then, such questions should be asked only at the end of the interview, when the informants feel more at ease with the interviewer.

7. Instructions to the informants. The questionnaire should provide necessary instructions to the informants. For example, the questionnaire should specify the time within which it should be sent back and the address at which it should be sent. Instructions about units of measurements, etc., should also be given. For instance, if there is a question on weight, it should be specified as to whether weight is to be expressed in pounds or kilograms or in some other units.

8. Objective type Questions. Avoid questions of opinion and keep to questions of fact. In factual studies, it is highly desirable that questions are so designed that objective answer may be forthcoming. For example, instead of asking the condition of a building, allow the informant or enumerator to state the condition in his own words. It is desirable to ask if a structure was in good condition, needed minor repairs, needed structural repairs or was unfit for use. No doubt, answer to such questions may not be completely objective but they can be readily tabulated. Similarly, while asking students how do they normaly travel to college, frame a question of the type:

How do you normally travel to college?

i) By bus (ii) By your own car (iii) By your own scooter

(iv) By taxi (v) On foot (vi) Any other

The respondent will tick mark the particular alternative applicable to him.

This type of question is known as multiple-choice question. It suggests several answer among which the respondent may choose. If a multiple choice question is used, all alternatives should be stated and a ‘don’t know’category be left in the questionnaire. Such questions not only facilitate tabulation but will take very little time of the respondent to fill the questionnaire. However, this type of question is excellent if most of the possible answers are both known and few in number. When the possible answers are numerous, a limited list – even if accompanied by "any other” category – may elicit response different from that which otherwise would be forthcoming. Multiple choice questions tend to bias result by the order in which alternative answers are given. When ideas are involved, the first item in the list of alternative has a favourable bias. The use of multiple- choice question is indicated only when the investigator is confident of the existence of a limited group of important alternatives and it should be avoided when the there are many possible responses of relatively equal significance.

9. “Yes” or “No” question. As far as possible the questions should be of such a nature that they can be answered easily in ‘Yes’ or ‘No’. Such questions pose a simple alternative to the respondent. This is an excellent technique if applied to situations where a clear-cut alternative exists. The questions “Do you own a car?”, “Are you married?”, “Did you vote in last election?” can easily be answered with a “yes” or “no”. However, when the alternatives is not clear-cut, the “yes” or “no” type question should be avoided. A question such as “Do you favour the Government policies?” usually cannot be answered with a simple reply. The Government has so many policies and only the most radical or partisam would favour or oppose them all. A typical citizen may endorse many, have no opinion on some and reject others. The “Yes” or “no” question in this case compels him to compress a variety of opinions into a simple alternative which may, in reality, not exist.

Sometimes a respondent cannot give a simple “yes” or “no” answer either because he has not yet made up his mind or because the lacks information on the topic. For example, the answer to the question “Are you in favour of public schools?” may not always be in ‘yes’ or ‘no’ because the respondent has not thought over it. In such cases additional alternative such as ‘do not know’, ‘undecided’; no opinion’ should be included.

10. Specific information questions and open-end questions. Specific information questions call for a specific item of information. For example, “What is your age?”, “How many children do you have ?”, etc. These questions are simple and direct and are well adapted to securing information of this type. Care should be taken to use this type of question only where the respondent can answer correctly. The open question does not pose alternatives or request specific information. It leaves the respondent free to make whatever reply he chooses. For example, the question, what should be done to enhance the practical utility of B.F.Sc course? Why do you use Colgate toothpaste or ‘Lux’ soap are open-end questions. In many ways open question is superior to other types—there is no danger of being unduly restrictive suggesting answers, posing false alternatives, and introducing some bias. It also may serve to interest the respondent in the interview itself, especially if he is asked his opinion at the outset. However, open questions are difficult to tabulate. Since no restriction is placed upon the variety of answers, many will often be forthcoming. This not only increased the labour involved but frequently leads to improper tabulation. Hence every effort must be made to minimize open questions in the questionnaire.

11. Questionnaire should look attractive. A questionnaire should be made to look as attractrive as possible. The printing and the paper used, etc., should be good and plenty of space should be left for answers depending upon the type of questions.

12. Questions requiring calculations should be avoided. Questions should not require calculcations to be made. For example, informants should not be asked yearly income, for in most cases they are paid monthly. Similarly, questions necessitating calculation of ratios and percentages, etc., should not be asked as it may take much time and the informant may not send back the questionnaire.

13. Pre-testing the questionnaire. The questionnaire should be pre-tested with a group before mailing it out. The advantage of pre-testing is that the shortcomings of the questionnaire can be discovered and it can be revised in the light of the tryout.

14. Cross-checks. If possible, one or more change the serial should be incorporated into the questionnaire to determine whether the respondent is answering at least the important questions correctly.

Method of tabulation. The method to be used for tabulating the results should be determined before the final draft of the questionnaire is made. If the results of the questionnaire are to be computerized, it is desirable to consult the computer experts before making a final draft.