The Social Context of Question Asking
The precise wording of questions plays a vital role in determining the answers given by respondents. This fact is not appreciated as fully as it should be, even in ordinary conversation. For example, a colleague mentioned that he needed to pick out granite for a kitchen countertop. The only day he could make the trip was the Saturday before Labor Day. Although he called on Friday to make certain the store was open, he arrived at the store on Saturday only to find a sign on the door that said "Closed Labor Day Weekend." When asked if he remembered what question he had asked the clerk at the store, he said, "I asked him what hours he was open on Saturday, and he replied 'Nine to five.'"
This story illustrates the basic challenge for those who engage in the business of asking questions. It illustrates not only the importance of the golden rule for asking questions-Ask what you want to know, not something else-but also, more important, the ambiguities of language and the powerful force of context in interpreting the meaning of questions and answers. Our colleague had unwittingly asked a perfectly ambiguous question. Did the question refer to Saturdays in general or the next Saturday specifically? The clerk obviously interpreted the question as referring to Saturdays in general. Our colleague meant the next Saturday and did not think his question could mean anything else until he arrived at the store and found it closed.
In everyday life, these types of miscommunications happen all the time. Most of the time they are corrected by further conversation or by direct questions that clarify their meaning. Sometimes they only get corrected when expected behavior does not occur, as was the case when the store turned out to be closed. But the stylized form of question asking used in surveys does not often provide feedback about ambiguities or miscommunications. We must depend on pretesting to weed out ambiguities and to help reformulate questions as clearly as possible-to ask about what we want to know, not something else.
The thesis of this book is that question wording is a crucial element in surveys. The importance of the precise ordering of words in a question can be illustrated by another example.
Two priests, a Dominican and a Jesuit, are discussing whether it is a sin to smoke and pray at the same time. After failing to reach a conclusion, each goes off to consult his respective superior. The next week they meet again. The Dominican says, "Well, what did your superior say?"
The Jesuit responds, "He said it was all right."
"That's funny," the Dominican replies. "My superior said it was a sin."
The Jesuit says, "What did you ask him?"
The Dominican replies, "I asked him if it was all right to smoke while praying."
"Oh," says the Jesuit. "I asked my superior if it was all right to pray while smoking."
Small Wording Changes that Made Big Differences
The fact that seemingly small changes in wording can cause large differences in responses has been well known to survey practitioners since the early days of surveys. Yet, typically, formulating the questionnaire is thought to be the easiest part of survey research and often receives too little effort. Because no codified rules for question asking exist, it might appear that few, if any, basic principles exist to differentiate good from bad questions. We believe, however, that many such principles do exist. This book provides principles that novices and experienced practitioners can use to ask better questions. In addition, throughout the book we present examples of both good and bad questions to illustrate that question wording and the question's social context make a difference.
Loaded Words Produce Loaded Results
Suppose a person wanted to know whether workers believed they