Writing and Delivering Survey Questions

The Science Of Writing Survey Questions

Writing survey questions is both and science and an art. There are well established “rules” for writing questions that a researcher must consider when crafting questions from scratch. Question writing is also an art in that the only way to become proficient at writing questions is to write questions. It is a process where experience helps. Learning from one’s failures helps. Let us first discuss the science of survey writing.

• Rule 1: Ensure that your questions are not biased.

• Rule 2: Ensure that your questions are not double barreled.

• Rule 3: Ensure that your question option choices are mutually exclusive.

• Rule 4: Ensure that your question option choices are exhaustive.

• Rule 5: Know your audience.

We will discuss each of these rules in greater detail.

Regarding Rule 1, the concept of bias in this context is any word or phrase within the body of the question – or the option choices – that might make the respondent lean toward one set of responses over another set of responses. In other words, if the question is written in a fashion that presupposes a “correct” answer, then the researcher has written a biased question. Biased questions lead to the collection of biased data, which in turn leads to biased research results – i.e., research that is essentially useless.

Biased questions lead the respondent. They presume a correct or better answer. Consider this example: To what extent do you agree that research methods courses help critical thinking skills? What about the language of this question makes it biased? To what extent do youagreethat research methods courses help critical thinking skills? The fact that the body of the question reads “agree” could influence a respondent to presume that “to agree” is the preferred answer. In reality, with opinion based survey questions, there is no preferred answer choice. A less biased question would read: To what extent do youagree or disagreethat research methods courses help critical thinking skills? The fact that this version includes “disagree” makes it less biased. My experiences have taught me that all questions have the of possibility of bias, and thus I refer to questions as being “more” or “less” biased in comparison to one another. The absence of bias is inherently difficult given that questions are crafted by humans who could unknowingly interject bias into survey questions.

An astute observer might argue that the order of the words “agree” and “disagree” in the body of that question might influence respondents as well. That is, the mere fact that agree appears before disagree might bias someone to choose agree over disagree. This would be a correct assumption. To remedy this, a researcher might re-word the question altogether or when administering the survey there will be two versions – one version where agree precedes disagree, which will be given to half of the total number of respondents selected to take the survey – and a second version where disagree precedes agree, which will be given to the other half.

Regarding Rule 2, a double barreled question expresses more than one thought yet only allows for one answer. Consider this example question: How would you feel about the federal government spending less money on defense and putting more money toward higher education? Options choices include: Yes, no, maybe. The problem here – and what makes this question double barreled – is that it asks two separate questions yet only allows for one answer.

Double Barreled:

How would you feel about the federal government spending less money on defense (Part 1) and putting more money toward higher education? (Part 2)

** One question with two parts

Not Double Barreled:

How would you feel about the federal government spending less money on defense?

How would you feel about the federal government putting more money toward higher education?

** Two separate questions each with one part