Data Collecting Methods: Getting Down To Basics

This resource corresponds to Module 4.

Avoid Ambiguity

Are you more than 18 years old? (Check one.)

____ No

____ Yes

Some 18 year olds might answer Yes because they passed their 18th birthday. Others might answer No because they have not reached their 19th birthday. Are you 18 years old or older? or Were you born on or before [date]? is less confusing. Just asking for their birth date would be the least ambiguous way to ask this question-although it would require additional data analysis to determine which respondents were over the age of 18 and might jeopardize confidentiality.

Define Any Terms That Could Be Confusing

Have you ever smoked cigarettes? (Check one.)

____ No

____ Yes

This question leaves smoking open to definition. Does it mean trying one puff of a cigarette? Does it mean regular cigarette use? If you want to find out if the respondent has ever tried a cigarette, you might want to ask: Have you ever tried cigarette smoking-even one or two puffs?

Use Clear Time Frames

On how many days did you drink an alcoholic beverage during the past month (Write in.)

____ Days (Enter a zero if you did not drink an alcoholic beverage in the
past month.)

In this case, respondents may interpret the past month to mean the past 30 days, the month before the current month, or only the days that have passed so far in this month. It would be less confusing to ask On how many of the last 30 days did you drink an alcoholic beverage?

Put the Question Last, After Definitions or Examples

How many days in the past 30 days have you used a designated driver? A designated driver is one person in a couple or group who does not drink alcohol so that he or she can drive the drinkers home. (Write in.)

____ Days (Enter a zero if you did not use a designated driver in the past
30 days.)

Respondents are likely to answer a question as soon as it is stated. They may skip the important definition in the second sentence. In this case, the definition is especially important, since a designated driver is being defined as someone who abstains from alcohol as opposed to someone who limits his or her drinking.

The solution is to place the definition at the beginning of the question: A designated driver is a person who does not drink alcohol so that he or she can drive friends who have been drinking home. How many days in the past 30 days have you used a designated driver?

The placement of response choices when reading questions to a respondent is also important. When the choices are presented at the outset of the question, the respondent often forgets the range of choices available. For example:

Were you very satisfied, somewhat satisfied, somewhat dissatisfied, or very dissatisfied with the written materials distributed at this training?

Place the question at the end of the item: How satisfied were you with the written materials distributed at this training? Were you very satisfied, somewhat satisfied, somewhat dissatisfied, or very dissatisfied?

In a face-to-face interview or telephone survey, the interviewer should instruct the respondent not to answer until all the choices have been read. You do not want the respondent to claim he or she was somewhat satisfied before he or she knows that a somewhat dissatisfied answer is an option.

Don't Ask for an Answer the Respondent Won't Know

During the past 12 months, on how many days did you drink an alcoholic beverage. (Write in.)

____ Days (Enter a zero if you did not drink an alcoholic beverage in the
past 12 months.)

Respondents may not be able to recall or to compute the number of days on which they drank alcohol over a year. Strategies for getting this information include the following:

Ask about alcohol use during the past 30 days.

Ask the respondent to estimate use by providing categories, such as no days, 1-2 days, 3-5 days, 6-9 days, 10-19 days, 20-30 days, 40 or more days.
Ask about the respondent's pattern of use. For example, respondents would be presented with such choices as: 5 to 6 days a week, 3 to 4 days a week, 1 to 2 days a week, 3 to 4 days a month, 1 to 2 days a month, less than one day a month.

Don't Ask for Information Beyond the Respondent's Personal Knowledge or Experience

How serious do you think the problem of substance use/abuse is among youth in this community as compared to other communities in Massachusetts?
(Check one.)

____ Much less serious
____ Somewhat less serious
____ About the same
____ Somewhat more serious
____ Much more serious

Many respondents may not be able to accurately answer this question. It presumes knowledge of youth substance use/abuse in their community and in Massachusetts as a whole. The question might be improved by adding Don't know as a choice, but it would still be unclear how/why those who did not choose that response made their judgment.

Asking this question might be a way of assessing if the community sees substance use/abuse as a serious problem, but it is a poor way to estimate the actual seriousness of substance use/abuse.

Ask about Actual Behavior, Knowledge, and Attitudes

How likely is it that you will use alcohol the next time you go to a party? (Choose one.)

____ Definitely will not use
____ Probably will not use
____ Probably will use
____ Definitely will use

Hypothetical questions do not provide accurate information, even about the future behavior of the respondent. It is far more accurate to predict future behavior by asking people about their past or current behavior.

Ask for One Piece of Information (Do Not Ask Compound Questions)

How satisfied were you that training was interesting and useful? (Choose one.)

____ Very satisfied
____ Somewhat satisfied
____ Somewhat dissatisfied
____ Very dissatisfied

Here, the respondent is asked to judge whether the training was an interesting experience and whether it was useful. The respondent may have found the training very interesting and useful. Or the respondent could have found the training boring, but not very useful. In this case, ask two questions: one about whether the training was interesting and another about whether the training was useful. And define useful, e.g., whether the training was relevant to their work.

Don't Ask about Contingencies

During the past 30 days, have you driven a car, motorcycle, or truck when you felt too intoxicated to drive safely? (Choose one.)

____ No
____ Yes

This question does not apply to people who did not drink alcohol in the past 30 days or did not drive in the past 30 days. Thus, make sure to include as the first response option "I did not drive a car, motorcycle, or truck in the past 30 days."