- Read the Module 3 workshop presentation.
- Read Activity 3: Getting What You’re Looking
For and post your responses on the Message Board.
- Participate in an online discussion of Activity
3, which will continue throughout the day. We encourage you
to check the Message Board periodically and participate in
the discussion.
Data collection can be intimidating, but
collecting your own data can have advantages over using existing
data. For example, data collection allows you to do the following:
- Obtain information about the people
in whom you are most interested. For example, existing
national data sources may provide general information on
Latinos living in the United States between the ages of 15
and 24. Collecting your own data can provide precisely the
information you want on immigrants and the children of immigrants
from Guatemala between 14 and 18 years of age who live in
your city.
- Obtain information related to a specific
issue. For example, you could collect information
that would tell you exactly where these young people go for
health information in your community (such as a particular
school-based clinic or hospital emergency room).
- Obtain information that is current.
Many state and national data efforts take years to collect,
analyze, and publish their data. Your own data collection can
provide much more up-to-date information about what is happening
in your community.
Next week, we will discuss three specific methods
of collecting data—surveys, focus groups, and key informant
interviews. But first, we will discuss a number of tasks common
to all data collection efforts. Many of the issues explored in
Module 1’s discussion of using existing data apply to collecting
new data. These include clarifying your purpose, deciding which
data will serve this purpose, finding out what has already been
done, working with local agencies, knowing when to stop, and
interpreting the data carefully.
In addition, anyone collecting data must do the
following:
We will describe each of these steps in detail,
below.
One advantage of collecting data is that it allows you to focus
exclusively on those people in whom you are most interested—your
target audience. Defining the target audience is a key step in
collecting data. When thinking about your target audience, keep
in mind the following:
- The target audience for your data
collection efforts may be defined by the audience served
by your prevention program. For example, prevention
programs that serve a Hispanic community, homeless youth,
or middle school children, will want to collect data on
those groups. However, you may want to also collect data
that allow you to compare your target audience with the
community as a whole (or other segments of the community).
This can lead to insights that make your prevention activities
more effective or that can support the allocation of resources
for your program. For example, you may want to compare
rates of drug and alcohol use among Hispanic youth in your
community with all youth in that age group (regardless
of ethnic or racial affiliation). Or, you may want to compare
homeless youth with youth who have homes or compare middle
school children with elementary and high school students
(which might tell you something about how substance use/abuse
patterns change with age).
- The target audience for your prevention
program may be determined by what you learn in your data
collection effort. If, for example, you received
funding to address substance use/abuse among young people,
in general, you may want to collect data on a broad range
of young people to determine if you should concentrate your
efforts on a specific group. Your data collection may reveal
that the young people in a particular neighborhood, ethnic
group, or age range are more at risk for substance use/abuse
than other young people.
It is also useful to collect data using categories
that allow you to compare your community (or target audience)
with those of other communities, your state, and the nation—as
well as the results of more specialized research in substance
use/abuse. One method of doing this is to try to collect data
using the age ranges or racial and ethnic categories used by
the United States Census Bureau or the National Center for Health
Statistics at the Centers for Disease Control and Prevention
(though this can be challenging since these agencies don’t
all use the same categories all the time.) Ultimately, it is
probably most important to collect information according to your
funders’ specified categories.
Collecting data involves creating (or adapting) questions that
produce the information you want. There are a number of questions
you should always ask yourself about any item you consider using
in a survey, focus group, or key informant interview:
- Is the question clear? If
everyone answering the question doesn’t interpret
the question in the same way, the answers they give
will not be consistent. For example, some young people,
if asked “Have you ever used drugs?” might
respond “Yes” because they have used
prescribed medications. If necessary, include information
to clarify the question. For example, you might want
to ask “Have you ever used drugs other than
those prescribed by a doctor?”
- Do people understand how you
want them to provide the information? For
example, someone could respond to the question “When
did you first smoke marijuana?” with “In
1984,” “When I was 13,” or “In
middle school.” Be specific and ask “How
old were you when you first smoked marijuana?” Or
use a multiple choice question that forces people
to respond in the form most useful for your analysis
(such as: a. Before age 12, b. Ages 12–3,
c. Ages 14–15, d. Ages 16–17, e. Age
18 or older)
- Are you asking more than one
question at a time (a compound question)? For
example, asking “Have you used marijuana
or alcohol?” and finding a substantial number
of respondents have, bypasses an essential piece
of information needed for designing a prevention
program: Whether these young people use marijuana,
alcohol, or both. Asking separate questions about
each subject will provide this important information.
- Will people be able to answer
the question? Students, for example, cannot
be expected to know “What percentage of students
in your school smoke marijuana?” (although
a drug counselor might be able to answer that question
correctly). Ask questions about the respondents’ personal
experiences, knowledge, attitudes, or recent and/or
important experiences.
- Will people answer the question
truthfully? Questions about behavior that
is illegal or socially stigmatized must be asked
in a way that encourages people to answer truthfully.
This is also true for questions whose answers could
prove awkward for the respondent if they were made
public (such as asking a school drug counselor
whether the school adequately supports the counseling
program). A key to getting truthful answers is
assuring people of confidentiality. (Confidentiality
will be discussed at greater length later in this
module, as well as in the modules on specific data
collection methods). Questions should not be asked
in ways that make the respondents feel guilty about
their behavior or attitudes or that imply there
is a “right” answer. The importance
of answering the questions honestly should be emphasized
in the introduction to the data collection process.
- What will I do with the answer
to this question? You should only collect
information that will help you design, target,
and evaluate your prevention program. However,
your data collection efforts must be broad enough
to reveal information and patterns that might not
be immediately evident. For example, asking questions
about the use of other drugs as part of a data
collection effort that primarily focuses on alcohol
abuse might reveal important information. However,
don’t expand the time it takes to collect
your data (and the time and expense for analyzing
it) by including a large number of questions of
purely academic interest.
- Has anyone asked these questions
before? It is quite likely that someone
has created (and tested) questionnaires or questions
that you can use or adapt. They may have had more
expertise and resources than you can afford. Often,
you can use or adapt these materials to collect
data. We will return to this topic in Module 4.
Please keep in mind that collecting new
data doesn’t necessarily mean developing your
own protocols, surveys, or questions. There are many
existing questions and core measures for questions
from which you can draw. We will discuss the benefits
of using these existing resources in more detail in
Module 4.
People cannot accurately answer questions they cannot understand.
Questions should be asked in language that is clearly understood
by the target audience. If your focus is (or includes) people whose
primary language is other than English, you may want to administer
surveys (or conduct focus groups or key informant interviews) in
their native language. A competent translator should be hired to
translate the questions, since nuances in translation can distort
the question. Finally, have the translation reviewed by someone
familiar with the particular culture and dialect of the people
of whom the questions will be asked.
Cultural competence also pertains to
those whose primary language is English. Questions
must be asked at an appropriate reading level (even
if they are read aloud or asked during an interview).
In most cases, use nontechnical language and avoid
professional jargon. However, if you are collecting
data from professionals, you may have to employ their
jargon (or clarify your terms) to be understood. For
example, law enforcement professionals might not consider “alcohol” to
be a drug. So if you ask a police officer about drug
abuse among local high school students, the response
may be limited to illegal substances unless you specify “illegal
use of drugs and alcohol.” Similarly, students
who may be unfamiliar with the term "amphetamine" may
know exactly where to purchase speed.
Other factors, such as gender and religion,
also influence people's understanding and/or interpretation
of survey questions. If you are creating a question
about gender, for example, your first instinct might
be to limit your response categories to "male" and "female." However,
it may be more appropriate to include categories for
male, female, trans-gendered, and questioning. By omitting
the additional categories, you may not only lose valuable
information, but may also alienate some respondents
who feel that the survey isn't for them. Similarly,
many cultures include alcohol use in their religious
ceremonies. Thus, in surveys asking about alcohol use,
it is important to include a disclaimer that excludes
use that is part of a religious ceremony.
The people involved in collecting data and the people from whom
the data is collected should understand how and why they are engaging
in the effort. You can present this information verbally, during
a training of the data collectors or before you administer a questionnaire
or begin a focus group or key informant interview. You should also
provide a written document that contains this information so participants
can refer to it later and so students, for example, can share it
with their parents.
The information you provide to subjects should include the following:
- Who is collecting data
- Why the data is being collected
- What types of questions will be asked
- Why/how the participants were chosen
- If participation is voluntary
- How long answering the questions will
take
- If the answers will be confidential
- If the data will be made public
- If and how the participants can obtain
a copy of the results
It is also important (and sometimes legally
required) that this information is communicated to
the parents of children from whom data is collected.
We will discuss this further in the section on maintaining
confidentiality and protecting respondents.
Pilot testing is a process of trying out your data collection instruments
and procedures with a small subset of your target audience to see
how well they work. A pilot test can reveal critical flaws like
ambiguous questions or interviews that take much too much time.
Data collection efforts (especially those involving large numbers
of people) should be pilot-tested before implementation. Strategies
for pilot-testing each data collection method will be discussed
in Modules 4, 5, and 6.
The data you collect will
be reliable only if it is collected consistently—that
is, if the data collection effort is administered
to all subjects in the same way. This is especially
true of surveys. It is important to communicate
the importance of consistency to everyone involved
in your effort. For example, it is extremely
important that everyone administering a classroom
survey read the material on confidentiality to
students exactly as written. They should not
paraphrase or condense the statement to “Your
responses will be confidential” or—even
worse—skip the statement entirely. Students
who didn’t hear the complete statement
may answer the questions less truthfully than
students in classrooms who were made fully aware
that no one will reveal their answers. This,
in turn, will affect the quality of the data
collected.
Data quality is often expressed in terms
of validity and reliability. Validity concerns whether
you are measuring the “true value” of what
you want to measure (for example, if you are collecting
information on actual alcohol use among young people,
as opposed to student perception—or misperception—of
their peers’ alcohol use). Reliability concerns
how well you consistently measure true value. Both
reliability and validity are affected by your choice
of target audience, the questions you ask, and whether
the data collection effort is culturally competent
and conducted skillfully and consistently.
Data collection must not harm the people from whom and about whom
you collect information. Collecting data could harm people in the
following situations:
- Information that could embarrass or
humiliate individuals, threaten their jobs, or subject
them to criminal action is not kept confidential.
- Information is solicited in a way
that makes people uncomfortable or raises troubling
personal or family issues.
There are federal and state laws for
data collection and research. These laws fall into three
general areas: confidentiality, parental consent, and
human subjects protection.
Confidentiality. Maintaining
confidentiality (or anonymity) of the data you collect
means that no one (except project staff collecting
or analyzing the data, in some situations) can connect
information from the data collection effort with the
individuals who provided that data. There are a number
of compelling reasons why most data collection should
be confidential:
- People are more likely to truthfully
answer questions (especially questions about drug
and alcohol use) when they know the information will
be confidential.
- Release of information that can be
traced to identifiable individuals can cause them
emotional—and in some cases, financial or legal—hardship.
- There are laws that protect confidentiality,
especially that of minors.
- Failure to protect confidentiality
could lead to lawsuits, should release of any data
prove harmful or embarrassing to individuals.
Methods of protecting confidentiality
will be discussed in Modules 4, 5, and 6.
Privacy and Parental Permission. Student
educational records are protected by the Family Educational
Rights and Privacy Act (FERPA) of 1974 and the Protection
of Pupil Rights Amendment (PPRA) of 1978. These acts
include provisions on privacy and parental consent
for data collection in schools. Federal regulations
on the protection of human subjects (discussed in
the next paragraph) also contain provisions for parental
permission when minors are used in research.
Human Subjects Protection.
All federally funded research (including data collection)
is governed by the Federal Policy for the Protection
of Human Subjects, which is administered by the Office
for Human Research Protections, United States Department
of Health and Human Services. Such research must be
reviewed and approved (or exempted from review) by
an Institutional Review Board—a committee that
determines that the research does not present risk
of harm to the research subjects. Most universities
and other agencies receiving federal research funds
have an Institutional Review Board. In many cases,
data collection that is not federally funded, or conducted
for programmatic purposes rather than research, is
excluded from this process.
Mandatory Reporting.
You and/or the people assisting in your data collection
efforts (such as teachers) may be subject to state
requirements mandating that certain professionals (including
health care providers, school staff, and others) report
any suspicions of child abuse. All 50 states have such
requirements. Even if your data collection effort does
not ask about child abuse, children may reveal this
information in a venue in which you or your data collectors
are legally or ethically required to breach confidentiality
and report these suspicions to the proper authorities.
An example would be a child in a focus group on family
alcohol use who revealed to a teacher that a parent
became violent when drinking and hit the child. You
must be prepared to fulfill your ethical and legal
responsibilities in these situations.
Your choice of data collection method will depend on a number of
factors, including the information you are seeking, your available
resources, and the type of access you have to the target audience.
Module 4 will discuss surveys, an efficient way to collect standardized
information from a relatively large number of people.
Please proceed to Activity
3. Getting What You’re Looking For.
Empirical research series: Writing
guide series (2003). Fort Collins, CO: Colorado
State University Writing Center. Available online
at http://writing.colostate.edu/guides.
Harding, W. (1999, February 6) Designing
and implementing questionnaires (a training for Massachusetts
Prevention Centers staff). Burlington, MA: Social Science
Research and Evaluation, Inc.
National Committee for Injury Prevention
and Control (1989). Injury prevention: Meeting
the challenge. New York: Oxford University Press.
Salant, P. and Dillman, D. (1994). How
to conduct your own survey. New York: John Wiley & Sons,
Inc.
Trochim, W. (2001). Research methods
database (2nd ed.). Cinninnati, OH: Atomic Dog
Publishing.
|
