Substance Abuse & Mental Health Services Administration

The Substance Abuse and Mental Health Services Administration (SAMHSA)

Data Collection Methods: Pros & Cons

MethodsDescriptionsProsCons
ArchivalData that have already been collected by an agency or organization and are in their records or archives
  • Low cost
  • Relatively rapid
  • Unobtrusive
  • Can be highly accurate
  • Often good to moderate validity
  • Usually allows for historical comparisons/trend analysis
  • Often allows for comparisons with larger populations
  • May be difficult to access local data
  • Often out of date
  • When rules for record-keeping are changed, makes trend analysis difficult or invalid
  • Need to learn how records were compiled to assess validity
  • May not be data on knowledge, attitudes, and opinions
  • May not provide a complete picture of the situation
Key Informant InterviewsStructured or unstructured one-on-one directed conversations with key individuals or leaders in a community
  • Low cost (assuming relatively few)
  • Respondents define what is important
  • Rapid data collection
  • Possible to explore issues in depth
  • Opportunity to clarify responses through probes
  • Sources of leads to other data sources and other key informants
  • Can be time consuming to set up interviews with busy informants
  • Requires skilled/trained interviewers
  • Accuracy (generalizability) limited and difficult to specify
  • Produces limited quantitative data
  • May be difficult to analyze and summarize findings
Focus GroupsStructured interviews with small groups of like individuals using standardized questions, follow-up questions, and exploration of other topics that arise to better understand participants
  • Low cost
  • Rapid data collection
  • Participants define what is important
  • Some opportunity to explore issues in depth
  • Opportunity to clarify responses through probes
  • Can be time consuming to assemble groups.
  • Produces limited quantitative data
  • Requires trained facilitators
  • Less control over process than key informant interviews
  • Difficult to collect sensitive information.
  • Accuracy (generalizability) limited and difficult to specify
  • May be difficult to analyze and summarize findings
SurveysStandardized paper-and-pencil or phone questionnaires that ask predetermined questions
  • Can be highly accurate
  • Can be highly reliable and valid
  • Allows for comparisons with other/larger populations when items come from existing instruments
  • Easily generates quantitative data
  • Relatively high cost
  • Relatively slow to design, implement, and analyze
  • Accuracy depends on who and how many people sampled
  • Accuracy limited to willing and reachable respondents
  • May have low response rates
  • Little opportunity to explore issues in depth

Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.

  • Behavioral Health is Essential to Health
  • Prevention Works
  • Treatment is Effective
  • People Recover
Health and Human Services Logo
SAMHSA’s mission is to reduce the impact of substance abuse and mental illness on America’s communities.

Substance Abuse & Mental Health Services Administration • 1 Choke Cherry Road • Rockville, MD 20857 • 1-877-SAMHSA-7