Guidelines for Determining if a Prevention Program, Policy, or Strategy is Evidence-Based

“Evidence-based” prevention refers to any prevention approach that research shows has been effective under a particular set of circumstances. Prevention experts agree on several criteria that qualify an intervention to be called evidence-based:

• It is included in Federal registries of evidence-based interventions
• It is reported (with positive effects on the primary targeted outcome) in peer-reviewed journals
• It has documented evidence of effectiveness, based on guidelines developed by SAMHSA/CSAP and/or the State. These guidelines include the following:

Guideline 1: The intervention is based on a theory of change that is documented in a clear logic or conceptual mode; AND

Guideline 2: The intervention is similar in content and structure to interventions that appear in registries and/or the peer-reviewed literature; AND

Guideline 3: The intervention is supported by documentation that it has been effectively implemented in the past, and multiple times, in a manner attentive to scientific standards of evidence and with results that show a consistent pattern of credible and positive effects; AND

Guideline 4: The intervention is reviewed and deemed appropriate by a panel of informed prevention experts that includes: well-qualified prevention researchers who are experienced in evaluating prevention interventions similar to those under review; local prevention practitioners; and key community leaders as appropriate, e.g., officials from law enforcement and education sectors or elders within indigenous cultures.

Experts in the field agree that the nature of evidence is continuous. The strength of evidence, or “evidence status,” of tested approaches will fall somewhere along a continuum from weak to strong. Strong evidence means that the approach “works”—that it generates a pattern of positive outcomes attributed to the approach itself, and that it reliably produces the same pattern of positive outcomes for certain populations under certain conditions.

Experts also agree that evidence becomes “stronger” with replication and field testing in various circumstances. However, experts do not agree on a specific minimum threshold of evidence or cutoff point below which evidence should be considered insufficient. Nor do they agree whether little evidence is equivalent to no evidence at all. Even evidence from multiple studies may be judged insufficient to resolve all doubts about the likely effectiveness of an approach designed for a different population or situation.

Strength of evidence is critical to selecting approaches that are likely to work, but it is not the sole consideration. Keep in mind two practical criteria:

• When deciding between two approaches, choose the one for which there is stronger evidence of effectiveness if the approach is similar, equivalent, and equally well-matched to the community’s unique circumstances.
• There may be a place for an approach with little or weak evidence of effectiveness—for example, when other approaches with stronger evidence do not fit local circumstances.

Resources: http://www.nrepp.samhsa.gov/ [1].

Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.