|
I am implementing a
evidence-based prevention program and want to ensure that I achieve the
program's intended outcomes. I recognize the importance of implementing
the program with fidelity, but I would like to adapt the program to meet
the unique needs of my community. Is there any research on how to effectively
strike a balance between fidelity and adaptation?
The literature on how evidence-based prevention programs actually work
is still quite sparse and the number of empirical studies is even smaller.
With the movement towards evidence-based prevention, researchers, however,
have begun to address these issues more seriously.
In an effort to isolate the elements of programs that are critical to
their success, CSAP researchers reviewed 21 model and effective programs
from the National Registry of Effective Prevention Programs (NREPP). Teams
of prevention researchers independently reviewed these programs' materials
and interviewed the developers. The researchers conducting this core components
analysis made the following recommendations:
1. Structure your intervention to initially focus on relationship building;
then follow with opportunities to practice behaviors learned.
2. Promote consistent messages via multiple informants such as parents,
peers, and teachers.
3. Combine ATOD content with strategies to promote life skills. Attend
to the characteristics of the target population that place them at-risk
for ATOD and structure activities to address these characteristics.
4. Use written, session-by-session, easy to follow curricula.
5. Incorporate your programs into existing networks (school, community,
church) by involving people from these networks in change efforts.
6. Tailor your program content to the culture and language of clients.
7. Eliminate logistical barriers to program participation.
8. Acknowledge and tailor your program to developmental influences.
9. Employ known and trained authorities to deliver the intervention (peers,
parents, teachers, guidance counselors, coaches, etc.).
10. Capitalize on client strengths; acknowledge weaknesses but do not
focus exclusively on them.
11. Establish and nurture long-term and effective partnerships with collaborating
agencies.
12. Involve parents. Plan social, recreational, and cultural events to
foster increased interaction among parents and youth. Attend to parental
deficits by providing skills training to enhance parental self-efficacy.
(Details of this study are available in Science-Based Programs and Principles,
2002 by Schinke, Brounstein, and Gardner) These recommendations are written
for those who want to design an effective prevention program. A program
implementer might want to adapt a policy of not making adaptations that
would alter any of these core components since these are elements that
other effective programs have in common and may be directly correlated
to program effectiveness. It should be noted that the Core Components
Analysis is not a rigorous scientific study and should not be relied upon
as theory. Consequently, making policy decisions based on the theory may
not ensure that the program is being effectively replicated or adapted.
The Controversy
The debate around fidelity/adaptation seems to fall into two camps. On
one side, there are many program developers who encourage that a program
be implemented as it was designed. They claim that if you want to be effective
then you must replicate the model with fidelity keeping adaptations to
a minimum. On the other side, there are researchers that claim that the
most realistic and effective approach to program implementation is to
strike a balance between fidelity to the original model and strategic
adaptations.
In a recent paper entitled Finding the Balance: A Practitioner's Guide
to Program Fidelity/Adaptation, Thomas Backer, provided practitioners
with a guide to addressing the balance of program fidelity/adaptation.
His recommends that practitioners use a combination of common sense and
good, well-structured planning to wrestle with the challenges of program
fidelity/adaptation". He states that there is ample evidence supporting
the fact that if you alter a program too much or in certain ways, it does
not work. Consequently, he believes that in order to maintain the effectiveness
of a given program the implementer needs to find the balance between fidelity
to the original program model and adapting the model strategically to
meet the unique needs of a particular community.
Backer has put forth a 12-step checklist for finding this balance. These
are useful guidelines for program implementers who want to retain the
effectiveness of a program, but recognize the need for adaptations. These
guidelines are:
1. DEFINE what you mean by fidelity/adaptation balance, and share
your definition with all those who will collaborate on implementing the
program.
2. Relate the THEORY behind the prevention programs you select
to fidelity/adaptation.
3. Relate what's known about the CORE COMPONENTS of the program
to fidelity/adaptation.
4. ASSESS fidelity/adaptation concerns as part of the larger effort
to measure community needs and assets.
5. Determine what RESOURCES are needed to deal with fidelity/adaptation
concerns, and how to present need for these resources to funders.
6. Look at what TRAINING the program developer offers that might
help you address fidelity/adaptation challenges.
7. Define how you'll MEASURE your success in achieving a reasonable
fidelity/adaptation balance, including whether you'll use a program developer's
FIDELITY INSTRUMENT.
8. Decide whether an individualized CONSULTATION with the program
developer on fidelity/adaptation issues would be feasible and useful.
9. INVOLVE THE COMMUNITY in thinking about fidelity/adaptation
concerns, as part of the larger implementation process.
10. Weave results from all these steps into a PLAN FOR ADDRESSING FIDELITY/ADAPTATION
BALANCE - as part of an overall implementation plan.
11. Include fidelity/adaptation issues in the design of the program's
EVALUATION.
12. Continue paying attention to fidelity/adaptation balance, and how
it may change over time, in the ONGOING OPERATION of the program.
Backer's guidelines are not based upon an empirical study, but rather
a qualitative study using methods such as interviews with program developers
and comparative analysis by analogy to similar situations in different
fields. Consequently, while these guidelines may be the best that current
research has to offer they are not exhaustive. Backer reviews each of
the guidelines in depth explaining the implementer's task, the expected
outcomes from the task, and some resources that may assist with the task.
Additional tips on fidelity/adaptation and related CAPT products:
- Conduct a needs assessment and use these data to select the best-fit
program for your particular situation. By doing this you can avoid making
unnecessary adaptations during program implementation. The Northeast
CAPT offers trainings on both assessment (Assessment: A Vital Preplanning
Activity http://www.captus.samhsa.gov/northeast/services/f2ftraining/assessment1.cfm)
and program selection (Selecting Appropriate Prevention Programs to
Meet Local Needs http://www.captus.samhsa.gov/northeast/services/f2ftraining/selecting1.cfm).
- If you make any major adaptations to a program, check with your evaluator
to see whether or not this will effect the evaluation of the program.
For instance, if you drop a violence prevention component from a multi-component
program you should not be held accountable for the corresponding outcomes.
Be sure to make the appropriate changes to your evaluation to reflect
this adaptation.
- Use CSAP's Achieving Outcomes: A Practitioner's Guide to Effective
Prevention document as a guide and/or checklist for program implementation
and evaluation. This document can be found on SAMHSA's prevention Platform
website at http://preventionplatform.samhsa.gov/
Please contact the Northeast CAPT at capt@edc.org for more information.
|
