This document presents an overview of process evaluation and how practitioners can use it to monitor program implementation.
Process evaluation looks at how program activities are delivered. It helps practitioners determine the degree to which an intervention was implemented as planned and the extent to which it reached the targeted participants. We all know that implementation quality is critical to maximizing the intended benefits and demonstrating strategy effectiveness. Process evaluation provides us with the tools to monitor quality. It also provides the information needed to make adjustments to strategy implementation in order to strengthen effectiveness.
Specifically, process evaluation can be used to:
Some central questions posed in process evaluation are: Who delivers the program? How often? To what extent was the program implemented as planned? How is the program received by the target group and program staff? What are barriers to program delivery? Was the data used to make program improvements/refinements? If so, what changes were made?
Outcome evaluation looks at results. It measures the direct effects of program activities on targeted recipients, such as the degree to which a program increased knowledge about alcohol and other drug use. But results don’t tell the whole story. Evaluation that only focuses on outcomes is sometimes called a “black box” evaluation because it does not take process evaluation into consideration. Disappointing outcome evaluation results can frequently be illuminated by examining how the program was implemented, the number of clients served, dropout rates, and how clients experienced the program. These are process evaluation questions. Positive evaluation results can also be explained: You can’t take credit for results if you can’t show that you did something that caused them to happen. Outcome evaluation only, without a process evaluation component, won’t provide information about why a program did or didn’t work.
Quality implementation means that the implementers of each strategy have:
Process evaluation measures should be designed to assess the degree to which implementers adhered to the items listed above.
If process data suggest the program was not effective, practitioners should work with project stakeholders to determine:
Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.