This resource corresponds to Day 1.

While there are many possible ways to structure your evaluation, the following designs are the most common.


One-Group, Post-Only Design (least expensive, least rigorous)

For this design, you administer a post-test (e.g., survey) to the intervention group after it has received the intervention. Though relatively inexpensive, this design does not allow you to measure changes from baseline (before the intervention), nor does it allow you to measure change in relation to other groups of people who did not take part in the intervention.

One-Group, Pre- and Post-Program Design

This design is slightly more rigorous because you assess the intervention group before and after program implementation. This allows you to compare where your group started with where they ended up and, thus, measure any changes that occurred in the interim. However, you still don’t know whether the program was responsible for producing the change. Alternative explanations are possible. For example, change may have occurred because participants matured over time, or because they “learned” something by taking the initial pretest.

Pre- and Post-Program with Comparison Group Design

For this design, you administer pre- and posttests to the intervention group and another similar group that does not receive the program. The addition of a comparison group helps you determine whether your target group would have improved over time even if it had not experienced your program. The more similar the two groups are with respect to variables that may affect program outcomes (e.g., gender, race or ethnicity, socioeconomic status, education), the more confident you can be that your program contributed to any detected changes. This design also helps control for test effects, since both groups got the pretest. However, this design does increase both the expense and complexity of your evaluation. It also leaves room for alternative explanations, since the program and comparison groups may differ in some undetected but important ways.

Pre- and Post-Program with Control Group Design (most expensive, most rigorous)

This design offers the greatest possibility of attributing evaluation outcomes to program activities. By randomly assigning individuals from the same target population to either an intervention or control group, all members of the target population have an equal chance of winding up in either group. This fact should ensure that members of the intervention and control groups are equivalent with respect to many key variables that might affect their performance on the pre- and posttests. Of the four designs discussed here, this is the most complex and expensive to conduct, but also provides the highest level of certainty that it was your program that caused any changes detected by your evaluation.

Issues to Consider When Selecting a Design

• Complex evaluations cost more, but allow for greater confidence in a study’s findings.
  
  
• Complex evaluation designs are more difficult to implement and so require higher levels of expertise in research methods and analysis.
  
  
• More complex designs may cause problems other than increased cost. For example, a typical problem with the pre- and posttest with control group design is resistance to being randomly assigned to a group that will not receive a potentially beneficial intervention.
  
  
• No evaluation design is immune to threats to its validity. There is a long list of possible complications associated with any evaluation study. However, your evaluator will help you maximize the quality of your evaluation study.
  
  
• Don’t assume generalizability! Just because an intervention works in one setting doesn’t mean that it will in others. It’s unlikely that your findings will apply to all groups and populations equally well.
  
  
• Some evaluation is better than none. Though you may not have the money or resources to conduct the evaluation of your dreams, start somewhere—even if that means using the least rigorous design.

For more information on designing an evaluation, visit
Evaluation Guides and Courses.