Western > Resources > Planning and Best Practices > Step 7 > Evaluation III

III.  How to Build Your Program Logic Model

A. What You Need to Know to Build Your Logic Model

In order to build a useful logic model, you will need to answer the following questions about the program:

1. What are the risk and protective factors to be addressed? (the goals)
2. What services and activities will be provided? (the strategies)
3. Who will participate in or be influenced by the program? (the target group)
4. How will these activities lead to expected outcomes? (the Theory of Change or "If-Then" Statement)
5. What immediate changes are expected for individuals, organizations, or communities? (the short-term outcomes)
6. What changes would the program ultimately like to create? (the long-term impacts).

1. What are the risk and protective factors to be addressed? (the goals)

The first thing you will need to know is what risk and/or protective factors you plan to address. If you have done a needs assessment, prioritized your needs, and identified resources, you should have a good idea about the goals that are important for your program to address (see steps 2, 3, & 4).

A note about terminology: The terms "goal," "objective," "outcome," and "impact" are often defined differently by different people for different purposes. For our purposes, these distinctions are not terribly important. We find that it is useful to think of "goals" as the answer to the question "What are issues that you would like the program to address?" (e.g., "the goal of the program is to address existing community laws and norms about ATOD use) and "outcomes" as the answer to: "What changes do you want to occur because of your program?" (e.g., "the outcome of the program will be to increase the number of community residents who believe teenaged smoking is dangerous").

Often, a program will have several different kinds of outcomes, including:

1. short-term or immediate outcomes, sometimes called objectives
2. long-term outcomes, sometimes called impacts.

One common problem in thinking about program goals is worrying too soon about how the goal will be measured. We suggest that you think about what your goals really are first, without worrying (yet) about measuring them. We'll get to measurement later.

STEP ONE: Using the logic model worksheet, fill in the risk and protective factors that your community seeks to address. These go in the first column, labeled "in order to address the level of this risk or protective factor"… These should follow from your needs assessment (Step Two), Prioritization Process (Step Three), and Resource Assessment (Step Four).

Click here for an example of some Risk/Protective factors from hypothetical program logic models

2. What services and activities will your program provide? (the strategies)

What are the activities involved in your program? That is, what will you actually be doing? It is very important to specify what activities you plan to do: A program that isn't implemented in the way it is planned is not likely to lead to the expected program outcomes.

When writing down your planned activities, try to answer the questions: "what are we going to be doing" and "when and how much are we going to do?"

What you do may include such things as: screening at-risk youth, providing after-school activities, delivering an educational curriculum, circulating red ribbons, writing letters to policy makers, hosting a community march, and so on.

"When and how much" refer to when the program or activities will be delivered (after school every day for 3 hours, a single day for 3 hours, etc).

If you have selected a model based on Best Practices, you should have a clear idea of the kinds of activities you will be implementing.

STEP TWO: For each risk factor that you are addressing, write down the planned activities in column 2 of the logic model worksheet, labeled "we will do the following program activities…" There may be multiple activities or program components that address a single risk factor, or one activity that addresses multiple risk factors.

Click here for some examples of program activities from hypothetical program logic models.

3. Who will participate in, or be influenced by, the program? (the target group)

The next important question is to whom is the program being delivered? That is, who is the recipient of your program, or whom do you expect to be influenced by your activities? If you are using the best practices guide, this may help you determine which groups are most likely to be influenced by your chosen strategy. You should also know whether the strategy you've chosen is universal, selective, or indicated.

STEP THREE: For each activity or component of your program, write down the specific groups that you expect will participate in or be influenced by the activity. Make sure and indicate how many you estimate will be participating in or influenced by the activity. Write this in the third column of your logic model worksheet, labeled "For these people and for this amount of time…."

Click here for some examples of descriptions of target groups from hypothetical program logic models

4. How will these activities lead to expected outcomes? (the Theory of Change)

The next step asks you to identify the assumptions underlying your program. That is, it asks you to think about why and how program activities are expected to lead to the desired outcomes. A very common problem in prevention programs is when the chosen program activities and strategies do not lead logically to the goals or outcomes that the program would like to achieve. That's why we recommend thinking through the assumptions of why and how you expect your program to lead to the desired changes. What are the steps that turn inputs into outputs into outcomes? You might think about this as a series of "if-then" relationships.

For example, if the program invests time and money to develop an inventory of drug-free summer activities, then youth will be more informed about what is available in the community. If youth know what is available, then they will be more likely to participate in these programs. If youth participate in alternative activities, then they will be more likely to develop friendships with non-using peers, and then be less likely to use ATOD themselves. Note that even in this very simple series of if-then statements there are a number of assumptions about the problem to be addressed, how the program will work and what it can achieve. For example, it assumes that:

• youth currently don't know about many available activities;
• the collaborative will have the necessary time, money, and expertise to develop the resource inventory;
• once the resource inventory is developed, people will use it, particularly the identified target group;
• knowing about the activities will lead youth to actually use the activities, and
• the activities will support development of new, positive peer relationships.When developing your map or logic model, think about the underlying assumptions. Are they realistic and sound? What evidence do you have to support your assumptions?

STEP FOUR: For each of the activities that you have proposed, write if-then statements that link how you expect that activity to lead to the expected short-term outcomes and long-term impacts. Write these in column 4 of your program logic model worksheet, labeled "We expect that this activity will lead to changes in these factors, which in turn will lead to our program goal."

Click here for some examples of descriptions of theories of change from hypothetical program logic models

Click here for some underlying assumptions of common prevention programs

5. What are the program's short and long-term goals/outcomes?

Short-Term Outcomes are the immediate program effects that you expect to achieve soon after the program is completed. For example, a parent education program is expected to improve the parents' family management skills.

[Click here to see some examples of short-term outcomes for hypothetical programs]

Long-Term Impacts, on the other hand, are the long-term or ultimate effects from the program. Let's follow our parent education program example one step further. We attempt to improve parents' family management skills, the immediate outcome, because we believe that improving parents' family management skills will ultimately help prevent or reduce their children's drug use, the long-term impact. However, research shows us that many factors (e.g., knowledge, attitudes, policy) must change, and much time must pass before we can detect any changes in the ultimate impact on drug use.

[Click here to see some examples of long-term impacts for hypothetical programs]

Issues in Defining Outcomes/Impacts

• There is no right number of outcomes/impacts. The number of outcomes/impacts selected by your collaborative will depend upon the nature and purpose of the collaborative, resources, size, and number of constituencies represented.
• The more immediate the outcome or impact, the more influence the program has over its achievement. In a parent training program, changes in participants' knowledge about substance abuse can be largely attributed to the education and training provided by the program.
• Conversely, the longer term the outcome or impact, the less direct influence a program will have on its achievement and the more likely other extraneous forces are to intervene. The extent of the final impact of decreased adolescent ATOD use is influenced by a variety of factors in the sociocultural, political, and economic environment.
• Because other forces affect an outcome/impact doesn't mean it shouldn't be included. Despite the influence of other factors on ATOD use, the program may wish to measure and track these outcomes/impacts in order to understand the rates of use in the community, and what effects a confluence of factors, including the specific program being implemented, may have on overall rates of use.
• Long-term impacts, however, should not go beyond the program's purpose or target audience. Think about what the program is designed to do—where its influence is likely to be felt—and focus the measurement at that level. Likewise, keep the measures focused on the targeted audience. In the above example, reductions in ATOD use by the children of the participating parents were expected to change for those specific children, not for the city as a whole.
• Confusing Outcomes with Outputs. Another common problem in thinking about program goals is confusing "outputs" with "outcomes." This is an important distinction. It is important to have goals about how many outputs your program will achieve: how many clients served, how many teachers trained, how many community events implemented, etc. These are what we might call "implementation goals": the program's goal for how much or how many events or activities are experienced. If your program doesn't successfully provide services, train teachers, or host events, then it is impossible for your program to ever change people and therefore reach its goals. However, outputs, or implementation goals, do not provide evidence that your program is creating change, the key to successful outcomes. Outcomes refer to changes produced (in individuals, communities, or systems) by your program. Outputs refer to the number of opportunities your program has to create these changes in the form of clients served, activities implemented, etc.

  
  a. What immediate changes are expected - for individuals, organizations, communities? (the short-term outcomes)

  • STEP FIVE: Write down the outcomes that you expect to see immediately as a function of your program. Remember, an outcome statement should be written so that it reflects the expected change in individuals, organizations, or communities. Write this in the 5th column of your program logic model worksheet, labeled "We will know these changes have occurred if":
    
    Note: In the framework, the short-term outcomes are likely to include some of the risk and protective factors that you specified as program goals. Make sure that there is a connection between the risk and protective factors being addressed and your long- and short-term outcomes!

  b. What changes would the program ultimately like to create? (the long-term impacts)

  • STEP SIX: Write down the long-term changes that you think this program is likely to influence eventually. There may be a number of long-term goals that your program could influence, but you should think about those that are most likely to flow directly and logically from the program activities and short-term outcomes you've specified. Write these in the 6th column of your program logic model worksheet, labeled "We will know we are reaching our goals if":

B. Reviewing Your Logic Model

Once your logic model is complete, you'll want to continue to use it for planning your evaluation. We also strongly recommend that you regularly review and update your logic model in order to see what has changed, keep track of progress, make modifications either in your work or your model, or when communicating to others about what the program is doing. Programs are usually not implemented exactly as planned, but are changed, adapted, and improved: Your logic model should provide a "picture" of these changes.

Previous | Outline | Next