Data Collecting Methods: Getting Down To Basics

This resource corresponds to Module 2.

Hi! I'm Scott Formica. I'm a researcher with a firm that works with local, state, and federal agencies and private organizations. We often use existing data to meet client and/or funder needs. It's often the most efficient and cost-effective way to meet our client's needs.

For example, we recently worked with a coalition in Cambridge, Mass., one of several communities that had received a demonstration grant to implement a research-based substance abuse prevention program in their schools. Cambridge wanted to implement Life Skills Training in several of their middle schools. We decided to use the evaluation tool created by the original program developer. This survey was designed specifically for the program to measure the intermediate and long-term variables that the program was designed to change. Students were given a pretest before the program and a posttest after the program. We also administered a follow-up questionnaire following booster sessions given to the students in the seventh and eighth grades.

In the third year of the project, the funder decided it wanted to conduct a cross-site evaluation that looked at some general, long-term outcomes across all the projects it had funded. The funder came up with a set of new items it was interested in looking at. Unfortunately, these didn't match what Cambridge was already measuring.

This was a problem. Collecting new data was going to take time and money that we didn't have. And we didn't want to add the new questions to the ones we were already asking, as this might have compromised the integrity of our instrument. Finally, since the program had already started, we could not collect baseline data on the new items the funder wanted measured. Without baseline data, we couldn't assess the impact of the program on the new items.

Fortunately, there was a solution to this problem. And that solution was using existing data. Since 1997, Cambridge had been administering a version of the Youth Risk Behavior Survey to all middle school students. As luck would have it, this survey included the very items that the funder wanted its projects to use. By looking at the Youth Risk Behavior Survey data for the schools in which the Life Skills Training program was being used, we were able to accommodate the needs of our funder without spending money we didn't have or compromising the integrity of our study. In other words, since Cambridge had an existing instrument in place, and had collected the data already, all we had to do was look at the existing data for the kids who were involved in the Life Skills Training program. Existing data came to our rescue, allowing us to satisfy our funder as well as provide some additional information about the effects of our project.

Scott Formica is a member of CSAP's Northeast CAPT's evaluation team and a researcher at Social Science Research and Evaluation, Inc.