Examining Data: The Value of Comparisons

Comparing your data to other, existing data provides a context for understanding your assessment or evaluation results. For example, survey data from your high school may seem, at a glance, to reveal high smoking rates among 9th graders. However, a comparison of these data to statewide data might actually show that your school’s smoking rates are much lower than the statewide average.

When examining data, it’s useful to make comparisons:

• Between the community now and the community some time in the past.
• Between the community and other similar communities
• Between the community and the state, as a whole
• Among different population groups in the community, including different age groups and genders.

For example, you can look at data from an indicator such as 30-day alcohol use and draw comparisons in different ways. 

• You can compare current 30-day use to past 30-day use, and find out if alcohol use is higher or lower than it once was.
• You can compare current use in your community to average rates across the State,  and find out if alcohol use in your community is  lower or higher than the State, overall. 
• You can compare usage rates of males and females, and discover if use among males is higher or lower than among females; or compare 14-year-olds and 17-year-olds, and see how their rates compare.  

Small Numbers

When looking at data and making comparisons, don’t ignore sample size. Small numbers can sometimes be misleading. For example, if data indicate that between 2003 and 2006 motor vehicle accidents increased by 100%, that may sound like a lot. However, there may have been only one motor vehicle accident in 2003 and two in 2006—still an increase of 100%, but one that is not so alarming. In this case, it is important to look at data both in terms of percentages and as actual numbers. And keep in mind the following:

• When small numbers are used to calculate rates or percentages, differences may be exaggerated.
• Low, or even very low, percentages of some behaviors should not be dismissed as unimportant, especially if the behavior has severe consequences.

Rates

Epidemiology uses rates to standardize data in order to be able to compare data across populations that are different sizes. A rate is a measure of the frequency with which a health event occurs in a defined population over a period of time. 

For example, if a town with a population of 30,000 has 500 arrests in a given year for driving under the influence (DUIs), you can divide 500 by 30,000 to get a rate of .017 arrests per person per year. To make the rate easier to understand, we can multiply by 1,000 and say that the rate of DUIs for the town is 17 per 1,000 people per year. Moreover, a neighboring town of 22,000 people with 367 DUI arrests would have the same rate (divide 367 by 22,000 and then multiply by 1,000).

The following calculation provides a rate per 1,000 people per time period:

For national data sources or larger population sizes, people will often calculate the rate with 10,000 or 100,000 population size. You can multiply by whatever size makes sense for your community.

Things to Keep in Mind

• Examine different kinds of data. Substance use and other behavioral health problems are complex so understanding them requires looking at different kinds of data, both quantitative and qualitative, to get an accurate and complete picture of the problems.
• Look for relationships and patterns. Numbers alone have no meaning. Look for patterns over time and relationships between data.
• Be aware that gaps exist. After determining what data exists, see what is missing. It is likely that here will be “gaps” or additional data that you will need to find (or collect) in to paint a complete picture of your community.
• Not all data is equal. Some data are more valid and available than others. Qualitative data tends to be less objective than quantitative data. Note these differences in your analysis.

Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.