Some data sets may have collection or quality issues that will affect your ability to obtain useful data. In some cases, you can overcome these barriers by working with the data providers (or your evaluator) to reconfigure the data in ways that meet your needs. In other cases, you simply may not be able to use the data or will need to keep their limitations in mind when drawing conclusions based on the data. Caveats about data limitations, and its possible consequences for your analysis, should be included in data reports.
Common barriers to obtaining useful data include the following:
- Data may be aggregated. Hospitals, for example, often combine adult and youth data or data across several communities. This can be frustrating if you are seeking information about youth in your town. The agency may be able to sort the data for you.
- Jurisdictions may overlap. For example, the jurisdiction boundary of your local police department may not correspond to that of the school district. A trauma center may draw patients from across your state.
- Time periods may be inconsistent or too short. Data from one agency may be organized by calendar year, another by fiscal year, and another by school year. The data may not be current enough or collected for a long enough time to track trends accurately.
- Data may be missing or incomplete. Information included in agency records and local data sets is often missing or incomplete. If the amount of missing data is large, the data may not provide an accurate picture of your community. This is especially true if some information is consistently missing, such as records from a particular school district or police precinct. Or, a failure to consistently record data (such as age or blood alcohol content) may make it impossible for you to analyze the data in ways that are useful for your efforts.
- Data categories may not meet your needs. For example, sub-categories such as race or ethnicity are not always determined or implemented consistently across organizations.
Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.