Service to Science Academies Go Virtual!

In times of shrinking travel budgets and tighter cost restrictions on meetings, federal agencies are struggling to find new ways of bringing people together to exchange information and receive training. To address these challenges, SAMHSA’s Service to Science Initiative has adapted its cornerstone training program from a face-to-face meeting into an interactive webinar series.

Service to Science is a national initiative that provides training and technical assistance (T/TA) to innovative and exemplary local substance abuse prevention programs to increase their capacity to provide more credible evidence of program effectiveness. Selected programs are assigned to work with evaluation TA providers to identify and address gaps in evaluation methods. However, program representatives sometimes lack a basic understanding of evaluation principles and do not know the right questions to ask when meeting with evaluation TA providers.

In the past, evaluation basics were delivered during centralized academies in which program representatives assembled to participate in face-to-face trainings on the following topics: becoming an evidence-based program, using frameworks for evaluation planning, and applying logic modeling to program development and evaluation. However, given changes in federal travel guidelines, assembling participants was no longer a viable option.

“It was clear that we needed to convert the face-to-face presentations to a virtual platform using webinar technology,” says Kim Dash, Chief of Service to Science. “But we also knew that it wasn’t practical to squeeze all of the presentation material into a webinar series. We had to decide what information was important to keep and how best to present that material in a new format.”

Working collaboratively, Dash and the Service to Science team pored over the existing materials and extracted the evaluation basics that they felt were most critical for participants to master. The result is the Service to Science Evaluation Primer webinar series, which helps program participants begin the process of evaluating their program’s prevention activities and learn more about the initiative. According to Dash, “The webinar series encourages participants to think through ways to make evaluation more feasible and useful, consider evaluation issues or gaps worth addressing, as well as clarify questions they may want to ask the technical assistance providers assigned to work with them.”

Until the webinar series was developed, the Service to Science model called for funneling programs that had less experience with evaluation into the centralized academies to learn the basics before they received customized T/TA, while those programs with more experience went straight to the onsite technical assistance component. Under the new model, all programs regardless of experience level attend all the Evaluation Primer webinars, which helps to assure that everyone is starting on the same page, says Adela Santana, Service to Science Lead for the Southeast Resource Team.

The Evaluation Primer series comprises three 1.5-hour webinars delivered on three consecutive days via Adobe Connect. So far, the webinars have been presented twice, once in December and January, with a final presentation scheduled for early February. Sixty-nine people viewed the first two webinars; initial feedback has been extremely positive.

Dr. Jerry Evans, Director of Research and Evaluation for Community Health Initiatives in Carbondale, Colorado, who participated in the January series, says, “The presentations were the best evaluation capacity-development seminars for community members I have participated in—period. I will use the language and concepts regularly in my consultations with Service to Science participants.”

Dash considers the webinars a work in progress. Evaluations of the webinar series that have already taken place are being aggregated, and Dash and her colleagues will use this information to continue to improve the webinar series. In the comments so far, participants and presenters acknowledged the inherent challenges of the webinar format—namely that “people can’t see one another and have a more difficult time interacting and networking than they would in person,” says Tehout Selameab, Service to Science Lead for the Central Resource Team. As participants become more comfortable using the interactive features of the technology, and networking electronically, this will become a bit easier in the future, according to Dash.

The Service to Science Virtual Evaluation Primer is also being adapted for use by tribal programs.

For more information on this SAMHSA initiative, contact the Service to Science [1] lead in your service area.