SAMHSA's Service to Science Initiative: Starter Guide

This guide is for new and potential Service to Science participants, Single State Agency/National Prevention Network representatives, and other stakeholders interested in learning more about SAMHSA's Service to Science Initiative. The guide is organized around a list of questions and answers frequently asked about this initiative. If you have any further questions, please do not hesitate to ask!

General Information

What is SAMHSA's Service to Science Initiative?

Service to Science is a national initiative for locally-developed and innovative prevention programs interested in demonstrating more credible evidence of effectiveness. Established in 2005 by the Substance Abuse and Mental Health Services Administration (SAMHSA), the initiative's long-range purpose is twofold:

to increase the number and array of effective prevention interventions from which states and communities can select to address substance abuse and related mental and behavioral health problems or the underlying factors associated with increased risk; and
to support innovative local interventions seeking to strengthen their capacity to demonstrate and document evidence of effectiveness.

Service to Science consists of a combination of training events, customized technical assistance, and financial incentives for evaluation capacity enhancement. Each year, Single State Agencies and/or National Prevention Network representatives nominate a new cohort of participating programs. Pending availability of funds, these programs then participate in onsite training and technical assistance consultations and receive follow-up technical assistance tailored to meet their individual evaluation needs. Participating programs also have the opportunity to compete for a one-year subcontract of up to $30,000 to be used to enhance their evaluation capacity (more on this below). The mini-subcontract competition and awards also are dependent on the availability of funds.

Who implements the Service to Science Initiative?

Service to Science is implemented by SAMHSA's Center for the Application of Prevention Technologies (CAPT). SAMHSA's CAPT provides training and technical assistance to states, tribes, jurisdictions, and communities across the United States to prevent and reduce substance abuse and related mental, emotional, and behavioral health problems across the lifespan. Service to Science onsite consultations are offered by the CAPT's five resource teams (RTs), located in each of the CAPT's five service areas (see last page for map depicting these areas).

Nomination and Application Process

Who nominates programs for participation in Service to Science?

State Alcohol and Drug Agencies/Single State Agencies (SSAs) or National Prevention Network (NPNs) representatives nominate programs for participation in Service to Science. Each year, SAMHSA asks each state to nominate up to five programs to participate in the initiative. Programs interested in applying should contact their state's SSA or NPN and ask to be considered for nomination. Programs may contact the appropriate CAPT Service to Science Lead in their service area, whose contact information is provided at the end of this guide.

What kinds of programs are eligible for participation?

Locally-developed programs eligible for Service to Science are innovative, responsive, practice-based, mindful, and committed.

Innovative: Innovative interventions represent new services, practices, policies or programs distinguished by their creativity, originality, and utility. Innovative programs do NOT include existing evidence-based programs or combinations of existing evidence-based programs. However, existing evidence-based programs significantly adapted for specific population groups (e.g., defined by gender, age, race and/or ethnicity) or to address substance abuse problems with which they were not originally tested do qualify as innovative.
Responsive: Responsive programs address local substance abuse prevention and related mental health needs, as well as gaps in service to address those needs, as identified through the application of SAMHSA's Strategic Prevention Framework—a five-step planning model based on epidemiological assessment, capacity building, planning, implementation and evaluation.
Practice-based: Practice-based innovations are based on staff work or experience in the field; that is, programs developed with or informed by input from the target population and/or based on practical experience working with the target population in the setting in which the program is delivered. Indeed, when local organizations apply to Service to Science, they are asked to demonstrate that they are rooted in the community they serve.
Mindful: These are programs that address, via culturally focused or other responsive strategies, the disparities in access, quality, and outcomes of prevention programming for vulnerable populations that historically have been underserved or inappropriately served by the behavioral health system.
Committed: Programs participating in Service to Science are expected to commit considerable time and effort to improving their evaluation capacity. Based on our experience, programs that commit time to the Service to Science process are more likely to achieve progress in building their evaluation capacity. To this end, and pending availability of FY 2013 funds, selected programs are invited to participate in intensive customized technical assistance (TA) through a two-day onsite consultation. Next, programs may request up to 30 hours of TA from our expert Evaluation TA Providers between December 2012 and July of 2013.

SAMHSA Priorities
In addition to the above criteria, SAMHSA is interested in programs that address SAMHSA Strategic Initiative (SSI) #1: Prevention of Substance Abuse and Mental Illness, Goals 1—4. These goals include the following:

With primary prevention as the focus, build emotional health, prevent or delay onset of, and mitigate symptoms and complications from substance abuse and mental illness.
Prevent or reduce consequences of underage drinking and adult problem drinking.
Prevent suicides and attempted suicides among populations at high risk, especially military families; lesbian, gay, bisexual, transgender, and questioning youth; and American Indians and Alaska Natives.
Reduce prescription drug misuse and abuse.

For more information on the SAMHSA Strategic Initiatives, see: http://store.samhsa.gov/shin/content/SMA11-4629/01-FullDocument.pdf [1]

SAMHSA is especially interested in innovative prevention programs targeting populations disproportionately affected by substance abuse, or by substance abuse associated with trauma and mental health problems, and for whom few established prevention interventions have been developed. These populations include, but are not limited to, specific racial and ethnic groups (i.e., African Americans, American Indian/Alaskan Native, Native Hawaiian and Pacific Islander youth, and Latina youth); lesbian, gay, bisexual, transgender and questioning individuals; people with disabilities; women and girls; transition-age youth; military families; and persons who face economic hardship or live in health care workforce shortage areas.

Programs considered ineligible are those that have participated in a previous year of the Service to Science Initiative; or lack funding level sufficient to operate for at least one year.

How can programs apply to participate in Service to Science?

Programs nominated for participation will be asked to complete a Service to Science application. The application includes two parts: (1) a short form to collect contact and program information; and (2) a narrative section that asks applicants to provide the following information on which they will be scored (with the exception of their program evaluation plan):

Program Rationale: A description of the general problem(s) the program addresses; factors that contribute to the problem(s) as well as its consequences; how the program addresses contributing factors or consequences identified; gaps in services and programming for the identified problem(s); and the program's anticipated substance abuse, and other relevant, outcomes.
Core Program Elements: The program's essential approach, strategies, methods, products or practices delivered, as well as when, where, why, and to whom these elements are delivered.
Program Innovativeness: A description of how the program represents a new service, approach, practice, or policy that has not been implemented by others in the field; how it was developed based on work or experience in the field; how it is distinguished from others by its creativity; how it is distinguished from others in terms of its originality; and how it is distinguished from others in terms of its utility or feasibility of implementation. Alternatively, if the program is an adaptation of an existing evidence-based program for a specific population group or substance abuse or related behavioral or mental health problem, how it is new or has not been tried by others in the field; how it was developed based on work or experience in the field; how it is suitable to the target population; and the degree to which it is feasible to implement.
Program Management: A description of how the program is implemented (i.e., organization capacity) and who is responsible for implementation.
Program Evaluation Plan: An overarching description of how evaluation has been conducted in the past or how the organization plans to conduct evaluation. (This section is for information purposes only and will not be scored).
Service to Science TA Plan: A description of how the program plans to use the evaluation technical assistance (TA) available through Service to Science to strengthen its evaluation capacity.

Programs should submit their completed application to SAMHSA's CAPT. (Guidance on submission is provided in the Service to Science Application.) The CAPT will review all the applications it receives and select participants based on how well their application scores. Approximately one month after submitting their applications, programs will receive notification of their status—accepted or not accepted. Accepted applicants will also receive contact information for the Evaluation TA Provider assigned to work with them.

Working with Evaluation Technical Assistance Providers

Who provides the technical assistance?

Each program participating in Service to Science is assigned a technical assistance (TA) provider. These evaluation experts will work with program representatives throughout the Service to Science process, starting with pre-consultation assessment (e.g., application review, pre-consultation questions) and through to the end of the follow-up TA period. (If the program is awarded a subcontract, TA Providers will provide additional assistance to carry out the activities proposed for the subcontract. See below for more information on the SAMHSA Service to Science subcontracts.) TA Providers are matched to programs based on program needs, TA Providers' areas of expertise, geographic considerations, and availability.

Onsite Consultations

What are the onsite consultations?

Every year, and pending availability of fiscal year funds, SAMHSA offers a number of onsite consultations, geared toward programs in more advanced stages of evaluation planning. They consist of up to two-day-long, onsite consultations at the program's desired location, attended by a TA Provider and as many program representatives as deemed appropriate (typically up to five).

When and where do onsite consultations take place?

The TA Provider and program representatives collaborate to determine the date and location of the onsite consultation. Soon after program acceptance to Service to Science, the program's assigned TA Provider will contact the program to schedule the consultation. Pending availability of FY2013 funds, consultations should take place between November 2012 and February 2013. Site visits usually last between eight and sixteen hours (over the course of two days)—depending on need and readiness to address evaluation issues.

Who should attend the onsite consultations?

In addition to the TA Provider, up to six program representatives can attend the onsite consultation. It is up to each program to determine who these program representatives might be. We recommend that they include the person or people who developed the program, the person or people who are responsible for implementing the program, and the person or people who are responsible for evaluating the program. Single State Agency and/or National Prevention Network representatives who nominated programs for participation in Service to Science are also encouraged to attend the onsite consultations in their states, jurisdictions, or territories; however, they are not required to do so.

What happens at the onsite consultation?

During the consultation, the program's TA provider visits the program's home organization and meets with program representatives and stakeholders. The purpose of the visit is to identify key areas of program need, answer questions about the current status of program evaluation plans, and provide targeted TA to address specific evaluation concerns or issues. Three overarching questions typically guide the discussion:

Is the need for the program apparent and clearly stated? Do program personnel make a case that the program is needed based on research evidence or existing explanatory theories?
Are program components and mechanisms clearly explained? Can program personnel explain why the various components of the program were chosen and implemented, and how these components are likely to abate the problem(s) or bolster the asset(s) the program is meant to address? Is the program described in such a way that someone else could obtain the same results?
How do we know the program works? Can the program explain what would have happened to subjects in the absence of the program? Does the program account for the anticipated change in outcomes of interest or is it possible that some other factors might account for the change? Consider, for example, the reliability and validity of measures, bias introduced through attrition or selection into the program or comparison condition, appropriateness of analytical strategies, and the ability to account for potential confounders.

At the end of the onsite consultation, the TA Provider will prepare an action plan that summarizes key highlights from the meeting and identifies follow-up action steps agreed to by all attendees.

How can programs prepare for the onsite consultation?

STS participation begins with program representatives attending three separate webinars in which they explore what it means to become an evidence-based program, determine how program logic models guide evaluation planning, and examine why evaluation planning follows specific steps. These webinars will prime participants for their onsite consultations in which they work with their assigned TA Provider to develop a plan for enhancing program evaluation activities. TA Providers will have all the materials programs submitted as part of their applications. However, program representatives should feel free to bring to the consultation any additional materials they think would be helpful (they can also send these to their TA Provider prior to the meeting). These might include:

logic model
description of the program's evaluation design
instruments (surveys, interview questions, focus group protocols, etc.)
evaluation results
evaluation reports

Will the consultation follow a set agenda?

The TA Provider will work with program representatives before the onsite consultation to develop an agenda that will guide the meeting discussion. These agendas typically include:

Introductions
The purpose of the Service to Science Initiative and the onsite consultation
Different paths for obtaining evidence-based recognition
Program implementation
Program rationale
Program's evaluation strengths and challenges
Action steps that emerged from discussion

After the Onsite Consultations

Will there be any technical assistance available after the onsite consultation?

After the onsite consultation, programs are entitled to up to 30 hours of follow-up TA. During the consultation, programs will work with their TA Provider to develop an action plan that will guide the follow-up TA they will receive. Follow-up TA is a key element of the Service to Science process and is meant to continue the learning community experience after the onsite consultation. Therefore, programs are encouraged to take advantage of the continued TA offered to them.

Programs can access follow-up TA as soon as the consultation is over and until October 31, 2013.

What do programs need to do to receive follow-up TA?

To receive follow-up TA, programs can contact their assigned TA Provider directly. If program representatives have any questions or concerns about follow-up TA, they also should feel free to contact their Service to Science Lead (see contact information below). In addition, TA Providers will contact their assigned programs at fairly regular intervals to assess program progress on or desire to address items in their action plans.

What type of help does Service to Science provide?

Through follow-up TA, programs receive various types of assistance from their assigned TA Provider to meet each program's specific needs. Some examples of TA include consultation from TA Providers on how to:

articulate the rationale or theory of change of the program
design a program evaluation plan
develop or identify existing measures of outcomes of interest
analyze data collected for evaluation purposes

What type of help does Service to Science not provide?

Although Service to Science TA Providers are prepared to help with program evaluation, they do not actually conduct program evaluations or prepare materials as part of the evaluation process. For example, they do not collect program data, summarize evaluation results, or prepare evaluation reports. They are able to review and comment on materials, instruments or reports.

Will the results of STS participants' TA be reported to anyone?

TA Providers report on the types of follow-up TA they provide to their assigned programs. This information is shared with the nominating entity (SSA or NPN) to provide them with information about program progress. In addition, CAPT evaluators will contact program participants approximately 15 months after selection into Service to Science and ask about the progress of their evaluation capacity building efforts. However, SAMHSA will not ask for participant data collected as part of evaluation efforts.

The Service to Science "Subcontracts"

What are these subcontracts referred to above?

Pending the availability of funds, the subcontracts are funding opportunities offered by the CAPT, on behalf of SAMHSA, to Service to Science participants at the end of the Service to Science process. The purpose of the subcontracts is to provide financial support to Service to Science participants to increase their evaluation capacity.

Who is eligible to apply?

On behalf of SAMHSA, the CAPT will accept proposals submitted in response to a Request for Proposals (RFP) for the subcontract funding period in 2013 pending availability of FY2013 funds. Participants can only apply once, and only during the year they participate in the initiative (e.g., programs participating in the STS Initiative in 2013 will only be able to submit proposals in response to the RFP to be released in 2013). Proposals submitted after the deadline specified in the RFP will not be accepted.

How many subcontracts are available?

Pending availability of funds, SAMHSA's CAPT will award approximately 25 subcontracts to programs participating in a Service to Science onsite consultation. Five of these subcontracts will be set aside for Pacific jurisdiction-nominated programs and another five for programs nominated by SAMHSA's Native American Center for Excellence.

How do I apply for a subcontract?

SAMHSA's CAPT will send a pre-solicitation notice to all Service to Science participants approximately two weeks before the Request for Proposals (RFP) for Building Evaluation Capacity for Evidence-based Interventions is released. That pre-solicitation will, most likely, be sent to you in June 2013. The RFP will be sent to you in July. Programs will have about 30 days to complete their proposals. All proposals will be reviewed and scored by a panel of experts. Awards will be announced approximately two months after they are submitted.

How much funding do the subcontracts provide?

Pending availability of funds, programs will be able to request up to $30,000 for a full-year.

What kind of assistance can programs get to develop their proposals?

Service to Science TA Providers cannot directly help programs prepare their subcontract proposals. However, program representatives can use the assistance received from their TA Provider to inform the content of their proposals. For example, if a TA Provider helped a program design an evaluation plan, the program can describe this evaluation plan in its proposal.

What kind of assistance can a program get if it is awarded a subcontract?

Programs awarded subcontracts receive an additional 10 hours of follow-up TA by their TA Provider—likely the same TA Provider that programs worked with previously during the onsite consultation and follow-up TA phases.

How long does the funding last?

Subcontract funding lasts for one year.

If a program doesn't apply this year, can it apply next year?

No, programs are only able to apply during the same year they participate in Service to Science.

What kind of reporting requirements go along with the subcontract?

SAMHSA's CAPT will ask subcontract recipients to submit two reports: one half-way through the funding year and one at the end of the funding year. Subcontract recipients will be asked to report on the progress they are making implementing the evaluation plan they outlined in their proposals. Subcontract recipients will not be asked to provide data collected as part of their evaluation efforts!

For more information about the subcontract, contact:

Kim Dash
Chief, Service to Science Initiative
SAMHSA's Center for the Application of Prevention Technologies
Phone: 617.618.2425
Email: kdash@edc.org [2]

Matt Biewener
Service to Science Subcontract Administrator
SAMHSA's Center for the Application of Prevention Technologies
Phone: 617-618-2328
Email: mbiewener@edc.org [3]

Contact Information

For information about Service to Science, please contact the Service to Science Lead in your CAPT service area:

National	Southeast
Kim Dash CAPT Chief, SAMHSA's Service to Science Initiative Phone: 617.618.2425 Email: kdash@edc.org [2]	Adela Santana Service to Science Lead SAMHSA's CAPT Southeast Resource Team Phone: 678.954.5796 Email: asantana@edc.org [4]
Central	Southwest
Tehout Selameab Service to Science Lead SAMHSA's CAPT Central Resource Team Phone: 763.232.7237 E-mail: tselameab@edc.org [5]	Nicole Diehl Service to Science Lead SAMHSA's CAPT Southwest Resource Team Phone: 405.325.0347 E-mail: nicoled@ou.edu [6]
Northeast	West
Maria Valenti Service to Science Lead SAMHSA's CAPT Northeast Resource Team Phone: 518.878.0296 Email: mvalenti@edc.org [7]	Eric Ohlson Service to Science Lead SAMHSA's CAPT West Resource Team Phone: 775.682.8562 Email: eohlson@casat.org [8]