|
I am a program
coordinator and have located several potential evaluators for my program.
Do you have any suggestions about how I can select from among them?
When hiring an evaluator it is a good idea to use a competitive process.
Hire an evaluator just the way you hire other staff. Recruit candidates
and assess their qualifications. Here are some important things to keep
in mind during the hiring process and some questions you can ask to help
narrow your selection pool:
-
Evaluators have biases just like everyone else. Ask candidates
what they think about programs like yours. Do they think your program
is trying to do something important? Do they think what you are doing
is likely to be effective and why? Obviously, it is sensible to avoid
hiring someone with strong prejudices against your program.
-
It’s important to ask candidates about their experience in
evaluating similar programs. First, if they have conducted evaluations
of similar programs, you can examine the evaluation reports they prepared.
Second, if they have worked on similar programs, they probably know
things that will reduce the time needed to do the study—and save
you money. For example, they will be familiar with instruments used
to evaluate similar programs.
-
Programs can save money and exercise more control over the evaluation
by having program staff carry out some evaluation tasks. For
example, staff rather than the evaluator might administer a survey
to students. Ask candidates if they are willing to let program staff
work on the study.
-
Who controls and owns the study? People almost never ask this
question when they hire an evaluator. You should insist the evaluator
obtain your prior approval for all public dissemination of the results.
-
Ask about the balance of responsibilities among program staff,
funder, and others. The basic question is: For whom does the evaluator
work? To whom does he or she report the findings first? Does he or
she send the final evaluation report to the funder without prior review
and approval by the program?
-
You need a detailed budget for the evaluation that itemizes
major tasks. The budget should also show who will work on the tasks
and the percentage of time they will spend on the tasks. This is important,
since in many evaluation firms the senior investigator assigns much
of the work to research assistants
-
The evaluator should be comfortable with writing and presenting
for professional audiences of other researchers. You can easily check
this by looking at reports and articles he or she has written. It
is also important that the evaluator be able to describe the study
and the findings in language that lay audiences can understand
-
Check references. People almost always do this when they hire
staff, but sometimes overlook it when it comes to an evaluator.
-
Ask for an Evaluation Plan. Ideally, you should have
candidates prepare a written plan for the evaluation before you hire
them. To get a good plan, you should provide the evaluator with clear,
preferably written, information about the goals and objectives of
the program, how the program operates, the clients you serve or will
serve, and so on.
Related CAPT products:
-
CSAP’s Northeast CAPT also offers a training entitled "Locating, Hiring,
and Managing an Evaluator" that examines many aspects of
the evaluation process including those above. For more information
about this training view an on-line training description, contact
the Single State Agency in your state, or call CSAP’s Northeast
CAPT at 888-EDC-CAPT.
-
Visit our web site to view brief descriptions of select evaluation
groups in the northeast region. These can be accessed from our home
page at http://captus.samhsa.gov/northeast/northeast.cfm
by clicking on the appropriate state on the map.
Please contact the Northeast CAPT at captonline@edc.org
for more information.
|
