Resources Related to Working with an Evaluator

American Evaluation Association
http://www.eval.org/find_an_evaluator/evaluator_search.asp [1]

The AEA has a searchable database of evaluators, firms and organizations involved in evaluation. The listing is voluntary on the part of the individual or firm; AEA provides this as a public service and inclusion does not constitute endorsement by AEA. The link is to a page from which you can browse all listings in alphabetical order by firm/organization name. In addition to the name, the firm’s location(s), a description of services provided, website link, and AEA members on staff with email addresses are provided. The database also can be searched by keywords or phrases and by the name of an individual or firm. Listings also can be generated for all entries from outside the U. S., all those within the U. S. and for entries for individual states.

Centers for Disease Control and Prevention. (2010). Learning and Growing Through Evaluation: State Asthma Program Evaluation Guide, Appendix D: Hiring an Evaluator. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Environmental Health, Division of Environmental Hazards and Health Effects, Air Pollution and Respiratory Health Branch.
http://www.cdc.gov/asthma/program_eval/Evaluation_Guide_COMBINED_FINAL_BOOKMARKED.pdf [2]

Although this is an appendix from an evaluation guide for state asthma programs, the information and guidelines are directly applicable or easily adaptable to evaluations in other settings. Further, although the title is Hiring an Evaluator, it covers the topic broadly and is a very thorough document. Section D.1 is a sample position description that includes general items under the headings of Principle Duties and Knowledge, Skills and Abilities. This is followed by very specific items organized around the six steps of the CDC Evaluation Framework: engage stakeholders, describe the program, focus the evaluation design, gather credible evidence, justify conclusions, and ensure use and share lessons learned. Section D.2 provides guidelines for working with an external evaluator, including lead and support roles for program staff and the evaluator for each of the six steps. The final section, D.3 presents a list of competencies to look for in an evaluator arranged under four headings: professional foundations, planning and designing an evaluation, implementing the evaluation plan, and managing the evaluation. Additional resources are provided at the end of the second and third sections.

Education Development Center, National Center for Mental Health Promotion and Youth Violence Prevention. (2004). Hiring an Evaluator and Managing an Evaluator. Newton, MA: The National Center.

The National Center for Mental Health Promotion and Youth Violence Prevention has produced two guides relevant to the process of hiring and working with an external evaluator. Both guides are based on material from a web-based course, Locating, Hiring, and Managing an Evaluator designed and implemented by EDC’s Northeast Center for Application of Prevention Technologies. The original course no longer is available.
Hiring an Evaluator
http://www.promoteprevent.org/publications/prevention-briefs/hiring-evaluator [3]

This document provides advice on how to locate and hire an evaluator, whether the project is going to hire and put the evaluator on staff or contract out for evaluation services. Sections discuss when to involve an evaluator in a project, how to find an evaluator, screening and assessing candidates, key items to discuss when interviewing potential evaluators, and guidelines for managing interview process.

Managing an Evaluator
http://www.promoteprevent.org/publications/prevention-briefs/managing-evaluator [4]

Once an evaluator has been identified, expectations must be clearly defined and procedures put in place to help ensure a smooth process and a good working relationship between the evaluator and program management and staff. This brief provides guidelines with respect to several elements important to process, including: creating an evaluation plan, coming to an agreement on the scope of work and elements of the contract, establishing a timeline for reports and other deliverables, and adhering to an agreed upon payment schedule.

Rabinowitz, P. (2003). The Community Toolbox, Section 4. Choosing Evaluators. University of Kansas Work Group on Health Promotion and Community Development and AHEC/Community Partners.
http://ctb.ku.edu/en/tablecontents/section_1351.aspx [5]

The Community Tool Box website provides numerous how-to tools designed to help practitioners with the different tasks necessary for community health and development. The section on choosing an evaluator is part of a larger chapter on program evaluation. This section of the Introduction to Evaluation chapter provides a well-written discussion of around four questions: Why pay attention to the selection of evaluators? When should you choose your evaluators? How should you decide between professional and community or other volunteer evaluators? What should you look for in choosing evaluators? Also included are a short checklist and power point that summarize the major points contained in the text. A list of potential questions to be used during a consultant selection interview also is provided.

Rutnik, Tracy and Campbell, Marty. (2002). When and How to Use External Evaluators. Baltimore, MD: The Association of Baltimore Area Grantmakers.
http://www.irvine.org/assets/pdf/evaluation/when_how_external_evaluator.pdf [6]

This document was developed as a resource to help inform nonprofits and foundations about the process of contracting an external evaluator. It is based on a review of the literature and the experiences of the authors and colleagues with whom they have worked. Following a discussion of its evolving use by nonprofits and foundations, the authors address issues pertaining to where to find evaluators, developing and reviewing responses to a RFP for an evaluation, contract and scope of work, costs, and tips for managing the evaluator and the overall process (including signs that the relationship is failing).

Stufflebeam, Daniel L. (1999). Evaluations Contracts Checklist. Kalamazoo, MI: University of Western Michigan, The Evaluation Center.

This checklist is designed to be used by evaluators and their clients to help them identify important issues that should be addressed as part of contract development. Fifty-one elements are included under 10 categories: basic considerations, information, analysis, reports, reporting safeguards, protocol, evaluation management, client responsibilities, budget, and review and control of the evaluation. A wide variety of checklists related to evaluation are available at http://www.wmich.edu/evalctr/checklists/checklist_topics/ [7]

U. S. Department of Health and Human Services. Administration for Children and Families, Office of Planning, Research & Evaluation. (2010). The Program Manager’s Guide to Evaluation (2nd ed.), Chapter 4: How Do You Hire and Manage an Outside Evaluator? Washington, DC: The Office.
http://www.acf.hhs.gov/sites/default/files/opre/program_managers_guide_to_eval2010.pdf [8]

This chapter is a well-organized for working with an external evaluator. The steps in the process are clearly discussed and prescriptions are provided for how to find an evaluator, what to do if you have trouble hiring an evaluator, how to develop a contract and the basic elements that should be included, the respective responsibilities of the evaluator and the program manager, and what to do if problems arise.
 

Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of Prevention Technologies contract. Reference #HHSS277200800004C. For training and/or technical assistance purposes only.