|
X. Evaluation Tools & Measures
G. Using Observational Data
One way to collect information is to observe the activities of project staff and participants. Observation is especially useful when conducting context and implementation evaluation because it may indicate strengths and weaknesses in the operations of your project, and may enable you to offer suggestions for improvement.
Information gathered through observation will allow you to:
- formulate questions which can be posed in subsequent interviews;
- examine the project's physical and social setting, staff and clientele characteristics, group dynamics, and formal and informal activities;
- become aware of aspects of the project that may not be consciously recognized by participants or staff;
- learn about topics that program staff or participants are unwilling to discuss; and
- observe how project activities change and evolve over time.
Despite its value as a strategy for data collection, observation has limited usefulness in certain situations. For example, observing certain events, such as medical consultations, would be inappropriate if the observation violates the confidentiality of the doctor-patient relationship. Other types of observation, although legal, could violate cultural values or social norms. For example, a male evaluator of a project serving pregnant women should not intrude visibly on program activities if his presence would be disruptive. If in doubt, it is a good idea for the evaluator to talk with the project director about those situations he or she would like to observe.
An evaluator should also recognize that even the most passive, unobtrusive observer is likely to affect the events under observation. Just because you observe it, do not assume that you are witnessing an event in its "natural " state.
With these considerations in mind, here are a few tips:
- Figure out how observation can be used to complement or corroborate the data you receive from other sources (e.g., surveys, interviews, focus groups). Establish goals for the observation, but be willing to modify them on site. You may even want to write some questions to guide your observations, such as: How do participants react to the project environment? Do they feel secure? Are staff members treated as equals? How do they address one another?
- Be practical. If your time is limited and the project you are evaluating is large or scattered over several sites, you will not be able to observe everything. Decide what is most important or what cannot be learned from other data sources and concentrate on these areas.
- Be systematic. Once you have established a focus (for example, staff relations), approach the matter from different angles. Observe the project at different times of day; observe a variety of different individuals; see if you can attend a staff meeting.
- Be prepared. Develop instruments for recording your observations efficiently so that you can concentrate on what is happening on site. If the instruments do not workwell, modify them.
- Be inquisitive. Set aside some time to discuss your observations with the project director or other staff members. This will help you to put the events you observed in context and gain a better understanding of what happened.
- Be open. If you do not expect the unexpected, you will not find it.
--Heraclitus, 5th century B.C.
Back
|
