Types of Evaluation
Evaluations fall into one of three categories:
- Process-based evaluation;
- Outcomes-based evaluation;
- Impact-based evaluation.
Choosing the most appropriate type of evaluation is guided by the availability of resources, and whether the evaluation is needed for internal or external purposes. (See Who Should Evaluate? in the Resources Sidebar.)
Process-based evaluations are used to understand how a program works and the way it delivers its results. They assess the activities that are being implemented, the materials that are used, and the people reached by the program. Process evaluation also has an explanatory purpose, because it can identify reasons why a program may not have succeeded.
Process-based evaluations are intended to answer the following questions:
- What is required to deliver the program in terms of resources, products, and services?
- How are individuals implementing the intervention trained?
- How are participants selected and recruited?
- What are considered to be the program’s strengths and weaknesses?
- What is the feedback from participants / partners about the implementation of the program?
Outcomes-based evaluations describe a program's achievements and are used to measure any immediate or direct effects on participants. They help to establish that these changes have occurred in response to the program.
For example, do the participants in a particular campaign know more after being exposed to it than they did before? Have their attitudes changed? Can any immediate effects on their behavior be observed?
Outcomes-based evaluations focus on the following questions:
- Which outcomes are being measured (behavior change or change in knowledge or awareness) and why?
- How will these outcomes be measured, specifically?
- Do the participants know more about the targeted subject area than before?
- Have attitudes changed?
- Can any immediate effects on participant behavior be observed or measured?
- What is the desired proportion of participants who will have undergone a change as a result of the intervention? Has this number been reached?
Impact-based evaluations look beyond the immediate effects of a program on participants and identify longer-term effects, as well as any unintended or unanticipated outcomes. The most successful type of impact-based evaluation tracks impact over extended periods of time, rather than simply examining conditions immediately before and after the intervention was implemented.
To be successful, outcomes- and impact-based evaluations require the following:
- Detailed information on the indicators that can be used to measure the desired outcomes;
- A thorough assessment of how best to gather the necessary information (see Data Collection);
- A reliable and rigorous method for analyzing and reporting findings (see Data Analysis and Interpretation).
Be aware that there can be moderating variables or confounding variables to consider that may complicate analysis of the relationship between exposure to your program and the outcomes being measured. Moderating variables are variables that change the magnitude (stronger or weaker) of the relationship between two other variables. Confounding variables can obscure or confound the relationship between the exposure and outcome variables.
Both outcomes- and impact-based evaluations require information about the conditions before and after the intervention was implemented and can be further enhanced by including a control or comparison group against which to measure the “exposed” group (i.e., the one that has received an intervention).
It is desirable to incorporate process, as well as either outcomes- or impact-based evaluation components into your program design and implementation. Monitoring the implementation process yields important data that may be used in evaluating program impact or assessing whether or not program objectives have been achieved.