With regard to intended purpose, there are three fundamental types of evaluation. They can: render judgement, encourage improvement and generate new knowledge (Patton 1997).
Summative evaluation, accreditation, quality control and audits are examples of judgment-intended evaluations. They follow a deductive method by setting clear criteria and standards with which to judge performance, often using quantitative measures. Judgment-intended evaluation often is commissioned by external parties (e.g., donors), and typically is performed by external evaluators. Such evaluation could increase the credibility of an IEA process, given its impartiality and objectivity.
Formative evaluation, empowerment evaluation (Fettermann 1996) and outcome mapping (Carden 2001) are examples of improvement or development-minded evaluations. The central intent of this type of evaluation is making things better over time. Improvement evaluation is inductive, posing open-ended evaluation questions. Evaluators are often internal; the participants, including some of the primary users, conduct the evaluation. SWOT analysis (Strength, Weaknesses, Opportunities and Threats), TQM (Total Quality Management), EMS (Environmental Management Systems) and ISO 14001 employ the evaluation of improvement approach. Improvement-intended evaluation could increase the legitimacy of the IEA process, given its users’ perspective.
Such evaluations often are applied to cyclical activities, like the IEA process, where performance improvement is expected over time. This improvement can involve change in behaviour (e.g., improved communication) or change in the state of the environment (e.g., improved water quality).
Outcome mapping (Carden 2001) focuses on changes in human behaviour, values, skills and knowledge, and acknowledges the complexity and the life cycle of the outcome. Some outcomes (e.g., institutional transformations) need decades to fully develop.
Knowledge-oriented evaluation—exemplified by action research, case studies, lessons learned and policy recommendations—has been gaining attention recently because of its capability to generate innovative ideas and deep insights for the intended users. Emerging knowledge can improve a known process, and break new ground. Evaluators can be both internal and external, and the intended users are actively engaged all along. Knowledge-oriented evaluation can increase the saliency of the process, given its potential to generate new knowledge that the user needs for a pressing decision (Bernd Siebenhüner 2005).
In reality, elements of all three categories can be found in an evaluation. For practical purposes it is important to select and focus on one dominant approach from the onset. As an IEA process intends to influence the policy and decision making processes, which generally happen in predictable cycles, a predominantly improvement-oriented evaluation is recommended.
Discussion Questions 1
- Why do you need to plan for monitoring and evaluating your IEA process and its impact at the beginning of the planning process?
- Why is improvement-oriented evaluation relevant to your IEA process?
Having decided on the intended purpose of your evaluation, the next step is to clarify who has interest in using the evaluation findings (users), and who will eventually implement the monitoring and evaluation (evaluators).