This framework takes a look at key attributes that enhance the IEA report’s effectiveness in influencing policy-makers. The notions of saliency, credibility and legitimacy—as key attributes of effective assessments—arise from earlier academic research that focused on better understanding the factors that determine the effectiveness of assessments (Box 2; Figure 3).
Box 2: Attributes of Effective Assessments
Through a five-year consultative process involving hundreds of professional evaluators internationally, the American Evaluation Association identified four criteria for effective evaluation: utility, feasibility, propriety and accuracy (Patton 1997). A couple of years later, the Social Learning Group’s international research team arrived at a similar conclusion from a different point of departure, namely studying what makes environmental assessments effective, and what makes them utilized. The Social Learning Group found that the user (i.e., policy-maker’s) perception of the assessment’s saliency, credibility and legitimacy was critical (Farrell and Jäger 2005).
Figure 3: Corresponding Attributes of Effective Evaluations
The saliency-credibility-legitimacy attribute triad acknowledges that the process is subject to political interests. We do not suggest that the process should bend to those interests, but emphasize the need to attract political attention when legitimacy and credibility are not convincing enough on their own merits. It also implies that without credibility and legitimacy, political saliency is not enough to attract and maintain attention.
The assessment of stratospheric ozone depletion is a good example, because it was perceived by policy-makers as salient, credible and legitimate:
- Salient. Because it addressed a global threat to survival that called for immediate attention and action from decision-makers.
- Credible. Because it involved high-profile research institutions from different countries, triangulating their observations and results.
- Legitimate. Because of the transparent process, engaging all relevant stakeholders and acknowledging their investment.
The Social Learning Group’s findings highlight another important point, the importance of involving user representatives which, in the case of an IEA, means key policy-makers. Through their participation in the assessment, these decision-makers can develop a sense of saliency in addition to being assured of credibility and legitimacy.