1. An independent evaluation of the five-year $25 million Millennium Ecosystem Assessment (MA) Project was carried out on behalf of UNEP in the first half of 2006. This report describes the evaluation approach and findings.
2. The objective of the MA was to assess the consequences of ecosystem change for human well-being and the scientific basis for actions needed to enhance the conservation and sustainable use of those systems and their contributions to human well-being. The MA consisted of an assessment of ecosystem services both at a global level and at a sub-global level through local, national and regional studies; the SGAs. Primary users were to be the international ecosystem-related conventions, regional institutions, UN agencies, national governments, civil society and the private sector.
3. The MA Project was coordinated by UNEP in partnership with other agencies. The original project budget was US$ 20.8 million plus $4 million project development funding. $7.0 million was provided by the GEF through UNEP, $4.2 million by UNF, $2.4 million by the Packard Foundation, $1.5 million by the World Bank and $0.8 million by UNEP. In-kind contributions were $7.3 million. Originally planned to run for four years to March 2005, the project was extended by 6 months to September 2005. Completion is expected in 2006.
Overall Findings
4. The MA was a highly complex and challenging project to design and implement on a global scale. There are many important positive aspects to the Project as well as some weaknesses. Most of the weaknesses are attributable either to strategic choices made during the Project design phase or to resource and time constraints that emerged during implementation. Project implementation and management were generally very effective. While it is too early to assess the impacts of the Project, the progress made towards most of the Project's major objectives and intended outcomes can be assessed:
Preparation and Design
5. High quality preliminary work under an Exploratory Steering Committee set the direction and engaged reasonably broad support for the MA. The decision to set the MA's technical objective as assessing the capacity of natural systems to support humanity proved both innovative and far sighted. Engaging the global scientific community to address this issue was critical to ensuring that the findings would be authoritative and credible.
6. A key decision was not to carry out the MA through an official inter-governmental process. This helped the Project engage more than 1,400 scientists and experts to carry out the assessment, virtually all of whom worked on a voluntary basis, an extraordinary contribution from the scientific community. Other benefits from working outside an inter-governmental process were the opportunities to engage private sector and civil society organizations in key decision-making roles on the MA Board, as well as greater autonomy and flexibility for the Project. Some important disadvantages from working outside an inter-governmental process included (i) a significant lack of awareness or engagement by political actors in both developed and developing countries, and (ii) a contribution to the present uncertainty over what should happen next now that the MA has been completed.
Major Achievements
7. The Project has achieved some clear successes:
• The MA has produced a series of credible, authoritative and high quality reports, with a very considerable volume of material well packaged for different audiences at varying levels of complexity.
• The MA emphasis on ecosystem services and their significance for human well-being is widely recognized as having made a major contribution to linking biodiversity conservation with poverty mitigation.
• The MA Conceptual Framework is widely regarded as an innovative and excellent technical analysis that seems likely to have a significant impact on the direction and approach of future applied research, which in turn may lead to more effective ecosystem management decisions and policies.
• The MA responded to and has successfully engaged the secretariats of the CBD and Ramsar.
• The level of interest in carrying out sub-global assessments (SGAs) as well as the number of SGAs actually undertaken (34) far exceeded expectations. Many of these SGAs are still continuing.
• The MA led to the emergence of a genuine global community for multi-scale ecosystem assessment that had not existed previously.
• The Project's capacity building goals appear to have been largely met.
• The MA and its implications are being discussed by various OECD government agencies and may be adopted in various forms. The MA also seems likely to have an impact on future GEF programming.
• Exceptionally able leadership was provided by the Project Director, with strong support from both the Board and the secretariat staff.
• All of these factors have contributed to keeping biodiversity conservation and ecosystem management on the international policy agenda.
Weaknesses
8. The Project's successes are mitigated by some significant weaknesses:
• There is little evidence so far that the MA has had a significant direct impact on policy formulation and decision making, especially in developing countries.
• The Project objectives call for the MA to be used in management and policy decisions, and anticipate the development of ‘implementation strategies'. Problems with these objectives include: (i) policy and decision makers were not a part of the MA process; and (ii) the MA has not produced tools, models or methods that can readily be applied by practitioners in the field.
• The lack of specific policy guidance in the MA has contributed to uncertainty on what should happen next and who is supposed to do what with the MA findings.
• Adequate financial resources were not available for communications and outreach after the assessment's major products were released starting in 2005.
• Few developing country SGAs were adequately funded. The quality of SGA products has been variable and most did not connect effectively with the global assessment. Relatively few of the SGAs engaged with local or national decision makers.
• The objectives, outcomes and initial expectations of the MA were probably too ambitious for a four-year project, even allowing for a six month extension.
• One year after the Project's major outputs started to become available, it is not clear what, if anything, should happen next.
Role of UNEP
9. UNEP was the GEF implementing agency and provided overall coordination for the MA Project. The agency played a relatively hands-off role, leaving the project leadership and secretariat to be relatively autonomous. UNEP deserves considerable credit for adopting more of a partnership than an oversight role.
10. The Project Director was hired as a senior UNEP staff member. Other UNEP staff, especially from DEWA, were involved in the partnership of organizations that planned the MA Project and then played a full and constructive role on the MA Board. UNEP has moved ahead and i nternalized some key elements of the MA approach into its own GEO process. Technical staff inputs from UNEP to the MA process were relatively limited, however.
11. Neither UNEP nor the other main Project partners appear to have taken any specific action to address the major weaknesses identified in the Project.
Worthwhile Use of Funds?
12. T he MA seems more likely to influence research agendas than policy agendas, depending on whether the links that the MA has highlighted between ecosystem management and human well-being are translated into tangible projects and programs. A categorical answer on the value of the MA Project is impossible to provide, as a lot depends on what happens next in terms of the MA's influence and impacts. Some of these impacts will happen spontaneously as more people and organizations become aware of the findings, some will happen through individual promotion or use of the MA by the many participants familiar with the process, and some will depend on so far unspecified plans to follow-up or possibly repeat the MA in some form.
13. The shortcomings documented here should not diminish the overall Project performance and the immense effort by many partners that went into moving this complex initiative forwards. As an innovative and largely unprecedented undertaking, the MA faced considerable uncertainty regarding how far the process could be taken or the level of impact generated, neither of which could be reliably predicted in advance.
Follow Up Activities
14. Various follow-up options have been considered by the MA core team and partners. These include: (i) further outreach and communication to ensure that the MA's findings and messages reach as broad an audience as possible; (ii) the production of a report focused on the MA's methodology; (iii) training and capacity-building on the MA's integrated ecosystem assessment approach; and (iv) continued coordination of the SGAs that are still underway. These potential activities all appear to have considerable value.
15. Some MA Board members have called for the assessment to be repeated at regular intervals, following the IPCC example.
16. The MA mid-term evaluation raised several key questions for any decision about the future of the MA, and these remain relevant:
• Should the MA continue, in some form, beyond its current assessment?
• If the MA is to continue, what form should it take?
• What relationship should future MA activities have to other organizations?
• Should the MA remain a multi-stakeholder process or become more intergovernmental?
17. The current unavailability of working models that can readily be used by policymakers to analyze ecosystems services and their trade-offs with development policies and resource allocations constrains the MA's potential for influencing environmental trends on the ground. Translation of the MA into operational methodologies and tools that will support decision making and policy setting seems absolutely critical, even though it is not clear at this point who should do this or how. The MA emphasis on ecosystem services and trade-offs and their links to human well-being have been welcomed by the conservation community as a bridge to development efforts focused on poverty mitigation, in other words making biodiversity more relevant to the needs of society. But the real test will be whether the international development community starts to take up and utilize tools and methods based on the MA approach, and when governments and private firms start to use these tools and methods to guide their investments.
Major Recommendations
18. Immediate priorities for follow-up to the MA are:
• An MA communications and outreach effort that engages more effectively with decision and policy makers, especially in developing countries.
• Using the MA findings to develop sets of operational tools and methods that can be adopted and applied by practitioners.
• Training potential users of these tools and methods, and implementing case studies to demonstrate their value and broader applicability, especially in developing countries.
19. These steps appear vital to maintaining the momentum of the MA and we would not anticipate they should collectively involve an investment of much more than $1 million, which does not seem excessive if it were to significantly enhance the impact of a $20 million project.
20. We encourage the MA stakeholders to develop and assess options for repeating the MA in some form in several years time, ranging from a full-scale repeat to a briefer, less expensive exercise focusing on particular topics related to the MA.
