Indicators are what make data relevant for society and for policy making. They help us make decisions or plans because they help us understand what is happening in the world around us. As a society, we tend to choose measures that reflect our values. On the other hand, the information we receive also shapes what we value.
Indicators have an important role in both informing and assessing policy (UNEP 1994). The World Bank (1997) stated that, “The development of useful environmental indicators requires not only an understanding of concepts and definitions, but also a thorough knowledge of policy needs. In fact, the key determinant of a good indicator is the link from measurement of environmental conditions to practical policy options.” Practical policy options imply a relationship between environmental and societal affairs. As any decision has a price, whether it is environmental or social, a policy’s impact ultimately depends on the priority of the decision-maker as influenced by the perceived priorities of that person’s constituents. Thus, the integration of policy areas must provide a solid platform for supporting the path toward sustainable development (Gutierrez-Espeleta 1998).
The value of indicators in policy making can be summarized as:
- providing feedback on system behaviour and policy performance;
- improving chances of successful adaptation;
- ensuring movement toward common goals;
- improving implementation; and
- increasing accountability.
Selecting good indicators
Because indicators influence decision making, it is important that the measures we use are proper ones. Poor indicators provide inaccurate and misleading information about what is being measured. An example of a poor indicator might be a measure that reflects change over a very long time scale when decision makers require knowledge about change over in a short time scale. In order to know the impact of fertilizer on land quality, it would be insufficient to measure and present just the soil organic matter, which changes on a decade long time scale. Inaccurate indicators could lead to policy actions that are over or under-reactive.
One of the challenges of selecting good indicators is that it may be easier to choose indicators based on ease of measurement or data availability, rather than what needs to be measured. As mentioned previously, filling data gaps can be a resource intensive process, which means that options in terms of indicator selection may be limited. Notwithstanding, it is still valuable for you to select indicators that have the best possible fit with the IEA process.
Part of the process of selecting good indicators is weighing them against a set of indicator criteria. Selecting indicators can be a balancing act, with trade-offs among such factors as ensuring they are relevant to society and policy-makers, scientifically sound and accurate, and easy to interpret with a reasonable degree of accuracy and precision.
The following criteria, drawn from the World Bank (1997) and OECD (1993) are commonly cited as useful in the indicator selection process.
- be developed within an accepted conceptual framework;
- be clearly defined, easy to understand and interpret, and able to show trends over time;
- be scientifically credible and based on high-quality data;
- be policy relevant;
- be relevant to users, politically acceptable and a basis for action;
- be responsive to changes in the environment and related human activities;
- provide a basis for international comparison by providing a threshold or reference value;
- be subject to aggregation (from household to community, from community to nation);
- be objective (be independent of the data collector);
- have reasonable data requirements (either data that are available or data that can be collected periodically at low cost); and
- be limited in number.
An important consideration is selecting the appropriate number of indicators. Too many indicators may create “noise” that is difficult to interpret, while too few indicators limit the scope of understanding. Selecting indicators based on a select set of priority issues is an increasingly common way of limiting the number of indicators.
|Figure 7: Linking values, issues,
indicators and performance
criteria in a participatory process
(Pintér, Zahedi and Cressman 2000).
Because indicators are intended to help inform decisions that affect society, indicators better serve society when they reflect the diverse perspectives held by multiple stakeholders, such as citizens and citizen groups, private and public sectors, and policy-makers. As shown in the following figure, participatory processes occur across the spectrum of indicator development, from an initial identification of broadly-held values and issues that inform indicator selection, to more focused tasks of setting indicator targets and criteria for performance.
An additional step not shown in Figure 5 is the process of communicating indicator results with stakeholders, and understanding how they interpret the results in relation to values and their world-views. Developing an effective participatory approach requires careful planning so that the people who need to be involved are involved in an appropriate way, taking into account available resources (See Section 2).
|Figure 8: DPSIR framework for
Source: DEWA 2006).
Indicators are developed based on priority issues. The orientation of indicators to issues as well as relationships among indicators (such as cause and effect relationships) is often structured using conceptual frameworks. In an IEA and in GEO, the conceptual framework is the Drivers – Pressure – State – Impacts – Responses (DPSIR) framework, which shows relationships between human activity and ecosystem well-being, as introduced in detail in Modules 1 and 5. The DPSIR framework used in GEO-4 is shown in Figure 6. The DPSIR framework is a variant of the Pressure – State – Response (PSR) framework originally developed by Rapport and Friend (1979) for Statistics Canada and also adopted by the OECD. Variations on the DPSIR framework include Driving-State-Response (DSR), which was originally used by the UN Division for Sustainable Development (UN-DSD), and the Pressure-State-Response framework used by Statistics Canada and the OECD.
The UNEP Human – Environment Interaction analytical approach: – built on the Drivers, Pressures, State and trends, Impacts and Responses (DPSIR) framework. It is multi-scalable and indicates generic cause and effect relations within and among:
- DRIVERS: The drivers are sometimes referred to as indirect or underlying drivers or driving forces and refer to fundamental processes in society, which drives activities having a direct impact on the environment;
- PRESSURES: The pressure is sometimes referred to as direct drivers as in the MA framework. It includes in this case the social and economic sectors of society (also sometimes considered as Drivers). Human interventions may be directed towards causing a desired environmental change and may be subject to feedbacks in terms of environmental change, or could be an intentional or un-intentional by-products of other human activities (i.e., pollution);
- STATE: Environmental state also includes trends, often referred to as environmental change, which could be both naturally and human induced. One form of change, such as climate change, (referred to as a direct driver in the MA framework) may lead to other forms of change such as biodiversity loss (a secondary effect of climate gas emissions);
- IMPACTS: Environmental change may positively or negatively influence human well-being (as reflected in international goals and targets) through changes in environmental services and environmental stress. Vulnerability to change varies between groups of people depending on their geographic, economic and social location, exposure to change and capacity to mitigate or adapt to change Human well-being, vulnerability and coping capacity is dependent on access to social and economic goods and services and exposure to social and economic stress; and
- RESPONSES: Responses (interventions in the MA Framework) consist of elements among the drivers, pressures and impacts which may be used for managing society in order to alter the human – environment interactions. Drivers, pressures and impacts that can be altered by a decision-maker at a given scale is referred to as endogenous factors, while those that can’t are referred to as exogenous factors.
The following is an example of how the DPSIR framework can be used to tell a story about an issue, such as the “state” of urban air quality.
Another type of framework is based on capital accounting. This framework focuses on changes in in physical, natural, human or social capital. The goal of this model, which is in use by the World Bank, is to ensure that “future generations receive as much or more capital per capita than the current generation” (World Bank 1997).
Types of capital include:
- physical capital – buildings, structures, machinery and equipment, and urban land etc.;
- natural capital – renewable and non-renewable natural resources;
- human capital – e.g., return on investment in education; and
- social capital – norms and social relations, social cohesion.
|Figure 9: Example of DPSIR
framework for urban air quality
Capital accounts first must be tracked and may also be reported in physical units. Using physical measures helps reduce ambiguities, but it leads to indicators being reported in different units, thus assessing overall progress and comparison between different jurisdictions is often difficult. As an optional subsequent step, some or all forms of capital may be converted into a monetary equivalent. This may help with aggregation, but economic valuation methods related to non-market goods and services is wrought with challenges, particularly when one intends to apply it consistently across a wide range of social and ecological issues, over large geographic areas, and regularly over time (Hardi and Muyatwa 2000).
Further methodological aspects on economic valuation are discussed in Module5.
Flow of indicator development
Indicator development often begins with a conceptual framework, followed by the selection of indicators based on criteria of suitability. Indicator development is often an iterative process, where a large number of environmental or sustainable development issues are narrowed down in successive rounds of dialogue with stakeholders and experts to a few high-level measures.
|Figure 10: Example of an indicator
development process from South
Source: Palmer Development Group
Figure 10 provides an example of the process used for indicator development in South Africa. The main steps are further described below.
Step 1 involved identifying a framework to guide the selection of indicators. The framework was based on a review of environmental and local government legislation, and consultation with stakeholders. It was built around core environmental mandates for local government, and if a core mandate was not present, then around the role of provincial and national government.
Step 2 involved drafting a set of indicators based on a set of criteria for indicator selection. The draft set of indicators was reviewed by local, provincial and national government, to ensure that the new indicators would have as consistent a format and language as pre-existing indicators. A workshop was then held to obtain feedback from stakeholders.
Step 3 involved further categorizing the indicators. Because municipalities and provinces across South Africa manage areas with different characteristics, and with different levels of resources, capacities, knowledge and available data, further categories were needed to reflect these differences. The indicator categories were then placed within the indicator framework.
Towards the end of the project, a workshop was held with stakeholders for three purposes: to finalize the draft set of indicators, to categorize the indicators into proposed sets, and to discuss issues related to the use of indicators by government. The workshop resulted in a draft set of categorized indicators and a number of recommendations from stakeholders directed towards the government department responsible for indicator reporting.
Core indicator sets
Once indicators have been identified, you can further reduce them into core and peripheral sets of indicators. Core or headline indicators provide clear and straightforward information to decision-makers and civil society on trends and progress for specific issues. Few in number (<30), core indicators are sometimes clustered around themes, parameters or dimensions to assist with understanding more complex situations. They do not, however, provide a comprehensive picture of the situation, including detailed relationships among different aspects being measured. More detailed, supporting indicators may be included in a peripheral set to provide a higher level of detail.
Several “core data/indicator” sets have been developed, mainly differing by geographic scope (i.e., country, region, global). Examples are the OECD Key Indicator Set, the EEA Core Data Set, the EU Structural Indicators, the GEO Core Data Matrix and the UN CSD Theme Indicator Framework. There are several other global core sets of environmental and sustainable development indicators, but the common aspect among all of those initiatives is that they attempt to model reality according to a previous agreement among stakeholders.
Core sets of indicators can also be defined at the regional level, such as the Latin America and Caribbean Initiative for Sustainable Development (ILAC)2. This Initiative has six themes, 26 goals and 32 indicators, and is still under revision by national focal points. Others regions also have core sets of indicators such as OECD countries, and NAFTA countries, among others (ILAC 2006). Among national initiatives the headline indicator system of the UK may serve as a useful example, particularly because of attempts to establish direct links between indicators and the country’s sustainable development strategy. Linking indicators to mainstream policy mechanisms and instruments, such as strategies, integrated development plans or budget processes helps realize the potential of indicators as pivotal decision-making, learning and information tools.
Example: Core indicator set
As shown in the GEO Core Indicator Data Matrix, the GEO Core Indicator Set is based on a series of theme areas that reflect global issues and trends for selected environmental issues. These theme areas include:
- fresh water;
- coastal and marine areas;
- urban areas;
- socio-economic; and
Each year, the list is updated with new indicators, based on the rise and fall of the importance of global issues. Amidst efforts to ensure data are collected using environmental monitoring, surveying and remote sensing, there remain many data gaps. Some examples of these gaps include waste disposal and management, land degradation and urban air pollution (UNEP 2006). Table 1 describes broad themes, issues and provides detailed information about data variables, lead indicators and lead sources for the data. The first section of the framework is shown in the text below, and the remainder of the framework is provided in Appendix A.
Table 1: GEO core indicator data matrix (Large table - Click here to view)
Example: UN-DSD Indicator Framework
In 1995, the UN Commission on Sustainable Development (then part of the UN Division for Sustainable Development) approved a work programme on indicators of sustainable development, in response to Chapter 40 of Agenda 21. The work programme included a list of approximately 130 indicators organized in the DSR framework. These, along with their corresponding methodology sheets, were published by the United Nations in 1996 in what became widely known as the first ‘Blue Book’. The indicators were tested in a number of volunteering countries. As a result, the number of sustainable development indicators in the core set was reduced to 58, and the DSR framework was replaced by a thematic framework, organized along the four “pillars” (social, environmental, economic, institutional) of sustainable development. The results were published in the second Blue Book in by UN-DSD in 2001.
In 2005, UN-DSD began a second review of its indicators, which was completed in late 2006 and resulted in a further streamlined and updated set of indicators (UN Department of Economic and Social Affairs, Division for Sustainable Development. 2006). Agreement has been reached to reduce the core set further, to retain the thematic approach, and to divide indicators among the four pillars, since it masks relationships among the issues. For the most up to date listing of the indicators, consult UN-DSD website: http://www.un.org/esa/sustdev/natlinfo/indicators/isd.htm.
Table 2: CSD Theme indicator framework from the UN Division for Sustainable Development, 3nd Edition (2007) (Large table - Click here to view)
See Exercise 4.4.1
2. Approved in November 2003 by the Forum of Ministers of Environment of Latin America and Caribbean.