An index consists of multiple indicators combined into a composite or aggregated unit. In the course of your IEA you have the option of using some accepted aggregate indices, develop new ones or choose to focus only on discrete indicators. While the development of indices is a complex task, indices have the potential to attract decision maker and media attention. In choosing your strategy you need to consider not only the needs of your target audience, but also you capacity to effectively work with aggregates.
Indices make it easier for you to interpret complex information on a wide range of topics. Indices are often used to assess and compare performance against benchmarks or among performers, as this is easier than comparing several discrete trends. They are best used as a starting point for discussion and attracting public interest on an issue. You may also use them as a tool to inform policy, in which case it is paramount that the index is both well constructed and accurately interpreted (Nardo 2005).
There are several potential drawbacks of indices. If poorly developed and communicated, indices can relay misleading information or be misinterpreted, leading to inappropriate policy decisions. Also, because the scope of an index is broad, it may miss specific issues that would otherwise be revealed using discrete indicators. Finally, because an index is ideally based on the best available data and indicators, issues that do not have associated data may not be included in the policy making process.
The selection of indicators for use in the index involves the use of a series of criteria to ensure that appropriate indicators are selected. Select indicators that fit within the overall framework of the index, lend themselves to aggregation, are based on high quality data, and preferably do not highly correlate to each other, as this would amplify the effect of certain indicators within the overall index.
The construction of indices is similar to creating a mathematical model. Indicator data are standardized using statistical tools, such as converting values to a scale of 0–100, so that all indicators can be added together. Indicators are then weighted and combined into a single index. Because the development of an index involves several steps that can result in variations in the final outcome, credibility is greater when the methodology used is transparent and well documented.
A key step in the process of combining indicators into an index involves assigning relative weights to individual indicators. Indicators with a higher weighting more strongly influence the outcome of the index than those with a lower weighting. The decision about how to assign weights can be based on various factors including societal values and indicator relevance to policy, as well as more objective factors, such as the robustness of the data.
If weighting is being determined by societal values and policy relevance, you will need to consult with experts, representatives of the public and politicians to better understand diverse perspectives on the issues. You can ask participants in your assessment process to rank various indicators based on perceived importance, and assign a monetary value to the issues they think are important, or choose indicators using a process of comparison (decision support). As this is very subjective, the weighting could be subject to scrutiny or perceived relevance over time as societal values change.
If weighting is determined using more objective measures, one approach to consider is to base weighting on quality and amount of data. A downside of this approach is that indicators containing lower quality or a smaller amount of data are penalized, even if the indicator reflects an important and relevant issue.
Indicators may also be weighted equally, as this avoids some of the challenges presented in this discussion. This approach could be supported by consultations and statistical tools that show minimal differences among the indicators selected.
To more fully comprehend the message relayed by an index, it is useful to disaggregate the index into component indicators and categories. This gives you a more detailed analysis into specific patterns or to answer questions of decision-makers working in a more detailed context (Nardo 2005).
Example: Well-known indices
Gross Domestic Product
Gross Domestic Product (GDP) is a well-known index measuring the size of a country’s economy. A common approach to measuring GDP is to add together consumer expenditures, business investments in capital, government expenditures on goods and services and net exports (GDP = C+I+G+NX). While GDP is frequently used as a proxy for standard of living, it is not a true measure of standard of living because only economic activity is shown. A country may have high exports, for example, but a low standard of living because of other factors. Also, a major nuclear accident, natural disaster or marine oil spill will raise the GDP. There are also difficulties when comparing GDP among countries, as different calculations may be used.
Human Development Index
|Figure 11: Human Development
Index (UNDP 2005).
The key dimensions of the UN Human Development Index (HDI) are longevity, knowledge and standard of living. For longevity, life expectancy is used to generate a sub-index. Adult literacy and combined primary, secondary and tertiary enrolment ratio are used to generate educational attainment or knowledge sub-index. Adjusted income is used to create the standard of living sub-index. These three sub-indices are arithmetically combined to produce the HDI. More than 170 countries are ranked by this index. HDI also has helped UNDP to generate a family of related indices, such as the Human Poverty Index (HPI), Gender Related Development Index (GDI), and Gender Empowerment Measure (GEM).
Figure 11 provides a snapshot from the Human Development Index (UNDP 2005).
Environmental Performance Index
|Figure 12: Environmental Performance
Index framework (Yale Center for
Environmental Policy and Law
et al., 2006)
The Environmental Performance Index, released in 2006 by The Yale Center for Environmental Policy and Law, measures policy performance towards two goals: reducing environmental stresses on human health and promoting ecosystem vitality and sound natural resource management. Sixteen indicators express six categories of commonly agreed-upon policy categories: Environmental Health, Air Quality, Water Resources, Productive Natural Resources, Biodiversity and Habitat, and Sustainable Energy (Yale Center for Environmental Policy and Law et al., 2006). The index framework is shown in Figure 12.
Indicator performance is measured using a “proximity-to-target” approach, based on a core set of environmental outcomes linked to policy goals. Countries are also ranked and compared on an issue-by-issue basis. Criteria for indicator selection include relevance, performance orientation, transparency and data quality. The indicators were selected based on a review of environmental policy literature, expert judgment and policy dialogue in the context of the Millennium Development Goals. The targets, which are the same for all countries, were based on international agreements, international standards, national authorities or prevailing consensus among scientists (Yale Center for Environmental Policy and Law, et al., 2006). Indicator weighting occurs using statistical analysis.
See Exercise 4.4.2