Included in the policy instrument scan as described in this module is an analysis of policy effectiveness. Before talking about policy effectiveness, we define what is meant by policy effects.
The European Environment Agency (EEA) notes that the effects of policy responses imply causality between policy and driver, pressure, state or impact. The intended and unintended effects can be determined based on scientific and social observation and analysis. Such an analysis as described by the EEA should be judgment free.
Determining the effect of an individual policy on a driver, pressure or an environmental state can be challenging because of the complexity of the causal chain of drivers and pressures for a range of environmental, social and economic issues. It is often easier and more accurate to attribute the change in a driver, pressure or an environmental state to a mix of policies, given that attribution is so difficult to establish for a single policy (refer to Figure 26).
That being said, analysing the effect of an individual policy or a small set of policies on a specific issue is not impossible. It often is just a matter of time and computational effort. Section 6.9 of this module presents an example of an advanced policy analysis related to SO2 emissions in the Netherlands.
|Figure 26: Effects of a mix of policies
and policy instruments on an
Different from policy effect, the EEA describes policy effectiveness as going beyond analysing effects, and judges how the actual effect measures up to the policy objective. This is a performance assessment of the policy.
In order to carry out an analysis of policy effectiveness it is necessary to identify performance criteria. In an ideal case, performance criteria and the requirement for a performance assessment are built into policies, and the criteria are easily associated with routinely monitored indicators and cited targets. If this is the case, assessment is relatively straightforward, assuming that both the indicators and criteria appropriately represent the effects of the particular policy.
Often, policies are designed either without clearly defined and specific performance criteria, or with criteria that are not necessarily related to environmental performance. This is often so for economic policies related to taxation, trade or investment. Although these may have very significant links to environmental issues—in fact they may be the key drivers of environmental change—their built-in evaluation criteria are usually limited to economic performance. This makes their evaluation particularly challenging from an environmental and sustainable development perspective.
Performance criteria can range from general and descriptive (for example, whether a policy is in compliance with broadly defined principles), to specific and quantitative (for example, a target value associated with a specific indicator on a specific time scale). In essence, they provide a basis for comparison between planned or desirable performance and actual performance. Table 10 provides examples for some main types of performance criteria.
Table 10: Main types of performance criteria for policies (based on Pintér, Zahedi and Cressman. 2000)
|Type of criteria
||Comparison with a documented best-case performance related to the same variable within another entity or jurisdiction. The policy is evaluated based on its impact in a given jurisdiction compared with conditions in the benchmark or reference jurisdiction.
Example: highest percentage of households connected to sewage system in a comparable jurisdiction.
||The value of a key variable that will elicit a fundamental and irreversible change in the behaviour of the system. The policy is evaluated based on its role in making the system move toward or away from the threshold in any given period.
Example: maximum sustainable yield of a fishery.
||A broadly defined and often formally accepted rule. If the definition of the
principle does not include a relevant performance measure, the evaluator should seek a mandate to identify one as part of the evaluation.
Example: the policy should contribute to the increase of environmental literacy.
||Nationally and/or internationally accepted properties for procedures or
environmental qualities. The policy is successful if it helps keep performance within specified limits.
Example: water quality standards for a variety of uses.
||Determined in a political and/or technical process taking past performance and desirable outcomes into account.
Example: official development assistance shall be 0.4 percent of national GNP.
State of energy access for rural households in India
The energy sustainability gauge
Source: The Energy and Resources Institute http://www.teriin.org/ee/gbr/fesa/fesa_report.htm (see Section 3)
|Figure 27: Energy sustainability gauge
for India showing energy access.
Indicator: Percentage of rural households with access
to electricity (Census of India)
Reference level: 100% by 2012 Government of India target.
Progress rate: Actual annual rate of growth
since 1991 is only 46% of that required to meet the
objective by 2012.
There is ahigh level of policy implementation,
relying mostly on subsidized tariffs and government
sponsored electrification schemes. The ineffective
targeting of subsidies has been, however, a cause of
concern both from the point of view of equitable
access as well as financial implications for the
government. Ongoing structural reforms and the
setting up of regulatory commissions are seeking to
address the issue.
Indicator: Percentage of rural households using electricity for fuel and lighting purposes as estimated by the Census of India.
Table 11: Policy mix directed at the issue (2002)
||Subsidized rural electricity tarifs
Subsidized loans under the Accelerated Rural Electrification Programme, REC, etc.
||Various central government schemes, e.g., The Minimum Needs Programme, the Kutir Jyoti
Programme, the Prime Minister’s Gramodaya Yojana
||Tariff Orders of State Electricity Regulatory Commissions
||The Rural Electrification Corporation
|Figure 28: Per cent of rural households
using electricity for fuel and lighting.
Performance Criteria: The Government of India target is 100 per cent electrification by 2012. Of the balance, the Tenth Plan proposes to cover all 62 000 villages that can be electrified through grid extension. The remaining 18 000 remote villages are to be electrified by 2011–12 through the use of non-conventional technologies (MoF 2002-03).
Analysis: Only 44 per cent of rural households have access to electricity, compared to 88 per cent of urban households. Out of the 597 258 inhabited villages in the country, about 80 000 are yet to be electrified, going by the current definition of village electrification. 32 Ten states have declared 100 per cent electrification of their villages (Tenth Five-Year Plan).
As indicated in the gauge, there is a high level of implementation of policies directed at the provision of electricity access in rural areas. Policies have mostly relied on subsidized tariffs and government-sponsored electrification schemes. Progress is, however, not as encouraging as indicated by the low level of access. Though statistics show that 86 per cent villages are “electrified,” available data indicates that only 44 per cent of the rural households have access to electricity. There are thus inadequacies in the current definition of village electrification itself, which needs to be changed so that a village is considered electrified only if a minimum number of households in that village are provided with electricity connections (Tenth Five-Year Plan).
See Exercise 18.104.22.168