CHAPTER 3 The Technocratic and Democratic Functions of the CAIR Regulatory Analysis – Reforming Regulatory Impact Analysis

CHAPTER 3

The Technocratic and Democratic Functions of
the CAIR Regulatory Analysis

NATHANIEL O. KEOHANE

Of the case studies considered in this report, the regulatory impact analysis for the Clean Air Interstate Rule (CAIR) provides particularly fertile ground for analysis and critique (EPA 2005a, henceforth “RIA”). The rule itself was far-reaching, mandating reductions of 60 to 70 percent in the emissions of two major criteria air pollutants—sulfur dioxide (SO2) and nitrogen oxides (NOx)— from power plants in the eastern United States. As a result, the stakes were very high. The annual costs of the controls proposed by the CAIR were projected to run into the billions of dollars—and yet they were dwarfed by estimated benefits of nearly $100 billion a year.

In the scale of its economic impact, as well as its proposed reliance on market-based policies, the CAIR was a coda to the immensely successful emissions trading programs for SO2 and NOx established by the 1990 Clean Air Act Amendments and the subsequent NOx Budget Program— programs that produced estimated benefits of $122 billion against costs of just $3 billion (Chestnut and Mills 2005). Since it was announced, the CAIR has been dogged by legal challenges, and its future remains uncertain. But the ambition and potential impact of the program make it a natural case study for the role of regulatory impact analysis in general and cost–benefit studies in particular.

This chapter is divided into two parts, following this introduction. The first part (the bulk of the chapter) offers a technical critique of the CAIR RIA as an exercise in applied cost–benefit analysis. In keeping with the themes identified in the Introduction to this report, I assess the performance of the RIA on a range of dimensions: the scope of alternatives considered; the estimation of costs and benefits, including the expression of benefits in monetary terms using willingness to pay; the consideration of equity and differential impacts among subpopulations; the discounting of delayed effects; and the treatment of uncertainty. The RIA is a more than competent example of cost–benefit analysis on a number of dimensions and should be praised for its innovative approach to considering uncertainty. However, I identify a number of areas where the analysis could have been substantially improved. I close the section with a set of recommendations for improving regulatory cost–benefit analysis.

The second part takes a broader view of the RIA as a public document. The starting point for the discussion is that a purely technical appraisal of the RIA is necessarily incomplete, because such a discussion presumes a formal role for analysis in guiding policy that the CAIR RIA lacked. As several commentators have pointed out, including Wendy Wagner in Chapter 4 of this report, the U.S. Environmental Protection Agency (EPA) did not have statutory authority to base its rule-making on cost–benefit grounds; indeed, the agency is expressly forbidden to use cost as a criterion in setting or implementing National Ambient Air Quality Standards under the Clean Air Act. Hence the CAIR RIA did not—could not—inform policymaking. As a formal matter, the RIA was carried out under Executive Order 12866, which requires agencies to submit an assessment of the costs and benefits of significant regulatory actions to the Office of Information and Regulatory Affairs (OIRA) in the Office of Management and Budget.

If the RIA could not offer guidance to policymakers, nonetheless it represented a potentially powerful means of informing citizens interested in evaluating those decisions after the fact. In contrast to the essentially technocratic function usually identified with regulatory analysis, this role suggests a democratic function that is often overlooked. The last section of this chapter explores the implications of this function. I argue that an emphasis on informing the public directs attention to order-of-magnitude judgments, estimation of marginal net benefits of stringency at a proposed policy, and above all transparency in presenting information, particularly involving benefits. Nonetheless, the technocratic and democratic roles complement one another, rather than competing with each other. An ideal regulatory analysis should fulfill both functions.

The RIA on Technical Grounds

Here I critique the RIA along several dimensions: scope, estimation of costs and benefits, equity, time discounting, and treatment of uncertainty. I close with an overall assessment, along with concrete recommendations for how this analysis, or any similar applied cost–benefit analysis, could be improved.

Scope

One of the most striking characteristics of the CAIR RIA is how narrow it is. The analysis evaluates the costs and benefits associated with only one emissions target, considers only one policy instrument, and compares the policy to a single baseline case representing one possible business-as-usual outcome.

First, the CAIR RIA is essentially an up-or-down assessment of the final rule versus the status quo. No other policy alternatives were considered in detail. Although the RIA conclusively demonstrates that the CAIR was preferable to the status quo on cost–benefit terms, it fails to show that the CAIR was better than an alternative policy. And there certainly were alternatives; what is more, EPA already had much of the information necessary to evaluate them. As Richard Morgenstern notes in Chpater 2, the rule itself was preceded by vigorous debate in Congress over several legislative proposals to reduce air pollution from power plants. These proposals ranged from the Bush administration's own Clear Skies legislation, introduced by Senators Inhofe (R-OK) and Voinovich (R-OH), to the more ambitious proposal by Senator Jeffords (I-VT). The cost to EPA of expanding the scope of the analysis, in terms of time and resources, would have been modest; as it was, the agency performed several runs of its Integrated Planning Model of the electric power sector and could have added more. Indeed, much of the necessary work was already done because EPA staff themselves had considered a range of policy scenarios in their assessment of the administration's Clear Skies legislation.1 The CAIR RIA, therefore, should have considered a range of alternatives to the policy that was eventually chosen.2

Second, the agency assumed that the states covered by the rule would implement a cap-and-trade program for SO2 and NOx emissions from electric generating units (EGUS), rather than using more prescriptive (and costly) approaches such as performance standards. But the states were free to regulate emissions in whatever manner and from whatever sources they chose. In defense of its approach, EPA argued that states could be expected to adopt a trading program for EGUS because it would be the least-cost approach. But the decision to focus solely on an emissions trading system effectively limited EPA’s analysis to a best-case scenario—and one it lacked authority to impose.

At the same time, EPA failed to demonstrate the cost savings that could be expected from using a market-based approach. By focusing on a single policy instrument, EPA ruled out a cost-effectiveness analysis—that is, a cost-based comparison among several policies that would achieve the same reduction in emissions. Such an analysis, however, would have been directly germane to EPA’s task under the Clean Air Act. As it turned out, costs were more relevant than benefits in issuing guidelines for state implementation plans (SIPS)—an ironic outcome, given the Clean Air Act's prohibition on using costs to set ambient air quality standards. In particular, Section 110(a) of the Clean Air Act—more precisely, Section 110(a)(2)(D)(i)(I)—requires SIPS to prohibit emissions in upwind states that “contribute significantly to nonattainment” of ambient air quality standards in downwind states. By anchoring EPA’s authority in the binary distinction between attainment and nonattainment in downwind states, the statute made the quantification of benefits largely irrelevant.3 On the other hand, in creating an emissions trading program for NOx in 1998, EPA had used cost-effectiveness as a criterion for defining the “significant contribution” of upwind states—a position that was upheld by the U.S. Court of Appeals in Michigan v. EPA.4

EPA was well aware of the importance of the cost-effectiveness criterion: the preamble to the official announcement of the CAIR in the Federal Register repeats the phrase “highly cost-effective” like a mantra (EPA 2005b). Yet the preamble offers a somewhat languid defense of the CAIR’s emissions targets as “highly cost-effective,” based on a comparison of the expected per-ton costs under the CAIR with cost estimates from a smattering of other air pollution controls, along with an ad hoc “knee of the curve” analysis that attempts to discern the level of abatement at which diminishing returns set in. Notably absent is any formal demonstration or evidence that the costs of an emissions trading program for EGUS would be substantially less than the costs of implementing more direct controls (e.g., technology standards requiring scrubbers or low-NOx combustion technologies).

The third and final dimension of scope concerns the choice of baseline scenario. Baseline scenarios embed a host of assumptions just as surely as policy scenarios do, even if they tend to be less obvious: outcomes under business as usual, and therefore the incremental costs and benefits of a particular policy, can be highly sensitive to assumptions about economic growth, electricity demand, technology, energy prices, and so on. Throughout almost all of its analysis of the CAIR, however, EPA compares the costs and benefits of its rule against a single business-as-usual scenario. The agency did run a few simulations assuming higher energy prices, but these are mentioned only occasionally and appear almost as an afterthought.

Estimation of Costs

The CAIR RIA makes two basic distinctions among types of costs: direct versus indirect costs, and private versus social costs. Direct costs refer to the actual costs of complying with the rule, such as installing the emissions control equipment (flue-gas desulfurization to reduce SO2 and selective catalytic reduction to abate NOx). These were estimated using EPA’s model of the electric power sector. Indirect costs, on the other hand, are the “ripple effects” in the economy as a whole that ultimately result from the regulation. For example, an increase in electricity rates—as the compliance costs are passed on to utility customers—will translate into higher production costs for businesses and lower real wages for workers, which can in turn lead to adjustments and changes in manufacturing output, investment, and labor supply. Gauging these indirect costs requires a general equilibrium model of the economy.

For the CAIR RIA, EPA estimated the general equilibrium (indirect) costs using the well-regarded Intertemporal General Equilibrium Model (IGEM), but these turned out to be negligible. For example, the projected decrease in gross domestic product (GDP) was only 0.03 of a percent relative to the business-as-usual case; even in energy-intensive industries, such as chemical manufacturing, impacts on output were well under 0.1 percent. As a result, EPA essentially, and quite reasonably, ignored the indirect costs and focused entirely on the direct costs in the bulk of its analysis.

Those direct costs, in turn, can be expressed either as private or social costs. In this case, the term social costs does not refer to the full social costs, taking into account the externality from pollution, which in this case would be a negative cost (i.e., a benefit). Rather, the wedge between private and social costs in this analysis is a much narrower one, including only taxes, which appear as costs to the firm but are transfers from the perspective of society, and the difference between private and social discount rates (with the private rate, representing the cost of capital, higher than the social rate). Appropriately, the agency used social costs as the basic measure of costs, although the presentation was somewhat confusing: only private costs appear in the summary table in the chapter of the RIA that discusses costs, whereas social costs appear in the tables in the executive summary.

A number of concerns arise regarding how EPA estimated the direct costs. Some aspects of EPA’s methodology understate the costs of the regulation. Not only did EPA assume that states would employ the recommended cap-and-trade system (which would achieve the emissions targets at lower cost than traditional command-and-control regulation), but in modeling the cost of that approach, EPA also essentially assumed a frictionless market with perfectly cost-minimizing utilities. This is far from an accurate description of reality. For example, ex post analyses of the SO2 trading program under the 1990 Clean Air Act Amendments have demonstrated that the costs of that program, although lower than they would have been under more prescriptive regulations, are substantially higher than what would have prevailed in a perfectly efficient market (Carlson et al. 2000; Keohane 2006).

On the other hand, several factors tend to overstate the costs of the regulation. For example, EPA’s model of the electric power sector uses a static representation of abatement technologies. Because technology would be expected to improve between 2005 (the date of the analysis) and 2015 (when the final emissions targets were in place)—especially given the additional spur provided by the regulation in question—ignoring technological change amounts to overstating the costs of compliance. This effect is reinforced by the use of conservative capital cost figures that, by EPA’s own admission, did not even reflect the most recent data available at the time of the analysis (EPA 2005a, page 7-19).

The cost analysis also fails to consider adjustments on the demand side. Higher electricity prices would dampen electricity demand, which in turn would bring compliance costs down because the lower electricity generation would translate into decreased emissions and less need for abatement. Again, EPA acknowledges this flaw in the RIA and goes out of its way to say that this might be significant (EPA 2005a, p. 7-20). On balance, the cost estimates in the RIA are probably overstated, as EPA itself acknowledges. Indeed, this is a common theme in discussions of regulatory cost estimates: Harrington, Morgenstern, and Nelson (2000) found that ex ante estimates of the cost of regulation tend to overstate the actual realized costs.

Nonetheless, it is hard to find too much fault with the agency. An ideal cost–benefit analysis would use up-to-the-minute cost data, incorporate technological change, take demand response into account, and model the frictions and transactions costs of real-world markets. But those things—especially the second and fourth items—are all at the frontier of economic research. Moreover, EPA is admirably up-front about the caveats and limitations of its analysis; indeed, all of the concerns listed above are raised by EPA itself. As I argue below, transparency is next to accuracy in cost–benefit analysis; indeed, it may be even more essential.

Estimation of Benefits

Despite the pitfalls associated with estimating the costs of environmental regulation, that exercise is widely viewed as straightforward compared with evaluating the benefits. The CAIR RIA is no exception to this rule.

In line with standard economic practice, EPA strives not only to quantify but also to monetize the benefits from reduced air pollution. It uses willingness to pay (WTP) as the basis of this valuation as far as possible (although the analysis resorts to other techniques in a few cases, such as cost-of-illness measures reflecting direct hospital costs and lost wages), and it accords more consideration to revealed preference methods of estimating WTP (such as using the variation in wages to estimate the value of a statistical life [VSL]) than to stated preference methods based on surveys. The fundamental justifications for these approaches are well known, even if they are hotly contested by critics: expressing benefits in dollar terms provides a convenient and consistent yardstick with which they can be compared to costs; WTP is an appropriate measure of economic value (and it is economics, after all, that provides the framework for analyzing and comparing costs and benefits); and revealed preference methods are generally considered more reliable measures of benefits than survey-based methods, which are prone to a number of biases.

A discussion of the merits or flaws of this basic approach is well beyond the scope of this chapter.5 For present purposes, I posit the appropriateness of the basic economic valuation paradigm and focus on three particular issues: the use of WTP in evaluating public decisions, the scope of the benefits estimation, and the transparency of the results. The first concerns the applicability to the public sphere of willingness-to-pay measures based on private decisions. It is not at all evident that people's willingness to trade off higher wages for greater risk in the workplace should be used to infer their valuation of reduced risk from premature mortality resulting from air pollution, but this is precisely the logic behind wage-based VSL estimates. Individuals may regard risks that they willingly accept in the workplace as fundamentally different from risks over which they have no control—such as the consequences of breathing air pollution caused by power plants hundreds of miles away. Moreover, even if those risks are considered to be commensurate, observed wage premiums offer an imperfect basis for estimating willingness to pay in general. The people who accept risky jobs have (almost by definition) preferences that are systematically different from those of the general population: in particular, they are likely to have a higher tolerance for risk.

Relying on stated preference measures cannot address these basic concerns. As several authors have argued, there is an important difference between people's valuation for amenities as consumers and their valuation as citizens.6 Indeed, experimental evidence suggests that social context matters: people value public goods differently when asked to state their valuation publicly, versus recording it privately (List et al. 2004). For all these reasons, determining a socially based citizen valuation for reduced risk would be a valuable topic for further research.7

A second and fundamental issue to consider in the benefits estimation is its scope: what gets measured and monetized, and what does not. The most striking aspect of the CAIR benefit analysis is how much it leaves out. As Richard Morgenstern discusses in Chapter 2 of this report, the list of unquantified and nonmonetized benefits is long and includes a variety of health benefits from lower air pollution (e.g., a reduction in premature mortality from ozone pollution or short-term exposure to particulates and a reduction in respiratory problems other than asthma) as well as virtually all nonhealth benefits, from impacts on commercial agriculture and fisheries to ecosystem functions.

Given that some benefits are expressed in dollar terms, the demands of completeness and consistency suggest that the largest possible fraction of benefits should be monetized. However, the difficulties inherent in monetizing benefits mean that some omissions are inevitable. Starting from that premise, the relevant questions are as follows: How should analysts determine which benefits to monetize? What should they do with impacts for which they lack willingness-to-pay estimates? How should they present their results?

To EPA’s credit, it captures what appears to be the most important benefit category, by several orders of magnitude: namely, the reduced premature mortality risk from ambient concentrations of particulate matter (PM). EPA estimates that the CAIR would prevent 17,000 premature mortalities each year when fully implemented, corresponding to annual benefits of $80 billion to $90 billion. By comparison, the next highest category of monetized benefits (reduced chronic bronchitis) is valued at just $3 billion. On the other hand, EPA also devotes a great deal of attention to benefit categories that amount to much less than rounding error: two pages of the RIA are given up to the details of assessing the impact of school absence days ($36 million), and other benefit categories include emergency room visits for asthma ($3.6 million) and lower and upper respiratory symptoms in children ($4 million and $5 million, respectively). It may be unfair to criticize EPA too much for analyzing these ultimately inconsequential (in an order-of-magnitude sense) effects of the regulation: perhaps the only way to determine how relatively small those benefits were was to conduct the analysis. Nonetheless, given the potentially significant benefit categories that were not monetized, EPA does not appear to have gotten the biggest “bang for its buck.”

Of the large number of benefit categories omitted from the estimate of monetized benefits, three stand out. The first of these is reduced premature mortality from ozone pollution. The benefits from reduced ozone-related mortality do not appear in the RIA’s estimate of total benefits, apparently because of doubts on the part of EPA’s Science Advisory Board (SAB) that the ozone effect could be distinguished from the mortality effects of PM. Nonetheless, on the basis of recent research on the subject, EPA suggests that reduced premature mortality from ozone could contribute an additional $3 billion annually to the estimated benefits—making it the second-largest single category of estimated benefits (though it would still be dwarfed by the benefits of lower particulate pollution). This estimate is based on meta-analyses of the literature on ozone pollution that were in press, but not yet published, at the time of the RIA. Thus, the quantification of ozone pollution appears to have been a case of the SAB’s recommendation lagging behind the literature. It is not too great a feat of inference to conclude that EPA staff disagreed with the SAB’s recommendation and would have strongly preferred to include the estimate of ozone-related benefits in their official total.

The facts of the case are similar in the second major missing category of benefits: reduced acidification of lakes and streams in the northeastern United States. Again, a study was available (Banzhaf et al. 2004) that would have allowed estimation of the monetary benefits from reduced acidification. In contrast to the case of ozone, however, EPA chose not to apply the study to the CAIR RIA on the grounds that the study was still undergoing peer review.

A third category of missing benefits concerns visibility. Here EPA’s action appears to have been more arbitrary and much less well explained. The agency states that it is able to quantify visibility impacts throughout the area affected by the new regulation; but it provides monetized benefit estimates only for Class I areas in the southeastern United States, without explaining why it limits the geographic scope of its analysis. As it is, EPA relies on a study of WTP for visibility improvements in the southwestern United States and extrapolates those results to the southeastern states; and hence is already obliged to use benefits-transfer methods to extrapolate; if transferring benefits in this way is valid, it ought to be equally valid for the other areas where the CAIR would improve visibility.

These omitted benefit categories raise a methodological concern about the criteria used to include or exclude relevant information from the scientific and economic literatures. In particular, EPA (or the SAB) seems too rigid in its distinction between acceptable and unacceptable studies. Given the problems and uncertainties inherent in any applied valuation analysis, this either-or approach is essentially arbitrary, especially when it means the difference between having some number and having no number at all. At the very least, such a bright-line approach leaves the results of a regulatory impact study subject to the whims of peer review timing and publication schedules. Meanwhile, efforts to get around the application of a strict rule may lead to asymmetric treatment of essentially similar cases, as in the example of ozone pollution versus acidification benefits.

An alternative and preferable approach would be both more flexible and internally consistent. EPA should present a range of estimates based on studies of varying degrees of authority. At a minimum, the range would include two estimates of benefits: one using only peer-reviewed studies with high confidence, and a second using the best available estimates even where they have not yet been published or where other concerns pertain (e.g., the age of the study). The former number, based on peer-reviewed studies, could still serve as the official or preferred estimate of benefits. The latter would provide a sense of the range of estimates. A “best available estimate” would also focus attention on EPA’s choice of which benefit categories to include and which to leave out, thereby helping to identify the categories of benefits that are most deserving of further research and study.8

Even with this more flexible approach, however, a complete accounting of monetized benefits will never be feasible. A third crucial issue to consider in benefits estimation, therefore, is transparency. What are the precise reasons for including or excluding a benefit category? How much does the decision not to include a benefit category matter for the overall results?

To help address these questions, a comprehensive list of all impacts should be presented, along with an indication of whether they were quantified or monetized, and a brief explanation for this choice (e.g., no willingness-to-pay information available; studies available but deemed unreliable; or quantification of impacts unavailable). The CAIR RIA takes a good first step in this direction by providing an exhaustive list of nonmonetized impacts. Even so, the analysis is inconsistent in explaining why these impacts were not monetized and what their magnitude might have been. In the case of ozone pollution, EPA (apparently chomping at the bit against the SAB’s restrictive recommendations) presents its own “informal” estimate and emphasizes its significance. In the other two cases, however, the agency is much vaguer about the potential benefits; the potential magnitude of missing visibility benefits is mentioned almost as an afterthought, and the potential benefits from reduced acidification are described as “substantial,” without any further elaboration. Yet strong evidence suggests that each of the missing benefits could be several hundred million dollars or more annually—at least an order of magnitude greater than several of the health-related benefit categories to which EPA devotes much more attention.

The lack of transparency is even more problematic in the case of other missing benefit categories. Regarding the remaining nonmonetized health benefits related to PM (involving low birth weight, pulmonary function, and other effects), EPA simply asserts that “we feel these benefits may be small relative to those categories we were able to quantify and monetize,” without any mention of the evidence on which the agency bases that judgment.

A further step toward improving transparency is explicitly acknowledging that the dollar-valued-benefit estimate is incomplete. The CAIR RIA scores well on this dimension, collecting nonmonetized impacts into a term “B,” which is then carried throughout the cost–benefit analysis. Although some observers criticize this approach as effectively ignoring a range of benefits by collapsing them into a single unknown parameter, acknowledging the missing impacts explicitly— putting them “on one side of the ledger”—is certainly preferable to the default alternative of assigning those impacts a zero value.

Distributional Incidence

Geography

One would expect that a cost–benefit analysis of an environmental regulation focused on the interstate transport of air pollutants would consider how the consequences of the policy were likely to vary with location. Indeed, the uneven incidence of the costs and benefits of pollution control provides the central rationale for the CAIR itself: upwind states should be held accountable for the impacts of their emissions on air quality in downwind states.

EPA’s analysis of costs largely reflects the central importance of geography, presenting region-specific estimates of the projected impacts on coal production and retail electricity prices (EPA 2005a, Tables 7-7 and 7-9). On the benefit side, however, EPA’s performance is more mixed. The RIA estimates the qualitative impacts on lakes and streams in three regions (EPA 2005a, Table 5-1), projects visibility impacts for each of 29 individual Class I areas (Table 3-10), and maps the expected percentage reductions in sulfur and nitrogen deposition (Figures 5-1 and 5-2). Similarly, the Notice of Final Rulemaking preamble (EPA 2005b) presents projections of precisely which counties will be in nonattainment for PM and ozone in 2010 and 2015 under both the base case and the CAIR, along with estimated ambient pollution concentrations.

Regarding the geographic distribution of monetized benefits, however, the RIA is silent. This must have been a conscious omission, although it is unacknowledged. EPA already estimates changes in air quality at the county level, and data on population density and hence exposure are used implicitly to translate those air quality changes into benefits. Thus, no new work would have been required to discuss how the estimated benefits are distributed geographically—but the resulting analysis would have been of considerable interest in understanding the impacts of the policy. Given that benefits are presented in monetary terms at the aggregate level, it is difficult to see why they should not be presented that way at local and regional levels as well.

Income

Any air quality regulation as sweeping as the CAIR can be expected to have disparate impacts across different income groups. A given rise in electricity rates has very different implications for rich and poor households, and may have regressive effects if not countered by other policy measures. The benefits are likely to be unevenly distributed as well. Air quality improvements may disproportionately benefit low-income households, to the extent that they are concentrated in urban areas or in places with poor initial air quality. On the other hand, visibility benefits are likely to accrue disproportionately to richer households, who are more likely to visit places such as national parks where visibility is most valuable.

The CAIR RIA, however, ignores distributional incidence across income groups. As in the case of geographic distribution of benefits, this would have required little extra work: EPA would only have had to match its county-level estimates of air quality improvements to similarly disaggregated data on average household income. Greater information on the distributional incidence of costs, meanwhile, could be gleaned by comparing estimated increases in electricity prices with average expenditures on electricity across households of different income levels (data already collected by the Energy Information Administration). The payoff from employing these approaches would have been a good deal of insight into how the consequences of the regulation fell on different groups.

The reluctance to report distributional effects may stem from a justified concern about the role of income in determining WTP. Strictly speaking, WTP depends on ability to pay and increases with income. According to this logic, benefits to richer people should be more highly valued. Although such an approach may be consistent with a narrow application of economic theory, it violates basic principles of fairness.9 Quite appropriately, EPA elects not to scale its measure of monetized benefits on the basis of cross-sectional variation in the income of the affected population.

At the same time, EPA’s decision not to adjust benefits for cross-sectional variation in income is apparently contradicted by its use of income adjustments over time. To account for economic growth, EPA assigns higher value to improved air quality in future years (when incomes will be higher in real terms), relying on estimates of income elasticities of WTP drawn from the economics literature. At first blush, this seems logically inconsistent with EPA’s decision not to take cross-sectional income variation into account. Indeed, the RIA itself directs attention to this problem, by characterizing income disparities across subpopulations and income growth over time as two manifestations of how income differences can affect WTP. Having thus lumped the two sources of variation together, EPA leaves itself little room to explain its decision to adjust WTP over time but not across income groups. As justification, the agency cites a statement by the SAB highlighting the “sensitivity of making such distinctions [among income groups], and because of insufficient evidence available at present” (EPA 2005a, 4-16). The clear implication is that if EPA (or the SAB) were more insulated from political “sensitivities,” or had greater evidence about the income elasticity of WTP, then it would be justified in adjusting WTP across populations.

This apparent contradiction can be resolved—but only by taking a conceptual step that is missing from EPA’s analysis. The measure of value that EPA desires (appropriately) to estimate, in assessing the benefits from its proposed policy, is not the willingness to pay of the actual affected population (which would depend on income, as well as age, education status, and so on), but rather the WTP of a representative U.S. population.10 In this conception, whether smog settles over a poor neighborhood or a rich suburb does not affect society's estimation of the damages caused, or the benefits of better air. Such an approach would seem to be a fundamental tenet of true environmental justice, and consistent with basic concerns of fairness and equity. Moreover, such an approach resolves the logical contradiction. Estimating the WTP of a representative population is perfectly consistent with making an adjustment for economic growth over time—as the United States as a whole gets wealthier, so does a representative population.

Discounting

The use of discounting to express future costs and benefits in present-value terms is at once one of the most standard and one of the most controversial approaches in applied cost–benefit analysis. The thorniest issues arise when comparing costs and benefits across long periods of time (e.g., decades or centuries), because discounting then carries with it an implicit judgment about intergenerational welfare comparisons. In the case of the CAIR, the benefits and costs were examined over a much shorter time horizon, in 2010 and 2015. Hence, no intergenerational comparisons are implicated. If the only use of discounting were to provide a common yardstick for costs incurred in 2010 (for example) with benefits realized in 2015, there would be little need for comment.

However, discounting still enters into the analysis in a fundamental way because of the lag time involved in the health consequences of exposure to air pollution. Following the recommendations of the SAB, EPA uses a segmented lag structure that allocates 30 percent of the PM-related mortality reductions to the first year, 50 percent to years 2 through 5, and the remaining 20 percent to years 6 through 20 (EPA 2005a, p. 4-45). To express the benefits from reduced mortality in present-value terms, therefore, EPA spreads the estimated reductions in mortality over 20 years and applies the standard Office of Management and Budget 3 percent and 7 percent discount rates to the resulting time profile of benefits.

This discounted lag approach has serious flaws. To begin with, as EPA notes, the lag structure is essentially arbitrary: because there is no “specific scientific evidence of the existence or structure of a PM effects lag,” the segmented approach is simply “intended to reflect the combination of short-term exposures in the first year, cardiopulmonary deaths in the 2- to 5-year period, and long-term lung disease and lung cancer” in the later years (EPA 2005a, 4-45–44-6). But when later effects are discounted, the choice of lag structure (as well as the choice of discount rate) matters considerably. Although EPA rightly performs a sensitivity analysis of the impact of the lag structure, that analysis underscores the problem: the estimated health benefits when no lag structure is applied are 250 percent greater than when an exaggerated (15-year) lag structure is applied and benefits are discounted at 7 percent.

What makes the choice of lag structure matter, of course, is the decision to discount future benefits. This may appear at first to be entirely unobjectionable—at least if one grants the appropriateness of discounting costs and benefits in principle. Upon reflection, however, very little justification seems to exist for discounting the reduced mortality from air pollution. The measure of benefits is the VSL. The underlying damage at issue—whose reduction is being quantified as a benefit and then discounted—is not an actual death, but rather an increase in risk. The benefit of cleaner air is an immediate reduction in the statistical likelihood of death—whether that death is imminent or lies in the distant future. Because the benefit is realized at the time that air quality improves, it should not be discounted.11

It may be that people are willing to pay more to reduce the chance that they will die within a year than to reduce their chance of dying within two decades. But this is not evident or deducible from first principles: some might well prefer a sudden heart attack to a drawn-out struggle with a chronic and debilitating disease. It is an empirical question. What is needed to resolve it is not simply better information about the lag structure, but also—and crucially—better information about how people value reductions in the risks of different kinds of deaths. Applying a discount rate, as EPA does, is a crude approach that imposes a particular and arbitrary assumption about how people value reductions in the risk of future death. Curiously, EPA appears largely oblivious to these considerations. The RIA is concerned only with whether the lag structure is correctly determined, but does not acknowledge that there is a more fundamental question about valuation at stake.12

As the discussion in the CAIR RIA makes clear, the issue of lagged health effects is a crucial one that will apply to many future regulatory analyses. A high priority for research, therefore, should be to gather better empirical estimates of willingness to pay for reductions in different kinds of mortality risk—at a minimum, distinguishing near-term impacts from chronic ones. Once such evidence is available, the proper approach will be to apply the appropriate measure of VSL at the time when the reduction in risk takes place—in other words, when the air becomes cleaner, not when the eventual mortality would have occurred. Until then, EPA should acknowledge the fundamental problem with discounting in this context and should include the case of zero lag (which is of course equivalent to an arbitrary lag structure with no discounting) as one of its core benefit estimates, rather than relegating it to a sensitivity analysis.

Treatment of Uncertainty

The CAIR RIA addresses uncertainty in three ways: through conventional sensitivity analysis (using parameter values chosen to represent plausible alternative assumptions), through Monte Carlo analysis using estimated distributions for dose–response parameters and health endpoints, and through Monte Carlo analysis using distributions of PM-related mortality impacts drawn from an expert elicitation process.

The results of all three approaches underscore the importance and pervasiveness of uncertainty in the case of the CAIR. As the sensitivity analysis shows, basing the estimated PM-related mortality impacts on results from the Harvard Six Cities study (Dockery et al. 1993), rather than the American Cancer Society study (Pope et al. 1995) used in the base case, more than doubles the estimated benefits of the CAIR. At the other extreme, assuming a fairly high (but still plausible) threshold for the effects of PM cuts the estimated benefits by 96 percent to less than $1 billion. The choice of income elasticity also has a sizeable effect because it interacts multiplicatively with the PM-related mortality effects that make up, by far, the greatest share of the benefits.

The Monte Carlo analyses demonstrate the extent of uncertainty even more clearly. Using estimated standard errors from the underlying studies that provided the basis for the health impacts, the Monte Carlo analysis finds a 90 percent confidence interval spanning an order of magnitude, from $26 billion to $210 billion (with a mean of $100 billion). When the Monte Carlo is based on the results of expert elicitation rather than estimated standard errors, the 90 percent confidence interval balloons, extending from $3 billion to $240 billion (around a mean of $74 billion).

This is an admirably varied, complete, and even innovative approach to assessing uncertainty. The three techniques employed by EPA represent a substantial and sophisticated effort to account for uncertainty, and they complement each other well. The conventional sensitivity analysis facilitates focused consideration of particular discrete and often qualitative alternatives; it answers such questions as (a) What happens if we ignore the lag structure for reductions in mortality risk? (b) What happens if we value all cases of chronic bronchitis? (c) What happens if we assume that willingness to pay is more or less sensitive to increases in income? The “classical” Monte Carlo approach allows for simultaneous consideration of multiple sources of uncertainty. It answers the question (conditional on the specified parameter distributions, of course), What is the central range or most likely magnitude of benefits? Finally, the expert elicitation approach combines the flexibility and scope of Monte Carlo analysis with a fuller and more nuanced treatment of uncertainty on a particular dimension—in this case, the dose–response curve for PM-related mortality impacts.

While one can quibble with how the techniques were applied in the case of the CAIR RIA, for the most part these concerns are minor. For example, the classical Monte Carlo analysis is applied only to health effects; impacts on ecosystems and visibility are treated as constants in the analysis. In practice, however, this omission probably matters little, in part because some of those same effects (in particular, visibility) are explored in the conventional sensitivity analysis; this is another example of complementarities among the three approaches. Similarly, the expert elicitation procedure was imperfect in a number of ways: only five experts were consulted, limited review appears to have taken place beforehand, no pre-elicitation workshop was held, and so on. But these criticisms hardly seem fair when one considers that the CAIR RIA represents the pilot phase of expert elicitation: it was explicitly designed as a trial run, and many of its deficiencies were remedied in subsequent applications. Indeed, the formal use of expert judgment to evaluate uncertainty represents an important innovation.

For all its merits, however, the uncertainty analysis in the CAIR RIA—like the selection of policy scenarios and the choice of which benefits to monetize—is too narrow in scope. Faced with pervasive uncertainty, the RIA considers only a subset of the sources of that uncertainty. Estimating benefits in dollar terms requires performing several independent analyses in sequence, with each link in the chain subject to uncertainty. Because the regulation proposed an emissions-trading system rather than mandates on individual power plants, a model of the electric power sector is required to translate the overall emissions targets into plant-level emissions estimates. Because the emissions affected by the regulation (SO2 and NOx) contribute to air pollution hundreds of miles away, and in different chemical forms (e.g., NOx combining with volatile organic compounds to produce ground-level ozone), analysts must employ necessarily imperfect models of pollution dispersion and atmospheric chemistry to translate emissions into pollution concentrations. In turn, those concentrations must be translated into effects on human health and ecosystems, using often poorly understood dose–response relationships. And finally, the physical impacts (16 number of premature deaths among human populations, or 16 percent of lakes and streams affected by acid deposition) must be expressed in monetary terms using often scant or incomplete measures of value, some of which depend on uncertain projections of population or income growth.

The problem is not that EPA fails to acknowledge these sources of uncertainty; indeed, in its discussion of benefits estimation the agency goes to great lengths to enumerate the uncertainties (EPA 2005a, Table 4.5). Rather, the problem is that the RIA addresses only a subset of the sources of uncertainty head-on. The sensitivity and Monte Carlo analyses explore the relationships between pollution concentrations and physical impacts, and between physical impacts and monetary values. But they give short shrift to the first two links in the causal chain outlined above— that is, the uncertainties in the distributions of pollution emissions and of ambient concentrations.

When the RIA does address the air transport models used to derive ambient concentrations, it does so in a self-referential way—evaluating their performance by comparison to the performance of other models or of other analyses. Thus, the model runs performed for the CAIR analysis are deemed “appropriate” simply because they are no worse than prior model runs. Although some statistics are provided on the predictive abilities of the models used relative to actual measured conditions, they are given without any context for the level of fractional error, for example, that might be deemed “good” or “bad” in an absolute sense. And no evidence is provided on the correlation between predicted and actual changes in air quality, even though the accuracy of predicted changes in air quality resulting from policy-induced changes in emissions is of central importance in the reliability of the model.

Meanwhile, the RIA includes essentially no discussion of the uncertainty in the spatial pattern of predicted emissions based on the projected outcome of emissions trading using EPA’s model of the electric power sector. Indeed, one is hard-pressed to find any recognition at all in the RIA that these projected emissions might be a major source of uncertainty in the analysis as a whole.13 This omission is especially glaring in light of the July 2008 vacatur ruling overturning the CAIR: one of the reasons cited by the court for its ruling was EPA’s failure to conclusively demonstrate any connection between its chosen regulatory approach (emissions trading) and the likely reductions in the contributions by sources in specific upwind states to downwind air quality.

EPA’s failure to sufficiently explore explicitly these two major sources of uncertainty—the impacts of regulations on emissions and of emissions on concentrations—constitutes a major gap in its analysis. This omission is all the more striking given how straightforward it would be, at least conceptually, to integrate these sources into its formal modeling of uncertainty—in particular, its Monte Carlo analyses. The air transport models, which are based on Gaussian plumes (i.e., probabilistic analyses of air movements), ought to be readily amenable to Monte Carlo analysis. Although more work would probably be required to formally model uncertainty in the spatial distribution of emissions under trading, that could be done as well—for example, by explicitly modeling the cost of pollution abatement at each individual EGU as a draw from a distribution rather than as a point estimate.

Recommendations

The CAIR RIA is, in many ways, an admirably comprehensive account of the benefits and costs associated with a particular regulation. EPA staff brought to bear a huge amount of relevant information. They carefully described how they conducted the analysis and, for the most part, explained why they made the choices they did. They presented many estimates in natural units as well as in dollar terms, and employed a set of sophisticated and complementary techniques to assess uncertainty.

Nonetheless, like any such document, the CAIR RIA had a number of weaknesses and blind spots that help to highlight ways that regulatory impact analysis could be improved. Distilling the preceding discussion yields a number of recommendations for regulatory impact analyses—some, but not of all, of these were followed in the case of the CAIR.

Scope of analysis. An RIA should consider multiple policy alternatives and, if possible, multiple policy baselines. When the policy instrument is not mandated by regulation—as in the case of the CAIR, which could suggest but not require an emissions trading program—the scope of policy alternatives considered should include other policy instruments as well as other targets.

Use of willingness-to-pay measures. Because the chosen value for the VSL plays a central role in the analysis, high priority should be placed on further research into appropriate VSL measures, particularly measures that explicitly capture public or social values rather than being derived purely from private risk-taking behavior.

Choice of primary sources. Although it is appropriate to base the “main” estimate of benefits only on studies meeting a well-defined and rigorous set of criteria (e.g., peer-reviewed articles published within a certain period of time), at least one additional benefit estimate should be presented that incorporates a wider set of studies, especially where doing so can expand the set of benefit categories considered. For example, in the CAIR RIA, a second estimate should have been presented that drew on still-unpublished but leading-edge research into the benefits from ozone-related mortality reductions and from visibility improvements.

Monetization of benefits estimates. If any benefits are expressed in monetary terms, then as many as possible should be expressed this way, and the reasons for not doing so should be clearly and fully explained on a case-by-case basis. Moreover, the existence of nonmonetized benefits should be explicitly acknowledged in the presentation of results—for example, through a generic term labeled “B.”

Distributional incidence. The distributional incidence of costs and benefits should be presented in depth—in particular, by geographic region and household income.

Use of a “representative population” for estimating benefits. Estimates of WTP should be defined with respect to a representative U.S. population. Benefits should be adjusted for income growth over time, but not for income disparities across subpopulations.

Discounting and lag structures for health effects. If health effects lag behind exposure, separate VSLS for different types of mortality (e.g., acute versus chronic impact) should be used rather than an arbitrary discounting approach. Because the benefit from the policy is a reduction in risk, that benefit should not be discounted—regardless of how far off in the future the death is likely to occur.

Uncertainty analysis. Multiple analyses of uncertainty should be used, including conventional sensitivity analysis, Monte Carlo analysis, and (if feasible) expert elicitation. To the extent possible, all sources of uncertainty should be explicitly accounted for, including uncertainty in modeling emissions and air quality, rather than just dose–response relationships and valuation.

The RIA as a Guide to Policymaking or a Source of Information

Having delved into the details of methodology and scope, we now pull back to a loftier vantage point and asks more fundamental questions: What can a regulatory impact analysis like the CAIR RIA achieve? What roles does it serve? Here, I identify two main functions, which I term the technocratic and democratic functions. Focusing solely or primarily on cost–benefit methodology as a technical input to policymaking (as in the above section) ignores the equally critical but often overlooked democratic function. Moreover, the two roles are complementary rather than mutually exclusive. Improving the performance of the RIA in informing and educating the public (the democratic function) cannot help but improve its usefulness to policymakers.

The Technocratic Function: The RIA as a Guide to Policymaking

The conventional view among efficiency-minded economists is that an RIA, and particularly the cost–benefit analysis at its heart, should guide policymakers in designing policy—setting the stringency of the emissions reductions required, selecting the appropriate policy instrument, and so on. According to this view, the RIA logically precedes the choice of policy. It should explore a range of relevant policy alternatives, illuminating the trade-offs among them and determining which would yield the greatest net benefits. Of course, most advocates of this view hasten to add that a cost–benefit analysis need not be narrowly determinative: economic efficiency need not be a necessary or sufficient criterion for sound policymaking (Arrow et al. 1996). Nonetheless, the role of the cost–benefit analysis in this framework is prescriptive: to provide the policymaker with a definitive assessment of the relative efficiency of various policy options.

As the section critiquing the RIA on technical grounds makes clear, the CAIR RIA performs well on many technical aspects of cost and benefit estimation. And yet the RIA fails to meet the most basic requirement of sound economic policy analysis: namely, the consideration of multiple alternatives. A document that considers the costs and benefits of the proposed policy only relative to the status quo cannot possibly have been used to design that policy.

Ironically, the CAIR RIA’s outward embrace of the technocratic ideal partly explains its failures. The benefits and costs are patiently catalogued, summed up, and presented to three significant digits as if decimal points will lead to better policy. Indeed, the RIA is almost compulsive in its precision—as illustrated by its patient exploration of categories of impacts (such as school absence days and asthma events) that do not even amount to rounding error, being measured in the tens of millions relative to total benefits in the tens of billions. For all its impressive features, the CAIR RIA is a document consumed by relatively minor details, providing little guidance or rationale for how the policy itself was chosen.

To be sure, the recommendations presented in this chapter's section critiquing RIA on technical grounds (as well as similar recommendations from other authors, including those represented in this report) offer suggestions for improving the RIA as a technical document. All the same, criticizing the RIA on the grounds that it did not provide the basis for informed policymaking misunderstands the statutory context. As noted in the introduction to this chapter, the Clean Air Act itself expressly forbids the EPA administrator to consider costs in setting ambient air quality standards. Indeed, EPA itself, in announcement of the CAIR (EPA 2005b), takes pains to explain that the required emissions reductions were based not on the RIA but rather on the agency's judgment of what was “highly cost-effective” and necessary to reduce the contribution of the states affected by the rule to ambient air pollution in downwind states. Far from playing a central role, the RIA is summarized cursorily on pages 144 to 151 of the 155-page document.

The Democratic Function: The RIA as a Source of Public information

If the RIA was not actually used to guide policy, what is its purpose? In Chapter 4 of this report, Wagner characterizes the CAIR RIA as a “litigation support document.” In the tradition of public choice analysis, she treats the RIA as a tool designed by EPA to support its rulemaking against legal challenge and evaluates it on that dimension.

But there is another alternative, and indeed one that is suggested by EPA itself in its introductions of recent RIAS: “to inform the public and states about the potential costs and benefits of implementing these important air quality standards.” What happens if we take seriously the proposition that a central aim of an RIA is (or ought to be) informing the public (and policymakers) ex post about the consequences of a decision—even if the decision, as in the case of CAIR, was made on other grounds? Three implications stand out.

Order-of-magnitude judgments

First, from this different vantage point, precision in estimating benefits and costs takes on less importance. Rather, the most useful and relevant pieces of information concern order-of-magnitude judgments: Are benefits likely to be greater than costs? With what degree of confidence?

Emphasizing order-of-magnitude impacts rather than precise figures communicates a more honest realization of the deep uncertainties involved in assessing regulatory policies as far-reaching as the CAIR. As noted above, the process of estimating the regulation's costs, and especially its benefits, involve a chain of reasoning—from regulation to emissions, emissions to concentrations, concentrations to physical impacts, and impacts to monetary values—in which each step involves great difficulty and fundamental uncertainty. The impression of staggering complexity is confirmed by the sensitivity of the results to key assumptions, as well as the extraordinarily large 90 percent confidence intervals found in the Monte Carlo studies.

Moreover, the measured sources of uncertainty just described amount to the known unknowns—the gaps in knowledge that can be catalogued, assessed, and assigned standard errors or other measures of uncertainty. On top of these lie what might be called the unknown unknowns.14 Several major categories of benefits are not monetized at all; as a result, they are absent from the discussions of standard errors or Monte Carlo analysis. And there may well be whole categories of benefits that are not accounted for at all. The impetus behind the SO2 trading system in the 1990 Clean Air Act Amendments, for example, was the problem of acid deposition in forests, lakes, and streams. Only later did the contribution of SO2 to ambient concentrations of PM—now considered to be far and away the most important benefit from reducing SO2 emissions—become known. This pervasive uncertainty suggests caution in drawing precise conclusions from the CAIR RIA or from any assessment of similarly complex regulations.

In the case of the CAIR, is fairly certain that benefits are greater than costs; after all, the regulation would pass a simple cost–benefit test even assuming the lower bound of the 90 percent confidence interval using expert elicitation ($3 billion). Indeed, one may reasonably conclude that benefits are much greater than costs—by at least an order of magnitude. Beyond these order-of-magnitude statements, however, it is hard to pin down any number with confidence. In a sterling example of mistaking precision for accuracy, the CAIR RIA presents results to three significant digits without regard to the considerable error bounds surrounding its estimates. A more honest approach would be to replace the precise numbers presented in the executive summary with a simple conclusion: “Based on the best available evidence, the benefits from the CAIR are at least an order of magnitude greater than the costs, with net benefits measured in the tens of billions of dollars annually.”

This is not to say that analysts should not seek appropriate precision in their estimates—but rather to argue that in presenting their main results, analysts should abstract from the precise numbers and offer conclusions that are consonant with the underlying uncertainties. Presenting results in order-of-magnitude terms would also direct attention toward a fruitful set of questions that are unanswered in the CAIR RIA. What are the most important sources of uncertainty in driving the results? How wide is the range of plausible benefits and costs? Perhaps most importantly, what would have to change to alter the basic conclusion from the analysis (i.e., reverse the sign of net benefits)?

Estimation of marginal net benefits of the proposed policy

Second, if the primary objective of the analysis is to describe the impacts rather than prescribe an outcome, identifying and considering a particular set of alternative policies may not be as crucial. Instead, it becomes more useful to estimate the marginal impacts of increasing or decreasing the stringency of the policy. Such an approach can help answer a question of considerable interest: How does the proposed policy compare to the economically efficient one?

Note that the performance of a given policy relative to the efficient one cannot be ascertained simply by computing its total benefit and cost. In the case of the CAIR, for example, EPA found projected net benefits of $83 billion to $99 billion a year at full implementation in 2015, even without considering a range of nonmonetized benefits; benefits outweighed costs by roughly 30 to 1. Such figures might seem to provide prima facie grounds for more stringent action. But, in fact, that conclusion does not follow at all: in principle, net benefits could have been even larger for a weaker policy (although this did not prove to be the case in the CAIR, as I discuss below). Simply calculating total benefits and costs does not shed light on marginal benefits and costs, which—as any economics student knows—must be equated to satisfy efficiency.

Nor is the mere consideration of a few alternative policies sufficient to determine how a given policy compares to the efficient one. Suppose policy A represents the status quo, policy B is a proposed regulation, and policy C is a more stringent alternative. An analysis finds that total net benefits are greatest under B, but greater under A than under C. Comparing A and C might suggest that the efficient policy must be less stringent than B; but in fact nothing rules out the possibility that the efficient outcome actually lies between B and C—meaning that the proposed policy is not stringent enough, rather than too stringent.

By comparing sufficiently many alternatives, of course, one could determine the efficient policy. In the limit, the analyst would estimate the marginal benefit and cost schedules (for all possible policies), with the efficient point lying at their intersection. In the real world, estimating these entire schedules may be infeasible. Nonetheless, it may be possible to estimate the marginal benefit and marginal cost of the proposed policy. Whether the proposed policy is more or less stringent than the efficient one can then be determined by computing marginal net benefit (marginal benefit minus marginal cost): if it is positive, the policy is too lax from an efficiency perspective; if negative, the policy is too stringent.

This approach was feasible in the case of the CAIR. EPA itself estimated the marginal cost of its proposed rule; other analysts had little trouble estimating marginal benefit. The advocacy group Environmental Defense Fund, using a methodology employed by EPA in prior analyses, made a back-of-the-envelope estimate that the benefits of SO2 reductions amounted to $15,000 per ton— more than an order of magnitude greater than EPA’s estimate of the marginal abatement cost (Shore et al. 2004).15 Researchers at Resources for the Future estimated that marginal benefits were between $1,800 and $4,700 per ton of SO2 reductions, well above EPA’s estimated marginal cost of $700 to $1,400, and $700 to $1,200 per ton of NOx reduction, somewhat less than EPA’s estimate marginal cost of $1,300 to $1,600 per ton (Banzhaf et al. 2004). On the basis of these estimates, the SO2 reductions—despite their impressive net benefits—proved to be too small from an efficiency perspective, while the NOx reductions may have been slightly too stringent.16

Transparent and accessible presentation of benefits and costs

Third, an eye toward informing the public rather than shaping policy also suggests that much more attention should be given to how costs and especially benefits are presented in the analysis.

A key step is to quantify the consequences of the policy in natural units (i.e., physical impacts) as well as in monetary terms. The monetization of costs and especially benefits is one of the most common targets for critics of cost–benefit analysis—and with some justification, given the large number of assumptions involved. On the other hand, proponents of cost–benefit analysis point out (with equal justification) that expressing impacts in dollars—or any other common metric— is a necessary step in aggregating disparate benefits and comparing them with costs in a consistent fashion.

The primary justification for boiling everything down to net benefits is to ensure that decisionmakers rely on some “objective” measure of value rather than substituting their own personal preferences. In this context, WTP is best seen as one possible system of weights, among many, to use in comparing disparate impacts. An estimate of net benefits is essentially a summary statistic; like all summary statistics, in compressing a great deal of information it leaves much out. If only a single set of weights is to be used (necessarily the case in deriving a single “societal” estimate for net benefits), then economic theory provides a strong argument that for all its faults, WTP based on revealed-preference measures is the best set of weights.17 Rather than viewing the calculation of net benefits as a central goal (the implicit approach taken in the CAIR RIA), however, the estimation of net benefits should be regarded as one means to a larger end: informing the public about the consequences of a proposed regulatory policy, and the trade-offs involved.

When the function of a regulatory analysis is defined as informing the public, the personal preferences of individuals take center stage. From this perspective, the goal of the analysis should be to give the reader enough information to answer the question, would I vote for this? This means supplying enough information that an individual reader can substitute her own weights on various policy outcomes—that is, her own preferences—rather than relying solely on the “objective” weights provided by willingness-to-pay measures.

Wherever possible, therefore, benefits ought to be quantified and presented in natural units (e.g., a reduction in the incidence of premature mortality or the increase in visibility in natural parks). The presentation of quantified benefits in natural units is a complement to, rather than a substitute for, the presentation of monetized benefits: the two sets of numbers convey distinct information. To its credit, EPA does present a variety of health impacts in natural units (see EPA 2005a, tables 4-16 and 4-17.

Benefits (and costs) could also be made more informative by conveying them in both total and per-capita terms. For example, health effects could be presented in terms of incidence rates as well as in totals. After all, what is being measured (and valued) is a reduction in risk, not a reduction in specific deaths.18 In addition to presenting the PM-related health impacts as 13,000 fewer deaths per year, therefore, EPA should present them as such-and-such a reduction in the risk of premature death (e.g., per 100,000 individuals). The total number conveys a sense of the magnitude of the program's impact (and could be compared to the total cost of the program); the reduction in risk is what is relevant for an individual (and could be compared to, say, the cost of the program per household).

Recognizing that individuals may have difficulty identifying their own preferences over unfamiliar nonmarket goods, a well-designed RIA could provide contextual clues to help readers make those assessments. A reader may wonder how much she, as an individual, ought to value the reduced risk associated with air pollution that this policy will achieve. To inform this contemplative process, an RIA could describe the underlying trade-offs by transposing them to other, more familiar spheres of decisionmaking. For example, in the case of the CAIR, the reduced mortality risk from air pollution might be described as follows:

This proposed regulation is estimated to cost roughly $2 billion annually and to prevent 13,000 premature deaths each year. The implied cost per avoided premature death is therefore $150,000. If one were to apply this same trade-off to other, more familiar decisions, it would be equivalent to an individual paying $15 per year to reduce his or her annual risk of dying by 1 in 10,000—equivalent to the risk from [smoking X cigarettes per day] [rock climbing at X elevation], and so on.

Similarly, results from revealed-preference studies could explicitly serve as guides to personal reflection, rather than being offered as “true” or “objective” measures of value. Continuing the example above, the RIA might point out that empirical studies of wage premiums suggest that workers earn, say, roughly $600 more in a year for every 1-in-10,000 increase in the risk of death.

Finally, the same considerations also suggest that RIAS should explicitly include discussions of other regulatory policies. In the conventional view of an RIA as a guide to policy, discussion of other policies is essentially irrelevant; the costs and benefits of the policy at hand are what matters. The CAIR RIA, for example, never mentions any other environmental regulation; it is as if the analysis takes place in a vacuum. In contrast, if the primary goal of the RIA is to inform the public, then a discussion of other regulatory policies—such as their costs and consequences—can provide crucial context.

Conclusion

In the last section I emphasized the democratic function of regulatory analysis—partly because, in the case of the CAIR, the technocratic function was essentially made moot by the Clean Air Act's prohibition on using costs in setting air quality standards. In general, however, an effective regulatory impact analysis should fulfill both the technocratic and the democratic functions—guiding policy makers ex ante and informing citizens ex post.

At a practical level, the technocratic function is already well enshrined in the regulatory review procress. Elevating the democratic function implies a change of approach. In particular, the team at OIRA responsible for reviewing and commenting on draft RIAS should include not only the typical assortment of economists, engineers, and scientists—i.e., technical experts—but also at least one member charged with assessing the transparency and informativeness of the review. One can imagine a new office of “OIRA ombudsman” serving in this new role.

If the technocratic and democratic functions would place somewhat different emphasis on different aspects of analysis, their combined effect would be complementary. A technically sound cost–benefit analysis is an obvious prerequisite for an informative one. Less apparent, but equally the case, is that a truly democratic regulatory analysis—one that is transparent and accessible to the lay reader, and designed to inform the public—will also provide better guidance to policymakers. In part, this is because the ultimate consumer of a cost–benefit analysis is often little more than an educated layperson, at least relative to the technical experts, steeped in the nuances of a particular regulation, who author the analyses. Policymakers need plain language, transparent presentation of results, and order-of-magnitude conclusions just as surely as the public does. Moreover, the process of making a regulatory analysis transparent and informative can only improve the clarity of thought and quality of reasoning that go into the document itself. Finally, from a dynamic perspective, the democratic function is integrally important to good policymaking in the long run: each successive regulation offers a chance to inform and educate the public, and thereby strengthen popular support for sound and well-designed regulations.

Notes

1. Personal communication from Sam Napolitano, Director of EPA’s Clean Air Markets Division, June 6, 2008.

2. The narrow scope of the CAIR RIA also runs counter to the language of Executive Order 12866, which mandated it. Section 6(C)(iii) of the executive order requires “An assessment, including the underlying analysis, of costs and benefits of potentially effective and reasonably feasible alternatives to the planned regulation... and an explanation why the planned regulatory action is preferable to the identified potential alternatives.”

3. In other words, the binary distinction does not recognize the magnitude of the benefits from moving from nonattainment to attainment. A large improvement in air quality in a very polluted county could still fail to bring that county into attainment, while a much smaller improvement in a marginal county could cross the threshold.

4. State of Michigan, Michigan Department of Environmental Quality and State of West Virginia, Division of Environmental Protection, Petitioners v. U.S. Environmental Protection Agency, Respondent, New England Council, Inc., et al., Instervenors, 213 F.3d 663 (D.C. Cir. 2000).

5. Later in this chapter, I probe into how a valuation approach based on willingness to pay could be supplemented with other information.

6. See, e.g., Sen (1995) and Sunstein (1997, Ch. 2).

7. Although there are principled reasons to question the use of a wage-based VSL for air pollution, the practical implications for cost–benefit analysis are less clear-cut, at least in the case of the CAIR. First, few alternatives exist. Indeed, EPA itself acknowledges the limitations of using wage–hedonic estimates of VSL, but defers to its own Scientific Advisory Board in continuing to rely on such estimates for valuation. Second, the number value EPA uses—$6 million in the year 2010—is on the high end of available estimates; for comparison, the cost–benefit cost analysis of the CAIR by Banzhaf et al. (2004) used a value of $2.25 million. While that comparison does not reveal whether $6 million is “high” or $2.25 million is “low,” it does insulate EPA from the charge that it chose a low-end estimate. Third, the estimated benefits from the CAIR are already far greater than the costs. Selecting a higher value for the VSL feels a little like running up the score.

8. Note that the proposal to base estimates of benefits (and costs, which should be treated symmetrically) on a wider range of studies is distinct from the treatment of uncertainty discussed later in this chapter. The point here is not to explore the consequences of this or that assumption, but rather to show the total impact on estimated benefits and costs from incorporating the very latest research, even if that research has not yet completed its journey through the publication process. The importance of taking leading-edge research into account is especially great in areas of active research, as in the case of the CAIR.

9. An alternative approach, equally consistent with economic theory, would be to base valuation on willingness to accept (WTA; also known as “equivalent variation” in welfare economics) rather than WTP (also known as “compensating variation”). While WTP is more commonly used in applied settings, there is no theoretical ground for preferring it, and under standard conditions it is weakly smaller than WTA.

10. See Revesz (1999, p. 967), who advocates on equity grounds that a uniform VSL be applied across all environmental programs on the basis of a representative population of the United States.

11. See Heinzerling (2000, 204–5) for a similar argument.

12. The only hint of these deeper issues comes in the context of EPA’s discussion of “uncertainties” surrounding the valuation of premature mortality, when the agency discusses the theoretically attractive but practically infeasible “survival curve” approach, which would account for the effect of improved environmental quality on the probability of survival as a function of age, health status, and so on (EPA 2005a, pp. 4-58).

13. Although EPA performed a sensitivity analysis to gauge the effects of alternative assumptions about energy prices and electricity demand, it appears to have considered the consequences only for estimated costs—not for the spatial pattern of emissions.

14. Although former Secretary of Defense Donald Rumsfeld captured the idea in his memorable phrase “unknown unknowns,” economists generally credit Frank Knight (1921) with distinguishing between risk, involving a number of possible events whose probabilities can be known in advance (i.e., known unknowns), and uncertainty, involving events whose likelihood is unknown and unmeasurable (i.e., unknown unknowns).

15. The author, although currently on the staff of Environmental Defense Fund, was not at the organization at the time that the CAIR was finalized or at the time of the comments.

16. It is worth noting that, in generating these benefit estimates, Banzhaf et al. used a much lower estimate for the value of a statistical life than EPA used in its analysis: $2.25 million versus $5.5 million.

17. That statement would surely be challenged by opponents of cost–benefit analysis. Nonetheless, as a conditional statement it seems hard to challenge. Conditional on the need to use a single set of weights, economic theory provides a strong basis for using WTP. Scholars uncomfortable with cost–benefit analysis have a stronger case in challenging the need to use a single set of weights, as I discuss in the text.

18. For a differing perspective, see Heinzerling (2000), who challenges the notion of a “statistical life.” But while Heinzerling argues that risk and death are distinct harms, the distinction makes little sense from an analytic perspective. What is relevant for the policy analyst is the incremental effect of the policy being considered. Consider an analogy to cigarette smoking. Smoking raises the risk of death from lung cancer, but many smokers never get cancer, and many nonsmokers do. An antismoking campaign could be evaluated by the resulting total reduction in the number of deaths from lung cancer (controlling for other factors) or by the change in the risk, i.e., the incidence of lung cancer in the affected population. Those two measures correspond to the same underlying effect (the latter is simply the former divided by the total population) and hence are equivalent: they do not represent distinct harms and should not both be counted.

References

Arrow, Kenneth J., Maureen L. Cropper, George C. Eads, Robert W. Hahn, Lester B. Lave, Roger G. Noll, Paul R. Portney, Milton Russell, Richard Schmalensee, V. Kerry Smith, and Robert N. Stavins. 1996. “Is There a Role for Benefit-Cost Analysis in Environmental, Health, and Safety Regulation?” Science 272 (12 April): 221–222.

Banzhaf, H. Spencer, Dallas Burtraw, and Karen Palmer. 2004. “Efficient Emission Fees in the U.S. Electricity Sector.” Resource and Energy Economics 26:317–341.

Carlson, Curtis, Dallas Burtraw, Maureen Cropper, and Karen L. Palmer. 2000. “Sulfur Dioxide Control by Electric Utilities: What Are the Gains from Trade?” Journal of Political Economy 108:1292–1326.

Chestnut, Lauraine G., and David M. Mills. 2005. “A Fresh Look at the Benefits and Costs of the U.S. Acid Rain Program.” Journal of Environmental Management 77:252–266.

Dockery, D.W., C.A. Pope, X.P. Xu, J.D. Spengler, J.H. Ware, M.E. Fay, B.G. Ferris, and F.E. Speizer. 1993. “An Association between Air Pollution and Mortality in Six U.S. Cities.” New England Journal of Medicine 329 (24): 1753-1759.

Harrington, Winston, Richard D. Morgenstern, and Peter Nelson. 2000. “On the Accuracy of Regulatory Cost Estimates.” Journal of Policy Analysis and Management 19 (2):297–322.

Heinzerling, Lisa. 2000. “The Rights of Statistical People.” Harvard Environmental Law Review 24:204–205.

Keohane, Nathaniel O. 2006. “Cost Savings from Allowance Trading in the 1990 Clean Air Act: Estimates from a Choice-Based Model.” In Jody Freeman and Charles D. Kolstad, eds., Moving to Markets in Environmental Regulation: Lessons from Thirty Years of Experience (New York: Oxford University Press), 194–229.

Knight, Frank H. 1921. Risk, Uncertainty, and Profit. Boston: Hart, Schaffner & Marx.

List, John A., Robert P. Berrens, Alok K. Bohara, and Joe Kerkvliet. 2004. “Examining the Role of Social Isolation on Stated Preferences.” American Economic Review 94:741–752.

Pope, C.A., III, M.J. Thun, M.M. Namboodiri, D.W. Dockery, J.S. Evans, F.E. Speizer, and C.W. Heath, Jr. 1995. “Particulate Air Pollution as a Predictor of Mortality in a Prospective Study of U.S. Adults.” American Journal of Respiratory Critical Care Medicine 151:669–674.

Revesz, Richard L. 1999. “Environmental Regulation, cost–benefit Analysis, and the Discounting of Human Lives.” Columbia Law Review 99:941.

Sen, Amartya. 1995. “Environmental Evaluation and Social Choice: Contingent Valuation and the Market Analogy.” Japanese Economic Review 46 (1):23-37.

Shore, Michael, Amber Munger, Cindy Copeland, and Vickie Patton. 2004. Stop Blowing Smoke in the Heartland: Protecting Human Health from Air Pollution. New York: Environmental Defense Fund. www.edf.org/documents/3887_blowingsmoke.pdf.

Sunstein, Cass R. 1997. Free Markets and Social Justice. New York: Oxford University Press.

U.S. Environmental Protection Agency (EPA). 2005a. Regulatory Impact Analysis for the Final Clean Air Interstate Rule. EPA-452/R-05-002 (March).

———. 2005b. “Rule To Reduce Interstate Transport of Fine Particulate Matter and Ozone (Clean Air Interstate Rule); Revisions to Acid Rain Program; Revisions to the NOx SIP Call.” Federal Register 70 (91): 25162–25405.