Shining a Bright Light on Regulators:  Tracking the Costs andBenefits of Federal Regulation


Shining a Bright Light on Regulators:  Tracking the Costs andBenefits of Federal Regulation

September 30, 1997 43 min read Download Report
Robert DePosada
Policy Analyst

Every year, the federal government issues thousands of regulations. According to the U.S. General Accounting Office (GAO), federal agencies produced 5,476 final rules between April 1, 1996, and August 21, 1997. Too many of these regulations are either unnecessary or poorly designed, and needlessly inefficient and expensive as well. Furthermore, federal agencies have not developed a system for making rational and well-informed decisions about efficiently allocating their limited resources in order to maximize the health, safety, environmental, and other benefits expected from these regulations.

There is bipartisan agreement that benefit-cost analysis can be a useful tool in evaluating the merits of entire regulations or parts of regulations. This consensus has been achieved slowly and painstakingly over many years, with the White House Office of Management and Budget (OMB) often taking the lead in educating both Congress and the public. As recently as December 1996, OMB stated that "The use of sound economic analysis in the design of regulations, such as the benefit-cost and cost-effectiveness analyses called for in President Clinton's E.O. [Executive Order] 12866, Regulatory Planning and Review,2  is vital to generating maximum health, safety, environmental and other benefits to society for the limited resources available."3

What Did Congress Require the OMB To Do?

Section 645(a) of the Treasury, Postal Services and General Government Appropriations Act, 1997 (Public Law 104-208), directs the Director of the Office of Management and Budget to submit to Congress, by September 30, 1997, a report providing:

"(1) estimates of the total annual costs and benefits of Federal Regulatory programs, including quantitative and nonquantitative measures of regulatory costs and benefits;

"(2) estimates of the costs and benefits (including quantitative and nonquantitative measures) of each rule that is likely to have a gross annual effect on the economy of $100,000,000 or more in increased costs;

"(3) an assessment of the direct and indirect impacts of Federal rules on the private sector, State and local government, and the Federal Government; and

"(4) recommendations from the Director and a description of significant public comments to reform or eliminate any Federal regulatory program or program element that is inefficient, ineffective, or is not a sound use of the Nation's resources."














Yet in its most recent report, entitled "Draft Report to Congress on the Costs and Benefits of Federal Regulations,"4  OMB appears to have lost the spirit of its convictions. It now sees benefit-cost analysis as but one tool among many-and, unfortunately, not a particularly important one. Despite many previous efforts, OMB now is wary of efforts to sum benefit and cost estimates and is unwilling to draw comparisons across federal regulatory agencies.

When Congress passed Section 645(a) of the Treasury, Postal Services and General Government Appropriations Act, 1997 (Public Law 104-208), and directed the OMB to submit a report by September 30, 1997, it clearly expected OMB to shine a bright light on the federal government's regulatory activities. By doing so, OMB would help satisfy the public's right to know the full range of consequences associated with federal regulation. With full disclosure, the report could better equip the American people to hold their elected officials, appointed agency heads, and agency civil servants accountable for decisions they make.

In issuing this Draft Report, OMB has demonstrated that it is possible to begin to get a handle on the size and scope of the federal regulatory system. As Senate Governmental Affairs Committee Chairman Fred Thompson (R-TN) recently noted, "Now we know that regulatory accounting is doable, and it does not impose an unjustifiable burden on the agencies."5 But there still is a long way to go before the American public can conclude safely that the federal government's regulatory system is working effectively and efficiently in the public interest, or that centralized oversight of federal regulation lies in good hands at OMB.

Despite almost two decades of experience reviewing proposed regulatory actions and regulatory analysis, the Office of Management and Budget may have lost the ability (or perhaps even the will) to provide cogent independent insight about either the magnitude of the benefit and cost estimates developed by federal agencies or the quality and reliability of these estimates. Furthermore, OMB is surprisingly critical of independent efforts to perform the evaluative functions that it apparently no longer feels sufficiently competent to perform itself.

Congress directed OMB to prepare the Draft Report in the belief that, unlike any other federal agency, OMB possessed the expertise, experience, and breadth of perspective necessary to do the job. For many years, OMB did not shy away from providing independent assessments either of the merits of proposed federal regulations or of the quality of the regulatory analysis supporting them. In the Draft Report, however, it eschews any responsibility for independent critical review of agency actions or analyses. Instead, it merely compiles estimates of costs and benefits provided by agencies without any commentary on their reliability.

This is not what Congress intended. Since OMB reviews all "economically significant" rules according to the principles of E.O. 12866, it is in a unique position to know the extent to which these estimates accurately reflect the impact of regulatory actions.

Congress, however, has had to resort to legislating that information on regulatory impacts be provided to the public. Thus, OMB's effort can be described as, at best, a very modest first step in the right direction. OMB could better serve Congress and the public with a more open discussion of the costs and benefits of regulation when it submits its final report to Congress on September 30, 1997. It should ensure that the estimates in the Draft Report are based on consistent, comparable, and reliable analyses. At a minimum, it should highlight when this is not the case. As previous OMB reports demonstrate, it is certainly capable of such analysis and evaluation.6 If OMB fails to meet, at a minimum, the statutory reporting requirements, Congress should send the report back to OMB and demand that it do so.

The Benefits of Regulatory Accounting

The Office of Management and Budget correctly points out in its Draft Reportthat "knowing the total costs and total benefits of all of the many and diverse regulations that the Federal government has issued provides little specific guidance for regulatory decisions."7 However, this does not suggest that information on total regulatory costs and benefits is not valuable. Though knowing total federal budget expenditures may not help in evaluating individual budgetary decisions, it would be hard for anyone to argue against keeping track of the size of the federal budget. As OMB itself observed in its FY 1993 budget, "just as forecasting budget outlays is key to the development of macroeconomic policy, knowledge of the mandated costs of regulation is important for the conduct of micro- and macro-economic policy."8

In an era of fiscal budgetary restraint, regulation is often an appealing substitute for government spending. Tracking spending is relatively easy because the federal outlays are clear and measurable. On the other hand, regulatory costs are harder to track. As pressure mounts to hold down on-budget spending, policymakers find it increasingly attractive to shift costs off-budget in the form of regulation. Beginning to account for the costs and benefits of regulation will help to show that real trade-offs on the allocation of America's resources are being made, and that they have real economic consequence.

In addition, on several occasions over the past two decades, OMB has heralded the merits of regulatory budgeting, and a budget-which would force regulators to act within limits-goes considerably beyond the simple regulatory accounting exercise that Congress initially asked OMB to prepare. Indeed, OMB has consistently supported regulatory budgeting even in the face of serious implementation concerns. For example, the 1991 Regulatory Program of the United States Government includes a section entitled "Regulatory Accountability and the Regulatory Budget" in which OMB concludes that a regulatory budget is one way to build on the existing regulatory oversight process and-most important-that despite the concern that uncertainties associated with estimating regulatory costs and benefits may be greater than those involved in fiscal budgeting, "they should not be insurmountable. For example, the spending forecasts for fiscal budgets do not have to be perfectly precise for the fiscal budget process to be effective in controlling spending. Likewise, the spending forecasts for regulatory budgets do not necessarily have to be absolutely accurate for the regulatory budget process to act as a constraining device for regulatory spending."9 In various other publications, OMB has advanced essentially the same position.10

In contrast to this previous OMB position, the Draft Reportsuggests that the uncertainties and difficulties of regulatory accounting are so huge as to be insurmountable. Although it contributes to the field of knowledge by highlighting areas in which more research on regulatory accounting would be useful, the OMB's Draft Report paints an inaccurately bleak picture of the state of the art of regulatory analysis.

How the OMB Reprot Could Be Improved


• Improve the usefulness and quality of the aggregate cost and benefit estimates.

• Broaden the review of "significant regulations."

• Provide an OMB assessment of the quality and reliability of agency economic analyses, including a checklist to assess compliance with OMB's economic analysis guidelines.

• Recommend changes in regulations or regulatory programs.








In requesting the report,11 Congress asked the Office of Management and Budget to provide (1) estimates of the total costs and benefits of federal regulatory programs; (2) estimates of the costs and benefits of economically significant rules; (3) an assessment of the direct and indirect impacts of federal rules on the private sector, as well as federal, state, and local governments; and (4) recommendations for reforming or eliminating ineffective regulations or ineffective federal programs.

OMB attempted to supply the information required in the first two of these mandates, though it could better serve Congress and the public with a more open discussion of the costs and benefits of regulation and a more critical review of regulatory programs and their supporting analyses. However, in the Draft Report, OMB barely addresses the latter two mandates and provides no quantitative assessment of the indirect effects of federal regulatory actions or actions for regulatory reform. There are a number of very doable, concrete ways by which Congress can require OMB to improve its report.

Improve the Usefulness and Quality of Aggregate Cost and Benefit Estimates

Historically, OMB has been in the forefront in raising concerns about the aggregate scale of regulatory costs and the absence of credible estimates of aggregate regulatory benefits. In its Draft Report, however, it appears paralyzed by the difficulties of generating reliable estimates and is peculiarly critical of others' independent attempts to develop such aggregate estimates. OMB can improve the Draft Report significantly by doing what Congress directed instead of offering excuses for avoiding its legislated responsibilities. Specifically, it can improve its estimates of the benefits and costs of federal regulation by:

Preventing the perfect from becoming the enemy of the good;

Putting the estimates of aggregate costs and benefits in some context;

Using the best available information;

Improving the quality of cost estimates; and

Improving the quality of benefit estimates.

Preventing the Perfect from Becoming the Enemy of the Good
In its Draft Report, OMB acknowledges the well-known limitations of aggregate benefit and cost estimates, and then spends an inordinate amount of time trying to argue that its own aggregate estimates are superior to those developed by others. Some readers of the Draft Report might conclude that OMB's purpose is primarily to undermine these independent estimates rather than to provide any new insights.

The Draft Report argues that it is impossible to estimate costs and benefits with any degree of precision, in part because of what it refers to as "the baseline problem." While it is true that characterizing an accurate baseline from which to estimate costs and benefits is important and difficult, OMB seems to throw up its hands because the "true counterfactual" never happened. It laments: "Could a civil society even exist without regulation? In other words, what do we use as the baseline for a world without any regulation?"12

Historically, OMB has assumed that as long as the rules remain on the books, the costs are attributable to the regulations. If OMB really believes that there is a rising secular baseline reflecting the assimilation of regulatory requirements into day-to-day life, however, it should be recommending that these rules be removed from the Code of Federal Regulations on the basis that they are no longer needed.

This hand-wringing might be a little more appropriate if OMB's approach to estimating costs depended on a macro model of regulatory costs and benefits that required as its baseline a world with no regulation. But OMB's estimate of benefits and costs is based on summing the incremental costs of individual regulations. On this incremental level, OMB recognizes problems with both ex ante and ex post estimates of costs. It also notes that most costs are based on direct expenditures rather than on consumer and producer surplus. Although the report claims this approach can overstate as well as understate costs, most regulatory economists probably would agree that the full social costs of regulation are substantially greater than the direct private cost.13

OMB also observes that existing estimates of total regulatory costs and benefits are based on studies which "vary in quality, methodology, and type…."14  Nevertheless, it should provide its best professional comparison of apples and oranges, and should recommend that agencies comply with common, consistent methods for estimating the costs and benefits of regulation. The inability of federal agencies to adhere to OMB's guidelines is a key reason for these variations. OMB should enforce its economic analysis guidelines to ensure greater consistency in agency estimates of costs and benefits.

Putting the Estimates of Aggregate Costs and Benefits in Context
OMB should put its estimates in some relative context. For example, how has this burden changed over time? The sheer growth in regulatory burden would be informative, yet OMB does not report it, other than in a passing reference that costs as a percent of GDP were roughly the same in 1997 as in 1988. OMB should present more information along the lines of that provided by Thomas Hopkins, Arthur J. Gosnell Professor of Economics at the Rochester Institute of Technology,15  who measures regulatory costs over time (often going back to 1977) and the incidence of those costs on households and by type and size of business.

An additional analytical refinement OMB should consider is looking at the relative impacts on different sectors of the economy. Since OMB has estimated total benefits as well as costs, it could contribute valuable information on the distribution of those benefits. For example, it would be valuable to gain a sense of the relative benefits and costs of regulations on different socioeconomic classes; even OMB has suggested in E.O. 12866 that distributional impacts need to be considered. With the growing interest in environmental equity, quantitative estimates of who bears the costs of, and gains the most benefits from, regulatory activity would enlighten policy decisions. OMB should begin to move in this direction, and one way to do so is to disaggregate the benefits and costs of each major rule reported by each major provision of the rule. In addition, since E.O. 12866 sought for the first time to have agencies include distributional analysis, OMB should summarize such analyses performed by the agencies.

Using the Best Available Information
OMB's estimates of benefits and costs are based not on the best information available to it, but on cost and benefit estimates reported in the literature for rules issued before 1990, and a combination of agency and other published estimates for the incremental costs and benefits of significant regulations issued since 1990. The legislative history for Section 645 suggests that, while Congress did not anticipate that OMB would start from scratch to conduct an original analysis, it should supplement existing information where needed. OMB's methodology can be strengthened considerably by making an effort to draw on such resources and information available to it in fulfilling its central regulatory review responsibility.

OMB should establish a consistent method for estimating the costs and benefits of rules by relying, largely but not exclusively, on data available to it from its rulemaking record. In addition, if the information is to be truly useful in the debate over how to reform regulations, the public needs to have enough information about how agencies derive their estimates in order to question or reproduce those results. OMB should begin to build a database on the costs and benefits of regulation, logging in a systematic way the information it receives as part of its review of significant regulations. In addition, for major rules not reviewed, it should still make an effort to capture basic estimates on costs and benefits in this database. OMB can and should work with the GAO and utilize its comprehensive database on regulations to identify those regulations that fall outside of its direct review but for which significant costs and benefits should be captured.

Improving the Quality of Cost Estimates
To derive an estimate of the cost of federal regulation, OMB first uses the estimates from two other sources to arrive at a 1988 baseline estimate of regulatory cost. These sources are the EPA (for environmental regulations) and Hahn and Hird (for other social and economic regulations).16  It then estimates the incremental costs of environmental and other social regulations from a review of major regulations individually reviewed by OMB under E.O. 12291 and E.O. 12866 between 1987 and 1996. OMB's Draft Report suggests that the costs are overstated by a factor of two:

While our estimates do not include the costs of regulations with costs below $100 million and there is a possibility that agencies understate the costs of proposed rules (Hopkins, 1992, p. 13), we believe that, if anything, the estimates overstate actual direct costs because of the rising baseline phenomenon discussed above. For example, as a sensitivity analysis, it does not seem implausible that, for environmental and other social regulations over ten years old, no more than half of compliance costs would likely be saved if these Federal regulations magically disappeared over night.17

The report goes on to conclude that the total cost of social regulations would fall by half, to $130 billion per year.

It is fortunate that OMB does not use this figure for its best estimate, because this statement is simply not true. In fact, the studies from which these costs are taken all reach the opposite conclusion: that costs are understated (largely because direct compliance costs do not reflect the full costs of regulation).18  OMB's own estimate of the costs of major rules published between 1987 and1996 includes a caveat that "the total costs of regulation are understated because not all major rules have quantified cost estimates and the costs of non-major rules are not included."19 Indeed, many regulatory costs, particularly those associated with so-called gatekeeper regulations that delay or limit the introduction of new products, impose significant costs in terms of both health and welfare that are not included at all in these regulatory cost statistics.

Furthermore, even the rising baseline phenomenon to which OMB refers (that is, that companies would continue to use certain technologies and management practices if regulations requiring them were lifted) does not suggest that these costs should not be attributed to regulation. They are regulatory costs, even if they would not be avoided with regulatory reform. Also, regulations themselves can create fears that raise the "baseline" unnecessarily. Regulation of r-DNA modified organisms, for example, creates the impression that these organisms are somehow more dangerous than most experts would agree they are. This unfounded fear imposes social costs because it inhibits the development and acceptance of new medical, environmental, and agricultural products that could improve the quality of life.

To estimate the total cost of economic regulation, OMB adds to the 1988 baseline from Hahn and Hird an estimate of the incremental costs based on modifications to calculations done by Hopkins.20  OMB excludes Hopkins's transfer and paperwork costs, and estimates costs that are $370 billion per year less. This new methodology is entirely inconsistent with OMB's approach in the past.

OMB's own economic analysis guidance21  suggests that "principles of full disclosure and transparency" should be applied to transfer costs and highlighted in some way, even if they cannot be quantified. When the federal government issues a regulation that redirects resources from one party to another (as in, for example, a change in food stamp eligibility requirements that will increase the number of beneficiaries and federal budget outlays), the reallocation of resources should be accounted for somewhere. Although transfer costs do not always represent net social costs, they should be addressed, listed separately, and quantified to the extent possible.

OMB also excludes Hopkins's paperwork costs, partly because it claims they already are included in the cost estimates supplied by agencies and reviewed by OMB; yet it is not on these estimates that OMB's aggregate cost figure is based. Moreover, there are thousands of paperwork generating rules that OMB never reviews because they are not "economically significant," so it is not clear how it would have captured these costs. OMB has probably underestimated significantly the costs of paperwork required by regulation. Under the Paperwork Reduction Act, it has access to the best information on paperwork costs of federal actions. This information should be presented clearly in the Draft Report.

OMB is critical of the comprehensive research done in the field of regulatory cost accounting, noting that it "produces large numbers and it creates confusion. It produces large numbers by including 'costs' that are not normally considered as part of the regulatory reform debate" (referring to the work of Thomas Hopkins).22

When it comes to paperwork, many individuals and small businesses are not likely to agree with the OMB statement that:

costs such as the burden of filling out income tax forms or doing the paperwork needed to get visas, passports, small business loans, and veterans benefits are not what one usually thinks about when worrying about the cost of regulation. Nor do we usually think that the income gained by farmers from price support programs or the increased sales by domestic businesses as a result of trade protection are costs of regulation.23

As recently as December 1996, OMB expressed a very different view. In its report on achievements under E.O. 12866, it expressed pride in the Administration's efforts to reduce paperwork costs, observing that "When people speak of regulatory burden, they are usually referring to recordkeeping or reporting requirements, i.e., paperwork."24

Most certainly, both these transfer costs and the paperwork burden are costs of regulation. Just as transfers provide real benefits to recipients, the costs they impose on other sectors are equally real. For example, American consumers have a right to know how much they are donating to farmers each year through price support programs. Moreover, programs like dairy price supports may well represent regressive taxes because they impose the greatest burden on lower-income families with children. As OMB notes in the first chapter of the Draft Report, "those who bear the costs of a regulation and those who enjoy its benefits often are not the same people."25  OMB should follow itsown guidance and not ignore the "distribution of the net effects of a regulatory alternative across the population and economy, divided in various ways (e.g., income groups, race, sex, industrial sector)."26

As noted, a few months ago, OMB claimed as regulatory reductions the very same "costs" it now dismisses so derisively. OMB's unjustified dismissal of independent regulatory accounting efforts and its selective approach to tallying costs have two unfortunate consequences. First, OMB has produced a less than objective estimate of the total costs of regulation, based on previously published estimates rather than on the unique and valuable information available to OMB. Second, OMB paints an inaccurately bleak picture of the state of the art of regulatory accounting.

Improving the Quality of Benefit Estimates
Just as OMB's Draft Report suggests that costs are overstated, it also suggests that benefits are likely to be understated: "We can only conclude that estimates of the total benefits of regulation will be understated by an unknown amount until all significant benefits are monetized."27  Yet the report recognizes that the source of the estimates (Hahn and Hird) cautions that they "are likely to substantially overstate actual net benefits."28 OMB should be objective in its presentations and not endeavor to create the impression of net benefits that are larger than it estimates.

OMB uses the Hahn and Hird estimates of the benefits of regulations in 1988 as its baseline. However, rather than examine the benefits estimates reported in the economic impact analyses for major rules issued since 1988, as it did for its estimate of the incremental costs of regulation, OMB relies on a ratio. This ratio is based on estimates of costs and benefits by Robert W. Hahn,29 who in 1996 reviewed agency estimates of costs and benefits presented in the economic impact analyses for rules issued since 1990. OMB offers no explanation for why it did not review these analyses itself, as it did for costs. Nor does it explain why it chose to use benefit/cost ratios based on Hahn rather than Hahn's estimate of benefits. The report states that OMB does not have adequate information to convert Hahn's present value estimates into annualized estimates, but that is a simple calculation. Using Hahn's discount rate of 5 percent and his 20-year time horizon, his present value of $500 million is equivalent to annualized benefits of about $40 billion per year. Using this $40 billion figure, rather than the $160 billion per year figure implicit in OMB's benefit figures, would alter OMB's estimates of net benefits dramatically.

Complicating a review of its approach, OMB does not report the incremental benefits of the three categories of regulation from 1988 to 1997. Though hidden in the aggregate cost and benefit totals of the Draft Report (see Table 1), it appears that OMB may have used the wrong best estimate from Hahn and Hird for its benefits of environmental regulation. (Hahn and Hird present a "best estimate" that is lower than the midpoint of their range.) OMB also may have used a number that is too low for the baseline benefits of social regulations. (This would increase the total benefits estimate.)

Despite the information available to OMB, its method here is overly simplistic and inconsistent with its approach to estimating costs. Better data are available, and OMB should use them to make a more robust estimate of the total benefits of federal regulation. At a minimum, OMB should present its assumptions clearly, to allow for more informed review, and use consistent approaches for estimating costs and benefits.

Broaden the Review of "Significant Regulations"

Congress required OMB to provide "estimates of the costs and benefits…of each rule that is likely to have a gross annual effect on the economy of $100,000,000 or more in increased costs." These commonly are referred to as "major" rules. Yet OMB summarizes only a small subset of such rules in its Draft Report. For its final report, OMB should respond to Congress's directive by expanding the scope of its review to capture many more of the rules that have a significant impact on the economy.

For reasons unexplained, OMB reported on the costs and benefits of just 22 major final regulations reviewed during the period between April 1, 1996, and March 31, 1997. Nothing in the legislation or legislative history suggests that Congress was interested only in information on a sample of rules issued during a one-year window. Moreover, OMB has significant previous work and resources from which to draw. In its Budget for Fiscal Year 1992, for example, OMB presented a table on the risks and cost-effectiveness of selected health and safety regulations published between 1970 and 1991. The database used to compile that table could provide the foundation for a more robust evaluation of costs and benefits of all significant rules, as Congress required in Section 645.

OMB neglected to report on 19major final regulations issued during its selected time period because they principally involve "transfers" from one group of citizens to another. As noted, such regulations often have serious effects on the allocation of resources and on individual and firm behavior. These transfer costs and benefits of rules should be quantified in the final report (and presented separately from social costs and benefits). OMB should include estimates of the real social costs and benefits associated with such transfer payments, as well as the magnitude of such transfer costs.

OMB also ignored major regulations issued by the various independent regulatory agencies, such as the Securities and Exchange Commission (SEC), Federal Communications Commission (FCC), Federal Energy Regulatory Commission (FERC), and Nuclear Regulatory Commission (NRC). For example, the SEC's controversial derivatives disclosure rules, issued in January 1997, certainly would fall under Congress's definition of an economically significant rule.

The GAO maintains a database of all regulations issued, both major and non-major, as part of its responsibility under the Congressional Review Act.30  According to the GAO's database, 65 major rules were issued during the time period covered by OMB's report (April 1, 1996, to March 31, 1997). The difference between OMB's count of 41 rules and the GAO's count of 65 reflects OMB's omission of major rules issued by independent regulatory agencies whose rules it does not review. However, for purposes of this OMB report,they should have been included. The fact that OMB does not review a rule does not mean it has no cost.

Finally, OMB revealed nothing at all about those regulations that were proposed but not finalized during the chosen reporting period. It correctly notes that during the period, "several particularly significant proposals [were] reviewed by OMB's Office of Information and Regulatory Affairs (OIRA): EPA's two proposals in November 1996 to revise the National Ambient Air Quality Standards [NAAQS] for Particulate Matter and Ozone; EPA's proposal in the summer of 1996 expanding the industries covered by the Toxic Release Inventory; and [the Food and Drug Administration's] January 1997 proposal regarding Animal Proteins in Ruminant Feed."31  The NAAQS rules, which were issued in final form in July 1997, arguably are the most expensive two rules ever issued. By the EPA's own estimates,32  the partial costs of the ozone rule far outweigh the benefits. The full cost of these two rules alone has been estimated at over $100 billion per year and, as such, would increase by one-third OMB's estimate of the total costs of all federal regulations. Both of these rules were issued in final form before the Draft Report was released for comment. Given the relative magnitude of the costs of these rules, they should have been included.

OMB's narrow interpretation of its mandate is not consistent with congressional intent. Its approach excludes a large number of rules that may provide large benefits and impose significant costs on society. As a result, the Draft Report presents a highly misleading and potentially biased picture of the costs and benefits of significant regulations.

Assess the Quality and Reliability of Agency Economic Analyses

OMB's narrow selection of rules might be more understandable if it had undertaken to conduct an independent analysis of the estimates of the costs and benefits of the covered rules. However, its estimates of the benefits and costs of significant federal regulations is simply a compilation of estimates provided by agencies in their regulatory impact analysis statements. OMB reports these agency estimates without providing any commentary on the reliability of those estimates or the quality of the analysis underlying them. Because it reviews all "economically significant" rules according to the principles of E.O. 12866, it is in a unique position to know the extent to which these estimates accurately reflect the impact (beneficial as well as negative) of regulatory actions. Although it narrowed the coverage of the report to a small fraction of relevant rules, OMB fails to provide any independent information, analysis, or evaluation.

OMB's failure to provide any independent evaluation of the quality and reliability of agency benefit and cost estimates would be much less worrisome if there were persuasive evidence that agencies in fact developed these estimates in compliance with OMB economic analysis guidance. Agency adherence to the Administration's own guidance would be a positive first step toward ensuring that federal regulations actually provide benefits that outweigh their costs. In 1995, the Clinton Administration opposed regulatory reform legislation by arguing that many of the proposed principles already were being implemented administratively by the Administration through vehicles such as E.O. 12866 on Regulatory Planning and Review. Indeed, the Draft Report suggests that the guidelines do have a role in regulatory over sight: "The principles that are described in detail in Best Practices are summarized here because they can serve as an introduction to how we have evaluated the studies on the costs and benefits of regulation discussed in the following chapters."33

If it is true that OMB evaluates rules based on this guidance, it certainly is not evident from the Draft Report. A review of some recent significant rules illustrates how agencies appear to be issuing significant rules based on analysis that blatantly violates OMB's basic principles for assessing benefits and costs as outlined in its Draft Report.

Problem #1: Costs and benefits are not measured relative to a realistic baseline
OMB's Draft Report urges that "costs and measured relative to a baseline...constructed to reflect policy in the absence of the regulation being evaluated, consistent with pending government actions, and applied equally to benefits and costs."34 The EPA's ozone NAAQS rule provides a vivid illustration that this is not followed consistently. The EPA's analytical baseline included (1) overestimates of the effectiveness and implementation of new regulations included in the baseline and (2) emission reductions from actions that, in fact, would be attributable to the standards being evaluated.

Problem #2: Agencies are not using consistent methodologies; hence, their results are not comparable. OMB's Draft Report states that:

Costs and benefits should be presented in a way to maximize their consistency or comparability…. It is important to keep in mind the larger objective of consistency-subject to statutory limitations-in the estimates of benefits applied across regulations and agencies for comparable risks. Failure to maintain such consistency prevents achievement of the most risk reduction from a given level of resources spent on risk reduction.35

This principle is very important if any meaningful form of regulatory accounting is to be conducted. Yet a comparison of methods and assumptions across the significant rules presented in OMB's report reveals that it is not being followed. For example, the Department of Transportation (DOT) and the Food and Drug Administration (FDA) used a discount rate of 7 percent, as directed by OMB Circular A-94, in their roadway worker protection and tobacco rules, respectively. The EPA, in its accidental release prevention rule, used a discount rate of 3 percent. For valuing reductions in premature mortality from their rules, both the EPA and the DOT estimated statistical lives saved. However, the values they apply differ by a factor of two-the EPA assigned a value of $5.4 million per life saved, while the DOT valued a statistical life at $2.7 million. The FDA, on the other hand, valued discounted life years (rather than statistical lives) at $117,000 per life year.

The FDA's approach provides more valuable information to decision-makers because it reflects the life expectancies of the beneficiaries of an action. The EPA Advisory Council on Clean Air Compliance Analysis recommended in October 1996 that "premature stated in terms of the expected number of life years saved, and that life years be valued instead of statistical lives." The same recommendation was made last year by a panel of 11 experts on regulatory analysis that includes, among others, Nobel laureate Kenneth J. Arrow: "a program that extends a life by thirty years should be valued more highly than one that extends it for three. Where policies are expected to extend a life, it is better to estimate the number of life-years extended than solely the number of lives."36

One major concern raised about the EPA's particulate matter (PM) NAAQS was that its projected benefits were dominated by its estimates of premature mortality, which the EPA valued in terms of statistical lives saved rather than life years. Yet exposure to particulates is expected to shorten life spans by two years37  or less.38  How can one compare the benefits of the PM rule, in which an additional two years of life is valued at $4.8 million, to the FDA's tobacco rule, in which the same benefit is assigned a value of $234,000? Certainly, this problem is not insurmountable if agencies follow consistent approaches.

Problem #3: Agencies do not evaluate alternative regulatory options
OMB's Draft Report states that "An economic analysis cannot reach a conclusion about whether net benefits are maximized-the key economic goal for good regulation-without consideration of a broad range of alternative regulatory options."39  This is arguably the most basic and most important of OMB's principles, yet it appears to be violated almost universally. The statutory basis for some regulations limits the types of alternatives the agencies are permitted to select; yet, even in those cases, information on the benefits and costs of alternatives outside the statutory framework can help decision-makers in the future.

In many cases, it appears that agencies select a preferred regulatory alternative and then evaluate its costs and benefits. For example, the final economic impact analysis for the Department of Transportation's roadway worker protection rule estimated benefits and costs only for the approach the DOT selected.40  Similarly, the final Economic Impact Analysis supporting the EPA's regulation of lead-based paint activities in target housing examined only the standards and requirements imposed by the rule.41

In its Hazard Analysis and Critical Control Point (HACCP) rule, the Department of Agriculture (USDA) analyzed two alternatives: the proposed rule and the final rule. In its final economic impact analysis, the USDA stated: "After considering broader regulatory approaches including market incentives and voluntary industry standards, Food Safety and Inspection Service (FSIS) has determined that effective process control is needed throughout the meat and poultry industry in order to minimize pathogen contamination of food products and lower the risk of subsequent foodborne illness."42  Yet the USDA's consideration of the "broader regulatory approaches" allowed under its statute did not include any estimates of their relative benefits and costs; it was not until after the USDA chose its preferred option that it estimated its costs and benefits.

Even in cases where agencies appeared to examine alternatives, closer examination reveals that those alternatives reflect minor alterations in the preferred regulatory strategy rather than viable policy options. For example, the final economic impact analysis for the EPA's accidental release prevention rule presented three alternatives: the final rule, an option that imposed final rule requirements on a greater number of sources, and the proposed rule.43 In both the USDA and EPA analyses, the benefits for each option were the same, and each agency could claim to maximize cost-effectiveness by choosing the alternative with the lowest costs.

Clearly, agency staffs have little incentive to present to policymakers any estimates of costs and benefits that might favor options they do not prefer, but a fair and reasoned determination of which alternatives should be selected requires objective analysis. One of the key benefits of executive oversight of individual agency actions should be to ensure that regulatory actions do more good than harm by requiring that they be supported by reliable analyses of their impacts (both beneficial and otherwise).

Problem #4: Policy is based on unsupported assumptions about costs and benefits
OMB's economic analysis guidelines state that "Where benefit or cost estimates are heavily dependent on certain assumptions, it is essential to make those assumptions explicit, and where alternative assumptions are plausible, to carry out sensitivity analyses based on the alternative assumptions."44 For several of the significant rules presented in the Draft Report, agenciesadmit that they have little information on how effective the rule will be at achieving its goals (for example, the USDA's HACCP rule, the EPA's accidental release rule, and the EPA's lead-based paint activities rule).

The benefits of the DOT roadway worker protection rule are dominated by its assumptions regarding worker productivity increases associated with a safer working environment. The DOT estimated these benefits using a range of percentage productivity increases, and coincidentally selected as its best estimate an effectiveness percentage that results in total benefits that just exceed total costs. Rather than presenting the range of possible benefits, OMB's Draft Report presents only the DOT's point estimate.

Problem #5: Agencies are not fully describing the uncertainties in their estimates
OMB's Draft Report also states that:

The large uncertainties implicit in many estimates of risks to public health, safety, or the environment make treatment of risk and uncertainty especially important. In general, the analysis should fully describe the range of risk reductions, including an identification of the central tendency in the distribution; risk estimates should not present either the upper-bound or the lower-bound estimate alone.45

However, commenters on OSHA's methylene chloride rule expressed concern that OSHA misrepresented the precision of its conclusions regarding the health benefits. Also, the economic impact analysis for the EPA's Ozone NAAQS proposal presented a large range in estimated benefits (between $12 million and $2.1 billion for partial attainment). The EPA noted that "the high estimate of the range includes the results of the mortality category, which comprises greater than 98% of the total estimate.…"46  The economic impact analysis does not clearly present the weight of the evidence against a causal relationship between ozone concentrations and daily mortality, though the EPA's best estimate (central tendency) of benefits, reported in an appendix to the economic impact analysis, excludes these questionable mortality benefits.

Problem #6: Agencies are not quantifying the benefits of their rules
OMB's Draft Report states that "[A]n attempt should be made to quantify all potential real benefits to society in monetary terms to the maximum extent possible, by type and time period."47  Yet, of the 22 rules included in OMB's review, only eight were supported with benefits estimates expressed in monetary terms. Among those, the approaches to estimating the benefits varied considerably, and estimated benefits were based on tenuous linkages between the regulatory action and the predicted benefits. For example, the USDA's hazard analysis and critical control points rule (HACCP) is not based on knowledge of (1) the extent of the problem (as in, say, cases of food-borne illnesses), (2) the degree to which contamination is caused at the manufacturing stage, or (3) the effectiveness of process controls. The USDA admits to "being without the knowledge to predict the effectiveness of the requirements in the rule to reduce foodborne illness."48

Before imposing rules that are expected to cost Americans hundreds of millions of dollars a year, agencies should have a better idea as to whether they will be effective. The EPA's toxic release rule illustrates the same problem. Regardless of the alternative approach it examined, the agency assumed the same net benefits. The large dollar benefits estimated for both these two rules are based on the assumption that huge health benefits will be achieved merely through requirements to write plans.

Problem #7: Agencies use questionable approaches in valuing benefits
OMB's Draft Report states:

Where market transactions are difficult to monitor or markets do not exist, analysts should use appropriate proxies that simulate willingness-to-pay based on market exchange.... Contingent-valuation methods have become increasingly common for estimating indirectly traded benefits, but the reliance of these methods on hypothetical scenarios and the complexities of the goods being valued by this technique raise issues about its accuracy in estimating willingness to pay compared to methods based on (indirect) revealed preferences.49

However, EPA's ozone NAAQS values health effects based on contingent valuation surveys of individuals to determine their hypothetical willingness to pay to avoid "incidents" such as a day of coughing.

Problem #8: Agencies are not measuring the opportunity cost of their actions
According to OMB's Draft Report, "The preferred measure of cost is the 'opportunity cost' of the resources used or the benefits forgone as a result of the regulatory action.... For example, the opportunity cost of banning a product (e.g., a drug, food additive, or hazardous chemical) is the forgone net benefit of that product, taking into account the mitigating effects of potential substitutes."50  Yet agencies almost always limit cost analysis to direct compliance costs. This is especially problematic in rules, such as the ozone and PM air quality standards, where compliance will require major lifestyle changes, or in the new, tighter standards for methylene chloride, where the most likely available substitute poses a greater imminent hazard.

In conclusion, OMB's economic analysis guidelines reflect sound principles and practices for regulatory analysis and review. However, in "reestablish[ing] the primacy of the agencies in regulatory decision making,"51  E.O. 12866 appears to have limited OMB's authority to enforce these guidelines. As the above discussion highlights, evidence abounds that agencies are ignoring the guidelines and issuing rules with little assurance that they will not cause more harm than good.

OMB should develop a simple checklist of the most important principles in its guidance to agencies ("Economic Analysis of Federal Regulations Under Executive Order 12866," dated January 11, 1996) and report whether or not the agencies complied with these principles for each rule. This should not be difficult to do for the Draft Report to be submitted by September 30, 1997.

Recommend Changes in Regulations or Regulatory Programs

Congress asked OMB, in its report, to make "recommendations to reform or eliminate any Federal regulatory program or program element that is inefficient, ineffective, or is not a sound use of the Nation's resources." However, OMB concludes in Chapter IV of its Draft Report that "we do not believe substantial economic evidence exists on which to base proposals for major reforms or elimination of social regulatory programs or their elements,"52 and then asks for public comment. OMB's evasive posture is a dramatic reversal of positions it has held since it was called the Bureau of the Budget. Historically, OMB has been the sole government agency willing and able to identify programs and policies that have failed to fulfill expectations. Advocates of good government in both political parties have depended on OMB to press for the reform or elimination of such programs. Ever since the beginning of executive review of regulations in the early 1970s, OMB has carried the same determination and judgment into the arena of federal regulations.

OMB's inability to include any recommendations for reform casts doubt on the White House's initiatives to "reinvent government," including "reinventing regulation," that kicked off in early 1995. According to the Draft Report, available information on the costs and benefits of regulation is too inadequate to conduct simple regulatory accounting or to make even the most general suggestions for improvements in regulatory programs. How, then, can anyone say the information was adequate for imposing huge new regulatory programs on society?

As Congress requested, OMB should provide its expert and professional opinion-based on two decades of experience working with the federal agencies on regulatory development and review-of the programs that should be reformed or eliminated. As Senator Thompson points out, "OMB does not have to be omniscient to propose such improvements, and its recommendations do not have to be based on perfect empirical data. Let's also use common sense and work together for the public good."53  Such recommendations have been made in the past, and OMB should not remain silent. OMB serves the American public, and that public has a right to know where there is a need for improvement. Absent such recommendations, the OMB report will be rendered misleading and devoid of credibility by suggesting that there is absolutely no room for improvement in the way the federal regulatory system operates.


This paper is adapted from comments submitted by the authors to the Office of Management and Budget on August 29, 1997.

2 Issued September 30, 1993.

3 U.S. Office of Management and Budget, More Benefits Fewer Burdens, Creating a Regulatory System That Works for the American People, December 1996, p. 13.

4 U.S. Office of Management and Budget, "Draft Report to Congress on the Costs and Benefits of Federal Regulations, Notice and Request for Comments," Federal Register, July 22, 1997, pp. 39351-39383; hereinafter cited as Draft Report.

5 Statement of Senator Fred Thompson, Congressional Record, July 17, 1997, p. S7701.

6 See U.S. Office Management and Budget, Regulatory Program of the United States Government, April 1, 1992-March 31, 1993, and earlier years.

7 Draft Report, p. 39359.

8 U.S. Office of Management and Budget, Budget of the United States Government, Fiscal Year 1993, pp. 1-397.

9 U.S. Office of Management and Budget, Regulatory Program of the United States Government, April 1, 1991-March 31, 1992, p. 6.

10 See U.S. Office of Management and Budget, OMB Mid-Session Review: The President's Budget and Economic Growth Agenda, July 24, 1992, pp. 396-401, and Budget Baselines, Historical Data, and Alternatives for the Future, January 1993, pp. 114-115.

11 See Treasury, Postal Services and General Government Appropriations Act, 1997 (Public Law 104-208), Section 645(a).

12 Draft Report, p. 39359.

13 See, for example, U.S. Environmental Protection Agency, Environmental Investments: The Cost of a Clean Environment, December 1990. See also M. Hazilla and R. Kopp, "The Social Cost of Environmental Quality Regulations: A General Equilibrium Analysis," Discussion Paper QE89-11, Resources for the Future, Washington, D.C., March 1989.

14 Draft Report, p. 39360.

15 Thomas D. Hopkins, Regulatory Costs in Profile, Center for the Study of American Business Policy Study No. 132, August 1996.

16 See Robert W. Hahn and John A. Hird, "The Costs and Benefits of Regulation: Review and Synthesis," Yale Journal on Regulation, Vol. 8, No. 1 (Winter 1991). See also EPA, Environmental Investments: The Cost of a Clean Environment.

17 Draft Report, p. 39362.

18 "Compliance costs do not fully reflect social costs because they neglect direct regulatory impacts that do not involve out-of-pocket costs as well as the intertemporal and secondary effects of environmental protection. In other words, they do not account for the dynamic, general equilibrium effects created throughout the economy that impose costs on industries and households not directly affected by regulation. Environmental protection imposes costs on virtually all economic entities-including the general public-that are largely hidden. Examples of social costs imposed by pollution controls that are not reflected in direct compliance cost estimates include lost or delayed production and consumption opportunities, reduced economic productivity, and higher price inflation. Some recent research suggests that compliance cost estimates may understate substantially the true long-term costs of pollution control." EPA, Environmental Investments: The Cost of a Clean Environment. See also Hahn and Hird, The Costs and Benefits of Regulation.

19 OMB, More Benefits Fewer Burdens, p. A5.

20 Thomas D. Hopkins, "Cost of Regulation: Filling the Gaps," Report Prepared for the Regulatory Information Service Center, Washington, D.C., August 1992.

21 U.S. Office of Management and Budget, "Economic Analysis of Federal Regulations Under Executive Order 12866," January 11, 1996.

22 Draft Report, p. 39361.

23 Ibid.

24 OMB, More Benefits Fewer Burdens, p. 28.

25 Draft Report, p. 39358.

26 Ibid.

27 Ibid., p. 39363.

28 Ibid., p. 39364.

29 See Robert W. Hahn, "Regulatory Reform: What Do the Numbers Tell Us?" in Risks, Costs, and Lives Saved: Getting Better Results from Regulation, ed. Robert W. Hahn (New York: Oxford University Press and AEI Press, 1996).

30 The Congressional Review Act is subtitle E of Title II (The Small Business Regulatory Enforcement Fairness Act) of the Contract With America Advancement Act of 1996, Public Law 104-121. This subtitle establishes an expedited process by which Congress may review and disapprove essentially all federal regulations. Agencies issuing final rules must submit each rule to Congress and the U.S. General Accounting Office as soon as practicable.

31 Draft Report, Chapter III, footnote 1.

32 U.S. Environmental Protection Agency, "National Ambient Air Quality Standards for Particulate Matter; Final Rule," Federal Register, Vol. 62, No. 138 (July 18, 1997), pp. 38651-38701.

33 Draft Report, p. 39357.

34 Ibid., p. 29258.

35 Ibid., p. 39358.

36 Kenneth J. Arrow et al., Benefit-Cost Analysis in Environmental, Health, and Safety Regulation-A Statement of Principles (Washington, D.C.: American Enterprise Institute, 1996), p. 13.

37 See Bob Herbert, "Bad Air Day," The New York Times, February 10, 1997, p. A15.

38 See Robert W. Crandall et al., "Clearing the Air," CATO's Journal of Regulation, No. 4 (1996), p. 44.

39 Draft Report, p. 39358.

40 U.S. Department of Transportation, Federal Railroad Administration, "Roadway Worker Protection Final Rule Regulatory Impact Analysis," September 27, 1996.

41 Abt Associates, TSCA Title IV, Sections 402(a) and 404, "Target Housing and Child-Occupied Facilities Final Rule Regulatory Impact Analysis," August 1996, pp. 3-14 to 3-23.

42 "Pathogen Reduction; Hazard Analysis and Critical Control Point (HACCP) Systems," Federal Register, July 25, 1996.

43 See U.S. Environmental Protection Agency, "Economic Analysis in Support of Final Rule on Risk Management Program Regulations for Chemical Accidental Release Prevention," May 21, 1996.

44 Economic Analysis Guidelines, January 11, 1996, p. 21.

45 Draft Report, p. 39358.

46 EPA, "National Ambient Air Quality Standards for Particulate Matter," Regulatory Impact Analysis, Table IX-3.

47 Draft Report, p. 39358.

48 OMB's Draft Report cites USDA's rule at 61 FR 38956.

49 Draft Report, p. 39358.

50 Ibid., p. 39358-39359.

51 Ibid., p. 39357.

52 Ibid., p. 39382.

53 Congressional Record, July 17, 1997, p. S7701.

54 During the 104th Congress, Representative Thomas Bliley (R-VA) introduced the Regulatory Accounting Act of 1995, which would have required the President, after a notice and comment period, to report to Congress on the total costs and benefits of agency regulations for the current and upcoming five fiscal years.

55 For a history of legislative proposals related to regulatory budgeting, see Clyde Wayne Crews Jr., Promise and Peril: Implementing a Regulatory Budget, Competitive Enterprise Institute, April 1996, pp. 7-9.

56 Ibid.

The Next Steps: What Congress Should Do

There are two basic steps Congress can take: It can make sure the OMB report does what the law requires, and it can make the report a permanent requirement. In addition, it should keep working toward a regulatory budget.

Make sure the report does what the law requires
Congress needs to hold OMB accountable for delivering on the requirements of the law. The committees of jurisdiction should make sure that OMB knows what is expected even before the final report is issued. When the final report is delivered to Congress by September 30, 1997, Congress should review it closely; and if the report fails to make a good-faith effort to address the statutorily required elements, Congress should refuse to accept it and should send it back to OMB, demanding that a good-faith effort be made.

Make the OMB regulatory accounting report-the public's "regulatory right-to-know" report-a permanent requirement
Rather than continuing to direct OMB through its appropriations to issue a report, Congress-at a minimum-should direct OMB to issue this report periodically, allowing adequate time for public notice and comment. Both the House and Senate are considering legislative proposals that would require OMB to submit a regulatory accounting report on a regular basis.54

Keep working toward the long-term goal: a regulatory budget
Over time, and with the refinement of cost-benefit methods and expansion of trend data on regulatory costs and benefits, Congress should move to full implementation of a regulatory budget. The President and Congress should establish a federal regulatory budget that places a ceiling on the total estimated cost that can be imposed on the economy each year by all federal regulations. If the budget total was reached by existing regulations, an agency wishing to add a regulation with additional costs would have to repeal or modify an existing regulation imposing the same or greater cost. A regulatory budget would place a politically accountable limit on the total burden that could be imposed on the economy by federal regulation. It also would give agencies a strong incentive to review existing regulations to discover which could be safely eliminated or modified to reduce costs. Overall, the government would be forced to prioritize its regulations and eliminate the least beneficial ones.

In the past, Congress has considered a number of proposed versions of a regulatory budget. During the 104th Congress, for example, Representative Lamar Smith (R-TX) introduced the Regulatory Accountability Act, which would not have created an overall regulatory cost ceiling but would have worked incrementally to establish a budget for each new law and for each piece of reauthorized legislation.55


Those in Congress and elsewhere who have supported the need to improve the way the federal government makes decisions about whether and how to regulate must take note of this draft OMB report. Although OMB's inaugural effort at accounting for the costs and benefits of regulation is disappointing, it also is a very modest and critical first step in the right direction. As Senator Thompson recently stated, "Estimating the total annual costs and benefits of Federal regulatory programs is like assembling a jigsaw puzzle, and some of the major sections have been assembled. OMB's first draft report will provide a foundation for further improvements."56

But there are several important improvements OMB has the expertise and ability to make right now. These include improving the usefulness and quality of the aggregate costs and benefits estimates, broadening the review of "significant" regulations, independently assessing the quality and reliability of agency economic analysis, and providing recommendations for changes in regulations or regulatory programs.

By requiring OMB to produce this report, Congress has demonstrated that it is possible to begin to get a handle on the size and scope of the federal regulatory system. There is strong bipartisan agreement that publicly disclosing more information about the costs and benefits of regulations will make the federal regulatory system much more accountable to the American public. Congress must hold the Office of Management and Budget accountable for producing a report that is completely responsive to its statutory requirements, and it must require that OMB continue to produce such reports well into the future.


Robert DePosada

Policy Analyst