The Heritage Foundation

Backgrounder #2984 on Economy

December 30, 2014

December 30, 2014 | Backgrounder on Economy

Accurate Budget Scores Require Dynamic Analysis

To predict the budgetary impact of a major federal policy accurately, analysts should take into account the policy’s potential macroeconomic effects. This approach, often called “dynamic analysis,” comports with the normal scholarly practice of macroeconomic research. Academic economists do not publish predictions about macroeconomic changes that fail to take into account the interplay and feedback among relevant variables. Congressional analysts should follow the best practices of macroeconomic research by using dynamic analysis and methodological transparency. The current practice of congressional analysts, as exemplified by the Congressional Budget Office’s score of the Dodd–Frank financial law, ignores macroeconomic and even behavioral responses to the law in question, misleading Congress with the pretense of precision.

Key Points

  1. Major federal policies change the size and composition of the broader economy, changing revenues and outlays in unrelated parts of the budget.
  2. Dynamic analysis is necessary for accurate scoring.
  3. Static analysis ignores the potentially large macroeconomic and budgetary effects that occur through indirect channels and thus offers Congress a pretense of precision.
  4. Academic macroeconomic research uses dynamic methods, and scholars do not publish predictions about variables they have not dynamically modeled.
  5. Dynamic analysis is more difficult and requires more skill and judgment of researchers.
  6. Transparency supports dynamic analysis by exposing methods to peer critique and correction.
  7. When the best available methods cannot predict a law’s macroeconomic effects, analysts should report that the budgetary impact is unknown. In such cases, an honest admission of ignorance is more informative than the false precision of a static estimate.

A‌s the new Congress considers changes to the Joint Committee ‌on Taxation (JCT) and Congressional Budget Office (CBO), it should promote improvement in the methods used for evaluating macroeconomic policy. Currently, the JCT and the CBO often use “static” scoring methods, which make very strong assumptions about equilibrium responses to policy changes, often assuming no change at all. A dynamic approach would be less dogmatic, allowing for and trying to calculate a range of potential effects, and it would improve the accuracy of budgetary estimates.

For the Dodd–Frank financial reform bill and the Affordable Care Act (ACA), Congress received inaccurate budget scores in part because they failed to account for likely macroeconomic effects. The CBO and the JCT scored the Dodd–Frank bill like the opening of a post office. Even though the legislation’s express purpose was to significantly change risk-taking and profit making in American finance, the CBO’s score made no attempt to quantify its macroeconomic impact.[1] CBO Director Doug Elmendorf testified to a House subcommittee, “CBO did not attempt to determine whether the estimated costs under the act would be smaller or larger than the costs of alternative approaches to addressing future financial crises and the risks such crises pose to the economy as a whole.”[2]

The CBO and JCT score of the massive ACA was slightly better: It at least allowed for some behavioral responses.[3] However, the score assumed no labor market impact, despite the ACA’s imposition of significant new requirements on the employer–employee relationship. Nor did the score begin to acknowledge potential effects on overall income. Later on, the CBO acknowledged that the ACA would reduce employment by making work less attractive.[4] Less employment means less personal income, less consumption and investment, less tax revenue, and greater government outlays. The possibility of such effects was not mentioned in the CBO’s initial score.

The potential budgetary impact of macroeconomic responses to Dodd–Frank or the ACA utterly dwarfs the static estimates considered by the CBO and the JCT.

As Congress considers how to improve the services that it receives from the JCT and the CBO, it should look to the economics profession for how best practices are typically applied in economic research. The most glaring gap is the persistence in Washington of inaccurate scoring of policies that may have large macroeconomic impacts.

Static Versus Dynamic Analysis

“Static” and “dynamic” are terms of art. In every economic analysis, some things are held constant (“static”) and some are allowed to change according to mechanical or equilibrium relationships (“dynamic”).[5] In budgetary analysis, the major line between dynamic and static analysis is whether the results account for likely changes in the level of income and tax revenue in the broader economy in response to the policy change.[6] If not, the analysts are implicitly assuming that the policy does not have macroeconomic effects. A major policy with no macroeconomic effect would be surprising.

Dynamic budgetary analysis relaxes assumptions, allowing policy to affect the overall size and makeup of the economy, and it uses well-established macroeconomic techniques to predict those effects. For small policy changes, the results may be negligible, and there will be little difference between the static and dynamic analysis. For major policy changes, the macroeconomic effects may be huge, and the dynamic analysis will produce a much more accurate budget score.

Scholarly Macroeconomic Analysis

The overwhelming majority of macroeconomic analysis is dynamic. Academic papers make predictions only about outcomes that they have dynamically modeled.

For example, economists interested in how the mismatch between workers and jobs adds to unemployment use a model that incorporates employer behavior, worker behavior, and a matching process. The model makes predictions for all forms of unemployment, not just “mismatch” unemployment, and the model’s results can be compared with the data to evaluate its usefulness. If a change in the matching function has feedback effects elsewhere in the job market, the model will pick those up.[7] A static approach—such as estimating the number of mismatched workers and assuming that changes in mismatch unemployment have no feedback effects on hiring or work decisions elsewhere in the labor market—would not be published in a reputable journal.

To demonstrate the total dominance of dynamic analysis in scholarly macroeconomics, I examined dozens of influential papers. The following section touches on widely cited papers in several fields that are relevant to federal policy. This paper reviews the macroeconomic articles published in current issues of three of the most competitive journals in economics. Dynamic analysis is ubiquitous in them. I found no research question that macroeconomists are comfortable answering using static methods.[8]

Policy-Relevant Research Fields Are Dynamically Grounded

These articles analyzed questions in several policy areas.

Fiscal Policy. Academic research on fiscal policy—the size of government and its debt dynamics—owe much to Alan Auerbach and Larry Kotlikoff’s 1987 book, Dynamic Fiscal Policy.[9] Applying dynamic methods to a current question, Richard Evans, Larry Kotlikoff, and Kerk Phillips model the path to fiscal collapse in a case in which politicians perpetually fail to address growing deficits.[10] There is an extensive dynamic literature on sovereign debt, including a landmark paper by Jeremy Bulow and Kenneth Rogoff.[11]

Optimal Taxation. The work of James Mirrlees undergirds the study of how best to design tax systems. Mirrlees’s landmark paper[12] begins by explicitly setting out its assumptions, a hallmark of good dynamic work. The field famously includes the counterintuitive result that under certain conditions, labor is best off when all taxes are paid by labor and none by capital.[13] Tony Atkinson and Joseph Stiglitz note that tax systems should be studied as a whole: “Throughout the paper, we have stressed the importance of the interactions between different taxes, and the fact that a piecemeal approach may be misleading.”[14] More recent work continues to resolve challenges in designing optimal utilitarian tax systems, even in relatively simple frameworks.[15]

Social Insurance. Government transfers such as Social Security and unemployment benefits are forms of social insurance. A review in the Handbook of Public Economics points out that behavioral responses to increased social benefits are “substantially larger than the labor supply elasticities typically found for men in studies of the effects of wages or taxes.”[16] Changes in eligibility rules can affect labor force participation and, hence, change total income and income tax revenue as well.[17] A theoretical paper by Nobel Prize winners Peter Diamond and James Mirrlees found that people’s ability to adjust their savings behavior in the presence of social insurance is a crucial constraint in designing optimal insurance schemes.[18] In a large economy such as the U.S., policies that reduce savings also reduce investment, which feeds directly into growth. Major changes to social benefit eligibility or generosity will have macroeconomic effects.

Other Examples. Every policy area has a related literature. In immigration, the dynamic effects on human capital acquisition may be large enough to reverse a static “brain drain.”[19] Bailouts of bad loans are only good policy under certain conditions, according to a dynamic model by Ben Bernanke and Mark Gertler.[20] An influential paper on financial regulation argued that existing models of financial crises were insufficiently dynamic and requires more extensive modeling of risk-taking and risk pricing under specific regulatory regimes.[21]

Peer-Reviewed Papers Use Dynamic Analysis

The current issues of three leading economics journals included 17 papers on macroeconomics or touching on macroeconomic issues.[22] All but two included full dynamic effects for all of the macroeconomic variables they considered.[23] The other two had limited dynamics, where one macroeconomic variable was dynamic, but second-order effects on other macro variables were not considered.[24] Another paper discussed software for working with dynamic models.[25] Not just the major advances in macroeconomic analysis use dynamic analysis; virtually all quality scholarship does.

A hypothetical paper that used static methods—either assuming away behavioral responses or failing to incorporate potentially large economy-wide outcomes and their feedback—would be received with a mix of puzzlement and hostility in peer review. What could possibly be learned from using such an obviously incomplete approach?

Budget Analysis Using Macroeconomic Dynamics

The transparent use of best practices is central to the job that the JCT and the CBO perform for Congress. Only dynamic analysis can indicate whether or not a major policy’s macroeconomic spillovers or feedback effects have important budgetary implications, and any analysis must be allowed to stand up to peer review and criticism.

An extensive discussion of the specifics of accurate scoring is beyond the scope of this paper, but the interested reader is referred for further reading to The Heritage Foundation’s volume The Secret Chamber or the Public Square?[26]

Dozens of economists have used dynamic analysis to produce accurate scores of diverse tax reform proposals. Because tax changes are quantifiable, the modeling work is straightforward (although painstaking). Publications of the Tax Foundation,[27] research papers by Larry Kotlikoff,[28] and the JCT’s existing macroeconomic modeling work provide a sample of such work.[29]

Dynamic analysis of policies other than taxation is more difficult. Evaluating the impact on the level of economic activity requires a rich modeling framework and rigorous consideration of the key channels through which the policy will work. The CBO and the JCT’s inaccurate approach to Dodd–Frank and the ACA is a poor alternative.

The challenges to estimating accurate scores for major economic policies will give even the best analyst ample opportunity for epistemic humility. That is a feature, not a bug. The penny-wise approach of static analysis has cost enough already.

The weaknesses of dynamic analysis are that it is neither easy nor foolproof. It requires more skill of the analyst and can seem opaque to policymakers. Static analysis is at least transparently wrong. Dynamic analysis requires more knowledgeable oversight and integrity on the part of analysts. Official budget analysis should embrace transparency as well as dynamics, in keeping with best practices in economics.

Finally, some problems that Congress seeks to address are so inherently complex that even the most sophisticated analysis may not be up to the task. In those cases, an honest admission of ignorance is more informative than the false precision of a static estimate.

These challenges can be met. Many economists and statisticians, some already on the congressional payroll, are capable of performing first-class dynamic analysis. Methodological transparency will quickly expose mistakes by congressional analysts to peer critique and correction. It is long past time for the CBO and the JCT to catch up with the economics profession and fully embrace dynamic analysis and accurate scoring of legislation.

—Salim Furth, PhD, is Senior Policy Analyst in Macroeconomics in the Center for Data Analysis, of the Institute for Economic Freedom and Opportunity, at The Heritage Foundation.

About the Author

Salim Furth, Ph.D. Research Fellow, Macroeconomics
Center for Data Analysis

Related Issues: Economy

Show references in this report

[1] Congressional Budget Office, “H.R. 4173: Dodd–Frank Wall Street Reform and Consumer Protection Act: Cost Estimate,” June 28, 2010, http://www.cbo.gov/sites/default/files/hr4173.pdf (accessed December 16, 2014).

[2] Douglas W. Elmendorf, “Review of CBO’s Cost Estimate for the Dodd-Frank Wall Street Reform and Consumer Protection Act,” testimony before the Subcommittee on Oversight and Investigations, Committee on Financial Services, U.S. House of Representatives, March 30, 2011, p. 12, http://www.cbo.gov/sites/default/files/03-29-dodd-frank.pdf (accessed December 16, 2014).

[3] Congressional Budget Office, “H.R. 3590, Patient Protection and Affordable Care Act,” March 11, 2010, http://www.cbo.gov/publication/21279 (accessed December 20, 2014).

[4] Congressional Budget Office, “The Budget and Economic Outlook: An Update,” August 2010, http://www.cbo.gov/sites/default/files/08-18-update.pdf (accessed December 16, 2014), and Congressional Budget Office, “Labor Market Effects of the Affordable Care Act: Updated Estimates,” February 2014, http://www.cbo.gov/sites/default/files/cbofiles/attachments/45010-breakout-AppendixC.pdf (accessed December 16, 2014).

[5] In other applications, the words “static” and “dynamic” have different meanings, such as lacking or having a time dimension. The phrase “comparative statics” is unrelated to the language of budget scoring.

[6] An even more static form of analysis—such as that used in scoring Dodd–Frank—is to assume that a policy has no immediate impacts on behavior. One need not waste space explaining how unscientific this approach is.

[7] The model alluded to here is Ayşegül Şahin et al., “Mismatch Unemployment,” American Economic Review, Vol. 104, No. 11 (November 2014).

[8] The author defines static methods as those that draw conclusions about macroeconomic aggregates that are assumed to be invariant to any modeled mechanism.

[9] Alan J. Auerbach and Laurence J. Kotlikoff, Dynamic Fiscal Policy (Cambridge, U.K.: Cambridge University Press, 1987), http://www.kotlikoff.net/sites/default/files/Dynamic%20Fiscal%20Policy_1.pdf (accessed December 20, 2014).

[10] Richard W. Evans, Laurence J. Kotlikoff, and Kerk L. Phillips, “Game Over: Simulating Unsustainable Fiscal Policy,” National Bureau of Economic Research Working Paper No. 17917, March 2012.

[11] Jeremy Bulow and Kenneth Rogoff, “Sovereign Debt: Is to Forgive to Forget?” American Economic Review, Vol. 79, No. 1 (March 1989).

[12] J. A. Mirrlees, “An Exploration in the Theory of Optimum Income Taxation,” The Review of Economic Studies, Vol. 38, No. 2 (April 1971).

[13] Christophe Chamley, “Optimal Taxation of Capital Income in General Equilibrium with Infinite Lives,” Econometrica, 1986, Vol. 54, No. 3 (May 1986).

[14] A. B. Atkinson and J. E. Stiglitz, “The Design of Tax Structure: Direct Versus Indirect Taxation,” Journal of Public Economics, Vol. 6, Nos. 1–2 (July–August 1976).

[15] For example, Stefania Albanesi and Christopher Sleet, “Optimal Taxation with Private Information,” The Review of Economic Studies, Vol. 73, No. 1 (January 2006), pp. 1–30.

[16] Alan B. Krueger and Bruce D. Meyer, “Labor Supply Effects of Social Insurance,” Chap. 33 in Alan J. Auerbach and Martin Feldstein, eds., Handbook of Public Economics, Vol. 4 (2002).

[17] Jonathan Gruber and Peter Orszag, “Does the Social Security Earnings Test Affect Labor Supply and Benefit Receipt?” National Bureau of Economic Research Working Paper No. 7923, September 2000.

[18] P. A. Diamond and J. A. Mirrlees, “A Model of Social Insurance with Variable Retirement,” Journal of Public Economics, Vol. 10, No. 3 (December 1978).

[19] Michel Beine, Frédéric Docquier, and Hillel Rapoport, “Brain Drain and Economic Growth: Theory and Evidence,” Journal of Development Economics, Vol. 64, No. 1 (February 2001).

[20] In a static model, bailouts are usually a good idea. Ben Bernanke and Mark Gertler, “Agency Costs, Net Worth, and Business Fluctuations,” The American Economic Review, Vol. 79, No. 1 (March 1989).

[21] Claudio Borio and Haibin Zhu, “Capital Regulation, Risk-Taking and Monetary Policy: A Missing Link in the Transmission Mechanism?” Journal of Financial Stability, Vol. 8, No. 4 (December 2012).

[22] The journals are the top two journals in economics and the top journal that focuses primarily on macroeconomics, according to the IDEAS/RePEc ranking. I used the August 2014 issue of the Quarterly Journal of Economics, the November 2014 issue of the American Economic Review, and the (incomplete) 2015 issue of the Journal of Monetary Economics. IDEAS, “IDEAS/RePEc Aggregate Rankings for Journals,” https://ideas.repec.org/top/top.journals.all.html (accessed December 12, 2014).

[23] Treb Allen and Costas Arkolakis, “Trade and the Topography of the Spatial Economy,” The Quarterly Journal of Economics, Vol. 129, No. 3 (August 2014); Nicola Gennaioli, Andrei Shleifer, and Robert Vishny, “Finance and the Preservation of Wealth,” The Quarterly Journal of Economics, Vol. 129, No. 3 (August 2014); Thomas Piketty and Gabriel Zucman, “Capital Is Back: Wealth–Income Ratios in Rich Countries 1700–2010,” The Quarterly Journal of Economics, Vol. 129, No. 3 (August 2014); Justin Caron, Thibault Fally, and James R. Markusen, “International Trade Puzzles: A Solution Linking Production and Preferences,” The Quarterly Journal of Economics, Vol. 129, No. 3 (August 2014); Şahin et al., “Mismatch Unemployment”; Thomas Chaney, “The Network Structure of International Trade,” American Economic Review, Vol. 104, No. 11 (November 2014); Alwyn Young, “Structural Transformation, the Mismeasurement of Productivity Growth, and the Cost Disease of Services,” American Economic Review, Vol. 104, No. 11 (November 2014); Daniel Aaronson, Fabian Lange, and Bhashkar Mazumder, “Fertility Transitions Along the Extensive and Intensive Margins,” American Economic Review, Vol. 104, No. 11 (November 2014); Benjamin F. Jones, “The Human Capital Stock: A Generalized Approach,” American Economic Review, Vol. 104, No. 11 (November 2014); Shouyong Shi, “Liquidity, Assets and Business Cycles,” Journal of Monetary Economics, Vol. 70 (March 2015); Kirstin Hubrich and Robert J. Tetlow, “Financial Stress and Economic Dynamics: The Transmission of Crises,” Journal of Monetary Economics, Vol. 70 (March 2015); Christopher L. House and Yusufcan Masatlioglu, “Managing Markets for Toxic Assets,” Journal of Monetary Economics, Vol. 70 (March 2015); Vasco Cúrdia et al., “Has U.S. Monetary Policy Tracked the Efficient Interest Rate?” Journal of Monetary Economics, Vol. 70 (March 2015); Lieven Baele et al., “Macroeconomic Regimes,” Journal of Monetary Economics, Vol. 70 (March 2015); and Conny Olovsson, “Optimal Taxation with Home Production,” Journal of Monetary Economics, Vol. 70 (March 2015).

[24] Raj Chetty et al., “Active vs. Passive Decisions and Crowd-Out in Retirement Savings Accounts: Evidence from Denmark,” The Quarterly Journal of Economics, Vol. 129, No. 3 (August 2014), and Marcel Fratzscher et al., “The Scapegoat Theory of Exchange Rates: The First Tests,” Journal of Monetary Economics, Vol. 70 (March 2015).

[25] Luca Guerrieri and Matteo Iacoviello, “OccBin: A Toolkit for Solving Dynamic Models with Occasionally Binding Constraints Easily,” Journal of Monetary Economics, Vol. 70 (March 2015).

[26] Dan R. Mastromarco, David R. Burton, and William W. Beach, eds., The Secret Chamber or the Public Square? What Can Be Done to Make Tax Analysis and Revenue Estimation More Transparent and Accurate (Washington, DC: The Heritage Foundation, 2005).

[27] Tax Foundation, “Articles: Taxes and Growth,” http://taxfoundation.org/articles-taxtopic/1085 (accessed December 16, 2014).

[28] Laurence Kotlikoff, “Research Articles by Laurence J. Kotlikoff,” http://www.kotlikoff.net/research?tid=48 (accessed December 16, 2014).

[29] The Joint Committee on Taxation, “Publications on Macroeconomic Analysis,” https://www.jct.gov/publications.html?func=select&id=4 (accessed December 16, 2014).