August 29, 2016 | Backgrounder on National Security and Defense
Despite significant post-9/11 reforms, the U.S. intelligence community (IC) is still not as effective as it could—and should—be. As soon as the 2004 Intelligence Reform and Terrorism Prevention Act passed, security experts warned that the act had not fully dealt with the challenges facing the IC. The critics pointed to lack of strategic analysis, politicization of intelligence, and the difficulties that the IC has learning from failures. The need for reform is made more urgent by the increasingly complex national security environment that the United States is facing, dominated by violent non-state Islamic extremists; anti-status-quo states China, Russia, and Iran; and weakened or alienated allies and partners around the globe on which the U.S. must depend for support in dealing with these threats and challenges. There are also well-grounded fears that this situation will be the “new normal” for at least the next decade. The IC must, therefore, become the kind of federated enterprise—organizationally, analytically, and culturally—that can constantly learn from, and adapt to, this highly volatile environment.
Despite the deep reforms of the U.S. intelligence community (IC) carried out after 9/11, including the creation of the Director for National Intelligence (DNI) and the National Combatting Terrorism Center (NCTC), there is widespread agreement that more remains to be done. This is not a new thought. Before the ink was dry on the 2004 Intelligence Reform and Terrorism Prevention Act (IRTPA), there were warnings from insiders as well as outside experts that the law had not fully dealt with the challenges facing the IC. The critics pointed to the anomalous position of the DNI, a neglect of strategic analysis, accusations of the politicization of intelligence, and the difficulties that the IC has with failure, learning, and adaptation, as signs that all was not well within the IC.
A series of stumbles over the past seven years have given credence to the red flags raised by these experts. In quick succession, the IC failed to predict the so-called Arab Spring, the resurgence of al-Qaeda, the adventurism of Putin, the aggressiveness of China, and a number of terrorist attacks on the U.S., from the Detroit “underwear bomber” to the San Bernardino massacre. The intelligence failures surrounding the Arab Spring were especially important, since the IC had not understood the implications of an entire series of seismic shifts in the strategic landscape, suggesting that there are serious problems with the analytical side of the community. It is noteworthy that the community’s intelligence collection—clandestine and open source—appears not to have focused on the deeper questions of regime stability and the underlying causes for the Arab Spring. In this Backgrounder, the focus will remain on analysis and improvements to the analytic aspects of the IC.
The need for reform is made more urgent by the increasingly complex national security environment that the United States is facing, one that is dominated by violent non-state Islamic extremists; anti-status-quo states China, Russia, and Iran; and a weakened or alienated set of allies and partners around the globe on which the U.S. must depend for support in dealing with these threats and challenges. There are also well-grounded fears that this situation will be the “new normal” for at least the next decade. The IC must, therefore, become the kind of federated enterprise—organizationally, analytically, and culturally—that can constantly learn from, and adapt to, this highly volatile environment in order to better support decision makers.
Before presenting potential solutions to the serious challenges that exist within the intelligence community, it is important to carefully describe the specific structural, analytical, and cultural issues that must be reformed if the community is to be made ready to deal with the new security environment.
Structural Challenges. There are two main structural challenges confronting the IC, each of which seems to contradict the other.
A Powerless DNI and Bureaucratic Bloat: Too Much Centralization or Too Little? The current organizational structure of the IC is dominated by 17 diverse agencies, with the DNI as the nominal head. There are also a few mission-organized units, such as the NCTC, that bring together intelligence professionals from all the agencies with their own organically grown workforce to address specific areas of concern. Criticisms of this structure have come from two completely different directions, with some experts arguing that the DNI has too little power, while others argue that there is already too much centralization and bureaucracy in the IC. These two criticisms are not mutually exclusive.
A number of observers and experts have noted that the Director of National Intelligence lacks any real control over the IC. Unlike the Secretary of State or Secretary of Defense, the DNI does not oversee his entire domain, nor do the various agencies, other than the CIA, report directly to him. The DNI also cannot dictate to the heads of the CIA or Defense Intelligence Agency (DIA) in the way that the Secretary of Defense, for instance, can issue orders to combatant commanders. This is an authorities issue, bureaucratic problem, and personality challenge combined. The IRTPA sets out the position of DNI as one that would coordinate and encourage cooperation, rather than a “command” position like the Secretary of Defense. And while the Director of Central Intelligence should report directly to the DNI, the powerful and independent-minded leadership and bureaucracy of the CIA reportedly resented the intrusion of another layer of administration into their affairs and have fought against DNI attempts to assert his legal authority. In addition, the personalities of the DNI and President matter: A determined leader might be able to use his authorities to enforce more unification of effort but only if supported by the President.
The problems created by the anomalous position of the DNI are legion: There is no central hub that can enforce change throughout the IC, make the entire community more adaptable, or root out and fire bad managers and leadership. The DNI must be prepared to exercise the firing authority often in tandem with the head of the Department where the poor performance is observed. If the President wants to enquire about issues handled by multiple agencies, the President can turn to the DNI, but obtaining an answer requires strong White House backing for the DNI as opposed to the DNI resting on relatively weak authorities in law or policy to push IC elements for answers.
For some observers, this is a feature, not a bug, since they worry about too much centralization within the IC rather than too little. The intelligence community’s bureaucracy, in their view, presents a serious challenge to sound and independent analysis and the creation of the DNI position has only compounded this problem. These experts point out that there are too many layers of management within the CIA in particular, creating unnecessary bureaucratic hurdles for intelligence professionals and separating the writers of intelligence products from their customers by as many as eight distinct managerial levels. In their opinion, adaptation and learning must come from multiple directions, and the relatively loose control of the DNI allows each agency to find the best way forward for dealing with the myriad challenges confronting the U.S.
Organized for Individual Missions, Not Cooperation. Despite the relative weakness of the DNI, there is no doubt that the creation of this office has facilitated better, though not perfect, collaboration and cooperation between the other 16 agencies that make up the IC. At the same time, the new bureaucratic layer did not deal with the disparate organizational structures within each of the intelligence community’s constituent parts. Every agency creates subdivisions according to what its leadership feels best fits its own mission, rather than working together to create an overall structure that would facilitate cooperation throughout the community. Even the recent reorganization by John Brennan of the CIA, while integrating the analytical and operational sides of that agency, does not seem to take into consideration the need to cooperate with other institutions. The result is that there are only a few issues, such as counterterrorism, where cross-agency cooperation is fully supported by organizational design and structure.
Analytical Challenges. Perhaps the most serious problems confronting the IC are those related to analysis—the core function for most of the community. The challenges come from many different directions and include pressures to maintain consensus, a failure to encourage the development of real subject-matter expertise, a neglect of open-source materials, and a strategic analysis gap.
Lack of Diversity in Analysis. The widely studied problems of groupthink, confirmation biases, and cultural norms have all had their role in limiting how intelligence analysts have perceived and described problem sets. The group-style of writing all products makes these problems a critical threat to dissenting opinions and alternative viewpoints from the lowest to the highest levels of the IC. There are also a few additional issues, the result of the specific circumstances under which intelligence professionals must work, that might impose greater uniformity on analysis than the data supports. The ongoing investigation into allegations of intelligence-analysis manipulation at Central Command’s intelligence shop, for instance, has exposed the problem of a preferred “narrative” that analysts were pressured to maintain. It has even been alleged that managers altered finished products, and that analysts’ jobs were threatened if the analysts did not toe the party line. The problem of potential politicization, then, whether directly by an Administration or by managers eager to please their political leadership, remains a serious one. Another IC-specific challenge is created by the need for consensus when writing the intelligence community’s premier products, such as a National Intelligence Estimate (NIE). This might lead to a less than full exposure and discussion of objections to received wisdom or the downplaying of alternative viewpoints.
Failure to Encourage Expertise. Personnel and promotion incentives within the IC encourage generalization rather than specialization and can hinder the development of real area, or subject-matter, expertise. There are, in addition, no incentives to discourage analysts from changing subjects frequently, preventing them from ever having enough time and study to become experts on any one topic. Richard Russell describes the almost palpable disdain held by managers at the CIA for PhDs, and the lack of encouragement to hire new analysts with doctorates. While there is certainly a need for generalists in the IC, the current career expectations discourage the formation of internal experts and force agencies to rely on external subject-matter experts. This is a serious issue for the new challenges that the U.S. faces in little-studied regions of the Muslim-majority world, since there are relatively few such subject-matter experts for the IC to rely upon and many of them, for ethical or political reasons, will not work with the IC.
An Open Source-Classified Divide. The IC naturally prizes classified intelligence and prioritizes it over other sorts of information. After all, if intelligence professionals simply read newspapers or other published sources to gather information, how would they differ from outside experts or reporters? But this natural tendency is unhelpful when it leads to neglect, or even downplaying, of the value of open-source materials. This is especially troubling when dealing with non-state actors like al-Qaeda and ISIS, which put out a great deal of public information about their plans, objectives, and even failings. The terrorists’ decisions to use Twitter, Telegram, and other social media, as well as to publish biographies of members, demonstrate that open source can also be a rich source of detailed information about jihadist groups, one that the IC, by discounting unclassified material, may be failing to fully exploit.
Strategic Thinking Lapses. The IC, driven by the demands of its customers as well as the community’s own hierarchy of importance, engages mostly in short-term, tactical analysis and thinking, rather than strategic and long-term consideration. Although it seems reasonable that current intelligence must be the focus of the majority of the IC’s analytical work, neglecting longer-term strategic analysis can create key vulnerabilities by de-emphasizing strategic warning, forecasting, and prediction. One exception to this general problem is Global Trends, produced by the National Intelligence Council every four years. However, this public document, which analyzes likely developments in national security issues over the succeeding two decades, is apparently a stand-alone product that might have little influence on analysis within the IC. No publicly released NIE, for instance, has ever cited Global Trends in its analysis or conclusions. In addition, a look at Global Trends documents since the publication’s inception in 1997 shows a clear focus on current challenges and a bias toward straight-line assumptions about future developments.
Cultural Challenges. Cultural problems can seem even more intractable than analytical challenges, since they are “baked into” the very essence and structure of institutions. In the IC, two cultural issues in particular have prevented the community from being an adaptable and learning organization.
Fear of Failure. With the caveat that there have indeed been actual intelligence failures, it is also true that the intelligence community is often the whipping boy for national security policy mistakes. It is far too easy for policymakers and political leadership to blame errors of judgment and policy on the IC rather than take responsibility for their own failings. The result is that the entire IC, and the CIA in particular, has developed a well-founded fear of failure and, consequently, of being blamed by the public and policymakers. This can lead to a number of serious problems, including risk aversion and an analytical tendency to overcompensate for previous errors by going too far in the other direction with subsequent assessments. In addition, the CIA’s response in 2013 and 2014 to the Senate detainee report shows that CIA leadership will go to great lengths to avoid being “stuck” with the blame for actions that are later called into question or outright discredited, a tendency perhaps sparked by the Church Committee process of the 1970s.
Problems with Learning. Mistakes should be opportunities for learning, but—despite some positive examples of learning and adaptation, best exemplified by George Tenet’s reform efforts during the 1990s and the lengthy process undertaken after 9/11—the IC as a whole has great difficulty looking at failures objectively and making adjustments to prevent mistakes in the future. It is also unclear whether there is an institutionalized learning process for the IC, one that would capture lessons learned from both successes and failures and pass them on to future generations of operators and analysts.
This set of challenges requires a series of deep reforms, not just one or two small tweaks to the community. Overall, the basic concept for reform is to help create a more flexible and adaptable institution that can evolve with the changing national security situation rather than requiring serious reforms every decade or so. This Backgrounder offers proposals to do this by creating stronger leadership for the IC while streamlining the bureaucracy, and by encouraging a good balance of cooperation, diversity of views, and learning by intelligence professionals throughout the entire community.
Meeting the Structural Challenges.
Overhauling the Bureaucracy. While the ODNI has improved cooperation and collaboration across the community, the DNI is relatively weak and must be given the power to actually control the IC, including the CIA. At the same time, the bureaucratic layers between analysts and their customers need to be reduced to allow more analytical freedom and diversity, and to prevent the massaging of information to fit a preferred narrative or achieve consensus by going to the lowest common denominator in making hard judgment calls.
Fixing Management Problems. The creation of this new structure for the IC makes flattening the current unwieldy bureaucracy all the more urgent. Yet there seems to be a serious dilemma for any attempt to rid the IC of its layers of management: No outside group of experts would be able to quickly grasp the complex system of administration of the IC and understand how best to reorganize the entire community for its many missions—while any group of insiders might be both too close to the problem for objectivity, and too timid in proposing changes.
Unifying the Community. While reorganization of the IC leadership can ensure good cooperation and coordination at the very highest levels of the community, it will not break down the walls that exist among the all-source analysis agencies (such as the CIA, DIA, or the Bureau of Intelligence and Research). Exposure to the culture and viewpoints of other agencies will also help to foster a broader range of views and undermine groupthink around the IC.
Encouraging Diversity of Views. Given the dangers of groupthink suggested by the investigation at Central Command, as well as the failures to foresee the Arab Spring, the resurgence of al-Qaeda, and other national security challenges, much more needs to be done to encourage diversity of views and analytical conclusions. One traditional answer would be to establish red teams, and these should indeed be an integral part of the intelligence community’s analytical and product writing processes. Yet red teams alone will not provide well-sourced, reliable, and widely disseminated new ways of understanding ongoing challenges. Established views will always be seen as the norm, while the concepts proposed by red teams will be understood as exceptional, marginal, and therefore perhaps ignored in writing NIEs or other major products.
Ending the Open Source–Classified Divide. Classified information must remain the centerpiece for intelligence analysis, not only because it is the sole provenance of the IC, but also because it is often unequaled in value and, if correctly interpreted, can give a view of the enemy that is closer to reality than open-source information alone. At the same time, given the difficult problem set and the new complex security environment that the U.S. is facing, the IC can no longer neglect open-source materials as it has lately been wont to do. Instead, the IC needs to return to the earlier way when publicly available sources were exploited in the decades-long fight with the Soviet Union—as an additional resource that can be combined with classified information to present a more complete picture of these problem sets.
Instilling Strategic Thinking. Finally, there is the issue of too little strategic and long-term analysis by the IC. While it is true that this can be customer-driven, and that current intelligence must remain the primary focus of analysts, a neglect of longer-term thinking and writing will force the IC to remain reactive rather than proactive when dealing with the many challenges ahead. Because tactical analysis and strategic thinking are two completely different sorts of mental activities, which generally require two different sorts of analysts, it is necessary to create a separate structure within each agency—as well as one within the NIC—to adequately cover this gap within the IC.
Allowing Failure While Encouraging Learning. Despite the very best efforts to encourage diversity of opinions, to create the capacity for strategic analysis, and to allow analysts to speak as clearly as possible to policy leaders, there will always be intelligence failures and mistakes. The important point is to make learning from experience, whether good or bad, an integral part of the training process throughout the IC. While not guaranteed to fix poor analysis, it will help to prevent repeated misdiagnoses, wrong analytical frameworks, and poor predictions, and also to foster adaptability. In addition, much more remains to be done to encourage agencies to admit errors and fix them, while providing some cover when Congress or policymakers attempt to blame the IC for errors that are the result of mistaken policies.
The recommendations presented here could make the intelligence community into a very different enterprise—more unified and with a clearer chain of command, leaner bureaucracy, better analysis, and a concept for strategic analysis that preserves the necessity for current intelligence. But the most important change would be to help the new IC become a more adaptable, learning organization. These proposals are not the final reforms that are necessary for the intelligence community. History shows that there will be unforeseeable events and processes that will require flexibility and changes to structure, analysis, and the culture of the IC. If these proposed reforms accomplished only one thing—to make the IC an institution that welcomes change rather than rejects it for fear of punishment, and that can evolve with the complex national security environment—they would have succeeded.—Mary Habeck is a Visiting Scholar at the American Enterprise Institute. Charles D. Stimson is Manager of the National Security Law Program and Senior Legal Fellow in the Center for National Defense, of the Kathryn and Shelby Cullom Davis Institute for National Security and Foreign Policy, at The Heritage Foundation.
 Michael Morell, The Great War of Our Time: The CIA’s Fight Against Terrorism—From al Qa’ida to ISIS (New York: Twelve, 2015), p. 180.
 For both failures: Mark Hosenball, Phil Stewart, and Matt Spetalnick, “Exclusive: Congress Probing U.S. Spy Agencies’ Possible Lapses on Russia,” Reuters, October 8, 2015, http://www.reuters.com/article/us-mideast-crisis-intelligence-exclusive-idUSKCN0S20CZ20151008 (accessed May 9, 2016).
 See, among many sources, Peter Baker and Carl Hulse, “U.S. Had Early Signals of a Terror Plot, Obama Says,” The New York Times, December 29, 2009, http://www.nytimes.com/2009/12/30/us/politics/30obama.html?_r=1&hp (accessed May 9, 2016); Karen DeYoung and Anne E. Kornblut, “Times Square Suspect’s Movements Raise Questions About Holes in Antiterror System,” The Washington Post, May 5, 2010, http://www.washingtonpost.com/wp-dyn/content/article/2010/05/04/AR2010050405357.html (accessed May 9, 2016); and Mark Viser, “House Panel Details Failures in Run-Up to Marathon Attack,” The Boston Globe, March 26, 2014, http://www.bostonglobe.com/news/nation/2014/03/26/congressional-report-details-intelligence-failures-prior-marathon-bombings/P1EJ4eGWdvt1K809brQ5VO/story.html (accessed May 9, 2016).
 U.S. Department of Justice, “The Intelligence Reform and Terrorism Prevention Act of 2004 (IRTPA),” Public Law 108–458, December 17, 2004, http://it.ojp.gov/PrivacyLiberty/authorities/statutes/1282 (accessed May 10, 2016).
 See, for instance, the fight between DNI Admiral Dennis Blair (ret.) and DCI Leon Panetta, which seems to have ended in the victory of the CIA over the DNI. Marc Ambinder, “The Real Intelligence Wars: Oversight and Access,” The Atlantic, November 18, 2009, http://www.theatlantic.com/politics/archive/2009/11/the-real-intelligence-wars-oversight-and-access/30334/ (accessed May 10, 2016), and David Ignatius, “Dennis Blair Erred—But He Had an Impossible Job,” The Washington Post, May 20, 2010, http://voices.washingtonpost.com/postpartisan/2010/05/dennis_blair_erred_--_but_he_h.html (accessed May 10, 2016).
 Most cogently expressed by Richard Posner, Uncertain Shield: The U.S. Intelligence System in the Throes of Reform (Lanham, MD: Rowman & Littlefield, 2006), especially pp. 17, 64, and 67.
 Richard L. Russell, Sharpening Strategic Intelligence: Why the CIA Gets It Wrong and What Needs to Be Done to Get It Right (Cambridge: Cambridge University Press, 2007), pp. 151–152.
 This is apparent from a quick perusal of the public organization charts of intelligence agencies: Central Intelligence Agency, “About CIA: CIA Organization Chart,” https://www.cia.gov/about-cia/leadership/cia-organization-chart.html (accessed May 10, 2016), and National Reconnaissance Office, “NRO Organization,” http://www.nro.gov/about/nro/orgchart.html (accessed May 10, 2016). For the NSA, see Marc Ambinder, “An Educated Guess About How the NSA Is Structured,” The Atlantic, August 14, 2013, http://www.theatlantic.com/technology/archive/2013/08/an-educated-guess-about-how-the-nsa-is-structured/278697/ (accessed May 10, 2016), and Ellen Nakashima, “National Security Agency Plans Major Reorganization,” The Washington Post, February 2, 2016, http://www.washingtonpost.com/world/national-security/national-security-agency-plans-major-reorganization/2016/02/02/2a66555e-c960-11e5-a7b2-5a2f824b02c9_story.html (accessed May 10, 2016).
 “CIA Director John Brennan Announces Major Reorganization to Address Intelligence ‘Gaps,’” Associated Press, March 6, 2015, http://www.foxnews.com/us/2015/03/06/cia-director-john-brennan-announces-major-reorganization-to-address.html (accessed May 10, 2016).
 Gregory F. Treverton, Intelligence for an Age of Terror (Cambridge: Cambridge University Press, 2009), pp. 161–162, and Rob Johnston, “Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study,” Center for the Study of Intelligence, 2005, pp. 21–25.
 Shane Harris and Nancy A. Youssef, “Exclusive: 50 Spies Say ISIS Intelligence Was Cooked,” The Daily Beast, September 9, 2015, http://www.thedailybeast.com/articles/2015/09/09/exclusive-50-spies-say-isis-intelligence-was-cooked.html (accessed May 10, 2016).
 Gregory F. Treverton and C. Bryan Gabbard, “Assessing the Tradecraft of Intelligence Analysis,” RAND, 2008, p. 7, http://www.rand.org/content/dam/rand/pubs/technical_reports/2008/RAND_TR293.pdf (accessed May 10, 2016).
 A constant complaint about the IC. See, for instance, Frederick P. Hitz, “Not Just a Lack of Intelligence, a Lack of Skills,” The Washington Post, October 21, 2001, p. B3.
 Russell, Sharpening Strategic Intelligence, p. 124.
 Stephen Marrin, Improving Intelligence Analysis: Bridging the Gap Between Scholarship and Practice (London: Routledge, 2011), pp. 67, et seq.
 Treverton and Gabbard assume that this is inevitable and will continue into the future, since “[g]iven the demands of current business, there simply is no time to ‘train up’ an analyst on a new, current, hot-button issue with any serious depth.” Treverton and Gabbard, “Assessing the Tradecraft of Intelligence Analysis,” p. 31.
 Jennifer E. Sims, “Understanding Ourselves,” in Jennifer E. Sims and Burton Gerber, eds., Transforming U.S. Intelligence (Washington, DC: Georgetown University Press, 2005), pp. 37–39.
 Noted by Russell, Sharpening Strategic Intelligence, pp. 136, et seq. The problem of too great a focus on short-term, tactical issues is mentioned by intelligence professionals themselves in Rob Johnston, “Analytic Culture in the U.S. Intelligence Community,” p. 15, and Treverton and Gabbard, “Assessing the Tradecraft of Intelligence Analysis,” p. 11.
 Office of the Director of National Intelligence, “National Intelligence Council,” http://www.dni.gov/index.php/about/organization/national-intelligence-council-global-trends (accessed May 10, 2016).
 This latter issue is one of the main points of John Diamond’s book The CIA and the Culture of Failure: U.S. Intelligence from the End of the Cold War to the Invasion of Iraq (Stanford: Stanford University Press, 2008). For an internal acknowledgement of this important problem, see Central Intelligence Agency, “Remarks by Deputy Director for Operations James L. Pavitt at the Foreign Policy Association,” June 21, 2004, http://fas.org/irp/cia/product/ddo_speech_062404.html (accessed May 10, 2016).
 Mark Mazzetti and Carl Hulse, “Inquiry by C.I.A. Affirms It Spied on Senate Panel,” The New York Times, July 31, 2014, http://www.nytimes.com/2014/08/01/world/senate-intelligence-commitee-cia-interrogation-report.html?_r=0 (accessed May 10, 2016), and Jason Leopold, “The Google Search that Made the CIA Spy on the US Senate,” Vice News, August 12, 2015, http://news.vice.com/article/the-google-search-that-made-the-cia-spy-on-the-us-senate (accessed May 10, 2016).
 For a good discussion on the need for more learning within the IC, see Treverton, Intelligence for An Age of Terror, pp. 136, et seq.