The Need to Protect Civil Liberties While Combating Terrorism: Legal Principles and the Total Information Awareness Program

Report Homeland Security

The Need to Protect Civil Liberties While Combating Terrorism: Legal Principles and the Total Information Awareness Program

February 6, 2003 17 min read

Authors: Paul Rosenzweig and Michael Scardaville

Since the September 11, 2001, terrorist attacks on the World Trade Center and the Pentagon, Congress, the media, and the general public have urged the intelligence, counterintelligence, and law enforcement agencies to improve their ability to discover and preempt terrorist plots before additional attacks occur. The criticism is colloquially rendered as the failure of government agencies to "connect the dots."

Yet, when the government begins the development of intelligence programs that would assist in "connecting the dots," Americans naturally also worry that government will overreach and seek to accumulate unwarranted power. This reaction may be in tension with the desire for increased security, but U.S. history shows that it is not unique to the post-September 11 era. The United States has experienced abuses of power in times of war and almost unilateral disarmament in times of seeming calm.

The war on terrorism changes the stakes in fundamental ways. No longer is the United States fighting against adversaries an ocean away--the war has come home to this continent. Yet the war against terrorism is likely to be a long one, and Americans cannot tolerate the long-term substantial degradation of their civil liberties as the price of public safety.

Many see this conundrum as irresolvable: Security must be balanced against civil liberty, and any improvement in one results in a diminution of the other. This is the wrong perspective: America is not limited to a zero-sum game. There are effective ways to limit the ability of the government to intrude into Americans' lives while increasing security. America must adhere to fundamental and firm principles of limited government, and it can do so while also answering the terrorist threat. The challenge is not an easy one, but few worthwhile things are.

The public controversy over the experimental (and unwisely named) Total Information Awareness (TIA) research program sponsored by the U.S. Department of Defense is an instructive case study of how the challenge might be met. In response to the threat of terrorism, the Defense Advanced Research Projects Agency (DARPA) in January 2002 opened the Information Awareness Office (IAO), which manages the TIA program. The program is an effort to develop the technological means to "put together the pieces of the puzzle" by (in part) allowing subject- and pattern-based queries of computer databases. Once developed, and if proven effective, technology being researched under TIA eventually could help federal agencies link government information systems together to provide a national intelligence fusion capability and a less costly way to access information already available to law enforcement and intelligence agencies. In other words, if DARPA's research (which is in its initial stages) is successful, a properly implemented TIA will provide intelligence, counterintelligence, and law enforcement agencies with a variety of powerful tools for unearthing suspected terrorists.

However, the concept of TIA has been criticized, most prominently by New York Times columnist William Safire.1 Since Safire's critique first ran, privacy advocates have voiced determined opposition to the program. It has been labeled "a big, scary Orwellian thing,"2 while Safire describes it as a "computerized dossier"3 on every American's private life. Other critics cite the potential for government misuse as a reason to forgo any effort to develop TIA.4 Questions about the program have begun to emerge on Capitol Hill.5 Some lawmakers have already offered amendments to kill the program6 and have called for its review by the Inspector General of the Department of Defense.7

The criticisms of the nascent TIA programs sound two distinct themes:

  1. That TIA will, by making access to data easier and more efficient, inappropriately magnify and enhance the government's power, and
  2. That TIA, when implemented, will (beyond increasing government power) allow the government access to data to which it does not currently have access and/or lower existing legal barriers to such access.

These concerns should be taken seriously. Indeed, we and many of our respected colleagues within The Heritage Foundation share these concerns. They stem from an understanding of America's founding history and recent unfortunate examples of government excess.8 In our considered judgment, however, these legitimate concerns are outweighed by the potential benefits of the TIA program under development, which may be implemented within existing legal and policy constraints that can prevent abuse of the program during criminal or national security investigations. Indeed, if TIA were the program its most vocal critics describe, we would join them, without reservation, in opposing it.

To an extent that is rare even in Beltway debates, however, the description of TIA offered by most of its critics is not accurate.9 DARPA certainly invited some of the criticism by adopting a name, symbol, and motto that have an Orwellian ring.10 It is a natural outgrowth of a healthy military culture that leaders label their operations with titles that convey overwhelming power.11 Regrettably, when military research projects are given those names--especially when they have potential civilian applications--it strikes many as naive or politically inept. The name does not, however, say very much about what the research project really attempts to achieve.

A more complete and accurate picture of TIA is necessary to foster the debate. Our examination has led us to the conclusion that a wholesale rejection of TIA's possibilities before its capacities are realized would be a serious mistake. Rather, the legitimate concerns call for us to devise thoughtful limits and protections against abuse and to understand the distinction between the foreign and domestic uses to which TIA might be put.

Development of TIA can and should continue, based upon the following foundations:

  • Strong congressional oversight;
  • High-level authorization for use of the technology and limited access to its product;
  • Implementation in a manner that does not alter or contravene existing legal restrictions on the government's ability to access data about private individuals;
  • Absolute protection for fundamental constitutional liberties;
  • Civil and criminal penalties for abuse; and
  • A sunset provision to terminate the program after a trial period.

Our analysis begins (as we believe it ought) with a summary of first principles. We then summarize our understanding of the nature and scope of the problem posed by terrorist threats and offer a more comprehensive summary of what the TIA programs are actually doing.12 We conclude with several policy recommendations that, in our view, address critics' concerns about privacy and government power while advancing continued research into a potentially powerful weapon against terrorism.


Some might say that discussion of any development of TIA is premature--that TIA has yet to grow beyond the concept stage and that discussion of the limits to be placed on the use of TIA should await its development. Although TIA is little more than a research project at this juncture, however, it is still prudent to consider appropriate safeguards on its use while in development and implementation.

Indeed, fundamental legal principles and conceptions of American government should guide the configuration of TIA rather than the reverse. The precise contours of any rules relating to the use of TIA will depend, ultimately, on exactly what TIA is capable of accomplishing--the more powerful the systems, the greater the safeguards necessary. As a consequence, the concerns of critics should be fully voiced and considered while the TIA research program is underway.

In general, TIA can and should be constructed in a manner that fosters both civil liberty and public safety. Certain overarching principles must animate the architecture of TIA and provide guidelines that will govern the implementation of TIA in the domestic environment. These are the same principles that should animate the consideration of any new program to combat global terrorism at home.

Most of the debate over new intelligence systems focuses on perceived intrusions on civil liberties, but Americans should keep in mind that the Constitution weighs heavily on both sides of the debate over national security and civil liberties. The President and other policymakers must respect and defend the individual civil liberties guaranteed in the Constitution when they act, but there is also no doubt that they cannot fail to act when we face a serious threat from a foreign enemy.

The Preamble to the Constitution acknowledges that the United States government was established in part to provide for the common defense. The war powers were granted to Congress and the President with the solemn expectation that they would be used. Congress was also granted the power to "punish...Offenses against the Law of Nations,"13 which include the international law of war, or terrorism. Besides serving as chief executive and commander in chief, the President also has the duty to "take Care that the Laws be faithfully executed,"14 including vigorously enforcing the national security and immigration laws.

Of course, just because the Congress and the President have a constitutional obligation to act forcefully to safeguard Americans against attacks by foreign powers does not mean that every means by which they might attempt to act is necessarily prudent or within their power.15 Core American principles require that TIA (and, indeed, any new counterterrorism technology deployed domestically) should be developed only within the following bounds:16

  • No fundamental liberty guaranteed by the Constitution can be breached or infringed upon.
  • Any increased intrusion on American privacy interests must be justified through an understanding of the particular nature, significance, and severity of the threat being addressed by the program. The less significant the threat, the less justified the intrusion.
  • Any new intrusion must be justified by a demonstration of its effectiveness in diminishing the threat. If the new system works poorly by, for example, creating a large number of false positives, it is suspect. Conversely, if there is a close "fit" between the technology and the threat (that is, for example, if it is accurate and useful in predicting or thwarting terror), the technology should be more willingly embraced.
  • The full extent and nature of the intrusion worked by the system must be understood and appropriately limited. Not all intrusions are justified simply because they are effective. Strip searches at airports would prevent people from boarding planes with weapons, but at too high a cost.
  • Whatever the justification for the intrusion, if there are less intrusive means of achieving the same end at a reasonably comparable cost, the less intrusive means ought to be preferred. There is no reason to erode Americans' privacy when equivalent results can be achieved without doing so.
  • Any new system developed and implemented must be designed to be tolerable in the long term. The war against terrorism, uniquely, is one with no immediately foreseeable end. Thus, excessive intrusions may not be justified as emergency measures that will lapse upon the termination of hostilities. Policymakers must be restrained in their actions; Americans might have to live with their consequences for a long time.

From these general principles can be derived certain other more concrete conclusions regarding the development and construction of any new technology:

  • No new system should alter or contravene existing legal restrictions on the government's ability to access data about private individuals. Any new system should mirror and implement existing legal limitations on domestic or foreign activity, depending upon its sphere of operation.
  • Similarly, no new system should alter or contravene existing operational system limitations. Development of new technology is not a basis for authorizing new government powers or new government capabilities. Any such expansion should be independently justified.
  • No new system that materially affects citizens' privacy should be developed without specific authorization by the American people's representatives in Congress and without provisions for their oversight of the operation of the system.
  • Any new system should be, to the maximum extent practical, tamper-proof. To the extent that prevention of abuse is impossible, any new system should have built-in safeguards to ensure that abuse is both evident and traceable.
  • Similarly, any new system should, to the maximum extent practical, be developed in a manner that incorporates improvements in the protection of American civil liberties.
  • Finally, no new system should be implemented without the full panoply of protections against its abuse. As James Madison told the Virginia ratifying convention, "There are more instances of the abridgment of the freedom of the people by gradual and silent encroachments of those in power than by violent and sudden usurpations."17


With those principles in mind, the discussion of TIA will also be well-served by a thorough understanding of the threat it is intended to address and the precise means by which it would address that threat.

The Terrorist Threat
The full extent of the terrorist threat to America cannot be fully known. Consider, as an example, one domestic aspect of that threat--an effort to determine precisely how many al-Qaeda operatives are in the United States at this time and to identify those who may enter in the future.

Although estimates of the number of al-Qaeda terrorists in the United States have varied since the initial attack on September 11, the figure provided by the government in recent, supposedly confidential briefings to policymakers is 5,000.18 This 5,000-person estimate may include many who are engaged in fundraising for terrorist organizations and others who were trained in some fashion to engage in jihad, whether or not they are actively engaged in a terrorist cell at this time. But these and other publicly available statistics support two conclusions: (1) no one can say with much certainty how many terrorists are living in the United States, and (2) many who want to enter in the foreseeable future will be able to do so.

Understanding the scope of the problem demonstrates the difficulty of assessing the true extent of the risk to the United States. Consider this revealing statistic: "[M]ore than 500 million people [are] admitted into the United States [annually], of which 330 million are non-citizens."19
Of these:

  • Tens of millions arrive by plane and pass through immigration control stations, often with little or no examination.20
  • 11.2 million trucks enter the United States each year.21 Many more cars do so as well: More than 8.5 million cars cross the Buffalo-Niagara bridges each year alone, and only about 1 percent of them are inspected.22
  • According to the U.S. Department of Commerce, approximately 51 million foreigners vacationed in the United States last year, and this figure is expected to increase to 61 million in three years.23
  • There are currently approximately 11 million illegal aliens living in the United States. Roughly 5 million entered legally and simply overstayed their lawful visit.24
  • Over half a million foreign students are enrolled in American colleges, representing roughly 3.9 percent of total enrollment, including:
  • 8,644 students from Pakistan;
  • A total of 38,545 students from the Middle East, including 2,216 from Iran, 5,579 from Saudi Arabia, and 2,435 from Lebanon, where Hezballah and other terrorist organizations train; and
  • About 40,000 additional students from North African, Central Asian, and Southeast Asian nations where al-Qaeda and other radical Islamic organizations have a strong presence.25

This, of course, is only part of the story. The other aspect of the danger to America is the new and unique nature of the threat posed by terrorists. Virtually every terrorism expert in and out of government believes there is a significant risk of another attack. Moreover, the threat of such an attack, unlike the threat posed by the Soviet Union during the Cold War, is asymmetric.

In the Cold War era, U.S. analysts assessed Soviet capabilities, thinking that their limitations bounded the nature of the threat the Soviets posed. Because of the terrorists' skillful use of low-tech capabilities (e.g., box cutters), their capacity for harm is essentially limitless. The United States therefore faces the far more difficult task of discerning their intentions. Where the Soviets created "things" that could be observed, the terrorists create transactions that can be sifted from the noise of everyday activity only with great difficulty. There can, therefore, be little doubt of the importance of research to better understand the value (or lack thereof) of sifting this mass of data. It is a problem of unprecedented scope, and one whose solution is imperative if American lives are to be saved.

The Total Information Awareness Program
The Department of Defense is experimenting with a number of possible technological approaches to solving this problem, collectively known as TIA.26 It is a research project to develop a variety of new software and hardware tools to improve the way the intelligence, counterintelligence, and law enforcement communities share information on suspected terrorist plans in order to prevent future attacks.

TIA can be a powerful collaborative network for agencies that have a counterterrorism mission. By fostering the sharing of information in existing databases, TIA can close the seams between organizations that have prevented early detection of foreign terrorists in the past. The program conducts research in issues relating to data search, pattern recognition, and information security. It is a multi-year feasibility study and development effort consisting of numerous related research initiatives that first began awarding contracts in 1997.27 A prototype of the more controversial technology is at least five years away.

This research has two intended uses: gathering foreign intelligence on non-Americans and gathering domestic information for intelligence and law enforcement purposes. The research also has two potential government applications: the relatively uncontroversial goal of establishing a much-needed intelligence fusion capability by permitting data integration from a variety of government-owned databases28 and the more controversial creation of a more efficient means of querying non-government databases holding information relevant to domestic terrorism investigations.29

The more controversial aspects of TIA relate to the second of these development projects insofar as it would operate domestically30--the effort to create technology to link databases and permit queries of those databases based upon models of potential terrorist behavior. As the accompanying appendix describes in substantially more detail, there are two aspects of this project: the development of the technological means for querying databases with widely varying data formats and the development of the technological means for conducting such queries while enhancing the privacy of the data being retrieved.

Terrorists preparing for an attack will leave an electronic trail of interactions with the government both outside (e.g., travel from Yemen to Germany) and within the United States (e.g., Customs declarations upon entry) through purchases, travel, and other activities, just as anybody else living in the modern world does. Through a subject-oriented query of databases containing this information, technology being developed by the IAO could be used to gain a more complete understanding of a suspect, his activities, and his relationships with others through an examination of this trail. Through a pattern-oriented query, TIA solutions linked to this information could be used to identify a terrorist based on intelligence data and detailed models of potential terrorist activities.31

Thus, for example, imagine if credible intelligence sources reported that the precursor components of Sarin gas were being smuggled into the United States by al-Qaeda operatives via flights originating in Germany during the month of February 2003. If TIA-based technologies were available today, a pattern-based inquiry of existing government databases might produce a list of non-resident aliens entering the United States during that period on flights meeting those specifications. This information might be cross-checked against other government databases identifying known or suspected terrorists. A subject-based data query might then be used to develop additional information about those identified as warranting further investigation. Their purchase, for example, of additional materials that might assist in the deployment of Sarin gas (canisters and the like) could conceivably be sifted from the information in non-government databases and used as a predicate for further investigation.

Because of the evident challenges to civil liberties that such capabilities would present, the TIA development program has built into its research agenda various measures designed to protect privacy by keeping personal data and irrelevant information out of government's hands. To insure this privacy protection, as part of its research, the IAO is developing technologies intended to prevent the examination of personal information and general misuse. These include, for example, information partitioning and selective revelation technology (that is, separating individual identification information from the underlying data). It also includes the increased use of filters and software to analyze data and remove information unrelated to the investigation.

The combined components of the TIA program would present the intelligence community with a powerful means to electronically intercept and process electronically stored data. Because of the potential power of these tools, the IAO is investing resources in technology that "can allow us to make substantial progress toward supporting both privacy and national security."32


Paul Rosenzweig
Paul Rosenzweig

Former Visiting Fellow, The Heritage Foundation (2009-2017)

Michael Scardaville

Former Policy Analyst