National Academies Press: OpenBook

Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment (2008)

Chapter: 2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals

« Previous: 1 Scoping the Issue: Terrorism, Privacy, and Technology
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

2
A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals

The government increasingly uses technologies, programs, and systems that involve the acquisition, use, retention, or sharing of information about individuals to fight terrorism or serve other important national goals. These systems are very diverse and in the counterterrorism context range from requiring identification to board airplanes or enter government buildings to telephone and e-mail surveillance and intensive mining of commercial records. For purposes of this framework, this chapter describes all of these, together with the people who operate them, as information-based programs because they have in common their reliance on information about individuals.

This chapter proposes a framework for evaluating and deploying technologies, programs, and systems that rely on personal data to prevent terrorism or to serve other important national goals. This framework establishes sets of criteria to address the likely effectiveness and the lawfulness and consistency with U.S. values of any proposed information-based program.

2.1
THE NEED FOR A FRAMEWORK FOR EVALUATING INFORMATION-BASED PROGRAMS

Although information-based programs are not new, advances in digital technology and the proliferation of digital information about individuals have expanded their variety, the interest in their use, and potentially their impact. As a result, information-based programs often raise difficult

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

questions about privacy and other civil liberties, cost, effectiveness, legality, and consistency with societal values.

These issues and the lack of consensus about how they should be evaluated have contributed to limiting the ability of public officials to make rational and informed choices about information-based programs for counterterrorism, research on potentially promising systems, and the availability of information about such systems and their use.

Many groups and individuals have considered how information-based programs should be evaluated and under what conditions they should be deployed. The U.S. Department of Defense Technology and Privacy Advisory Committee,1 the U.S. Department of Homeland Security Privacy and Integrity Advisory Committee,2 the Markle Foundation Task Force on National Security in the Information Age,3 and the McCormick Tribune Foundation’s Cantigny Conference on Counterterrorism Technology and Privacy4 are among the many groups—inside and outside government—to address these vital issues. There is a striking degree of consistency among their recommendations and also in the extent to which they have not been implemented.

Building on the work of these prior efforts and informed by the members’ experiences and research, the committee designed a framework to guide public officials charged with making decisions about the development, procurement, and use of information-based programs. Its purpose is not to impose bureaucratic compliance requirements, but rather to assist well-meaning people at every level of government to do their jobs better, to enhance their effectiveness in countering terrorist threats, to facilitate the wise and timely implementation of new programs, to invest limited government resources wisely, and to ensure that basic American values are not compromised when doing so. The committee also intends the framework to assist judges and policy makers responsible for approving or evaluating those decisions, legislators in crafting the law that governs these programs, and the press and the public in their broad and critical oversight of government activities.

This framework not only shares much in common with the recommendations of prior groups, but it is also consistent with many of the widely recognized standards that already guide information technology procurement, deployment, and use decisions in industry and other areas

1

See Technology and Privacy Advisory Committee, Safeguarding Privacy in the Fight against Terrorism, Department of Defense, Washington, D.C., March 2004, available at http://www.cdt.org/security/usapatriot/20040300tapac.pdf.

2

See http://www.dhs.gov/xinfoshare/committees/editorial_0512.shtm.

3

For more information, see http://www.markletaskforce.org/.

4

See “The Cantigny principles on technology, terrorism, and privacy,” National Security Law Report 27(1):14-16, February 2005.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

of government. Although this framework is necessarily broader, since it reaches far beyond information technology, it mirrors many of the best practices reflected in the Control Objectives for Information and Related Technologies (COBIT), the IT Infrastructure Library (ITIL), International Organization for Standards (ISO) 17799, and the standards promulgated by the National Institute of Standards and Technology (NIST), among others.

In short, the individual elements of what the committee proposes are not wholly new. They reflect much of the wise advice that the government has received—and largely failed to implement—many times before, advice that both it and the private sector do follow in other areas. It is the committee’s hope that by adding to this prior work the breadth of experience, knowledge, and expertise reflected in its membership, it can offer a comprehensive framework that policy makers will, in fact, implement. It is the integration of the individual elements that the committee does think is new.

At the heart of this framework are two sets of questions: First, is an information-based program effective or likely to be effective in achieving its intended goal—in short, does it work? Second, does the program comply with the law and reflect the values of society, especially concerning the protection of data subjects’ civil liberties?

Although these questions are posed as having yes-no answers, any serious application of the framework will almost certainly result in information on how effective and how protective of civil liberties any given information-based program is. This is critical knowledge when determining which of many competing systems, if any, should be developed, acquired, or deployed, and how they might be used or improved. For any potential program, policy makers will have to exercise sound judgment in deciding whether the program is sufficiently effective and sufficiently protective of privacy to warrant proceeding with it, although such judgment should be undertaken after the framework has been applied rather than before.

The questions posed by this framework should be asked not only of all new information-based programs, but also of existing programs today, at regular intervals in the future, and any time that a program is to be altered or put to a different use, to ensure that scarce resources are invested wisely; tools are used appropriately, lawfully, and consistently with societal values; and the best protection is pursued for national security and civil liberties. As discussed in greater detail below, achieving such goals requires routine monitoring, ongoing auditing, and clear, competent oversight. In short, the application of the framework is an ongoing process that should last throughout the operational lifetime of a program.

Technology can aid considerably in the application of the framework,

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

and the effectiveness with which the framework addresses many issues can be enhanced through the use of technology—for example, the creation of immutable audit records and the continuous, automated analysis of those records. But technology alone is not sufficient. What is most critical is that the tools necessary to ensure compliance with the framework—whether or not they are technological—be built into information-based programs to the greatest extent possible and internalized into the processes by which they are developed, acquired, deployed, and used.

The framework is deliberately and necessarily broad because it is designed to apply to all information-based programs. As a result, not all of the points addressed by the framework may be applicable to all programs. Points that are inapplicable should be noted explicitly, along with a clear explanation of why they are inapplicable. The fact that a point is difficult to address should not be a justification for ignoring it. Honest, well-reasoned responses are far more useful to system developers, users, and overseers than none at all, and incomplete or erroneous responses can be supplemented or corrected as additional experience with a program is gained.

The framework and the processes by which it is implemented need to be evaluated regularly and revised as necessary to ensure that it is achieving these objectives. The fact that the framework is undoubtedly imperfect is no reason for avoiding it. Too frequently the argument is heard that national security is too important and the terrorist threat too great to pause to ask hard questions of the systems to be deployed to protect the nation. In the committee’s view, that is the wrong approach. It is precisely because national security is important and the threats to it are great that it is so important to ensure that the systems to be deployed to protect the nation are effective and are consistent with U.S. values.

2.2
EVALUATING EFFECTIVENESS

The first inquiry about an information-based program is concerned with effectiveness: whether a program achieves its intended purpose (i.e., Does it work?), with what precision it does so (i.e., How well does it work?), how it might be made to work better in the future, and how its effectiveness compares with that of other available alternatives. For example, grounding all airplanes would be a highly effective technique for preventing terrorist bombings of airplanes in flight, but it would not be a workable solution because it would also keep millions of law-abiding passengers from flying. As this example suggests, ineffective or overly broad programs often create significant side effects that extend far beyond the immediate impact on the data subjects.

It is impossible in the abstract to establish acceptable levels of effec-

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

tiveness because the level that society demands of any given program is likely to depend on the severity and likelihood of the consequences it is designed to guard against and the burden on individuals and overall cost of the program designed to prevent those consequences.

What matters is that policy makers and government officials responsible for developing, purchasing, deploying, and using information-based programs systematically evaluate the effectiveness of those programs and assess whether they are warranted in light of their likely effectiveness. This is seldom easy, and it is made more difficult by four factors: the rapid change in technologies and applications, the evolving nature of terrorist threats, the fact that so much of the information about terrorist threats and countermeasures is classified, and the reality that dealing with broad-based terrorist threats will require many programs to be scalable to a level far beyond what is typically required in industry or academic settings.

The following criteria are designed to assess and enhance effectiveness in light of these challenges. They are intended to ensure that the nation invests its human, technological, and financial resources wisely. They should be addressed before a new information-based program is procured or deployed and, as appropriate, at regular intervals during the development and use of such a program.

  1. There should be a clearly stated purpose for the information-based program. It is impossible to assess a program’s effectiveness without knowing what it was intended to accomplish. A clear, precise objective is the foundation for any system.

  1. Is that objective worthwhile?

  2. Is it legally appropriate?

  3. Is there a demand or need for it?

  4. Is it already being accomplished or could it be accomplished through less intrusive or less costly means?

A system’s purpose should be the basis for judging if the system is appropriate, and thereafter a basis for assessment of the system and for audits of its use. The purpose may be updated in response to changed circumstances or new experience with the system, but changes to the purpose should be explicit.

  1. There should be a sound rational basis for the information-based program and each of its components. Is there a scientific foundation for the system? For most information-based programs, the rational basis will have to take into account not only how individual components work in a laboratory, but also how they will work together and in connection with other systems in the field. This inquiry is likely to involve not only computer science, statistics, and related fields, but also a range of other social and behavioral sciences.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  1. There should be a sound experimental basis for the information-based program and each of its components. Experimental science, and much of engineering as well, generally involves a logical progression from theory to simulations to laboratory tests, to small-scale field tests, to larger scale tests. In the rush to find quick responses to pressing national security concerns, there is a natural tendency to want to skip one or more of these phases, but the hundreds of millions of dollars wasted on systems that did not go through appropriate experimentation and subsequently did not work suggest that such omissions seldom pay off.

  1. Does the system work to achieve its stated purpose?

  2. Has the new system been shown to work in simulations or laboratory settings or has it been field-tested?

  3. Did the test conditions take into account real-world conditions?

  4. Has it been applied to historical data to determine if it accurately accomplished its objective?

  5. Have experimental successes been replicated to demonstrate that they were not coincidence?

  6. Has the system been subjected to critical analysis, challenge, and likely countermeasures (for example, through “red-teaming”)?5

  1. The information-based program should be scalable. A system for enhancing security that appears promising in the laboratory may well fail in the field if it cannot be scaled up to deal with the real-world flood of data (or even the physical demands of conducting background checks or security scans at airports). Testing scalability has been a special challenge in this area because of the difficulty of obtaining data sets for testing of appropriate size and complexity. In some instances, Congress has proven too quick to rush to judgment on potential systems that were being tested but not deployed, and administration officials have been insufficiently frank about the need for data for testing. Testing on a data set of adequate size is essential to predicting the scalability and therefore the effectiveness of any information-based program.

  2. There should be a clearly stated set of operational or business processes that comprehensively specify how the information-based program should operate in the organization, including who interacts with the program, whether programmatically for input, analysis, or obtaining results, or operationally for maintenance and modification, and with what authority; the information sources and how they are processed; and how the operations defined by the processes contributes to achieving

5

“Red-teaming” refers to the practice of conducting realistic “blind” tests against a system. Such tests are blind in the sense that the operators of the system do not know that they are being tested, and realistic in the sense that the testers are free to do most or all of the things that actual terrorists might or could do in challenging the system.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

the stated purpose. This criterion addresses issues related to operational integration of the program with the organization.

  1. The information-based program should be capable of being integrated in practice with relevant systems and tools inside and outside the organization. For example:

  1. Does the system interact effectively with the sources of information on which it relies?

  2. If it requires combining data, can it do so in practice to yield meaningful results, at the necessary speed, while maintaining an appropriate level of information integrity?

  3. Can the end product of the system be acted on meaningfully by people or other systems?

  1. Information-based programs should be robust. This requires not only that the program work reliably in the field, but also that it not easily be compromised by user errors or circumvented by countermeasures. Investments in programs that are easily undercut or avoided are rarely sound.

  2. There should be adequate guarantees that the data on which the information-based program depends are appropriate and reliable. Data should be stored as long as necessary, but they should be deleted when appropriate and regularly updated if they are needed by the system on an ongoing basis.

  1. Are there adequate guarantees of the information’s validity, provenance, availability, and integrity? Such guarantees are particularly important if a failure to meet the guarantees might adversely affect an individual.

  2. Are the data easily compromised or manipulated so that the system can be defeated?

  1. An information-based program is no better than the data on which it relies, and too many proposals for systems that initially appeared promising foundered when questions were raised about the adequacy and reliability of the source data.

  2. The information-based program should provide for appropriate data stewardship, a term that refers to accountability for program resources being used and protected appropriately according to the defined and authorized purpose. The data must be protected from unlawful or unauthorized disclosure, manipulation, or destruction. In addition, there should be technologies and/or procedures built into the system to ensure that privacy, security, and other data stewardship and governance policies are followed.

  3. There should be adequate guarantees of objectivity in the testing and assessment of the information-based program. In the race for success stories and government contracts in the fight against terrorism, there is

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

a clear tendency to promote systems that lack appropriate guarantees of objectivity in the testing of their effectiveness. This is unacceptable when spending public money, especially when the stakes are so high. No agency or vendor should do all of the testing on the information-based programs it is promoting. Academics typically depend on peer review. That may be more difficult when the systems involved are classified, but it is the standard that the government should be seeking to achieve through appropriate measures. Often scientists or other experts with clearances can help test and evaluate the test results on systems they have not been involved in developing. Technical advisory committees, with members with appropriate clearances, are useful. Third-party assessment even within the government, so that one agency tests another’s systems, would help bring independence to the development and evaluation process. The government should assess independently the effectiveness of any system that it is considering purchasing or deploying. To the extent possible, testing should be blind—to both researchers and research subjects—so that the risk of biasing the outcome is diminished. The causes of failures should be documented so that they can be avoided in developing future systems, or reexplored as technologies and data sources evolve. Failures, as well as successes, should be reported together with what the agency has learned about the cause of those failures.

  1. There should be ongoing assessment of the information-based program. No system, no matter how well designed or tested, will be perfect. There will always be not only unforeseen issues, but also entirely foreseeable ones, such as erroneous or mismatched data, false positives, and false negatives. Assessment is critical to detecting errors, correcting them, and improving systems to reduce errors in the future. Assessment is also essential to ensuring that the system is used properly and only for appropriate purposes. Are there mechanisms for detecting, reporting, and correcting errors? Are there monitoring tools and regular audits to assess system and operator performance?

  2. The effectiveness of the information-based program and its compliance with these key requirements should be documented. Documentation is necessary to ensure that these critical issues are addressed during the development of new information-based programs, and also to respond to subsequent inquiries about their effectiveness. Satisfactory documentation should be required before any information-based program is procured or deployed. When such a system uses personally identifiable information or otherwise affects privacy, the documentation should be examined by an entity, such as an independent scientific review committee, that is capable of evaluating the scientific evidence of effectiveness outside the agency promoting the new system.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

2.3
EVALUATING CONSISTENCY WITH U.S. LAW AND VALUES

The second inquiry is concerned with whether a proposed (or existing) information-based program is consistent with U.S. law and values. Lawfulness is more likely to be binary: a proposed action either is or is not against the law. U.S. society expects its government to obey the law, and it is required by the Constitution to do so. In addition, because technologies and events usually outpace law, it is necessary to constantly consider what types of information-based programs should be lawful. In short, are they consistent with the values of U.S. society?

The values inquiry is always difficult, especially in the context of a diverse and pluralistic society like that of the United States. But it is essential in order to respect the values that undergird the system of government and bind people together. Evaluating information-based programs in light of values is also essential because the Supreme Court has limited the Fourth Amendment to protect only “reasonable expectations” of privacy, and it has found that reasonableness is measured in part by what society is willing to accept as reasonable and in part by what individuals’ subjective expectations are. An awareness of society’s values and individual expectations is therefore critical for understanding what expectations of privacy the law is likely to regard as reasonable and therefore afford legal protection. In addition, paying attention to core values is necessary to avoid creating a race to the bottom—in which the public begins to accept uses of personal data only because the law permits them.

There are also practical, utilitarian reasons for concern about values. Promising antiterrorism systems may be derailed, even ones well within existing law, because they so offend popular and political understandings of privacy that go beyond existing legal requirements.

The determination as to whether a proposed system is lawful, or should be lawful, often requires evaluating the effectiveness of the system in light of its purpose, cost, and the consequences if it fails. As a result, while clear and unambiguous (bright-line) legal rules are desirable, they inevitably rely on subjective judgments that overlap with the effectiveness criteria described above. For example, the precision and accuracy of a system are key aspects of any determination of legality in which individual rights are involved. If the government obtains a warrant to tap a specified phone line but taps another line instead, it has probably broken the law. Or if a surveillance order from a court requires the government to delete nonrelevant communications but it fails to do so, the entire court order and all of the evidence obtained through it can be thrown out. Understanding a program’s effectiveness is also often necessary because the law requires the government and courts to assess whether there are any equally effective but less intrusive means of accomplishing the purpose.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

In the absence of an assessment of effectiveness, such a requirement is impossible to satisfy.

Effectiveness also matters from the standpoint of values, not so much as a requirement of a specific law, but as a commonsense or even an ethical requirement. Any intrusion on privacy would be entirely unjustified if it were not accompanied by some reasonable chance of accomplishing a worthwhile purpose. If an intrusion is perforce ineffective, it would seem by its very nature unwarranted. (Of course, the converse is not necessarily true—it may be that even effective programs should not be deployed because they do offend the ethical sensibilities of the citizenry.)

The following criteria are therefore designed not only to ensure that a proposed system is lawful in the face of existing laws, but also to reduce the impact on privacy that might otherwise render the system either unlawful in the future or politically impractical. They should be addressed by agency officials before a new information-based program is procured or deployed and, as appropriate, at regular intervals during the development and use of such a system. The committee also believes that the criteria should be useful to judicial and congressional officials as they evaluate new and existing programs and determine the boundaries of the nation’s laws protecting privacy and other civil liberties. The criteria are divided into three categories to facilitate their application.

2.3.1
Data

  1. Need for personal data. The need for personal data to accomplish the stated purpose and the specific uses for personal data should be clearly identified. Personal data should not be used unless they are reasonably necessary to achieve the stated objective and effective in doing so. Alternatives should be explicitly considered to determine whether there are equally effective means of achieving the same purpose that rely less on personal data (or on less personal data). Such alternatives are usually preferable.

  2. Sources of data. The sources of those personal data should be clearly identified. It must be lawful for the source to supply the data and for the agency to obtain them.

  3. Appropriateness of data. The personal data should be determined to be appropriate for the intended use, taking into account the purpose(s) for which the data were collected, their age, and the conditions under which they have been stored and protected. Data quality, integrity, and provenance should be assessed explicitly and determined to be appropriate for the intended use and objective. In addition, information-based programs should not rely exclusively on data that relate to the exercise of

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

rights protected by the First Amendment (i.e., freedom of expression, the press, assembly, religion, and petition).

  1. Third-party data. Because using personal data from other government agencies or from private industry may present special risks, such third-party data should be subject to additional protections:

  1. The agency should take into account the purpose for which the data were collected, their age, and the conditions under which they have been stored and protected when determining whether the proposed information-based program is appropriate.

  2. If data are to be used for purposes that are inconsistent with those for which they were originally collected, the agency should specifically evaluate whether the inconsistent use is justified and whether the data are appropriate for such use.

  3. Because of the difficulty of updating, overseeing, and maintaining the accuracy and context of data that have been copied from place to place, data should be left in place whenever possible (i.e., in the hands of the third parties that originally controlled those data). If this is impossible, they should be returned or destroyed as soon as practicable.

  4. Private entities that provide data to the government on request or subject to judicial process should be reasonably compensated for the costs they incur in complying with the government’s request or order.

2.3.2
Programs

  1. Objective. The objective of the information-based program should be clearly stated. That objective must be lawful to pursue by the agency developing, procuring, or deploying the program.

  2. Compliance with existing law. The information-based program should comply with applicable existing law.

  3. Effectiveness. Using scientifically valid criteria, the information-based program should be demonstrated to be effective in achieving the intended objective.

  4. Frequency and impact of false positives. The information-based program should be demonstrated to yield a rate of false positives that is acceptable in view of the purpose of the search, the severity of the effect of being identified, and the likelihood of further investigation.

  5. Reporting and redress of false positives. There must be in place a process for identifying the frequency and effects of false positives and for dealing with them (e.g., reporting false positives to developers to improve the system, correcting incorrect information if possible, remedying the effects of false positives as quickly as practicable), as well as a specific locus of responsibility for carrying out this process.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  1. Impact on individuals. The likely effects on individuals identified through the information-based program should be defined clearly (e.g., they will be the subject of further investigation for which a warrant will be sought, they will be subject to additional scrutiny before being allowed to board an aircraft, and so on).

  2. Data minimization. The information-based program should operate with the least personal data consistent with its objective. Only the minimally necessary data should be accessed, disseminated, or retained. This has long been a requirement of U.S. surveillance law, although it has been rendered largely irrelevant in recent years as technology and applications have evolved so that vast streams of data are recorded and stored, rather than just limited, relevant elements. Moreover, the proliferation of digital data and dramatic reductions in the costs associated with sharing and storing data mean that even irrelevant data are routinely retained by the government indefinitely. Giving new force to minimization requirements is essential to avoiding the situation of government maintaining ubiquitous data records that threaten to invade personal privacy and overwhelm efforts to use data effectively to enhance security. Whenever practicable, the information-based program should rely on personal data from which information by which specific individuals can be commonly identified (e.g., name, address, telephone number, Social Security number, unique title) has been removed, encrypted, or otherwise obscured.

  3. Audit trail. The information-based program should create a permanent, tamper-resistant record of when data have been accessed and by whom. Continuous, automated analysis of audit records can help ensure compliance with applicable laws and policies. This is especially important when sensitive or potentially sensitive data are involved.

  4. Security and access. The information-based program should be secured against accidental or deliberate unauthorized access, use, alteration, or destruction. Access to such an information-based program should be restricted to persons with a legitimate need and protected by appropriate access controls, taking into account the sensitivity of the data.

  5. Transparency. The information-based program should be developed, deployed, and operated with the greatest transparency possible, consistent with its objective. Persons affected by the program and the public generally should be informed as fully as practicable of the existence of the program, its purpose, cost, the laws and regulations under which it operates, the measures in place for assessing its effectiveness and protecting privacy, and the process for reporting and obtaining redress of grievances concerning its operation.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

2.3.3
Administration and Oversight

  1. Training. All persons engaged in developing or using information-based programs should be trained in their appropriate use and the laws and regulations applicable to their use.

  2. Agency authorization. No information-based program that involves the acquisition, use, retention, or sharing of personally identifiable information should be developed, procured, or deployed until a senior agency official, preferably one subject to Senate confirmation, has certified in writing that it complies with the requirements of this framework.

  3. External authorization. The deployment or use of any information-based program that relies on sensitive personally identifiable information, personally identifiable information collected surreptitiously, personally identifiable information that has been obtained from a third party without individual consent, or personally identifiable information that is being used for a purpose that is incompatible with that for which it was originally collected should be conditioned on an appropriately specific authorization from a source external to the information-based program.6 Typically, this would be authorization by an appropriate court (federal Article III, Foreign Intelligence Surveillance, or state), but Congress may provide for other forms of external authorization.

  4. Auditing for compliance. Information-based programs should be audited not less than annually to ensure compliance with the provisions of this framework and other applicable laws and regulations. The party conducting such audits may or may not be in the department responsible for the program but should operate and report independently of the program in question.

  5. Privacy officer. Before an agency develops, procures, or deploys an information-based program, it should have in place a policy-level privacy officer. The privacy officer would be responsible for ensuring the training of appropriate agency personnel on privacy issues; assisting in the design and implementation of systems to protect privacy; working with the general counsel, inspector general, other appropriate officials in

6

he specificity of the authorization required in any given instance is an issue that changing technologies have highlighted in the context of the wiretapping of voice calls. For example, for criminals who use throwaway cell phones, authorizations that grant wiretap authority to law enforcement agencies only for specific phone numbers are obviously much less useful than authorizations that grant wiretap authority for all phones that a specific individual might use. Furthermore, the committee expects that the issue of specificity will become more important as the scope of information sought becomes broader. Because the nature of the appropriate specificity depends on the particular information needs of a given program, it is impossible for the committee to specify in advance in its broad framework the appropriate level of specificity. However, it does note that policy makers should make explicit decisions regarding the appropriate level of specificity.

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

the agencies to ensure compliance with such systems; providing advice and information on privacy issues and tools for protecting privacy; and advising agency leaders and personnel on privacy matters and the implementation of this framework.

  1. Reporting. An agency that develops, procures, or deploys an information-based program should report to Congress not less than annually, or more frequently as required by law, on the use of the system; its effectiveness; the nature, use, and timeliness of redress mechanisms; and the integrity of the system and the data on which it relies. The report should be made public to the greatest extent possible.

2.4
A NOTE FOR POLICY MAKERS: APPLYING THE FRAMEWORK IN THE FUTURE

In times of crisis, policy makers are often pressured into making important decisions with inadequate information and too little time for consultation and deliberation. When those decisions involve laws concerning information-based programs, the consequences can be especially significant and long-lasting. Law inevitably tends to lag behind technology, yet dramatic technological changes can alter the scope of laws overnight. So, for example, when the Supreme Court excluded records maintained by third parties from the scope of the Fourth Amendment in 1976, it created a situation in which, 30 years later, because of the proliferation of digital records maintained by third parties, almost all information about individuals would be accessible to the government without judicial authorization.

The committee intends the entire framework proposed in this chapter to be useful to policy makers in outlining issues to be addressed through legislation or regulatory policy, as well as in proposing specific steps for ensuring that the nation fights terrorism effectively and consistently in accord with its core values. However, the breadth and variety of information-based programs, as well as the constantly changing capacity of technology, make crafting legislation governing those programs and protecting civil liberties a difficult task. To further facilitate effective legislation to achieve these critical goals, the committee presents this additional brief discussion of how the framework might be applied in the legislative context.

In the committee’s view, all such legislation should specifically address the following eight areas (many specific elements of which have already been described above):

  1. Agency competency. Is the agency being authorized to operate or use the information-based program competent to do so? Is the program

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

consistent with its mission? Is it staffed appropriately? Are its staff trained appropriately? Does it have a policy-level chief privacy officer? Does it have a culture of respecting the law and civil liberties?

  1. Purpose. Does the information-based program have a clearly articulated purpose against which its effectiveness and impact on civil liberties can be assessed? Are there appropriate protections to guard against mission creep or repurposing of the program without careful deliberation? Will that purpose remain valid in the face of countermeasures or likely technological changes? Are there procedures in place for reevaluating that purpose?

  2. Effectiveness. Are there appropriate guarantees that the information-based program and each of its components are effective? Are credible processes in place to measure effectiveness and to ensure continual assessment of effectiveness and efforts to improve effectiveness? Are measures of effectiveness documented?

  3. Authorization. Are requirements in place for authorization by an identified, accountable official both before an information-based programs is created, procured, or deployed and before such programs are applied to personal data about a specific individual? Does the authorization for applying the program to a specific individual come from a court or other source external to the agency operating the program, especially if the data gathering or use is covert?

  4. Data. Are there reasonable guarantees that the personal data to be used by an information-based program are appropriate, sufficiently accurate for the stated purpose, and reliably available on a timely basis? Are there protections to ensure that only necessary personal data are used, retained no longer than necessary, and protected against accidental or deliberate misuse? Are the data and the manner in which they are obtained consistent with U.S. values? Does their use deter the exercise of constitutionally protected rights?

  5. Redress. Are there robust systems in place to identify errors, such as false positives, use them systematically to improve information-based programs, and provide rapid, effective redress to affected individuals?

  6. Assessment. Are there reliable tools for assessing the performance of information-based programs and their compliance with applicable laws and regulations, as well as for acting on those assessments? Are the results of ongoing assessment documented?

  7. Oversight. Is the information-based program subject to meaningful oversight from both inside and outside the agency, including from Congress? Are the program and its oversight mechanism transparent to the public and the press to the greatest extent possible? If transparency is impossible, are there reliable means for heightened independent agency, judicial, and/or congressional oversight?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

2.5
SUMMARY OF FRAMEWORK CRITERIA

2.5.1
For Evaluating Effectiveness

  1. Is there a clearly stated purpose for the information-based program?

    • Is that objective worthwhile?

    • Is it legally appropriate?

    • Is there a demand or need for it?

    • Is it already being accomplished or could it be accomplished through less intrusive or less costly means?

  1. Is there a sound rational basis for the information-based program and each of its components?

    • Is there a scientific foundation for the system?

  1. Is there a sound experimental basis for the information-based program and each of its components?

    • Does the system work to achieve its stated purpose?

    • Has the new system been shown to work in simulations or laboratory settings or has it been field-tested?

    • Did the test conditions take into account real-world conditions?

    • Has it been applied to historical data to determine if it accurately accomplished its objective?

    • Have experimental successes been replicated to demonstrate that they were not coincidence?

    • Has the system been subjected to critical analysis, challenge, and likely countermeasures (for example, through “red-teaming”)?

  1. Is the information-based program scalable?

    • Has it been tested on a data set of adequate size to predict its scalability?

    • Has it been tested against likely countermeasures or changes in technologies, threats, and society?

  1. Is there a clearly stated set of operational or business processes that comprehensively specify how the information-based program should operate in the organization?

  2. Is the information-based program capable of being integrated in practice with related systems and tools?

    • Does the system interact effectively with the sources of information on which it relies?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  • If it requires combining data, can it do so in practice to yield meaningful results and at the speed necessary?

  • Can the end product of the system be acted on meaningfully by people or other systems?

  1. Is the information-based program robust?

    • Can it easily be compromised by user errors?

    • Can it easily be circumvented by countermeasures?

  1. Are there appropriate guarantees that the data on which the information-based program depends are appropriate and reliable?

    • Are there adequate guarantees of the information’s validity, provenance, availability, and integrity?

    • Are the data easily compromised or manipulated so that the system can be defeated?

  1. Does the information-based program provide for appropriate data stewardship?

    • Are the data protected from unlawful or unauthorized disclosure, manipulation, or destruction?

    • Are there technologies and/or procedures built into the system to ensure that privacy, security, and other data stewardship and governance policies are followed?

  1. Are there adequate guarantees of objectivity in the testing and assessment of the information-based program?

    • Has there been peer review or its equivalent?

    • Has the program been evaluated by entities with no stake in its success?

    • Have test results been evaluated by independent experts?

    • Was testing blind—to both researchers and research subjects—whenever possible?

  1. Is there ongoing assessment of the information-based program?

    • Are there mechanisms for detecting and reporting errors?

    • Are there monitoring tools and regular audits to assess system and operator performance?

  1. Have the effectiveness of the information-based program and its compliance with these key requirements been documented?

    • Has the documentation been examined by an entity capable of evaluating the scientific evidence of effectiveness outside the agency promoting the new system?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

2.5.2
For Evaluating Consistency with Laws and Values

The Agency

  1. Does the agency have in place a policy-level privacy officer?

  2. Does the agency report to Congress not less than annually, or more frequently as required by law, on the use of its information-based programs, their effectiveness, the nature and use of redress mechanisms, and the integrity of the programs and the data on which they rely? Is that report made public to the greatest extent possible?

  3. Have all persons engaged in developing or using information-based programs been trained in their appropriate use and the laws and regulations applicable to their use?

The Program

  1. Is the objective of the information-based program clearly stated? Is that objective lawful for the agency developing, deploying, or using the program to pursue?

  2. Does the information-based program comply fully with applicable existing law?

  3. Has the information-based program been demonstrated to be effective in achieving the intended objective? Is that demonstration based on scientifically valid criteria?

  4. Has the information-based program been demonstrated to yield a rate of false positives that is acceptable in view of the purpose of the search, the severity of the effect of being identified, and the likelihood of further investigation?

  5. Is there a process in place for identifying the frequency and effects of false positives and for dealing with them (e.g., reporting false positives to developers to improve the system, correcting incorrect information if possible, remedying the effects of false positives as quickly as practicable), as well as a specific locus of responsibility for carrying out this process?

  6. Have the likely effects on individuals identified through the information-based program been defined clearly (e.g., they will be the subject of further investigation for which a warrant will be sought, they will be subject to additional scrutiny before being allowed to board an aircraft, and so on)?

  7. Does the information-based program operate with the least personal data consistent with its objective? Does it access, disseminate, and retain only minimally necessary data? Have data by which specific individuals can be commonly identi-

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×

fied (e.g., name, address, telephone number, Social Security number, unique title) been removed, encrypted, or otherwise obscured whenever possible?

  1. Does the information-based program create a permanent, tamper-resistant record of when data have been accessed and by whom? Does it provide for continuous, automated analysis of audit records?

  2. Is the information-based program developed, deployed, and operated with the greatest transparency possible, consistent with its objective?

  3. Is the information-based program secured against accidental or deliberate unauthorized access, use, alteration, or destruction? Is access to the information-based program restricted to persons with a legitimate need and protected by appropriate access controls, taking into account the sensitivity of the data?

  4. Has (or will) a senior agency official, preferably one subject to Senate confirmation, certified (or will certify) in writing that the information-based program complies with the requirements of this framework?

  5. If the information-based program relies on sensitive personally identifiable information, personally identifiable information collected surreptitiously, personally identifiable information that has been obtained from a third party without individual consent, or personally identifiable information that is being used for a purpose that is incompatible with that for which it was originally collected, have its deployment and use been conditioned on authorization from a source external to that in which the information-based program will exist, and have they been approved by an external authority (e.g., an appropriate court or other authority)?

  6. Is the information-based program audited not less than annually to ensure compliance with the provisions of the proposed framework and other applicable laws and regulations?

The Data

  1. Are personal data necessary to accomplish the objective of a given information-based program? Are the specific uses for personal data clearly identified? Are there equally effective means of achieving the same purpose that rely less on personal data (or on less personal data)?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  1. Are the sources of personal data clearly identified? Is it lawful for the source to supply the data and for the agency to obtain the data?

  2. Are the personal data appropriate for the intended use, taking into account the purpose(s) for which the data were collected, their age, and the conditions under which they have been stored and protected? Do the data relate solely to the exercise of rights protected by the First Amendment (i.e., freedom of expression, the press, assembly, religion, and petition)?

  3. If an information-based program uses personal data from other government agencies or from private industry, are the following additional protections in place?

    • Have the purpose for which the data were collected, their age, and the conditions under which they have been stored and protected been taken into account when determining whether the proposed information-based program is appropriate?

    • If data are to be used for purposes that are inconsistent with those for which they were originally collected, has the agency specifically evaluated whether the inconsistent use is justified and whether the data are appropriate for such use?

    • Are the data being left in place whenever possible? If this is impossible, are they being returned or destroyed as soon as practicable?

    • Is the agency reasonably compensating private entities that provide data to the government on request or subject to judicial process for the costs they incur in complying with the government’s request or order?

2.5.3
For Developing New Laws and Policies

  1. Agency competency

    • Is the agency being authorized to operate or use the information-based program competent to do so?

    • Is the program consistent with the agency’s mission?

    • Is the agency staffed appropriately?

    • Are its staff trained appropriately?

    • Does it have a policy-level chief privacy officer?

    • Does it have a culture of respecting the law and civil liberties?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  1. Purpose

    • Does the information-based program have a clearly articulated purpose against which its effectiveness and impact on civil liberties can be assessed?

    • Are there appropriate protections to guard against mission creep or repurposing of the program without careful deliberation?

    • Will the program’s purpose remain valid in the face of countermeasures or likely technological changes?

    • Are there procedures in place for reevaluating the program’s purpose?

  1. Effectiveness

    • Has the information-based program been demonstrated to be effective in achieving the intended objective?

    • Is that demonstration based on scientifically valid criteria?

    • Are there credible processes in place to measure effectiveness and to ensure continual assessment of effectiveness and efforts to improve effectiveness?

    • Are measures of effectiveness documented?

  1. Authorization

    • Are there requirements in place for authorization by an identified, accountable official both before an information-based program is created, procured, or deployed and before such programs are applied to personal data about a specific individual?

    • Does the authorization for applying the program to a specific individual come from a court or other source external to the agency operating the program, especially if the data gathering or use is covert?

  1. Data

    • Are personal data necessary to accomplish the objective of a given information-based program?

    • Are the specific uses for personal data clearly identified?

    • Are there equally effective means of achieving the same purpose that rely less on personal data (or on less personal data)?

    • Are there protections to ensure that only necessary personal data are used, retained no longer than necessary, and protected against accidental or deliberate misuse?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  • Does the information-based program operate with the least personal data consistent with its objective?

  • Does the program access, disseminate, and retain only necessary data?

  • Have data by which specific individuals can be commonly identified (e.g., name, address, telephone number, Social Security number, unique title, and so on) been removed, encrypted, or otherwise obscured whenever possible?

  • Are there reasonable guarantees that the personal data to be used by an information-based program are appropriate, sufficiently accurate for the stated purpose, and reliably available?

  • Are the sources of those personal data clearly identified?

  • Is access to the information-based program restricted to persons with a legitimate need and protected by appropriate access controls, taking into account the sensitivity of the data?

  • Is it lawful for the source to supply the data and for the agency to obtain the data?

  • Are the data and the manner in which they are obtained consistent with U.S. values?

  • Does their use deter the exercise of constitutionally protected rights?

  • If an information-based program uses personal data from other government agencies or from private industry, are the appropriate additional protections in place?

  1. Redress

    • Is there a process in place for identifying the frequency and effects of false positives and for dealing with them (e.g., reporting false positives to developers to improve the system, correcting incorrect information if possible, remedying the effects of false positives as quickly as practicable, and so on)?

    • Have the likely effects on individuals identified through the information-based program been defined clearly (e.g., they will be the subject of further investigation for which a warrant will be sought, they will be subject to additional scrutiny before being allowed to board an aircraft)?

    • Has the information-based program been demonstrated to yield a rate of false positives that is acceptable in view of the purpose of the search, the severity of the effect of being identified, and the likelihood of further investigation?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
  • Are there robust systems in place to identify errors, such as false positives, use them systematically to improve information-based programs, and provide rapid, effective redress to affected individuals?

  1. Assessment

    • Are there reliable tools for assessing the performance of information-based programs and their compliance with applicable laws and regulations, as well as for acting on those assessments?

    • Does the information-based program create a permanent, tamper-resistant record of when data have been accessed and by whom?

    • Does it provide for continuous, automated analysis of audit records?

    • Is the information-based program audited not less than annually to ensure compliance with the provisions of this framework and other applicable laws and regulations?

    • Are the results of ongoing assessment documented?

  1. Oversight

    • Is the information-based program subject to meaningful oversight from both inside and outside the agency, including from Congress?

    • Are the program and its oversight mechanism transparent to the public and the press to the greatest extent possible?

    • If transparency is impossible, are there reliable means for heightened independent agency, judicial, and/or congressional oversight?

Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 44
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 45
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 46
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 47
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 48
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 49
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 50
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 51
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 52
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 53
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 54
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 55
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 56
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 57
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 58
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 59
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 60
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 61
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 62
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 63
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 64
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 65
Suggested Citation:"2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals." National Research Council. 2008. Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12452.
×
Page 66
Next: 3 Conclusions and Recommendations »
Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment Get This Book
×
Buy Paperback | $67.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

All U.S. agencies with counterterrorism programs that collect or "mine" personal data -- such as phone records or Web sites visited -- should be required to evaluate the programs' effectiveness, lawfulness, and impacts on privacy. A framework is offered that agencies can use to evaluate such information-based programs, both classified and unclassified. The book urges Congress to re-examine existing privacy law to assess how privacy can be protected in current and future programs and recommends that any individuals harmed by violations of privacy be given a meaningful form of redress.

Two specific technologies are examined: data mining and behavioral surveillance. Regarding data mining, the book concludes that although these methods have been useful in the private sector for spotting consumer fraud, they are less helpful for counterterrorism because so little is known about what patterns indicate terrorist activity. Regarding behavioral surveillance in a counterterrorist context, the book concludes that although research and development on certain aspects of this topic are warranted, there is no scientific consensus on whether these techniques are ready for operational use at all in counterterrorism.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!