national-security purpose. The question of effectiveness must be assessed through rigorous testing guided by scientific standards. The TIA research program proposed an evaluation framework, but none of the results of evaluation have been made public. Some testing and evaluation may have occurred in a classified setting, but neither this committee nor the public has any knowledge of results. Research on how large-scale data-analysis techniques, including data mining, could help the intelligence community to identify potential terrorists is certainly a reasonable endeavor. Assuming that initial research justifies additional effort on the basis of scientific standards of success, the work should continue, but it must be accompanied by a clear method for assessing the reliability of the results.

Even if a proposed technology is effective, it must also be consistent with existing U.S. law and democratic values. First, one must assess whether the new technique and objective comply with law. In the case of TIA, DARPA presented to Congress a long list of laws that it would comply with and affirmed that “any deployment of TIA’s search tools may occur only to the extent that such a deployment is consistent with current law.” Second, inasmuch as TIA research sought to enable the deployment of very large-scale data mining over a larger universe of data than the U.S. government had previously analyzed, even compliance with then-current law would not establish consistency with democratic values.

The surveillance power that TIA proposed to put in the hands of U.S. investigators raised considerable concern among policy makers and the general public. That the program, if implemented, could be said to comply with law did not address those concerns. In fact, the program raised the concerns to a higher level and ultimately led to an effort by Congress to stop the research altogether.

TIA-style data mining was, and still is, possible because there are few restrictions on government access to third-party business records. Any individual business record (such as a travel reservation or credit-card transactions) may have relatively low privacy sensitivity when looked at in isolation; but when a large number of such transaction records are analyzed over time, a complete and intrusive picture of a person’s life can emerge.

Developing the technology to derive such individual profiles was precisely the objective of the TIA program. It proposed to use such profiles in only the limited circumstances in which they indicated terrorist activity. That may be a legitimate goal and could ultimately be recognized explicitly as such by law. However, that the program was at once legal and at the same time appeared to cross boundaries not previously crossed by law-enforcement or national-security investigations gives rise to questions that must be answered.