Skip to Content

Government Surveillance

Report: Data Mining Ineffective Anti-Terrorist Tool

A new National Research Council report cautions that government data mining programs cannot effectively identify patterns of terrorist activity. Pattern-based or predictive data mining was singled out as likely to generate huge numbers of useless leads. Because of this, the authors warned, pattern-based data mining should not be used to deny a person rights and liberties. This mirrors past conclusions that CDT and others have drawn about data mining efficacy.

The Committee that drafted the October 7th report, entitled "Protecting Individual Privacy in the Struggle Against Terrorists," recommended that all U.S. data mining programs be re-evaluated according to criteria set forth in the 376-page document. The authors – which included former Secretary of Defense William Perry – made the case that even well-managed data mining efforts are of only limited usefulness and can infringe on Americans’ privacy.

The Committee recommended that new data mining programs that use sensitive personal information, or personally identifiable information obtained from a third party such as a data broker, should require authorization from a court or other entity designated by Congress. This recommendation is particularly striking because this would be a new role for courts, and because the Department of Homeland Security, which runs a number of such data mining programs, co-sponsored the effort to write the report.

In another key finding, the Committee determined that U.S. privacy laws, including the Privacy Act and the Electronic Communications Privacy Act, are woefully out of date. The report recommended that the next Congress and administration update this legal patchwork to keep pace with modern technology and strengthen inadequate privacy protection. CDT has arrived at this same conclusion years ago and has pressed the issue in Congress.

Recognizing that Americans increasingly live their lives online and also leave behind many digital tracks from their everyday activities, the authors decried the wholesale capture of this data as favoring quantity over quality. The report argued that identifying signs of terrorist activity in huge gobs of information is highly problematic, particularly if the system vacuuming up the data is automated. Rather, the report concluded that current data mining and behavioral surveillance efforts are more likely to produce a host of false positives that target innocent people as threats. The poor data quality, rate of error and overall secrecy surrounding such programs undermines both Americans’ trust in intelligence agencies and the ability of those agencies to protect national security.

The authors recommended that data based counterterrorism programs build privacy protections into their systems prior to implementation, and that greater protections be installed into existing systems. The report provides a framework reminiscent of Fair Information Practices to aid agencies evaluating their data mining programs. The criteria focus on ensuring the program is effective and minimally invasive. For example, programs should collect the minimum amount of data necessary and offer a redress process to remedy privacy violations.