The NSA revelations should open a discussion about how necessary it is to collect massive data – and to which extent.
Data mining technology is only as good as the inherently human effort to determine which data are relevant. This is art as much as science. Unless we begin to value this critical human effort, data mining will not yield results that make us safer. Americans have always excelled at technological innovation and admired the alluring rationality of science. Today, our defense establishment seeks to find ways to eliminate human error from the all-too-human practice of war. Thus, the development of smartweapons able to hit targets with superhuman precision.
The era of supercomputers has arguably given rise to the greatest optimism ever about the primacy of technology. The ability for a supercomputer to „crunch“ incredibly large data sets has allowed some to argue that we can bypass human analysis altogether. The underlying belief in the „power of big data“ inverts reality. It isn't the data that are powerful. It's the people with their insightful grasp of the context of a particular phenomenon. And it is their ability to build algorithms that capture expression of those contexts in massive data sets that we should be focusing on.
Right after the NSA issue broke, former CIA director Leon Panetta's chief of staff, Leon Bash, remarked that, if you're looking for a needle in the haystack, you need a haystack.
Not so. Actually, what you need is an accurate narrative, or theory, about the needle and how to characterize it. Moreover, its characteristics must leave digital indicators, if you are planning to search in a digital haystack. There must be enough examples of needles in the world for researchers to be certain that they can distinguish a needle from a stalk of dried grass. Without a precise sense of how to recognize a needle, all you will get are a lot of false positives. Any process for selecting particular data from a larger set represents a story about the world outside the data.
It has been almost a quarter of a century since Princeton professor Orley Ashenfelter used statistics on rainfall and temperature to predict the quality of Bordeaux wines. The reason that Ashenfelter could compute the value of a wine using statistics is that he had developed a strong theory about how rainfall and temperature combine to produce good wine. In other words, he imposed a pre-existing story onto data and correctly collected those particular data that served the story.
The content of our online searches may not be the best data for analyzing political violence. But it is easier to collect the data than it is to develop an on-the-ground nuanced understanding of behind-the-scenes conspiracy building in, for example, Peshawar.
In other words, if you are looking for a needle, and collection technology makes it easy to build a haystack, it would be an entirely understandable tendency for you to elevate the importance of the haystack. The task facing those seeking to use data mining to support counterterrorism is not fundamentally different from the detective work that has always faced the investigator.
Intelligence is the job of selecting and putting together evidence into a feasible narrative. But it also requires having a nuanced sense of which evidence to look for to fill in the developing story. Poor stories will lead to poor data extraction and collecting more data will not solve the problem. We will not make the necessary advance with an imbalanced focus on the technological capability, without an equally strong focus on our human capabilities.
We need critical thinking that helps us defend against our own biases, knowledge of societies and histories in which we are engaged, and imaginative and nuanced understanding of how statistical data do (and do not) express social patterns. In order to understand problems that are fundamentally social and political, such as international terrorism, analysts need encouragement from their leadership to relentlessly interrogate their own narratives.
Is the story we are imposing on these data the right one? Are we exploring the right data? Are we using these data because they are the right source for insight, or simply because they are available? This encouragement can be reflected in the allocation of resources to projects that develop the human side of cyber-security. And it needs to be reflected in the education and training of national security professionals, in the hiring process and via a general culture of appreciation for the degree to which cyber is a human endeavor.
Above all, we Americans should recognize our technological bias and our tendency to tell ourselves that technology has self-generating power. Perhaps that means developing greater faith in our ability to stay critically engaged in a complex world using the power of knowledge and imagination. That would be an excellent starting point to learn the lesson from the NSA story.
Amy Zalman ist Informationsbeauftragte des US-Verteidigungsministerium.
© The Globalist 2013
("Die Presse", Print-Ausgabe, 16.06.2013)