The problem with Big Data

Two interesting items

Another Overhyped Fad

It is intriguing that most of the more optimistic claims about the possibilities of big data come not from intelligence analysts or their policy customers but from people in information technology. This is self-serving, to be sure, but it does raise concerns about hype. Beyond examples such as the NSA program or link analysis, there have not been many concrete examples of specific applications of big data. Some do exist: the terrorism connections so noted; the details of weapons systems; or economic data. However, no amount of data will get at some of the key questions uppermost on the minds of policy makers: intentions. What North Korea or Iran or any other nation or leader will do next is not very susceptible to data. Moreover, as one senior policy official remarked, “I do not want data, I want knowledge and insight.” This is an extremely important point: our customers want knowledge and insight, not data. And no; data in and of itself will not necessarily produce knowledge and insight, any more than crowds will produce wisdom.

A Longtime Tool of the Community

Let’s stipulate that today’s big-data mantra is overhyped. Too many technology vendors are busily rebranding storage or analytics as “big data systems” under the gun from their marketing departments. That caricature rightly is derided by both information technology cognoscenti and non-techie analysts.