Anti-terrorism software

I noted this Photon Courier post on Monday. But the failure of the FBI's information-sharing software deserves much more attention than it is getting.

It clear that one problem with the project is that many people are involved, but no one was responsible for its success. The evidence is right here:

SAIC has said it believes the problem was caused largely by the FBI: specifically, too many specification changes during the development process...an SAIC executive asserted that there were an average of 1.3 changes per day during the development.

When you are processing that many spec-changes, no one is driving the bus. This is common with software projects and is a big reason why so many of them fail. They keep expanding and morphing.

This huge complexity is responsible for software's permanent crisis: if you build a big enough program, it is almost impossible to make it come out right. Studies show that the average commercial software project takes 50% longer that it was supposed to, and one project in four is abandoned.... The 'beta test' is the industry's admission of failure-- the procedure whereby a product that is known to be flawed, but is nonetheless as good as the manufacturer can make it, is handed to expert users in the hopes they will find some of the remaining bugs. [ David Gelernter, Machine Beauty, 1998]

I wrote about this and other problems with software development here.

The complexity problem also stands out when you examine the success story highlighted by Photon Courier. The counter-sniper system had a very specific objective and, hence, the developers and engineers had a very focused mission.

Another reason for the failure could be that the FBI over-estimated what data-sharing could accomplish. Too many people treat data, information, knowledge, and intelligence as though they are roughly the same thing. Thus, they belive, that if police agencies share data they will automatically generate better intelligence.

I think that it is much more fruitful to think in terms of Haeckel's Hierarchy which distinguishes between these terms in very useful ways. (The Hierarchy is discussed in this PDF.)

The first thing that stands out is that the elements that separate data from information (context) or intelligence from information (inference) are elements that computers do not handle well. This is even more true with "knowledge" which Haeckel explicitly describes as "subjective and [which] resides in humans, not in databases or books."

In short, you cannot get rid of the human element by substituting hard drives, high-speed networks, and a template. Computers are a blunt tool for generating better intelligence. No amount of money, time, or effort can change that.

UPDATE: See additional thoughts here.