The psychological impact of false positives

False positives, or the act of marking legitimate content as being malicious, are an unfortunate but unavoidable consequence of rapid response security technologies. They are relatively rare, unseen events, that make the news only when something goes horribly wrong. Security filters in both the real world and in the electronic world all work on pretty much the same principles. The computer security filters look for the presence of a set of "features", such as a set of binary strings or the libraries used by a given program, while real world security filters look for weapons in carry-on bags or other suspicious activities.

Both types of systems have error rates that are defined, measured, and hopefully reduced. Type I errors, or false positives, are mistakes where something is marked malicious when it is really clean, while Type II errors, or false negatives, are mistakes where malicious content is marked legitimate. Type I errors consists of legitimate mail going to your spam folder or travelers being told they can't fly due to their name being on a secret list. Type II errors consists of viruses that infect machines that are already running anti-virus and guys who smuggle shoe-bombs onto airliners.

Acceptable numbers for false positive and false negative errors depend on the relative cost of an error and when the last error occurred. As a society we tolerate relatively high false positive error rates as long as false negative errors are zero for the screening of potential terrorists, while we don't tolerate false negative errors at all for desktop anti-virus, as it would affect legitimate software.

As errors fade into memory, our risk tolerance changes. Today, we are far less tolerance of false positives in anti-virus systems, as not too many people remember the days of when the Michelangelo virus would nuke a chunk of the hard drive on a given day. American citizens are also less tolerant of false positives when it comes to finding terrorists, as the current political climate has clearly shown.

All of you are aware that Google had a brief false positive issue on Saturday morning that was rapidly remedied. Their technology is important for providing the average user with a first-line filter for removing web-borne threats. I hope that people will continue to trust it, even with their small increase in false positive rate.