The latest airline explosion plot has sparked a predictable overreaction by the much-beloved TSA, which once again demonstrates that it lacks a basic understanding of precision and recall.
Let's recap, shall we ? Suppose you're trying to find all elements of a subset S in a universe U, and return as an answer the set H.
Precision measures the quality of your answer; what fraction of the elements of H are indeed in S ? The higher the precision, the lower the false-positive rate.
Recall measures the efficacy of the procedure; what fraction of the elements of S did you actually find ? The higher the recall, the lower the false-negative rate.
As any statistician will tell you, false-positives and false-negatives are complementary; increase one, and the other decreases. Search engines need high precision, (they also need good ranking, but that's a different story).
The TSA is clearly going for high recall. Banning bottles of water will surely eliminate any future plans for liquid explosives that use water, but it also eliminates the many (how shall I say) *innocent* uses of water ?
p.s I don't mind the short term enforcement of stricer guidelines while law enforcement attempts to figure out what kinds of explosives were being designed. I just have no faith that the new draconian regulations will be relaxed any time soon, considering how long it took to allow us to carry nail clippers on board flights again.
Suresh, here's something also well-written, expressing part of what you express above (but cutting out the CS terminology):
ReplyDeletehttp://walkerw.blogspot.com/2006/08/knowledge-is-best-defense-against.html
Posted by Amit C
Seems the answer is simple enough though. I don't hear of any bombs exploding in Northern Ireland, do I?
ReplyDeleteAs long as you keep wagging wars and being arogant, and you refuse to have reasonable politics, people are going to try to fight back.
It is also quite statistical. You upset 60 million people here, 60 million people there, statistically, you are going to get just enough desperate and crazy people to create lots of problems.
The recipe for security is simple, really: do no evil.
You'll always have nut cases, but statistically, these are going to be rare and easily outgunned.
Posted by Daniel Lemire
"do no evil" is exemplified in Northern Ireland ? You should go back and read an issue of the Atlantic Monthly from earlier this year (I think April). It tells the tale of one of the british double agents inside the IRA, and provides a snapshot of the 'dirty war' (their term, not mine) waged by British intelligence against the IRA.
ReplyDeleteI'd be hard pressed to describe anything that any of the warring parties did there as "not evil".
Posted by Suresh
Suresh, maybe you should become a consultant to the TSA? You are from Bell labs after all. :)
ReplyDeletePosted by Anonymous
The problem is, of course, there is no scope for error. Even if you implement a strategy that has 99% recall, you still have many planes being hijacked. In such high-risk situations, I think it is right to maximise recall at the cost of precision.
ReplyDeleteAnd why, oh why, does one need nail clippers on the plane? Given a choice between not clipping my nails on a plane and being blown apart at 30k feet over the Atlantic, I personally would choose the former.
Posted by Banjo