Evaluating models: beyond accuracy

In 1957 Cyril Cleverdon, librarian of the College of Aeronautics at Cranfield in England, began a series of experiments funded by the US National Science Foundation. The question sounded dull: which way of indexing technical papers helps people find what they need?

His method outlived the answer. The Cranfield tests built a fixed collection, in its best-known form 1,400 documents and 225 requests, and decided in advance which documents were relevant to each request. Any indexing system could then be scored on the same ground with two numbers: how many of the relevant documents it found, and how many of the documents it found were relevant.

Those two numbers are recall and precision. Every classifier you build is still judged by them.