Decision trees, random forests and boosting

Leo Breiman spent seven years as an academic probabilist, then left to work as a freelance consultant for thirteen more before joining the statistics department at Berkeley in 1980. Consulting changed how he judged a model: by whether it predicted well on new data, not by how elegant its assumptions were.

In 1984 he co-wrote Classification and Regression Trees with Jerome Friedman, Richard Olshen and Charles Stone. In 2001 he published two papers that still shape the field. One, in the journal Machine Learning, introduced random forests. The other, “Statistical Modeling: The Two Cultures”, estimated that only about 2% of statisticians worked his way, and argued that the rest should pay attention.