Lesson 5 of 81986–20018 min
Decision trees and ensembles
The first volume of the journal Machine Learning, in 1986, carried a paper by the Australian computer scientist J. Ross Quinlan: “Induction of Decision Trees”. It described ID3, an algorithm that builds a tree of yes/no questions from examples by always asking next the question that carries the most information, in Claude Shannon's sense of the word.
The idea is easy to state and still at the heart of the models that win most competitions on tabular data. What changed is that we stopped trusting a single tree and started averaging hundreds of them.
The rest of this lesson is part of Pro
Create a free account to read the first lesson of every path, and go Pro for the rest.
Sources
- Induction of Decision Trees (J. R. Quinlan, Machine Learning 1, 81–106, 1986) — Springer
- Bagging Predictors (L. Breiman, Machine Learning 24, 123–140, 1996) — Springer
- Decision Trees — scikit-learn documentation