Lesson 6 of 81967–20077 min
k-nearest neighbours and k-means
In 1967 Thomas Cover and Peter Hart published a short paper in IEEE Transactions on Information Theory about the laziest classifier imaginable: to label a new point, find the closest training point and copy its label. No training at all.
Their result was startling. With enough data, the error of this rule is at most twice the Bayes error, the lowest error any classifier could achieve on that problem. Half of all the information needed for classification, they wrote, is contained in the nearest neighbour.
The rest of this lesson is part of Pro
Create a free account to read the first lesson of every path, and go Pro for the rest.
Sources
- Nearest Neighbor Pattern Classification (T. M. Cover, P. E. Hart, IEEE Transactions on Information Theory 13(1), 1967) — IEEE
- Nearest Neighbors — scikit-learn documentation
- Clustering: k-means — scikit-learn documentation