Lesson 5 of 819867 min
How a neural network learns
In October 1986 the journal Nature published a short paper by David Rumelhart, Geoffrey Hinton and Ronald Williams: “Learning representations by back-propagating errors”. It described how a network with hidden layers, units that are neither inputs nor outputs, could adjust all of its weights to reduce its errors.
That was the answer to the perceptron's dead end. With hidden layers a network can represent features nobody gave it, and solve problems like XOR that a single unit cannot. In 2024 Hinton shared the Nobel Prize in Physics with John Hopfield “for foundational discoveries and inventions that enable machine learning with artificial neural networks”.
The rest of this lesson is part of Pro
Create a free account to read the first lesson of every path, and go Pro for the rest.
Sources
- Learning representations by back-propagating errors (D. Rumelhart, G. Hinton, R. Williams, Nature 323, 533, 1986) — NASA Astrophysics Data System (record of the Nature paper)
- 2024 Physics Nobel Prize: John Hopfield and Geoffrey Hinton — Brown University, Department of Physics