Four rounds on how a network learns. In each round you first predict what will happen and lock your answer, then test it on the real loss surface of the lecture, which you can turn with the mouse or a finger, and only then see the proof. The data are those of the lecture: the median house value of the 20,640 block groups of California (1990 census) against their median income, and the 64-64 network of slide 17.
The gold diamond marks the least-squares minimum; the red ball is gradient descent. Drag to turn, scroll or pinch to zoom.
Round 1 · The narrow valley
- 1 Predict
- 2 Test
- 3 Prove
Round 2 · Just above the edge
- 1 Predict
- 2 Test
- 3 Prove
Round 3 · Two fixes
- 1 Predict
- 2 Test
- 3 Prove
Round 4 · A real network
- 1 Predict
- 2 Test
- 3 Prove
Loss surface: mean squared error (MSE) of value = a + b × income, computed exactly from five sample moments of the data; gradient descent and momentum use the formulas of analysis_L2.py (start a = 4, b = −0.5, or b = −1 when standardised). Network: two hidden layers of 64 rectified linear units (ReLU), plain stochastic gradient descent, batch 128, 30 epochs; curves of seed 0, final MSE of three seeds (60/20/20 split, seed 20261108). Code: lectures/L2/analysis_L2.py and interactive_data_L2.py on GitHub.

