Neural Networks and Deep Learning

Lecture 3 · Game: Stop early or train longer? A game on training curves

Stop early or train longer? A game on training curves

You train a neural network and must decide when to stop. Each chart is a real run of Lecture 3: the training loss and, on most cards, the loss on a validation set the network does not train on. Pick one of three epochs. The right answer is the epoch whose network makes the smallest error on new data: a test set nobody looked at during training. Nine charts, about four minutes.

Seven charts come from German credit applications (600 for training, 200 for validation, 200 for test); two from card payments split by time. The loss is the cross-entropy: the lower, the better the predicted probabilities. On the first three charts you see only the training loss, as in a report that shows the fit to the training data. The model you play against is early stopping: it picks the epoch with the lowest validation loss.

Class board for the projector

Data: German credit (Hofmann, 1994; UCI), 1,000 applicants split 60/20/20 at random; card fraud of the ULB Machine Learning Group and Worldline (OpenML 1597), split by time. Runs: the networks of analysis_L3.py (seed 20261108), repeated with the test loss recorded after every epoch. Code: lectures/L3/make_game_L3.py.