Lecture 3
Regularisation and generalisation
Prof. dr. Daniel Traian PELE
Bucharest University of Economic Studies (ASE), Department of Statistics and Econometrics
Romanian Academy, Institute for Economic Forecasting (IPE)
A network that fits its training data perfectly can still fail on the next applicant or the next payment. This lecture is about the gap between the two: why it opens, how penalties, dropout, early stopping and batch normalisation close it, and how to design a validation scheme that measures it honestly. The case study is card fraud, where the event is rare, the data arrive in time order and a careless split can make a mediocre detector look excellent.
We use three real datasets: 600 training applicants of German credit from the University of California, Irvine (UCI) repository, Californian census block groups, and 28,924 card payments of the Université libre de Bruxelles (ULB) and Worldline, with all 492 frauds. Every model is a multilayer perceptron (MLP) trained in PyTorch, compared with logistic regression and, for fraud, with extreme gradient boosting (XGBoost), always with several seeds.