Neural Networks and Deep Learning

Lecture 3: Regularisation and generalisation

Lecture 3

Regularisation and generalisation

Prof. dr. Daniel Traian PELE

Bucharest University of Economic Studies (ASE), Department of Statistics and Econometrics
Romanian Academy, Institute for Economic Forecasting (IPE)

A network that fits its training data perfectly can still fail on the next applicant or the next payment. This lecture is about the gap between the two: why it opens, how penalties, dropout, early stopping and batch normalisation close it, and how to design a validation scheme that measures it honestly. The case study is card fraud, where the event is rare, the data arrive in time order and a careless split can make a mediocre detector look excellent.

We use three real datasets: 600 training applicants of German credit from the University of California, Irvine (UCI) repository, Californian census block groups, and 28,924 card payments of the Université libre de Bruxelles (ULB) and Worldline, with all 492 frauds. Every model is a multilayer perceptron (MLP) trained in PyTorch, compared with logistic regression and, for fraud, with extreme gradient boosting (XGBoost), always with several seeds.

What you will be able to do

  • Diagnose overfitting from training and validation curves, and explain the bias-variance trade-off and double descent.
  • Write weight decay, L1 penalties, dropout, early stopping and batch normalisation with their formulas.
  • Design a validation scheme without leakage: cross-validation, time-ordered splits, resampling inside the training part only.
  • Evaluate a fraud detector with precision, recall, the area under the precision-recall curve (PR-AUC) and the cost of errors.

Lecture plan

  1. Generalisation and overfittingWhat the generalisation gap is, how it shows in training curves, how test error decomposes into bias, variance and noise, and why very large models can generalise again.
  2. Penalties on the weightsWeight decay and the L1 penalty constrain the size of the weights; they correspond to Gaussian and Laplace priors.
  3. Dropout, early stopping and batch normalisationThree regularisers that act through training rather than through the loss, measured on German credit, card fraud and California housing.
  4. Validation designHow to measure generalisation honestly: cross-validation, the noise of a single split, and the leaks that inflate scores.
  5. Case study: card fraud detectionRare events need their own metrics, weights for the sampling and a cost-based alert budget.

Generalisation and overfitting

What the generalisation gap is, how it shows in training curves, how test error decomposes into bias, variance and noise, and why very large models can generalise again.

A model is judged on applicants it has never seen

Slide 5 in the PDF

The generalisation gap \(R(\hat f)-\hat R_n(\hat f)\) compares the expected loss on new data with the training loss. Regularisation is anything that lowers \(R\) at the price of a higher \(\hat R_n\), usually written as a penalised objective \(\hat R_n(f)+\lambda\Omega(f)\).

Without regularisation, a large MLP memorises 600 applicants in a few epochs

Slide 6 in the PDF

An MLP 256-256 has 78,593 parameters for 600 applicants. Its validation loss is lowest at epoch 3 (0.536) and reaches 2.187 after 150 epochs, while the training loss goes to zero; dropout 0.5 with weight decay only slows the rise (1.767). Logistic regression stays at 0.570.

Chart: Without regularisation, a large MLP memorises 600 applicants in a few epochs

Training and validation cross-entropy of an MLP 256-256 on German credit.

Source: UCI Machine Learning Repository, Statlog German credit. Data: 600 training and 200 validation applicants; MLP 256-256.

In practice: a training loss near zero on credit data is a warning, not a success.

The test error splits into bias, variance and noise

Slide 7 in the PDF

For \(y=f(x)+\varepsilon\) the expected squared error of a model fitted on a random sample decomposes as

\[\mathbb E\big[(y-\hat f_{\mathcal D}(x))^2\big]=\big(f(x)-\mathbb E\hat f_{\mathcal D}(x)\big)^2+\operatorname{Var}\big(\hat f_{\mathcal D}(x)\big)+\sigma^2\]
(3.1)

Regularisation in (3.1) trades a little squared bias for a large reduction in variance; the noise \(\sigma^2\) is a floor no model can beat.

Past the interpolation threshold, more hidden units lower the test error again

Slide 8 in the PDF

With \(p\) random, frozen rectified linear units (ReLU) and a minimum-norm output layer fitted on \(n=\) 400 block groups, the test mean squared error (MSE) is 0.781 at \(p=\) 50, explodes at the interpolation threshold \(p=n\) (\(1.0\times10^{6}\)) and falls again to 0.932 at \(p=\) 8,000: the double descent of Belkin et al. (2019).

Chart: Past the interpolation threshold, more hidden units lower the test error again

Training and test MSE against the number of random ReLU features, median of 10 draws.

Source: StatLib, California housing, 1990 census (Pace and Barry, 1997). Data: 400 training and 5,000 test block groups per draw, 10 draws.

Section summary

Overfitting shows as falling training loss and rising validation loss; the bias-variance trade-off explains the classical U-curve, and double descent shows that very large models can generalise again.

Penalties on the weights

Weight decay and the L1 penalty constrain the size of the weights; they correspond to Gaussian and Laplace priors.

Weight decay as an L2 penalty and as a Gaussian prior on the weights

Slide 10 in the PDF

Adding \(\frac\lambda2\|\mathbf w\|^2\) to the loss turns each stochastic gradient descent (SGD) step into \(\mathbf w\leftarrow(1-\eta\lambda)\mathbf w-\eta\nabla\mathcal L\), and the penalised minimum is the maximum a posteriori (MAP) estimate under a Gaussian prior. With Adam the two differ; Adam with decoupled weight decay (AdamW) restores uniform shrinkage.

An L1 penalty sets weights exactly to zero

Slide 11 in the PDF

The L1 penalty \(\lambda\|\mathbf w\|_1\) has a corner at zero; its proximal (soft-thresholding) step \(w\leftarrow\operatorname{sign}(u)\max(|u|-\eta\lambda,0)\) sets small weights exactly to zero, the lasso of Tibshirani (1996).

First-layer weights under L1 and L2 penalties

Slide 12 in the PDF

On German credit, 67.2 % of the 3,072 first-layer weights end below 0.01 in absolute value with L1, against 26.3 % with L2; the test AUC is 0.768 and 0.761.

Chart: First-layer weights under L1 and L2 penalties

First-layer weights of an MLP with 64 hidden units under L1 and L2 penalties (log counts).

Source: UCI Machine Learning Repository, Statlog German credit. Data: 600 training applicants; MLP with 64 hidden units, 3,072 first-layer weights.

Section summary

Weight decay shrinks all weights in proportion, the L1 penalty sets many exactly to zero; both are priors in disguise.

Dropout, early stopping and batch normalisation

Three regularisers that act through training rather than through the loss, measured on German credit, card fraud and California housing.

Dropout trains an ensemble of thinned networks that share their weights

Slide 14 in the PDF

Inverted dropout multiplies each hidden unit by \(m_j/(1-p)\) with \(m_j\sim\) Bernoulli\((1-p)\) during training and uses the full network at test time. With \(H\) droppable units the network samples from \(2^H\) thinned sub-networks that share their weights (Srivastava et al., 2014).

On 600 applicants, regularisation does not close the gap to logistic regression

Slide 15 in the PDF

Over 5 folds, the MLP 256-256 reaches AUC 0.749 without regularisation, 0.750 at the best weight decay and 0.755 at the best dropout rate, while logistic regression reaches 0.772. Weight decay does shrink the network (weight norm 18.0 at \(\lambda=0\), 10.6 at \(\lambda=3\)), but on so little data a smaller model is the better cure.

Chart: On 600 applicants, regularisation does not close the gap to logistic regression

Held-out AUC of an MLP 256-256 on German credit against weight decay and dropout rate.

Source: UCI Machine Learning Repository, Statlog German credit. Data: 1,000 applicants, 5-fold cross-validation, no early stopping (150 epochs).

Early stopping: keep the weights of the best validation epoch

Slide 16 in the PDF

Early stopping keeps the weights of the epoch with the lowest validation loss and stops after \(P\) epochs without improvement. Near a quadratic minimum, \(t\) steps of size \(\eta\) act like an L2 penalty of strength about \(1/(\eta t)\) (Prechelt, 1998; Bishop and Bishop, 2024).

On card fraud, validation loss bottoms out early while training loss keeps falling

Slide 17 in the PDF

On the time-ordered fraud split, the validation loss of an MLP 256-256 is lowest at epoch 4 (0.0159); with a patience of 20 epochs training stops at epoch 24, and after 120 epochs the validation loss would be 0.0593.

Chart: On card fraud, validation loss bottoms out early while training loss keeps falling

Training and validation loss on card fraud, with the best epoch and the patience window.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: time-ordered split: 17,354 training, 5,785 validation transactions.

Batch normalisation standardises each unit over the mini-batch

Slide 18 in the PDF

Batch normalisation (BN) standardises each unit over the mini-batch and then rescales it with learned \(\gamma_j,\beta_j\); at test time running averages replace the batch statistics (Ioffe and Szegedy, 2015).

Batch norm lets a deep MLP train at a rate at which it otherwise diverges

Slide 19 in the PDF

A 6-layer MLP on California housing trained with SGD and momentum diverges at \(\eta=0.1\) in 3 of 3 seeds without BN; with BN the same rate trains, with a best validation MSE of 0.283. At \(\eta=0.01\) the best values are 0.265 without and 0.277 with BN, the BN curve being noisier.

Chart: Batch norm lets a deep MLP train at a rate at which it otherwise diverges

Validation MSE of a 6-layer MLP with and without batch normalisation at two learning rates.

Source: StatLib, California housing, 1990 census (Pace and Barry, 1997). Data: 6 hidden layers of 64 units, SGD with momentum 0.9, 3 seeds.

Section summary

Dropout and penalties shrink capacity, early stopping is the most effective cheap regulariser on our fraud data, and batch norm is first of all an optimisation aid.

Validation design

How to measure generalisation honestly: cross-validation, the noise of a single split, and the leaks that inflate scores.

Hold-out, cross-validation and nested cross-validation

Slide 22 in the PDF

\(K\)-fold cross-validation averages the loss of \(K\) models, each evaluated on the fold it did not see. Nested cross-validation adds an inner loop for tuning, so the outer estimate measures the whole procedure.

One random hold-out split can move the AUC by several points

Slide 23 in the PDF

Over 200 random 70/30 splits of German credit, the test AUC of the same logistic model ranges from 0.702 to 0.842, with 90 % of splits between 0.743 and 0.816.

Chart: One random hold-out split can move the AUC by several points

Test AUC of logistic regression over 200 random hold-out splits of German credit.

Source: UCI Machine Learning Repository, Statlog German credit. Data: 1,000 applicants, 200 random stratified splits.

Information leakage into the validation set

Slide 24 in the PDF

Four leaks recur in business data: preprocessing fitted on all rows, resampling before the split, ignoring time, and inputs that are only known after the decision (Kaufman et al., 2012). For time series, use expanding-window or blocked validation (Bergmeir and Benítez, 2012).

Oversampling before the split overstates performance

Slide 25 in the PDF

The same MLP on the same payments gets PR-AUC 0.843 with a time-ordered split, 0.853 with a random split, and 0.921 when every fraud is copied 20 times before splitting: copies end up on both sides.

Chart: Oversampling before the split overstates performance

Estimated PR-AUC under three evaluation designs, 5 seeds each.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: the same 28,924 transactions, MLP 128-128 with early stopping, 5 seeds.

Section summary

Validation must mimic deployment: one small split is noisy, random splits ignore time, and resampling before the split leaks.

Case study: card fraud detection

Rare events need their own metrics, weights for the sampling and a cost-based alert budget.

Fraud is rare and clustered in time

Slide 27 in the PDF

The full data hold 284,807 payments with a fraud rate of 0.173 %; our file keeps every fraud and 10 % of the legitimate payments, each weighted 10. The inputs are 28 principal component analysis (PCA) scores and the log amount. The hourly fraud rate reaches 2.27 %, against a median of 0.131 %.

Chart: Fraud is rare and clustered in time

Hourly fraud rate over the two days, with the start of validation and test.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: September 2013, 2 days, 284,807 transactions (weighted sample).

Performance measures for rare events: precision, recall and PR-AUC

Slide 28 in the PDF

Precision is \(TP/(TP+FP)\), recall \(TP/(TP+FN)\), and the PR-AUC averages precision over recall levels; a random score gets a PR-AUC equal to the fraud rate (Saito and Rehmsmeier, 2015). Every legitimate payment counts with weight 10 in \(FP\), so precision is that of the full population.

On the test hours, early stopping is the regulariser with the clearest gain

Slide 29 in the PDF

With 60 fixed epochs and 5 seeds, the test PR-AUC is 0.796 with no regularisation, 0.842 with early stopping, 0.787 with weight decay, 0.798 with dropout, 0.798 with batch norm and 0.841 with all combined.

Chart: On the test hours, early stopping is the regulariser with the clearest gain

Test PR-AUC of six regularisation settings, 5 seeds each.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: time-ordered test set: last 5,785 transactions (75 frauds); 5 seeds.

The regularised MLP compared with logistic regression and gradient boosting

Slide 30 in the PDF

On the last 20 % of the two days, PR-AUC is 0.789 for logistic regression, 0.840 for the regularised MLP (5-seed ensemble) and 0.814 for XGBoost; at 80 % precision they catch 74.7 %, 80.0 % and 80.0 % of the 75 test frauds.

Chart: The regularised MLP compared with logistic regression and gradient boosting

Precision-recall curves on the test hours, population-weighted.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: time-ordered test set: 5,785 transactions, 75 frauds, population-weighted.

Class weights and the prior correction for undersampled data

Slide 31 in the PDF

A model trained on data where legitimate payments were kept with rate \(\beta\) overstates the odds of fraud by \(1/\beta\); the corrected probability is \(p=\beta p_s/(\beta p_s-p_s+1)\) (Dal Pozzolo et al., 2015). Class weights and resampling move the effective cut-off but hardly change the ranking.

Number of alerts and the share of fraud losses avoided

Slide 32 in the PDF

If a missed fraud loses its amount and each alert costs a review of 5 euro (EUR), the test-period losses fall from 7,729 EUR with no alerts to 3,093 EUR for logistic regression, 2,694 EUR for the MLP and 2,695 EUR for XGBoost at their best alert counts.

Chart: Number of alerts and the share of fraud losses avoided

Total test-period cost against the number of alerts sent to review.

Source: ULB Machine Learning Group and Worldline, credit-card transactions (OpenML 1597). Data: test period (last 20 % of the two days); review cost 5 EUR per alert.

In practice: the alert budget, set from amounts and review costs, is the business decision; the model only ranks.

Explore it visually

Free interactive tools and illustrated guides for this lecture; most run in the browser.

  • TensorFlow Playground: choose the spiral data set, noise 50 and 10% training data, and a network with several hidden layers: the test loss stays above the training loss, which is overfitting.
  • TensorFlow Playground: set the regularisation to L2, then L1, with rate 0.01: L1 drives many weights to zero, drawn as thin connections, while L2 shrinks them all.

Section summary

For rare events, report PR-AUC, precision at a workable recall and the cost of alerts, on a time-ordered test set and with several seeds.

Videos: the intuition behind the mathematics (in English)

Abbreviations

AdamWAdam with decoupled weight decay
AIartificial intelligence
AUCarea under the receiver operating characteristic curve
BNbatch normalisation
EUReuro
GNNgraph neural network
GraphSAGEgraph sample and aggregate (a GNN architecture)
IEEE-CISInstitute of Electrical and Electronics Engineers, Computational Intelligence Society
MAPmaximum a posteriori
MLPmultilayer perceptron
MSEmean squared error
OpenMLOpen Machine Learning platform
PCAprincipal component analysis
PR-AUCarea under the precision-recall curve
ReLUrectified linear unit
ROCreceiver operating characteristic
SGDstochastic gradient descent
SHAPShapley additive explanations
UCIUniversity of California, Irvine (Machine Learning Repository)
ULBUniversité libre de Bruxelles
XAIexplainable artificial intelligence
XGBoostextreme gradient boosting

References

  • Będowska-Sójka, B., Wójcik, P. & Pele, D. T. (2026). Early warning systems for cryptocurrency markets: Predicting `zombie' assets using machine learning. The North American Journal of Economics and Finance, 81, 102543. doi:10.1016/j.najef.2025.102543
  • Belkin, M., Hsu, D., Ma, S. & Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32), 15849–15854. doi:10.1073/pnas.1903070116
  • Bergmeir, C. & Benítez, J. M. (2012). On the use of cross-validation for time series predictor evaluation. Information Sciences, 191, 192–213. doi:10.1016/j.ins.2011.12.028
  • Bishop, C. M. & Bishop, H. (2024). Deep Learning: Foundations and Concepts. Springer. doi:10.1007/978-3-031-45468-4
  • Chen, T. & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of KDD 2016, 785–794. doi:10.1145/2939672.2939785
  • Dal Pozzolo, A., Caelen, O., Johnson, R. A. & Bontempi, G. (2015). Calibrating probability with undersampling for unbalanced classification. IEEE Symposium Series on Computational Intelligence, 159–166. doi:10.1109/SSCI.2015.33
  • Diaconu, D. & Pele, D. T. (2026). Explainable AI in financial fraud detection: Evidence from the IEEE-CIS fraud dataset. Proceedings of the International Conference on Business Excellence, 20(1), 118–132. doi:10.2478/picbe-2026-0015
  • Hastie, T., Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer. doi:10.1007/978-0-387-84858-7
  • Hoerl, A. E. & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67. doi:10.1080/00401706.1970.10488634
  • Hofmann, H. (1994). Statlog (German Credit Data). UCI Machine Learning Repository. doi:10.24432/C5NC77
  • Ioffe, S. & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of ICML, 448–456. arXiv:1502.03167
  • Kaufman, S., Rosset, S., Perlich, C. & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4), 1–21. doi:10.1145/2382577.2382579
  • Loshchilov, I. & Hutter, F. (2019). Decoupled weight decay regularization. International Conference on Learning Representations. arXiv:1711.05101
  • Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B. & Sutskever, I. (2021). Deep double descent: where bigger models and more data hurt. Journal of Statistical Mechanics, 124003. doi:10.1088/1742-5468/ac3a74
  • Pace, R. K. & Barry, R. (1997). Sparse spatial autoregressions. Statistics & Probability Letters, 33(3), 291–297. doi:10.1016/S0167-7152(96)00140-X
  • Prechelt, L. (1998). Early stopping — but when? In Neural Networks: Tricks of the Trade, LNCS 1524, 55–69. Springer. doi:10.1007/3-540-49430-8_3
  • Saito, T. & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), e0118432. doi:10.1371/journal.pone.0118432
  • Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15, 1929–1958.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society B, 58(1), 267–288. doi:10.1111/j.2517-6161.1996.tb02080.x

Text written for the web version of the lecture; numbers, charts and sources come from the slides and their Quantlets.