Neural Networks and Deep Learning

Lecture 11 · Game: Which feature drove the decision?

Which feature drove the decision?

A bank's network has approved or refused a real home-equity loan application. For each of 8 applicants you see the file and the network's decision, and you choose, among four features, the one that pushed this decision the most. Then you see the answer of SHAP (SHapley Additive exPlanations, Kernel SHAP) and whether LIME (local interpretable model-agnostic explanations) agrees.

The network is the multilayer perceptron (MLP) of Lecture 11, trained on 3,576 other loans (area under the receiver operating characteristic curve, AUC, 0.937 on the test loans); it refuses a loan when its probability of default is at least 50%. The right answer is the option with the largest absolute Kernel SHAP contribution to the log-odds of default; LIME's answer is the option with the largest absolute LIME slope × standardised value. Both explanations were computed with the settings of the lecture.

Class board for the projector

Data: HMEQ home-equity loans (Baesens, Roesch and Scheule, 2016; 5,960 loans). The 30 applicants of the game are drawn at random from the test applicants explained in the lecture (12 refused and 18 approved by the network; seed 20261108); every game draws 8 of them. Kernel SHAP and the seed comparison come from the lecture's cached results; LIME is recomputed with the lecture's settings (5,000 samples, default kernel width, seed 0). Code: lectures/L11/make_game_L11.py.