Regulators require explanations of automated decisions
Slide 6 in the PDFThree texts frame the European practice. The AI Act (Regulation 2024/1689) lists creditworthiness assessment of natural persons among high-risk uses (Annex III), requires providers to make the system transparent to those who deploy it (article 13) and gives a person affected by a decision the right to an explanation of the role the system played (article 86). The GDPR, article 22 and recital 71, adds safeguards against decisions based solely on automated processing. The European Banking Authority (EBA) guidelines on loan origination and monitoring (2020) require that models used for credit decisions can be explained. Outside the EU, the Federal Reserve Supervisory Guidance on Model Risk Management (SR 11-7) asks for conceptual soundness, independent validation and ongoing monitoring of every model.
There are business reasons too: explanations reveal leakage (a feature that should not be there), help credit officers trust a score and let the bank tell a customer why the answer was no. The caveat runs through the whole lecture: an explanation that changes when nothing important has changed is a compliance risk, not a safeguard.



NN_L11_shap_global











