Statistical Approaches to Probability-of-Default Model Validation
Summary
The document concerns validation of credit risk models that estimate probability of default. The question raises binomial testing, the Hosmer–Lemeshow chi-square test, and tolerance testing as possible tools for assessing model accuracy, calibration, and discriminatory power, and asks how such tests should be combined. The response does not explain the tests or propose an aggregation method; it provides a bibliography of academic and professional works on validating default probability models.
The cited material includes research on backtesting probability of default, exposure at default, and loss given default; stress testing; multiple testing procedures; statistical approaches to PD validation; and internal credit models. This makes the document useful as a starting point for further study rather than a standalone validation procedure. It gives no evidence comparing the tests, no criteria for choosing among them, and no guidance on sample size, dependence, or interpreting conflicting validation results.
Key ideas
- PD model validation can involve tests of calibration and discriminatory performance.
- The question names binomial, Hosmer–Lemeshow, and tolerance tests but does not explain their implementation.
- The response directs readers to published work on PD model validation and related credit risk measures.
- No procedure is given for combining test results into an overall accuracy assessment.
Tags
Full text
# Model Validation Aggregation Documentation (Binomial, Hosmer-Lemeshow, Tolerance) - Credit Risk et cetera # Model Validation Aggregation Documentation (Binomial, Hosmer-Lemeshow, Tolerance) - Credit Risk et cetera I came across some document that says for a PD (Probability of Default) model in order to assess its accuracy you need to first look at the Binomial Test, then the Hosmer-Lemeshow Chi-square test, then the Tolerance test etc. The problem is this document has no references at all. Are you away of some books and/or online resources that deal with such tests and how you aggregate in order to assess the Accuracy/Discriminatory power et cetera of Credit Risk Models / PD models etc? ## Answer by Dimitri Vulis (score 3, accepted) https://quant.stackexchange.com/a/59247 Take a look at these: Bauke Maarse. Master Thesis: Backtesting Framework for PD, EAD and LGD (2012) https://essay.utwente.nl/61905/1/master_B._Maarse.pdf Fábio Yasuhiro Tsukahara, Herbert Kimura, Vinicius Amorim Sobreiro, Juan Carlos Arismendi Zambrano. Validation of default probability models: A stress testing approach (2016) https://doi.org/10.1016/j.irfa.2016.06.007 Sebastian Döler. Validation Of Credit Default Probabilities Via Multiple Testing Procedures. (2010) https://arxiv.org/pdf/1006.4968.pdf Stefan Blochwitz, Marcus Martin, Marcus Martin, Carsten S. Wehn. Statistical Approaches to PD Validation (2006). https://doi.org/10.1007/3-540-33087-9_13 Luisa Izzi, Gianluca Oricchio, Laura Vitale. Validation of Internal Credit Models. (2012) https://doi.org/10.1057/9780230361188_6
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.