Consumer Credit Scoring: Data Checks, Logistic Models, and Expert Judgment

# Expert System for Credit Scoring


# Expert System for Credit Scoring












I am working on a credit scoring model for a credit without a garanty (consumer credit).

I'm looking for a methodology for an expert system for credit scoring decision that doesn't base on statistical methods. I tried several statistical methods on my data (logistic regression, random forest) - without sucess. My variables are not predictive enough and predictions are too poor to build a scorecard.

I am modeling a binary variable with levels 0 and 1 where 1 is the default and I have several predictive variables.

Now I'm looking for a method that combines the information of the data with an expert system (human knowledge)? What can I do?

Does a method/algorithm like this exists? What about Apriori?

## Answer by Quantopik (score 6, accepted)

https://quant.stackexchange.com/a/18819

The best model used by practitioners to model consumer credit is the logistic regression model; now, you said that you tried modelling by using logistic model without success, but I want trying to suggest you the step-by-step procedure I would use in such case.

To model consumer credit, I'd follow the following procedure:

- Sample Analysis:

I analyze the distribution of such sample, verifying that it has approximately the same distribution of the population or benchmark. The sample size should be adequate (at least 50k observations over at least 5/7 years observations, according to what is suggested by Basel agreement).

In such step, you can perform this kind of analysis by using the Population Stability Index, given, for each variable, by:

$PSI$ $=$ $\sum\limits_{i=1}^n (P_{i,1}-P_{i,2})*ln\frac{P_{i,1}}{P_{i,2}}$

where $P_{i,1}$ is the percentage of observations in the j-class of the population/benchmark you considered and $P_{i,2}$ is the the percentage of observations in the j-class of the sample.

Compute the PSI for each variable in your sample, by verifying that the PSI is less than 10%; the threshold is empirical and depends on the country/instrument you're analyzing.

Eliminate from the sample variables that are not stable on the basis of the PSI.

- Data Quality Analysis:

analyze the sample values pinpointing the percentage of missing values, anomalies in the values (as, for instance, value 999.999) and error values according to the economic sense (as, for instance, negative values for variables that should be necessarily positive, like the loan value). I'd eliminate all variables with missings, anomalies or error values with a percentage greater than 5%.

- Logistic Regression Analysis

Implement a logistic by regressing the target variable on the independent ones, by eliminating the variables that are not significative at the 5% level; eliminate the variables with correlation greater than 30% (again, this is an empirical rule, so, use that as rule of thumbs and not as threshold point). After that, run a stepwise variable selection model, by keeping the significant variables only.

Now, this is the basic step-by-step procedure I'd use to model consumer credit market. Following this one, you should get an AUROC about equal to 80% (70% on the test/validation set). If it is not, you could calibrate/adjust your model by introducing expert/judgemental variables in. This means asking your colleagues to define what variables, according to them, influences the probability of default and adjust the weight the variables have in the model by capping them; this is the simpler way. Secondly, you could develop a parallel model constructed on judgemental way entirely (for instance, make a questionnaire about the variables could affect the PD and ask to several practitioners to compile that and, finally, take the more significative variables.

If you're not a practitioner, the following ones are the most common variables affecting the PD:

- Loan/Credit amount;

- Debt-To-Income ratio;

- Age;

- Job;

- Wage/salary amount;

- Credit Bureau score;

- Past consumer hystory (if the customer bought something in the past and was be able to pay back or not the debt);

and so on... Unfortunately, it does not exist an algorithm to develop expert-based credit risk model. Indeed, the expert judge is based on the experience of the bank teller/manager and such experience changes according to the country in which the employer. The consumer credit market changes according to a lot of things, such as the country culture, the rules, the law, the way the Central Banks applies Basel agreement, and so on. It is not possible to adapt a model build to perform in UK to a Latin country, like Spain, Italy, etc

Hope this helps.