Assess Prime and Subprime Segmentation in Loan Data
Article Quant Q&A · Author: Revolucion for Monica
Summary
This question concerns whether borrower loans in a Lending Club dataset can be segmented into prime and subprime categories using the available grade and sub-grade fields. The dataset excerpt includes many borrower and loan characteristics, alongside the lender-assigned grade and sub-grade. The author considers using k-means on selected features to see whether clusters correspond to those labels, but is unsure how to apply the method beyond a single dimension.
The document does not provide an answer, a definition of prime versus subprime, a proposed feature set, or any clustering results. Consequently, it offers no evidence that unsupervised clusters would reproduce the lender’s grades. The central analytical issue is that grade labels and clusters are different constructs: validating a segmentation would require defining the target categories and choosing an evaluation approach, while also handling feature scaling and potential leakage from existing grade fields. Those methodological steps are not developed in the source, so the note is useful mainly for framing the classification question.
Key ideas
- The dataset includes lender-assigned loan grades and sub-grades alongside borrower characteristics.
- The author asks whether loans can be grouped as prime and subprime from those features.
- K-means is raised as a possible way to discover clusters, but no clustering is performed.
- The document provides no criteria for defining prime status or validating agreement with existing grades.
Tags
Full text
# How to check if the people segmentation based on prime and sub prime loans is accurate? # How to check if the people segmentation based on prime and sub prime loans is accurate? Using Lending Club dataset I have a dataframe with characteristics of loans of some borrowers. Here is their distribution of the sub-grades: ``` >>>plt.figure(figsize=(16, 6)) >>>sns.countplot(x="sub_grade", data=result) ``` I wonder if I can easily segment people based on prime and sub prime loans (and are "prime" and "sub prime" synonyms of "grades" and "sub-grades" ?) Here is an excerpt of the dataset : ``` funded_amnt_inv term int_rate installment grade sub_grade emp_title emp_length home_ownership annual_inc verification_status issue_d loan_status pymnt_plan url desc purpose title zip_code addr_state dti delinq_2yrs earliest_cr_line inq_last_6mths mths_since_last_delinq mths_since_last_record open_acc pub_rec revol_bal revol_util total_acc initial_list_status out_prncp out_prncp_inv total_pymnt total_pymnt_inv total_rec_prncp total_rec_int total_rec_late_fee recoveries collection_recovery_fee last_pymnt_d last_pymnt_amnt next_pymnt_d last_credit_pull_d collections_12_mths_ex_med mths_since_last_major_derog policy_code application_type annual_inc_joint dti_joint verification_status_joint acc_now_delinq tot_coll_amt tot_cur_bal open_acc_6m open_act_il open_il_12m open_il_24m mths_since_rcnt_il total_bal_il il_util open_rv_12m open_rv_24m max_bal_bc all_util total_rev_hi_lim inq_fi total_cu_tl inq_last_12m acc_open_past_24mths avg_cur_bal bc_open_to_buy bc_util chargeoff_within_12_mths delinq_amnt mo_sin_old_il_acct mo_sin_old_rev_tl_op mo_sin_rcnt_rev_tl_op mo_sin_rcnt_tl mort_acc mths_since_recent_bc mths_since_recent_bc_dlq mths_since_recent_inq mths_since_recent_revol_delinq num_accts_ever_120_pd num_actv_bc_tl num_actv_rev_tl num_bc_sats num_bc_tl num_il_tl num_op_rev_tl num_rev_accts num_rev_tl_bal_gt_0 num_sats num_tl_120dpd_2m num_tl_30dpd num_tl_90g_dpd_24m num_tl_op_past_12m pct_tl_nvr_dlq percent_bc_gt_75 pub_rec_bankruptcies tax_liens tot_hi_cred_lim total_bal_ex_mort total_bc_limit total_il_high_credit_limit revol_bal_joint sec_app_earliest_cr_line sec_app_inq_last_6mths sec_app_mort_acc sec_app_open_acc sec_app_revol_util sec_app_open_act_il sec_app_num_rev_accts sec_app_chargeoff_within_12_mths sec_app_collections_12_mths_ex_med sec_app_mths_since_last_major_derog hardship_flag hardship_type hardship_reason hardship_status deferral_term hardship_amount hardship_start_date hardship_end_date payment_plan_start_date hardship_length hardship_dpd hardship_loan_status orig_projected_additional_accrued_interest hardship_payoff_balance_amount hardship_last_payment_amount disbursement_method debt_settlement_flag debt_settlement_flag_date settlement_status settlement_date settlement_amount settlement_percentage settlement_term 749995 7605 NaN NaN 15000 15000 15000.0 36 months 13.56 509.47 C C1 Driver 4 years RENT 55000.0 Not Verified 2018-12-01 Current n NaN NaN debt_consolidation Debt consolidation 171xx PA 17.15 0.0 Oct-2012 0.0 NaN NaN 9.0 0.0 7277 40.0 14.0 w 14316.22 14316.22 1001.99 1001.99 683.78 318.21 0.0 0.0 0.0 Feb-2019 509.47 Mar-2019 Feb-2019 0.0 NaN 1 Individual NaN NaN NaN 0.0 0.0 21229.0 0.0 2.0 1.0 3.0 8.0 13952.0 85.0 1.0 1.0 4778.0 61.0 18200.0 1.0 0.0 2.0 4.0 2359.0 2457.0 69.7 0.0 0.0 74.0 54.0 12.0 8.0 0.0 35.0 NaN 8.0 NaN 0.0 2.0 5.0 2.0 2.0 4.0 7.0 10.0 5.0 9.0 0.0 0.0 0.0 2.0 100.0 50.0 0.0 0.0 34632.0 21229.0 8100.0 16432.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN N NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Cash N NaN NaN NaN NaN NaN NaN 749996 35399 NaN NaN 8000 8000 8000.0 36 months 6.46 245.05 A A1 Realtor 2 years MORTGAGE 100000.0 Not Verified 2018-12-01 Current n NaN NaN debt_consolidation Debt consolidation 231xx VA 9.90 1.0 Sep-2001 0.0 21.0 NaN 13.0 0.0 17646 51.7 29.0 w 7389.00 7389.00 692.18 692.18 611.00 81.18 0.0 0.0 0.0 Feb-2019 245.05 Mar-2019 Feb-2019 0.0 NaN 1 Individual NaN NaN NaN 0.0 3323.0 326631.0 1.0 2.0 0.0 0.0 31.0 13286.0 57.0 2.0 3.0 2173.0 54.0 34100.0 0.0 3.0 0.0 3.0 27219.0 12434.0 26.0 0.0 0.0 82.0 206.0 2.0 2.0 2.0 2.0 63.0 17.0 21.0 0.0 4.0 5.0 5.0 11.0 3.0 10.0 24.0 5.0 13.0 0.0 0.0 0.0 2.0 82.8 40.0 0.0 0.0 381832.0 30932.0 16800.0 23370.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN N NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN DirectPay N NaN NaN NaN NaN NaN NaN 749997 8452 NaN NaN 40000 40000 40000.0 60 months 16.14 975.71 C C4 Director of Operations 3 years MORTGAGE 157630.0 Verified 2018-12-01 Current n NaN NaN credit_card Credit card refinancing 972xx OR 16.61 1.0 May-2004 0.0 18.0 NaN 11.0 0.0 15761 56.7 21.0 w 39562.28 39562.28 1155.05 1155.05 437.72 717.33 0.0 0.0 0.0 Feb-2019 1208.85 Mar-2019 Feb-2019 0.0 NaN 1 Individual NaN NaN NaN 0.0 0.0 474781.0 1.0 4.0 1.0 1.0 5.0 59310.0 38.0 0.0 0.0 9029.0 43.0 27800.0 0.0 5.0 0.0 1.0 43162.0 9453.0 59.8 0.0 0.0 71.0 175.0 27.0 5.0 2.0 27.0 NaN NaN NaN 0.0 3.0 4.0 3.0 4.0 6.0 6.0 13.0 4.0 11.0 0.0 0.0 0.0 1.0 95.2 66.7 0.0 0.0 542479.0 75071.0 23500.0 100389.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN N NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Cash N NaN NaN NaN NaN NaN NaN ... ``` I was wondering if I should use a k-mean on some features with the same number of sub-grades and see if it returns the same `sub_grade` classes. Yet I only know how to do k-means on one dimension. I use python3
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.