Bank Credit Default Case Study¶
Exploratory Data Analysis¶
Business Understanding :¶
The loan providing companies find it hard to give loans to the people due to their insufficient or non-existent credit history. Because of that, some consumers use it as their advantage by becoming a defaulter. Data is collected about a hypothetical finance bank which specialises in lending various types of loans to urban customers. EDA is used to analyse the patterns present in the data. The bank can use this data to ensure that the applicants capable of repaying the loan are not rejected.
When the bank receives a loan application, the bank has to decide for loan approval based on the applicant’s profile. Two types of risks are associated with the bank’s decision:
- If the applicant is likely to repay the loan, then not approving the loan results in a loss of business to the bank
- If the applicant is not likely to repay the loan, i.e. he/she is likely to default, then approving the loan may lead to a financial loss for the bank.
The data contains the information about the loan application at the time of applying for the loan. It contains two types of scenarios:
- The client with payment difficulties: he/she had late payment more than X days on at least one of the first Y instalments of the loan in our sample,
- All other cases: All other cases when the payment is paid on time.
When a client applies for a loan, there are four types of decisions that could be taken by the client/bank):
- Approved: The bank has approved loan Application
- Cancelled: The client cancelled the application sometime during approval. Either the client changed her/his mind about the loan or in some cases due to a higher risk of the client he received worse pricing which he did not want.
- Refused: The bank had rejected the loan (because the client does not meet their requirements etc.).
- Unused offer: Loan has been cancelled by the client but on different stages of the process.
This case study uses EDA techniques to understand how consumer attributes and loan attributes influence the tendency of default.
Business Objectives¶
This case study aims to identify patterns which indicate if a client has difficulty paying their installments which may be used for taking actions such as denying the loan, reducing the amount of loan, lending (to risky applicants) at a higher interest rate, etc. This will ensure that the consumers capable of repaying the loan are not rejected.
In other words, we have tried to understand the driving factors (or driver variables) behind loan default, i.e. the variables which are strong indicators of default. The bank can utilise this knowledge for its portfolio and risk assessment.
Data Understanding¶
- 'application_data.csv' contains all the information of the client at the time of application. The data is about whether a client has payment difficulties.
- 'previous_application.csv' contains information about the client’s previous loan data. It contains the data whether the previous application had been Approved, Cancelled, Refused or Unused offer.
Analysis Approach & Conclusions:¶
- The purpose of this analysis is to give insights on credit default analysis and suggest customer parameters which have a clear indication of default.
- The Analysis is done on current loan application data and previous loan application status data.
- The current loan application is used to explore for insights on the factors that contribute to payment difficulties / loan default.
- A merged data set of current loan application and previous application data is analysed to give insights on factors that contribute to Loan Approval, Rejection, Cancellations and unused offers.
- The risk of default could broadly divided into
- Risk attributed to Credit Good [Credit Good Risk]
- Risk attributed to Consumer liabilities / behaviour [Consumer related Risk]
- Loan application is approved based on below examination,
- If applicant is new customer, then we look at Credibility and Demographics
- If applicant is old customer, then we look at Credit history and Previous rejection causes.
The following impressions have been drawn from the analysis
Univariate Analysis¶
Gender
- There is a very high proportion of Female applicants (65.8%)
- Only 7% of total female applicants have payment difficulties compared to 10% of total male applications.
- It is possible that female applicants are safer borrowers than male.
Owning Realty
- Applicants who do not own Real-estate seem more likely to face payment difficulties than compared to the group of people owning the Real-estate.
Contact Address vs Work Address
- Applicants that live and work in same city are at less risk of payment difficulties.
Education
- Higher the education, lesser is the risk of payment difficulties.
Marital Status
- Applicants with FAMILY_STATUS as seperated or single or have higher risk of payment difficulties compared to that of Widow and Married.
- For the purpose of this analysis, civil marriage and married are treated equal. They might be merged into the one category.
Number of Days Employed
- The applicants with payment difficulties have lesser median days of employment compared to "all other cases". So, greater the DAYS_EMPLOYED, lesser is the risk of payment difficulties.
Bivariate Analysis¶
Credit Amount and Price of Credit Goods
- Lower Credit Amount and Lower Price of Credit Goods have higher cases of Payment difficulties
Gender & Marital Status
- Customers most attractive to the bank by likelihood of default, are 'Female Widows' > 'Married Females' = 'Separated Females' > 'Single Females' > 'Married Males' > 'Male Widowers' > 'Single Males' = 'Separated Males'
- The bank may focus their marketing campaigns on converting applications of Female Widows, Married Females, Separated Females.
- May note that the dataset is skewed towards Females than Males.
Goods Price
- Median of "Approved" Category is 95,208.48. Any amount above this has high risk of getting refused or being cancelled.
Income and Annuity
- Income Range and Annuity have a mild correlation with Payment difficulties
Education & Client Type
- Bank should look into the reasons why Repeat customers who are cancelling their offers. It could also focus on loans catered to academic degree holders who are first time customers.
Top 10 correlations for 'No default'¶
- AMT_GOODS_PRICE & AMT_CREDIT
- AMT_GOODS_PRICE & AMT_ANNUITY
- AMT_ANNUITY & AMT_CREDIT
- AGE_YEARS & DAYS_EMPLOYED
- AMT_ANNUITY & AMT_INCOME_TOTAL
- AMT_GOODS_PRICE & AMT_INCOME_TOTAL
- AMT_INCOME_TOTAL & AMT_CREDIT
- AGE_YEARS & EXT_SOURCE_3
- DAYS_LAST_PHONE_CHANGE & EXT_SOURCE_2
- DAYS_EMPLOYED & AMT_INCOME_TOTAL
Top 10 correlations for 'Default' are :¶
- AMT_GOODS_PRICE & AMT_CREDIT
- AMT_CREDIT & AMT_ANNUITY
- AMT_ANNUITY & AMT_GOODS_PRICE
- AGE_YEARS & DAYS_EMPLOYED
- EXT_SOURCE_2 & DAYS_LAST_PHONE_CHANGE
- EXT_SOURCE_3 & AGE_YEARS
- AGE_YEARS & AMT_GOODS_PRICE
- AGE_YEARS & AMT_CREDIT
- AMT_GOODS_PRICE & EXT_SOURCE_2
- AMT_CREDIT & EXT_SOURCE_2
Look below for a detailed analysis¶
In [98]:
#importing required Libraries.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
import sidetable
from tabulate import tabulate
import warnings
warnings.filterwarnings('ignore')
In [99]:
#setting max number of columns to be displayed to 100, to get a better view.
pd.set_option('display.max_columns',100)
pd.set_option('display.max_rows',100)
from IPython.display import HTML
HTML('''<script>
code_show_err=false;
function code_toggle_err() {
if (code_show_err){
$('div.output_stderr').hide();
} else {
$('div.output_stderr').show();
}
code_show_err = !code_show_err
}
$( document ).ready(code_toggle_err);
</script>
''')
Out[99]:
Loading Application Data¶
In [100]:
application_data=pd.read_csv('application_data.csv')
In [101]:
#checking shape
application_data.shape
Out[101]:
In [102]:
application_data.info()
In [103]:
application_data.describe()
Out[103]:
In [104]:
def color_red(value):
'''
Colors elements in a dateframe
green if nulls <50% and red if
>=50%.
'''
if value >=50:
color = 'red'
elif value <50:
color = 'green'
return 'color: %s' % color
#Finding Percentage of Missing values in each and every column
missing_data = np.round(100*application_data.isnull().sum()/len(application_data),2)
missing_data = pd.DataFrame(data={'Column Name' :missing_data.index, 'Null Percentage' : missing_data.values})
missing_data.style.applymap(color_red,subset=['Null Percentage'])
Out[104]:
In [105]:
#Finding the columns that have more than or equal to 50% null values and storing it to columns_to_drop.
columns_to_drop=application_data.columns[100*application_data.isnull().sum()/len(application_data)>=50]
columns_to_drop
Out[105]:
In [106]:
#dropping the columns where the null values are >= 50%.
application_data.drop(labels=columns_to_drop,axis=1,inplace=True)
In [107]:
#verifying if the columns are dropped.
application_data.shape
Out[107]:
Imputation Consideration¶
Finding the best values to impute below columns that have <=13% of null values¶
In [108]:
#Finding columns with <= 13% missing columns
null_info = pd.DataFrame(100*application_data.isnull().sum()/len(application_data))
null_info.columns = ['Null Percentage']
null_info[(null_info['Null Percentage'] > 0) & (null_info['Null Percentage'] <=13)]
Out[108]:
Columns To Impute¶
We have chosen the following columns to consider the best value to impute the missing values with
- AMT_ANNUITY
- AMT_GOODS_PRICE
- NAME_TYPE_SUITE
- CNT_FAM_MEMBERS
- OBS_60_CNT_SOCIAL_CIRCLE
AMT_ANNUITY Imputation¶
In [109]:
# Boxplot to check for outliers
application_data['AMT_ANNUITY'].plot.box()
plt.title('\n Box Plot of AMT_GOODS_PRICE')
# Calculating Quantiles
print('Quantile\tAMT_ANNUITY')
application_data['AMT_ANNUITY'].quantile([0.5,0.8,0.85,0.90,0.95,1])
Out[109]:
From the above box plot of AMT_ANNUITY, there are a lot of outliers.
Calculating Quantiles confirms the same. There is a huge jump from 95% value to max value .
Hence, Median : 24903 is the best value to impute the missing values in AMT_ANNUITY, since mean is not robust to outliers.
AMT_GOODS_PRICE Imputation¶
In [110]:
print('Quantile\tAMT_GOODS_PRICE')
application_data['AMT_GOODS_PRICE'].plot.box()
plt.title('\n Box Plot of AMT_GOODS_PRICE')
application_data['AMT_GOODS_PRICE'].quantile([0.5,0.8,0.85,0.90,0.95,1])
Out[110]:
The above box plot show significant number of outliers. This confirmed by the quantiles.
There is a huge jump from 95 percentile value to Maximum value.
In this case, since there are many outliers, Mean is not a good representation of the data since it is not robust to outliers.
Hence ,Median : 450000 is the best metric to impute missing values
NAME_TYPE_SUITE Imputation :¶
In [111]:
print('Data type of NAME_TYPE_SUITE : ',application_data['NAME_TYPE_SUITE'].dtype,'\n\n')
print('Category\tNormalized Count\n\n',application_data['NAME_TYPE_SUITE'].value_counts(normalize=True))
data = application_data['NAME_TYPE_SUITE'].value_counts(normalize=True)
plt.bar(data.index,data.values)
# data.hist()
plt.xticks(rotation=90)
plt.ylabel('Normalized Value Counts')
plt.title('\nNAME_TYPE_SUITE');
NAME_TYPE_SUITE is a categorical variable.
The best metric to impute missing values is Mode of the data.
From the above plot, 'Unaccompanied' is the Mode
Hence, Unaccompanied is the best value to impute the missing values.
CNT_FAM_MEMBERS Imputation¶
In [112]:
application_data['CNT_FAM_MEMBERS'] = application_data['CNT_FAM_MEMBERS'].astype('category')
print('Data type of CNT_FAM_MEMBERS : ',application_data['CNT_FAM_MEMBERS'].dtype,'\n\n')
print('Fly Mems | Value Counts\n',application_data['CNT_FAM_MEMBERS'].value_counts(normalize=True))
(application_data['CNT_FAM_MEMBERS'].value_counts()).sort_index().plot(kind='bar')
plt.title('\nNo of Family Members vs Value Counts');
EXT_SOURCE_2 Imputation¶
In [113]:
print('Data type of EXT_SOURCE_2 : ',application_data['EXT_SOURCE_2'].dtype,'\n\n')
application_data['EXT_SOURCE_2'].plot.box()
plt.title('\nEXT_SOURCE_2');
print('Quantile\tValue')
application_data['EXT_SOURCE_2'].quantile([0.5,0.8,0.85,0.90,0.95,1])
Out[113]:
In [114]:
round(application_data['EXT_SOURCE_2'].mean(),2)
Out[114]:
EXT_SOURCE_2 is the credit rating by an external source. It is a numerical continuous variable. The above bar plot and the quantiles show no outliers. In this case Mean and Median are very close. Because there are no outliers, Mean : 0.51 could be used to impute missing values.
Checking for Disguised Missing Values¶
In [115]:
def cat_value_counts(column_name) :
print(tabulate(pd.DataFrame(application_data.stb.freq([column_name])), headers='keys', tablefmt='psql'))
print(pd.DataFrame(application_data[column_name]).stb.missing(),'\n\n\n')
In [116]:
#Replacing XAN with np.nan in Gender column :
application_data['CODE_GENDER'] = application_data['CODE_GENDER'].replace('XNA',np.nan)
cat_value_counts('CODE_GENDER')
Since CODE_GENDER is a categorical variable, the missing values could be imputed with Mode : F
In [117]:
# replacing Unknown in NAME_FAMILY_STATUS with np.nan
application_data['NAME_FAMILY_STATUS'] = application_data['NAME_FAMILY_STATUS'].replace('Unknown',np.nan)
cat_value_counts('NAME_FAMILY_STATUS')
Since NAME_FAMILY_STATUS is a categorical variable, the missing values could be imputed with Mode : Married
In [118]:
# replacing XNA values in ORGANIZATION_TYPE
application_data['ORGANIZATION_TYPE'] = application_data['ORGANIZATION_TYPE'].replace('XNA',np.nan)
cat_value_counts('ORGANIZATION_TYPE')
Since ORGANIZATION_TYPE is a categorical variable, the missing values could be imputed with Mode : Business Entity Type 3
Data Type Checks¶
In [119]:
pd.DataFrame(application_data.dtypes)
Out[119]:
In [120]:
application_data[['DAYS_BIRTH','DAYS_EMPLOYED','DAYS_REGISTRATION','DAYS_ID_PUBLISH']].head()
Out[120]:
The DAYS_BIRTH,DAYS_EMPLOYED,DAYS_REGISTRATION,DAYS_ID_PUBLISH have negative values, since they are in past with respect to date of application.
To standard size these columns for the purpose of analysis, we could converted them to their absolute values
In [121]:
columns_to_convert = ['DAYS_BIRTH','DAYS_EMPLOYED','DAYS_REGISTRATION','DAYS_ID_PUBLISH']
application_data[columns_to_convert] = application_data[columns_to_convert].abs()
application_data[columns_to_convert].head()
Out[121]:
In [122]:
# checking columns with binary values
values_per_column = application_data.nunique().sort_values()
col_values_dtype = pd.DataFrame(index=values_per_column.index, data= {'Unique Values' : values_per_column.values, 'Data Type' : application_data.dtypes})
col_values_dtype
Out[122]:
The above columns are 'categorical' variables with only two values. But they have been read as 'int' datatype
We could convert them into 'categorical' data type
In [123]:
# converting to category data type
convert_to_cat = col_values_dtype[col_values_dtype['Unique Values']<=8].index
application_data[convert_to_cat] = application_data[convert_to_cat].astype('category')
In [124]:
# check if the columns are converted
values_per_column = application_data.nunique().sort_values()
new_categories = pd.DataFrame(index=values_per_column.index, data= {'Unique Values' : values_per_column.values, 'Data Type' : application_data.dtypes})
new_categories
Out[124]:
Checking for Outliers¶
Checking for outliers in the following numerical columns¶
- AMT_INCOME_TOTAL
- AMT_CREDIT
- AMT_ANNUITY
- AMT_GOODS_PRICE
- DAYS_BIRTH
In [125]:
# Adding a new column "AGE_YEARS" using 'DAYS_BIRTH' with age in years
def days_to_years(x) :
if x < 0 :
x = -1*x
return x//365
application_data['AGE_YEARS'] = application_data['DAYS_BIRTH'].apply(days_to_years)
application_data['AGE_YEARS'].describe()
Out[125]:
In [126]:
# Box plots of the above numerical variables
outlier_check_col = ["AMT_INCOME_TOTAL","AMT_CREDIT","AMT_ANNUITY","AMT_GOODS_PRICE","DAYS_BIRTH"]
fig,ax = plt.subplots(3,2)
fig.set_figheight(15)
fig.set_figwidth(15)
ax[0,0].set_yscale('log')
ax[0,0].set(ylabel ='Income in Log Scale')
application_data[outlier_check_col[0]].plot.box(ax=ax[0,0],);
application_data[outlier_check_col[1]].plot.box(ax=ax[0,1]);
application_data[outlier_check_col[2]].plot.box(ax=ax[1,0]);
application_data[outlier_check_col[3]].plot.box(ax=ax[1,1]);
ax[2,0].set(ylabel ='Age In Days')
application_data[outlier_check_col[4]].plot.box(ax=ax[2,0]);
ax[2,1].axis('off')
print('Box Plots of "AMT_INCOME_TOTAL","AMT_CREDIT","AMT_ANNUITY","AMT_GOODS_PRICE","DAYS_BIRTH" \n')
In [127]:
## Calculating quantiles for the above columns
pd.options.display.float_format = '{:,.2f}'.format
for col in outlier_check_col :
print(col,'\n',application_data[col].quantile([0.5,0.8,0.85,0.90,0.95,1]),'\n\n')
Outliers in Numerical Columns¶
From the above box plots and quantile calculations, we see that
AMT_INCOME_TOTAL has huge jump from 95th Percentile (337,500.00) to Maximum value (117,000,000.00) {346 times 95th Percentile}.
Apart from this jump, there are several values > 1.5 times IQR. All of these are deemed outliers.
To avoid skewed results of analysis, values greater than 95th percentile value may be capped to 95th Percentile value (337,500.00)
AMT_CREDIT also has a huge jump from 95th Percentile (1,350,000.00) to Maximum value (4,050,000.00) {3 times 95th percentile}.
Apart from this jump, there are several values > 1.5 times IQR. All of these are deemed outliers.
To avoid skewed results of analysis, values greater than 95th percentile value may be capped to 95th Percentile value (1,350,000.00)
Similarly , AMT_ANNUITY, AMT_ANNUITY have outliers and they could be capped to 95th percentile values.
DAYS_BIRTH has no outliers
Binning Continuous Variables¶
In [128]:
# AMT_INCOME_TOTAL
min_income = int(application_data['AMT_INCOME_TOTAL'].min())
max_income = int(application_data['AMT_INCOME_TOTAL'].max())
bins = [0,25000,50000,75000,100000,125000,150000,175000,200000,225000,250000,275000,300000,325000,350000,375000,400000,425000,450000,475000,500000,10000000000]
intervals = ['0-25000', '25000-50000','50000-75000','75000-100000','100000-125000', '125000-150000', '150000-175000','175000-200000',
'200000-225000','225000-250000','250000-275000','275000-300000','300000-325000','325000-350000','350000-375000',
'375000-400000','400000-425000','425000-450000','450000-475000','475000-500000','500000 and above']
application_data['AMT_INCOME_CAT']=pd.cut(application_data['AMT_INCOME_TOTAL'],bins,labels=intervals)
print('Income Range\t Count')
print(application_data['AMT_INCOME_CAT'].value_counts())
income_cat = application_data['AMT_INCOME_CAT'].value_counts()
plt.hist(application_data['AMT_INCOME_CAT'])
plt.title('\n Income Range vs No of Applications')
plt.xticks(rotation=90);
In [129]:
#AMT_CREDIT
bins = [0,150000,200000,250000,300000,350000,400000,450000,500000,550000,600000,650000,700000,750000,800000,850000,900000,1000000000]
intervals = ['0-150000', '150000-200000','200000-250000', '250000-300000', '300000-350000', '350000-400000','400000-450000',
'450000-500000','500000-550000','550000-600000','600000-650000','650000-700000','700000-750000','750000-800000',
'800000-850000','850000-900000','900000 and above']
application_data['AMT_CREDIT_RANGE']=pd.cut(application_data['AMT_CREDIT'],bins=bins,labels=intervals)
application_data['AMT_CREDIT_RANGE'] = application_data['AMT_CREDIT_RANGE'].astype('category')
print('Credit Range\t Count')
credit_range = application_data['AMT_CREDIT_RANGE'].value_counts()
print(credit_range)
plt.hist(application_data['AMT_CREDIT_RANGE'])
plt.xticks(rotation=90)
Out[129]:
Data Imbalance¶
In [130]:
# Target Variable - 1: Client with Payment difficulties, 0 : All other cases
cat_value_counts('TARGET')
From the above, you can see that this data set contains 91.9% of records about Clients with Payment difficulties and 0.08% records about all other cases.
Ratio of classes = .919/0.08 = 11.375
This represents an huge Data Imbalance
The dataset is skewed towards 'Clients with Payment difficulties'
Data Set division wrt TARGET¶
In [131]:
application_data0=application_data.loc[application_data["TARGET"]==0]
application_data1=application_data.loc[application_data["TARGET"]==1]
Columns Selection for Analysis¶
In [132]:
columnsForAnalysis = ['SK_ID_CURR', 'TARGET', 'NAME_CONTRACT_TYPE', 'CODE_GENDER',
'FLAG_OWN_CAR', 'FLAG_OWN_REALTY', 'CNT_CHILDREN', 'AMT_INCOME_TOTAL',
'AMT_CREDIT', 'AMT_ANNUITY', 'AMT_GOODS_PRICE',
'NAME_INCOME_TYPE', 'NAME_EDUCATION_TYPE', 'NAME_FAMILY_STATUS',
'NAME_HOUSING_TYPE',
'DAYS_EMPLOYED','FLAG_MOBIL', 'FLAG_CONT_MOBILE',
'FLAG_EMAIL', 'OCCUPATION_TYPE', 'CNT_FAM_MEMBERS',
'REGION_RATING_CLIENT_W_CITY',
'REG_CITY_NOT_LIVE_CITY',
'REG_CITY_NOT_WORK_CITY', 'LIVE_CITY_NOT_WORK_CITY',
'ORGANIZATION_TYPE', 'EXT_SOURCE_2', 'EXT_SOURCE_3',
'DAYS_LAST_PHONE_CHANGE' ,'AGE_YEARS', 'AMT_INCOME_CAT',
'AMT_CREDIT_RANGE']
Analysis¶
Univariate Analysis¶
In [133]:
# LOAN DATA
In [134]:
# function for categorical variable univariate analysis
def cat_univariate_analysis(column_name,figsize=(10,5)) :
# print unique values
print('TARGET 0\n', application_data0[column_name].unique(),'\n')
print('TARGET 1\n',application_data1[column_name].unique(),'\n')
# column vs target count plot
plt.figure(figsize=figsize)
ax = sns.countplot(x=column_name,hue='TARGET',data=application_data)
title = column_name + ' vs Number of Applications'
ax.set(title= title)
for p in ax.patches:
height = p.get_height()
ax.text(p.get_x()+p.get_width()/2,
height + 10,
format(height),
ha="center")
# Percentages
print('All Other Cases (TARGET : 0)')
print(tabulate(pd.DataFrame(application_data0.stb.freq([column_name])), headers='keys', tablefmt='psql'),'\n')
print('Clients with Payment Difficulties (TARGET : 1)')
print(tabulate(pd.DataFrame(application_data1.stb.freq([column_name])), headers='keys', tablefmt='psql'),'\n')
In [135]:
# NAME_CONTRACT_TYPE
cat_univariate_analysis('NAME_CONTRACT_TYPE')
Client's with All other cases (Target=0)¶
- Out of 'All other cases', 90.21% have taken 'Cash Loans' and 9.79% have taken 'Revolving Loans'
Client's with Payment Difficulties (Target=1)¶
- Out of all 'Client's with payment difficulties', 93.54% have taken 'Cash Loans' and 6.46% have taken 'Revolving Loans'
In [136]:
# function for numerical variable univariate analysis
def num_univariate_analysis(column_name,scale='linear') :
# boxplot for column vs target
plt.figure(figsize=(8,6))
ax = sns.boxplot(x='TARGET', y = column_name, data = application_data)
title = column_name+' vs Target'
ax.set(title=title)
if scale == 'log' :
plt.yscale('log')
ax.set(ylabel=column_name + '(Log Scale)')
# summary statistic
print('All Other Cases (TARGET : 0)')
print(application_data0[column_name].describe(),'\n')
print('Clients with Payment Difficulties (TARGET : 1)')
print(application_data1[column_name].describe())
In [137]:
# AMT_CREDIT
num_univariate_analysis('AMT_CREDIT')
Amount of credit¶
- The median amount of credit taken by Clients having payment difficulties' is almost same as 'All other cases'.
- You could notice that the IQR of Target 1 is less than Target 0, which implies that more proportion of client's with payment difficulties take credit amounts centered around the median amount than 'All other cases'
- Comparing the outliers of Target 1 and 0, we could see that Clients with payment difficulties favour lower amount of credit than all other cases. Conversly, people with higher amount of credit have are more probably to fall in the category 'All other cases' than 'Clients with Payment Difficulties'
In [138]:
#AMT_ANNUITY
num_univariate_analysis('AMT_ANNUITY','log')
Amount of Annuity¶
- There is not much difference in Amount of Annuity between Clients with Payment Difficulties and All other cases.
- The median and mean are around the same.
- However, Clients with Payment difficulties have annuity amounts which are less dispersed than All other cases.
In [139]:
#AMT_GOODS_PRICE
num_univariate_analysis('AMT_GOODS_PRICE')
Goods Price Amount¶
- With respect to
AMT_GOODS_PRICE, there is no significant difference between Clients with Payment difficulties and All another Cases.
In [140]:
#REGION_RATING_CLIENT_W_CITY
cat_univariate_analysis('REGION_RATING_CLIENT_W_CITY')
Bank City Rating¶
The bank has rated cities into three categories [1,2,3]
- Among 'All other cases' , 74.7% belong to category 2 cities followed by 13.7% in category 3 cities and 11.5% in category 1 cities.
- 73.1% of Clients with Payment difficulties belong to cities of category 2 followed by 20.1% belonging to category 3 and 6.6% in category 1
One can see that there are higher cases of Payment Difficulties in category 2 cities (20.1%) compared to All other cases (13.7%). Also , there are less cases of Payment difficulties in city category 1 (6.6%) compared to All other cases (11.5%)
In [141]:
#EXT_SOURCE_2
num_univariate_analysis('EXT_SOURCE_2')
Client Credit Rating By External Source - 2¶
- The median credit rating of clients with Payment difficulties (0.44) is less than those with all other cases (0.57)
- We can say that there is higher probability of client with less credit rating to have payment difficulties.
- Further, there are some outliers in 'All other cases'. These could be clients with no credit history and hence their credit ratings are marked as zero.
In [142]:
#EXT_SOURCE_3
num_univariate_analysis('EXT_SOURCE_3')
Client Credit Rating By External Source - 3¶
- The median credit rating of clients with Payment difficulties (0.38) is less than those with all other cases (0.55)
- We can say that there is higher probability of client with less credit rating to have payment difficulties.
In [143]:
#CODE_GENDER
cat_univariate_analysis('CODE_GENDER')
In [144]:
print(application_data['CODE_GENDER'].value_counts(),'\n')
print('Proportion of Females (Target 0) :', round(100*188278/202448,2))
print('Proportion of Females (Target 1) :', round(100*14170/202448,2))
In [145]:
print('Proportion of Males (Target 0) :', round(100*94404/105059,2))
print('Proportion of Males (Target 1) :', round(100*10655/105059,2))
Gender¶
- There are very high proportion of Female applicants (65.8%)
- Only 7% of total female applicants have payment difficulties compared to 10% of total male applications.
- It is possible that female applicants are safer borrowers than male.
In [146]:
def cat_proportions(column_name) :
values = application_data[column_name].unique()
values = values.to_numpy()
values.tolist()
data0 = application_data0[column_name].value_counts().to_dict()
data1 = application_data1[column_name].value_counts().to_dict()
data = application_data[column_name].value_counts().to_dict()
for i in values :
if i in data0 and i in data1 and i in data :
print('Proportion of '+ str(i) + ' in Target 0 : ', round(data0[i]*100/data[i],2))
print('Proportion of '+ str(i) + ' in Target 1 : ', round(data1[i]*100/data[i],2),'\n' )
In [147]:
# FLAG_OWN_CAR
cat_univariate_analysis('FLAG_OWN_CAR')
cat_proportions('FLAG_OWN_CAR')
In [148]:
application_data['FLAG_OWN_CAR'].value_counts(normalize=True)
Out[148]:
Owning A Car¶
- 34% of total customers own a car
- Among customers who own a car, 92.76% belong to 'All other cases' category and 7.24% have payment difficulties.
- Among customers with Payment difficulties, 30.5% own a car
In 'All Other Cases', 34.3% own a car
Customers having cars lower probability of having Payment Difficulties.
- Customers owning cars in 'Clients with Payment difficulties ' : 8.5%
- Customers owning cars in 'Clients with Payment difficulties ' 7.24
In [149]:
# FLAG_OWN_REALTY
cat_univariate_analysis('FLAG_OWN_REALTY')
cat_proportions('FLAG_OWN_REALTY')
Owning Realty¶
- In Overall population of clients, 69.36% of people own real estate.
- In that group of people owning the real estate, 7.96% of people are having payment difficulties and the rest 92.04% belong to 'other cases'.
- In Overall population of clients, 30.63% of people do not own Realestate.
- In that group of people not owning the real estate, 8.32% of people are having payment difficulties and the rest 91.68% belong to 'other cases'.
On closer comparision of above observations, we can infer that people who do not own Real-estate seem more likely to face payment difficulties than compared to the group of people owning the Real-estate.
In [150]:
#NAME_EDUCATION_TYPE
cat_univariate_analysis('NAME_EDUCATION_TYPE')
plt.xticks(rotation=90)
cat_proportions('NAME_EDUCATION_TYPE')
Highest Education¶
- 1.24% of the applicants are having education type of 'Lower secondary'
- Out of that 1.24%, 10.93% of them are likely to face Payment difficulties.
- 71% of the applicants are having education type of 'Secodary/seconday special' .
- Out of that 71%, 8.93% of them are likely to face Payment difficulties.
- 3.34% of the applicants are having education type of 'Incomplete higher'.
- Out of that 3.34%, 8.48% of them are likely to face Payment difficulties.
- 24.34% of the applicants are having education type of 'Higher education'.
- Out of that 24.34%, 5.36% of them are likely to face Payment difficulties.
- 0.05% of the applicants are having education type of 'Academic degree'.
- Out of that 0.05%, 1.83% of them are likely to face Payment difficulties.
We can clearly infer from the above observations that higher the education, lesser is the risk of payment difficulties.
In [151]:
#NAME_FAMILY_STATUS
cat_univariate_analysis('NAME_FAMILY_STATUS')
plt.xticks(rotation=90)
cat_proportions('NAME_FAMILY_STATUS')
Marital Status¶
- 5.2% of applicants have their family status as'Widow'
- Out of that 5.2%, 5.82% of them are likely to face Payment difficulties.
- 63.87% of applicants have their family status as'Married'
- Out of that 63.87%, 7.56% of them are likely to face Payment difficulties.
- 6.42% of applicants have their family status as'Seperated'
- Out of that 6.42%, 8.19% of them are likely to face Payment difficulties.
- 14.77% of applicants have their family status as'Single / not married'
- Out of that 14.77%, 9.81% of them are likely to face Payment difficulties.
- 9.68% of applicants have their family status as'Civil marriage'
- Out of that 9.68%, 9.94% of them are likely to face Payment difficulties.
Based on the above obervations, we can infer that the applicants with NAME_FAMILY_STATUS as seperated or single or have higher risk of payment difficulties compared to that of Widow and Married.
For the purpose of this analysis, civil marriage and married are treated equal. They might be merged into the one category
In [152]:
#NAME_HOUSING_TYPE
cat_univariate_analysis('NAME_HOUSING_TYPE',figsize=(10,5))
plt.xticks(rotation=90)
cat_proportions('NAME_HOUSING_TYPE')
Housing Type¶
- 0.85% of applicants have their housing type as'Office apartment'
- Out of that 0.85%, 6.57% of them are likely to face Payment difficulties.
- 88.73% of applicants have their housing type as'House/ apartment'
- Out of that 88.73%, 7.8% of them are likely to face Payment difficulties.
- 0.36% of applicants have their housing type as'Co-op apartment'
- Out of that 0.36%, 7.93% of them are likely to face Payment difficulties.
- 3.63% of applicants have their housing type as'Municipal apartment'
- Out of that 3.63%, 8.54% of them are likely to face Payment difficulties.
- 4.82% of applicants have their housing type as'with parents'
- Out of that 4.82%, 11.7% of them are likely to face Payment difficulties.
- 1.58% of applicants have their housing type as'Rented apartment'
- Out of that 1.58%, 12.31% of them are likely to face Payment difficulties.
Based on the above obervations, we can infer that the applicants with NAME_HOUSING_TYPE as 'Municipal apartment' or 'with parents' or 'Rented apartment ' have higher risk of payment difficulties compared to that of other housing types.
In [153]:
#AMT_INCOME_CAT
cat_univariate_analysis('AMT_INCOME_CAT',figsize=(20,5))
plt.xticks(rotation=90)
cat_proportions('AMT_INCOME_CAT')
In [154]:
# We will calculate the pecentage of "Clients with Payment difficulties" for every income category.
t0=application_data0['AMT_INCOME_CAT'].value_counts()
t1=application_data1['AMT_INCOME_CAT'].value_counts()
prop = 100*t1/(t1+t0)
print(tabulate(pd.DataFrame(prop), headers=['AMT_INCOME_CAT','PERCENTAGE'], tablefmt='psql'))
Income Category¶
- By the above table of 'AMT_INCOME_CAT' vs 'percentage of customers with payment difficulties' in that range, we can infer that as the income increases, applicant is less likely to face payment difficulties(except in exceptional cases, such as 450000 - 475000).
In [155]:
#CNT_CHILDREN
#Lets convert this into a categorical variable
application_data['CNT_CHILDREN']=application_data['CNT_CHILDREN'].astype('category')
application_data0['CNT_CHILDREN']=application_data0['CNT_CHILDREN'].astype('category')
application_data1['CNT_CHILDREN']=application_data1['CNT_CHILDREN'].astype('category')
In [156]:
#CNT_CHILDREN
cat_univariate_analysis('CNT_CHILDREN',figsize=(20,5))
column_name='CNT_CHILDREN'
cat_proportions('CNT_CHILDREN')
# values = application_data[column_name].unique()
# values=values.dropna()
# values = values.to_numpy()
# values.tolist()
# data0 = application_data0[column_name].value_counts().to_dict()
# data1 = application_data1[column_name].value_counts().to_dict()
# data = application_data[column_name].value_counts().to_dict()
# for i in values[:7]:
# print('Proportion of '+ str(i) + ' in Target 0 : ', round(data0[i]*100/data[i],2))
# print('Proportion of '+ str(i) + ' in Target 1 : ', round(data1[i]*100/data[i],2),'\n' )
In [157]:
# Calculating the pecentage of "Clients with Payment difficulties" for every income category.
t0=application_data0['CNT_CHILDREN'].value_counts()
t1=application_data1['CNT_CHILDREN'].value_counts()
prop = 100*t1/(t1+t0)
print(tabulate(pd.DataFrame(prop), headers=['No of Children','Percentage'], tablefmt='psql'))
Number of Children¶
- By looking at the plot and above table, we can clearly infer that starting from 0 to 6, applicants having 0 children have lowest risk of facing payment dificulties. With the increase in number of children, the risk of payment difficulties also increase.
- Above 6, there might be some data quality issues. We can even expect that this is error while taking the data from thr applicants, because the numericals above 6 are not practical in most of the cases.
- If the CNT_CHILDREN > 6 is true, then there is 0 probabilty that the applicant might face the payment difficulties.
In [158]:
#CNT_FAM_MEMBERS
cat_univariate_analysis('CNT_FAM_MEMBERS',figsize=(20,5))
cat_proportions('CNT_FAM_MEMBERS')
Number of Family members¶
- When we look at the plot and proportions above, we can clearly infer that increase in family members is directly proportional to increase in risk of facing payment difficulties.
- Above 6, we cant infer because there were not enough applicant samples.
In [159]:
#DAYS_EMPLOYED.
num_univariate_analysis('DAYS_EMPLOYED','log')
Number of Days Employed¶
- If we observe the above box plot, we can clearly notice the outliers, so lets concentrate on median.
- The applicants with payment difficulties have lesser median and IQR compared to "all other cases". So, greater the DAYS_EMPLOYED, lesser is the risk of payment difficulties.
In [160]:
#NAME_INCOME_TYPE
cat_univariate_analysis('NAME_INCOME_TYPE',figsize=(15,5))
plt.xticks(rotation=90)
cat_proportions('NAME_INCOME_TYPE')
Income type¶
- 52% of applicants have their NAME_INCOME_TYPE as'Working'.
- Out of that 52%, 9.59% of them are likely to face Payment difficulties.
- 23% of applicants have their NAME_INCOME_TYPE as'Commercial associate'
- Out of that 23%, 7.48% of them are likely to face Payment difficulties.
- 7% of applicants have their NAME_INCOME_TYPE as'State servant'
- Out of that 7%, 5.75% of them are likely to face Payment difficulties.
- 18% of applicants have their NAME_INCOME_TYPE as'Pensioner'
- Out of that 18%, 5.39% of them are likely to face Payment difficulties.
From the above Graph and proportions, we can infer that Pensioners and state servants have lower risk of facing payment difficulties compared to the Working and Commercial associates.
Unemployed,Student,Businessman and Maternity leave cannot be inferred because of the reason that they are not well populated.
In [161]:
#DAYS_LAST_PHONE_CHANGE
application_data['DAYS_LAST_PHONE_CHANGE'] = np.abs(application_data['DAYS_LAST_PHONE_CHANGE'])
application_data0['DAYS_LAST_PHONE_CHANGE'] = np.abs(application_data0['DAYS_LAST_PHONE_CHANGE'])
application_data1['DAYS_LAST_PHONE_CHANGE'] = np.abs(application_data1['DAYS_LAST_PHONE_CHANGE'])
num_univariate_analysis('DAYS_LAST_PHONE_CHANGE')
Days Since Phone Number Was Changed.¶
- From the above boxplot, we can clearly notice that the median for 'all other cases' is 776 days and the median for 'Clients with payment difficulties' is 594 days.
- By this, we can clearly infer that,lesser the days since last phone change, greater is the risk of payment difficulties
In [162]:
#FLAG_CONT_MOBILE
cat_univariate_analysis('FLAG_CONT_MOBILE',figsize=(15,5))
cat_proportions('FLAG_CONT_MOBILE')
Flag Whether Mobile Phone Was Reachable¶
- From the above plot, we can observe that 0.18% applicants mobile numbers were not reachable.
- From the people whose mobiles numbers were reachable, 8.07% of them are likely to face Payment difficulties, whereas the people whose mobile numbers were'nt reachable, 7.84% of them are likely to face Payment difficulties.
From these we cannot infer a relationship between FLAG_CONT_MOBILE and TARGET variable.
In [163]:
#FLAG_EMAIL.
cat_univariate_analysis('FLAG_EMAIL',figsize=(15,5))
cat_proportions('FLAG_EMAIL')
Whether E-mail Was Provided¶
- From the above plot and proportions, we can clearly see that 94.3% of applicants have not given the E-mail address.
- Out of 94.3%, 8.08% are likely to have payment difficulties.
- From the above plot and proportions, we can clearly see that 5.7% of applicants have provided the E-mail address.
- Out of 5.7%, 7.88% are likely to have paymenr difficulties.
From these we cannot infer a relationship between FLAG_EMAIL and TARGET variable because they stand in similar grounds.
In [164]:
#FLAG_MOBIL
cat_univariate_analysis('FLAG_MOBIL',figsize=(15,5))
cat_proportions('FLAG_MOBIL')
Mobile Number¶
- 99.9% of them have given their mobile numbers. Out of those 99%, 8.07% have payment difficulties.
- There is only one applicant who failed to provide the mobile number, since this number is small and negligible, we cannot infer using this data.
In [165]:
#LIVE_CITY_NOT_WORK_CITY
cat_univariate_analysis('LIVE_CITY_NOT_WORK_CITY',figsize=(15,5))
cat_proportions('LIVE_CITY_NOT_WORK_CITY')
Contact Address vs Work Address¶
- From the above plot and proportions, we can clearly notice that 82% of them live and work in same city.
- Out of that 82%, 7.66% of applicants likely to have payment difficulties.
- From the above plot and proportions, we can clearly notice that 17% of them donot live and work in same city.
- Out of that 17%, 9.97% of applicants have payment difficulties.
By this, we can clearly infer that, applicants that live and work in same city are at less risk of payment difficulties.
In [166]:
#REG_CITY_NOT_LIVE_CITY.
cat_univariate_analysis('REG_CITY_NOT_LIVE_CITY',figsize=(15,5))
cat_proportions('REG_CITY_NOT_LIVE_CITY')
Permanent Address vs Contact Address¶
- From the above plot and proportions,92% of the applicant's permanent and contact address are same.
- Out of that 92%, 7.72% of applicants likely to have payment difficulties.
- From the above plot and proportions, 8% of the applicant's permanent and contact address are not same.
- Out of that 8%, 12.23% of applicants likely to have payment difficulties.
By this, we can infer that, applicants that same permanent and contact address are likely to have less risk of payment difficulties.
In [167]:
#REG_CITY_NOT_WORK_CITY.
cat_univariate_analysis('REG_CITY_NOT_WORK_CITY',figsize=(15,5))
cat_proportions('REG_CITY_NOT_WORK_CITY')
Permanent Address vs Work Address¶
From the above plot and proportions, we can notice that 77% of applicants permanent address and work address are same. Coming to other side, 23% of applicants permanent address and work address are not same.
Looking at the proportions of "Clients with payment difficulties" on both sides, we can infer that the applicants whose permanent adress and work adress matches are less likely to have payment difficulties.
Bivariate Analysis¶
In [168]:
#AMT_ANNUITY, AMT_INCOME_TOTAL vs TARGET
column_names = ['AMT_INCOME_CAT','AMT_ANNUITY']
plt.figure(figsize=(16,8))
sns.barplot(x = column_names[0],y = column_names[1],hue='TARGET',data = application_data)
plt.title(column_names[0] + ' vs '+ column_names[1] + ' vs Target')
plt.xticks(rotation=90);
In [169]:
plt.figure(figsize=(16,8))
# sns.catplot(x = column_names[0],y =column_names[1],hue='TARGET',data = application_data, kind='violin',height=8,aspect=4);
sns.violinplot(x = 'AMT_INCOME_CAT',y='AMT_ANNUITY',hue='TARGET',data = application_data,split=True, inner="quartile")
plt.title(column_names[0] + ' vs '+ column_names[1] + ' vs Target')
plt.xticks(rotation=90);
Income and Annuity¶
- Both plots show a gradual increase in the amount of annuity with increase in income.
- Also, as the income increases there are higher number of outliers i.e the propensity to take on very high annuities compared to most customers in the same income range.
- It can be seen that in low income ranges, the median no of "Clients with Payment difficulties' is slightly higher.
- And the in high income ranges,the median no of "Clients with Payment difficulties' is slightly lower.
Income Range and Annuity have a mild correlation with Payment difficulties
In [170]:
#AMT_CREDIT, AMT_GOODS_PRICE vs TARGET
column_names = ['AMT_CREDIT', 'AMT_GOODS_PRICE']
fig,ax = plt.subplots(1,2)
fig.set_figheight(8)
fig.set_figwidth(15)
ax[0].set(title = column_names[0] + ' vs '+ column_names[1] + ' vs Target');
sns.scatterplot(x=column_names[0],y=column_names[1],hue='TARGET',data=application_data, alpha=0.8,ax=ax[0])
plt.xticks(rotation=90);
plt.title(column_names[0] + ' vs '+ column_names[1] + ' vs Target for AMT_GOODS_PRICE < 1000000');
sns.scatterplot(x=column_names[0],y=column_names[1],hue='TARGET',data=application_data[application_data['AMT_GOODS_PRICE'] <=1000000], alpha=0.8,ax=ax[1])
plt.xticks(rotation=90);
Credit Amount and Price of Credit Goods¶
- There is an overall linear relationship between Credit Amount and Price of Credit Goods.
- Notice that more clients with payment difficulties are the ones who have borrowed lesser money for lesser priced goods.
- And most outliers do not have payment difficulties.
Lower Credit Amount and Lower Price of Credit Goods have higher cases of Payment difficulties
In [171]:
#NAME_CONTRACT_TYPE vs REGION_RATING_CLIENT_W_CITY vs TARGET
column_names = ['NAME_CONTRACT_TYPE','REGION_RATING_CLIENT_W_CITY']
application_data.groupby(column_names)['TARGET'].value_counts(normalize=True)
Out[171]:
In [ ]:
In [172]:
sns.catplot(x='REGION_RATING_CLIENT_W_CITY', hue='TARGET', col="NAME_CONTRACT_TYPE", kind="count", data=application_data);
Type of Loan and City Rating¶
- In cash loans, Category 3 Cities have 7% more probability of payment difficulties than Category 1 Cities.
- In Revolving loans, Category 3 Cities have 4% more probability of payment difficulties than Category 1 Cities.
Although this just confirms correlation between the City Rating and default, now we know that city rating has a higher impact on Cash Loans than Revolving Loans
In [173]:
# EXT_SOURCE_2 vs EXT_SOURCE_3 vs TARGET
creditScores = ['EXT_SOURCE_2','EXT_SOURCE_3']
plt.figure(figsize=[8,8])
sns.scatterplot(x=creditScores[0],y = creditScores[1], hue='TARGET',data = application_data, alpha=0.5);
plt.title(creditScores[0] + ' vs '+ creditScores[1]+ ' vs '+ 'TARGET');
Credit Rating¶
- Checking Credit Ratings from two different sources confirms that trend of customers with lower credit ratings have a higher default rate.
In [174]:
#FLAG_OWN_CAR vs FLAG_OWN_REALTY vs TARGET
application_data.groupby(['FLAG_OWN_CAR','FLAG_OWN_REALTY'])['TARGET'].value_counts(normalize=True)
Out[174]:
Owning A Car & Realty¶
From the above table , we see that customers with no car or realty have a high probability of default (9%), followed by customers who don't have a car but own realty (8%)
Since there's not much difference posed by owning a car or realty vs not owning them, we cannot infer any trend in default using these variables
In [175]:
#CODE_GENDER vs NAME_FAMILY_STATUS vs TARGET
# For the sake of this analysis since Civil marriage and Married are same, clubbing them into married category
application_data['NAME_FAMILY_STATUS'].replace('Civil marriage','Married', inplace=True)
ax = sns.catplot(x='NAME_FAMILY_STATUS', hue='TARGET', col="CODE_GENDER", kind="count", data=application_data, height = 5, aspect=2);
In [176]:
# classification over both categories
pd.DataFrame(application_data.groupby(['CODE_GENDER','NAME_FAMILY_STATUS'])['TARGET'].value_counts(normalize=True))
Out[176]:
In [177]:
application_data.groupby(['CODE_GENDER','NAME_FAMILY_STATUS'])['TARGET'].value_counts(normalize=True)\
.unstack()\
.plot(
layout=(2,2),
figsize=(8,6), kind='barh', stacked=True);
Gender & Marital Status¶
- Looking at the above values, customers most attractive to the bank by likelihood of default, are
'Female Widows' > 'Married Females' = 'Separated Females' > 'Single Females' > 'Married Males' > 'Male Widowers' > 'Single Males' = 'Separated Males'
The bank may focus their marketing campaigns on converting applications of Female Widows, Married Females, Separated Females
May note that the dataset is skewed towards Females than Males
In [178]:
# NAME_HOUSING_TYPE vs NAME_INCOME_TYPE vs TARGET
income_housing = pd.DataFrame(application_data.groupby(['NAME_HOUSING_TYPE','NAME_INCOME_TYPE'])['TARGET'].value_counts(normalize=True))
income_housing.columns = ['Normalized Count']
income_housing
Out[178]:
In [179]:
application_data.groupby(['NAME_HOUSING_TYPE','NAME_INCOME_TYPE'])['TARGET'].value_counts(normalize=True)\
.unstack()\
.plot(
layout=(2,2),
figsize=(8,6), kind='barh', stacked=True);
In [180]:
income_housing[np.in1d(income_housing.index.get_level_values(2), 1)].sort_values(by='Normalized Count', ascending=True)
Out[180]:
Housing Type & Income Type¶
- The above tables list the categories in the order of least likelihood of Payment difficulties
- The top 5 categories are :
- State servant living in Co-op apartment
- State servant living in Office apartment
- Pensioner living in Co-op apartment
- Pensioner living With parents
- Pensioner living in House / apartment
The bank may focus their marketing campaigns on converting applications of the above categories
Pensioner living With parents category doesn't seem right and may be further checked if it's an anomaly
In [181]:
#FLAG_EMAIL vs FLAG_MOBIL vs TARGET
pd.DataFrame(application_data.groupby(['FLAG_EMAIL','FLAG_MOBIL'])['TARGET'].value_counts(normalize=True))\
.unstack()\
.plot(
layout=(2,2),
figsize=(8,6), kind='barh', stacked=True);
In [182]:
pd.DataFrame(application_data.groupby(['FLAG_EMAIL','FLAG_MOBIL'])['TARGET'].value_counts())
Out[182]:
Email & Mobile Information¶
- 8% of customers who have not provided Email but have provided a Mobile Number have had payment difficulties
- Similarly 8% of customers who provided both Email and Mobile Number have had payment difficulties
Hence we cannot infer any conclusive trend in payment difficulties using these variables
In [183]:
# DAYS_LAST_PHONE_CHANGE vs FLAG_CONT_MOBILE vs TARGET
sns.barplot(x = 'FLAG_CONT_MOBILE', y= 'DAYS_LAST_PHONE_CHANGE', hue='TARGET',data = application_data)
plt.title("FLAG_CONT_MOBILE vs DAYS_LAST_PHONE_CHANGE vs TARGET")
Out[183]:
In [184]:
sns.violinplot(x = 'FLAG_CONT_MOBILE', y= 'DAYS_LAST_PHONE_CHANGE', hue='TARGET', split=True, data = application_data,inner="quartile", height=5, aspect=1)
plt.title("FLAG_CONT_MOBILE vs DAYS_LAST_PHONE_CHANGE vs TARGET")
Out[184]:
Days Since Phone Number Changed & Whether was Phone was Reachable¶
- Notice that the number of clients with reachable phones is higher than unreachable phones. (~900 days vs ~350 days)
- Among clients with reachable phones, Clients with Payment difficulties have median newer phone numbers than All other cases. (See violin plot for FLAG_CONT_MOBILE = 1 )
- Among clients with unreachable phones, the difference is less (~50 days vs ~ 200 days). In fact, the median age of phone number is same for defaulters and others.
We could conclude that clients with reachable and older phone numbers are better borrowers than other categories.When the phone is unreachable, last date of phone number change doesn't matter.
In [185]:
application_data['DAYS_EMPLOYED'].plot.hist()
Out[185]:
In [186]:
#DAYS_EMPLOYED & NAME_EDUCATION_TYPE vs TARGET
fig,ax = plt.subplots(1,2)
fig.set_figheight(5)
fig.set_figwidth(15)
fig.suptitle(t="DAYS_EMPLOYED & NAME_EDUCATION_TYPE vs TARGET");
ax[0].tick_params(axis='x',rotation=90)
sns.barplot(x = 'NAME_EDUCATION_TYPE', y= 'DAYS_EMPLOYED', hue='TARGET',data = application_data,ax=ax[0])
ax[1].tick_params(axis='x',rotation=90)
# ax[1].set_yscale('log')
sns.violinplot(x = 'NAME_EDUCATION_TYPE', y= 'DAYS_EMPLOYED', hue='TARGET', split=True, data = application_data,inner="quartile", height=5, aspect=1, ax=ax[1])
#sns.swarmplot(x = 'NAME_EDUCATION_TYPE', y= 'DAYS_EMPLOYED', hue='TARGET', data = application_data, ax=ax[1])
Out[186]:
Employment Period & Education¶
- The barplot between Days Employed and Education clearly shows that the clients holding an Academic Degree and having held employment for a long time have very low probability of default (difference between Target 0 , 1 of ~60000)
- Among clients with Secondary education, there are two groups, ones who are recently employed and others with long term employment. The recently employed clients in this category have equal probability to default or not. But the clients long term employment have lower probability to default.
The bank may place emphasis on education and employment period. Higher education and long term employment are good indicators of no default.Further clients with secondary education are far more attractive borrowers when they have held long term employment than otherwise
In [187]:
#CNT_CHILDREN & CNT_FAM_MEMBERS vs TARGET
subset = application_data[['CNT_CHILDREN','CNT_FAM_MEMBERS','TARGET']]
subset = subset.dropna().astype('int')
childrenvsFamily = pd.pivot_table(index='CNT_CHILDREN', columns = 'CNT_FAM_MEMBERS',aggfunc=np.mean,data=subset)
childrenvsFamily
Out[187]:
Children & Family Members¶
- These variables are chosen to verify the amplified liabilities of clients having both more children and more family members.
- The most attractive sub-categories are the clients with 1 or 2 family members and 0 children.
- This is followed by clients having 1 or 2 children with 3 or 4 family members.
Note that more than 5 children is deemed an error. This needs further analysis
Small families with less children has low risk of default
In [188]:
# NAME_EDUCATION_TYPE vs AMT_INCOME_TOTAL vs TARGET
plt.figure(figsize=(10,8))
sns.violinplot(x = 'NAME_EDUCATION_TYPE', y= 'AMT_INCOME_TOTAL', hue='TARGET', split=True, data = application_data,inner="quartile", height=5, aspect=3)
plt.yscale('log')
plt.xticks(rotation=90);
Type of Income vs Amount of Income¶
- For the same average income across income types, clients with Secondary Education have the lowest probability of default.
- Among clients with an Academic Degree, median income of clients with payment difficulties is higher than all other cases.
- Among clients with Higher Education, Incomplete Higher Education and Lower Secondary education, the median income has no correlation with risk of default.
In [189]:
#'NAME_INCOME_TYPE vs NAME_EDUCATION_TYPE'
subset = application_data[['NAME_INCOME_TYPE','NAME_EDUCATION_TYPE','TARGET']]
subset = subset.dropna()
subset['TARGET'] = subset['TARGET'].astype('int')
# pivot table for percentage of default for education type vs income type
incomeTypevsEdu = pd.pivot_table(index='NAME_EDUCATION_TYPE', columns = 'NAME_INCOME_TYPE',aggfunc=[np.mean],data=subset)
incomeTypevsEdu
Out[189]:
In [190]:
# pivot table of the count of defaults for education type vs income type
pd.pivot_table(index='NAME_EDUCATION_TYPE', columns = 'NAME_INCOME_TYPE',aggfunc='count',data=subset)
Out[190]:
Education Type & Income Type¶
- From the above pivot tables we could say that clients with Academic Degree have the least default risk.
- The highest volume of applications are from clients with Secondary education. Among them, the least default risk is posed by Pensioners.
- The next highest in terms of volume of applications is from clients with Higher Education. Among these, the least risk is posed by Pensioners and State Servants.
- Note that there's not enough application from Businessmen with Higher Education to conclusively say they have very low risk of default.
- Highest default risk is posed by Unemployed clients. Among them unemployed clients with Secondary Education have the higest risk of default.
In [191]:
sns.violinplot(x = 'NAME_INCOME_TYPE', y= 'AMT_INCOME_TOTAL', hue='TARGET', split=True, data = application_data,inner="quartile", height=5, aspect=3)
plt.yscale('log')
plt.xticks(rotation=90);
Correlation Analysis¶
Correlation For Target : 0¶
In [192]:
# function to correlate variables
def correlation(dataframe) :
cor0=dataframe[columnsForAnalysis].corr()
type(cor0)
cor0.where(np.triu(np.ones(cor0.shape),k=1).astype(np.bool))
cor0=cor0.unstack().reset_index()
cor0.columns=['VAR1','VAR2','CORR']
cor0.dropna(subset=['CORR'], inplace=True)
cor0.CORR=round(cor0['CORR'],2)
cor0.CORR=cor0.CORR.abs()
cor0.sort_values(by=['CORR'],ascending=False)
cor0=cor0[~(cor0['VAR1']==cor0['VAR2'])]
return pd.DataFrame(cor0.sort_values(by=['CORR'],ascending=False))
In [193]:
# Correlation for Target : 0
# Absolute values are reported
pd.set_option('precision', 2)
cor_0 = correlation(application_data0)
cor_0.style.background_gradient(cmap='GnBu').hide_index()
Out[193]:
Top 10 correlations for Target : 0 are¶
- AMT_GOODS_PRICE & AMT_CREDIT
- AMT_GOODS_PRICE & AMT_ANNUITY
- AMT_ANNUITY & AMT_CREDIT
- AGE_YEARS & DAYS_EMPLOYED
- AMT_ANNUITY & AMT_INCOME_TOTAL
- AMT_GOODS_PRICE & AMT_INCOME_TOTAL
- AMT_INCOME_TOTAL & AMT_CREDIT
- AGE_YEARS & EXT_SOURCE_3
- DAYS_LAST_PHONE_CHANGE & EXT_SOURCE_2
- DAYS_EMPLOYED & AMT_INCOME_TOTAL
Correlation For Target : 1¶
In [194]:
# Correlation for Target : 1
# Absolute values are reported
pd.set_option('precision', 2)
cor_1 = correlation(application_data1)
cor_1.style.background_gradient(cmap='GnBu').hide_index()
Out[194]:
Top 10 correlations for Target 1 are :¶
- AMT_GOODS_PRICE & AMT_CREDIT
- AMT_CREDIT & AMT_ANNUITY
- AMT_ANNUITY & AMT_GOODS_PRICE
- AGE_YEARS & DAYS_EMPLOYED
- EXT_SOURCE_2 & DAYS_LAST_PHONE_CHANGE
- EXT_SOURCE_3 & AGE_YEARS
- AGE_YEARS & AMT_GOODS_PRICE
- AGE_YEARS & AMT_CREDIT
- AMT_GOODS_PRICE & EXT_SOURCE_2
- AMT_CREDIT & EXT_SOURCE_2
- Merged data analysis can be found here