This chapter describes data preparation techniques for different churn models. The central topic is data sampling as preparation for building churn models, especially for predictive models. The chapter shows how to construct a data sample that will reflect business reality and show good performance regarding building predictive models. A significant part of the chapter is dedicated to construction of derived variables, which are a direct reflection of expert knowledge used within churn models. Beside data preparation for predictive models, the chapter also describes data preparation techniques for other methods usable for churn modeling like survival models, fuzzy expert systems, K-mean clustering, etc. The attribute relevance analysis chapter described different techniques for attribute importance detection usable in churn modeling. It gave descriptions with examples of how to make an attribute relevance analysis for predictive churn models in case of binomial target variables, as well in case of multinomial target variables. This chapter covers dummy variable construction and profiling techniques based on attribute relevance analysis, as well as logic checks from the perspective of business users.
Top5.1 Data Preparation For Predictive Churn Modeling
Predictive churn models are almost synonym for churn modeling. Building predictive churn models are most exploited way in churn solution development.
Aimed outputs from those models are churn probability in defined future period of time.
There are few recommended characteristics, which every predictive model should have, and predictive churn models are not exception:
- •
Reliability
- •
Usability
- •
Stability
- •
Robustness
Reliability in light of churn predictive modeling means, that model should have significant power to predict which customers/consumers/clients will make churn in defined future period (e.g. 6 months). Predictive power could be measured using statistical measures like Kolmogorov Smirnoff test, or ROC curve on test sample. This methodology will be explained in details in following chapter.
Usability means that developed model has integrated business logic, and that it is in line with business perception of existing customer portfolio.
Stability is important characteristic for the models, which should be used periodically, and it implies that model should not contain unstable variables, which could cause instability of the whole model and imprecise probability calculation.
A robustness criterion implies that model is resistant on business environment changes and resistant on market changes. It is unrealistic to expect that it is possible to develop completely resistant model on significant market condition changes, or portfolio structure changes. Robustness means that model will not overreact and will not become unusable in short period of time after market conditions start to change, or portfolio structure start to change.
Each predictive modeling project demands almost 80% of time spending on data preparation. Contrary to rooted belief that data preparation process consists on Extract Transform Load processes, data quality improvement, or data extraction from different data sources only, it is much more broader process.
Data preparation starts with data sample construction planning.
Planning data sample construction for building predictive models is shown in Figure 1.
Figure 1.
Data sample construction for building predictive models
C- Observation period for active contracts; Od – Observation development point; He – End of outcome period point; Hd – Outcome period for development sample
Origination of data sample construction starts at observation development point (Od). At this point all active contracts/accounts/clients/buyers that started to use our products/services at observation period (C) should be included into data sample. Development data sample should contain socio-demographic data and behavioral characteristics for all active contracts/accounts or clients/buyers. Socio-demographic data are attributes related to client socio-demographic characteristics like age, residential status, work details and related stuffs. It is mostly static attributes which values should be determined at observation development point (Od).
Behavioral characteristics are related to contracts/accounts/client/buyer behavior and it usually should be calculated from transactional data. Behavioral attributes for example gives us information (if we talk about telecommunication company) about average call duration in previous three months, number of inbound calls in previous three months, trend of outbound calls in previous three months and similar values.
Experience shows that behavioral attributes are most important variables for churn model construction, and socio-demographic attributes are like a spice which gives flavor to the churn models.
Churn flag (churn = true or false) is generated if client broke or not broke relationship with company within outcome period for development sample (Hd). Length of outcome period should be determined empirically using methods like Cox regression or by experience, or in combination by using both approaches.
Next important topic is data quality check. It consolidates: