KKBox Churn Study

Background and data

A subscription business loses customers at renewal. Using public data from KKBox, a large Asian music-streaming service with carrier-style billing, this analysis examines predictors of churn and how well we can predict next month's churners.

The data cover 2015 to 2017: 392 million daily usage records, 23 million billing transactions and 6.8 million member profiles. Two questions drive the work. When in the customer lifecycle do new subscribers leave, and what changes that timing? And using only what is known before a month begins, how many of that month's churners can be identified?

1Most churn happens at the first renewal

The analysis follows every new paying subscriber who joined between March 2015 and December 2016 (564K people) until they left or the data ended. Subscribers still active when the data ended count only for the months we saw, so recent sign-ups do not make retention look better than it is.

Figure 1. Share of new subscribers still active, by months since first payment

Split by:

Shaded bands show the 95% confidence range; with this many subscribers they are thinner than the lines. Curves stop where fewer than 200 subscribers remain.

Figure 2. Share of remaining subscribers who leave in each month, by billing type

Auto-renewManual renewal

Manual-renewal customers leave in a spike at their first expiry and again near month 7, when 180-day plans end. Auto-renew customers leave at a low, steady rate.

Figure 3. How each factor changes the risk of leaving, holding the others equal (below 1 = less likely to leave)

2Predicting next month's churners

A model learned from 993K subscribers whose memberships expired in February 2017, using only what was known before February began. It was then applied, unchanged, to 971K subscribers expiring in March. That mirrors real use: learn from last month, score next month.

Figure 4. Share of the month's churners reached, by share of subscribers contacted, riskiest first

Contact by model scoreContact at random

Figure 5. Actual churn rate in March for each tenth of subscribers, ranked by the model's risk score

Model diagnostics: accuracy, calibration and drivers

Table A. Out-of-time performance (March 2017) as each data source is added

AUC is the probability that a random churner is ranked above a random non-churner (0.5 is chance). PR-AUC reflects performance on the rare class; the March base rate is 9.0%.

Figure A. Calibration on the March test set: predicted vs. observed churn rate in ten equal-size risk groups (log scale)

As trained (February)After recalibrationPerfect calibration

Figure B. Features with the largest influence on predicted risk: mean absolute SHAP value (log-odds) across 20K March subscribers

3Data and methods

Data and definitions

The source is the WSDM-KKBox Churn Prediction Challenge on Kaggle, released by KKBox for research with anonymized member IDs. Churn follows the company's own rule: no new paid subscription within 30 days of the membership expiring, with cancellations moving the expiry date forward. The survival cohort includes members registered in 2015 or later whose first paid transaction falls between March 2015 and December 2016. Requiring two months of prior history avoids mistaking a returning member for a new one.

Modeling

All model inputs are dated before the month in which the label is decided, which rules out leakage from renewals that have already happened. Validation is out-of-time on the following month rather than a random split, which would overstate accuracy. The model was early-stopped on a 15% in-time holdout. Calibration was corrected by re-estimating intercept and slope on a 5% labelled sample of the new month and evaluated on the remaining 95%. Survival estimates use Kaplan–Meier with Greenwood intervals and a Cox model on a 200K sample stratified by sign-up quarter, with proportionality checked month by month.