When Customers Churn at Renewal: Was It the Price or the Project?

of these choices that appears simple till it’s a must to measure it. A buyer’s introductory price expires, the bill goes up, and also you wish to know whether or not the worth change damage retention. Easy sufficient in concept.

The issue is that one thing else is nearly at all times occurring on the identical time. The initiative that drove the unique buy, whether or not it was a system migration, compliance push, gross sales transformation, or product launch, has wrapped up. The workforce that championed the instrument has moved on to the subsequent factor. And the product that when felt important is quietly turning into a line merchandise somebody goes to query.

So when the client churns, the account workforce says it’s the value. The retention technique workforce says the use case ran its course. Product says the platform by no means acquired previous the unique purchaser. Everybody has a concept and a spreadsheet to again it up.

Which attribution you land on issues, not abstractly, however by way of what you do subsequent.

Every methodology under builds a distinct counterfactual for a similar occasion. Choosing the right one will not be the onerous half. Understanding which query you are attempting to reply earlier than you open a pocket book, that’s the place most of those analyses go sideways.

Outline the query earlier than the strategy

Earlier than you contact the information, you’ll want to resolve what you’re truly making an attempt to estimate. The identical churn occasion at renewal can produce three meaningfully totally different numbers relying on what you’re asking:

• The promo-cohort impact. What was the typical churn influence on prospects whose introductory low cost expired? The finance workforce often desires this quantity as a result of it strains up with how renewal income will get reported.
• The initiative-completion impact. What was the churn influence on prospects whose unique adoption use case had concluded by renewal? The retention technique workforce desires this one as a result of it speaks as to if the product achieved sticky worth or simply served a venture.
• The joint impact and its interplay. What occurred to prospects who confronted each on the identical time, value improve and worth exhaustion arriving collectively? This quantity is nearly at all times bigger than both pressure alone would predict, and it’s often the one that truly explains the churn spike.

These will not be the identical quantity and they don’t reply the identical query. Treating them as interchangeable is the most typical mistake I see in renewal churn analyses, and it’s often what retains the account workforce versus retention technique debate getting in circles.

The Setup

The artificial dataset has 10,000 B2B prospects noticed round their renewal dates. Every has two flags: promo_expired (did their introductory price finish at renewal?) and initiative_complete (had the unique use case concluded earlier than renewal?). One factor value flagging upfront: initiative_complete must be outlined utilizing pre-renewal alerts, issues like buyer relationship administration (CRM) milestones, implementation completion, or buyer success well being scores. In the event you infer it from declining utilization after the actual fact, you’ll find yourself calling early churn conduct a reason for churn fairly than a symptom of it. The true results baked into the simulation:

• Baseline 6-month churn (neither pressure): 8%
• Promo expiry alone: +5 pp (13% churn)
• Initiative completion alone: +4 pp (12% churn)
• Each forces collectively: +14 pp (22% churn), a +5 pp interplay surplus above the additive expectation of 17%

import numpy as np
import pandas as pd
import statsmodels.formulation.api as smf

RNG = np.random.default_rng(158)   # seeded RNG for reproducibility
N   = 10_000                      # variety of prospects

# True results baked into the information, what every methodology ought to get well.
TRUE_BASELINE    = 0.08   # 8% baseline 6-month churn (neither pressure)
TRUE_PROMO       = 0.05   # +5 pp from promo expiry alone
TRUE_INITIATIVE  = 0.04   # +4 pp from initiative completion alone
TRUE_INTERACTION = 0.05   # +5 pp extra carry when BOTH forces hit

prospects = pd.DataFrame({
    'customer_id':         np.arange(N),
    'promo_expired':       RNG.selection([0,1], N, p=[0.45, 0.55]),
    'initiative_complete': RNG.selection([0,1], N, p=[0.50, 0.50]),
    'arr_usd':             RNG.lognormal(10.5, 0.8, N),  # annual rev
    'tenure_months':       RNG.uniform(10, 14, N),
    'n_seats':             RNG.integers(5, 200, N),      # seats bought
})

# Every buyer's churn likelihood = baseline + promo + init + interplay.
# The interplay time period solely fires when BOTH forces are lively.
churn_prob = (
    TRUE_BASELINE
    + TRUE_PROMO       * prospects['promo_expired']
    + TRUE_INITIATIVE  * prospects['initiative_complete']
    + TRUE_INTERACTION * prospects['promo_expired']
                       * prospects['initiative_complete']
)
prospects['churned'] = (RNG.uniform(measurement=N) < churn_prob).astype(int)

Technique 1: Distinction-in-Variations

Enterprise query: What was the typical churn influence of promo expiry on prospects who truly confronted a value improve at renewal, and does that influence differ relying on whether or not the use case had already concluded?

Technique-specific estimand: Common remedy impact of promo expiry on the promo-expired cohort, with a triple interplay time period to detect whether or not the worth shock is amplified when initiative completion co-occurs.

Figuring out assumption: Parallel traits. Absent promo expiry, the churn trajectory of expired and non-expired prospects would have tracked one another inside comparable initiative-completion teams across the renewal date.

To run this, you’d combination prospects into cohort-week cells across the renewal date, with every row representing a cohort’s weekly churn price and the variety of prospects nonetheless in danger that week. The triple interplay lets the mannequin detect whether or not the promo shock is amplified when the use case has additionally concluded:

# A 'cohort' is the (promo_expired, initiative_complete) cell: 4 cohorts.
# Every row within the panel is one cohort in a single week, with that cohort's
# weekly churn price and the variety of prospects nonetheless in danger that week.
# week = 0 is the renewal date; unfavourable weeks are pre-renewal.
panel = build_cohort_week_panel(prospects) # lengthy format: cohort x week

panel['post'] = (panel['week'] >= 0).astype(int) # 1 if post-renewal
panel['A'] = panel['promo_expired'] # rename for readability
panel['B'] = panel['initiative_complete']

# 'put up * A * B' expands to: put up, A, B, put up:A, put up:B, A:B, put up:A:B.
# Weighting by at_risk provides larger cohort-weeks extra affect.
did_model = smf.wls(
    'churn_rate ~ put up * A * B',
    information = panel,
    weights = panel['at_risk'],
).match(cov_type='HC3') # heteroskedasticity-robust normal errors

# Coefficients to learn:
# put up:A = promo shock when the initiative remains to be ongoing
# put up:B = initiative shock when the promo has not expired
# put up:A:B = extra churn when each forces hit in the identical week

A word on initiative_complete. It’s not randomly assigned and it correlates with issues that independently predict churn: buyer measurement, how lengthy the unique purchaser has been on the firm, and product match. Controlling for covariates helps, however what you can’t do is let the mannequin outline it. Measure it earlier than the renewal determination utilizing CRM or buyer success milestones, not from utilization patterns you observe after the client has already began disengaging.

Failure mode: anticipation. Renewal quotes exit early. If prospects begin buying alternate options the second they see the brand new price, the pre-period is already contaminated. Test the event-study plot earlier than you belief the coefficient.

Studying the outcome. The triple interplay time period, put up:A:B, is what the setup is constructing towards. A constructive coefficient there means the worth shock bites tougher when the use case has already light. In the event you see that, a reduced renewal bill won’t repair it.

Technique 2: Regression with interplay phrases

Enterprise query: What are the separate results of value improve and venture completion, and do they work together?

Technique-specific estimand: Predominant results and interplay coefficient from a regression that explicitly fashions each forces and their joint time period.

Figuring out assumption: No unmeasured confounders, ample overlap throughout all 4 situations, accurately specified practical kind.

# Buyer-level regression. Final result: 1 if buyer churned inside 6 months.
# np.log1p(x) = log(1 + x); used to regulate for skewed greenback/depend covariates
# (annual income, seat counts) so a number of massive prospects don't dominate.
# The * operator under expands to: principal results of A and B AND their interplay.
interaction_model = smf.ols(
    'churned ~ promo_expired * initiative_complete'
    '       + np.log1p(arr_usd) + np.log1p(n_seats)',
    information=prospects,
).match(cov_type='HC3')   # HC3 = heteroskedasticity-robust normal errors

# Coefficients (illustrative, matching simulation fact):
#   promo_expired:                     +0.049   (b1, principal impact of A)
#   initiative_complete:               +0.041   (b2, principal impact of B)
#   promo_expired:initiative_complete: +0.051   (b3, interplay A x B)

One factor that journeys folks up: b1 will not be ‘the impact of promo expiry.’ It’s the impact of promo expiry when initiative_complete equals zero. As soon as the initiative has additionally concluded, the marginal impact of promo expiry is b1 + b3, the place b3 is the interplay coefficient. The total image:

Impact of promo expiry, initiative ongoing:     b1      = +0.049
Impact of promo expiry, initiative full:    b1 + b3 = +0.100
 
Impact of initiative completion, promo ongoing: b2      = +0.041
Impact of initiative completion, promo expired: b2 + b3 = +0.092

Failure mode: collinearity. In the event you bought loads of prospects into the identical wave of transformation work, promo expiry and initiative completion will likely be correlated by development. When that occurs, b1, b2, and b3 get onerous to separate and the usual errors will flag it. At that time, report the joint prediction for every cohort fairly than making an attempt to interpret the coefficients individually.

Studying the outcome. That interplay coefficient is as massive as both principal impact by itself. A buyer going through each forces is not only further at-risk, they’re in a basically totally different state of affairs. That’s what ought to drive the industrial response.

Technique 3: Shapley worth attribution

Enterprise query: On condition that each forces collectively induced 14 pp of incremental churn, how a lot of that ought to every pressure be accountable for, for the needs of price range allocation and renewal technique?

Technique-specific estimand: Honest allocation of the joint churn influence throughout the 2 causal forces, utilizing Shapley values from cooperative sport concept.

Figuring out assumption: The coalition worth estimates v(S), the place S is a subset of the drivers and v(S) is the incremental churn attributable to that subset, are credible. They arrive from the regression or experiment above, not from a confounded mannequin.

With simply two drivers, Shapley is definitely fairly intuitive. Every driver retains its standalone contribution, after which the 2 break up the interplay surplus evenly. Promo expiry will get its 5 pp plus half the 5 pp interplay. Initiative completion will get its 4 pp plus the opposite half. The code makes this concrete:

from itertools import permutations
import math

# A 'coalition' is any subset of drivers lively collectively.
# v(S) = the incremental churn (in pp) attributable to coalition S.
# These coalition values come from the interplay regression above:
v = {
    frozenset():                    0,   # neither driver lively
    frozenset(['promo']):           5,   # promo expiry alone
    frozenset(['init']):            4,   # initiative completion alone
    frozenset(['promo', 'init']):  14,   # each, consists of +5 pp interplay
}

# 'gamers' = the drivers we're allocating credit score throughout.
# For every ordering of gamers, every participant's 'marginal contribution' is
# how a lot the coalition worth grows when that participant joins.
# Shapley worth = common marginal contribution throughout all orderings.
def shapley_values(v, gamers):
    n   = len(gamers)
    phi = {p: 0.0 for p in gamers}      # accumulator for every participant
    for perm in permutations(gamers):    # strive each ordering
        coalition = frozenset()           # begin with no drivers lively
        for participant in perm:
            # how a lot does the coalition develop when this participant joins?
            marginal     = v[coalition | {player}] - v[coalition]
            phi[player] += marginal
            coalition    = coalition | {participant}
    # common throughout all n! orderings
    return {p: spherical(phi[p] / math.factorial(n), 2) for p in gamers}

print(shapley_values(v, ['promo', 'init']))
# {'promo': 7.5, 'init': 6.5}  # sums to 14 pp, the total joint impact

The factor value repeating right here. Shapley is an allocation rule. It distributes credit score pretty given the coalition values you feed it, nevertheless it can’t repair dangerous inputs. In case your v(S) estimates come from a confounded regression, your Shapley shares are confounded too. The mathematics is clear; the causal work nonetheless has to occur upstream.

Studying the outcome. A 7.5 to six.5 break up will not be a sign to place 54% of your retention price range into pricing and 46% into buyer success. It’s a sign that you just want each, and that the timing of the renewal supply issues as a lot as what’s in it.

Selecting between the strategies

There isn’t any universally right methodology right here. The proper selection depends upon what query you’re answering and what your information can truly help. In observe, I run a couple of:

When the identification checks, the interplay mannequin, and the attribution layer all level in the identical route, I’m snug presenting the outcome. After they diverge, that’s value understanding earlier than you convey something to a stakeholder assembly. A pointy disagreement between strategies is often telling you one thing about which assumption will not be holding.

Translating the impact into income and LTV

Getting a churn coefficient will not be the identical as getting a pricing advice. The identical churn improve can nonetheless be web constructive if the worth carry is massive sufficient. You must propagate it ahead earlier than whether or not the change truly labored.

# LTV = anticipated income per buyer over a hard and fast horizon (in months).
# survival[m] = likelihood the client remains to be subscribed in month m.
# Multiply by month-to-month MRR and sum: undiscounted 2-year LTV.
def ltv(monthly_churn, monthly_mrr, horizon=24):
    months   = np.arange(horizon)
    survival = (1 - monthly_churn) ** months
    return (survival * monthly_mrr).sum()

# Convert 6-month churn charges into month-to-month churn charges.
# (1 - p)^(1/6) is the month-to-month survival price; subtracting from 1 provides month-to-month churn.
baseline_monthly = 1 - (1 - 0.08) ** (1/6)   # 0.0138 month-to-month churn
treated_monthly  = 1 - (1 - 0.22) ** (1/6)   # 0.0406 month-to-month churn

old_mrr = 1_000   # pre-renewal month-to-month recurring income (MRR)
new_mrr = 1_130   # post-renewal MRR (+13% value improve)

baseline_ltv = ltv(baseline_monthly, old_mrr)   # $20,550
treated_ltv  = ltv(treated_monthly,  new_mrr)   # $17,546

# Internet 2-year LTV change per buyer: -$3,004
# The 13% value improve doesn't offset the accelerated churn.

# Breakeven: what new MRR would restore the baseline 2-year LTV?
price_grid = np.linspace(1_000, 1_600, 1_000)
ltv_grid   = [ltv(treated_monthly, p) for p in price_grid]
breakeven  = price_grid[np.searchsorted(ltv_grid, baseline_ltv)]
# Breakeven MRR: ~$1,324  (a 32% improve, not the 13% that shipped)

Studying the outcome. On this state of affairs, the 13% value improve pays for itself within the quarter it ships however eats by medium-term buyer worth. To interrupt even on 2-year LTV on the identical churn price, you’d should be charging roughly $1,324, a 32% improve fairly than the 13% that went out. That’s not a niche you shut with a distinct value level. The underlying use-case drawback must be addressed first.

The decomposition is the deliverable. The causal estimate is simply the enter.

A number of closing pitfalls

1. Correlated task. The land-and-expand gross sales movement creates a pure correlation between promo expiry and initiative completion. You bought the client on an enormous initiative and gave them a year-one deal to get them transferring. Now each issues are expiring on the identical time by design. Cross-sectional variation alone won’t untangle them. You want timing variation, comparability cohorts, or an eligibility cutoff.
2. Anticipation. The pre-period solely stays clear if prospects don’t react to the renewal quote earlier than the official price-change date. After they do, the parallel-trends assumption breaks earlier than the remedy even fires, and the DiD coefficient picks up the early response fairly than the worth shock itself. The event-study plot is your first line of protection; a slope within the pre-period weeks is the inform.
3. Estimand drift. The most typical mistake I see in renewal churn reads is bringing the mistaken quantity to the assembly. The promo-cohort common impact, the interplay coefficient, and the Shapley allocation are three totally different solutions to a few totally different questions. Know which one you’re presenting and why.
4. Attribution with no determination. Shapley will get you to an allocation. It doesn’t get you to a plan. A 54/46 break up between value and use-case exhaustion is helpful context for a dialog, not a price range instruction. Somebody nonetheless has to resolve what the precise retention intervention is.

All code on this article runs finish to finish on the artificial dataset. The total pocket book with the cohort-week panel development, diagnostic plots, and sensitivity checks is on GitHub and runnable immediately in Colab.

Workers Information Scientist centered on causal inference, experimentation, and determination science. I write about turning ambiguous enterprise questions into decision-ready evaluation.

Extra like this on LinkedIn 👇

🔗 LinkedIn