Content
I stopped trusting health scores years ago. At least the traditional, rules-based ones. Let me explain.
Every CS team has one, and almost all of them were built the same way. You pick a few signals: usage, logins, support tickets, maybe NPS. You decide how much each one matters and assign weights. You set thresholds for red, yellow and green.
Here’s what bothers me. Every single one of those decisions is a guess.
Educated guesses, sure. Smart people in a room making reasonable calls based on what they know. But still guesses. Which signals matter is someone’s best judgment. How much each one counts is someone’s best judgment. Where the line sits between yellow and red is usually a number settled on in a meeting a year ago by someone who may not even work there anymore.
And I kid you not, when I joined my last org that is exactly what happened. They had just launched a health score they’d been building for the better part of a year. Rules based. Weighted thresholds. The whole thing.
I scrapped it.
I didn’t trust what it was telling me, and a score you or your CSMs don’t trust is worse than no score at all.
The actionability problem with health scores
Say the score does catch something. An account flips to red. Your CSM opens the dashboard and sees it.
Now what? It could be red because usage dropped, because a stakeholder went quiet, because they logged three escalations last month, or because the product isn’t delivering what they bought it for. A composite score flattens all of that into one number. It flags the account and leaves the hard part to the CSM.
The weighting makes it worse. If adoption carries 40% of the score and stakeholder engagement carries 10%, an account can lose its economic buyer and still sit at yellow because usage is holding steady.
The weights you picked determine what the score can even see. Get them wrong and the model will confidently hide the thing that’s actually about to kill the renewal. A CSM working off a flag instead of a driver either picks the wrong intervention or does nothing at all, because they don’t know where to start.
Health score vs. churn risk indicator: two tools, two jobs
Here’s where I differ from most of the industry. Most CS orgs run everything through one composite score: strategic account health, short-term renewal risk, CSM prioritization, all of it. I get why. One number is easier to manage than two.
Those are two completely different questions, and one tool can’t answer both well. So across my CS orgs, I’ve kept two separate tools.
Health score
The long view. Is this account on track across the lifecycle?
- Built top-down from your assumptions
- Drives client conversations, QBRs, expansion planning and coaching
Churn risk indicator (CRI)
The short view. What is happening right now that could derail this renewal?
- Built bottom-up from your historical data
- Drives save plans and retention forecasting
Some people call the second one a renewal probability score, or RPS, which honestly I like better as a name. The industry doesn’t generally make this distinction. This is how my teams have done it.
The health score is the leading indicator
This is the part most people get backwards. A health score, built well, tells you how an account is progressing across the lifecycle. It tracks trajectory: whether the customer is getting where they said they wanted to go. That’s what gives you room to act while there’s still something to act on.
Ours lived in a dashboard with separate dimensions rather than a single number:
- Customer journey progress: milestones completed on time, time to value, and outcomes realized
- Product adoption maturity: how deeply customers use the core and advanced features tied to ROI
- Partnership strength: an active champion, access to the decision maker, and more than one thread into the account
- Cohort benchmark: how the account compares with similar companies by size and industry
That separation is what made it useful. When one dimension dipped, that was the signal to go talk to the client, understand what was happening in that specific area, and work out how to course correct together. A CSM seeing journey progress slip has a completely different conversation than one seeing partnership strength thin out.
The churn risk indicator tells you something is already wrong
The CRI has a different job entirely: what is happening right now that could derail this account. And honestly, by the time the CRI fires, you’re already behind. Something has shifted. The question is whether you can still do anything about it.
Your annual physical tracks your cholesterol, your blood pressure, your heart health and your glucose levels. That’s the health score. It’s watching the trend so you can change course before something breaks. A diagnosis of a disease is something else. That’s the CRI. It’s specific, it’s urgent and it means the thing you were trying to avoid has already started.
You need both. If your only tool is the CRI, you’re running a CS org that reacts well and prevents nothing.
How we built our churn prediction model
When I built our first CRI, AI wasn’t where it is now. I needed a data scientist and I was lucky to have access to one.
She recommended a logistic regression model. I had no idea what that meant at the time. What it does is look at your historical outcomes, which customers renewed and which churned, and work out which signals actually predicted that outcome and by how much.
The output is a probability rather than a color. This account sits at 73% likely to renew. That one is at 34%. And it shows you which signals are driving that number, so your CSM sees the why alongside the score.
She pulled our client data across all time, from the beginning of the company to the present. I wanted coverage across adoption, sentiment and client engagement, so we worked through it step by step rather than throwing everything in at once.
- We started with adoption. We narrowed it down to four metrics with the strongest correlation to churn. That became our MVP, and we launched with it to see how it held up. It did well.
- Then we tried to add more. Support tickets and sentiment came first. Ticket volume, CSAT scores and ticket length didn’t correlate to churn in any meaningful way. Then we tried relationship signals, pulling every email and meeting logged in our CRM. Still not conclusive. Looking back, I think we hit a capability limit as much as a data limit, and I can only imagine what AI could do with that same dataset now. Either way, adding them would have muddled the score, so we left them out.
Comparing each account against itself
Enterprise customers use our product very differently than SMB customers. New customers use it differently than customers who’ve been with us for years. Different usage patterns, different rhythms, different definitions of normal. On top of that, some customers have seasonality baked into how their business runs.
A universal threshold breaks immediately under that. A 30% drop in usage might be a serious warning sign for one customer and completely normal for another. And an account with a predictable quiet period every year gets flagged as at risk every single year, which trains your CSMs to ignore the score.
So we built the CRI to compare each account against itself. It looks at the last 30 days and measures that against the account’s own historical pattern rather than a fixed rule applied across the book.
That one change made the score far more trustworthy. The question stopped being whether usage was below some arbitrary line and became whether this customer was behaving differently than this customer normally behaves.
How we used it for retention forecasting
The CRI is what my CSMs and I used for forecasting. Every CSM used it to call their own book: which accounts are at risk, what the probability says, and what they’re doing about it. Then I used the aggregate to build the retention and churn forecast I presented to leadership.
What made it genuinely useful was that it told us which accounts needed real work. Those accounts got a save plan with specific actions and owners, because we could see what was driving the risk and had enough runway to do something about it.
The MVP stood. Two years later we ran an accuracy check on it and it was predicting churn well in our mid-touch and SMB segments, which is where most of our customer volume sat. That was a real win.
There’s no excuse anymore
AI has changed what’s possible here, and that’s the part I want CS leaders to hear.
Everything I just described took a data scientist and months of work. You can build this yourself now with AI tools and your own historical data. Or you can buy a platform that already does it properly.
The old approach of picking signals, assigning weights and setting thresholds made sense when this kind of modeling was out of reach for a CS team. It’s not out of reach anymore. It’s just the old way of doing something that can now be done much better.
The question worth sitting with
Is your health score actually predicting renewal in your customer base, or is it sorting your accounts into colors based on assumptions nobody ever tested?
Customerscore.io
On the option of buying rather than building, I want to tell you about a partnership I’ve just started.
Their philosophy on scoring is almost identical to mine. The platform doesn’t ask you to design a health score. It trains a predictive model on your own customer base, comparing the customers who stayed with the ones who left, and it learns what churn actually looked like in your data.
Two things about their approach matter most to me.
- Every score shows the drivers behind it. Champion left the company. No QBR in two quarters. Seat usage down 41% before renewal. Your CSM sees the probability and the reason in the same place, which solves the actionability problem I described earlier.
- The model looks at each customer individually, including their seasonality. This is the same problem we had to solve ourselves. If an account has the same quiet period every year and has always renewed, that’s their normal rather than a risk.
I’m a Customerscore partner, which means I earn a commission if you become a customer. I say yes to very few of these, and I said yes here because I sat through their demo nodding along at a product that solves the exact problem I spent months solving by hand. I wish this had existed when I was building ours.
Exclusive perks for CS RevSpeak readers
Book a demo and mention CS RevSpeak. When you sign up for an annual plan, you can choose either perk.
Mention CS RevSpeak when they ask how you heard about them.
Frequently asked questions
What is the difference between a customer health score and a churn risk indicator?
A health score is the long view. It tracks whether an account is on track across the lifecycle and drives client conversations, QBRs, expansion planning and coaching. A churn risk indicator is the short view. It flags what is happening right now that could derail a renewal, is built from your historical churn data, and drives save plans and retention forecasting.
Why do traditional health scores fail to predict churn?
Rules-based health scores rely on assumptions: which signals matter, how much each one weighs, and where the thresholds sit. Those choices are rarely tested against real renewal outcomes, and a composite score hides the driver behind a red account, so CSMs can’t tell what to do about it.
How do you build a churn prediction model for customer success?
Start with your historical outcomes: which customers renewed and which churned. A model such as logistic regression identifies which signals actually predicted churn and by how much, and outputs a renewal probability with the drivers behind it. Start with the signals that correlate most strongly, compare each account against its own historical pattern, and add new signals only when they improve accuracy.
