20 quick insights on Customer Success and SaaS metrics with Dave Kellogg

Did you catch our Customer Success and SaaS metrics crash-course webinar with leading SaaS expert Dave Kellogg, of Dave Kellogg Consulting?

The audience had a ton of great questions that we couldn’t let go unanswered. Following the webinar, we invited Dave to give his rapid-fire takes on tracking the retention of auto-renew customers, calculating customer lifetime value as a startup, comping CSMs on expansion, determining the importance of measuring time to value, and much more.

We’d like to extend a huge thanks to Dave for his expert insights below, which will help you use Customer Success and SaaS metrics in a more nuanced and purposeful way.

You can read part one of our Q&A recap here.

Q: How do you think about calculating gross margin/cost of goods for pureplay SaaS?

A: First, you need to separate subscription cost of goods (COGS) from overall COGS. The latter includes professional services COGS, whereas the former is the pure cost of running the services (e.g., tech ops, AWS, technical support). In some ways, I just delegate this to accounting because it’s a generally accepted accounting principle (GAAP) and, while I’m interested in how they do it, ultimately, you’re just taking it right off the profit and loss (P&L).

Q: In Dave’s version of logo retention rate*, how do you treat customers on contracts that allow early-outs?

A: I’d ignore early-outs in available to renew (ATR) churn calculations. I think the right way to treat them is to build a history, and then do some sort of reserve against them. See your local CFO. The other answer: if they’re not a customer, they don’t count in churn; so maybe you don’t count them as customers until the end of the out period.

* To calculate Dave’s version of logo retention rate, divide the number of customers up for renewal that renewed by the number of customers up for renewal.

Q: If most of your customers are on auto-renew, do you use the regular logo retention rate or the ATR rate?

A: I love auto-renews. In the spirit of the Golden Rule, you should let customers know BEFORE you do the auto-renew so they have time to opt out. Then, they are just regular renewals, but you’ve optimized the process for the normal occurrence which is hopefully a renewal. They count as ATR because they have the choice not to renew. On logo/count-based vs. dollar-based rate, the real issue is how similar your customers are to each other. See this thread on the topic.

Q: How do you measure retention with a mixture of auto-renew customers and finite terms?

A: With a mix, there’s one driving question: did the customer have the chance to renew or not? If they did, then they’re in the ATR pool (i.e., the denominator); if they did not, they’re not. The idea is two separate metrics. One that says: of the whole base, here’s the percentage who renewed. The second says: of those who had an option, here’s the percentage who renewed. They measure different things. The former is better as a financial measure; the latter as a proxy for customer satisfaction score (CSAT).

Q: Should you track partial churn (i.e., the logo stays with you, but they consume less, so their monthly recurring revenue decreases)?

A: Yes, strongly. Logo churn misses this because the numerator is only logos. Annual recurring revenue (ARR)-based churn (dollars renewed divided by dollars up for renewal) does not. That’s why I like ARR-based if only given one.

Q: Do you ever see gross revenue retention without contractions?

A: No. In my world, gross revenue retention (GRR) is a dollar-based revenue retention metric that includes all forms of shrinkage, including shrinkage to zero! If you want to exclude customers who shrank to non-zero amounts, use logo churn.

Q: What’s the rationale for keeping the cadence on monthly recurring revenue versus annual recurring revenue?

A: If you sign month-to-month contracts that say “$1K/month,” and then call that “$12K ARR,” I think it’s disingenuous. It’s $1K of monthly recurring revenue (MRR). It gets tricky when you’re 90% annual contracts but someone wants to, e.g., renew for only six months. In that case, I would multiply by two to annualize to ARR, but see your local CFO for more.

Q: Are reactivations ever counted in net revenue retention? If not, where should they be counted when measuring company performance?

A: Like in baseball, I call these “saves” – and yes, they are technically included because net revenue retention (NRR) is a cohort-based measure. Let’s say we take a 1/1/20 cohort and customer X is in at $100K. Then, they churn on their renewal date of 3/1/20. We sell them hard, and they regret leaving and sign up at $150K on 10/1/20. On 1/1/21, they’re worth $150K, and ergo a $50K expansion, everything in between doesn’t matter. I love cohorts – the flows don’t matter, only the two snapshots do.

Q: Should you track auto-renewals differently from enthusiastic opt-in renewals?

A: No, not for the first-order metrics. Any renewal is a renewal. Any churn is a churn. Sometimes people want to say “uncontrollable” churn to exclude companies that went out of business (but that’s still real churn) or that got acquired (but wait, why couldn’t you upsell the parent company? If they loved-loved you, maybe you could). At the first order, exclude reasons. I’d try to capture what I think you’re talking about with net promoter score (NPS)/CSAT, intent to renew or – you gave me a new idea – renewal desire! Think: do you want to renew? Do you intend to renew? Cool!

Q: Should you use quarterly retention rate to measure a CSM’s performance?

A: So hard. As we discussed, it’s all about the hand you got dealt. To make this easy, let’s say the company’s target NRR is 105% and the hands (i.e., book of businesses) were divided perfectly fairly. Then, the answer is 105%. I want you to do as well as the average CSM. However, if you got the “hard cases,” then I’d dial that back. As discussed, it’s about comparing the actual result to an honest forecast.

Q: For customer acquisition cost, how far back should the numerator and denominator “lag”?

A: If you look at first principles, you should lag it by your average sales cycle length. BUT NO ONE DOES THAT. Just lag it by a quarter and save yourself the headache (and enable comparability), unless you have a really, really good reason not to.

Q: Should you attempt to calculate customer lifetime value if you’re just starting out?

A: You shouldn’t. The good news is, if you do one-year deals, it’s – by definition – perfect in your first year! Measure CSAT along the way, as well as implementation time and CSAT. If a customer doesn’t get implemented, odds are they won’t renew.

Q: Should you use metrics to measure the effect of CSM touchpoints on retention?

A: I’m not a big fan of activity metrics, so no. I suspect the real issue is process compliance. If you have a process, they should execute it and the score is actually binary.

Q: Should you add a sales accelerator to the comp plan of CSMs who have more difficult accounts?

A: Yes, that’s one way to do it, and a good one for would-be seller CSMs! Think: “We’ve handed you a hard set of accounts. We think they’re going to renew at 75%. Your on-target earnings (OTE) are based on hitting that 80%. I’ll start levering you up after that.” But that all depends on an honest forecast, which needs to be made by CS ops or sales ops – and not by the person who’s being paid to beat it!

Q: How do you measure the performance of non-revenue-based Customer Success goals?

A: In your CSAT/NPS survey, I’d add two questions. First, how would you rate your CSM on a scale of 1-10 on both effectiveness and relationship (so they can say “we like Joe but he’s not effective,” in effect)? Second, who is your favorite person at the company? If your CSMs aren’t winning that often or at least getting votes, well, it’s a problem. Their job is to be the good guys!

Q: How important is it to track time to value?

A:  Critical, but in the sense that you want customers getting value within time T. It’s not about making T 10% shorter. It’s about ensuring they’re up and ready to go so they can actually use the system and see if they like it. It’s a necessary, but not a sufficient, condition for renewal.

Q: What metrics help show a CSM’s added value to an organization?

A: Unless you’re at real scale, you can have fun with open-ended survey questions: tell me three things about your CSM. Then, you summarize what they said at the ops reviews. Or maybe you ask the customer to tell you three things they liked and three things they didn’t like. Either way, the answer will show if the CSMs are actually involved with the customers and if the customers know them (make responses optional). The other trick is to list attributes about the company, and then do a regression to correlate them to renewal. Use attributes like company cares about my success; company hires good people; company has great CSMs; company is a technology leader; or company is easy to do business with.

Q: How does using a pay-as-you-go model change the metrics you track?

A: Well, the first thing it does is blow up ATR churn because everyone is ATR all the time. So, you don’t need that metric. It’s a key question because you’re blowing up the basic “contract renewal” paradigm of SaaS as well. To me, the spirit changes from “who likes you and will renew” to more financial modeling about how much people are going to use in the future.

Q: How should you comp a CSM under an expansion revenue-share model?

A: The beauty of CSM comp plans is they’re usually not that leveraged. If a seller is 50/50 base/variable, a CSM might be 70/30, so the first answer is it matters less. I’m a little rusty on CSM comp plans but my inclination would be to comp 20% on CSAT and 80% on NRR. Think: “My name is Dave, and my job is to get your renewal and have you expand at the standard rate. I know that won’t happen if you’re not happy and getting value, so let’s talk about that.”

Q: How do comp both CSMs and salespeople on an expansion?

A: You pay sales on new ARR regardless of whether it’s from new or existing customers, and you pay CSMs on an NRR target. In effect, you’re paying both on expansion within customers.

The Customer Success metrics your investors want to see

Do you know what Customer Success metrics your board really cares about? Are you facing a project like a merger or acquisition that requires you to work with new investors or existing investors in a new capacity? Learn how to tell a compelling story with metrics in our blog, “How Customer Success can use metrics to better engage investors and boards.”