Stack ranking: your worst reps have good weeks too

If I asked you to name the bottom few reps in your team before opening this week's quality report, I suspect you'd do reasonably well. The same names have a habit of coming back.

There are two ways to read that. One is that the weekly report isn't telling you much. Maybe the scores are too noisy. Maybe the same people keep getting hard calls. Or maybe you've built a ranking that only confirms what everybody already knows. The other reading is less comfortable. The report is right, you knew who was struggling months ago, and not much has changed since.

We looked at more than two years of weekly scores to see which reading fits the data. The answer wasn't as simple as I expected. The people at the bottom move around a lot more than you might think. They just have an annoying habit of coming back.

One in four

Each week, we ranked reps by their score and split them into four groups. If the rankings were basically random, being in the bottom quarter this week would tell you nothing about next week. Your chance of ending up there again would be one in four.

In reality, it's closer to two in three.

Over longer periods that number barely falls. After a month, a little under two thirds of those reps are still in the bottom quarter. After three months it's still about six in ten. If we drop weeks where a rep had very few calls, the pattern gets stronger, not weaker.

The top of the ranking is just as stable. Most of the shuffling happens in the middle. That makes sense to me. Weekly scores are noisy, as I've written about before, but noise doesn't make a ranking useless. If two people are close together, one unusually good or bad week can swap their places. If they're a long way apart, it takes much more than that.

There is a catch. Some of what looks like rep performance is really team performance. If one team always scores lower than another, ranking everyone across the organisation will keep putting that team's people near the bottom. When we rank people only against their own team, the repeat rate drops to a little over half.

That's still about twice the one-in-four baseline, and it stays steady over three months. I think it's the more useful number for a manager. If you're deciding who needs help, compare people doing about the same job in about the same conditions. Don't put the whole organisation on one leaderboard.

They don't stay at the bottom

Take the reps who are in the bottom quarter this week and follow them for three months. A little under half of them will have at least one week in the top half during that period.

I had to think about that one for a bit. These aren't people stuck at the bottom of every report for three months. Nearly half will, at some point, have a week where they beat half their colleagues. They can clearly do better, and the weekly data catches them doing it.

They just don't seem to keep them.

That's a different coaching problem from someone who simply can't do the job. If I know Sarah has never shown good objection handling, I probably need to teach her objection handling. But say Sarah spent most of the quarter near the bottom, and I also have two weeks where she was well inside the top half. Now my question changes. What did she do differently in those weeks, and why didn't it stick?

That seems like a much better place to start a coaching conversation than "your objection handling score is 52".

It also fits something we found when looking at single calls. In Stop coaching from the best call of the week, three quarters of reps had at least one call among the best in their organisation that week. A great call shows what somebody did once. It doesn't show what they normally do.

The same seems true one level up. A good week isn't necessarily a new level either.

The names you already know

This leaves me looking at the boring old weekly ranking rather differently.

Say the same rep sits near the bottom for three months but sometimes shows they can do much better. I'm less interested in whether the report has found a weak performer. We settled that about eleven Mondays ago.

I'd want to know why the better performance isn't becoming normal.

Maybe there are lead types they handle well. Maybe one manager brings out better calls than another. Maybe coaching gives a lift that fades again. Or maybe they know exactly what good looks like but fall back to old habits when the queue gets busy. The score can't answer those questions. With every call underneath it, you can go and look.

That, I think, is where a weekly report earns its place. Telling a manager for the twelfth time that Sarah is in the bottom quarter is not useful. The useful part is putting Sarah's good weeks next to her ordinary ones and asking what changed.

So if the same names are at the bottom of your report again next Monday, don't blame the metric straight away. Pick one of them, find a week where they weren't at the bottom, and start there.

They've already shown you they can do better. The interesting question is why they stopped.

CallCoach is how we score every call, which is what lets you put a rep's good week next to their ordinary ones. Try it on your own calls and find the week your bottom-quarter rep wasn't.

---

All CallCoach data in this post is aggregated, anonymised performance data from hundreds of thousands of calls across a variety of industries. We report the trends we spot; full statistical analysis is beyond the scope of a blog post.