Stack ranking: your worst reps have good weeks too

If I asked you to name the bottom few reps in your team before opening this week's quality report, I suspect you'd do reasonably well. The same names have a habit of coming back.

There are two ways you can interpret that. One is that the weekly report isn't telling you very much. Perhaps the scores are too noisy, the same people keep getting difficult calls, or you've simply built a ranking that confirms what everybody already knows. The other possibility is less comfortable: the report is right, you knew who was struggling months ago, and nothing much has changed since.

We've been looking at more than two years of weekly scores to work out which of those explanations fits the data. The answer isn't quite as simple as I expected, because the people at the bottom move around considerably more than you might think. They just have an annoying tendency to come back.

One in four

Each week, we ranked reps by their score and divided them into four groups. If those weekly rankings were basically random, being in the bottom quarter this week would tell you nothing about next week. Your chance of ending up there again would be one in four.

In reality, it's closer to two in three.

Even over longer periods that number barely falls. After a month, a little under two thirds of those reps are still in the bottom quarter. After three months it's still about six in ten. If we remove weeks where a rep had very few calls, the persistence becomes stronger rather than weaker.

The top of the ranking is similarly stable, while most of the shuffling happens in the middle. That makes intuitive sense to me. Weekly scores are noisy, as I've written about before, but noise doesn't necessarily make a ranking useless. If two people are fairly close together, an unusually good or bad week can easily swap their positions. If they're a long way apart, it takes rather more than that.

There is a wrinkle though. Some of what looks like rep performance is actually team performance. If one team consistently scores lower than another, ranking everyone across the organisation will keep putting people from that team near the bottom. When we rank people only against their own team, the persistence drops to a little over half.

That's still roughly twice the one-in-four baseline, and it remains surprisingly stable over three months. But I think it's the more useful number for a manager. If you're trying to decide who needs help, compare people doing roughly the same job under roughly the same conditions rather than throwing the whole organisation onto one leaderboard.

They don't stay at the bottom

Take the reps who are in the bottom quarter this week and follow them for three months. A little under half of them will have at least one week in the top half during that period.

I had to think about that one for a bit. These aren't people who spend three months permanently stuck at the bottom of every report. Nearly half will, at some point, have a week where they outperform half their colleagues. They can clearly produce better results, and the weekly data actually catches them doing it.

They just don't seem to keep them.

That's a rather different coaching problem from someone who simply can't do the thing. If I know Sarah has never demonstrated good objection handling, I probably need to teach her objection handling. But if Sarah has spent most of the quarter near the bottom and I've also got two weeks where she was comfortably in the top half, my question changes. What was she doing differently in those weeks, and why didn't it stick?

That seems like a much better place to start a coaching conversation than "your objection handling score is 52".

It also connects nicely with something we found when looking at individual calls. In Stop coaching from the best call of the week, three quarters of reps had at least one call that ranked among the best calls in their organisation that week. A great call tells you what somebody managed to do once; it doesn't tell you what they normally do.

Apparently the same is true one level up. A good week isn't necessarily a new level either.

The names you already know

This leaves me looking at the boring old weekly ranking rather differently.

If the same rep appears near the bottom for three months, but occasionally demonstrates that they can perform considerably better, I'm less interested in whether the report has correctly identified a weak performer. We established that about eleven Mondays ago.

I'd want to know why the better performance isn't becoming normal.

Perhaps there are particular lead types they handle well. Perhaps one manager brings out better calls than another. Maybe coaching produces an improvement which disappears again, or they know exactly what good looks like but fall back to old habits when the queue gets busy. The score can't answer those questions, but with every call underneath it you can go looking.

And that, I think, is where a weekly report earns its keep. It isn't particularly useful to tell a manager for the twelfth time that Sarah is in the bottom quarter. The useful bit is being able to put Sarah's good weeks next to her ordinary ones and ask what changed.

So if the same names appear at the bottom of your report again next Monday, I wouldn't immediately blame the metric. I'd pick one of them, find a week where they weren't at the bottom, and start there.

They've already shown you they can do better. The interesting question is why they stopped.

CallCoach is how we score every call, which is what lets you put a rep's good week next to their ordinary ones. Try it on your own calls and find the week your bottom-quarter rep wasn't.

---

All CallCoach data in this post is aggregated, anonymised performance data from hundreds of thousands of calls across a variety of industries. We report the trends we spot; full statistical analysis is beyond the scope of a blog post.