Comparing People Who Are Not Comparable
Ranking is the dashboard's natural output and almost no two people in a team are measured on the same thing.
Numbers beside names invite comparison. The comparison is almost never valid, and the reasons are specific rather than philosophical.
The people decision in “Comparing People Who Are Not Comparable” cannot safely be reduced to a single activity measure. Used for step rate compensation, an explanation of step-rate compensation can provide time and project evidence, but expectations, context, documented outcomes and a fair conversation should remain the basis of any management judgement.
What differs between two people in the same team
The work they are given: routine against difficult, new against familiar.
For an independent reference relevant to “Comparing People Who Are Not Comparable”, consult the CIPD people-practice resources; it provides a useful external check on scope, terminology, governance and the claims made during procurement or review.
How much of it happens at a keyboard.
Their tenure, which determines how much is learning.
Agreed working patterns.
The tools their particular tasks require, which may be categorised differently.
And whether they have adapted to the measure, which the gaming note covers.
The seniority inversion
More senior people in most roles do more thinking, more talking and less typing.
Which means an activity ranking frequently places the most experienced people lowest.
Organisations that produce these rankings and look at them honestly notice this, and it is the clearest single demonstration that the measure is not measuring what it claims.
The role problem within a team
A support team where one person handles escalations and three handle routine queries.
The escalation specialist resolves fewer items, spends longer on each, and does the hardest work.
Volume comparison ranks them last, and nothing in the data corrects for it.
What a valid comparison would require
Same work, same difficulty, same tools, same tenure, same working pattern, same job shape.
Which essentially never holds between two real people.
The honest conclusion is that individual comparison on this data is not available, rather than that it needs adjusting.
The aggregate exception
Comparing a team this quarter against the same team last quarter is a valid comparison, because the population is the same.
Comparing two teams doing different work is not.
Which means the useful views are temporal rather than cross-sectional, and most dashboards default to the opposite.
What to do with a genuine outlier
Ask, with an open question, about the work.
Expect the explanation to be something the data could not contain, because it usually is.
And treat the figure as the reason you asked rather than as the subject of the conversation.
The configuration answer
If individual comparison cannot be valid, do not produce it.
Disable the ranking views.
A capability that invites an invalid conclusion will produce one, and policy does not prevent what the interface encourages.
What to check
Does your deployment produce rankings?
Where do your most senior people sit in them?
Could you name two people in one team doing genuinely comparable work?
And has a ranking ever been shown to a manager?