How to Define a Metric So Two Teams Get the Same Number

The argument is about the definition, not the data

Two people open two dashboards, both built honestly on the same warehouse, and read out different figures for the same week. Nobody has broken anything. The events fired, the pipeline ran, both queries are correct. They are counting different things and calling them by the same word, and the meeting that follows is spent reconciling arithmetic instead of deciding anything.

This guide is about writing the definition down so that stops happening. It is not about instrumentation — where events fire, what a tracking plan contains, which tool collects them — and it is not about choosing which metrics your company ought to care about. Assume the data arrives and that somebody has already decided the number matters. The work here is saying precisely what it is.

Start with the denominator

Most metrics that cause arguments are ratios, and almost all of the disagreement lives underneath the line. A conversion rate measured over visits, over people, over people who reached the relevant page, and over people who were eligible to buy are four different numbers, and every one of them is defensible. What is not defensible is leaving it unstated, because each reader will quietly fill the gap with the denominator they would have chosen.

Write the population first, in a sentence someone outside the team could apply without a follow-up question. Then write what is being counted within it. The order matters: teams that start from the numerator tend to define the denominator as wherever the numerator came from, which is how a rate ends up computed over the very group the numerator selected, and never falls.

Decide what counts as one

Every count needs a unit and a rule for collapsing repeats. Is somebody who signs up twice with two addresses one customer or two? If a support conversation moves to a second channel, is that one case or two? Does a person using three devices count once? These sound like edge cases until somebody measures what share of the total they are.

Double counting rarely arrives as a mistake in a query. It arrives when two sources are added together — the app and the website, self-serve and sales-assisted, two regions that both claim the same multinational customer — where the overlap is small enough to escape notice and large enough to change a decision. If a number is a sum across sources, the definition has to say how the overlap is handled, even when the honest answer is that it currently is not.

Fix the clock

A date is not one thing. An event has the moment it happened, the moment it reached your systems, and the moment the row was written or last updated, and reports built on each will disagree. They disagree most visibly at the boundaries, which is exactly where months and quarters are read and targets are judged. Say which timestamp the metric uses, and say it in a way that survives the table being rebuilt.

Say the time zone too, and choose one for the whole company rather than letting each report follow the viewer's. And where one event is attributed to an earlier one — a purchase to a campaign, a renewal to an onboarding cohort — the length of that window is part of the definition, not a setting buried in whichever tool computes it.

Say when the number is final

Almost no figure is complete on the day it first appears. Payments settle, refunds land, a queue backs up overnight, a partner reports weekly, an offline client syncs when it reconnects. The most recent bucket of any chart is therefore the lightest, and a team reading a fresh dashboard every morning will keep seeing a decline that is only the shape of arrival.

The definition should carry a maturity: how long after the period closes the figure stops moving, and how far back a later correction is expected to reach. Then show an incomplete period as incomplete rather than cropping it off the chart, which is the usual workaround and removes precisely the days people are asking about.

Name the exclusions, because they are the argument

Every real metric excludes something: internal accounts, test data, automated traffic, employees, sandbox environments, refunds and chargebacks, cancelled orders, one enormous customer whose behaviour swamps everyone else. These decisions are usually made once, by whoever wrote the first version of the query, and then inherited in silence by people who never knew a decision was made.

List them explicitly and keep the list where the number is read. A reader who knows what was left out can judge whether the figure answers their question; a reader who does not will assume it covers everything, and that assumption is always more generous than the truth. The same applies to sampling: if the figure comes from a share of the data rather than all of it, that belongs in the definition rather than a tooltip.

Give it one owner and a way to change

A definition with no owner is a definition that forks. Name a person accountable for each headline metric — not a committee, and not whichever analyst picks up the request — and make changing it their decision. Then agree in advance what happens to history when it does change, because the change itself is rarely the hard part.

Restating a number is legitimate and often correct; restating it silently is what costs you the room's trust. If a corrected definition moves last quarter's figure, publish both and say what changed. Keep a dated log of every change to every headline metric. It takes minutes to maintain and it answers the question that otherwise consumes an afternoon: whether the number moved, or whether the definition did.

Put the definition where the number is

A definition living in a document nobody opens is not doing the job. Attach it to the chart: one sentence carrying the population, the thing being counted, the clock, the exclusions and the owner, visible beside the figure it describes. If your tooling lets a metric be defined once and reused, better still: the closer the definition sits to the query, the fewer chances there are to reimplement it slightly differently.

The test of whether any of this worked is worth actually running. Ask two people from different teams to produce the same number independently, from the definition alone. If they agree, you have a definition. If they do not, you have found where it is ambiguous — cheaper to discover in an exercise than in the meeting where a decision turned on it.