Ground Truth

Ground truth is the reference answer a system's output is scored against — the label that says what the correct result for this case was. The phrase implies something discovered, like bedrock, and the most useful thing to know about it is that it is manufactured. Someone decided, under a set of instructions, at a particular time, and the score you later report inherits every property of that decision. For a narrow task the manufacture is uncontroversial: an invoice total either matches the invoice or it does not, and two careful people will agree. For most of the work an AI feature is actually asked to do, it is not. Was that summary faithful? Was that reply appropriate to an annoyed customer? Was that classification the right one when the message genuinely contains two intents? These have several defensible answers, and the moment a set of them is frozen as truth, disagreement stops being visible and starts being counted as error. That has three consequences worth acting on. Record who decided and under which version of the instructions, because a label is a judgement with a date and the instructions change. Keep the disagreements rather than discarding them; cases where trained people split are the most informative rows in the set, and averaging them away removes exactly the material that would tell you where the task is genuinely ambiguous. And separate the two failures a low score can represent, since a system marked wrong may have produced an answer a reasonable person would accept while the reference happens to record a different one. Where the truth is contested, say so on the page rather than in a footnote, and report the ambiguous cases separately from the ones with a single defensible answer. The same caution applies when a reference set arrives from outside. A set someone else built encodes their instructions, their edge-case decisions and the moment they were made, none of which is visible in the file, so the first useful thing to do with it is to have two of your own people label a sample and see whether they agree with it.