How to Evaluate AI Output Quality Without a Data Team

Why impressions are not enough

The usual way an AI feature is judged is that someone changes the prompt, tries it three times, likes what they see, and ships. This fails for a specific reason: model output varies between calls, the three inputs you happen to try are rarely the hard ones, and nobody remembers what the previous version did on the same input. Without a fixed set of inputs and a consistent way of scoring them, you cannot distinguish a real improvement from a good mood, and you will not notice the day a change makes something else worse.

Start with twenty real inputs

The single highest-value thing a small team can do is collect a test set of real inputs. Twenty is enough to start and far better than none. Take them from actual usage rather than inventing them, and deliberately include the awkward ones: the empty field, the very long input, the request in another language, the question just outside what the feature is meant to handle, the one that produced a complaint. A set made only of well-behaved examples will tell you everything is fine.

For each input, write down what an acceptable answer looks like. Not the exact words — a short description of the properties that matter. "Names the correct plan tier, does not invent a price, stays under four sentences." That description is what makes scoring repeatable by someone who did not write the prompt.

Score in a way you can repeat

Use a small number of pass/fail checks rather than a ten-point quality feeling. Three to five criteria per output works well: is it factually grounded in the supplied context, is it in the requested format, does it refuse when it should, is the length within bounds. Binary criteria are far more consistent between people and between weeks than a score out of ten, and they tell you what broke rather than only that something did.

A spreadsheet is a legitimate tool here. Inputs down the side, one column per version, a cell per criterion. Most teams do not need an evaluation framework until they are running this loop often enough to find it tedious — which is the right moment to automate it, not before.

Let the model do the first pass, but check it

Once the criteria are written down, a model can apply them to its own output at a fraction of the cost of a human read. This works well for mechanical checks — format, length, whether a required field is present, whether a claim appears in the supplied context. Before trusting it, score thirty outputs both ways and see how often the model agrees with you. If it agrees most of the time, use it for the bulk and read a sample by hand. If it does not, the criteria are probably ambiguous, and fixing them helps the humans too.

Interpreting a result honestly

Two habits will keep you from fooling yourself. First, run the whole set on both versions, not just the cases you were trying to fix — the most common outcome of a prompt change is that it fixes three inputs and breaks two others, and only a full run reveals that. Second, remember that twenty inputs is a small sample: a change from sixteen passes to seventeen is not evidence of anything. Treat small movements as noise and look for changes big enough to see, or grow the set.

Keep the loop alive

An evaluation set is only useful if it is run. Attach it to the moments that already exist: before a prompt change goes live, when you switch models, and on a fixed schedule regardless. Add every reported failure to the set as a new case, which turns support complaints into permanent regression coverage. And revisit the set every few months — real usage moves, and a test set that reflects last year's inputs quietly stops measuring the product you now have.