Guardrail Metric

A guardrail metric is a measure you watch not to improve but to make sure it does not get worse. Every optimisation target can be moved by means nobody intended: signups rise when the pricing page hides the price, activation improves when a step is skipped that prevented refunds later, engagement climbs when notifications become intrusive. Guardrails are the counterweights declared in advance, so a change that wins on the primary metric and loses somewhere important is recorded as a loss rather than a success. Useful guardrails are chosen per experiment from the plausible side effects of the specific change, plus a small standing set the whole team watches: refund or chargeback rate, support contact rate, unsubscribe rate, page latency and error rate. They are usually evaluated differently from the primary metric — you are not trying to prove an improvement, only to detect a degradation large enough to matter, so a threshold agreed beforehand is more useful than a significance test that a low-traffic guardrail will rarely clear. Two habits keep them honest. Write them down before the experiment starts, because a guardrail chosen afterwards tends to be one that happens to look fine. And define the action in advance: which guardrail movements block a rollout, which require a follow-up, and who decides. Without that, a guardrail is a chart someone screenshots after the decision has already been made.