Feedback Signal

A feedback signal is any evidence about whether a piece of AI output was useful. The explicit kind is the thumbs up and down attached to a response; the implicit kind is everything the user did next — copied it, edited it, asked the same thing again in different words, escalated, or left. The implicit signals are more plentiful and usually more honest. Explicit ratings have a shape worth knowing before you build on them. Almost nobody rates, so the volume is a small and self-selected slice. Negative ratings dominate, because irritation motivates a click and satisfaction does not. And the button conflates several different complaints: wrong, slow, badly formatted, correct but not what I meant, and I disagree with the policy behind it. A raw ratio computed from that is not a quality measure, though a sharp change in it is still a useful alarm. Implicit signals carry more information and need more care. A user rephrasing the same question within a minute is a strong sign the first answer failed; a heavy edit before sending is a partial failure; an immediate escalation to a person is close to a definitive one. Each needs to be defined once and instrumented deliberately, because if the events are not planned they will be reconstructed later from whatever happened to be logged, which is how teams end up measuring their interface rather than their model. The reason to collect any of it is to make a labelled set. Individually these signals are noisy, but the disliked and rephrased cases are a queue of real failures with real inputs, and reviewing a sample of them yields evaluation cases and prompt fixes that no synthetic test set would have produced. That pipeline — signal, sample, review, add to the evaluation — is the whole value. Two habits protect it. Ask for the reason alongside the rating in a form somebody will actually complete, and close the loop visibly at least sometimes, because a feedback control that never changes anything trains users to stop using it, at which point the signal thins out exactly as the system gets busier.