Repetition Penalty

A repetition penalty is a decoding parameter that lowers the probability of tokens the model has already produced, to stop generation collapsing into loops. Left unpenalised, a model can fall into a groove — repeating a phrase, restating the same bullet, or emitting the same sentence until it hits the token limit — because each repetition makes the next one look more likely given the context. Implementations vary: some divide the score of any previously seen token by a constant, some subtract a fixed amount (a presence penalty), and some scale the subtraction by how often the token has appeared (a frequency penalty). The distinction matters when you tune them, because a presence penalty discourages a word from appearing twice at all, while a frequency penalty only pushes back once it becomes common. The failure mode of over-penalising is subtler than the failure it fixes. Push the value too high and the model starts avoiding words it legitimately needs to repeat — a product name, a variable, a column heading in a table, the same key in a JSON object — which produces output that reads oddly or, in structured generation, does not parse. For that reason a repetition penalty is usually a poor tool for anything with a schema: constrained decoding or a JSON mode is the right control there. Treat it as a mild default for free prose, verify it against a real sample of your own outputs, and be aware that looping is often a symptom of a weak prompt or an over-long context rather than a decoding problem.