Model Versioning

Model versioning is the discipline of identifying exactly which model produced a given behaviour, and of pinning that identity in the code path rather than accepting whatever the provider currently serves under a friendly name. It is the difference between a system whose behaviour changes when you change it and one whose behaviour changes on a schedule you do not control. The need is easy to underestimate because nothing appears to break. A hosted model referenced by an alias can be updated underneath you, and the update is usually an improvement in aggregate. Your product does not experience aggregates. It experiences the prompts you wrote against the previous behaviour, the output formats your parser expects, the refusal boundaries your users learned, and the edge cases you fixed by wording things a particular way. An upstream change can improve a benchmark and break the one workflow you depend on, and because your code did not change, nobody looks there first. Pinning to an explicit version is therefore the default worth adopting. That version identifier belongs in configuration, gets recorded with every stored output, and appears in evaluation results, so that a shift in quality can be traced to a model change rather than argued about. When an output is logged without the version that produced it, the log is a record of what happened and not of why, which is exactly half of what an incident investigation needs. A pin is only useful with a plan for leaving it. Hosted versions are retired, and a pinned version with no migration path becomes an expiry date you have not diarised. The routine is unremarkable and works: watch deprecation notices, run the current evaluation set against the candidate version, compare on your data rather than on published claims, migrate behind a flag, and keep the old version available for as long as the provider allows so a rollback is a configuration change rather than a rewrite. Versioning applies to more than the model weights. The prompt, the retrieval configuration, the tool definitions and the sampling parameters all shape behaviour, so the thing worth versioning is the whole configuration, treated as one deployable artefact. A change to any part should produce a new identifier, because a system where the prompt can move independently of the model version is one where two teams can each be certain nothing changed. Self-hosted deployments trade one problem for another. Nothing moves without you, which removes the surprise, but you now own the weight files, the quantisation, the serving stack and the inference parameters, and any of those can alter output. The same rule applies: the version is whatever combination actually produced the behaviour, and it is worth being able to name it exactly.