How to Move Between AI Vendors Without a Rewrite

Why this comes up sooner than teams expect

The model you build on will not be the model you run in two years. Providers deprecate old versions on their own schedule, prices move, capability leadership changes hands, and a customer eventually asks for a deployment in a region or an environment your current vendor does not serve. None of this is a crisis if the switch costs a week. It becomes a crisis when provider-specific details have spread through the codebase and nobody can say what would break.

Put the provider behind one door

The single most effective decision is to keep all calls to a model behind one internal interface. Your application asks for a completion, a classification or an embedding; one module knows which vendor and which SDK answers. This is not a large abstraction and it should not try to be a universal wrapper — it just has to be the only place where a vendor's names, parameters and response shapes appear.

The two things that most often leak past this line are error handling and streaming. Rate-limit responses, retry semantics and streaming chunk formats differ between providers, and code that branches on a vendor's specific error type or parses its stream format directly is code you will rewrite. Normalise both at the boundary: your application should see your own error types and your own stream events.

Own your prompts, examples and evaluations

Prompts are portable assets and should live in your repository, versioned, not pasted into a vendor console. The same applies to few-shot examples and any schema you ask the model to fill. When you move, these are the inputs you re-test — and they are also the thing that most often needs tuning, since two models rarely respond identically to the same wording.

The evaluation set is the piece that makes a migration decidable at all. With a fixed set of realistic inputs and written acceptance criteria, switching is an experiment: run both, compare, decide. Without one, the question "is the new model good enough?" has no answer except opinion, and teams either stay put out of fear or move and discover the regressions from customers.

What is genuinely hard to move

Be honest about the parts that are not just a client swap. A fine-tuned model does not transfer — a new provider means retraining, which needs the dataset you hopefully kept. Embeddings are worse than they look: vectors from different models are not comparable, so changing embedding model means re-embedding and re-indexing your entire corpus, and running both during the transition if you cannot take downtime. Anything relying on a provider-specific feature — a hosted assistant abstraction, a proprietary tool-calling extension, a caching mechanism with its own semantics — is a rewrite of that feature, and it is worth knowing which of those you have adopted before you need to leave.

The cheap insurance

Three habits keep the door open at almost no cost. Make the model an environment-level setting rather than a constant, so you can point at a different one without a deploy. Log the model name and version with every request, so that when quality shifts you can tell whether the model changed under you. And run your evaluation set against one alternative provider occasionally — quarterly is plenty. That single exercise tells you whether the escape hatch actually works, converts an assumption into a measurement, and usually surfaces one or two leaks past the boundary while they are still cheap to fix.

Routing is a strategy, not just a fallback

Once the interface exists, sending different tasks to different models becomes an option rather than a project. Simple classification does not need your most capable model, and a cheaper one often matches it on narrow work while costing a fraction. That is worth doing on its own merits, and it has a useful side effect: a system already talking to two providers has proved it can talk to a third, and the failure of any one of them stops being an outage.