
I do not choose models from a comparison chart. Routing gets decided by the walls I hit with a vendor: a feature that does not ship, an outage on the one day I cannot afford one. What we run today is one gateway under everything, a circuit breaker for the bad day, and each model on the job it does well for what it costs.
Routing starts with a gap
For six months I asked two vendors for the same feature: scheduled execution. That means recurring agent runs, unattended, started from the command line. Both vendors collect feature requests in public, and I asked in public, more than once. Other things I asked for did arrive, in time. Loops did not, on either side, and after long enough building them stopped being optional.
So we built a local scheduler. It survives restarts, records every run and drives whichever coding agent is doing the work that day through a guarded adapter.
The wait taught me a second thing. How long a gap stays open after it has been clearly flagged tells me which vendor to lean on for what.
Using several providers is a hedge. Every vendor will eventually rate-limit you, ship something later than you need it, or go down on the wrong day. A setup with one provider has no answer for that day.
One gateway under everything
The coding tool I use every day sits on a gateway. Clients hold one gateway key, and the provider keys live behind it. Traffic goes to whichever compatible providers are configured, chosen by price or speed, and the calling application does not need to know which one served a request. The routing lives in one place instead of being copied into every system that needs a model.
Policy sits next to the routing. A request should never be sent quietly to a region or a class of provider the caller did not approve. The default stays local-first: no hosted calls at all unless someone configured them on purpose.
The circuit breaker matters only on the bad day, which is why it has to be built before that day. Where a system can use either the user’s own API key or a managed one, it first lists the routes that work: which keys are available, which providers are healthy, which are cheaper. Then the breaker takes a provider that is having a bad afternoon out of rotation, so it does not take every request down with it.
Which model should do which job?
I assign models to roles on purpose. A fast model is the executor and does the coding. A model I trust to plan and debug across a larger surface orchestrates. The cheapest model in the stack does quality assurance, because that check needs discipline about one narrow thing more than it needs a frontier model.
That cheap check works like this. After every turn a hook gives it three things: the last user prompt, the last agent message and the files that changed. It reads them, then either adds tasks the agent missed or stops the agent from declaring the work finished too early.
Inside our orchestration framework the split runs the other way. Cheap models produce: they write the code, run the test, fill the form. A stronger model reviews the result, and it has exactly one way to approve: an explicit tool call that is logged and can be checked later. A paragraph saying the work looks good does not count. The cheap tier fires constantly and the reviewer fires once per meaningful unit of work, which is what makes it affordable. The reviewer must also be a different model from the one it checks, because a model reviewing its own output tends to agree with itself. I have written about this shape in Actor and verifier loops.
Both arrangements rest on one rule. Decide which part of the work is judgment and which is grind, and price them differently on purpose.
Vendor philosophy is a routing input too. OpenAI’s approach makes you define the tools, the execution logic and the state management yourself. You get more control and more code to own. Claude’s approach works out more of it for you, with built-in tools that work immediately. A structured workflow with heavy compliance suits the first. An exploratory task suits the second.
Tools earn their place by measurement
Tools get routed the same way. Someone asked me in public which MCP servers, the tool servers an agent connects to, actually matter. I named five: web search, browser automation, a developer-tools inspector, a UI-component server and, optionally, a documentation server. Everything else should be a command-line tool, because every server wired into an agent spends context describing itself before the agent has done any work.
A model or a tool earns its place in the routing table by a measured number.
On one project we replaced the shell tool with a thinner execution path based on JavaScript. It measured out to roughly half as many tool calls and, on some batch actions, up to seventy percent fewer tokens. That number justified the swap, the same way evals justify a choice of model, the subject of What evals are for.
The vendors will keep shipping on their own schedule. The measuring and the routing stay with me.


