I run different LLMs for different jobs — and migrated a live fleet when the economics changed.
Frontier models for core reasoning. Cheap tiers for scheduled crons. A cheaper model for memory extraction. A local model for high-volume work at zero marginal cost. When a billing change pulled agent usage out of a subscription and onto full API rates, I re-routed the whole fleet — without dropping the operation.
Always-on agents make token cost a real operating expense.
The fleet never sleeps — so every token has a price.
Running an always-on agent fleet on frontier models gets expensive fast. Cost isn’t a rounding error; it’s a monthly line item I have to manage like any other.
Pay-as-you-go API rates made running everything on one frontier provider unsustainable — a $250 credit top-up was roughly two to four weeks of runway at the fleet’s burn rate.
Then the provider announced that agent-SDK and third-party agent usage would be pulled out of the subscription into a separate credit pool billed at full API rates. The subsidized “free on the plan” model the fleet leaned on was going away. I had to keep an always-on operation running affordably without gutting quality.
Model selection as a routing problem — not a standardization.
I refused to standardize on one model. Each task class got the cheapest model that could do its job to standard.
Core reasoning ran on the top frontier model available. Scheduled crons ran on cheaper GPT tiers. Memory extraction ran on a cheaper Claude tier. For the efficiency layer I stood up a local Qwen 3.5 model — chosen specifically for tool-calling reliability, since its job is driving tools inside the agent runtimes — served locally at zero frontend cost.
I also built the cost guardrails: I load-balanced provider accounts to spread the burn, and I removed the metered provider from auto-fallback so a timed-out cron couldn’t silently bleed API money.
The local model was a cost decision, not a privacy one — I chose Qwen 3.5 because it drives tools reliably and runs at zero marginal cost, and I'm precise about that.
— Design note · efficiency layer rationaleThe per-task routing map.
One router, one policy: each task class flows to the model that fits it on cost and capability. A frontier model carries reasoning with a fallback wired behind it; a local model absorbs the high-volume work at zero marginal cost.
The router applies one per-task policy: core reasoning to GPT-5.5 via Codex, crons to cheaper GPT tiers, memory extraction to Sonnet 4.5, and high-volume background work to a local Qwen 3.5 model at $0 marginal cost. Opus 4.8 stays wired as a fallback, and the metered provider was pulled out of automatic fallback so timed-out jobs can’t silently bill API money.
Reasoning Layer
Cron Tier
Memory Layer
Local Efficiency Layer
Runtimes
Cost Levers
The math that drove the decisions.
Cost figures verbatim from the internal record — the operator's own accounts and the provider's published tiers. These are economics, not performance metrics.
These are cost figures, not results.
Every number above is an economics input — provider pricing tiers and the operator’s own account spend, taken verbatim from the record. There are no traffic, revenue, or quality-improvement metrics attached to this project, and none should be inferred. The work here is disciplined cost engineering with quality tradeoffs weighed explicitly — not a claim that “cheaper is better.”
Which task ran which model.
The policy in one view — task class, the model it routed to, and why that model won the slot.
| Task class | Model (post-migration) | Runtime / plan | Why this model |
|---|---|---|---|
| Core reasoning | GPT-5.5 · Opus 4.8 fallback | OpenAI Codex on subscription | Best available reasoning at a subscription rate an always-on brain can afford |
| Scheduled crons | gpt-5-mini · gpt-5.2 | OpenAI cheaper tiers | Repetitive, low-stakes jobs don’t need frontier tokens |
| Memory extraction | Sonnet 4.5 | Max OAuth (repoint to local planned) | Cheaper Claude tier fits extraction; local offload was the next move |
| High-volume background | Qwen 3.5 9B (local) | LM Studio · MLX backend | Reliable tool-calling at zero marginal cost — its job is driving tools |
| Fallback | Opus 4.8 | Wired behind the primary brain | Quality safety net; metered provider removed from auto-fallback |
Two cutovers, one live operation.
Model choice was a lever I actively pulled — per task and per cost event — not a one-time switch.
The fleet's core reasoning ran on the top frontier Claude model, subsidized under the subscription plan.
First cost-driven migration: the primary brain and all scheduled crons cut over to GPT-5.4 high, on a fresh $100 Pro 5x account.
The migration oscillated — heavier Claude use returned when quality demanded it. This is the point: routing is an ongoing cost-vs-quality optimization, not a clean one-way switch.
After the billing change, the brain moved to GPT-5.5 on a subscription plan, a local Qwen 3.5 model went live for the efficiency layer, and Opus 4.8 stayed wired as fallback — "exactly what we wanted."
A status check shows the orchestrator briefly back on Opus 4.8 — evidence that model choice stayed an active lever, not a settled decision.
I set the policy and made every cost call.
I read the billing change, ran the per-plan economics, chose the new brain, and directed both migrations while keeping the operation live.
I selected Qwen 3.5 for the local layer on tool-use reliability grounds, and I designed the cost guardrails — splitting provider accounts to spread burn, removing the metered provider from auto-fallback, and keeping Codex Fast Mode off. Model strategist and operator: I owned the routing policy and pulled the lever every time the cost or quality picture moved.