SaaS AI Cost Control in 2026: Caching, Routing, Fallbacks

Your provider invoice and your MRR dashboard are getting closer together, and nothing broke. Usage grew, and model spend grew with it, linearly, because that is how per-token pricing works. Before you read further, open the last invoice and find one number: how many of your input tokens were cache reads. If you can't find it in under a minute, that is your first task.

Most advice on LLM cost optimization stops at "set a usage limit." A limit is a circuit breaker. It stops the worst month but leaves the average one untouched. The savings sit in four layers underneath it: a prompt cache, a semantic cache trusted with one narrow slice of traffic, a router that sends most requests to a cheap model, and a fallback chain that degrades instead of failing.

This post walks through those layers in the order I'd build them, with prices and limits checked against vendor documentation on 30 September 2026. Prices change often, so every number below carries its source and you should re-check it before you budget.

Why does LLM cost optimization fail at the usage-limit stage?

A usage limit caps your worst month without lowering your average one, because the cost sits in the shape of every request rather than in how many requests you send.

Consider a typical request in a B2B product: a system prompt, a tool schema, tenant configuration, a few thousand tokens of retrieved context, then a user message of forty tokens. You pay full input price for all of it, and the first four parts are identical to the last ten thousand requests. That redundancy is the leak. Volume is only the multiplier.

There is also a hard-stop version of the limit you should know about. Anthropic's rate-limit documentation says that when your organization reaches its monthly spend cap, requests return HTTP 429 with no retry-after header, and retrying, including the SDKs' automatic retries, fails until access resumes. A cap protects your budget. It also switches your product off for every tenant at once, so treat it as the last line and not the plan.

How do prompt caching and LLM model routing cut the bill?

Prompt caching bills a repeated prefix at a fraction of the input price, and LLM model routing sends each request to the cheapest model that can handle it, and both work without changing what the product does.

What does Anthropic prompt caching pricing look like?

Anthropic's pricing page lists a five-minute cache write at 1.25x the base input price, a one-hour write at 2x, and a cache read at 0.1x for most models, with 0.05x on Claude Opus 5.5 and 0.025x on Claude Fable 5.1 and Claude Mythos 5.1. The same page says caching pays off after one read for the five-minute duration and after two reads for the one-hour duration. OpenAI's prompt-caching guide describes a similar shape: caching is on by default, and on GPT-5.6 and later a cached read costs 0.1x the uncached input rate, with a 1,024-token minimum.

Where do I place the breakpoint so the cache actually hits?

Put the breakpoint on the last block that is identical across requests. Anthropic's prompt-caching documentation warns about the common mistake: a breakpoint on a block that contains a timestamp or the user message writes a fresh cache entry on every request and never gets a read, so you pay the write premium for nothing. Three more rules from the same page matter in production:

1. Invalidation runs in order. Prefixes are built as tools, then system, then messages, and a change at one level invalidates that level and everything after it. Editing a tool definition throws away the cache for the whole prompt.
2. Short prompts don't cache, and nothing tells you. The minimum cacheable prefix is 512 tokens on some models and 4,096 on others, and below it the request is processed without caching and no error is returned.
3. You get four breakpoints. Automatic caching uses one of them if you combine it with explicit ones.

The second benefit doesn't appear on the pricing page. For most Claude models, cache read tokens don't count toward the input-tokens-per-minute limit, and Anthropic's own example describes a 2,000,000 ITPM limit with an 80% cache hit rate handling about 10,000,000 input tokens per minute. Claude Haiku 3.5 is an exception, but it is retired on Anthropic's own API. Verify the win by checking that cache_read_input_tokens is non-zero in the response usage, not by trusting the config.

Which model should get the request?

Route on a signal you already have, default to the cheap tier, and escalate only on evidence that the cheap answer failed.

The research version is RouteLLM, where Ong and colleagues train routers on preference data and report that costs can fall by over 2 times in certain cases without compromising response quality. The production version is usually simpler, because you know things a benchmark can't: which feature made the call, which plan the tenant is on, and whether a tool result is already in context.

The price gaps are what make routing worth the effort. Base input and output prices per million tokens on Anthropic's pricing page:

The same pattern shows up at OpenAI. Its pricing page lists GPT-5.6 Luna at $0.20 input and $1.20 output, Terra at $2 and $12, and Sol at $4 and $20, and it now also lists a GPT-6 line. Catalogs move fast enough that a threshold tuned last quarter may be tuned against arithmetic that has stopped being true.

A routing rollout that holds up:

1. Ship the cheap tier as the default. A router that defaults to the expensive model stays there, because nobody gets paged for spending money.
2. Escalate on logged evidence. A structured-output parse failure, a refusal, a low confidence field, or a user pressing regenerate are all signals you can record today.
3. Track escalation rate per feature. A feature that escalates most of the time needs pinning to the strong model and pricing accordingly, not a cleverer router.
4. Re-run the arithmetic every quarter, the way you'd renew a TLS certificate.

Is a semantic cache worth building?

Only for a bounded set of questions, and only with an evaluation suite, because the threshold that makes it profitable is also the one that makes it wrong.

Portkey's April 2026 threshold analysis reports a benchmark that makes the trade-off concrete:

Read the accuracy column twice. Raising the threshold from 0.80 to 0.99 cut savings from 84.6% to 15.8% while accuracy moved by well under one point. In that benchmark, then, a stricter threshold mostly buys you fewer hits, not fewer wrong answers. This suggests a semantic cache needs a domain filter and a review step, not just a higher number. Portkey's own guidance is to start in the 0.90 to 0.95 range and to aim for accuracy above 99% in production.

The failure mode is the real cost. A bad semantic hit returns a plausible answer with HTTP 200, so no error rate exists to alert on. A prompt-cache miss just costs you a normal call. The two layers deserve very different levels of trust.

Where it earns its place is a bounded question space, such as support macros or glossary lookups, or the stable set of questions new tenants ask in their first week in a B2B SaaS product. Cache those and send everything else past it.

What stops one tenant from eating the whole AI budget?

Enforce per-tenant ceilings on the server, before the model call, with an hourly limit and a daily one, and keep a fallback that degrades instead of retrying.

On Callidus OS, the multi-tenant clinic SaaS I built solo on React, TypeScript and Firebase, the case study documents the Calia assistant running on Vertex AI through the @google/genai SDK, with 30-per-hour and 100-per-hour rate limits, thirteen injection regex patterns and a 90-day GDPR cleanup. Everything lives under tenants/{tenantId}/ in Firestore, with tenantId and role carried in JWT claims. That gives the budget check a tenant to attribute spend to before it does anything else. Consider a hypothetical: on a £15/month plan, one enthusiastic user running ninety queries in an afternoon is a real cost you can only size if you know your per-query price. Enforce the limit client-side and you've written a suggestion, not a limit.

How should fallbacks behave when a provider says stop?

Back off with jitter, switch provider, then return a degraded answer that says it is degraded. OpenAI's rate-limit guide recommends exponential backoff with jitter and warns that unsuccessful requests count toward your per-minute limit, so resending in a tight loop cannot work. Anthropic's 429 responses carry a retry-after header for ordinary rate limits, and honoring it beats guessing. The saving hides in the last step: a graceful "try again shortly" costs nothing, and a retry storm bills you repeatedly for answers nobody received.

The metric that matters is AI cost per tenant per month, sorted descending, because the tail is where margin goes. For the broader question of which features deserve metering at all, see AI SaaS solutions for business. If you are still choosing what to build on, the SaaS MVP stack guide covers the layer below.

In what order should I build all this?

Instrument first, cache second, cap third, route fourth, add fallbacks fifth, and treat the semantic cache as optional.

1. Instrument. Log input, output and cached tokens, model, tenant and feature on every call. Without this everything else is guesswork.
2. Turn on prompt caching. Highest saving per hour of work, no product change.
3. Add per-tenant ceilings. They convert your worst-case bill from unbounded to arithmetic.
4. Add routing, cheap tier first, and watch escalation rates for two weeks before touching a threshold.
5. Add fallbacks with backoff and jitter.
6. Consider a semantic cache last, for a bounded question set, with a backtested threshold. The same tooling-moves-faster-than-your-assumptions discipline runs through my AI-augmented development workflow.

Open your most recent invoice and calculate cache reads as a share of total input tokens. What is the number, and which feature is responsible for the biggest gap between it and 100%?