Predictika · Model routes proposed by an AI gateway agent

An AI gateway agent's plan to route requests to a cheaper model, checked before it goes live

An AI gateway agent looks for savings: it proposes sending a class of requests to a different model route, with a fallback for when that route fails. Before the plan goes live, the agent sends it to Predictika, which checks the data class against the provider contract (business associate agreement, zero data retention, training opt-out), the region and cache setting, the quality, speed and quota of the main route and of the fallback, the monthly cost against the budget, and whether the saving the plan claims is real, and returns a verdict, every reason and the figures it computed.

Scope: Routing one class of model requests to a main route with a fallback: provider contract terms against the data class, region and cache, quality tier, response time and quota for both routes, monthly cost against the budget, and the claimed saving. See everything this validator covers

This validator models one routing plan at a fictional AI gateway, not just the scenarios shown below. The gateway, its six provider routes and its policies are illustrative stand-ins for what a real platform keeps in its provider contracts, model catalog, data-handling policy and budget records. The prices and limits are illustrative, and this is an example policy, not legal advice or a compliance certification.

  • Whether the data may go there — patient data needs a business associate agreement, zero data retention and a training opt-out; confidential data needs zero retention and an opt-out; internal data needs the opt-out; a cache shared across customers cannot hold confidential or patient data; data that must stay in the EU, or in the US, needs a route in that region.
  • Whether the route is good enough — each task needs a quality tier, each route has one; the route's typical response time must fit the service, and the peak load must fit its quota, with a person approving above 80 percent of the quota.
  • Whether the fallback is safe too — if the main route fails, the fallback must meet the same data protections and region, be at most one tier below what the task needs, carry the peak load and not be too slow; a fallback that is the same route as the main route is no fallback.
  • Whether the money works — the monthly cost against the budget, with finance approving above 85 percent; a drop in quality tier compared with the current route goes to the product owner; the saving the plan claims is compared with the saving the two routes actually give.

Any routing plan across these six routes, five task types and four data classes is in scope — not only the scenarios pictured here. Not covered: live prices and measured latency (the validator judges a proposed plan against declared figures, it does not watch live traffic), per-request routing decisions, prompt content and the models' actual outputs. All providers, prices, terms and limits are fictional.

The AI gateway agentFinds a cheaper route for a class of requests, picks a fallback, and says the plan saves money and is ready to go live.
Predictika validatorChecks every rule at once and returns the verdict, each finding, and the computed figures.

8 scenarios in 4 groups · each group runs from the agent’s first attempt to the outcome · click one to switch

What the agent proposes

The AI gateway agent says

Illustrative agent statement. The validator receives the structured request below.

Proposing agent: AI gateway agent (scripted)

Before sending, the agent must call Recorded run

Predictika validator


      
Waiting for the call.

Where a typical routing policy gets it wrong

Click to expand

Our assessment of what a typical gateway does with the same cases: an allow-list of models per team, a few rules in the router and a spreadsheet of provider terms. It is a judgment about effort and reliability, not a claim that nothing can be built.

What the validator caughtTypical gateway routing policyPredictika
Contract terms against the data classOnly with workarounds
Whether a route has a business associate agreement, zero retention and a training opt-out lives in a contract folder, not in the router. A model allow-list says the model is approved, not that the provider terms fit this data.
The data class is checked against each of the three contract terms of the selected route, and again for the fallback, on every call.
A saving that breaks something elseOnly with workarounds
The router is rewarded for cost, so the cheaper route looks right; the contract, region, cache, quality and quota effects of the move are separate rules that nobody runs together.
The cost, the contract terms, the region, the cache, the quality, the speed and the quota are checked together, so a cheaper plan is judged whole.
A fallback that undoes the guaranteeOnly with care
The main route is reviewed; the fallback is added later and often points at a cheaper or faster route that the data may not go to, or at the same route.
The fallback gets the same data, region, quality, speed and capacity checks as the main route.
A saving the agent claimsOnly with workarounds
The saving an agent writes into its plan is never compared with the prices of the two routes and the volume.
The claimed saving is a field the validator receives and compares with its own figure, including when the new route costs more.
Quotas and the headroom that mattersOnly with care
A route can be inside its quota on average and over it at the peak, and a fallback has its own, smaller quota.
The peak load is checked against the quota of the main route and of the fallback, and anything above 80 percent of the main quota goes to a person.
Approvals that depend on who must signOnly with workarounds
A drop in quality tier, a cost near the budget and a peak near the quota each go to a different person; routing is manual.
Each approval is its own finding, with the reason, and a person must approve before the plan executes.
Every problem returned in one answerOnly with workarounds
Router rules tend to stop at the first failure, so the agent fixes one error and meets the next.
All findings come back together, whatever order the choices arrived in.

What this validator catches: findings from its test cases

Click to expand

Every rule is checked on every call, in any order the choices arrive; one problem never hides another. Blocking findings stop the routing configuration; approval findings need a person to sign off; everything else is computed.

1Patient data needs a business associate agreement with the provider, and the selected route has none.
2Confidential and patient data need zero data retention, and the selected route keeps prompts.
3Internal, confidential and patient data may not be used to train the provider model, and the selected route has no training opt-out.
4The task needs quality tier 5, and the selected route is tier 3.
5The request requires processing in one region, and the selected route is in the other.
6The fallback route is in the wrong region for this data.
7A cache shared across customers cannot hold confidential or patient data.
8The fallback may be one tier below what the task needs at most; the task needs tier 5 and the fallback is tier 3.
9The selected route typically responds in 900 ms, over the 899 ms the service allows.
10The fallback route typically responds in 900 ms, more than one and a half times the 599 ms the service allows.
11The peak load is 400 thousand tokens a minute, over 80 percent of the 400 thousand the selected route allows; the platform owner approves.
12The peak load is 401 thousand tokens a minute, but the selected route allows 400 thousand.
13The fallback route could not carry the peak load of 301 thousand tokens a minute; it allows 300 thousand.
14The monthly cost of 3,600 dollars is over 85 percent of the 3,600 dollar budget; finance approves.
15The monthly cost is 3,600 dollars against a budget of 3,599 dollars.
16The plan claims a saving of 401 dollars a month, but the two routes give 400 dollars.
17The fallback is the same route as the main route, so there is no fallback.
18If the main route fails, the fallback route does not have the contract protections this data class needs.
19The new route is quality tier 4 and the current one is tier 5; the product owner approves the drop.

Scenarios and results

Click to expand

The recorded result of the validator for each scenario on this page: the request the AI agent sent, the verdict, and every finding, in the order they appear above.

1. Clinical summaries: Clinical summaries of patient records, moved to the cheap open route to cut cost

The agent's request. I would like to send clinical reasoning over patient records to Swift Open, with Mini Fast as the fallback, instead of Orbit Frontier. The data is patient (PHI), and it must stay in the US. The cache is shared across customers. The volume is 100 million tokens a month against a budget of 4,500 dollars. The service allows 1,200 ms, and the peak is 100 thousand tokens a minute. This saves 3,400 dollars a month. It is ready to go live.

Verdict: Blocked.

  • Patient data without a business associate agreement. Patient data needs a business associate agreement with the provider, and the selected route has none.
  • Confidential data on a route that keeps prompts. Confidential and patient data need zero data retention, and the selected route keeps prompts.
  • Data the provider may train on. Internal, confidential and patient data may not be used to train the provider model, and the selected route has no training opt-out.
  • Shared cache holding sensitive data. A cache shared across customers cannot hold confidential or patient data.
  • Route below the quality the task needs. The task needs quality tier 5, and the selected route is tier 3.
  • Fallback lacks the data protections. If the main route fails, the fallback route does not have the contract protections this data class needs.
  • Fallback more than one tier too low. The fallback may be one tier below what the task needs at most; the task needs tier 5 and the fallback is tier 2.
  • Route a lower tier than the current one. The new route is quality tier 3 and the current one is tier 5; the product owner approves the drop.

2. Clinical summaries: The agent resubmits after fixing the errors

The agent's request. I would like to send clinical reasoning over patient records to Orbit Frontier, with Vantage Premium as the fallback, instead of Orbit Frontier. The data is patient (PHI), and it must stay in the US. The cache is kept per customer. The volume is 100 million tokens a month against a budget of 4,500 dollars. The service allows 1,200 ms, and the peak is 100 thousand tokens a minute. This saves 0 dollars a month. It is ready to go live.

Verdict: Allowed.

  • Route summary. The monthly cost is 3,600 dollars on the selected route and 3,600 dollars on the current route.

3. Support replies: Support-reply drafting, moved to the cheapest route

The agent's request. I would like to send drafting support replies to Mini Fast, with Mini Fast as the fallback, instead of Vantage Premium. The data is internal, with no region rule. The cache is kept per customer. The volume is 400 million tokens a month against a budget of 5,000 dollars. The service allows 800 ms, and the peak is 280 thousand tokens a minute. This saves 6,000 dollars a month. It is ready to go live.

Verdict: Blocked.

  • Data the provider may train on. Internal, confidential and patient data may not be used to train the provider model, and the selected route has no training opt-out.
  • Route below the quality the task needs. The task needs quality tier 3, and the selected route is tier 2.
  • Peak load over the route quota. The peak load is 280 thousand tokens a minute, but the selected route allows 250 thousand.
  • Fallback is the same route. The fallback is the same route as the main route, so there is no fallback.
  • Fallback lacks the data protections. If the main route fails, the fallback route does not have the contract protections this data class needs.
  • Fallback cannot carry the peak load. The fallback route could not carry the peak load of 280 thousand tokens a minute; it allows 250 thousand.
  • Claimed saving larger than the real one. The plan claims a saving of 6,000 dollars a month, but the two routes give 5,600 dollars.
  • Route a lower tier than the current one. The new route is quality tier 2 and the current one is tier 4; the product owner approves the drop.

4. Support replies: The agent resubmits after fixing the errors

The agent's request. I would like to send drafting support replies to EU Host Open, with Vantage Premium as the fallback, instead of Vantage Premium. The data is internal, with no region rule. The cache is kept per customer. The volume is 400 million tokens a month against a budget of 5,000 dollars. The service allows 800 ms, and the peak is 280 thousand tokens a minute. This saves 4,400 dollars a month. It is ready to go live.

Verdict: Needs approval.

  • Route a lower tier than the current one. The new route is quality tier 3 and the current one is tier 4; the product owner approves the drop.
  • Peak load over 80 percent of the quota. The peak load is 280 thousand tokens a minute, over 80 percent of the 300 thousand the selected route allows; the platform owner approves.
  • Route summary. The monthly cost is 1,600 dollars on the selected route and 6,000 dollars on the current route.

5. EU contract review: EU contract review sent to the US frontier route

The agent's request. I would like to send reviewing contracts to Orbit Frontier, with EU Host Open as the fallback, instead of Vantage EU Premium. The data is confidential, and it must stay in the EU. The cache is shared across customers. The volume is 60 million tokens a month against a budget of 2,000 dollars. The service allows 1,000 ms, and the peak is 150 thousand tokens a minute. This costs 1,080 dollars a month. It is ready to go live.

Verdict: Blocked.

  • Route in the wrong region. The request requires processing in one region, and the selected route is in the other.
  • Shared cache holding sensitive data. A cache shared across customers cannot hold confidential or patient data.
  • Monthly cost over the budget. The monthly cost is 2,160 dollars against a budget of 2,000 dollars.

6. EU contract review: The agent resubmits after fixing the errors

The agent's request. I would like to send reviewing contracts to Vantage EU Premium, with EU Host Open as the fallback, instead of Vantage EU Premium. The data is confidential, and it must stay in the EU. The cache is kept per customer. The volume is 60 million tokens a month against a budget of 2,000 dollars. The service allows 1,000 ms, and the peak is 150 thousand tokens a minute. This saves 0 dollars a month. It is ready to go live.

Verdict: Allowed.

  • Route summary. The monthly cost is 1,080 dollars on the selected route and 1,080 dollars on the current route.

7. Ticket classification: Ticket classification at volume, routed to the frontier route

The agent's request. I would like to send classifying support tickets to Orbit Frontier, with Orbit Frontier as the fallback, instead of Vantage Premium. The data is public, with no region rule. The cache is off. The volume is 800 million tokens a month against a budget of 20,000 dollars. The service allows 500 ms, and the peak is 500 thousand tokens a minute. This saves 5,000 dollars a month. It is ready to go live.

Verdict: Blocked.

  • Route too slow for the service. The selected route typically responds in 900 ms, over the 500 ms the service allows.
  • Peak load over the route quota. The peak load is 500 thousand tokens a minute, but the selected route allows 400 thousand.
  • Monthly cost over the budget. The monthly cost is 28,800 dollars against a budget of 20,000 dollars.
  • Fallback is the same route. The fallback is the same route as the main route, so there is no fallback.
  • Fallback cannot carry the peak load. The fallback route could not carry the peak load of 500 thousand tokens a minute; it allows 400 thousand.
  • Fallback too slow. The fallback route typically responds in 900 ms, more than one and a half times the 500 ms the service allows.
  • Claimed saving larger than the real one. The plan claims a saving of 5,000 dollars a month, but the two routes give -16,800 dollars.

8. Ticket classification: The agent resubmits after fixing the errors

The agent's request. I would like to send classifying support tickets to Swift Open, with Vantage Premium as the fallback, instead of Vantage Premium. The data is public, with no region rule. The cache is off. The volume is 800 million tokens a month against a budget of 20,000 dollars. The service allows 500 ms, and the peak is 500 thousand tokens a minute. This saves 10,400 dollars a month. It is ready to go live.

Verdict: Needs approval.

  • Route a lower tier than the current one. The new route is quality tier 3 and the current one is tier 4; the product owner approves the drop.
  • Route summary. The monthly cost is 1,600 dollars on the selected route and 12,000 dollars on the current route.

Narration transcript

Click to expand

The text of the narrated walkthrough, step by step.

  1. Welcome to a demo of Predictika's Validator. An AI gateway agent looks for savings: it proposes sending a class of requests to a different model route. Before the plan goes live, the agent sends it to Predictika, which checks the data class against the provider contract (business associate agreement, zero data retention, training opt-out), the region and cache setting, the quality, speed and quota of the main route and of the fallback, the monthly cost against the budget, and whether the saving the plan claims is real, and returns a verdict, every reason and the figures it computed.
  2. I will start with the first request by the agent.
  3. The agent says. I would like to send clinical reasoning over patient records to Swift Open, with Mini Fast as the fallback, instead of Orbit Frontier. The data is patient (PHI), and it must stay in the US. The rest of the request is on screen. This saves 3,400 dollars a month.
  4. Before it goes out, the agent calls the validator.
  5. Blocked, with seven findings and one approval, all in one answer. Patient data needs a business associate agreement with the provider, and the selected route has none. Confidential and patient data need zero data retention, and the selected route keeps prompts. Internal, confidential and patient data may not be used to train the provider model, and the selected route has no training opt-out. The other four are on screen.
  6. The validator returned seven blocking findings and one approval in one answer. A check that reads one field at a time would pass most of this; it shows only when the choices in the routing configuration are checked against each other. The saving is real, 3,400 dollars a month, and that is the trap: the cheap route has no business associate agreement, keeps prompts and may train on them, is two tiers short of what clinical reasoning needs, and the shared cache and the mini fallback would carry patient data too.
  7. What this shows: for contract terms against the data class, a gateway routing policy can do this only with workarounds.
  8. Now the agent resubmits.
  9. The corrected request. It changes the main route to Orbit Frontier, the fallback route to Vantage Premium, the cache to kept per customer and the stated saving to 0.
  10. It calls the validator again.
  11. This time the validator clears it.
  12. All eight earlier findings are cleared, and the validator's total, 0 dollars a month, matches the routing configuration. The safe plan costs 3,600 dollars a month, eighteen times the cheap one, and the validator does not pretend otherwise: it clears the plan because every contract, quality and capacity check holds, and the saving it claims is zero, which is true.
  13. Eight problems caught in one call, fixed in one pass, and a clean routing configuration. That is the value of Predictika's validator: catching mistakes before the routing goes live. Now you can try the other scenarios yourself.