Automatic model routing

What it does

If the seller it picked fails before the first byte, the request moves to the next seller, then to the next model. The response names the model that served it: the JSON model field and the x-inferit-model header. The router chooses from the request's size and features, never from what the prompt says.

auto is a model id like any other, so OpenAI-compatible clients can send it: the OpenAI SDKs and OpenCode. In a client that has its own Auto mode, such as Cursor, name the model inferit/auto so the two cannot be confused. Everything else works as it does for a named model: streaming, API keys and their limits, x402, the minimum discount.

curl https://api-production-c74b9.up.railway.app/v1/chat/completions \
  -H "Authorization: Bearer $INFERIT_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "qwen/qwen-2.5-14b-instruct", "messages": [{"role": "user", "content": "Summarise this diff."}]}' \
  -D - -o reply.json | grep -i '^x-inferit-'
# x-inferit-model: the model that served it (with auto, the one Inferit picked)
# x-inferit-route: auto:balanced (direct for a named model)
# x-inferit-auto-choice: 1 (model auto only)

Live now

The models model auto can choose from at this moment, in its order, with each one's AA index and its price for the reference request. The card reads GET /v1/routing and the market when this page loads; until it has a live pick it shows the example request.

Policies

• Ids are matched without regard to case, after trimming. The prefix inferit/ is accepted (inferit/auto, inferit/auto:cheap) for clients that want a vendor-qualified id.
• Any other id that starts with auto: or inferit/auto answers 400 invalid_auto_policy. No catalog model is named auto, and a seller cannot list it.
• GET /v1/models lists the three auto rows first (owned_by: "inferit"), each with an auto object: policy, pick, candidates and ranking. The pick is computed for a reference request of 3,000 input and 1,000 output tokens, so it is indicative: a request with other needs may get another model.

How a model is chosen

1. Fit

A catalog model is a candidate for a request when every check passes. They run in this order; the first that fails is the model's reason in an error:

2. Score

For each candidate m, C is the all-in estimate of this request at the model's cheapest healthy offer within your limits (the input estimate and the output limit, in µ), and Q is its quality score, an integer from 0 to 100. With C_min the lowest candidate estimate:

value(m) = Q(m) − λ × log2( max(C(m), 1) / max(C_min, 1) )

In words: with auto, a model that costs twice as much for this request must score 6 points higher to win. λ is a product setting, not a measurement; a change to it gets a changelog line and an update to this page.

Worked example. Model A scores 40 and costs 168 µ for the request; model B scores 52 and costs 600 µ. log2(600 / 168) = 1.84, so B's value is 52 − 11.0 = 41.0, above A's 40: auto picks B, auto:cheap picks A, auto:best picks B. Had B cost 1 000 µ, its value would be 52 − 6 × 2.57 = 36.6, below 40, and auto would pick A.

3. Attempts

The request goes to the first model's offers in rank order (the cheapest healthy seller first), then to the second model's, then to the third's: at most 3 models, and at most 3 attempts in all by default. A failing seller is replaced by the same model's next seller before the request moves to another model. Failover happens only before the first byte; nothing is retried once content has started streaming. The same market, body and headers always give the same order.

At equal price, a seller that answers with no cold start goes before one that has to wake up; it never goes before a cheaper one. A seller that sleeps when idle gets its declared wake time on top of the first-byte wait, 140 seconds in all at most (availability).

x-inferit-objective: latency keeps its meaning inside a model (it orders that model's sellers within a 10% price band). It does not change which model is chosen.

Hints

Optional limits, sent as body fields (in the OpenAI SDKs: extra_body) or as headers, for clients that cannot add body fields but can send provider headers, such as OpenCode. If both are sent, the stricter value wins.

• Strictest wins: max_price_per_m takes the smaller value, min_context and min_quality the larger.
• A bad value answers 400 invalid_request, naming the field. min_context or min_quality with a named model answers 400 invalid_request: they apply to model auto only.
• Hint fields never reach a seller: the API forwards only the chat fields a seller needs.
• Units: µ, one millionth of a dollar; 400000 per 1M tokens is $0.40 per 1M tokens.

import os
from openai import OpenAI

inferit = OpenAI(base_url="https://api-production-c74b9.up.railway.app/v1", api_key=os.environ["INFERIT_API_KEY"])
reply = inferit.chat.completions.create(
    model="auto:cheap",
    messages=[{"role": "user", "content": "Extract the dates from this text: ..."}],
    # Hints are body fields; clients that cannot add body fields send the x-inferit-* headers instead.
    extra_body={"min_context": 32000, "min_quality": 5, "max_price_per_m": 400000},
)
print(reply.model)  # the model that served it

curl https://api-production-c74b9.up.railway.app/v1/chat/completions \
  -H "Authorization: Bearer $INFERIT_API_KEY" \
  -H "content-type: application/json" \
  -H "x-inferit-min-quality: 5" \
  -H "x-inferit-max-price-per-m: 400000" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello"}]}'

Hints only narrow what auto can choose from now (candidates_now in GET /v1/routing). When that list is empty, name a model: max_price_per_m works with a named model too.

curl https://api-production-c74b9.up.railway.app/v1/chat/completions \
  -H "Authorization: Bearer $INFERIT_API_KEY" \
  -H "content-type: application/json" \
  -H "x-inferit-max-price-per-m: 400000" \
  -d '{"model": "qwen/qwen-2.5-14b-instruct", "messages": [{"role": "user", "content": "Hello"}]}'

What the response says

The other headers are unchanged: x-inferit-attempts, x-inferit-offer, x-inferit-buyer-cost-micro (all response headers). Browsers can read all of them (CORS).

x402 and auto

• Without a key, the 402 quote is priced for the model auto picks: the worst case of this body at that model's first offer, never below 10 000 µ ($0.01). The 402 carries x-inferit-model and x-inferit-route, and its message names the model.
• The paid retry tries that model first. If it stopped being a candidate between the quote and the payment, the next candidates are tried, each only if its worst case fits the payment. If none fits, the answer is 503 no_available_offers, and the payment stays in your balance.
• The model is chosen before any payment is taken: when no model can serve the request, the call without a key gets 503 no_eligible_model instead of a 402, and a paid retry gets it before its payment is settled.
• The quote is pinned to the body and the routing headers, the hint headers included: a retry with a different hint gets a new 402.

Agents: agents.md has the whole x402 exchange.

Errors

error.reasons counts every model considered by its first failing check (the reasons above), and error.reasons.models lists the models for no_offer, no_healthy_offer, below_min_discount and over_max_price.

{
  "error": {
    "message": "No model can serve this request right now: 2 have no healthy seller, 1 has a context window under 300,000 tokens, 16 have no seller on the whitechain rail, 7 have no published quality score.",
    "type": "service_unavailable",
    "code": "no_eligible_model",
    "reasons": {
      "considered": 26, "unscored": 7, "not_servable": 0, "context_too_small": 1, "needs_tools": 0, "needs_vision": 0,
      "unsupported_input": 0, "below_min_quality": 0, "no_offer": 16, "no_healthy_offer": 2, "below_min_discount": 0, "over_max_price": 0,
      "models": { "no_offer": ["…"], "no_healthy_offer": ["…", "…"], "below_min_discount": [], "over_max_price": [] }
    }
  }
}

The quality ranking

Read on 2026-10-06. score = the model's Artificial Analysis Intelligence Index, rounded to the nearest integer. Each catalog model is matched to the row the source publishes under the model's own id (its default configuration, e.g. 'Qwen3.6 27B (Reasoning)'); sourceName and sourceId record the match. Models the source does not list, or lists only under a different release than the catalog's, are left out (no fallback map is used in this table). estimatedBySource marks values the source itself labels as estimated.

A score describes the model as its vendor released it. Sellers often serve a quantized build (for example 4-bit or 8-bit weights, shown as the offer's quantization when the seller declares it), which can answer somewhat differently; the ranking does not measure each seller's build.

• Artificial Analysis Intelligence Index, read 2026-10-06: artificialanalysis.ai/leaderboards/models. Version: v4.3 (the leaderboard label; the methodology note on artificialanalysis.ai/models reads v4.3.2).

Catalog models without a score (auto never routes to them; name them to use them):

• qwen/qwen-2.5-14b-instruct: Not listed by Artificial Analysis (it lists Qwen2.5 Instruct 72B, Qwen2.5 Coder 32B and 7B) nor by the LMArena text leaderboard (checked 2026-10-06).
• mistralai/mistral-nemo: Not listed with an Intelligence Index value by Artificial Analysis.
• deepseek/deepseek-v4-pro: Catalog release is 0423 (OpenRouter); the source lists V4 Pro 0424 and V4 Pro 0813 only, so the match is not certain.
• deepseek/deepseek-v4-flash: Catalog release is 0423 (OpenRouter); the source lists V4 Flash 0420 and V4 Flash 0731 only, so the match is not certain.
• anthropic/claude-sonnet-5.5: Closed model: a reference row, not servable; needs no score.
• openai/gpt-6.1-sol: Closed model: a reference row, not servable; needs no score.
• google/gemini-3.8-flash: Closed model: a reference row, not servable; needs no score.

The table is kept with the catalog: when the catalog takes a new price snapshot, the scores are read again the same day. The same table, and the models auto can choose from now, are at GET /v1/routing as JSON (API reference).

What it does not do

• It chooses from the request's size and features, never from what it says.
• Tools and images are model-level flags from the catalog snapshot: a model supports the feature, and a given seller's server may still not.
• Hints are per request. A key cannot carry a default policy or price limit in this version.
• Audio and file parts go only to a model you name.

Name a model instead

Send a catalog id or an alias (GET /v1/models lists them) and Inferit picks only the seller: the cheapest healthy seller of that model, as it always has. max_price_per_m works with named models too: when no healthy offer of the model is within it, the answer is 503 no_available_offers, with retry-after. When no seller of the model serves a context window big enough for the request, the answer is 400 context_length_exceeded. min_context and min_quality do not apply to a named model. A named model needs no score: unscored models are served by name, so when auto answers no_eligible_model, a model with a live offer (a best_price in GET /v1/models) still answers by name.