LiteLLM

1. Get an API key

Create a key in the Buy workspace (Quickstart, steps 1 to 6) and export it. The key spends your prepaid balance: test ITC on the testnet, a test token with no monetary value.

export INFERIT_API_KEY='ik_...'

The base URL is https://api-production-c74b9.up.railway.app/v1. Any OpenAI-compatible client works with it (chat completions, streaming included).

2. The SDK

Prefix the Inferit model id with openai/: LiteLLM strips the prefix and sends the rest unchanged, slashes included (openai/qwen/qwen-2.5-14b-instruct sends qwen/qwen-2.5-14b-instruct). response.model is the model that answered.

# pip install litellm
import os
import litellm

# "openai/" tells LiteLLM to send the rest of the name, unchanged, to an
# OpenAI-compatible base URL. "auto": Inferit picks the model.
response = litellm.completion(
    model="openai/qwen/qwen-2.5-14b-instruct",
    api_base="https://api-production-c74b9.up.railway.app/v1",
    api_key=os.environ["INFERIT_API_KEY"],
    messages=[{"role": "user", "content": "Say hello in five words."}],
)
print(response.choices[0].message.content)
print(response.model)  # the model that answered

3. The proxy

To give several apps one endpoint, list Inferit in the proxy's config: one entry for model auto and one per catalog model you want to name. The proxy reads your Inferit key from INFERIT_API_KEY. It also needs a master key of its own (LITELLM_MASTER_KEY), which your apps send to it: recent LiteLLM versions refuse to start the proxy without one.

# litellm --config config.yaml     (pip install 'litellm[proxy]')
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY   # what your apps send to the proxy
model_list:
  - model_name: inferit-auto             # Inferit picks the model
    litellm_params:
      model: openai/auto
      api_base: https://api-production-c74b9.up.railway.app/v1
      api_key: os.environ/INFERIT_API_KEY
  - model_name: inferit-named            # one catalog model; Inferit picks the seller
    litellm_params:
      model: openai/qwen/qwen-2.5-14b-instruct
      api_base: https://api-production-c74b9.up.railway.app/v1
      api_key: os.environ/INFERIT_API_KEY

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'content-type: application/json' \
  -d '{"model":"inferit-named","messages":[{"role":"user","content":"Say hello in five words."}]}'

Model auto, or a named model

With "auto" as the model, Inferit picks the model: the best value on a published quality ranking for this request, at that model's cheapest healthy seller (Automatic routing). The response names the model that answered, in its model field and the x-inferit-model header.

Auto needs a ranked model with a live seller. GET /v1/routing lists the models it can choose from now in candidates_now; while that list is empty, a request with auto answers 503 no_eligible_model. Then name a catalog model with a live seller instead, such as qwen/qwen-2.5-14b-instruct, and Inferit picks the seller. The samples on this page send auto while it has a candidate, and that model otherwise.

This guide pays with an API key. To pay per request with x402 and no key, see Agents.

Checked

The SDK sample and the proxy config ran against the hosted testnet API on 2026-10-13, with a named model (auto had no candidate that day).