# LiteLLM

> Use Inferit from LiteLLM: the SDK call with the openai/ prefix and Inferit’s base URL, and a proxy config with model auto and a named model. Testnet: test ITC, no monetary value.

LiteLLM sends any model whose name starts with openai/ to an OpenAI-compatible base URL, so Inferit works in the LiteLLM SDK and behind the LiteLLM proxy.

## 1. Get an API key

Create a key in the [Buy workspace](https://inferit.app/buy) ([Quickstart](https://inferit.cc/docs/quickstart.md), steps 1 to 6) and export it. The key spends your prepaid balance: test ITC on the testnet, a test token with no monetary value.

```bash
export INFERIT_API_KEY='ik_...'
```

The base URL is `https://api-production-c74b9.up.railway.app/v1`. Any OpenAI-compatible client works with it (chat completions, streaming included).

## 2. The SDK

Prefix the Inferit model id with `openai/`: LiteLLM strips the prefix and sends the rest unchanged, slashes included (`openai/qwen/qwen-2.5-14b-instruct` sends `qwen/qwen-2.5-14b-instruct`). `response.model` is the model that answered.

_chat.py_

```python
# pip install litellm
import os
import litellm

# "openai/" tells LiteLLM to send the rest of the name, unchanged, to an
# OpenAI-compatible base URL. "auto": Inferit picks the model.
response = litellm.completion(
    model="openai/qwen/qwen-2.5-14b-instruct",
    api_base="https://api-production-c74b9.up.railway.app/v1",
    api_key=os.environ["INFERIT_API_KEY"],
    messages=[{"role": "user", "content": "Say hello in five words."}],
)
print(response.choices[0].message.content)
print(response.model)  # the model that answered
```

## 3. The proxy

To give several apps one endpoint, list Inferit in the proxy's config: one entry for model auto and one per catalog model you want to name. The proxy reads your Inferit key from `INFERIT_API_KEY`. It also needs a master key of its own (`LITELLM_MASTER_KEY`), which your apps send to it: recent LiteLLM versions refuse to start the proxy without one.

_config.yaml_

```yaml
# litellm --config config.yaml     (pip install 'litellm[proxy]')
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY   # what your apps send to the proxy
model_list:
  - model_name: inferit-auto             # Inferit picks the model
    litellm_params:
      model: openai/auto
      api_base: https://api-production-c74b9.up.railway.app/v1
      api_key: os.environ/INFERIT_API_KEY
  - model_name: inferit-named            # one catalog model; Inferit picks the seller
    litellm_params:
      model: openai/qwen/qwen-2.5-14b-instruct
      api_base: https://api-production-c74b9.up.railway.app/v1
      api_key: os.environ/INFERIT_API_KEY
```

_call the proxy_

```bash
curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'content-type: application/json' \
  -d '{"model":"inferit-named","messages":[{"role":"user","content":"Say hello in five words."}]}'
```

## Model auto, or a named model

With `"auto"` as the model, Inferit picks the model: the best value on a published quality ranking for this request, at that model's cheapest healthy seller ([Automatic routing](https://inferit.cc/docs/routing.md)). The response names the model that answered, in its `model` field and the `x-inferit-model` header.

Auto needs a ranked model with a live seller. `GET /v1/routing` lists the models it can choose from now in `candidates_now`; while that list is empty, a request with auto answers `503 no_eligible_model`. Then name a catalog model with a live seller instead, such as `qwen/qwen-2.5-14b-instruct`, and Inferit picks the seller. The samples on this page send auto while it has a candidate, and that model otherwise.

This guide pays with an API key. To pay per request with x402 and no key, see [Agents](https://inferit.cc/agents).

## Checked

The SDK sample and the proxy config ran against the hosted testnet API on 2026-10-13, with a named model (auto had no candidate that day).

---

Canonical page: https://inferit.cc/docs/guides/litellm
All pages as markdown: https://inferit.cc/llms.txt
Testnet demo: test tokens, no monetary value.