Generate a LiteLLM proxy config.yaml with the provider prefix that decides where requests go, credentials as environment variable names rather than values, and a load-balanced deployment group if you need one.
Worked setups you can load into the form above. Each one is a decision the generator makes differently, and the reason it makes it.
The alias is named after the role, so swapping the model behind it is a config edit rather than a deploy. The prefix is what routes the request.
The model field is a deployment name you chose rather than the model, and api_base and api_version are both mandatory with no defaults.
The openai/ prefix is the wire format, not the company. With no api_base every request leaves your network and is billed by somebody else.
A repeated model_name is a load-balanced group rather than a duplicate key. Each entry gets its own numbered credential, because they are different endpoints.
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
LiteLLM routes on it. Without `anthropic/` or `azure/` the request goes to OpenAI, which returns a 404 naming a model that genuinely exists elsewhere. The error points at the model, not at the routing.
Instead:Prefix every non-OpenAI model. OpenAI is the one with an empty prefix, because it is the default.
Azure serves models under a deployment you named, and that name is usually not the model's. `azure/gpt-4o` against a deployment called `prod-gpt4o` gives a DeploymentNotFound, and nothing suggests this field means something different on this provider.
Instead:Use the deployment name from the portal, and set api_base and api_version as well: both are mandatory on Azure and neither has a default.
It is how LiteLLM declares a load-balanced deployment group, and the router picks between the entries per request. Removing the repeat removes the load balancing, which people then reimplement in application code.
Instead:Repeat it deliberately, and give each entry its own credentials, its own api_base and its own rpm and tpm.
The prefix means the OpenAI wire format, not the OpenAI company. Without an api_base pointing at your vLLM or Ollama server, every request leaves your network and is billed by somebody else.
Instead:Set api_base to your endpoint, usually ending in /v1, which is where most OpenAI-compatible servers mount.
The file gets committed. A key in git history is not removed by rotating it afterwards, and it is in every clone.
Instead:Use `os.environ/NAME`. LiteLLM resolves it at load time, which keeps the file safe to commit.
A LiteLLM config has two model fields and they mean different things. Getting them the wrong way round is the first hour of using it, and the error the proxy returns points at neither.
Your application calls the alias. LiteLLM looks up the alias and calls the real model. Keeping them separate is the point of running a proxy at all: changing provider becomes a config edit rather than a deploy. Name the alias after the role, something like fast or smart, and it survives the model behind it being replaced.
model_list:
- model_name: smart <- what your code sends
litellm_params:
model: anthropic/claude-sonnet-5 <- what LiteLLM callsLiteLLM routes on it. Leave it off and the request goes to OpenAI, because that is the default, and OpenAI returns a 404 for a model name that exists somewhere else. The error names a model you know is real, which is why this one costs an afternoon rather than a minute.
model: claude-sonnet-5 404 from OpenAI
model: anthropic/claude-sonnet-5 correct
azure/DEPLOYMENT not the model name
bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
openai/anything plus api_base, for a local serverAzure OpenAI serves models under a deployment you named yourself, and that name is usually not the model's. Putting gpt-4o there when the deployment is called prod-gpt4o gives a DeploymentNotFound, and nothing in the message suggests this one field means something different on this provider than on every other one. It needs api_base and api_version too, both mandatory and neither defaulted.
Two entries with the same model_name are a load-balanced deployment group, and the router picks between them per request. People reimplement this in their own code because a repeated key looks like a mistake. Each entry gets its own credentials and its own endpoint, which is what makes it useful across regions or across accounts with separate quotas.
It is how you point LiteLLM at vLLM, Ollama or LM Studio, and the thing that makes it point at yours rather than OpenAI's is api_base. Without one, a config that reads like a local setup sends every request to OpenAI, and the first sign is a bill. Most OpenAI-compatible servers mount at /v1, and leaving that off is the other half of the same mistake.
With it on, a parameter the target model does not support is deleted rather than rejected. A request asking for a JSON response format, a seed or a specific tool choice can silently lose that requirement and come back looking fine. Keep it for genuine cross-provider routing, and turn it off while developing so an unsupported parameter is an error you see.
Every credential field on this form takes a variable NAME. The generated config refers to os.environ/NAME and never to a value, because a config file is a file that gets committed, and a key in your history is not removed by rotating it afterwards. If you paste a real key into a field, it is turned into a variable name and the value is discarded. This page also generates the config from the fields alone: it validates nothing against your account, so a deployment name that does not exist looks exactly like one that does.