liteLLM Release Notes

This release is published as ghcr.io/berriai/litellm:v1.106.1. See the GitHub release and the full releases page

v1.106.1 is a patch release on top of v1.106.0. It backports six fixes from main and the token counting fixes that shipped in v1.104.4 onto stable/1.106.x, so upgrading from v1.104.4 keeps them. There are no new database migrations. The v1.106.1 tag points at 978b4d1

Key Highlights

• Token counting from v1.104.4: Anthropic-format count requests for OpenAI and Gemini models are converted the same way /v1/messages inference converts them, and count responses say where the number came from
• Bare MCP tool names: a tools/call with a bare tool name on a single-server scope goes to that server, so a cold replica no longer answers Tool not found
• Clean vector store metadata: vector stores created through the proxy send the provider only the caller's metadata, so later metadata updates no longer fail with 400

New Models / Updated Models

Bug Fixes

• Anthropic
  • /v1/messages requests no longer drop a thinking block whose text is empty but whose signature is set. Only an empty, unsigned block is removed - PR #41362
  • Chat completions keep signed thinking blocks with empty text, and drop a thinking block only when its signature is missing. A /v1/messages request that sets prefix_mismatch_behavior: error now gets the binding mismatch 400 back instead of a retry with thinking stripped. The Anthropic, Azure AI and Bedrock Invoke configs keep thinking.block_binding when they rebuild thinking - PR #46342

LLM API Endpoints

Bugs

• Token counting
  • Anthropic-format count requests for OpenAI and Gemini models are converted the same way /v1/messages inference converts them
  • Count responses carry litellm_count_source, which is provider, token_count_model or local_estimate. When a provider count fails and local counting is on, the response is a local estimate labelled local_estimate
  • New model_info settings token_count_framing and token_count_model tune the count for one deployment. When a count model answers, the response also carries litellm_count_model
  • The bundled cost map adds token_count_framing and token_count_tokenizer values for some models
  • These fixes come from PR #47389, with PR #47446 and PR #47454 keeping earlier Google countTokens request shapes and prompt plus messages inputs working as before
• Responses API bridge
  • A chat completion bridged to /responses put each Responses message item in its own choice, so a [message, message, function_call] reply put the tool call on choices[1]. Message text, annotations, reasoning and tool calls now fold into one choices[0], and finish_reason is tool_calls when any tool call is present - PR #46099
• Vector Stores
  • Vector stores created through the proxy filled all 16 OpenAI metadata slots with proxy context, so every later metadata update failed with HTTP 400, and caller keys such as endpoint were overwritten. Create and update now send the provider exactly the caller's metadata, while logs, tags and limits still see the proxy context. The opt-in db_scripts/strip_vector_store_internal_metadata.py script removes LiteLLM's keys from stores created before this release - PR #47021

Management Endpoints / UI

Bugs

• Team keys can read and cancel their own team's batches and download the finished results. The same model check applies to file, fine-tuning, vector store, character and video ids - PR #47067

MCP Gateway

• A tools/call with a bare tool name on /{server_name}/mcp, or with one server in x-mcp-servers, returned Tool not found on a cold replica since v1.103.0. A request scoped to exactly one server now routes a bare name to that server, and a bare name that two servers share goes to the server the path names - PR #46225

All of these fixes and the version bump are backported in PR #47517

PR roll-up by ownership area

• LLM API Endpoints: 5
• Models & Providers: 2
• Auth & Management: 1
• MCP: 1

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.106.0...v1.106.1

Original source

This release is published as ghcr.io/berriai/litellm:v1.105.2. See the GitHub release and the full releases page

v1.105.2 is a patch release on top of v1.105.1. It backports seven fixes from main and the token counting fixes that shipped in v1.104.4 onto stable/1.105.x, so upgrading from v1.104.4 keeps them. There are no new database migrations. The v1.105.2 tag points at fc43640

Key Highlights

• Token counting from v1.104.4: Anthropic-format count requests for OpenAI and Gemini models are converted the same way /v1/messages inference converts them, and count responses say where the number came from
• Tool calls on choices[0]: a chat completion bridged to /responses returns its text, reasoning and tool calls in one choice, so agents that read only choices[0] run the tool
• Signed empty thinking blocks: /v1/messages and chat completions keep a thinking block that has a signature but no text, so replayed turns keep their reasoning

New Models / Updated Models

Bug Fixes

• Anthropic
  • /v1/messages requests no longer drop a thinking block whose text is empty but whose signature is set. Only an empty, unsigned block is removed - PR #41362
  • Chat completions keep signed thinking blocks with empty text, and drop a thinking block only when its signature is missing. A /v1/messages request that sets prefix_mismatch_behavior: error now gets the binding mismatch 400 back instead of a retry with thinking stripped. The Anthropic, Azure AI and Bedrock Invoke configs keep thinking.block_binding when they rebuild thinking - PR #46342

LLM API Endpoints

Bugs

• Token counting
  • Vertex AI partner models, such as Claude on Vertex AI, count the system prompt and tools
  • Anthropic-format count requests for OpenAI and Gemini models are converted the same way /v1/messages inference converts them
  • Count responses carry litellm_count_source, which is provider, token_count_model or local_estimate. When a provider count fails and local counting is on, the response is a local estimate labelled local_estimate
  • New model_info settings token_count_framing and token_count_model tune the count for one deployment. When a count model answers, the response also carries litellm_count_model
  • The bundled cost map adds token_count_framing and token_count_tokenizer values for some models
  • These fixes come from PR #47389, with PR #47446 and PR #47454 keeping earlier Google countTokens request shapes and prompt plus messages inputs working as before
• Responses API bridge
  • A chat completion bridged to /responses put each Responses message item in its own choice, so a [message, message, function_call] reply put the tool call on choices[1]. Message text, annotations, reasoning and tool calls now fold into one choices[0], and finish_reason is tool_calls when any tool call is present - PR #44346, PR #46099

Management Endpoints / UI

Bugs

• Team keys can read and cancel their own team's batches and download the finished results. The same model check applies to file, fine-tuning, vector store, character and video ids - PR #47067

Spend Tracking, Budgets and Rate Limiting

• Streamed chat completions sent through a model alias are now priced with the deployment's model instead of the alias, the same way non-streamed requests are. An alias that points at a deployment with no pricing now bills $0 on streamed requests, which matches what the same request billed without streaming - PR #44341

Performance / Loadbalancing / Reliability improvements

• Bursts of requests right after a model is added no longer get 400 Invalid model name. A request that waited on the registry lock now uses the model when another worker already loaded it, instead of spending the shared resync budget - PR #44277

All of these fixes and the version bump are backported in PR #47519

PR roll-up by ownership area

• LLM API Endpoints: 5
• Models & Providers: 2
• Auth & Management: 1
• Spend / Budgets / Rate Limits: 1
• Performance: 1

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.105.1...v1.105.2

Original source

This release is published as ghcr.io/berriai/litellm:v1.103.6. See the GitHub release and the full releases page.

v1.103.6 is a patch release on top of v1.103.5. It backports two fixes from main onto stable/1.103.x, so upgrading from stable/1.102.x keeps them. There are no new database migrations. The v1.103.6 tag points at dce7feb.

What's Changed

• When the upstream realtime API refuses the websocket handshake, for example with a 401 from Azure, the client now gets an error event that names the upstream HTTP status. The socket then closes with 1008 for 401 and 403, 1013 for 429 and 1011 for anything else. Before, the client saw a bare 1006 or 1011 close with no event, and the Azure relay could leave the client socket open - PR #42388
• Non-streaming requests of 256 KiB or more on the chat to Responses bridge log their spend once instead of twice, so key, team and daily spend are no longer counted twice for one request - PR #44508

Both fixes and the version bump are backported in PR #47522.

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.103.5...v1.103.6

Original source

Deploy this version

This release is published as ghcr.io/berriai/litellm:v1.106.0. See the GitHub release and the full releases page.

These notes cover everything since v1.105.0. Changes added to the release line after rc.1, through rc.4 and the fixes backported before the stable tag, are under Included after the v1.106.0-rc.1 cut.

Customer-facing changes come first. Test, CI and internal changes are listed at the bottom.

Breaking Changes

These callouts cover user-facing behavior that differs from v1.105.0, the previous stable release.

• The response cache skips requests with more than 4 messages by default. The new cache_params.max_messages defaults to 4, so longer conversations and Responses inputs are neither looked up nor stored, exact or semantic. Raise it, or set it to null to remove the limit. See PR #43878
• Responses WebSocket sessions are capped at 60 minutes by default. Sessions on /v1/responses close with code 1000 after general_settings.responses_websocket_session_limit_seconds (default 3600, maximum 7200), idle or not. The old 30 second first-message timeout is gone. See PR #44433
• The caller's LiteLLM key is never forwarded to MCP servers, including x-litellm-api-key configured under a server's Extra Headers. Give upstream servers their own credential through a static header or OAuth. See PR #45401
• PATCH /credentials/{name} no longer renames a credential. A different credential_name returns 400. Use the new display_name to relabel a credential, or create a new one to change its name. See PR #43148
• Decisions requests that send media a model cannot read now return 400. Images on /v1/systemone, audio and videos on either route, and TypeSafe assets used to be dropped with a 200. They now return 400 naming the field unless drop_params is set, and OpenAI decision models now receive System One images, which are billed. See PR #46628
• /v1/responses/compact now checks previous_response_id the same way /v1/responses does. Another user's or team's id returns 403, and so does an id the proxy never issued unless general_settings.allow_unmanaged_response_ids is true. Ids handed out by the websocket /v1/responses route are refused for every key. See PR #46827

Key Highlights

• New models on day one: Claude Haiku 5.5 across Anthropic, Bedrock, Bedrock Mantle, Vertex AI, Azure AI and Perplexity, Kimi K3 and GLM 5.3 on Bedrock, Mistral Large 4, and Gemini Nano Banana 2.1, among 59 new catalog entries
• Lens as a standalone Rust service: trace ingestion and investigations move out of the gateway, with always-on investigations, a Findings inbox, signals, end-user feedback and datasets built from real traces
• Admin UI Home page: the dashboard opens on a Home page with a usage summary, recent launches, blog updates and a model leaderboard, alongside a Decisions tab in the Playground, a redesigned Usage page and a Cmd+K command palette
• More Decisions providers and media: Laya, Bespoke Nimble and Databricks serve Decisions and act as Auto Router classifiers, Cloudflare Clef reads images, Clef-omni reads audio and video, and TypeSafe Jev 1.14 takes images. Reka joins as an OpenAI-compatible provider
• MCP Gateway controls: rate limits cover every MCP operation with a new server-level rpm, OAuth adds Client ID Metadata Document support, and the caller's LiteLLM key is never forwarded upstream
• Observability: Arize on OTel v2 works per team, Prometheus can cap series per metric, and litellm.agent() gains an in-process tool-calling loop

Included after the v1.106.0-rc.1 cut

• Cloudflare Clef reads images on both Decisions routes, the new cloudflare/clef-omni reads audio and video, and TypeSafe Jev 1.14 (typesafe/jev-1.14.0) takes images through assets, box questions and the opt-in TypeSafe-Danger-Zone-* headers (rc.4) - PR #46628
• Deployments can set model_info.mode: decisions, which behaves the same as mode: evaluation for health checks and the Admin UI (rc.4) - PR #46628
• Home page What's new cards show the newest LiteLLM blog posts tagged day 0 support, so new launches appear without a proxy upgrade (rc.2) - PR #46309
• The yes / no Decisions answer type is labelled "Yes / No" instead of noul (rc.2) - PR #45933
• Setting custom_prometheus_metadata_labels: ["project_alias"] no longer breaks the project per-model rate limit gauges or skips the other metrics for project keys (rc.3) - PR #46491
• streaming_buffer_until_moderated: true on the Decision Model guardrail holds a streamed answer until the decision model passes it (rc.3) - PR #46383
• /v1/responses/compact works with the caller's own previous_response_id, which used to fail with a 400 from OpenAI - PR #46827
• Redis Cluster with max_connections set waits for a free connection under load instead of switching Redis off for 60 seconds - PR #47006

New Providers and Endpoints

New Providers (3 new providers)

PROVIDER SUPPORTED LITELLM ENDPOINTS DESCRIPTION Reka /v1/chat/completions, /v1/responses, /v1/messages Reka as an OpenAI-compatible provider under reka/, with Responses and Messages bridged to its chat completions Laya /v1/decisions, Auto Router Laya decision models and an OSS classifier for the Auto Router Bespoke /v1/systemone, Auto Router Bespoke Nimble decision models and an OSS classifier for the Auto Router

Expanded provider endpoint support

PROVIDER ENDPOINT WHAT YOU CAN DO Databricks /v1/decisions, Auto Router Call Databricks ai_decide as a Decisions provider and Auto Router classifier

New Models / Updated Models

New Model Support (59 new models)

Counts represent new catalog identifiers compared with v1.105.0, including aliases and regional variants. Prices below are the values bundled in this release, in USD; runtime pricing-map reloads can update them. Input and output columns show base token rates; long-context, cache, image-token, and other specialized rates depend on the model.

The release also adds and updates provider support, model catalog and pricing, LLM API endpoints, SDK functionality, Admin UI features, Lens and Agent Traces, guardrails, observability, spend tracking, budgets, rate limiting, MCP Gateway controls, performance and reliability improvements, Docker and Helm support, dependency updates, documentation, tests, CI and internal tooling.

Tests, CI and Internal Changes

These 166 PRs change tests, CI, contributor tooling, release packaging, or Rust runtime scaffolding that is not yet wired into a user-facing path. They do not change proxy or SDK behavior on their own.

PR roll-up by ownership area

Customer-facing PRs: 397. Tests, CI and internal PRs: 166. Total: 563

• Other (tests, CI, internal): 166
• Models & Providers: 107
• UI: 77
• Logging & Tracing: 72
• Performance: 41
• LLM API Endpoints: 26
• Spend / Budgets / Rate Limits: 25
• Auth & Management: 22
• MCP: 15
• Guardrails: 11
• Docs: 1

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.105.0...v1.106.0

Original source

v1.103.5 is a patch release on top of v1.103.4. It fixes how LiteLLM turns a Responses API reply back into a chat completion, which affects /chat/completions calls with tools on GPT-5.4 and newer and on responses/ deployments. The v1.103.5 tag points at dec1691.

What's Changed

• Tool calls no longer land on a separate choices[1], so agents that only read choices[0] run them again - PR #46099
• Multiple tool calls in one non-streaming reply now get their own index instead of all being 0 - PR #46109

Both fixes are backported in PR #46949

Original source

This release is published as ghcr.io/berriai/litellm:v1.105.1. See the GitHub release and the full releases page.

v1.105.1 is a patch release on top of v1.105.0. It backports two spend tracking fixes from main onto stable/1.105.x. There are no new database migrations and no breaking changes. The v1.105.1 tag points at d665dc1.

Key Highlights

• Batched model usage writes: the model leaderboard rollup no longer upserts LiteLLM_DailyModelUsage on every request, which removes the row lock contention that dropped SpendLogs rows when heavy traffic went to one model
• Single spend log on the Responses bridge: non-streaming requests of 256 KiB or more sent through the chat to Responses bridge log their spend once instead of twice

Spend Tracking, Budgets and Rate Limiting

• The daily model usage rollup added with the model leaderboard in v1.105.0 wrote one LiteLLM_DailyModelUsage upsert per request. At high request rates for a single model those upserts fought over the same row and held Prisma connections, so some SpendLogs rows were never written. Rollup rows are now queued in memory and written in batches by the existing spend log flush job. See PR #44243
• Non-streaming requests on the chat to Responses bridge whose messages were 256 KiB or larger ran the async success logging twice, which wrote two SpendLogs rows and counted key, team and daily spend twice. The two handlers now run one after the other on the same logging object, and the second one skips if the first already logged. See PR #44508

Both fixes and the version bump are backported together in PR #46313.

PR roll-up by ownership area

• Spend / Budgets / Rate Limits: 2

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.105.0...v1.105.1

Original source

Deploy this version

docker run \
  -e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
  -e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
  -e STORE_MODEL_IN_DB=True \
  -p 4000:4000 \
  docker.litellm.ai/berriai/litellm:1.104.3

This release is published as ghcr.io/berriai/litellm:v1.104.3. See the GitHub release and the full releases page

Breaking Changes

Decisions requests that send safety_identifier to a provider other than OpenAI now return 400 instead of 200 with the identifier silently dropped. This covers Perplexity, TypeSafe, OpenRouter, Strands Decider and Cloudflare on /v1/decisions, /v1/systemone and litellm.decisions(). Set drop_params: true on the deployment, under litellm_settings, in the request body or on litellm.drop_params to keep the old behavior. /v1/systemone requests are also logged with call type asystemone instead of adecisions. See PR #44955 and PR #45846

v1.104.3 is a patch release on top of v1.104.2. It backports the Decisions follow-ups from main: two more providers, Strands Decider on AgentCore, a litellm.systemone() SDK entry point, OpenRouter billing from the cost OpenRouter reports, a decision model guardrail and decision-model Auto Router setup. There are no new database migrations. The v1.104.3 tag points at 3645fe0

Decisions API

• New providers: hosted_vllm/ (requires api_base, or HOSTED_VLLM_API_BASE) and azure_ai/ for Microsoft-Decision-1, joining OpenAI, Perplexity, TypeSafe, OpenRouter, Cloudflare and Strands Decider. See the Decisions docs
• Strands Decider on AgentCore: a strands_decider/ deployment whose api_base is an AgentCore runtime ARN calls InvokeAgentRuntime, signed with the deployment's aws_* credentials
• SDK: the new litellm.systemone() takes the System One format and litellm.decisions() takes the OpenAI format. litellm.decisions(state=...) still works and warns that it is deprecated in favor of litellm.systemone()
• safety_identifier: providers that cannot take it return 400 unless drop_params drops it. OpenAI keeps passing it through from either route
• Spend tracking: OpenRouter Decisions calls and the /openrouter/alpha/decisions pass-through bill the cost OpenRouter reports in usage.cost
• Decision model guardrail: a new decision_model guardrail screens messages and tool-call arguments with admin-written yes/no questions, each set to block or log above its threshold, and is configured from the Admin UI and the Guardrail Garden
• Auto Router: a new router can use an onboarded decision model deployment as its classifier, and existing router configurations keep working
• Admin UI and docs: TypeSafe and Strands Decider are in the Add Model provider list, and Swagger UI describes /v1/decisions and /v1/systemone

New Model Support (1 new model)

PROVIDER | MODEL | CONTEXT WINDOW | INPUT ($/1M TOKENS) | OUTPUT ($/1M TOKENS)
Azure AI | azure_ai/Microsoft-Decision-1 | - | $0.042 | $0.00

What's Changed

• Dispatch Decisions through per-provider configs and the shared HTTP handler - PR #45129, PR #45130, PR #45442
• Add the hosted vLLM Decisions provider - PR #45501
• Add azure_ai/Microsoft-Decision-1 and support it on the Decisions API - PR #45639, PR #45673, PR #45717
• Call a Strands Decider on an AgentCore runtime ARN - PR #45678
• Add litellm.systemone() and keep litellm.decisions() to the OpenAI format - PR #45846
• Refuse safety_identifier on providers that cannot take it unless drop_params drops it - PR #44955
• Bill OpenRouter Decisions and the Decisions pass-through from usage.cost (the Decisions and pass-through billing parts only) - PR #45640
• Add TypeSafe and Strands Decider to the Add Model provider list, and describe both Decisions routes in Swagger UI - PR #45335, PR #45838
• Add the decision model guardrail with an Admin UI flow - PR #45663, backported in PR #45900, with the Add Guardrail provider list fixed after both backports landed in PR #45936
• Pick an onboarded decision model as the Auto Router classifier - PR #45913, backported in PR #45914
• Rebuild the Admin UI bundle - PR #45924

The Decisions follow-ups and the version bump are backported together in PR #45906

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.104.2...v1.104.3

Original source

v1.104.2 is a patch release on top of v1.104.1. It adds the Decisions API routes and providers. There are no new database migrations or breaking changes: the Decisions routes are new on this line. The v1.104.2 tag points at fc3920b

Decisions API

The proxy now serves decision models, which answer typed questions (yes/no predicates, choices and scores) about an input. Requests go through the usual keys, budgets, spend tracking and guardrails. See the Decisions docs

• Two request formats
  • POST /v1/decisions and POST /decisions take OpenAI's Decisions API format: input plus a questions list, answered as an answers list with OpenAI-style usage
  • POST /v1/systemone and POST /systemone take the System One format: state plus a named questions map, answered as an answers map keyed by question name
  • Both routes go through one shared format inside the proxy, so any Decisions model can be called on either route
• Providers: openai/ (gpt-6-luna), perplexity/, typesafe/, openrouter/, cloudflare/clef and cloudflare/clef-flash, and self-hosted strands_decider/ (requires api_base)
• OpenAI: images, multi-message input, safety_identifier, names and refusals pass through on /v1/decisions. The provider reads litellm.api_key, litellm.openai_key, litellm.api_base, OPENAI_BASE_URL and OPENAI_API_BASE in the same order as other OpenAI calls
• Validation before any upstream call: a missing state, a malformed question, more than 128 questions, a single-option question or a body that is not JSON returns 400. Image input sent to a provider that only accepts text returns a clear 400
• Spend tracking: per-token pricing from the cost map, with cached and cache-write input tokens billed at their own rates
• Health checks: /health probes mode: evaluation deployments with a one-question Decisions call, and health_check_params can override its state and questions
• Guardrails: OpenAI-format responses include guardrail_information when the caller sets include_guardrail_response
• Pass-through: an existing pass_through_endpoints entry at /v1/decisions keeps answering that path, while /decisions serves the native route

New Model Support (6 new models)

PROVIDER MODEL CONTEXT WINDOW INPUT ($/1M TOKENS) OUTPUT ($/1M TOKENS) Perplexity perplexity/pplx-decider-v1-27b 262K $0.04 $0.00 Cloudflare cloudflare/clef 65K $0.24 $0.00 Cloudflare cloudflare/clef-flash 65K $0.09 $0.00 Cloudflare cloudflare/@cf/cloudflare/clef 65K $0.24 $0.00 Cloudflare cloudflare/@cf/cloudflare/clef-flash 65K $0.09 $0.00 Strands Decider strands_decider/strands-decider-2B-hobson-v19 - $0.00 (self-hosted) $0.00

The existing gpt-6-luna entry now lists /v1/decisions among its supported endpoints

What's Changed

• Add a unified /v1/decisions endpoint for System One-compatible providers - PR #44236
• Serve the System One format at /v1/systemone and OpenAI's format at /v1/decisions - PR #45184
• Add OpenAI as a Decisions provider behind a shared decisions format - PR #45214

All three are backported in PR #45190

Full Changelog

Original source

LiteLLM v1.102.4 fixes double-counted spend on the chat to Responses bridge

Deploy this version

This release is published as ghcr.io/berriai/litellm:v1.102.4. See the GitHub release and the full releases page.

v1.102.4 is a patch release on top of v1.102.3. It fixes spend that was logged twice for large non-streaming chat requests. There are no new database migrations or breaking changes. The v1.102.4 tag points at 0de9a17.

Spend logged once on the Chat to Responses bridge

Since v1.102.0, a large non-streaming /v1/chat/completions request to an openai/responses/* model was logged twice. The request showed two rows on the logs page, and key, team and daily spend went up by twice the real cost.

• One request now queues exactly one success log, and the logged result is the provider's own response
• Streaming requests and the success handler itself are unchanged

What's Changed

• Log spend once for large non-streaming requests on the chat to Responses bridge - PR #44508, backported in PR #45215

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.102.3...v1.102.4

Original source

This release is published as ghcr.io/berriai/litellm:v1.104.1. See the GitHub release and the full releases page.

v1.104.1 is a patch release on top of v1.104.0. It updates locked Python and Admin UI dependencies and the pgbouncer bundled in the Docker images. There are no new database migrations or breaking changes. The v1.104.1 tag points at 48963be.

pgbouncer 1.26.0

The Docker images build pgbouncer from source. Wolfi's openssl-dev package now installs OpenSSL 4.0, which pgbouncer 1.25.2 does not compile against, so image builds on this line stopped working. The bundled pgbouncer is now 1.26.0, the same version main ships, and it builds and runs against OpenSSL 4.0.

Dependency updates

• Python: langgraph-sdk 0.4.2 to 0.4.4, mako 1.3.12 to 1.4.2, multidict 6.7.1 to 6.9.1 and werkzeug 3.1.8 to 3.1.9
• Admin UI: next 16.3.3 to 16.3.6, moment 2.30.1 to 2.31.0 and brace-expansion 5.0.9 to 5.0.12, plus the transitive lockfile updates that come with them

What's Changed

• Refresh locked Python and Admin UI dependencies - PR #44777
• Bump the bundled pgbouncer to 1.26.0 - PR #45012

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.104.0...v1.104.1

Original source

Deploy this version

This release is published as ghcr.io/berriai/litellm:v1.100.5. See the GitHub release and the full releases page.

v1.100.5 is a patch release on top of v1.100.4. It updates locked Python and Admin UI dependencies and the wolfi-base image digest. There are no new database migrations or breaking changes. The v1.100.5 tag points at c80ccaa.

Dependency updates

• Python: fsspec 2026.4.0 to 2026.6.0, gitpython 3.1.60 to 3.1.62, langgraph-sdk 0.4.2 to 0.4.4, mako 1.3.12 to 1.4.2, multidict 6.7.1 to 6.9.1, oauthlib 3.3.1 to 4.0.0, pyjwt 2.13.0 to 2.15.0, pypdf 6.16.1 to 6.19.0, tornado 6.5.8 to 6.5.9, urllib3 2.7.0 to 2.8.0 and werkzeug 3.1.8 to 3.1.9
• Admin UI: next 16.2.11 to 16.3.6, moment 2.30.1 to 2.31.0, brace-expansion 5.0.9 to 5.0.12, js-yaml 4.3.1 to 4.3.2 and sharp 0.35.0 to 0.35.4, plus the transitive lockfile updates that come with them
• Base image: the wolfi-base digest moves forward to pick up glibc 2.44-r6

The Admin UI test tooling moves to Vitest 4, and a few Admin UI and Python tests are updated to match. None of this changes runtime code.

What's Changed

• Refresh locked Python and Admin UI dependencies, and move the Admin UI tests to Vitest 4 - PR #44774
• Bump wolfi-base digest to pick up glibc 2.44-r6 - PR #42643

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.100.4...v1.100.5

Original source

This release is published as ghcr.io/berriai/litellm:v1.103.3. See the GitHub release and the full releases page.

Breaking Changes

The proxy exits when database setup fails at startup instead of serving against an outdated schema. Set ENFORCE_PRISMA_MIGRATION_CHECK=false to keep the old behavior. See PR #44205.

Upgrading from v1.102.x or earlier

The two LiteLLM_SpendLogs index migrations from v1.103.0 are now no-ops. Build the indexes online yourself, outside a transaction:

CREATE INDEX CONCURRENTLY IF NOT EXISTS "LiteLLM_SpendLogs_api_key_startTime_idx" ON "LiteLLM_SpendLogs" ("api_key", "startTime");
CREATE INDEX CONCURRENTLY IF NOT EXISTS "LiteLLM_SpendLogs_litellm_call_id_idx" ON "LiteLLM_SpendLogs" ("litellm_call_id");

v1.103.3 is a patch release on top of v1.103.2. Startup exits on database setup failure, the LiteLLM_SpendLogs index migrations no longer build anything, streamed requests through an alias are billed correctly, and the base image and several dependencies are bumped. The v1.103.3 tag points at ecae261.

Bug fixes

Streamed chat completions through a dotted alias like claude-opus-4.6 recorded $0 spend, because pricing looked up the alias instead of the deployment's model. Streamed requests are now priced from the deployment's model, the same way as non-streamed ones, and logged streamed responses still show the alias. An alias that points at a deployment with no pricing now records $0 on streamed requests too, matching the non-streamed behavior. See PR #44341.

Chat replies bridged from the Responses API put the tool calls in a separate choices[1], so apps that read only choices[0] never ran the tool. choices[0] now carries both the text and the tool calls with finish_reason: "tool_calls". See PR #44346.

What's Changed

• Startup exits on database setup failure, and the LiteLLM_SpendLogs index migrations no longer build anything - PR #44205, PR #44286
• fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model - PR #44341
• fix(responses): merge bridged tool calls into the same choice as the text - PR #44346
• Bump wolfi-base digest to pick up glibc 2.44-r6 - PR #42643
• Dependency bumps: pyjwt, pypdf, tornado, urllib3, gitpython, litellm-proxy-extras 0.4.100.post1 - PR #44352, PR #44225
• test(e2e): move the Together structured-output and text-completion tests to models that still answer - PR #44437

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.103.2...v1.103.3

Original source

This release is published as ghcr.io/berriai/litellm:v1.103.2. See the GitHub release and the full releases page.

Upgrading

There are no new database migrations or breaking changes on top of v1.103.1. If you are upgrading from v1.103.0 or earlier, read the breaking changes in the v1.103.1 notes first: Admin UI and lite CLI users sign in once more after upgrading, and upgrades from v1.102.x or earlier add an index on LiteLLM_SpendLogs.

lite CLI users must log in again

Applies if you are upgrading from a release older than v1.103.1. After upgrading the proxy, every lite CLI user has to run lite login once more. Until they do, the CLI keeps sending its old session token and its requests to the proxy fail.

v1.103.2 is a patch release on top of v1.103.1. It brings Claude Code auto mode through the gateway, attributes CLI session spend to the user behind it, and restores the pass-through endpoint handling that v1.103.0 changed. The v1.103.2 tag points at f69b210.

Claude Code auto mode works through the gateway

Claude Code auto mode sends a safeguards field on /v1/messages and expects safeguard_results in the reply. Earlier releases dropped the field, so /status showed Auto mode server: Disabled and every tool call fell back to the client-side classifier.

The proxy now forwards safeguards and anthropic-beta unchanged on native Anthropic /v1/messages, and forwards safeguards plus the dangerous-tool-use-2026-09-03 beta to Bedrock Invoke and Vertex AI. Azure AI Foundry gets the same passthrough, with the beta added whenever the request carries safeguards, since Foundry rejects safeguards without it.

CLI session spend is attributed to the user

Spend from litellm-proxy login sessions showed up in /user/daily/activity/aggregated under a 64-character hash with null key_alias and user_email. On busy days the name lookup for those hashes could also hit the statement timeout and return nothing.

New CLI session spend is now recorded under the per-user cli-session- alias. Hashes written before the upgrade get their owner back from the daily spend rows, and the key name lookup reads at most two spend log rows per key, so it no longer times out on large spend tables.

Pass-through endpoints are back to their pre-v1.103.0 handling

With STORE_MODEL_IN_DB=true, v1.103.0 let the config file own general_settings.pass_through_endpoints. That broke three things: config pass-throughs with forward_headers: true stopped forwarding Authorization, creating, updating or deleting a pass-through in the Admin UI was rejected once the config declared any, and os.environ/ targets were sent upstream unresolved and returned a 500.

Config and DB pass-throughs are merged again on every DB sync and config reload, a pass-through saved in the Admin UI starts serving right away next to the config ones, and forwarded headers reach the backend unchanged. If your config declares pass-throughs, any pass-throughs stored in the DB on other paths start serving again after the upgrade, as they did before v1.103.0.

What's Changed

• fix(anthropic): forward safeguards and anthropic-beta unchanged on native /v1/messages - PR #42152
• fix(anthropic): forward Claude Code safeguards and dangerous-tool-use beta to Bedrock Invoke and Vertex on /v1/messages - PR #42288
• fix(anthropic): forward the dangerous-tool-use beta to Azure AI Foundry - 44115c5
• fix(azure_ai): add the dangerous-tool-use beta when Foundry gets safeguards - 3b00260
• fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token - PR #40541
• fix(proxy): recover session key owners from daily spend for usage attribution - PR #43642
• fix(proxy): look up hashed key names with two spend log rows per key - PR #43656
• fix(proxy): restore pre-config-wins handling of pass-through endpoints - PR #43962
• chore(release): bump litellm-enterprise 0.1.69 -> 0.1.69.post1 for stable/1.103.x - fe87252

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.103.1...v1.103.2

Original source

Deploy this version

docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.101.4

This release is published as ghcr.io/berriai/litellm:v1.101.4. See the GitHub release and the full releases page

Upgrading

There are no new database migrations or breaking changes on top of v1.101.3. If you are upgrading from v1.101.2 or earlier, read the breaking change in the v1.101.3 notes first: Admin UI and lite CLI users sign in once more after upgrading

lite CLI users must log in again

Applies if you are upgrading from a release older than v1.101.3. After upgrading the proxy, every lite CLI user has to run lite login once more. Until they do, the CLI keeps sending its old session token and its requests to the proxy fail

v1.101.4 is a patch release on top of v1.101.3. It lets the Straiker guardrail use v3 platform keys and stops a shared api_version default from breaking Azure Content Safety guardrails. The v1.101.4 tag points at f471923

Straiker guardrail supports v3 platform keys

Straiker v3 keys (sk_agt_...) got a 401 on every guarded request, because the guardrail only knew the v1 webhook route. With a v3 key the guardrail now relays traffic to /api/v3/detect, an x-s6r-agent request header names the calling app, and a block from Straiker reaches the caller as a 400 with the tenant's block message. Existing v1 keys see no change

A Straiker guardrail whose config carries a leftover api_version other than v1 or v3, for example 2024-09-01 copied from an Azure block, now loads with a warning and picks its route from the key prefix. Without this, the proxy would have started without the guardrail and served requests unchecked

Azure Content Safety guardrails work without api_version

The shared guardrail params defaulted api_version to Javelin's v1, so Azure Content Safety guardrails without an explicit api_version called Azure with api-version=v1 and got a 404, which failed every request when the guardrail was default_on. The shared default is now unset, Azure guardrails fall back to 2024-09-01, and a stored v1 on an Azure guardrail created through POST /guardrails is treated as unset, so those guardrails start working after the upgrade without edits. Javelin still fills in v1 itself

What's Changed

• fix(guardrails): stop the Javelin api_version default leaking into Azure Content Safety - PR #41941
• feat(guardrails): straiker guardrail speaks the v3 platform API (/api/v3/detect) - PR #41880
• fix(guardrails): treat an unknown straiker api_version as unset instead of skipping the guardrail - PR #43956
• chore(deps): bump pyjwt to 2.14.0 - bdc4212

Full Changelog

Original source

Deploy this version

docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.103.1

This release is published as ghcr.io/berriai/litellm:v1.103.1. See the GitHub release and the full releases page

Breaking Changes

Upgrading from v1.102.x or earlier adds an index on LiteLLM_SpendLogs, which blocks spend log writes while it builds at startup. On a large table that can take a long time. To avoid this, build the index yourself before upgrading with CONCURRENTLY, which does not block writes. The migration then finds it and skips the build. See PR #37983

SET statement_timeout = 0;
CREATE INDEX CONCURRENTLY IF NOT EXISTS "LiteLLM_SpendLogs_api_key_startTime_idx" ON "LiteLLM_SpendLogs" ("api_key", "startTime");

Run it outside a transaction, from a session that stays connected until it finishes. A failed concurrent build leaves an invalid index behind, and the migration would skip that too, so check it before upgrading. This must return true, otherwise run DROP INDEX CONCURRENTLY "LiteLLM_SpendLogs_api_key_startTime_idx"; and build it again:

SELECT indisvalid FROM pg_index WHERE indexrelid = '"LiteLLM_SpendLogs_api_key_startTime_idx"'::regclass;

If your spend logs table is partitioned, Postgres cannot build its index concurrently. Build a matching index concurrently on each partition first, then run the same CREATE INDEX without CONCURRENTLY on the parent, which only attaches them

Session tokens issued before the upgrade stop working. Admin UI and lite CLI users sign in once more after upgrading. During a rolling upgrade, pods on the old and new versions reject each other's session tokens, so finish the rollout before asking users to sign in again. Virtual keys, the master key and stored credentials are unaffected. See a80eaca

lite CLI users must log in again

After upgrading the proxy, every lite CLI user has to run lite login once more. Until they do, the CLI keeps sending its old session token and its requests to the proxy fail

v1.103.1 is a patch release on top of v1.103.0. It carries one change: the session tokens the Admin UI and the lite CLI receive after sign-in now use their own encryption context and a header-safe format. Both the Docker image and the PyPI package were built from 580bde9

UI and CLI session tokens get their own format

Session tokens were encrypted with the same routine the proxy uses for stored credentials, so they carried a v2:gcm: prefix and base64 padding. Basic auth parsers split on the first : and browsers reject : and = in WebSocket subprotocols, so Langfuse pass-through and the realtime playground could not use them. About one login in 262,144 also produced a token starting with sk-, which the proxy then treated as a virtual key and rejected with a 401

Session tokens are now AES-256-GCM encrypted under a context of their own and returned as litellm_login_ followed by unpadded base64url. They pass through any header, are easy to spot in logs, and are checked only as session tokens. Stored credentials keep their current encryption, so there is nothing to migrate

What's Changed

• refactor(auth): bind UI/CLI session tokens to their own AES-GCM context - a80eaca
• chore(lint): scope a TRY004 suppression to the bearer-token salt key check - d3bf006

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.103.0...v1.103.1

Original source