Bifrost HTTP Transport Release v2.3.2

🔒 Security

Go 1.27.3 - Security release for CVE-2026-111943 in crypto/x509: certificate chains through a small set of intermediates from a specific Certificate Authority are no longer accepted by Verify, which covers every TLS connection Bifrost makes to providers, MCP servers and other upstreams. Bifrost now builds with Go 1.27.3, and the Docker images (Dockerfile, Dockerfile.local, Dockerfile.redhat) move to golang:1.27.3-alpine3.24. Custom Go plugins must be rebuilt with Go 1.27.3 to load.

🔧 Maintenance

Dependency Upgrades - Bumped core to v1.13.1 and framework to v1.10.1 across all modules

🗄️ Database Migrations

No new database migrations in this release.

Installation

Docker

docker run -p 8080:8080 maximhq/bifrost:v2.3.2

Binary Download

npx @maximhq/bifrost --transport-version v2.3.2

Docker Images

• maximhq/bifrost:v2.3.2 - This specific version
• maximhq/bifrost:latest - Latest version (updated with this release)

This release was automatically created with dependencies: core v1.13.1, framework v1.10.1. All plugins have been validated and updated.

Bifrost HTTP Transport Release v2.3.0
✨ Features
Agent Gateway (A2A) - Bifrost now proxies A2A agents. Register an upstream agent by its agent card URL (POST /api/agents, POST /api/agents/inspect, or the new top-level agents array in config.json, reconciled by name at startup) and clients reach it at /agents/a2a/<name>/.well-known/agent-card.json, /agents/a2a/<name>/json-rpc (JSON-RPC and SSE streaming) and /agents/a2a/<name>/rest/..., plus an optional shared gRPC listener routed by <agent>.<a2a_grpc_base_domain>:<a2a_grpc_port>. The runtime caches cards and clients, refreshes a stale interface on 404/405/410/426, supports header, OAuth and TLS upstream auth, runs A2A plugin hooks, and delivers push notifications through a durable, retried outbox (/agents/a2a/<name>/push/callback, authenticated with a per-config token). Push callbacks to loopback or private addresses stay blocked unless the new server.a2a_allow_private_push_callbacks is on (default off). Access is governed per virtual key with agent_grants (omit to keep, [] to clear, unknown agents rejected atomically) and allow_by_default per agent; ungranted or disabled agents return 403. Every A2A operation and stream event gets its own log row (relational, ClickHouse, or hybrid with object-store payloads) in a new Agent Logs workspace with session grouping, raw capture and DAC-bounded filters, LLM and MCP logs carry the A2A correlation id, and A2A client spans plus the a2a.client.operation.duration metric reach connectors. The dashboard adds an agent sheet with auth fields and a card preview, push-config UI and an Agent Gateway settings page; Helm adds bifrost.agents[], bifrost.client.a2aExternalClientUrl, bifrost.server.a2aGrpcBaseDomain and bifrost.server.a2aGrpcPort
Warp AI Assistant (alpha) - Warp (formerly Odin) is a new in-dashboard agent that answers questions about the deployment's logs, spend, latency and governance in plain language, using read-only tools (query_logs, count_logs, get_log_detail, get_request_trace, query_metrics, query_usage_by, query_model_performance, render_chart, describe_filter_space, describe_virtual_key, describe_user_limits, ask_user). It asks multiple-choice clarifying questions, draws inline charts, links to matching rows on the Logs page, resolves "today", "yesterday" and bare dates in the asker's time zone, and filters by agent name and x-bf-agent-correlation-id. Admins set a default model plus up to 20 more models users can switch between, temperature, reasoning_effort and max iterations (2 to 20) at /api/warp/config; chat streams from /api/warp/chat with history under /api/warp/conversations (100-conversation cap). Warp calls the gateway in-process, its calls are logged and governed under the app name Warp, signed-in users without an access profile are served, and it only reads rows the asker may see. Governance refusals return their own error codes (access_denied, budget_exceeded, rate_limited, model_blocked), and the chat panel is a resizable dock. Gated by the warp feature flag, on by default in OSS and off by default in Enterprise
Warp Semantic Log Search - Completed LLM logs can be embedded into a vector store so Warp's semantic_search_logs finds conversations by meaning. The embedding provider, model, dimension and namespace are set in Warp settings; new logs are indexed live, and older logs by a resumable, concurrent background backfill (/api/warp/log-index/backfill, /backfill/status, /backfill/cancel, /api/warp/log-index/status). Restricted callers get results prefiltered to rows they can see, the similarity threshold is one 0 to 1 scale on every vector store (default 0.5, Weaviate included), and a finished backfill is hidden once the embedding space changes
GPT Live Sessions - Bifrost now proxies OpenAI GPT Live sessions over WebSocket and WebRTC at /v1/live/sessions, with a sideband /v1/live/sessions/{session_id}/attach WebSocket and recording downloads at /v1/live/sessions/{session_id}/content (logged as live_content). Voice time is billed in 30-second windows from input_cost_per_second on a live pricing row and backend Responses calls are billed per turn, with governance checked at each window and one session log row carrying transcript, delegations and costs. Live has its own live allowed request type (realtime does not imply it). Sessions are bound to the virtual key that opened them, keys are picked only from those governance allows, later billing units on an excluded key return 403, and sideband attachments run through the plugin pipeline as live_sideband with their own log row. Failed backend delegations are recorded on the delegation instead of failing the session, an upstream that ends without session.closed gives the client 502 server_error, a failed WebRTC setup still records the 15-second minimum, x-bf-direct-key works on WebSocket upgrades, and ?provider= or x-model-provider picks the provider on attach and content routes
Cloudflare Workers AI Provider - New first-party cloudflare provider for chat completions (streaming included), Responses, embeddings and model listing against Workers AI. Every key needs cloudflare_key_config.account_id (supports env. references); the base URL defaults to https://api.cloudflare.com/client/v4. Clef decision models (clef, clef-flash) are served natively on /v1/decisions with image input, and the Cloudflare {result, success, errors} envelope is unwrapped. A migration dialog in the UI moves an existing custom provider named cloudflare onto the new provider
Provider-Injected MCP Tools - A provider can carry injected_tools, a list of {name, mcp_client_name, tool_name} entries. Bifrost adds each MCP tool to every chat and Responses request for that provider (past client and request tool filters), runs it server side when the model calls it (non-streaming, streaming chat, streaming Responses and the Anthropic passthrough path), and returns only the final answer with usage summed across turns. An entry named web_search replaces any native web search the client sends; any other name replaces a client tool of the same name. Configure it in the provider's new Tools tab, the provider API or config.json. Depth is capped by mcp_agent_depth
OAuth-Minted Keys for OpenAI-Compatible Providers - Keys on openai and on custom providers with an openai base can set oauth_key_config instead of a static value. Bifrost mints a bearer from the token endpoint with the client_credentials grant (HTTP Basic or auth_style: "body") or the jwt_bearer grant (RFC 7523, RSA/EC signed assertions), caches it per credential, refreshes it a minute before expiry and backs off on rejected credentials. A non-empty value always wins. Available in the API, config.json, Helm and a new "OAuth (minted token)" tab in the key form. Databricks OAuth M2M and GitHub Copilot now mint through the same shared token cache, so a rejected Databricks service principal is backed off instead of hitting the token endpoint on every request
Decisions API in OpenAI Shape - POST /v1/decisions now takes input (text, messages with image parts, or structured data) and an ordered questions list, each with type (predicate, choice, score), optional name, instructions, criteria, choices or levels, plus safety_identifier. Answers come back as a list in question order, with Laya answer fields, response routing and state/truncation usage fields passed through. New /openai/v1/decisions and /openai/decisions routes (also under /langchain, /litellm, /pydanticai) accept the OpenAI SDK's native request; an OpenAI-based provider's body is relayed unchanged and others are rebuilt into OpenAI's response shape
Native Decision Models on More Providers - /v1/decisions now reaches native decision endpoints on OpenAI (gpt-6-luna), Perplexity (pplx-decider-), Ollama (local System One models nimble, tev1, clef, clef-flash, laya on /v1/systemone, with keep_alive and base64 images) and OpenRouter (TypeSafe Jev models and any model the datasheet marks supports_decisions, listed live with response headers forwarded). These providers returned unsupported_operation before (thanks @hcavarsan!)
Decision Emulation Is Opt-In - Decision requests to a model with no native decisions endpoint are no longer emulated through the model's Responses API by default. They return unsupported_operation with a message naming the model and convert_decision_to_responses. Enable emulation with the new compat setting convert_decision_to_responses (UI: Compatibility, "Emulate Decisions API on non-decision models") or per request with x-bf-compat: ["convert_decision_to_responses"]
TypeSafe as a Custom Provider Base - Custom providers can use base_provider_type: "typesafe", so a second TypeSafe-compatible endpoint (for example self-hosted Laya or Nimble) runs with its own keys, name, allowed_requests and request_path_overrides for decisions and list_models, with keyless listing. The dashboard offers TypeSafe as a base format and adds a Decisions toggle to allowed requests
Decision Pricing Rates - New datasheet and override fields input_cost_per_token_decisions and output_cost_per_token_decisions price decisions served natively by chat-row models such as gpt-6-luna; rows without them fall back to the normal token rates. Decision requests on variable-price models no longer bill a negative cost
Complexity Router Decision-Model Classifier - The complexity router can now classify with a decision model (TypeSafe Jev by default; Laya, Nimble, Clef and OpenAI decision models such as gpt-6-luna also work), as the primary classifier or as the semantic classifier's fallback. Only user turns are sent (previous_message_count, timeout), per-tier guidance (definition, signals, examples) can be edited with shipped defaults returned as jev_defaults on the status endpoint, and decision calls are billed under the routing cost with a RequestType. GET /api/models?decisions=true lists only models that serve decisions natively, decisions is a request type in pricing overrides and routing CEL rules, decision logs show their questions, and the LLM fallback classifier no longer sends temperature: 0. The dashboard has a stepped classifier setup and settings sheets
Durable Queue Framework - New framework/queue package: a durable queue with consumer groups, at-least-once delivery, per-key ordering, retries, dead-lettering and lease heartbeats. Backends: in-memory, and a logstore type that keeps queue tables in the logs database on SQLite (single process), Postgres (multi-instance, SKIP LOCKED claims, LISTEN/NOTIFY wakeups) and ClickHouse. Available to framework consumers; the gateway does not use it yet
Multimodal Embeddings - /v1/embeddings input now accepts content parts (text, image, audio, video, file, tokens) and per-item params overrides, served by Cohere, Gemini (including :batchEmbedContents) and Vertex (including gs:// URIs, multimodalembedding@001 video segments and embedContentConfig.documentOcr/audioTrackExtraction). Bedrock gains Nova multimodal (amazon.nova-2-multimodal-embeddings-v1:0) and Titan multimodal (amazon.titan-embed-image-v1) embeddings, encoding_format maps to Cohere embedding_types, and Parasail serves embeddings. OpenAI-compatible providers get only encoding_format and dimensions (Mistral no longer returns 422 on task_type), and the semantic cache hashes task_type, title and auto_truncate. Embedding inputs are stored in a new embedding_input log column and shown in the log view
Shell, apply_patch and Programmatic Tool Callers - The Responses API now supports OpenAI's shell tool with shell_call/shell_call_output items (streaming included) and the apply_patch tool with apply_patch_call/apply_patch_call_output items and diff stream events. allowed_callers is translated between OpenAI's programmatic and Anthropic's versioned code_execution_ callers on chat and Responses. Gemini executableCode/codeExecutionResult parts map to code_interpreter_call items, and code_interpreter reaches Gemini as codeExecution. Shell, code interpreter, image generation and computer tools are dropped for Bedrock and Bedrock Mantle, and the semantic cache treats shell_call and apply_patch_call as tool calls
Retrieve a Single Model - New GET /v1/models/{model} returns one model's metadata, implemented for OpenAI, Anthropic (with alias resolution), Cohere, Gemini, Mistral and Ollama, with shutdown_date where the provider announces one. Gated by the model_retrieve allowed request and exempt from budgets and rate limits
xAI Realtime - xAI is now a realtime voice provider on the WebSocket gateway, including client-secret minting through /v1/realtime/client_secrets. Browser clients can authenticate with the xai-client-secret.<token> subprotocol as well as openai-insecure-api-key.<token>; an empty, whitespace-containing or conflicting subprotocol credential returns 400 before the upgrade, and mint requests that carry session settings are rejected (send them with session.update). Responses streaming keeps the provider stop reason from the newest chunk that has one, and the logging plugin marks refusal, content_filter and safety stops as errors of type model_refusal
Bedrock Passthrough - New /bedrock_passthrough route forwards the three Bedrock operations Bifrost has no native route for: InvokeAgent (streamed as AWS sends it), knowledge-base Retrieve, and ApplyGuardrail. Any other path, method or malformed identifier returns 400. InvokeAgent and Retrieve are SigV4-signed with the key's AWS credentials (an API-key-only key is refused with 400), and AWS status, body and X-Amzn-* headers come back unchanged. Azure prompt_filter_results and per-choice content_filter_results are now passed through on chat responses
Vertex AI Workload Identity Federation from AWS - Vertex keys can set vertex_key_config.aws_workload_identity (audience, service_account_email, token_lifetime_seconds, aws_region, aws_role_arn) to authenticate from EKS IRSA, Pod Identity, ECS task roles or EC2 instance profiles with no stored GCP or AWS key. gcloud-generated AWS external_account JSON now also resolves AWS credentials through the AWS SDK chain. New "Workload Identity (AWS)" tab in the key form
Web Search Priced per Request - Provider-hosted web search is now reported as usage.tool_usage.web_search.num_requests (top-level tool_usage on Responses) for Anthropic, Gemini, Perplexity and OpenAI Responses, and billed with the new pricing field web_search_cost_per_request. Existing values are copied over by migration, and datasheet rows still carrying search_context_cost_per_query are read as the new field
Ignore Provider-Reported Cost - New per-provider ignore_provider_cost setting (UI: provider Pricing tab, default false) discards the usage.cost a provider reports, for providers such as Cortecs that report credits, and prices requests from the model catalog and pricing overrides for logs, budgets, telemetry and streamed totals. The client response keeps the provider's figure
Compat: Route Reasoning-Only Models to Responses - New compat setting force_reasoning_only_models_to_responses sends chat completion requests with tools to the Responses API when the datasheet marks the model as not supporting reasoning_with_tool_calls, and converts the answer back to chat format, so reasoning is kept instead of turned off. Defaults to true, also available per request through x-bf-compat
Chat reasoning.mode and reasoning.summary - A chat request carrying reasoning.mode (standard or pro) on OpenAI or Azure is served through the Responses API so pro is honoured; other providers report it as a dropped compat param. Chat requests can also set reasoning.summary (auto, concise, detailed), forwarded when chat is served through the Responses API
DeepSeek Served Model Tracking - When DeepSeek serves a different model than requested (retired IDs silently upgraded), the served model is recorded as ServerSideFallbackModel for pricing and routing info, and a datasheet server_side_model covers the Anthropic-compatible endpoint, which echoes the requested ID
Failure Classification and Smarter Key Rotation - Every failed attempt now gets a failure_class on the attempt trail (credential, quota, model_access, model_gone, region_blocked, rate_limit, transient, caller_fault, timeout, cancelled, unknown), read from provider error codes rather than status alone. Credential and quota failures hold a key out for every model; keys refused for the model, a retired model or a region block are rotated past, and permanent per-key failures walk the key pool even at max_retries: 0. Attempts Bifrost ends itself are recorded as timeout (504) or cancelled (499). Provider retry hints (Retry-After, retry-after-ms, Gemini RetryInfo) are exposed as extra_fields.retry_after_ms, clamped to 1s to 5m
Time-to-First-Token Cutoff - Routing rule targets accept ttft_timeout_ms (1 to 300000). A streaming attempt that does not produce its first token in time is cut off and handed to the next fallback without cancelling the request; the last attempt is never cut off, and non-streaming requests are unaffected. A cut-off attempt fails with code ttft_timeout and status 504 on the attempt trail. The rule editor has a "Time to first token cutoff (ms)" field per target, also in Helm
Requested Route in Routing Info - When a routing rule, load balancer or session router rewrites the model or provider, the caller's original route is returned as routing_info.requested_provider / requested_model and in the x-bifrost-routing-info-requested-provider and x-bifrost-routing-info-requested-model response headers
Gateway Request ID Header - Every traced response now carries x-bifrost-request-id, the ID the request is logged under (GET /api/logs/{id}), next to x-bifrost-trace-id. Provider response headers whose names start with x-bifrost- are no longer forwarded to the caller
Deprecated Models Hidden from Browsing - GET /api/models without a query now leaves out deprecated models and their counts; with a query they are kept but sorted last. Pass include_deprecated=true to get them all
Provider Objects Bound to the Creating Virtual Key - Files, batches, videos, containers (and container files) and cached contents created through a virtual key are now recorded in a new provider_objects table. Retrieve, content, delete, cancel, results and remix on another key's object return 404, list pages are narrowed to the caller's own objects, and indirect references (a batch on another key's input file, a container with file_ids, a request naming cached_content) are checked the same way under every id form the provider accepts. A failed ownership lookup returns 500 instead of an unchecked page, and id lists over 1000 no longer fail large list pages. A new Restrict Provider Objects to the Creating Virtual Key switch under Settings > Security (client.enforce_vk_object_ownership, default true, applied without a restart) turns the restriction off; creators are still recorded while it is off
File-Based Batches Under Model-Restricted Virtual Keys - A batch defined by input_file_id or input_blob is now judged by what the provider actually runs. On OpenAI and Azure (and custom providers built on them), which run the models named inside the file, the virtual key must allow every model of the provider. On Gemini, Vertex and Bedrock the request model must be present and allowed. A batch create that sends a file or blob together with inline requests returns 400
Passthrough Route Hardening - Passthrough routes (/openai_passthrough, /anthropic_passthrough, /azure_passthrough, /genai_passthrough and siblings) now forward only POST requests to a fixed allowlist of inference paths per provider (chat, completions, responses, embeddings, moderations, audio, images, generateContent, countTokens, predict and similar). Paths that could change the upstream host (not rooted, //, userinfo @, schemes, backslashes, .., control characters, percent-encoded forms included) are refused, the final URL must stay on the provider's host, and x-model-provider can only confirm the route's provider. The model is read strictly from the JSON body, and governance checks every model a body names, including each task of a Runware task array
Management API Refuses Unlisted Browser Origins - While dashboard auth is disabled or unconfigured, and on deployments with config_store.enabled: false, browser requests to /api/* (and /ws and /metrics without a config store) from an origin that is not localhost, the gateway's own origin, or listed in client.allowed_origins (patterns such as https://.example.com work, a bare * does not count) now return 403. Requests without an Origin header (CLI, SDKs), whitelisted routes and inference routes are unchanged, and enabling dashboard auth lifts the auth-off check
Real Auth Required for Secret References and Private Targets - Requests admitted only because dashboard auth is off now get 403 when they: use env./vault. references in MCP client credential fields (headers, connection string, OAuth, token exchange, TLS) on POST /api/mcp/client or PUT /api/mcp/client/{id}; create, re-point or redeliver to a webhook endpoint with allow_private_network; set an outbound endpoint (collector_url, metrics_endpoint, push_gateway_url, endpoint, url, base_url) or any env./vault. reference on a built-in plugin (OTEL, telemetry); set, change or remove a Vertex key's vertex_key_config.auth_credentials (the JSON names the hosts the Google client dials); or delete MCP tool logs via DELETE /api/mcp-logs. config.json provisioning is not affected (thanks @xujiantop-crypto!)
Auth Config Changes Need Proof of Control - When an admin account exists but dashboard auth is switched off, re-enabling auth or replacing the admin username or password through PUT /api/config now requires current_password (the stored admin password) or a valid setup_token in the request. The Security page asks for the current password in that case
Request Decompression Limits - Compressed request bodies may now expand to at most 100 times their compressed size (1 MiB floor, client.max_request_body_size_mb ceiling), and a process-wide in-flight budget of four times the body cap bounds buffered decompression. The budget is charged for bytes actually inflated, so ordinary concurrent gzip traffic is not refused
Access Profiles for Teams, Business Units and Customers - (enterprise) A team, business unit or customer can hold an access profile instead of its own budgets and rate limit, and every virtual key under it is governed by that profile. Virtual keys can now be owned by a business unit (business_unit_id), and a business unit holds at most one budget. config.json accepts access_profile on governance.teams, governance.customers and governance.business_units, roles accept the new business-unit-data and customer-data data access scopes, and the dashboard hides a key's own limit editors when a profile governs it
Per-Virtual-Key Content Logging Control - Virtual keys have a tri-state disable_content_logging: omitted or null inherits client.disable_content_logging, true keeps that key's request and response content out of the log store and strips it from OpenTelemetry export (MCP arguments included), false keeps content in the log store even when the client setting turns it off. The per-request x-bf-disable-content-logging header still wins. The virtual key editor has an Inherit / Off / On select, and the CLI has --disable-content-logging
Auto-Delete Expired Virtual Keys - Virtual keys can set delete_after_expire (requires expires_at, 400 otherwise). A daily job deletes flagged keys once they expire and posts one notification with the counts. The expiry picker has a matching option
Global Proxy in config.json and for All Outbound Traffic - The global outbound proxy can be declared as a top-level proxy_config in config.json (enabled, url, username, password, no_proxy, timeout, skip_tls_verify, enable_for_scim, enable_for_inference, enable_for_api) and in Helm. With enable_for_api on, MCP connections, MCP OAuth discovery, model catalog sync, OpenTelemetry export, webhooks, skill fetches and plugin downloads go through it, each keeping its own SSRF policy. With enable_for_inference on, providers without their own proxy_config now actually send inference through it (the flag was stored but never read before), and multi-entry no_proxy lists match on both fasthttp and net/http paths
Adaptive Routing Incidents and Attempt Metadata - (enterprise) Adaptive Routing has an Incidents page in the sidebar. Load balancer decisions for each attempt are written to log metadata under the reserved bifrost_alb_ prefix (caller headers cannot spoof them) and hidden from the log filter sidebar and metadata grid. Docs cover held keys, incidents and the /api/adaptive-routing endpoints
Governance Dashboard Updates - (enterprise UI) New Org Chart page under Governance and an enterprise Home landing page; Business Units navigation checks its own BusinessUnits permission; new BackgroundJobs RBAC resource; Warp follows RBAC; create, edit, delete and rotate controls follow the viewer's permissions with a tooltip explaining why they are disabled; the customers list shows a server-side team count
SSO and SCIM Updates - Every SSO provider's SCIM block accepts attributeProjectMappings (add users to existing projects when an IdP attribute matches a value or glob; memberships granted by a rule are revoked when it stops matching) and bulkSyncInterval (directory reconcile interval, default 24h, from 1h to 30d), in config.json and Helm. The generic SSO provider accepts a login displayName, and a new identity sync guide aligns the IdP setup docs
Audit Logs: Omit IP Addresses - (enterprise) New audit_logs.omit_ip_addresses (Helm omitIpAddresses, default false) stops recording client IPs in audit logs. New events store no request_ip or initiator host, older IPs are hidden from APIs, exports and archives (not deleted), and IP search and filters are disabled. HMAC signatures stay valid
Attribution Context Keys Deprecated - The raw attribution context keys (BifrostContextKeyGovernanceVirtualKeyID, ...TeamID, ...CustomerID, ...BusinessUnitID, ...ProjectID, ...BudgetIDs, ...RateLimitIDs, BifrostContextKeyUserID, ...UserName, ...UserEmail and related variants) are deprecated in favour of the ctx.Grant() accessors. Values and behavior are unchanged
MCP Client Target Validation at Connect Time - MCP clients registered while dashboard auth is off now carry a server-set require_public_target flag, and every later dial (reconnects, per-call connections, proxied hops) is held to public addresses, not only the registration-time DNS check. Existing clients default to false
MCP OAuth Server Hardening - Bifrost's MCP OAuth authorization server accepts remote client callback URIs only when listed in client.oauth2_server_config.allowed_redirect_uris (loopback and cursor://anysphere.cursor-mcp are always allowed), re-checks existing registrations at authorization and token exchange, and shows the exact callback on the consent page. Oversize DCR/authorize fields (client_name, scope, redirect_uris, state, code_challenge) return 400 before anything is stored
MCP OAuth Private Network Allowlist - OAuth-protected MCP servers on localhost or a private network can now be added. List hostnames, IPs or CIDRs in client.mcp_oauth_private_allowlist (or mcp.tool_manager_config.oauth_private_allowlist, which wins) and the MCP OAuth chain (discovery, dynamic client registration, token exchange, refresh) may reach them for clients registered by an authenticated admin or declared in config.json. Link-local and cloud metadata addresses stay blocked, invalid entries stop boot, and edits apply to the next connection
Virtual Key Rotation Revokes MCP OAuth Grants - Rotating a virtual key now revokes the MCP OAuth refresh tokens minted with its old value and any consented but unexchanged authorization code, in the same transaction as the rotation. OAuth consent in vk mode requires the key's current value
MCP Server Instructions - The instructions an upstream MCP server returns from its initialize handshake are now captured, persisted with discovered tools and refreshed with them. Bifrost forwards them on its own /mcp gateway and injects them into MCP-bearing chat (as a leading system message) and Responses (appended to instructions) requests as tagged <mcp_server name="..."> blocks, scoped to the servers the request can see. Caps are mcp.tool_manager_config.max_instructions_per_client (default 4096 bytes), max_instructions_total (default 16384 bytes) and a per-client max_instructions_length override. In code mode the instructions appear in the .pyi stubs
MCP Connection Recovery and Tool Refresh - POST /api/mcp/client/{id}/refresh-tools (and a Refresh tools action in the client table) re-discovers a client's tools immediately, for every client type including per-user and per-call clients. Tool calls that fail because the server dropped its session ("session terminated", "session not found", "need to re-initialize") trigger an immediate reconnect and, where safe, a retry; a failed live-connection check now triggers the documented reconnect, and a failed make-before-break re-dial keeps the live connection. Health probes retry timeouts but treat dead sessions and auth rejections as permanent. POST /api/mcp/client/{id}/reregister (dashboard: "Reauthorize with a new client") registers a new dynamic OAuth client and runs consent against it when the upstream forgot the old client_id
Code Mode Execution Limits - Code mode now runs in-process on a pinned Starlark runtime with per-execution budgets instead of child processes. New mcp.tool_manager_config.code_mode_limits (also client.mcp_code_mode_limits through the config API, the MCP settings page and Helm) sets max_source_bytes (default 64 KiB), max_steps (1,000,000), max_memory_bytes (64 MiB), max_log_bytes (64 KiB), max_tool_calls (64), max_value_bytes (1 MiB, minimum 1024) and max_nesting_depth (64, maximum 1000). Cyclic or oversized results are rejected before serialization
Disabled MCP Clients on Virtual Keys - Each mcp_configs entry on a virtual key now shows its MCP client's disabled value, and the docs explain that a disabled client offers no tools even when the key lists it
MCP Library Additions - FXMacroData and Parlel are added to the community MCP library (thanks @roberttidball!)
Semantic Cache Scoped by Identity - Semantic cache entries are now scoped to the identity that wrote them: the request's virtual key plus its user, when the request is attributed to one. The scope is part of the params hash, so it covers direct hits and semantic search alike, and a body field named identity_scope cannot claim another identity's scope. The stored cache_key is now the raw value the caller sent (the vk: / raw:vk: rewriting is gone), so DELETE /api/cache/clear-by-key/{cacheKey} clears every identity's entries under that key. New disable_identity_scoping option (default false, UI switch "Scope Cache by Identity") shares entries across identities
Semantic Cache Skips Tool-Call Responses by Default - Responses that carry client-executed tool calls (chat tool_calls, Responses function_call, custom_tool_call, computer_call, local_shell_call, mcp_call, mcp_approval_request) are no longer written to the semantic cache unless cache_tool_call_responses is true. Server-side tool items such as web search are still cached
Semantic Cache Response Headers - Responses that went through a semantic cache lookup now carry x-bifrost-cache- headers mirroring cache_debug: x-bifrost-cache-hit (always sent when a lookup ran), plus -id, -hit-type, -hit-latency-ms, -requested-provider, -requested-model, -similarity, -threshold, -provider-used, -model-used and -input-tokens when known. This works on streams and on the Anthropic, GenAI, Bedrock and Cohere routes, whose bodies have no extra_fields
Trace Failure Attributes and End-to-End TTFT - The root span now carries bifrost.error.type, error.type, gen_ai.error, gen_ai.error.code and the HTTP status, including for requests rejected before dispatch, and streaming requests get an end-to-end time-to-first-token attribute that includes queueing, plugins and key selection
Raw Payload Export to Connectors - Connectors can receive the exact provider request and response bodies. Set export_raw_payloads: true on an OTel profile (or BigQuery) to attach bifrost.raw_request / bifrost.raw_response to LLM spans. It requires store_raw_request_response on the provider, is suppressed by disable_content_logging, and drops bodies over 256 KB. Failed requests export the raw request and the provider's error body. Exposed in the OTel profile form and Helm
Per-Connector Attribute Denylist - New excluded_attributes on OTel profiles (and the Datadog, Kafka, Pub/Sub, Splunk and BigQuery connectors) drops listed span attributes by exact name, such as gen_ai.request.tools, before export. BigQuery takes column names (for example tools); trace_id and timestamp cannot be excluded. Unknown names are ignored
Cost Breakdown on Connector Spans - LLM spans now carry a per-category cost breakdown under bifrost.cost.* (input/output/additional sides with text, audio, image, cached read/write, request, reasoning, citation, search queries, guardrail, MCP, semantic cache and routing). gen_ai.usage.cost is still emitted. Plugins can call ctx.CalculateCostBreakdown(resp) for the same split
App Label on Metrics - Requests are classified by User-Agent into an app value (Claude Code, Cursor, Codex CLI, Cline, Windsurf, API, Other), exported as the bifrost.app span attribute and an app label on request-level Prometheus metrics
Datadog Metric Dimensions - New Datadog metric_dimensions lists x-bf-dim-* request dimension names to add as DogStatsD metric tags. Dimensions already reached Datadog spans; metrics stay untagged unless listed, to bound cardinality
Lower Tracing Overhead - Tracing now does only the work attached connectors read: attribute maps, content summaries and export snapshots are skipped when no connector is attached, plugin spans are created only when a connector consumes them, and LLM spans carry a typed payload instead of an untyped attribute bag. Connectors that predate the new interfaces still get everything
OTel HTTP Semantic Conventions - The root HTTP span now emits the stable OTel HTTP attributes http.request.method (unknown methods become _OTHER with http.request.method_original), http.response.status_code, http.route (the matched route template), url.path, url.scheme and user_agent.original. The deprecated http.method, http.url, http.user_agent and http.status_code are still emitted
Response Headers Plugin Hook - New HTTPTransportResponseHeadersHook runs once, in reverse registration order, just before response headers are sent, for buffered and streaming responses alike, so plugins can set headers on SSE streams. Content-Length, Transfer-Encoding and Connection cannot be changed and the status code is read-only. Native .so plugins may export the symbol optionally
Session Grouping in Logs - The grouped logs view collapses all requests that share a session_id under the session's earliest root request, with expandable session rows, session totals and a redesigned expander (tree lines, turn and fallback labels). GET /api/logs gains group_sessions=true (requires roots_only=true; ignored when session_id, parent_request_id or request_id is set), and collapsed rows carry session_child_count, session_total_cost and session_total_tokens
Background Jobs Panel - New GET /api/sidekiq/jobs and POST /api/sidekiq/jobs/{id}/cancel list every pending, running or recently finished background job (cost recalculation, Warp log indexing) with progress, and stop one. The notification panel has a Background jobs tab with count badges, Cancel / Stop job and Clear finished. Access uses View and Update on BackgroundJobs
Exclude Request Types from Object Storage - New logs_store.object_storage_exclude_request_types lists request types (the log object value, for example list_models, embedding, chat_completion_stream) whose logs stay in full in the logs database instead of being offloaded to S3/GCS. Streaming and non-streaming variants are separate types. Logs with hidden content are still cleared from the DB row and not uploaded
Above-100k Token Pricing Tier - The cost engine now applies the datasheet's _above_100k_tokens rates (input, output, cache read, 5m and 1h cache write) once a prompt exceeds 100,000 tokens, as Claude Haiku 5.5 bills. Higher tiers (128k, 200k, 272k) still take precedence. The custom pricing override form has the new fields
Guardrail Detections - Guardrail metadata can now record detect-only policy matches (detections: phase, rule, guardrail, reason and sanitized categories, never the matched text) alongside judge calls. Docs add the A10 Guardrails integration, regex allowed_values, profile execution order, and the prompt guardrails judge model requirements (max_output_tokens default 1000)
Unified Provider and Model Selectors - Every provider and model picker in the dashboard (provider keys and deployments, CEL builder, routing rules, pricing overrides, model catalog, limits, caching and embedding settings, prompts, model access and the add-provider menu) now uses one ProviderSelector and one ModelSelector, with icons, server-side search and paging, multi-select chips and custom-provider icon fallback. Provider lists in tables collapse into a +N chip
Entity Picker Infinite Scroll - Virtual key, team, customer, virtual MCP and MCP client pickers load further pages on scroll instead of stopping at the first 20 results, and fetch more when excluded rows leave the list short
Responsive Toolbars - Virtual Keys, Webhooks and provider card toolbars collapse to icon-only buttons at narrow widths
Clearer Client Tool Calls in Logs - A Responses log that ends on a client-side tool call labels it as a call (not a result), shows "no arguments" for empty arguments, and links to the next request in the session that carries the tool result
Sign-In Redirect on MCP Auth Links - When mcp_enable_temp_token_auth is off, visitors opening an OAuth consent or MCP per-user-headers link are sent to /login and back instead of seeing "invalid link". A 401 now carries error.code temp_token_auth_disabled or temp_token_rejected when relevant
Bifrost CLI - The bifrost CLI covers the published API: api (generated catalog of OpenAPI operations with search and tags), resource commands (models, responses, videos, providers, ...), chat, infer, request/http, context, auth, usage, keys, configure/unconfigure (reversible native-agent setup with receipts and rollback) and completion. It authenticates to the gateway (SSO in the interactive launcher and authenticated harness launches, with its own browser auth identity). Output is --output table|json|yaml|raw, and destructive operations require --yes
Helm Chart 2.1.44 - The chart can now set the config fields added since 2.1.43, including proxy_config, entity access_profile, the business-unit-data/customer-data scopes, routing ttft_timeout_ms, storage.configStore/storage.logsStore statementTimeout and idleInTransactionSessionTimeout, bifrost.client.compat.forceReasoningOnlyModelsToResponses, bifrost.client.deleteExpiredVirtualKeys, bifrost.client.oauth2ServerConfig.allowedRedirectUris, virtual key delete_after_expire/disable_content_logging/business_unit_id, bifrost.mcp.clientConfigs[].requirePublicTarget, semantic cache cache_tool_call_responses, object-form routing rule fallbacks, bifrost.governance.complexityAnalyzerConfig.jev.criteria and bifrost.mcp.toolManagerConfig.maxInstructionsPerClient/maxInstructionsTotal. New topologySpreadConstraints spreads replicas across nodes and zones, and extraContainers adds sidecars (native sidecars go in initContainers with restartPolicy: Always, refused on Kubernetes older than 1.29)
⚙️ Upgrade Notes
These entries change a default, a status code, a response shape or a Go SDK type. The matching entries above describe each change in full.
Agent Gateway (A2A) - optional; set client.a2a_external_client_url for push and stable card URLs, and server.a2a_grpc_base_domain + server.a2a_grpc_port (plus wildcard DNS) for gRPC. Custom schemas.Tracer implementations must add ReleaseTrace.
Warp AI Assistant (alpha) - on by default in OSS; set feature_flags.flags.warp.enabled: false to keep it hidden.
Cloudflare Workers AI Provider - the name cloudflare is reserved; migrate a custom provider named cloudflare with the dashboard dialog or recreate it with cloudflare_key_config.account_id.
Decisions API in OpenAI Shape - map-form /v1/decisions bodies are deprecated (still served); update Go SDK code to the new BifrostDecisionRequest fields.
Decision Emulation Is Opt-In - decisions on non-decision models return unsupported_operation; set compat.convert_decision_to_responses: true or send x-bf-compat: ["convert_decision_to_responses"] to keep emulation.
Multimodal Embeddings - Go SDK BifrostEmbeddingRequest.Input is now []EmbeddingInputItem; HTTP bodies are unchanged.
xAI Realtime - refusal and content-filter stops are logged as model_refusal errors; exclude that error type from error-rate alerts if needed.
Web Search Priced per Request - rename search_context_cost_per_query to web_search_cost_per_request in config.json overrides; read counts from usage.tool_usage.web_search.num_requests.
Compat: Route Reasoning-Only Models to Responses - turned on for existing deployments; set compat.force_reasoning_only_models_to_responses: false to keep the old behavior.
Chat reasoning.mode and reasoning.summary - the /openai chat route strips reasoning.summary; send it through extra_params there.
Failure Classification and Smarter Key Rotation - a model missing on every key returns the provider's 404 instead of 502 upstream_credentials_exhausted; update alerts on that error and on fail_reason values.
Gateway Request ID Header - read the gateway request ID from x-bifrost-request-id; upstream x-bifrost- headers are no longer forwarded.
Deprecated Models Hidden from Browsing - add include_deprecated=true to GET /api/models to keep deprecated models.
Provider Objects Bound to the Creating Virtual Key - set client.enforce_vk_object_ownership: false if virtual keys share files, batches, containers or cached contents.
File-Based Batches Under Model-Restricted Virtual Keys - use a * model grant for OpenAI/Azure file batches, send an allowed model for Gemini/Vertex/Bedrock file batches, and never mix a file with inline requests.
Passthrough Route Hardening - use passthrough only for POST inference paths with rooted paths, the route's own provider and one string model key.
Management API Refuses Unlisted Browser Origins - add cross-origin browser apps to client.allowed_origins or enable dashboard auth.
Real Auth Required for Secret References and Private Targets - enable dashboard auth or send X-Bifrost-Setup-Token for MCP env./vault. references, private webhooks, plugin endpoints, Vertex auth_credentials and DELETE /api/mcp-logs.
Auth Config Changes Need Proof of Control - send auth_config.current_password or setup_token when re-enabling auth or rotating admin credentials while auth is off.
Request Decompression Limits - bodies expanding over 100x return 413; send them uncompressed or raise client.max_request_body_size_mb.
Access Profiles for Teams, Business Units and Customers - move config.json limits of governed entities and their keys onto the access profile.
Global Proxy in config.json and for All Outbound Traffic - set enable_for_inference: false to keep inference direct, and list direct hosts in proxy_config.no_proxy.
Audit Logs: Omit IP Addresses - make SIEM rules accept both audit typeURI values.
Attribution Context Keys Deprecated - read attribution in custom plugins from ctx.Grant().
MCP Client Target Validation at Connect Time - register private MCP servers as an authenticated admin or in config.json, and use http, sse or stdio as connection_type.
MCP OAuth Server Hardening - list remote OAuth callback URIs in client.oauth2_server_config.allowed_redirect_uris before upgrading.
Virtual Key Rotation Revokes MCP OAuth Grants - re-authorize MCP clients after rotating their virtual key.
MCP Server Instructions - bound injected instructions with max_instructions_per_client, max_instructions_total or per-client max_instructions_length.
Code Mode Execution Limits - raise fields under mcp.tool_manager_config.code_mode_limits for scripts that exceed the new budgets.
Semantic Cache Scoped by Identity - expect a cold cache; set disable_identity_scoping: true to share entries across identities.
Semantic Cache Skips Tool-Call Responses by Default - set cache_tool_call_responses: true to keep caching them.
App Label on Metrics - add the new app label to recording rules or alerts that match exact label sets.
OTel HTTP Semantic Conventions - move dashboards to http.request.method, http.response.status_code, http.route and url.path; the span name and http.url drop the query string.
Response Headers Plugin Hook - Go types implementing HTTPTransportPlugin must add HTTPTransportResponseHeadersHook (a no-op is fine).
Exclude Request Types from Object Storage - only exclude request types whose content you do not need retained in object storage.
Helm Chart 2.1.44 - set bifrost.client.oauth2ServerConfig.issuerUrl when mcpServerAuthMode is oauth or both, or helm template fails.
Outbound Fetch Redirect Hardening - test-fire private webhooks with an authenticated caller; point datasheet URLs at their final location or use file://.
Provider base_url Shape Checked - remove userinfo, query strings and fragments from base_url.
Direct Keys Stay on Their Provider - pass ?provider= or x-bf-model-provider on direct-key model listings.
config.json Governance Limits Reconciled Correctly - read virtual key and provider config limits from the key's model configs after the migration.
Partial Provider Key Updates - send a field explicitly to clear it.
Weights Must Be Valid Numbers - use weights of 0 or more for keys and greater than 0 for routing targets.
Duplicate Routing Rule Priority Returns 409 - treat 409 as a request error instead of retrying on 500.
Dashboard Rankings Under DAC - treat the ranking row id: "other" like unassigned.
MCP OAuth and Client Fixes - expect 404, not 500, for an unknown MCP client.
Implicit Prompt Caching for gpt-5.6+ Now Takes Effect - set prompt_cache_options.mode: "explicit" on the request to opt into explicit mode.
Stream Errors Get a Real HTTP Status - update retry logic and alerts that assumed every stream error was 400.
Bedrock Incomplete Converse Streams - truncated Converse streams now end with 502.
Replicate File List Cursor - pass the after cursor exactly as returned; others get 400 invalid pagination cursor.
Anthropic Error and Stop Fidelity - branch on documented Anthropic error types instead of api_error.
Error Metrics Report the Real Status - repoint alerts on status_code="unknown" and add error_type to OTel queries.
OTLP HTTP Metrics to https Endpoints - set tls_ca_cert or keep insecure: true for self-signed collectors.
Connector Export Gaps - group OTel datapoints by bifrost_instance_id instead of service_instance_id, and review telemetry custom_labels for cardinality.
Postgres Scale Fixes - plan IO for the background index build; set statement_timeout and idle_in_transaction_session_timeout to "0" to keep server defaults.
ClickHouse Large Team Scopes - set logs_store.config.max_query_size if a ClickHouse settings profile constrains it.
Semantic Cache Vector Store Consistency - on Weaviate, set threshold to 2 * old - 1 to keep the old hit rate.
🐞 Fixed
Outbound Fetch Redirect Hardening - Pricing and model-parameter datasheets, the MCP library catalog, OAuth2 discovery, registration and token calls, webhook delivery and the config API URL check now enforce the private-network dial policy on every connection and redirect hop, not only the first URL. Failed URL checks on PUT /api/config no longer echo transport details
Encrypted Session and Virtual Key Lookups - The legacy plaintext fallback for sessions and virtual keys now matches only rows stored in plaintext, so an encrypted row's ciphertext can no longer be used as a session token or virtual key value
Virtual Key Header Redacted in Traces - x-bf-vk is now treated as a credential and redacted in captured request headers, so observability connectors no longer export it
Provider base_url Shape Checked - Provider base_url on create and update must be an origin with an optional path; URLs with credentials, a query (including a bare ?) or a fragment are rejected before anything is stored or dialed
Direct Keys Stay on Their Provider - A caller's x-bf-direct-key is no longer sent to every configured provider on an unscoped model listing, and drop-in routes (/openai/v1/models, /anthropic/v1/models, /genai/v1beta/models) list only their own provider. A refused direct key returns the provider's own error (for example OpenAI's 401) instead of 502 upstream_credentials_exhausted, and no session key binding is created when the direct key serves
Every Provider Egress Path Uses proxy_config - Side fetches (image and document URLs), credential exchanges (Vertex OAuth, Azure Entra ID, Databricks OAuth) and Bedrock runtime calls now go through the provider's proxy_config instead of connecting directly. https:// proxies are reached over TLS (system roots, ca_cert_pem, or skip_tls_verify), proxies are dialed dual-stack so IPv6 proxies work, proxy handshakes are bounded at 30s, runtime proxy changes reach already-created clients, and DisablePathNormalizing is honoured through the proxy-aware client
No Shutdown Panic From Request-Scoped Work - Work started while serving a request no longer derives its context from the fasthttp request, which could panic with "missing cancel error" during shutdown
config.json Governance Limits Reconciled Correctly - Budg

Plugin Release: telemetry v1.9.0

feat: add app label to request-level metrics

feat: derive metric labels from the enrichment dimension registry

feat: key health and rotation metrics follow the new failure classes, including timeout and cancelled

feat: implement HTTPTransportResponseHeadersHook as a no-op

fix: label error metrics with the effective HTTP status instead of unknown

fix: apply plugin custom_labels and resolve the plugin per request after reloads

fix: record billed usage from cancelled and failed requests

fix: build each metric's label set in a fresh slice so custom labels do not alias

chore: upgraded core to v1.12.0 and framework to v1.9.0

Installation

Update your go.mod to use the new plugin version

go get github.com/maximhq/bifrost/plugins/[email protected]

This release was automatically created from version file: plugins/telemetry/version

Plugin Release: semanticcache v1.7.0

feat: scope cache entries by virtual key and user, with a disable_identity_scoping opt-out

feat: cache_tool_call_responses (default false) skips caching responses that carry client-executed tool calls, shell_call and apply_patch_call included

feat: record lookup outcome for x-bifrost-cache-* response headers, including streams

feat: implement HTTPTransportResponseHeadersHook as a no-op

fix: isolate cache entries by request family (thanks @Atharva-Kanherkar!)

fix: snapshot the response before PostLLMHook returns so async cache writes do not race core cleanup

fix: placeholder vector on embedding failure and stricter config validation

fix: include embedding task params in the cache hash

chore: build cache keys from the new embedding input shape and the new oauth_key_config key schema

chore: upgraded core to v1.12.0 and framework to v1.9.0

Installation

Update your go.mod to use the new plugin version

go get github.com/maximhq/bifrost/plugins/[email protected]

This release was automatically created from version file: plugins/semanticcache/version

Plugin Release: routing v1.2.0

feat: decision-model complexity classifier (TypeSafe Jev, Laya, Nimble, Clef and OpenAI decision models) as primary or semantic fallback, with per-tier guidance and decision cost tracking

feat: send complexity classification decisions in the list shape

feat: ttft_timeout_ms sets the first-token deadline for streaming attempts

fix: LLM fallback classifier no longer sends temperature 0

chore: read embedding input through the new item shape

chore: upgraded core to v1.12.0 and framework to v1.9.0

Installation

Update your go.mod to use the new plugin version

go get github.com/maximhq/bifrost/plugins/[email protected]

This release was automatically created from version file: plugins/routing/version

Plugin Release: otel v1.6.0

feat: export A2A client spans and the a2a.client.operation.duration metric

feat: add export_raw_payloads profile option

feat: add excluded_attributes profile option

feat: export bifrost.cost.* breakdown attributes

feat: read the typed LLM span payload

feat: OTLP exporters dial through the global API proxy

feat: strip exported content for virtual keys that turn content logging off

feat: implement HTTPTransportResponseHeadersHook as a no-op

fix: report effective status and add error_type to bifrost_error_requests_total

fix: honour the https scheme on OTLP HTTP metrics endpoints

fix: rename the per-replica datapoint label to bifrost_instance_id

chore: upgraded core to v1.12.0 and framework to v1.9.0

Installation

Update your go.mod to use the new plugin version

go get github.com/maximhq/bifrost/plugins/[email protected]

This release was automatically created from version file: plugins/otel/version