Agent Router Release Notes

First release as Agent Router: MCP security hardening, bare MCP tool names, caller-supplied MCP credentials, AWS OpenAI-compatible and TypeSafe backends, and Envoy Gateway v1.9.

Agent Router v1.2.0 is the first release under the project's new name (formerly Envoy AI Gateway). Nothing you deploy is renamed: the aigateway.envoyproxy.io API group, the CRDs, the Helm charts, the container images, and the aigw CLI all stay the same. This release hardens MCP authentication and cross-namespace references. It adds bare tool names, caller-supplied credentials, and policy merging to MCPRoute, plus two new backend schemas (AWSOpenAI and TypeSafe). It also moves to Envoy Gateway v1.9.2, Gateway API v1.6, and Gateway API Inference Extension v1.6. Several security fixes change behavior, so read the breaking changes before you upgrade.

✨ New Features

Agent Router

Same project, new name

Envoy AI Gateway is now Agent Router, an Agentic AI Foundation project with the same code, maintainers, and Apache 2.0 license. The website moved to theagentrouter.ai and the repository to theagentrouter/agent-router; old links redirect. Your manifests, Helm values, and automation keep working unchanged.

MCP Gateway

Bare tool and prompt names with prefixMode: Never

Clients that hardcode tool names, such as interactive MCP Apps, no longer have to see the __ prefix. Set prefixMode: Never on the route or on individual backendRefs. A Never-mode backend must list its tools in toolSelector.include, and the controller marks the route NotAccepted if two Never-mode backends expose the same name. Prompts are exposed bare only when they are listed in the new promptSelector.include. Resources always keep the prefix. The default is still Always.

Caller-supplied credentials with injectionPolicy: IfNotPresent

Let users bring their own upstream token, for example a personal GitHub PAT forwarded with forwardHeaders, and fall back to a shared service key when they don't. With IfNotPresent, the configured API key is injected only when the target header is missing. This applies to header injection only.

Inherit Gateway-level policies with mergeType

The SecurityPolicy and BackendTrafficPolicy that Agent Router generates for an MCPRoute used to replace any Gateway- or listener-level policy. Set securityPolicy.mergeType or backendTrafficPolicy.mergeType to StrategicMerge or JSONMerge to keep a Gateway-wide rate limit or ext-auth policy in force alongside MCP OAuth.

OAuth for identity providers with non-standard metadata paths

oauth.authorizationServerMetadataUrl points the controller at the exact RFC 8414 metadata document when the issuer URL doesn't lead to it. The controller fetches this document while reconciling, so it needs network access to the identity provider. If the fetch fails, reconciliation fails.

Groundwork for the next MCP spec

The initialize response now negotiates the protocol version with the client and backends instead of always answering 2025-06-18. Merged list responses also carry ttlMs and cacheScope caching hints. Support for the stateless 2026-07-28 MCP specification is in progress and not yet enabled.

Providers & API Compatibility

AWS OpenAI-compatible endpoint (AWSOpenAI)

Send OpenAI-format /v1/chat/completions and /v1/responses requests to Bedrock Runtime's OpenAI-compatible /openai/v1 endpoint without a translation hop. Requests are signed with SigV4 through an AWS BackendSecurityPolicy. The prefix defaults to openai/v1. Other endpoints return 422.

TypeSafe System One (TypeSafe)

Route TypeSafe's Jev decision model through the gateway at /typesafe/v1/systemone with API-key auth. Request bodies pass through unchanged apart from a model name override, and token usage feeds the standard metrics and llmRequestCosts. Streaming is not supported.

Per-backend anthropic-beta filtering

One anthropic-beta value that a provider rejects no longer fails the whole request. AIServiceBackend.spec.headerValueFilters drops listed values (Denylist, the default) or keeps only listed values (Allowlist). It applies to /v1/messages traffic to GCPAnthropic and AWSAnthropic backends. Bedrock also accepts four more beta flags, including thinking-token-count-2026-05-13.

Claude and Gemini request coverage

Claude Opus 5 and 5.5 accept reasoning_effort on Vertex AI and Bedrock, and structured output (response_format) on Vertex AI. Chat-completion tools forward eager_input_streaming to Claude. Gemini on Vertex AI accepts vLLM-style structured_outputs (json, regex, choice).

OpenAI cache_write_tokens

Prompt-cache writes reported as cache_write_tokens by OpenAI Chat Completions and the Responses API now count toward cache-creation token metrics and costs. Translated Claude and Bedrock responses report the same value.

Model Catalog & Observability

Hide aliases from /v1/models

Set excludeFromModelsEndpoint: true on an AIGatewayRoute rule to keep internal or backwards-compatible model aliases out of /v1/models and /anthropic/v1/models. Requests to those models still route normally.

Per-backend GenAI metrics

All four gen_ai.* metrics now carry a gen_ai.backend attribute (gen_ai_backend in Prometheus) set to the backend's namespace/name. This lets you compare latency, time to first token, and token usage across providers serving the same model.

Inference & Operations

Gateway API Inference Extension v1.6

An InferencePool without endpointPickerRef is now reported as Accepted=False (EndpointPickerRefMissing) instead of breaking routing. appProtocol: kubernetes.io/h2c is honored for cleartext HTTP/2 model servers.

Safer controller rollouts

The controller reports Ready only after its caches sync, so Envoy Gateway no longer calls the extension server with an empty view of your resources during a rollout. The chart adds a startup probe and a controller.priorityClassName value.

🔗 API Updates

• AIGatewayRouteRule.excludeFromModelsEndpoint: Optional boolean (default false) in v1alpha1 and v1beta1.
• AIServiceBackend.spec.headerValueFilters: Optional list (max 16, one per header) of {name, mode, values}. mode is Denylist (default) or Allowlist; values holds up to 64 entries. A CEL rule rejects the field unless the schema is GCPAnthropic or AWSAnthropic, and only anthropic-beta is honored today. v1beta1 only.
• VersionedAPISchema.name: AWSOpenAI and TypeSafe: Two new schema values. TypeSafe defaults its version to v1.
• GatewayConfig.spec.extProc.metadataForwardingNamespaces: Optional list (max 32) of dynamic metadata namespaces that Envoy forwards to the external processor. It is required for credentialOverride.fromDynamicMetadata. Under Envoy Gateway mergeGateways, every Gateway in the class shares the union of these namespaces.
• MCPRoute.spec.prefixMode and backendRefs[].prefixMode: Always or Never; unset means Always. The per-backend value takes precedence. v1beta1 only.
• MCPRoute.spec.backendRefs[].promptSelector: Filters prompts with include, includeRegex, exclude, or excludeRegex (max 32 each). Exclude rules win over include rules. v1beta1 only.
• MCPRoute.spec.securityPolicy.mergeType and spec.backendTrafficPolicy.mergeType: Envoy Gateway MergeType; Replace is rejected. Unset keeps the previous override behavior.
• MCPRoute.spec.backendRefs[].securityPolicy.apiKey.injectionPolicy: Always (default) or IfNotPresent. IfNotPresent cannot be combined with queryParam. v1beta1 only.
• MCPRoute.spec.securityPolicy.oauth.authorizationServerMetadataUrl: Optional URI, up to 1024 characters.
• New MCPRoute validation for JWT-based CEL: securityPolicy.oauth is now required when an authorization rule or backendSelector rule's cel expression references auth.jwt. See Breaking Changes.

Deprecations

• cache_creation_input_tokens in translated OpenAI-format responses: Responses translated from Claude and Bedrock now report cache_write_tokens alongside the legacy cache_creation_input_tokens field. Read cache_write_tokens instead; the legacy field will be removed in v1.3.0.

⚠️ Breaking Changes

• Random MCP session encryption seed: The Helm chart no longer defaults to the well-known default-insecure-seed. It generates a random seed, stores it in the Secret -mcp-session-encryption, and reuses it on later upgrades. Upgrading therefore ends active MCP sessions, and clients must re-initialize. To keep sessions during the upgrade, set controller.mcp.sessionEncryption.fallback.seed=default-insecure-seed and remove it once clients have reconnected. If you render the chart with helm template (common with GitOps), set controller.mcp.sessionEncryption.existingSecret or seed. Otherwise every render generates a new seed.
• MCP OAuth tokens must match oauth.issuer: The generated JWT provider now checks the token's iss claim. Tokens with a missing or different issuer, including a trailing-slash mismatch, get 401. Make sure securityPolicy.oauth.issuer matches your identity provider's iss exactly.
• JWT data in MCP CEL requires securityPolicy.oauth: The gateway reads request.auth.jwt claims and scopes, and the session subject, only from a JWT that Envoy has verified. Without securityPolicy.oauth, claims are empty, and new CRD validation rejects authorization or backendSelector CEL that references auth.jwt. Existing routes that break this rule keep their stored spec, but every update is rejected. At runtime their claim lookups fail, and therefore deny, and their scope checks evaluate to false. Add oauth or remove the JWT references. MCP sessions are now bound to the verified subject; reusing a session ID as a different user returns 401.
• MCP CEL evaluation errors now deny: An authorization or backendSelector CEL expression that fails at runtime, for example by reading a header that isn't present, used to be skipped. It now denies: tools/call returns 403, the tool is hidden from tools/list, and the backend is left out of the session. Write expressions defensively, for example "x-team" in request.headers && request.headers["x-team"] == "blocked".
• MCP OAuth discovery honors spec.headers: On an MCPRoute with both headers matches and oauth, the OAuth well-known endpoints now apply the same header matches. Clients that run OAuth discovery without those headers get 404.
• Cross-namespace Secrets in BackendSecurityPolicy need a ReferenceGrant: azureCredentials.clientSecretRef, gcpCredentials.credentialsFile.secretRef, and the OIDC clientSecret under AWS, Azure, and GCP oidcExchangeToken were read from other namespaces without any check. The secretRef of apiKey, azureAPIKey, anthropicAPIKey, and awsCredentials.credentialsFile ignored its namespace, so a cross-namespace reference failed or silently used a Secret with the same name from the policy's own namespace. All of these now honor the namespace, and a missing grant sets the policy to NotAccepted. Already-issued tokens keep working until they expire, so the failure may appear late. Create a ReferenceGrant in the Secret's namespace (see Upgrade Guidance).
• Ungranted cross-namespace backends no longer reach the data plane: An AIGatewayRoute backendRef to an AIServiceBackend or InferencePool in another namespace without a ReferenceGrant already marked the route NotAccepted, but its credentials were still written to the external processor's config. Such backends are now left out. Working setups already have the grant; this affects only routes that were already NotAccepted.
• ReferenceGrant to.name is now enforced: A ReferenceGrant to entry that sets name used to authorize every object of that group and kind in its namespace. It now authorizes only the named object. References to other objects, such as another Secret, AIServiceBackend, or InferencePool, are NotAccepted and left out of the data plane. Check grants that set name before upgrading.
• Per-tenant token limits are now enforced: With the QuotaPolicy Distinct fix, tenants can start receiving 429 at the limits you configured. Review per-tenant limits before upgrading.
• External processor -configPath removed: The external processor reads only its config bundle (-configBundlePath), and the controller no longer writes the legacy filter-config.yaml Secret. Existing legacy Secrets are left in place and can be deleted. This matters only if you run the external processor yourself, or still have Envoy pods created before config bundles existed (v1.0.x era). Restart those pods before upgrading.

🐛 Bug Fixes

• Credential rotation no longer serves stale tokens: Fixes a v1.1.0 regression: the controller read backend credential Secrets from its informer cache, so a rotated credential could miss the filter config and upstream calls failed until the next reconcile. Secrets are read from the API server again. Credential changes in a BackendSecurityPolicy that targets an InferencePool now also reach the Gateway's filter config.
• Revoking a ReferenceGrant takes effect immediately: Deleting a ReferenceGrant, or removing a from entry, now reconciles the affected AIGatewayRoutes (including InferencePool backends) and BackendSecurityPolicies (including OIDC client secrets). Access they relied on used to stay in place until an unrelated reconcile.
• Gateways with TCP or TLS listeners translate again: A listener without an HTTP connection manager, such as a TCPRoute or TLS-passthrough listener, made the extension server fail translation for the whole Gateway. Those listeners are now skipped.
• Controller crash on combined priority backends: Adding a backendRef before its AIServiceBackend existed could panic and crash-loop the controller. The controller now logs the mismatch and continues.
• Self-signed webhook survives helm upgrade: With the default self-signed webhook certificate, an upgrade or Argo CD sync wiped the webhook's caBundle, so Envoy pods failed admission until the controller restarted. The chart now embeds the CA bundle.
• credentialOverride.fromDynamicMetadata now receives metadata: In v1.1.0 Envoy never forwarded the metadata namespace to the external processor. Requests silently used the configured credential, or failed with 401 when fallbackToConfigured: false. List the namespace in GatewayConfig.spec.extProc.metadataForwardingNamespaces (see Upgrade Guidance).
• Token usage is always captured on streams: Duplicate stream_options in a streaming request could stop include_usage from reaching the provider, which bypassed token accounting. Usage metadata is now also recorded when a client disconnects right after the final SSE frame. Responses API streams framed with CRLF now report usage, model, and tracing data.
• Clear 422 errors instead of 500s: Calling an endpoint that a backend's schema doesn't support (for example /v1/responses to Vertex AI Gemini) returns 422 Unprocessable Entity. Oversized JSON schemas sent to Vertex AI return 422 once they exceed 50,000 nodes.
• Lower external processor memory: Model-name strings used as metric labels no longer keep entire request and response bodies in memory, which was most noticeable with large-context requests.
• Per-tenant token limits charge the right tenant: With QuotaPolicy Distinct header selectors, the end-of-stream token cost went into one shared bucket instead of the caller's own. See Breaking Changes for the effect on tenants.
• Claude translation fixes: Structured outputs keep the property order you declared. Streamed extended-thinking text is no longer empty. The web_search_20260209 tool on /v1/chat/completions is now forwarded to Claude instead of rejected with 422; use streaming, because non-streaming responses currently return only the first text block. Tracing of long Claude streams no longer allocates quadratically or panics on malformed delta indices.
• MCP interoperability: Backends that send application/json; charset=utf-8 (for example the MCP Java SDK) no longer have their list results dropped. OAuth metadata discovery now treats an empty or non-JSON document as a miss and tries the next well-known URL.
• aigw run base URL handling: A path in OPENAI_BASE_URL or ANTHROPIC_BASE_URL (for example OpenRouter's /api/v1) is now used as the request prefix instead of being dropped. Base URLs that aren't http or https are rejected.
• Unresolvable InferencePool routes are rejected: When Envoy Gateway couldn't resolve an InferencePool, the extension server used to accept a route that only returned 500. It now rejects that route so the error is visible.

📖 Upgrade Guidance

Upgrading from v1.1 needs no CRD storage migration, and every new field is optional. Several security fixes change behavior, though, so work through the steps that apply to you before you upgrade. Upgrade from v1.1.x, not directly from v1.0.x.

1. Upgrade Gateway API and Envoy Gateway first

   Envoy Gateway v1.9 requires the Gateway API v1.6 CRDs. If you use the standard channel, move any TCPRoute and UDPRoute manifests to gateway.networking.k8s.io/v1 before you upgrade the CRDs. Then upgrade the CRDs and Envoy Gateway. The Agent Router values file is unchanged:

   helm template eg-crds oci://docker.io/envoyproxy/gateway-crds-helm \
   --version v1.9.2 \
   --set crds.gatewayAPI.enabled=true \
   --set crds.envoyGateway.enabled=true \
   | kubectl apply --force-conflicts --server-side -f -
   

   helm upgrade -i eg oci://docker.io/envoyproxy/gateway-helm \
   --version v1.9.2 \
   --namespace envoy-gateway-system \
   -f https://raw.githubusercontent.com/theagentrouter/agent-router/v1.2.0/manifests/envoy-gateway-values.yaml
   

   Envoy Gateway v1.9 has its own breaking changes. Read its v1.9.0 and v1.9.2 release notes. The ones most likely to affect AI traffic:

   • Lua EnvoyExtensionPolicy is disabled by default. Enable it with extensionApis.enableLua.
   • mergeType on a SecurityPolicy or BackendTrafficPolicy is only accepted on route targets.
   • Tracing client sampling now defaults to 0%.
   • EndpointSlice indexing can increase controller memory.

   Do not enable Envoy Gateway's new EnvoyProxy mergeBackends option with Agent Router yet. It shares clusters between routes, which breaks how Agent Router resolves the backend for a request.

2. Plan for the MCP session seed change

   The chart now generates a random MCP session encryption seed, so upgrading ends active MCP sessions. Pick one:

   • Accept a reconnect. Do nothing. Clients re-initialize their sessions.

   • Keep sessions during the upgrade. Add the old seed as a fallback in your values for this upgrade, then remove it once clients have reconnected:

     controller:
       mcp:
         sessionEncryption:
           fallback:
             seed: default-insecure-seed
     

   • GitOps or helm template. The chart can't read back a seed it generated earlier, so it generates a new one on every render. Supply your own Secret instead:

     kubectl create secret generic mcp-session-seed -n envoy-ai-gateway-system \
     --from-literal=seed="$(openssl rand -hex 32)"
     

     Then set controller.mcp.sessionEncryption.existingSecret: mcp-session-seed.

3. Check your MCPRoutes

   • securityPolicy.oauth.issuer must match your identity provider's iss claim exactly, including any trailing slash.
   • Any authorization or backendSelector CEL that references request.auth.jwt needs securityPolicy.oauth on the same route.
   • Make CEL expressions null-safe, because evaluation errors now deny. For example, use "x-team" in request.headers && request.headers["x-team"] == "blocked" instead of request.headers["x-team"] == "blocked".
   • On routes with both headers and oauth, make sure clients send those headers during OAuth discovery.

4. Add ReferenceGrants for cross-namespace credential Secrets

   If a BackendSecurityPolicy references a credential Secret in another namespace, create a grant in the Secret's namespace. This now applies to every credential type: API keys, AWS credentials files, Azure client secrets, GCP credentials files, and OIDC client secrets.

   apiVersion: gateway.networking.k8s.io/v1beta1
   kind: ReferenceGrant
   metadata:
     name: allow-backend-security-policy-secrets
     namespace: credentials # the Secret's namespace
   spec:
     from:
     - group: aigateway.envoyproxy.io
       kind: BackendSecurityPolicy
       namespace: ai-backends # the BackendSecurityPolicy's namespace
     to:
     - group: ""
       kind: Secret
       name: openai-api-key # optional: limit the grant to one Secret
   

   A to entry with name now authorizes only that object. Without name, it covers every object of that kind in the namespace. If an existing grant sets name, make sure it lists every Secret, AIServiceBackend, or InferencePool that other namespaces reference.

5. Turn on metadata forwarding for credentialOverride.fromDynamicMetadata

   If you use per-request credentials from dynamic metadata, list the metadata namespace in a GatewayConfig, and reference that GatewayConfig from the Gateway with the aigateway.envoyproxy.io/gateway-config annotation:

   apiVersion: aigateway.envoyproxy.io/v1beta1
   kind: GatewayConfig
   metadata:
     name: my-gateway-config
     namespace: default # same namespace as the Gateway
   spec:
     extProc:
       metadataForwardingNamespaces:
       - envoy.filters.http.ext_authz
   

6. Upgrade Agent Router

   helm upgrade -i aieg-crd oci://docker.io/envoyproxy/ai-gateway-crds-helm \
   --version v1.2.0 \
   --namespace envoy-ai-gateway-system
   

   helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm \
   --version v1.2.0 \
   --namespace envoy-ai-gateway-system
   

   The chart and image names keep the ai-gateway prefix. Only the project name changed.

7. After the upgrade

   • Dashboards: GenAI metrics now carry a gen_ai_backend label. Panels that don't aggregate with sum by (...) will split into one series per backend.
   • Legacy Secrets: the controller no longer writes the old filter-config.yaml Secrets. You can delete them.
   • Inference Extension: if you use InferencePool, upgrade the Gateway API Inference Extension CRDs to v1.6.x and redeploy your endpoint picker.

📦 Dependencies Versions

Go 1.27.1

Updated from Go 1.26.4.

Envoy Gateway v1.9.2

Built on Envoy Gateway v1.9.2, up from v1.8.1.

Envoy v1.39

Envoy Proxy v1.39.1, as shipped by Envoy Gateway v1.9.2.

Gateway API v1.6.1

Updated from v1.5.1. Envoy Gateway v1.9 requires the v1.6 CRDs.

Gateway API Inference Extension v1.6.2

Updated from v1.0.2.

MCP Go SDK v1.7.0

Unchanged from v1.1.

⏩ Patch Releases

🙏 Acknowledgements

v1.2 came from 39 contributors, many of them contributing for the first time. Special thanks to:

• Ignasi Barrera, who led the security hardening across MCP authentication, session binding, and cross-namespace references.
• Hritik Raj, who built the groundwork for the 2026-07-28 MCP specification.
• Deepika Agrawal, Aishwarya Raimule, and Huabing (Robin) Zhao, who added MCP prefix mode, credential injection, policy merging, and OAuth metadata URLs.
• Yang Haoran, hustxiayang, and Xiaolin Lin, who added provider coverage and fixed streaming bugs.
• zirain, who handled the Gateway API Inference Extension v1.6 upgrade and conformance.
• Everyone who reported bugs, reviewed PRs, and joined the community meetings during the move to Agent Router.

🔮 What's Next

Work already in flight for upcoming releases:

• The stateless 2026-07-28 MCP specification, switched on end to end.
• OAuth token exchange for MCP backends, so the gateway can swap a client token for a backend-scoped one.
• A dedicated MCPBackend CRD, decoupling backend configuration from MCPRoute.
• Longer quota windows (weekly, monthly, and yearly) and quota-aware routing.
• Agent-to-agent (A2A) traffic, building on the A2A preview guide.

The roadmap is community-driven. Join us and help shape it.

Original source

Token counting, per-request credentials, stream idle timeout, MCP hostname routing, OpenTelemetry GenAI tracing, and HTTP CONNECT egress.

Envoy AI Gateway v1.1.0 is the first minor release on the stable 1.x API. It adds token counting across providers, per-request upstream credentials, stream idle timeout with failover, MCP hostname routing and CEL backend selection, optional OpenTelemetry GenAI tracing, and HTTP CONNECT egress. Upgrading from v1.0 needs no CRD migrations; the only operator-facing change is Helm's restricted controller security context.

✨ New Features

Token Counting APIs

vLLM-compatible /tokenize across providers

Count tokens for a prompt without generating a completion. Chat messages and completion prompts use the vLLM tokenize request shape against vLLM, Vertex AI Gemini, GCP Anthropic, AWS Bedrock Converse, and AWS Anthropic. Native Anthropic clients can use /anthropic/v1/messages/count_tokens; Responses API clients can use /v1/responses/input_tokens (OpenAI and Azure OpenAI).

Anthropic-format /anthropic/v1/models

Claude clients that call Anthropic's model list endpoint now get the gateway's configured catalog in Anthropic's response shape, scoped by hostname the same way /v1/models is.

Per-Request Credentials & Egress

Per-request upstream credentials via credentialOverride

A trusted filter can supply the backend credential per request instead of sharing one static key in BackendSecurityPolicy. Source from Envoy dynamic metadata (preferred) or a request header the gateway strips before the backend. API keys, Anthropic, Azure, GCP, and AWS SigV4 are supported.

HTTP CONNECT forward proxy

GatewayConfig.spec.forwardProxy tunnels upstream LLM connections through an HTTP CONNECT proxy (host:port). Bedrock requests through PrivateLink now sign over the actual upstream host, so VPC endpoints no longer fail with a silent 403.

Streaming Resilience

Stream idle timeout with failover

AIGatewayRouteRule.streamIdleTimeout bounds how long Envoy waits without upstream bytes on a streaming response. If it fires before the first token, a retry policy can fail over to the next backend; mid-stream it returns 504. Pair with a BackendTrafficPolicy whose retry covers reset.

MCP Gateway

Hostname routing and backend selection

MCPRoute.spec.hostnames scopes an MCP endpoint to specific hosts. backendSelector restricts which backends a session fans out to via CEL rules evaluated at initialize (default Deny). Initialize now advertises merged backend capabilities, and the controller shares one Envoy Gateway Backend per namespace.

Observability

OpenTelemetry GenAI conventions and Grafana dashboard

Set AI_GATEWAY_TRACING_SEMCONV=gen_ai on the ext-proc to emit gen_ai.* span attributes. An example dashboard for gen_ai_* Prometheus metrics ships in examples/monitoring/grafana-dashboard.json. Controller and ext-proc logs accept --logFormat=json; MCP resource reads populate mcp_resource_uri in access-log metadata.

Provider Translation & API Compatibility

Structured output for Claude on Vertex AI

JSON-schema constrained decoding now works for Claude models on Vertex AI that advertise output_config. OpenAI reasoning_effort is forwarded to Bedrock as reasoning_config.

Responses API tool_search, unknown tools, and Codex inputs

The Responses API accepts the built-in tool_search tool and additional_tools, preserves unknown tool types, and accepts Codex-style agent input items.

Helm & Operations

Controller hardening and sharded config secrets

The chart can emit a PDB, honour topologySpreadConstraints, and attach extra podLabels. Filter configuration is split across multiple Secrets so large Gateway configs no longer hit Kubernetes' 1 MiB Secret size limit.

🔗 API Updates

• AIGatewayRouteRule.name: Optional unique name copied onto the generated HTTPRoute rule. route-not-found is reserved.
• AIGatewayRouteRule.streamIdleTimeout: Optional Gateway API duration; unset means no per-try idle timeout.
• BackendSecurityPolicy.spec.credentialOverride: Optional per-request credential source. Exactly one of fromRequestHeaders or fromDynamicMetadata. fallbackToConfigured defaults to true.
• GatewayConfig.spec.forwardProxy: Optional HTTP CONNECT proxy. address is required (host:port).
• MCPRoute.spec.hostnames: Optional list of hostnames (max 16).
• MCPRoute.spec.backendSelector: Optional CEL selector evaluated at session initialize. defaultAction defaults to Deny.

⚠️ Breaking Changes

• Helm controller security context defaults: The Helm chart now sets a restricted podSecurityContext and container securityContext on the controller (non-root UID/GID 65532, all capabilities dropped, no privilege escalation, RuntimeDefault seccomp). Previously both were empty. If you run a custom controller image that cannot use UID 65532, override controller.podSecurityContext and controller.securityContext in your values.

🐛 Bug Fixes

• Stale Content-Encoding on translated error responses: Translated error bodies no longer keep the upstream Content-Encoding header. Clients no longer fail to decode an uncompressed error payload that still claimed to be gzip or brotli.
• Anthropic ping events keep streams alive: Anthropic ping SSE events now emit an empty chunk on translated paths, so intermediaries and clients that time out on silence no longer drop a healthy Claude stream.
• Gateway local replies are not reprocessed as upstream responses: Ext-proc no longer treats Envoy local replies (rate-limit, timeout, auth) as provider payloads. Local error bodies pass through instead of being force-translated into a malformed LLM response.
• Anthropic cache_control on the Bedrock path: Prompt-caching cache_control markers now translate onto AWS Bedrock, including tool-result cache control and Bedrock tool cache points as their own tools[] element. Cached prefixes that previously silently dropped now actually hit Bedrock's cache.
• Bedrock reasoning_content serialized as a plain string: Non-streaming and streaming Bedrock reasoning content is emitted as a plain string, with signatures carried via thinking_blocks. Clients that expected a string and previously received an object no longer fail to parse Claude-on-Bedrock reasoning.
• Unsupported Anthropic tool types error instead of dropping: Unknown Anthropic tool types now fail the request rather than being silently stripped. You get a clear error instead of a tool-less prompt that looks like it succeeded.
• Cross-namespace InferencePool backendRef honours namespace: An InferencePool referenced from another namespace is now resolved in that namespace instead of the route's. Cross-namespace inference pools that previously 404'd at reconcile now work when the ReferenceGrant allows them.
• Anthropic streaming cache tokens no longer double-counted: Input and cache token counts on Anthropic message_delta events are merged once, including on the OpenAI→Anthropic streaming path. Usage metrics and access logs no longer inflate cached-token cost on streaming Claude calls.
• Credential leak in Envoy ext-proc debug logs: Envoy HTTP and ext-proc debug logging is capped, and credential header mutations are always redacted in debug output. Passthrough endpoints also redact user content. Debug-level logging is no longer a credential-exfiltration path.
• Gemini streaming tool-call finish reasons and reasoning tokens: Newer Gemini models such as gemini-3.5-flash now map streaming tool-call finish reasons correctly, and reasoning tokens are counted in streaming metrics. Streaming Gemini tool use no longer reports a wrong finish reason or drops thinking-token usage.

📖 Upgrade Guidance

Upgrading from v1.0 requires no CRD migrations. New fields are optional and backward compatible. Review the Helm security-context change if you override the controller image.

If the official controller image works for you, no action is needed. To restore empty security contexts:

controller:
  podSecurityContext: {}
  securityContext: {}

JSON logs: set controller.logFormat and extProc.logFormat to json. Enable a PDB with controller.podDisruptionBudget.enabled: true when you run more than one replica.

📦 Dependencies Versions

Go 1.26.4

Continues on Go 1.26.4, same as v1.0.

Envoy Gateway v1.8.1

Built on Envoy Gateway v1.8.1 for proven data plane capabilities.

Envoy v1.38

Leveraging Envoy Proxy v1.38.1 for battle-tested networking.

Gateway API v1.5.1

Support for Gateway API v1.5.1 specifications.

Gateway API Inference Extension v1.0.2

Continued integration with Gateway API Inference Extension v1.0.2 for intelligent endpoint selection.

MCP Go SDK v1.7.0

Updated from v1.6.1 to v1.7.0 for the latest Model Context Protocol features.

⏩ Patch Releases

🙏 Acknowledgements

Thank you to everyone who made v1.1 possible

🔮 What's Next

The roadmap is community-driven — join us and help shape it.

Original source

The first stable, generally available release. The v1beta1 control-plane API is now covered by our stability guarantee — with 16 LLM providers, an MCP gateway, multimodal and audio endpoints, enterprise observability, and multi-tenant routing, all on the proven Envoy Gateway foundation.

General Availability

• Stable v1beta1 API
• SemVer Guarantee
• 16 Providers
• MCP Gateway
• Multimodal & Audio
• Enterprise Observability
• Multi-Tenant Routing

Envoy AI Gateway v1.0.0 marks General Availability. With this release the core control-plane API — AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute, all served at v1beta1 — is declared stable: within the 1.x series we will not make breaking changes to it unless required by a critical security fix, and any such change will ship with a documented migration path. Upgrading from v0.7 requires no changes to your resources. 1.0 brings together everything built since the first release in February 2025: a single OpenAI-compatible API across 16 providers with cross-provider translation, a full Model Context Protocol gateway, multimodal and audio endpoints, enterprise-grade observability, and multi-tenant, quota-aware routing — all as an additive layer on CNCF Envoy Gateway. This is the foundation we set out to build when we said “Onward to 1.0”: a stable base you can run production AI infrastructure on.

🎉 What 1.0 Means

1.0 is not just another feature release — it is a commitment. From the very first release we said the major version would arrive once we had a first stable control-plane API. With v1.0, that moment is here: the core CRDs you author every day are now a stable contract you can build on with confidence.

Practically, General Availability means three things:

• A stable API. Your AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute resources — all served at v1beta1 — will not break under you within the 1.x series.
• Predictable upgrades. Upgrading the controller will not break a valid, migrated configuration, and any change that ever requires action will ship with a documented path.
• A complete platform. Everything assembled since v0.1 — 16 providers, a full MCP gateway, multimodal and audio endpoints, enterprise observability, and multi-tenant routing — is now production-ready on the proven Envoy Gateway foundation.

Our API stability commitment

For stable releases, we will never break the APIs unless there is a critical security issue, and we will always provide a migration path in the release notes if we ever must. Following Semantic Versioning, the v1beta1 control-plane API will remain backward compatible for the entire 1.x series — breaking changes would only ever land in a future 2.0. See the full support policy.

🧭 The Road to 1.0

When v0.1 shipped in February 2025, Envoy AI Gateway put a unified API in front of two providers. 1.0 is the culmination of everything built since:

• 16 LLM providers
  From 2 at launch to 16 today — all behind one OpenAI-compatible API.
• Full endpoint coverage
  Chat, completions, embeddings, images, audio, and the Responses API.
• MCP gateway
  Multiplex, route, and authorize Model Context Protocol servers.
• Enterprise observability
  OpenTelemetry + OpenInference tracing and GenAI token metrics.
• Multi-tenant routing
  Hostname scoping, model virtualization, and quota-aware limits.
• A stable v1beta1 API
  A SemVer-backed contract you can build production on.

✨ The 1.0 Feature Surface

These are the capabilities the stable 1.0 control plane brings together.

A Stable, Versioned Control-Plane API

Core CRDs now covered by the stability guarantee

The control-plane API you build on — AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute, all served at v1beta1 — is now a stable contract. Within the 1.x series these APIs will not change in a breaking way unless required by a critical security fix, and any such change will ship with a documented migration path. You can standardize on Envoy AI Gateway without tracking a moving target.

Universal LLM Access

One OpenAI-compatible API across 16 providers

Reach OpenAI, Azure OpenAI, Google Gemini, Google Vertex AI, AWS Bedrock, Anthropic, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, and the Tetrate Agent Router Service through a single endpoint. Point your application at the gateway once and switch or mix providers without changing client code.

Cross-provider request/response translation

Translate between provider protocols transparently — Anthropic /v1/messages to OpenAI /v1/chat/completions, and Anthropic Messages to AWS Bedrock Converse and InvokeModel — including streaming, tool use, reasoning/thinking blocks, and images, so clients reach any backend without rewriting their integration.

Model virtualization with modelNameOverride

Expose stable, application-facing model names while the gateway maps them to provider-specific models. Enables A/B testing, gradual migrations, and multi-provider strategies without touching client code, and guards against provider lock-in.

Full Endpoint Coverage

Chat, completions, embeddings, and images

/v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/images/generations are supported across compatible providers, all behind the same auth, rate limiting, and observability.

Audio: transcription, translation, and speech

/v1/audio/transcriptions, /v1/audio/translations, and /v1/audio/speech bring speech-to-text and text-to-speech workloads through the gateway alongside the rest of your AI traffic.

OpenAI Responses API and multimodal inputs

The /v1/responses endpoint is supported, including on Azure OpenAI backends, and chat requests accept image, audio_url, and video_url content parts for compatible backends.

MCP Gateway

Aggregate and route Model Context Protocol servers

Multiplex multiple MCP servers behind one endpoint with MCPRoute, including tool routing and include/exclude filtering, so clients see one unified tool surface.

Fine-grained, CEL-based authorization

Enforce per-tool authorization. tools/list applies the same rules as tools/call, so callers only discover the tools they are allowed to invoke — no leaked tool names, no wasted LLM turns.

Per-backend header forwarding with JWT claim projection

Forward selected request headers and project JWT claims to individual MCP backends, enabling identity-aware access to downstream tool servers.

Traffic Management & Multi-Tenancy

Hostname-based multi-tenant routing

Serve different model sets per hostname from a single Gateway with AIGatewayRoute.spec.hostnames; the /v1/models endpoint scopes its response to the models for the matching host.

Token- and quota-aware rate limiting

Rate limit on model tokens and per QuotaPolicy, with backend rate limit filter injection to enforce quota-based throttling against upstream provider limits.

Provider fallback and InferencePool support

Automatic failover across providers, plus intelligent endpoint selection for self-hosted models via the Gateway API Inference Extension.

Provider Authentication & Compliance

BackendSecurityPolicy for upstream authentication

Centralize provider credentials with API key, AWS, Azure, and GCP cloud-native identity — including GKE Workload Identity via Application Default Credentials — keeping secrets out of application code.

Request/response body redaction

Redact sensitive request and response bodies to meet compliance requirements without losing the rest of your observability.

Enterprise Observability

OpenTelemetry tracing with OpenInference

Full request-lifecycle tracing, compatible with AI evaluation tools like Arize Phoenix, for deep visibility into every LLM and MCP request.

GenAI token metrics and reasoning-token accounting

Prometheus metrics for token usage, time-to-first-token, and inter-token latency, with separate accounting for reasoning tokens so cost attribution stays accurate for thinking models.

🔗 API & Stability

• The v1beta1 API is now stable: v1.0 does not change the API surface. Instead it elevates the existing v1beta1 CRDs to a stable contract under our support policy: no new apiVersion is introduced and no resource migration is required. New fields added during the 1.x series will remain backward compatible.

⚠️ Breaking Changes

None. v1.0 introduces no breaking changes. The v1beta1 API is unchanged — 1.0 declares it stable rather than altering it — so there is no apiVersion bump and no resource migration. If you are running v0.7, your existing resources work as-is.

🛡️ Support & Compatibility Policy

With 1.0, the project's support policy applies in full:

• API compatibility. The v1beta1 CRDs are stable for the 1.x series. New fields are added in a backward-compatible way; breaking changes are reserved for a future major version and would ship with a migration path.
• Controller upgrades. Upgrading the controller will not break a valid configuration. As with every release, upgrade at most two minor versions at a time, following any documented migration steps.
• Envoy Gateway compatibility. Each release is built on the latest stable Envoy Gateway (and therefore Envoy Proxy); keep Envoy Gateway up to date before upgrading Envoy AI Gateway.
• End of life. A release is supported until two releases after it, consistent with prior versions.

📖 Upgrade Guidance

Upgrading from v0.7 is a drop-in change — there are no API or resource changes:

1. Update the Helm chart / controller image to the v1.0.0 release.
2. Roll out as usual. Your existing v1beta1 resources require no edits.

If you are on an older release, upgrade one or two minor versions at a time and follow the migration steps in each series' release notes (notably the v0.6 promotion of the core CRDs to v1beta1) before moving to 1.0.

📦 Dependencies Versions

Go 1.26.4

Built with Go 1.26.4.

Envoy Gateway v1.8.1

Built on Envoy Gateway v1.8.1 for proven data plane capabilities.

Envoy v1.38

Leveraging Envoy Proxy v1.38.1 for battle-tested networking.

Gateway API v1.5.1

Support for Gateway API v1.5.1 specifications.

Gateway API Inference Extension v1.0.2

Continued integration with Gateway API Inference Extension v1.0.2 for intelligent endpoint selection.

MCP Go SDK v1.6.1

Built on MCP Go SDK v1.6.1 for the latest Model Context Protocol features.

⏩ Patch Releases

🙏 Acknowledgements

1.0 belongs to everyone who got us here. Our deepest thanks to:

• The maintainers across Tetrate, Bloomberg, Tencent, and Nutanix, and the many independent contributors who shaped the project through code, reviews, and weekly community meetings.
• The early adopters — including LY Corporation, Alan by Comma Soft, and NRP — who used, tested, and fed back what mattered.
• The broader Gateway API, Envoy, and CNCF communities whose standards this project is built on.

🔮 What's Next (Beyond 1.0)

A stable API is a starting line, not a finish line. On the roadmap:

• A dedicated MCPBackend CRD, decoupling MCP backend configuration from MCPRoute.
• Deeper MCP authorization across tools, resources, and prompts.
• Fuller quota-aware routing that automatically steers around rate-limited upstreams.
• More provider translation paths and expanded multimodal support.

The roadmap is community-driven — join us and help shape it.

Original source

Hostname-based routing, Anthropic-to-Bedrock Converse translation, audio transcription/translation endpoints, Azure OpenAI Responses API, quota-aware rate limiting, Claude Opus 4.7 reasoning, and MCP authorization-aware tool listing.

Envoy AI Gateway v0.7.0 adds hostname-based routing to AIGatewayRoute, enabling multi-tenant deployments where different hostnames expose different model sets through a single Gateway. A new Anthropic Messages → AWS Bedrock Converse translator lets Anthropic-native clients reach Bedrock without switching protocols. OpenAI audio transcription and translation endpoints arrive alongside Azure OpenAI Responses API support. Quota-aware rate limiting takes its first step with backend rate limit filter injection for QuotaPolicy. Claude Opus 4.7 gains full reasoning support including the display parameter and xhigh effort tier. Anthropic-to-OpenAI translation now handles reasoning blocks and images end-to-end. MCP tools/list responses respect authorization rules, and multimodal support grows with audio_url and video_url content types. Several SSE streaming and provider translation bugs are fixed.

✨ New Features

Multi-Tenant Hostname Routing

Hostname-based model scoping on AIGatewayRoute

Serve different model sets from a single Gateway by assigning hostnames to each AIGatewayRoute. The /v1/models endpoint automatically returns only the models declared by routes matching the request's Host header, so tenants on teamA.ai.example.com and teamB.ai.example.com each see their own catalog without separate Gateways. Wildcard hostnames (*.ai.example.com) are supported following the Gateway API hostname matching rules.

Provider Translation

Anthropic /v1/messages → AWS Bedrock Converse API

Send requests in Anthropic Messages format and have them translated to Bedrock's Converse and ConverseStream APIs automatically. Supports text, images, tool use, thinking blocks, and streaming — so Anthropic-native clients can reach any Bedrock model without changing their integration. Complements the existing OpenAI → Bedrock Converse and Anthropic → Bedrock InvokeModel paths.

Reasoning and image support for Anthropic-to-OpenAI translation

The Anthropic /v1/messages → OpenAI /v1/chat/completions path now handles thinking/reasoning content and image blocks end-to-end. Thinking config (enabled/disabled/adaptive) passes through, thinking and redacted_thinking blocks are preserved in multi-turn conversations, and image blocks (base64 and URL) convert to OpenAI image_url format. Previously these were silently dropped.

Claude Opus 4.7 and Mythos Preview reasoning

Full support for Claude Opus 4.7's reasoning features: the display parameter (summarized/omitted) controls thinking content visibility, and xhigh joins the reasoning effort tiers for long-horizon agentic and coding tasks. Both claude-opus-4-7 and claude-mythos-preview models are recognized for effort-based thinking control.

Custom request paths for Anthropic backends via prefix

The prefix field on VersionedAPISchema now works for Anthropic-schema backends, producing endpoints like /{prefix}/messages instead of the default /v1/messages. Useful for routing to Anthropic-compatible providers that use a non-standard path.

Anthropic anthropic-beta header forwarded to AWSAnthropic

The anthropic-beta request header is now mapped into the anthropic_beta body field when routing to AWSAnthropic backends, so beta features like extended thinking and token counting work through the gateway without manual body rewriting.

OpenAI API Compatibility

Audio transcription and translation endpoints

Full data-plane support for OpenAI's /v1/audio/transcriptions (Whisper transcription) and /v1/audio/translations (Whisper translation) endpoints. These accept multipart/form-data requests containing audio files, enabling speech-to-text workloads to flow through the gateway with the same auth, rate limiting, and observability as other traffic.

Azure OpenAI Responses API

The OpenAI-compatible /v1/responses endpoint now works with Azure OpenAI backends, routing requests to Azure's /openai/responses?api-version=... path while preserving existing request and response handling. Azure users get Responses API support without changing client code.

audio_url and video_url content types

OpenAI chat completion requests can now include audio_url and video_url content parts, enabling multimodal audio and video inputs for compatible backends like vLLM with phi-4-mm and Qwen 3.5 models.

Quota-Aware Routing

Backend quota rate limit filter injection

First step toward quota-aware routing: the controller now injects a backend rate limit filter when a QuotaPolicy is attached to an AIServiceBackend. The QuotaPolicy controller reconciles the policy, builds rate limit descriptor trees, and configures the rate limit service. This enables per-backend request throttling based on upstream provider quotas.

MCP Gateway

Authorization-filtered tools/list responses

MCP tools/list now applies the same authorization rules used by tools/call, omitting tools the caller isn't authorized to invoke. Prevents unauthorized callers from discovering tool names and avoids wasted LLM turns on tools that would fail at call time.

Observability

Smarter log redaction preserves developer-authored metadata

Debug log redaction (--enableRedaction) no longer masks developer-authored schema metadata that was previously over-redacted: tool definition description and parameters, tool call function.name, response_format.json_schema, and guided_json are now visible in debug logs. User-provided content and AI-generated text remain redacted, making debug logs significantly more useful without compromising privacy.

🔗 API Updates

AIGatewayRoute.spec.hostnames: New optional field accepting a list of hostnames for hostname-based request filtering. When specified, the generated HTTPRoute includes these hostnames, and the /v1/models endpoint scopes its response to models from matching routes. Follows Gateway API hostname semantics including wildcard support.

AIGatewayRoute.spec.rules capped at 15: Maximum rules per AIGatewayRoute reduced from 128 to 15 to match the Gateway API HTTPRoute limit (one slot is reserved for a controller-injected catch-all rule). To configure more rules on the same Gateway, split them across multiple AIGatewayRoute resources.

VersionedAPISchema.prefix supported for Anthropic: The prefix field now applies to Anthropic-schema backends in addition to OpenAI. The version field is ignored for Anthropic; use prefix for custom paths. Note: prefix is ignored for AWSAnthropic and GCPAnthropic as these override paths internally.

QuotaPolicy rate limit filter injection (runtime enforcement): The QuotaPolicy CRD (introduced as API-only in v0.6) now has its first runtime behavior: when attached to an AIServiceBackend, a backend rate limit filter is injected to enforce quota-based throttling. Full quota-aware routing across multiple backends is planned for future releases.

🐛 Bug Fixes

SSE parser handles fields without space after colon: The SSE event parser now correctly handles fields formatted as data:{json} (no space after the colon), in addition to the standard data: {json}. Fixes silent field drops when proxying responses from providers that omit the optional space.

Responses API streaming SSE buffering: OpenAI Responses API and speech streaming translators now buffer incomplete SSE events across response body chunks instead of treating each chunk as self-contained. Fixes dropped or mangled events when TCP segment boundaries split an SSE event mid-frame.

Responses API token usage from incomplete and failed streams: Token usage is now captured from response.incomplete and response.failed SSE events, not just response.completed. Streams that hit max_output_tokens or encounter post-generation failures no longer report zero tokens.

Nil output guard in AWS Bedrock response translator: Bedrock can return HTTP 200 with no output field (e.g. guardrail interventions or UnknownOperationException). Previously this caused a nil-pointer panic in the ext-proc; now it returns a clean error to the caller.

Comprehensive Gemini finish-reason mapping: Gemini finish reasons like SAFETY, BLOCKLIST, RECITATION, MALFORMED_FUNCTION_CALL, and others now map to their correct OpenAI equivalents instead of all falling through to content_filter. Unknown reasons map to error rather than silently misreporting as a content filter event.

Empty delta in GCP Vertex AI streaming chunks: Streaming response chunks from GCP Vertex AI that lack candidate content now emit an empty delta object instead of omitting the field, conforming to the OpenAI streaming contract and fixing parse errors in strict clients.

Typeless assistant output messages in Responses API: Multi-turn Responses API inputs that include assistant messages without an explicit type: "message" field (e.g. from OpenCode) now parse correctly. Previously these were treated as easy-input messages, causing unmarshalling failures on output_text content blocks.

📖 Upgrade Guidance

Using Hostname-Based Routing

To serve different model sets per hostname, add hostnames to your AIGatewayRoute:

apiVersion: aigateway.envoyproxy.io/v1beta1
kind: AIGatewayRoute
metadata:
  name: team-a-route
spec:
  hostnames:
    - "team-a.ai.example.com"
  rules:
    - matches:
        - headers:
            - name: x-ai-eg-model
              value: gpt-4o
      backendRefs:
        - name: openai-backend

Routes without hostnames remain accessible on all hosts. When at least one route uses hostname scoping, the /v1/models endpoint automatically returns only the models for the matching host.

Rules-Per-Route Limit

AIGatewayRoute.spec.rules is now capped at 15 (down from 128) to match the Gateway API HTTPRoute limit. If you have routes with more than 15 rules, split them across multiple AIGatewayRoute resources attached to the same Gateway.

Using Anthropic Prefix

If you route Anthropic traffic to a provider with a non-standard path, use the prefix field:

schema:
  name: Anthropic
  prefix: /custom/v2 # produces /custom/v2/messages

Note: prefix is ignored for AWSAnthropic and GCPAnthropic backends as they override paths internally.

Adopting Claude Opus 4.7 Reasoning

If you use Claude Opus 4.7 or Mythos Preview models, note that display defaults to omitted (unlike earlier Claude models which default to summarized). To receive summarized thinking content, set display: "summarized" explicitly. The new xhigh reasoning effort tier is available for long-horizon agentic tasks.

📦 Dependencies Versions

Go 1.26.2

Continues on Go 1.26.2, same as v0.6.

Envoy Gateway v1.7.0

Built on Envoy Gateway v1.7.0 for proven data plane capabilities.

Envoy v1.37

Leveraging Envoy Proxy v1.37.0 for battle-tested networking.

Gateway API v1.4.1

Support for Gateway API v1.4.1 specifications.

Gateway API Inference Extension v1.0.2

Continued integration with Gateway API Inference Extension v1.0.2 for intelligent endpoint selection.

MCP Go SDK v1.6.0

Updated from v1.4.1 to v1.6.0 for the latest MCP protocol features and fixes.

⏩ Patch Releases

🙏 Acknowledgements

We extend our gratitude to all contributors who made this release possible. Special thanks to:

• The growing community of adopters for their valuable feedback and production insights
• Everyone who reported bugs, submitted PRs, and participated in design discussions
• The Envoy Gateway team for their continued collaboration

🔮 What's Next

We're already working on features for future releases:

• Full quota-aware routing — building on the rate limit filter injection landed in v0.7 to route around rate-limited upstreams automatically across multiple backends
• MCPBackend CRD — a dedicated custom resource for MCP backend servers, decoupling MCP backend configuration from MCPRoute
• Expanded multimodal support — additional audio, video, and image generation backends across cloud providers
• Deeper MCP authorization — finer-grained policy across tools, resources, and prompts
• More provider translation paths — filling coverage gaps across Anthropic, Bedrock, and Vertex AI

Original source

Envoy AI Gateway v0.6.0 marks the first production-ready API surface: the core CRDs (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, MCPRoute) are now served at v1beta1. AWS Bedrock gains a native InvokeModel path for Claude alongside Titan embeddings via the OpenAI /v1/embeddings contract. Gemini gets first-class embeddings and Anthropic-style prefix context caching. Cross-provider clients can hit Anthropic's /v1/messages endpoint on any OpenAI-compatible backend, and a single reasoning_effort knob now works across Anthropic, OpenAI, and Gemini. Operators get GKE Workload Identity via Application Default Credentials, configurable webhook host networking, request/response body redaction for compliance, and the Go 1.26.2 + Envoy 1.37 + Envoy Gateway 1.7 baseline. Two breaking changes land in v0.6 — AIGatewayRoute.spec.filterConfig is removed (move to GatewayConfig), and the deprecated version-as-prefix behavior on VersionedAPISchema is removed (use prefix). See Upgrade Guidance below.

✨ New Features

AWS Bedrock

Native InvokeModel API for Claude

Send requests to Claude models on Bedrock through Bedrock's native InvokeModel endpoint, complementing the existing Converse API path. Useful when applications already speak the Anthropic Messages format and want a thin translation layer.

OpenAI → Bedrock Titan embeddings translation

Call Amazon Titan embedding models on Bedrock through the standard OpenAI /v1/embeddings contract. Switch embedding providers without changing client code. Cohere and other Bedrock embedding models are not yet covered and will follow in a later release.

Anthropic and Cross-Provider Translation

Anthropic /v1/messages endpoint on OpenAI backends

Expose any OpenAI-compatible backend through Anthropic's Messages API. Lets Claude-style clients reach OpenAI, Azure OpenAI, or any other OpenAI-compatible provider behind the gateway without rewriting requests.

Structured output for Claude models

Pass JSON schema constraints through to Claude so responses conform to your declared shape. Available on Anthropic and AWS Bedrock Claude backends today; GCP Vertex AI Claude is excluded pending upstream provider support.

Cleaner handling when max_tokens is omitted on Anthropic requests

Requests without an explicit max_tokens no longer crash the translator; they're forwarded so the provider returns a normal validation error. Removes a long-standing footgun when forwarding OpenAI-shaped requests through the Anthropic path.

Adaptive thinking for claude-opus-4.6

Translate Claude's new adaptive thinking mode end-to-end. Adaptive lets the model decide thinking depth per request rather than committing to a fixed budget, so callers can opt in without bespoke provider code.

Unified reasoning_effort across Anthropic, OpenAI, and Gemini

A single OpenAI-style reasoning_effort value (low / medium / high / xhigh) now maps onto Anthropic's thinking budgets and Gemini 3's thinking controls. One client knob, three providers.

Gemini Provider

Gemini embeddings translation

Use Gemini embedding models through the OpenAI /v1/embeddings contract, completing Gemini coverage alongside chat completions and Responses.

Gemini context caching with prefix-style API

Activate Gemini's context caching using the same Anthropic-style cache_control prefix surface already supported elsewhere. Cut input token costs on long, repeated system prompts without a Gemini-specific code path.

Gemini reasoning surfaced as thinking blocks

Non-streaming Gemini reasoning is now exposed as both string content and structured thinking_blocks, matching the shape clients already use for Anthropic responses. Streaming responses still surface reasoning as string content only.

OpenAI API Compatibility

Responses API — context management and richer streaming

Second wave of Responses API work fills in context management and improved streaming so the /v1/responses path is closer to parity with /v1/chat/completions. If you held off on /v1/responses due to missing features, retest now.

Compatibility with open-source Responses API implementations

Improved compatibility with non-OpenAI implementations of the Responses API (e.g. open-source inference servers that expose a /v1/responses endpoint), broadening which Responses-aware clients can sit in front of the gateway.

Text-to-speech endpoint /v1/audio/speech

Route OpenAI text-to-speech requests through the gateway, so audio workloads benefit from the same auth, rate limiting, and observability as chat traffic.

MCP Gateway

Per-backend header forwarding with rename

MCPRouteBackendRef.forwardHeaders accepts a list of inbound headers to forward to each backend, optionally renaming them on the way out. Each MCP backend can receive its own set of headers (e.g. trace context, tenant identifiers, per-user auth) without a single route-wide rule.

JWT claim forwarding to MCP backends

Project verified JWT claims into outbound headers via MCPRouteOAuth.claimToHeaders, enabling identity-aware tool execution at backend MCP servers without re-authenticating downstream.

Exclude / excludeRegex on tool selectors

MCPToolFilter now supports deny patterns (literal exclude and regex excludeRegex) alongside the existing include rules. Useful when a backend exposes more capabilities than a given route should surface.

Tool name in access logs and response metadata

Tool invocations now carry the tool name in dynamic metadata (key mcp_tool_name), so per-tool debugging, dashboards, and access-log fields are straightforward to wire up.

Per-backend capability tracking

The gateway tracks which MCP server feature flags (tools, prompts, resources, logging, completions) each backend supports and merges them across a route. Capability negotiation now reflects what's actually reachable, so clients don't get told a feature is available when no reachable backend implements it.

Authentication and Identity

GKE Workload Identity via Application Default Credentials

GCP backends now authenticate using the standard ADC chain when neither credentialsFile nor workloadIdentityFederationConfig is set in the BackendSecurityPolicy. Workloads running on GKE pick up Workload Identity automatically — no static service account JSON secret needed.

Security and Privacy

Request and response body redaction

Strip or mask sensitive fields in request and response bodies before they hit logs, traces, or metrics. Lets you keep observability on while meeting privacy and compliance constraints.

Observability

OTLP access logging auto-configured by aigw

Standalone aigw wires up OTLP access logging out of the box when an OTLP endpoint is configured (via OTEL_EXPORTER_OTLP_ENDPOINT), removing a manual step from local-dev and demo paths.

Default agent-session-id → session.id header mapping

Spans and logs now correlate by session.id automatically when clients send the agent-session-id header, so agent frameworks like Goose get session correlation with zero config. Override or disable via OTEL_AIGW_REQUEST_HEADER_ATTRIBUTES. Metrics never default to session IDs (high cardinality).

ReasoningToken cost type

LLMRequestCostType now includes ReasoningToken, so you can budget and bill against thinking tokens separately from input, output, and cache cost types.

Response model metadata

Responses now carry the resolved upstream model in metadata, which clients and downstream tools can read to confirm exactly which model served a request (useful when routes use model aliasing or fallback).

OTEL attribute count cap removed for large contexts

Removed the OTEL span attribute count limit so long-context requests no longer have parts of their trace silently dropped.

Operations and Extensibility

Custom webhook port and host network

The conversion webhook can now bind to a configurable port (controller.mutatingWebhook.port) and run on the host network (controller.hostNetwork), smoothing installs in clusters with restrictive admission webhook networking such as GKE private clusters.

Lua filter slot after the AI ExtProc stage

Lua filters can now be attached after the AI ExtProc stage in the standard filter chain, so you can do last-mile request shaping (header rewrites, body tweaks) without writing a custom EnvoyExtensionPolicy.

Route-scoped LLM request costs with global defaults

Set GatewayConfig.spec.globalLLMRequestCosts for fleet-wide defaults and override per-route at AIGatewayRoute.spec.llmRequestCosts. Makes per-tenant or per-backend cost tracking straightforward without per-route boilerplate.

🔗 API Updates

• Core CRDs promoted to aigateway.envoyproxy.io/v1beta1: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute are now served at v1beta1, signaling that the core API and MCP routing surface are stable enough for production use. v1alpha1 versions remain registered with deprecation warnings so existing manifests continue to apply during the upgrade window.
• MCPRouteBackendRef.forwardHeaders: New per-backend list of headers to forward, with optional rename. Replaces the need for a single route-wide header forwarding rule when backends expect different headers.
• MCPRouteOAuth.claimToHeaders: Configure which verified JWT claims should be projected into outbound headers to MCP backends.
• MCPToolFilter.exclude/excludeRegex: Tool selectors now support exclusion alongside inclusion, with both literal and regex forms.
• LLMRequestCostType.ReasoningToken: New cost type for thinking-token usage, complementing the existing input, output, and cache cost types.
• GatewayConfig.spec.globalLLMRequestCosts: Fleet-wide cost defaults that individual AIGatewayRoute.spec.llmRequestCosts entries can override.
• Preview: QuotaPolicy API (v1alpha1, no runtime enforcement yet): New CRD surface for declaring upstream-provider quota policies, laying the groundwork for quota-aware routing. Currently API-only — no controller reconciliation or enforcement is wired up. Track it for future releases; do not rely on it as a working feature today.

⚠️ Breaking Changes

• AIGatewayRoute.spec.filterConfig removed: The filterConfig field on AIGatewayRoute has been removed. Move external-processor configuration (resources, env vars, image overrides) to a GatewayConfig resource referenced from the Gateway via the aigateway.envoyproxy.io/gateway-config annotation. v0.5 deprecated the resources subfield with a pointer to GatewayConfig; v0.6 removes the entire filterConfig struct, so anything still set there must move now.
• VersionedAPISchema.version no longer acts as an endpoint prefix for OpenAI-schema backends: The legacy behavior deprecated in v0.5 — using the version field as a path prefix for OpenAI-schema backends — has been removed. Use the dedicated prefix field instead (e.g. prefix: /v1beta/openai for Gemini's OpenAI-compatible API, prefix: /compatibility/v1 for Cohere).

🐛 Bug Fixes

• Webhook cache race during extProc injection: Fixes a race where freshly applied AIGatewayRoute resources could miss extProc injection on first reconcile because the conversion webhook read from a stale cache. Scripted apply-then-curl tests should see fewer flakes.
• Field ownership preserved on updates: Controllers no longer claim ownership of fields they don't manage during updates. If you co-deploy the AI Gateway controller alongside other operators that touch adjacent fields (e.g. service mesh injectors, policy controllers), expect fewer reconcile churn loops on shared resources.
• Orphan cleanup for MCPRoute backendrefs: Resources tied to MCPRoute backend references are now cleaned up when the route or reference is removed, fixing a leak that could leave stale config in the cluster.
• Standalone Envoy startup failures surfaced by aigw: aigw now reports standalone Envoy startup failures cleanly instead of hanging or printing an unhelpful trace, making local dev and CI loops much faster to diagnose.
• Bedrock Titan embeddings dataplane route: Restored the Envoy route for Titan embeddings in dataplane tests so Titan workloads exercise the full pipeline.
• Hardened bearer token parsing: Malformed Authorization: Bearer headers used to panic the MCP subject extractor; they now return a clean auth failure and the request falls through to the standard auth failure path.
• Request context propagation in PostTranslateModify: Kubernetes client calls inside PostTranslateModify now honor request cancellation and deadlines, reducing stuck reconciles when the parent request is canceled.
• Case-sensitive JSON marshalling and unmarshalling: JSON encoding now consistently honors case, fixing subtle mismatches when round-tripping fields whose names differ only in case (visible previously as occasional 400s on certain provider payloads).
• Secret rotation propagates to MCPRoute: Updates to a Secret referenced by an MCPBackendRef are now reflected in the live configuration, matching how BackendSecurityPolicy already handled secret updates. Operators rotating MCP backend credentials no longer need to bounce the route.
• MCP proxy handles compressed Accept-Encoding from upstreams: The MCP proxy now correctly handles compressed Accept-Encoding values from upstream requests, fixing failures when MCP backends advertise gzip or other compression schemes.
• aigw standalone accepts IP addresses for endpoints: In standalone mode, aigw previously assumed endpoints were hostnames; you can now point OpenAI and MCP env config at IP addresses (e.g. 127.0.0.1), making local dev against loopback addresses work.

📖 Upgrade Guidance

Migrating from filterConfig to GatewayConfig

The filterConfig field on AIGatewayRoute has been removed in v0.6. If you previously configured the external processor (resources, environment variables, image overrides) via filterConfig on individual routes, move that configuration to a GatewayConfig resource and reference it from the Gateway.

Migrating VersionedAPISchema.version to prefix

The deprecated v0.5 behavior of using VersionedAPISchema.version as an endpoint path prefix for OpenAI-schema backends has been removed in v0.6. Use the dedicated prefix field instead.

Adopting v1beta1 APIs

AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute are now served at aigateway.envoyproxy.io/v1beta1. Existing v1alpha1 manifests continue to work via conversion, but new manifests should target v1beta1 directly.

Switching GCP backends to Workload Identity

If you're running on GKE, drop static service-account keys and let the gateway pick up Application Default Credentials. Configure your BackendSecurityPolicy for GCP with the appropriate workload identity binding on the controller's service account; no serviceAccountJSON secret is required.

📦 Dependencies Versions

Go 1.26.2

Updated to Go 1.26.2 to pick up the latest security and performance fixes.

Envoy Gateway v1.7.0

Built on Envoy Gateway v1.7.0 for the newest data plane capabilities and stability fixes.

Envoy v1.37

Leveraging Envoy Proxy v1.37.0 for the latest networking and security features.

Gateway API v1.4.1

Support for Gateway API v1.4.1 specifications.

Gateway API Inference Extension v1.0.2

Continued integration with Gateway API Inference Extension v1.0.2 for stable intelligent endpoint selection.

MCP Go SDK 1.4.1

Updated to modelcontextprotocol/go-sdk v1.4.1 for the latest MCP protocol features and fixes.

⏩ Patch Releases

🙏 Acknowledgements

We extend our gratitude to all contributors who made this release possible. Special thanks to:

• The growing community of adopters for their valuable feedback and production insights
• Everyone who reported bugs, submitted PRs, and participated in design discussions
• The Envoy Gateway team for their continued collaboration

🔮 What's Next

We're already working on features for future releases:

• Quota-aware routing — building on the new backend quota policy API to route around rate-limited upstreams automatically
• Deeper MCP authorization — finer-grained policy across tools, resources, and prompts
• Expanded provider coverage — additional embeddings, audio, and image generation backends across cloud providers
• More efficient large-context handling — continued improvements to streaming, memory use, and tracing for long-context workloads

Original source

Multi-gateway configuration, prompt caching cost savings, fine-grained MCP authorization, OpenAI Responses API, and Google Search grounding for Gemini

Envoy AI Gateway v0.5.0 makes multi-gateway deployments easier with the new GatewayConfig CRD, cuts costs with prompt caching for AWS Bedrock and GCP Claude, and unlocks fine-grained access control with CEL-based MCP authorization. Developers gain OpenAI Responses API support, Google Search grounding for Gemini, and the ability to mutate request bodies per-route. Under the hood, the switch to sonic JSON processing reduces latency across all requests.

✨ New Features

Gateway Configuration

New GatewayConfig CRD

Introduces a new GatewayConfig custom resource for gateway-scoped configuration. Reference it from a Gateway via the aigateway.envoyproxy.io/gateway-config annotation to configure the external processor container with environment variables, resource requirements, and other Kubernetes container settings. Multiple Gateways can share the same GatewayConfig for consistent configuration.

Configurable endpoint prefixes

Added prefix field to VersionedAPISchema for configuring API endpoint prefixes. Useful for routing to backends with OpenAI-compatible APIs that use non-standard prefixes, such as Gemini's /v1beta/openai or Cohere's /compatibility/v1.

OpenAI API Support

OpenAI Responses API /v1/responses

Full support for OpenAI's new Responses API endpoint with streaming and non-streaming modes, function calling, MCP tools support, reasoning, multi-turn conversations, and native multimodal capabilities. Includes complete token usage tracking and OpenInference tracing.

Provider Caching Enhancements

Prompt caching for AWS Bedrock Claude

Reduce costs and latency by reusing cached system prompts with Bedrock Anthropic models. The gateway handles cache point markers automatically and tracks both cache creation and cache hit tokens separately for accurate billing visibility.

Prompt caching for GCP Vertex AI Claude

Same cost-saving prompt caching now works with Claude models on GCP Vertex AI. Cache your system prompts and few-shot examples to cut input token costs on repeated requests.

MCP Gateway Enhancements

Fine-grained authorization with CEL, JWT claims, and external auth

Comprehensive authorization system for MCP routes. Write expressive CEL rules using request attributes (HTTP method, headers, JWT claims, tool names and call arguments), enforce access based on JWT claim values from your identity provider, or delegate to external gRPC/HTTP authorization services. Control which users can access which tools with precision.

Real-time tool list synchronization

MCP clients automatically stay in sync when you update MCPRoutes. The gateway sends notifications/tools/list_changed notifications, so clients refresh their available tools without manual intervention or reconnection.

Stdio server proxy in standalone mode

Run command-line MCP tools (like npx-based servers) without code changes. The aigw CLI proxies stdio-based MCP servers over HTTP, letting you integrate CLI tools directly into your AI workflows.

Improved OAuth metadata discovery

OAuth well-known endpoints now serve at the MCPRoute path prefix, ensuring MCP clients correctly discover authorization configuration even when multiple routes have different security policies.

Inference Extension

Security policies for inference pools

Apply BackendSecurityPolicy to InferencePool resources, ensuring consistent authentication across all dynamically-selected inference endpoints in your pool.

Gemini Provider Enhancements

Ground responses with live web search

Give Gemini models access to real-time information via the google_search tool type. Filter results by domain, set blocking confidence thresholds, and restrict time ranges to ensure responses use current, relevant sources.

Consistent thinking configuration across providers

Use the same thinking configuration for both Anthropic and Gemini models. Write provider-agnostic code that works with extended thinking features regardless of which model backend processes the request.

Gemini 3 reasoning and image quality controls

Fine-tune Gemini 3 behavior with thinking_level (control reasoning depth) and media_resolution (balance image quality vs. speed). These settings gracefully degrade on older Gemini versions.

Visibility into model reasoning

When thinking is enabled, the gateway extracts and surfaces thought summaries from Gemini responses, helping you understand how the model reached its conclusions.

Enterprise web search integration

Connect Gemini to your organization's search infrastructure with the enterprise_search tool type. Configure custom data sources for grounding responses in enterprise-specific knowledge.

Traffic Management

Route-level body mutation

Inject or remove JSON fields in request bodies per-backend without application changes. Use bodyMutation to add service_tier, set custom parameters, or strip internal fields before requests reach different providers. Route-level settings override backend defaults for flexible multi-backend configurations.

AWS Bedrock service tier control

Choose between standard, flex, priority, and reserved for Bedrock requests. Set service_tier to prioritize latency-sensitive workloads or optimize costs for batch processing, with automatic fallback handling.

Observability Enhancements

Per-provider cost attribution

Track spending across AI providers with the new gen_ai.provider.name metric attribute. Filter dashboards and alerts by provider to identify cost drivers and optimize your multi-provider strategy.

Full tracing for Anthropic Messages API

Debug Claude model requests end-to-end with OpenInference-compliant tracing for the native /messages endpoint. View prompts, responses, and timing in Arize Phoenix or any OpenTelemetry-compatible observability platform.

Cohere Rerank visibility

Track reranking operations in your traces. Full OpenTelemetry support for Cohere's v2 rerank endpoint captures query, documents, and relevance scores for debugging RAG pipelines.

Performance and Operations

Faster request processing with sonic JSON

Reduced gateway latency by migrating to bytedance/sonic for JSON encoding and decoding. Expect measurable latency improvements and lower CPU usage, especially for large payloads and high request volumes.

Faster cross-namespace reference validation

Reduced controller reconciliation time with optimized ReferenceGrant indexing. Cross-namespace backend references now validate faster, improving startup time and configuration change responsiveness.

Improved MCP proxy throughput

MCP proxy now reuses HTTP connections across requests, eliminating per-request connection overhead. This significantly improves throughput when proxying to multiple MCP backend servers.

🔗 API Updates

• New GatewayConfig CRD: New custom resource for gateway-level configuration. Spec includes extProc.kubernetes for container settings (resources, env vars, image overrides). Reference from Gateway via aigateway.envoyproxy.io/gateway-config annotation.
• VersionedAPISchema.prefix: New prefix field replaces overloading version for endpoint path customization. Example: prefix: "/v1beta/openai" for Gemini's OpenAI-compatible API.
• AIGatewayRouteRuleBackendRef.bodyMutation: New field with set (list of field/value pairs) and remove (list of field names) for request body manipulation.
• LLMRequestCostType.CacheCreationInputToken: New cost type for tracking tokens written to cache, separate from CachedInputToken (tokens read from cache).
• MCPRouteSecurityPolicy authorization fields: New authorization block with defaultAction (Allow/Deny), rules array supporting cel expressions, JWT scopes/claims, and tools targeting. New extAuth field for external authorization delegation.
• BackendSecurityPolicy.targetRefs expansion: Now accepts InferencePool (group: inference.networking.x-k8s.io) in addition to AIServiceBackend.

Deprecations

• AIGatewayFilterConfigExternalProcessor.resources: The resources field in AIGatewayFilterConfigExternalProcessor is deprecated. Use GatewayConfig for gateway-scoped resource configuration instead. This field will be removed in the next v0.6.
• version field as prefix for OpenAI schema: Using the version field as a prefix for OpenAI schema is deprecated. Use the new prefix field instead. The legacy behavior will be removed in the next v0.6.

🐛 Bug Fixes

• AWS Bedrock Claude streaming reliability: Streaming responses from Bedrock Claude models now complete correctly. Previously, some streamed responses could be truncated or malformed.
• Gemini streaming token counts: Token usage in Gemini streaming responses now matches OpenAI format, fixing integrations that parse usage from stream chunks.
• Multi-chunk Gemini tool calls: Tool calls that span multiple streaming chunks now have correct indices, preventing function calling errors with Gemini.
• GCP Claude reasoning content: Reasoning/thinking content now correctly passes through for Claude models on GCP Vertex AI in both requests and responses.
• Zero-weight backend references: Backend references with zero weight no longer cause routing errors, allowing gradual traffic shifts via weight changes.
• Umbrella chart image pull secrets: Helm deployments within umbrella charts now correctly inherit global.imagePullSecrets when chart-level secrets aren't set.
• GCP global region backends: Vertex AI backends configured with global region now work correctly instead of failing during setup.
• Accurate per-token latency metrics: Fixed integer truncation in time_per_output_token calculation that caused incorrect latency measurements for fast responses.
• Anthropic token counting: Improved accuracy of input and output token counts for Anthropic models, ensuring billing metrics match provider reports.

📖 Upgrade Guidance

Migrating to GatewayConfig

If you're using AIGatewayFilterConfigExternalProcessor.resources for container resource configuration, migrate to the new GatewayConfig CRD:

1. Create a GatewayConfig resource with your desired configuration:

   apiVersion: aigateway.envoyproxy.io/v1alpha1
   kind: GatewayConfig
   metadata:
     name: my-gateway-config
     namespace: default
   spec:
     extProc:
       kubernetes:
         resources:
           requests:
             cpu: "100m"
             memory: "128Mi"
           limits:
             cpu: "500m"
             memory: "512Mi"
         env:
           - name: OTEL_EXPORTER_OTLP_ENDPOINT
             value: "http://otel-collector:4317"
   

2. Reference the GatewayConfig from your Gateway:

   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: ai-gateway
     annotations:
       aigateway.envoyproxy.io/gateway-config: my-gateway-config
   

Migrating Endpoint Prefix Configuration

If you're using the version field as a prefix for OpenAI-compatible backends, migrate to the new prefix field:

Before:

schema:
  name: OpenAI
  version: "/v1beta/openai" # Deprecated usage

After:

schema:
  name: OpenAI
  prefix: "/v1beta/openai"

📦 Dependencies Versions

Go 1.25.6

Updated to Go 1.25.6 for improved performance and security fixes.

Envoy Gateway v1.6

Built on Envoy Gateway v1.6 for enhanced data plane capabilities and stability improvements.

Envoy v1.36

Leveraging Envoy Proxy v1.36.4 with the latest networking and security features.

Gateway API v1.4.0

Support for Gateway API v1.4.0 specifications.

Gateway API Inference Extension v1.0.2

Integration with Gateway API Inference Extension v1.0.2 for stable intelligent endpoint selection.

⏩ Patch Releases

🙏 Acknowledgements

We extend our gratitude to all contributors who made this release possible. Special thanks to:

• The growing community of adopters including Bloomberg, LY Corporation, Alan by Comma Soft, and NRP for their valuable feedback and production insights
• Everyone who reported bugs, submitted PRs, and participated in design discussions
• The Envoy Gateway team for their continued collaboration

🔮 What's Next

We're already working on exciting features for future releases:

• Additional provider integrations - AWS Bedrock InvokeModel API support for Claude and GPT models, Gemini embeddings, and Azure/AKS workload identity
• Batch inference APIs - Support for batch processing to improve throughput for high-volume workloads
• Advanced caching strategies - Prompt cache key and retention controls for OpenAI chat completions
• Upstream provider quota policies - New API for managing upstream provider quotas
• Sensitive data redaction - Request and response body redaction for protecting sensitive information

Original source

Envoy AI Gateway v0.4.0 is a major release introducing Model Context Protocol (MCP) Gateway with full OAuth and server multiplexing support, OpenAI Image Generation API, first-party Anthropic provider integration, and native Anthropic Messages API support for AWS Bedrock. This release also brings guided output decoding for GCP Vertex AI/Gemini, cross-namespace resource references, enhanced authentication for AWS and Azure, and comprehensive observability improvements across all providers.

✨ New Features

Model Context Protocol (MCP) Gateway

New MCPRoute CRD

Introduces MCPRoute custom resource for routing MCP requests to backend MCP servers, enabling unified AI API for multiple MCP backends.

Complete MCP spec implementation

Includes streamable HTTP transport, JSON-RPC 2.0 support, and MCP spec-compliant OAuth 2.0 authorization with JWKS validation and Protected Resource Metadata.

Server multiplexing and tool routing

Aggregates multiple MCP servers behind a single endpoint with intelligent tool routing, tool filtering (exact match and regex patterns), and collision detection.

Upstream authentication

Supports both OAuth-based authentication and API key authentication for secure backend MCP server communication with configurable headers.

Session management

Implements MCP session handling with encryption, rotatable seeds, and graceful session lifecycle management.

Anthropic Provider Support

Direct api.anthropic.com support

Native integration with Anthropic's API at api.anthropic.com, complementing existing GCP Vertex AI Anthropic support.

AWS Bedrock native Anthropic Messages API

Support for Claude models on AWS Bedrock using the native Anthropic Messages API format instead of the generic Converse API, enabling full feature parity with direct Anthropic API including prompt caching and extended thinking.

Anthropic API key authentication

Native x-api-key header-based authentication matching Anthropic's API conventions and SDK patterns for direct Anthropic connections.

Passthrough translator with token usage tracking

Efficient passthrough translation layer that captures token usage and maintains API compatibility while minimizing overhead for both direct and AWS Bedrock Anthropic endpoints.

Standalone CLI auto-configuration

Auto-configuration from ANTHROPIC_API_KEY environment variable in standalone mode for zero-config deployments.

Guided Output Support for GCP Vertex AI/Gemini

Guided regex support

Constrains model outputs to match specific regular expressions for GCP Vertex AI/Gemini models, enabling structured text generation.

Guided choice support

Restricts model outputs to predefined choices for GCP Vertex AI/Gemini models, ensuring responses conform to expected values.

Guided JSON support

Ensures model outputs are valid JSON conforming to specified schemas for GCP Vertex AI/Gemini models, with OpenAI-compatible API translation.

Provider-Specific Enhancements

OpenAI Image Generation /v1/images/generations endpoint

End-to-end support for OpenAI's image generation API including request/response translation, Brotli encoding/decoding, and full protocol compatibility.

OpenAI legacy /v1/completions endpoint

Full pass-through support for OpenAI's legacy completions endpoint with complete tracing and metrics, ensuring backward compatibility.

Azure OpenAI embeddings support

Native support for Azure OpenAI embeddings API with proper protocol translation and token usage tracking.

AWS Bedrock reasoning tokens

Full support for reasoning/thinking tokens in AWS Bedrock responses for both streaming and non-streaming modes, properly exposing extended thinking processes in Claude models.

GCP Vertex AI safety settings

Support for GCP-specific safety settings configuration, allowing fine-grained control over content filtering and safety thresholds for Gemini models.

GCP Gemini streaming token accounting

Accurate completion_tokens reporting in streaming usage chunks for Gemini models, ensuring proper token accounting during streaming responses.

Cross-Namespace Resource References

Cross-namespace AIServiceBackend references

AIGatewayRoute can now reference AIServiceBackend resources in different namespaces, enabling multi-tenant and organizational separation patterns.

ReferenceGrant validation

Comprehensive ReferenceGrant integration following Gateway API patterns, with automatic validation and clear error messages when grants are missing.

Enhanced Upstream Authentication

AWS SDK default credential chain

Support for AWS SDK's default credential chain including IRSA (IAM Roles for Service Accounts), EKS Pod Identity, EC2 Instance Profiles, and environment variables, eliminating need for static credentials or OIDC settings

Azure API key authentication

Native Azure OpenAI API key authentication using the api-key header, matching Azure SDK conventions and console practices.

Traffic Management and Configuration

Header mutations at route and backend levels

New headerMutation fields in both AIServiceBackend and AIGatewayRouteRuleBackendRef enable header manipulation with smart merge logic for advanced routing scenarios.

InferencePool v1 support

Updated to Gateway API Inference Extension v1.0, providing stable intelligent endpoint selection with enhanced performance and reliability.

Cached token usage tracking for actual token usage reporting

Captures and reports cached token statistics from cloud providers (Anthropic, Bedrock, etc.), providing accurate cost attribution for prompt caching features.

Standalone Mode and CLI

Docker image support

Official Docker images for the aigw CLI published to GitHub Container Registry, enabling containerized standalone deployments with proper health checks and lifecycle management.

Multi-provider auto-configuration

Zero-config standalone mode with automatic configuration from OPENAI_API_KEY, AZURE_OPENAI_API_KEY, or ANTHROPIC_API_KEY environment variables. Generates complete Envoy configuration with OpenAI SDK compatibility.

MCP server configuration

Native MCP support in standalone mode via --mcp-config and --mcp-json flags, enabling unified LLM and MCP server configuration in a single aigw run invocation without Kubernetes.

XDG Base Directory standards

Proper separation of configuration, data, state, and runtime files following XDG Base Directory specification, improving organization and enabling better cleanup and management of aigw state.

Enhanced readiness monitoring

Improved Envoy readiness detection and status reporting in standalone mode, providing clear insights into when the gateway is ready to accept traffic with better error messages.

Consolidated admin server

Unified admin server on a single port serving both /metrics and /health endpoints, simplifying monitoring and health check configuration.

Improved error handling

aigw CLI now fails fast and exits cleanly if external processor fails to start, preventing silent failures and improving debugging experience.

Type-safe Kubernetes client SDK

Generated client libraries for all AI Gateway CRDs following standard Kubernetes client-go patterns, enabling developers to build controllers, operators, and custom integrations with type safety.

Observability Enhancements

MCP operations observability

Comprehensive monitoring, logging, and tracing for MCP operations with configurable access logs and metrics enrichment for MCP server interactions and tool routing.

Image generation tracing and metrics

OpenInference-compliant distributed tracing and OpenTelemetry Gen AI metrics for image generation requests with detailed request parameters and timing information.

OpenTelemetry native metrics export

Support for OTEL-native metrics export (in addition to Prometheus), enabling integration with Elastic Stack, OTEL-TUI, and other OTEL-native observability systems. Includes console exporter for ad-hoc debugging.

Embeddings tracing implementation

Complete OpenInference-compliant tracing for embeddings operations, complementing existing chat completion tracing.

Enhanced /messages endpoint metrics

Distinct metrics for Anthropic's /messages endpoint, providing accurate attribution separate from /chat/completions endpoints.

Original model tracking

Metrics now track both the original requested model and any overridden model names, providing accurate attribution in multi-provider and model virtualization scenarios.

🔗 API Updates

• New MCPRoute CRD: Introduces MCPRoute custom resource with comprehensive fields for MCP server configuration, tool filtering, authentication policies (OAuth and API key), and Protected Resource Metadata.
• Cross-namespace references in AIGatewayRoute: Added namespace field to AIGatewayRouteRuleBackendRef, enabling cross-namespace backend references with ReferenceGrant validation.
• Header mutations at route and backend levels: Added headerMutation fields to both AIServiceBackend and AIGatewayRouteRuleBackendRef for backend-level and per-route header manipulation with smart merge logic.
• New AWSAnthropic API schema: Added AWSAnthropic schema for Claude models on AWS Bedrock using the native Anthropic Messages API format, providing full feature parity with direct Anthropic API.
• Anthropic API key authentication: Added AnthropicAPIKey to BackendSecurityPolicy for x-api-key header authentication.
• Azure API key authentication: Added AzureAPIKey to BackendSecurityPolicy for api-key header authentication.
• AWS credential chain support: BackendSecurityPolicy AWS auth now supports SDK default credential chain when credentials are not explicitly provided.
• InferencePool v1: Updated to support Gateway API Inference Extension v1.0 (inference.networking.k8s.io/v1) instead of v1alpha1.
• Enforced Backend resource requirement: Added CRD validation to AIServiceBackend explicitly requiring Envoy Gateway Backend resources (Kubernetes Service is not supported).

📦 Dependencies Versions

Go 1.25.3

Updated to Go 1.25.3 for improved performance and security.

Envoy Gateway v1.5+

Built on Envoy Gateway v1.5+ for proven data plane capabilities and enhanced features. This version is also fully compatible with the upcoming v1.6

Envoy v1.35+

Leveraging Envoy Proxy v1.35+ battle-tested networking capabilities.

Gateway API v1.4.0

Support for Gateway API v1.4.0 specifications.

Gateway API Inference Extension v1.0

Integration with Gateway API Inference Extension v1.0 for stable intelligent endpoint selection.

🔮 What's Next (beyond v0.4)

We're already working on exciting features for future releases:

• Advanced MCP features - Further enhancements to the MCP protocol support
• Additional LLM provider integrations - Expanding the ecosystem of supported LLM providers
• Enhanced Performance - Improving the runtime performance of the gateway

Original source

Release version introducing intelligent inference routing with Endpoint Picker Provider, enhanced observability features, Google Vertex AI support, and enhanced provider integrations.

Envoy AI Gateway v0.3.0 introduces intelligent inference routing, expanded provider support (including Google Vertex AI and Anthropic), and enhanced observability with OpenInference tracing and configurable metrics. Key features include Endpoint Picker Provider with InferencePool for dynamic load balancing, model name virtualization, and seamless Gateway API Inference Extension integration.

✨ New Features

Endpoint Picker Provider (EPP) Integration

Gateway API Inference Extension Support

Complete integration with Gateway API Inference Extension v0.5.1, enabling intelligent endpoint selection based on real-time AI inference metrics like KV-cache usage, queue depth, and LoRA adapter information.

Dual Integration Approaches

Support for both HTTPRoute + InferencePool and AIGatewayRoute + InferencePool integration patterns, providing flexibility for different use cases from simple to advanced AI routing scenarios.

Dynamic Load Balancing

Intelligent routing that automatically selects the optimal inference endpoint for each request, optimizing resource utilization across your entire inference infrastructure with real-time performance metrics.

Extensible Architecture

Support for custom endpoint picker providers, allowing implementation of domain-specific routing logic tailored to unique AI workload requirements.

Expanded Provider Ecosystem

Google Vertex AI Production Support

Google Vertex AI has moved from work-in-progress to full production support, including complete streaming support for Gemini models with OpenAI API compatibility.

Anthropic on Vertex AI Integration

Complete Anthropic Claude integration via GCP Vertex AI, moving from experimental to production-ready status with multi-tool support and configurable API versions for enterprise deployments.

Enhanced Gemini Capabilities

Improved request/response translation for Gemini models with support for tools, response format specification, and advanced conversation handling, making Gemini integration more robust and feature-complete.

Strengthened OpenAI-Compatible Ecosystem

Enhanced support for the broader OpenAI-compatible provider ecosystem including Groq, Together AI, Mistral, Cohere, DeepSeek, SambaNova, and more, ensuring seamless integration across the AI provider landscape.

Observability Enhancements

OpenInference Tracing Support

Added comprehensive OpenInference distributed tracing with OpenTelemetry integration, providing detailed request tracing and performance monitoring for LLM operations. Includes full chat completion request/response data capture, timing information, and compatibility with evaluation systems like Arize Phoenix.

Configurable Metrics Labels

Added support for configuring additional metrics labels corresponding to HTTP request headers. This enables custom labeling of metrics based on specific request headers like user identifiers, API versions, or application contexts, providing more granular monitoring and filtering capabilities.

Embeddings Metrics Support

Extended GenAI metrics support to include embeddings operations, providing comprehensive token usage tracking and performance monitoring for both chat completion and embeddings API endpoints with consistent OpenTelemetry semantic conventions.

Enhanced GenAI Metrics

Improved AI-specific metrics implementation with better error handling, enhanced attribute mapping, and more accurate token latency measurements. Maintains full compatibility with OpenTelemetry Gen AI semantic conventions while providing more reliable performance analysis data.

Infrastructure and Configuration

Model Name Virtualization

Added a new modelNameOverride field in the backendRef of AIGatewayRoute, enabling flexible model name abstraction across different providers. This allows unified model naming for downstream applications while routing to provider-specific model names, supporting both multi-provider scenarios and fallback configurations.

Unified Gateway Support

Enhanced Gateway resource management by allowing both standard HTTPRoute and AIGatewayRoute to be attached to the same Gateway object. This provides a unified routing configuration that supports both AI and non-AI traffic within a single gateway infrastructure, simplifying deployment and management.

🔗 API Updates

• BackendSecurityPolicy TargetRefs: Added targetRefs field to BackendSecurityPolicy spec, enabling direct targeting of AIServiceBackend resources using Gateway API policy attachment patterns.
• Gateway API Inference Extension: Allows InferencePool resource of Gateway API Inference Extension v0.5.1 to be specified as a backend ref in AIGatewayRoute intelligent endpoint selection.
• modelNameOverride in the backend reference of AIGatewayRoute: Added modelNameOverride field in the backend reference of AIGatewayRoute, allowing for flexible model name rewrite for routing purposes.

Deprecations

• backendSecurityPolicyRef Pattern: The old pattern of AIServiceBackend referencing BackendSecurityPolicy is deprecated in favor of the new targetRefs approach. Existing configurations will continue to work but should be migrated before v0.4.
• AIGatewayRoute's targetRefs Pattern: The targetRefs pattern is no longer supported for AIGatewayRoute. Existing configurations will continue to work but should be migrated to parentRefs.
• AIGatewayRoute's schema Field: The schema field is no longer needed for AIGatewayRoute. Existing configurations will continue to work but should be removed before v0.4.
• controller.envoyGatewayNamespace helm value is no longer necessary: This value is no longer necessary and is redundant when configured.
• controller.podEnv helm value will be removed: Use controller.extraEnvVars instead. The controller.podEnv value will be removed in v0.4.

📖 Upgrade Guidance

For users upgrading from v0.2.x to v0.3.0:

1. Upgrade Envoy Gateway to v1.5.0 - Ensure you are using Envoy Gateway v1.5.0 or later, as this is required for compatibility with the new AI Gateway features.

2. Update Envoy Gateway config - Update your Envoy Gateway configuration to include the new settings as below. The full manifest is available in the manifests/envoy-gateway-config/config.yaml file as per the getting started guide.

   --- a/manifests/envoy-gateway-config/config.yaml
   +++ b/manifests/envoy-gateway-config/config.yaml
   @@ -43,9 +43,19 @@ data:
   extensionManager:
   hooks:
   xdsTranslator:
   +     translation:
   +       listener:
   +         includeAll: true
   +       route:
   +         includeAll: true
   +       cluster:
   +         includeAll: true
   +       secret:
   +         includeAll: true
   post:
   - - VirtualHost
   - Translation
   + - Cluster
   + - Route
   

3. Upgrade Envoy AI Gateway to v0.3.0

4. Migrate Gateway target references - Update from the deprecated AIGatewayRoute.targetRefs pattern to the new AIGatewayRoute.parentRefs approach after the upgrade to v0.3.0.

5. Migrate backendSecurityPolicy references - Update from the deprecated AIServiceBackend.backendSecurityPolicyRef pattern to the new BackendSecurityPolicy.targetRefs approach after the upgrade to v0.3.0.

6. Remove AIGatewayRoute.schema - remove the schema field from AIGatewayRoute resources after the upgrade to v0.3.0, as it is no longer used.

📦 Dependencies Versions

Go 1.24.6

Updated to latest Go version for improved performance and security.

Envoy Gateway v1.5

Built on Envoy Gateway for proven data plane capabilities.

Envoy v1.35

Leveraging Envoy Proxy's battle-tested networking capabilities.

Gateway API v1.3.1

Support for latest Gateway API specifications.

Gateway API Inference Extension v0.5.1

Integration with Gateway API Inference Extension for intelligent endpoint selection.

⏩ Patch Releases

🙏 Acknowledgements

This release represents the collaborative effort of our growing community. Special thanks to contributors from Tetrate, Bloomberg, Tencent, Google, Nutanix and our independent contributors who made this release possible through their code contributions, testing, feedback, and community participation.

The Endpoint Picker Provider integration represents a significant milestone in making AI inference routing more intelligent and efficient. We appreciate all the feedback and testing from the community that helped shape this feature.

🔮 What's Next (beyond v0.3)

We're already working on exciting features for future releases:

• Support for MCP Protocol - Handle routing for MCP requests on the Envoy AI Gateway
• Restore support for referencing Kubernetes Services in AIServiceBackend to enable seamless integration with Kubernetes-native backends
• What else do you want to see? Get involved and open an issue and let us know!

Original source

Release version introducing Azure OpenAI integration, sidecar architecture, cross-backend failover, and a CLI tool.

v0.2.0 — June 5, 2025

Envoy AI Gateway v0.2.0 builds upon the solid foundation of v0.1.0 with focus on expanding provider ecosystem support, improving reliability and performance through architectural changes, and enterprise-grade authentication support for Azure OpenAI.

✨ New Features

Azure OpenAI Integration

Full Azure OpenAI Support

Complete integration with Azure OpenAI services with request/response transformation for the unified OpenAI compatiple completions API.

Upstream Authentication for Azure Enterprise Integration

Support for accessing Azure via OIDC tokens and Entra ID for enterprise-grade authentication for secure and compliant upstream authentication.

Enterprise Proxy URL Support for Azure Authentication

Enhanced Azure authentication with proxy URL configuration options for enterprise proxy support.

Flexible Token Providers

Generalized token provider architecture supporting both client secret and federated token flows

Architecture Improvements

Sidecar + UDS External Processor

Switched to sidecar deployment model with Unix Domain Sockets for improved performance and resource efficiency

Enhanced ExtProc Buffer Limits

Increased external processor buffer limits from 32 KiB to 50 MiB for larger AI requests. Users can now configure CPU and memory resource limits via filterConfig.externalProcessor.resources for better resource management.

Multiple AIGatewayRoute Support

Support for multiple AIGatewayRoute resources per gateway, removing the previous single-route limitation. This enables better organization, scalability, and management of complex routing configurations across teams.

Certificate Manager Integration

Integrated cert-manager for automated TLS certificate provisioning and rotation for the mutating webhook server that injects AI Gateway sidecar containers into Envoy Gateway pods. This enables enterprise-grade certificate management, eliminating manual certificate handling and improving security.

Cross-Backend Failover and Retry

Provider Fallback Logic

Priority-based failover system that automatically routes traffic to lower priority AI providers as higher priority endpoints become unhealthy, ensuring high availability and fault tolerance.

Backend Retry Support

Configurable retry policies for improved reliability and resilience against AI provider transient failures. Features include exponential backoff with jitter, configurable retry triggers (5xx errors, connection failures, rate limiting), customizable retry counts and timeouts, and integration with Envoy Gateway's BackendTrafficPolicy.

Weight-Based Routing

Enhanced backend routing with weighted traffic distribution, enabling gradual rollouts, cost optimization, and A/B testing across multiple AI providers

Enhanced CLI Tools

aigw run Command

New CLI command for local development and testing of Envoy AI Gateway resources.

Configuration Translation

aigw translate for translating Envoy AI Gateway Resources to Envoy Gateway and Kubernetes CRDs.

🔗 API Updates

• AIGatewayRoute Metadata: Added ownedBy and createdAt fields for better resource tracking.
• Backend Configuration: Moved Backend configuration back to RouteRule for improved flexibility.
• OIDC Field Types: Specific typing for OIDC-related configuration fields.
• Weight Type Changes: Updated Weight field type to match Gateway API specifications.

Deprecations

• AIServiceBackend.Timeouts: Deprecated in favor of more granular timeout configuration.

🐛 Bug Fixes

• ExtProc Image Syncing: Fixed issue where external processor image wouldn't sync properly.
• Router Weight Validation: Fixed negative weight validation in routing logic.
• Content Body Handling: Fixed empty content body issues causing AWS validation errors.
• First Match Routing: Fixed router logic to ensure first match wins as expected.

⚠️ Breaking Changes

• Sidecar Architecture: The switch to sidecar + UDS model may require configuration updates for existing deployments.
• API Field Changes: Some API fields have been moved or renamed - see migration guide for details. Please review the migration guide for details.
• Timeout Configuration: Deprecated timeout fields require migration to new configuration format.
• Routing to Kubernetes Services: Routing to Kubernetes services is not supported in Envoy AI Gateway v0.2.0. This is a known limitation and will be addressed in a future release.

📖 Upgrade Guidance

For users upgrading from v0.1.x to v0.2.0:

1. Review usage of any deprecated API fields (particularly AIServiceBackend.Timeouts).
2. Update deployment configurations if using custom replica configurations - the replicas field in AIGatewayFilterConfigExternalProcessor is now deprecated due to the new sidecar architecture.
3. Remove routing to Kubernetes services - currently, Envoy AI Gateway does not support routing to Kubernetes services. This is a known limitation and will be addressed in a future release.

📦 Dependencies Versions

Go 1.24.2

Updated to latest Go version for improved performance and security.

Envoy Gateway v1.4

Built on Envoy Gateway for proven data plane capabilities.

Envoy v1.34

Leveraging Envoy Proxy's battle-tested networking capabilities.

Gateway API v1.3

Support for latest Gateway API specifications.

⏩ Patch Releases

v0.2.1 — June 9, 2025

Bug Fix

Quick bug fix release addressing an AWS authentication issue discovered in the v0.2.0 release.

Bug Fixes

• AWS Request Authentication: Fixed the authentication issues with AWS Bedrock integration for larger payloads.

🙏 Acknowledgements

This release represents the collaborative effort of our growing community. Special thanks to contributors from Tetrate, Bloomberg, Google, and our independent contributors who made this release possible through their code contributions, testing, feedback, and community participation.

There are those who engage in conversations, provide feedback, and contribute to the project in other ways than code, and we appreciate them greatly. Ideas, suggestions, and feedback are always welcome.

🔮 What's Next (beyond v0.2)

We're already working on exciting features:

• Google Gemini & Vertex Integration
• Anthropic Integration
• Support for the Gateway API Inference Extension
• Endpoint picker support for Pod routing
• What else do you want to see? Get involved and open an issue and let us know!

Original source

The first official release of Envoy AI Gateway! This milestone includes unified API for LLM providers, token-based rate limiting, traffic management, and Kubernetes native integration.

🎉 The First Official Release!

🎉 The inaugural release of Envoy AI Gateway! This milestone release establishes the foundation for unified AI service management with support for major LLM providers, comprehensive rate limiting, and Kubernetes-native integration.

This foundational release establishes Envoy AI Gateway as a production-ready solution for managing AI services in Kubernetes environments, providing the essential building blocks for enterprise AI infrastructure.

Installation

For installation instructions, see our Getting Started Guide.

✨ Core Features

Unified LLM Provider API

OpenAI Integration

Complete support for OpenAI's GPT models with chat completions and embeddings

AWS Bedrock Support

Native integration with AWS Bedrock for Claude, Titan, and other foundation models

Request/Response Transformation

Automatic translation between different provider APIs for consistent experience

Traffic Management & Rate Limiting

Token-Based Rate Limiting

Intelligent rate limiting based on actual token consumption rather than request count

Backend Load Balancing

Weighted routing and load balancing across multiple AI service backends

Route-Level Controls

Fine-grained traffic management at the route level with flexible matching

Kubernetes Native Integration

Custom Resource Definitions

Native Kubernetes CRDs for AIGateway, AIServiceBackend, and routing configuration

Operator Pattern

Kubernetes operator for automated lifecycle management and configuration

RBAC Integration

Full integration with Kubernetes RBAC for secure access control

API Design

The v0.1.x series establishes the core API design for Envoy AI Gateway:

• Initial API Design: Core CRDs including AIGateway, AIServiceBackend, and AIGatewayRoute
• Gateway API Integration: Built on top of Kubernetes Gateway API specifications
• OpenAPI Schema: Complete OpenAPI v3 schema for all custom resources

📦 Dependencies Versions

Go 1.22

Built with Go 1.22 for optimal performance and security

Envoy Gateway v1.2

Built on Envoy Gateway for proven data plane capabilities

Envoy v1.30

Leveraging Envoy Proxy's battle-tested networking capabilities

Gateway API v1.1

Implementing latest Gateway API standards

⏩ Patch Releases

v0.1.5 — April 3, 2025

Bug Fix · ExtProc

Final patch in the v0.1.x series addressing ExtProc image synchronization.

Bug Fixes

• ExtProc Image Syncing: Fixed external processor image synchronization issues
• Container Registry: Improved container image handling and registry integration

v0.1.4 — March 20, 2025

Bug Fix · AWS

Additional AWS validation and error handling improvements.

Bug Fixes

• AWS Validation: Fixed validation error handling for AWS Bedrock requests
• Error Response Format: Improved error response formatting for better debugging

v0.1.3 — March 14, 2025

Bug Fix · Streaming · Metrics

Streaming improvements and enhanced observability with GenAI metrics.

New Features Observability Enhancements

• GenAI Metrics: Added comprehensive metrics for AI service performance and usage tracking

Bug Fixes

• Chat Completion Streaming: Fixed streaming response handling for chat completions
• Metrics Collection: Resolved issues with metrics collection and reporting

v0.1.2 — March 5, 2025

Bug Fix · Images

Container image and deployment fixes for improved reliability.

Bug Fixes

• Image Tag Fixes: Fixed rotator and ExtProc container image tag synchronization
• Deployment Stability: Improved deployment reliability and image pulling

v0.1.1 — February 28, 2025

Bug Fix · AWS

Quick bug fix release addressing AWS tooling issues discovered in the initial release.

Bug Fixes

• AWS CLI Tools: Fixed authentication issues with AWS Bedrock integration
• Configuration Validation: Improved validation for AWS-specific configuration parameters

🙏 Acknowledgements

The v0.1.0 release represents months of collaborative effort from our founding community. Special thanks to early contributors and adopters who helped shape the initial architecture and provided invaluable feedback during the alpha and beta phases.

This milestone release sets the foundation for Envoy AI Gateway's future growth and establishes our commitment to providing enterprise-grade AI service management capabilities.

🔮 What Came Next

The success of the v0.1.x series paved the way for v0.2.0, which introduced:

• Azure OpenAI Integration
• Sidecar Architecture for Better Performance
• Cross-Backend Failover and Retry Logic
• Enhanced CLI Tools

v0.1 is now complete

The v0.1.x release is now complete. For the latest features and improvements, check out v0.2.

Original source