OpenRouter Release Notes

Today we’re launching Private Endpoints. OpenRouter already puts the public models your team uses behind one API. Now it can serve the deployments only your team has too. If you run a fine-tuned model, a dedicated deployment of a public model, or capacity you’ve reserved with a provider, you can route it through OpenRouter alongside everything else in the catalog. Your team keeps the API, model slug, and routing preferences they use today. Private endpoints are only visible within your organization.

Private Endpoints is an add-on to the Enterprise plan. Ask your account representative to enable it for your organization, or talk to the team.

What you can do with it

Fine-tuned models and reserved capacity used to mean a second integration next to OpenRouter. Now you register the deployment as a private endpoint and call it through the API you already use, with public providers as an optional backup if it goes down or hits its rate limit. Only the organizations you approve can see it, and it never shows up in public model lists, rankings, search, charts or benchmarks.

If your organization has a dedicated Azure deployment of DeepSeek V4 Pro for example, you can register it as a private endpoint for deepseek/deepseek-v4-pro and your apps keep calling that slug. For your organization it moves to the front of the provider list by default, and Model Fallbacks, Provider Selection and the rest of routing work as they do for any provider. OpenRouter reaches it with your Azure key through BYOK, so Azure bills you directly.

Reserved capacity gets this treatment too. Point a private endpoint at it and your organization’s traffic goes there first by default. When it’s rate-limited or unavailable, OpenRouter can send the overflow to public providers, within the provider preferences and BYOK fallback settings you’ve set.

Setting up an endpoint

Once Private Endpoints is enabled, an organization admin registers the deployment from the Private Endpoints page in your account. The deployment has to be hosted by a provider we support for BYOK, since OpenRouter reaches it with your own key. It also has to serve a model that’s already in the OpenRouter catalog, since it inherits that model’s slug and capabilities. You can attach your own pricing so the endpoint shows up accurately in cost reporting, and you declare whether the deployment retains prompts.

Before anything routes to the new endpoint, OpenRouter sends one real request through the provider key saved in one of your workspaces. That request proves the key is accepted, the deployment speaks the OpenAI-compatible format, streaming works, usage comes back for billing, and the model ID matches what you registered. Once it passes, you activate the endpoint and workspaces in your organization can call it by the model slug. Until then it stays hidden and can’t be called.

Managing endpoints from code

If you run more than a handful of endpoints, or you keep your infrastructure in code, you can skip the settings page. The Private Endpoints API covers the whole lifecycle, from registering a draft and running the test checks through activation, pricing updates, pausing, and deletion. It’s authenticated with an organization Management API key, and one create call registers, tests, and activates an endpoint.

curl https://openrouter.ai/api/v1/private-endpoints \
-H "Authorization: Bearer $OPENROUTER_MANAGEMENT_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: gpt-4o-prod" \
-d '{
"model_permaslug": "openai/gpt-4o-2024-08-06",
"provider_slug": "openai",
"base_url": "https://api.openai.com/v1",
"upstream_model_id": "gpt-4o-2024-08-06",
"pricing": { "prompt": "0.0000025", "completion": "0.00001" },
"activate": { "workspace_id": "<workspace-id>" }
}'

If the test checks fail, the draft is kept and comes back with the failed checks, so you can patch it and try again. The Idempotency-Key header makes retries safe (a repeated key returns the endpoint the first request created). The OpenRouter Terraform provider wraps all of this in an openrouter_private_endpoint resource, so terraform apply registers, tests, and activates the endpoint and terraform destroy removes it.

Good to know

• Marking an endpoint private doesn’t make it a Zero Data Retention endpoint. Your provider handles prompts exactly as it did before, and OpenRouter records the data policy you declare when you register the deployment.
• A private endpoint can use In-Region Routing if OpenRouter can work out a region from its address, or if you declare one during setup. Requests to us.openrouter.ai or eu.openrouter.ai then route to your deployment inside that region. The In-Region Routing guide lists the supported providers.
• Your provider key has to be saved under BYOK in each workspace that calls the endpoint, and the test request during setup needs it there too.
• Each organization can have up to 10 private endpoints by default, and endpoints that are still being set up count toward that limit.

Get started

Private Endpoints is an add-on to the Enterprise plan. Ask your account representative to enable it for your organization, or talk to the team.

Once it’s enabled, an organization admin can add your first endpoint from the Private Endpoints page in your account or with a Management API key through the Private Endpoints API.

Read the docs for more info.

Original source

ElevenLabs is now on OpenRouter with nine Text to Speech models and two Speech to Text models.

If your app already talks to a language model through OpenRouter, it can now speak and listen too. Narrate long-form content with direction over tone and delivery with Eleven v4, give your agent faster spoken replies with v4 Turbo, or transcribe conversations with speaker labels with Scribe v2. You don’t need a separate ElevenLabs plan. Just call POST /api/v1/audio/speech and POST /api/v1/audio/transcriptions with your OpenRouter API key.

Every ElevenLabs model is 50% off OpenRouter’s list price for all OpenRouter customers through October 19, 8am PT

Which model to use

Start with three models and reach for the rest when a job needs them:

• Narration, characters, product videos: Eleven v4. Audio tags let the script carry the performance.
• Voice agents and live replies: Eleven v4 Turbo, ElevenLabs built it for real-time use, at half the v4 rate.
• Transcripts, meeting notes, captions: Scribe v2, with speaker labels and word timestamps.

Use Multilingual v2 when long-form audio needs speed control, and Flash v2.5 for high-volume speech of up to 40,000 characters per request. Scribe v2 Medical is fine-tuned for medical terms.

Text to Speech

ElevenLabs prices its models in two tiers. Eleven v4, v3, and Multilingual v2 bill the full per-character rate, and v4 Turbo, v3 Conversational, Flash, and Turbo bill half of it. Current per-character prices are on each model page in the Text to Speech collection.

Speech to Text

Both Scribe models use the same request shape and the same per-second price. Realtime Scribe v2 over WebSocket isn’t part of this launch.

Walkthrough: from a first MP3 to a voice agent

We’ll build a small product tour in three steps. First Eleven v4 narrates it. Then a voice agent answers questions about it, and finally Scribe v2 turns the team’s review meeting into notes that show who said what. If a request doesn’t behave the way you expect, check the tips at the end.

Set your key once:

export OPENROUTER_API_KEY = "your-api-key"

Step 1: Narrate a product tour with Eleven v4

Start with one line of narration:

curl https://openrouter.ai/api/v1/audio/speech \
  --fail-with-body \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elevenlabs/eleven-v4",
    "input": "[cheerfully] Welcome to the tour. [whispering] Let me show you the shortcut first.",
    "voice": "george",
    "response_format": "mp3"
  }' \
  --output intro.mp3

voice takes any of ElevenLabs’ 21 premade voices by name (george here); names aren’t case-sensitive. Note the explicit response_format — see Tips for your first requests for both.

The bracketed words are audio tags. Eleven v4 and v3 read tags such as [whispering], [laughing], [sighs], and [curious] as delivery directions instead of speaking them. Tags count toward your billed characters.

A full tour script is longer than one request allows (10,000 characters on v4, 5,000 on v3), so split it on paragraph boundaries. ElevenLabs-only settings go under provider.options.elevenlabs. Pass the neighbouring text as previous_text and next_text so intonation stays consistent across the joins, and fix a seed so reruns are more repeatable.

Step 2: Answer questions with a voice agent

Now let users ask the tour questions out loud. A voice agent chains three models: Scribe v2 for the question, a chat model for the answer, and Eleven v4 Turbo for the reply. ElevenLabs builds v4 Turbo for real-time use, and it bills at half the v4 rate.

Step 3: Turn the review meeting into speaker-labelled notes

When your team meets to review the tour, Scribe v2 can tell you who said what. Upload the recording as multipart, ask for verbose_json with word timestamps, and set diarize under provider.options.elevenlabs. Every word in the response then carries a speaker. If you know how many people were in the meeting, pass num_speakers to improve the split.

Each entry in words has word, start, end, confidence, speaker (an integer), and speaker_label (speaker_0, speaker_1, and so on). tag_audio_events adds entries such as (laughter) alongside the words. Group consecutive words by speaker to get turns, then send the turns to a chat model for a summary and action items.

Uploads are capped at 25 MB, which covers about 27 minutes of 128 kbps MP3.

For longer recordings, pass a public URL and ElevenLabs downloads the file directly, so the 25 MB cap doesn’t apply.

Very long files can still hit the 180-second upstream request timeout, so split recordings that run long.

Tips for your first requests

1. Pick a voice by name or ID. For voice, pass any of ElevenLabs’ 21 premade voices by name: george, sarah, adam, alice, bella, brian, charlie, daniel, jessica, roger, will and more (full list on each model page). Names aren’t case-sensitive. To use any other voice, pass its ElevenLabs voice ID as voice — copy it from the ElevenLabs voice library. Voices you clone or design live in your own ElevenLabs account, so use those with BYOK.
2. Ask for mp3 when you want a file you can play. If you leave out response_format, our speech endpoint returns raw PCM (16-bit little-endian mono at 24 kHz). That suits an audio pipeline, but most players can’t open it. Set "response_format": "mp3" to get a playable 44.1 kHz, 128 kbps file.
3. Use Multilingual v2 or Flash for speed control. Multilingual v2 and Flash take speed from 0.7 to 1.2. Eleven v4 and v4 Turbo have no speed control and reject a non-default speed, so leave it out on those models and change the pacing with audio tags instead.
4. Split long scripts. Each model has a per-request character limit (see the model table). Audio tags count toward the limit and toward your billed characters. Split on paragraph boundaries as in step 1, and pass previous_text and next_text so the joins sound natural.
5. ElevenLabs-specific settings go under provider.options.elevenlabs. Text to Speech accepts seed, previous_text, next_text, language_code, apply_text_normalization, apply_language_text_normalization, pronunciation_dictionary_locators, voice_settings (stability, similarity_boost, style, use_speaker_boost, speed), and output_format (an ElevenLabs format name such as mp3_22050_32 or pcm_16000; it must match your response_format codec). Speech to Text accepts diarize, num_speakers, diarization_threshold, tag_audio_events, no_verbatim, and seed. num_speakers and diarization_threshold need diarize: true, and you can set one or the other, not both. Use the field names exactly as listed. Keys that aren’t on these lists are ignored; a listed key with a bad value returns a 400.
6. Scribe returns three-letter language codes. You can send a two-letter language hint (en). The detected language comes back as an ISO 639-3 code (eng).

Getting started

1. Create an OpenRouter API key.
2. Make your first request with the curl example in Step 1 above.
3. For every parameter, see the Text to Speech and Speech to Text guides.

Text to Speech is billed per character of input, counted as Unicode code points, so an emoji or a CJK character counts once. Audio tags count as characters. Scribe is billed per second of audio, and the usage.cost field on every transcription response tells you what that request cost. We pass through ElevenLabs pricing without markup; see each model page for the current rate.

Already have an ElevenLabs account?

Add your key with BYOK to use your own cloned voices, pronunciation dictionaries, and higher-quality formats such as mp3_44100_192.

Create an OpenRouter API key, or try a model first in the Playground. The Text to Speech and Speech to Text guides have the full request reference.

Original source

Say you’re building an agent that answers questions about a CSV a customer uploaded. The model needs to run Python to get the answer. Where does that code run?

One option is to run it yourself. That means a sandbox platform such as E2B or Modal, or your own container on Docker, plus the work of keeping it away from your database and the internet, capping its runtime, and patching the image.

A server-side code execution tool is the other option. You add the tool to your API request, the model decides when it needs to run something, and the provider runs the commands in its own sandbox and returns the output to the model inside the same request. A sandbox here means an isolated Linux container with its own filesystem, a time limit, and no network access by default.

This article covers four providers that offer server-side code execution today, what each sandbox can and can’t do, what it costs in latency and money, and the jobs that still need a sandbox you operate yourself.

Tl;dr

• A server-side code execution tool runs the model’s commands in the provider’s sandbox during your API request. You don’t provision, patch, or secure a container.
• OpenAI, Anthropic, and Google each run code for their own models. Our openrouter:shell tool runs commands for any model on the Responses and Messages APIs, and our openrouter:bash tool does the same on the Messages API only. Both tools are in beta.
• Our sandbox is an isolated container scoped to your account and workspace, with outbound network access off by default and per-command limits on runtime and output size. Sandbox time is billed at $0.0001 per second with a 30-second minimum for a new or sleeping container.
• A sandbox platform you operate yourself is still the right choice for a custom base image, GPU work, or a session that runs for hours.

What server-side code execution means

A model never runs anything itself. When it calls a tool, it emits a request naming the tool and the arguments, and something has to carry that out. With a client-side tool, that something is your application code or the agent framework you build on. Your application receives the call, runs it, and sends the result back in a follow-up request. With a server-side tool, the provider runs the call on its own infrastructure and returns the result to the model in the same request, so your application has no handler for it. Server-side code execution is the second kind.

You may already use tools that work this way. Web search lets a model look something up on the live web, and web fetch lets it read the contents of a URL. In both cases you add one entry to the request and the provider does the rest. Code execution applies the same pattern to running commands.

How a tool call becomes a command that runs

These steps apply to any server-side code execution tool. Where a field name appears, it’s the one our shell tool uses.

1. You include the tool in the request’s tools array.
2. The model decides it needs to run something and emits a call carrying one or more shell commands.
3. The provider runs those commands in order inside a sandboxed container.
4. Each command’s standard output, standard error, and outcome go back to the model. The outcome is either an exit code or a timeout.
5. The model reads the results and either answers you or runs more commands in the same request.

Steps two through five repeat until the model answers. The model runs something, reads the output, decides whether it needs another command, and goes again. That loop is where a code execution tool earns its place, because the model can check its work against real output instead of guessing.

We cap that loop. The max_tool_calls field sets how many server-tool steps one request may take. Our server tools reference puts both the default and the maximum at 30.

How a hosted sandbox differs from one you run yourself

Running your own sandbox means owning the parts the provider would otherwise own. You pick a base image, provision the compute, wire in an SDK to start runs and read output, and manage each run’s lifecycle. You also own the security boundary.

A hosted tool trades that control for one entry in a JSON array. You don’t size the container, patch it, or operate it. We built openrouter:shell for the case where the work is a handful of short commands per request.

Who offers hosted code execution today

This article covers OpenAI, Anthropic, Google, and OpenRouter. Agent SDKs and dedicated sandbox platforms are separate categories, and both come later in the article. What separates the four providers is which models each one runs code for.

OpenAI runs a hosted shell for OpenAI models

OpenAI’s shell tool runs commands in a container OpenAI manages, on the Responses API. OpenAI documents the hosted runtime as Debian 12 with a default working directory of /mnt/data. Commands run without sudo, and interactive TTY sessions aren’t supported. Preinstalled languages listed in the documentation include Python 3.11, Node.js 22.16, Java 17, PHP 8.2, Ruby 3.1, and Go 1.23.

Hosted containers have no outbound network access by default. To enable it, an organization admin configures an allowlist in the OpenAI dashboard and you set network_policy on the container environment in the request. A container can be reused across requests by passing its id in a container_reference environment, and its expiry is set when the container is created. OpenAI also has a separate code interpreter tool for Python.

Anthropic runs Python and Bash for Claude models

Anthropic’s code execution tool runs Python and Bash in a sandbox Anthropic manages, on the Messages API. The documented environment is a Linux x86_64 container with Python 3.11, 5 GiB of RAM, 5 GiB of workspace storage, and one CPU. Internet access is disabled and no outbound connections are permitted, so Claude works with the preinstalled libraries and can’t install a package during a run.

Three tool versions exist, and every supported model accepts all three. code_execution_20250825 supports Bash commands and file operations. code_execution_20260120 adds Python interpreter state that persists between requests, which depends on Anthropic’s programmatic tool calling and isn’t available on Claude Haiku 4.5. Containers expire 30 days after creation. After about 5 minutes of inactivity a container is checkpointed, and a request with its id inside the 30-day window restores it.

Google runs Python for Gemini models

Google’s code execution tool runs Python in a sandbox Google manages, enabled with a code_execution entry in the request’s tools. The documentation states that the model can only generate and execute Python, that the code environment has a maximum runtime of 30 seconds, and that you can’t install your own libraries. Google publishes the list of libraries the environment includes.

We run a hosted shell for any model

The three tools above each work with one company’s models. Ours works with any model on the Responses and Messages APIs, because we run the sandbox at the routing layer rather than inside one model provider.

We ship two code execution tools. openrouter:shell mirrors the shape of OpenAI’s hosted shell tool and works on both the Responses API and the Messages API. openrouter:bash mirrors the shape of Anthropic’s bash tool and works on the Messages API only.

Both tools are in beta, so the API may change. Sandboxed execution runs on the global openrouter.ai endpoint only. The in-region endpoints don’t offer the shell tool, and Chat Completions rejects both tools with a 400 that names the APIs that do support them.

Set engine to openrouter on either tool and the commands run in our sandbox. The default engine is auto. For openrouter:shell, auto keeps a provider’s native hosted shell where one exists and routes to our sandbox otherwise. For openrouter:bash, auto returns the tool call to your application to run client-side, and nothing executes on our servers.

Run a command in our sandbox

Here is a complete request that runs two commands and reads the results back.

[Code example omitted]

We ran this request on September 22, 2026. The model sent both commands in one shell call, and each command came back with its own outcome. The full os-release output runs to several lines and is trimmed to its first two here.

[Output example omitted]

The sandbox reported Ubuntu 22.04.5 LTS with Python 3.11.14 on that date. The runtime image can change, so read the version from the container rather than hard-coding it. The server tools reference covers the remaining parameters.

Agent SDKs that wrap a sandbox for you

If you build on an agent SDK rather than calling an API directly, some SDKs wrap one of these tools.

The OpenAI Agents SDK ships CodeInterpreterTool, which runs code in OpenAI’s sandbox, and ShellTool, which runs either in your local runtime or in an OpenAI-hosted container depending on how you configure its environment. Check which mode you configured before assuming a command ran remotely. On our side, openrouter:shell is an entry in the tools array like any other, so it goes into an OpenRouter Agent SDK loop the same way it goes into a raw request.

What still needs a sandbox of your own

A hosted tool fits short, bounded work such as running a script, transforming a file, or checking a result. Anything that needs a specific base image or a GPU falls outside all four hosted tools and needs a sandbox platform you run yourself.

Comparison

Every hosted-tool cell comes from the vendor’s own documentation linked above, except the OpenRouter runtime cell, which is what the sandbox reported when we ran the request above. The self-managed column describes a sandbox you run yourself rather than any one platform.

[Comparison table omitted]

What our sandbox enforces

The point of a sandbox is that you don’t have to trust the model. A command you never intended can’t reach the network or anyone else’s container, and it stops at a hard limit on how long it runs and how much it prints. Everything in this section describes our sandbox. The table above shows where the other three differ.

The limits that apply to every command

We run each command in an isolated container, separate from the infrastructure serving your request and from your machine, and scoped to your account and workspace. You set your own ceilings with timeout_ms for how long a command may run and max_output_length for how much it may print.

timeout_ms defaults to 120,000 ms and can’t exceed 300,000 ms.

max_output_length defaults to 16,384 characters per stream and can’t exceed 65,536. A shell call with more than 100 commands is rejected.

Outbound network access is off unless you turn it on. We checked this on September 22, 2026 by sending a request with no network_policy and asking the model to fetch https://example.com with curl, printing only the HTTP status code and giving up after 5 seconds. The command printed 000, which is what curl prints when no response arrives, and exited with code 28, a curl timeout.

[Output example omitted]

To open the network, set a network_policy allowlist of up to 50 hostnames or glob patterns. Only ports 80 and 443 are reachable, and the policy is fixed when the container starts.

pip install needs both pypi.org and files.pythonhosted.org in the allowlist.

Prompt injection and what the sandbox limits

Giving a model a shell exposes you to prompt injection. If your agent reads a web page, a support ticket, or a file someone uploaded, an attacker can hide instructions in that text telling the model to ignore your prompt and run something else.

The limits above apply whether the model is following your prompt or an attacker’s. A command run under an injected instruction can’t reach any host outside the network_policy you configured, and has no network access at all when you leave the policy off. It can’t reach another tenant’s container, and it stops at the same timeout. An allowlist widens what an injected command can reach, and allowed_domains: ["*"] permits unrestricted egress, so keep the allowlist to the hosts the job needs.

You can also detect an attempt that arrives in the request itself. Prompt-injection detection in a workspace guardrail checks the user-supplied message content of each incoming request against regex patterns for common injection techniques, before we forward the request to the model. It doesn’t inspect what a server tool fetches after that point, so a page, file, or command output the model reads through a tool needs a control of your own, such as reviewing the sandbox output your application receives. A match does one of three things, depending on the action you configure.

• Flag records the detection and forwards the request unchanged.
• Redact replaces the matched span with [PROMPT_INJECTION] and forwards the sanitized request.
• Block rejects the request with a 403 before it reaches the model.

Where more than one guardrail applies, the strictest action wins, in the order block, redact, flag. The detection isn’t exhaustive and can produce false positives, so measure the match rate against your own traffic in flag mode before you enforce redact or block. You can report a false positive from the Logs page.

What a hosted sandbox costs you in seconds and dollars

A hosted sandbox adds seconds to a request and leaves you with no infrastructure to run. It also gives you a container you can return to.

Files survive between requests

Commands run in /workspace/home, and we save the changed files under that directory after every command. Send a stable session_id, or set a container id in the tool’s environment configuration, and every request with that id reaches the same container and the same files. A session_id must use only letters, digits, _, and -. An id with any other character is ignored, and we pick the container as if you sent no session_id, which means the most recent container_id in the replayed conversation if there is one, and otherwise a new container for that request. When a session_id is longer than 20 characters, we use only the last 20, so two long ids that end the same way share a container. A container_reference id can be 1 to 40 characters from the same set and isn’t truncated, so use it when you need the id to be exact. We confirmed this on September 22, 2026 by writing a file in one request and reading it back in a second request that shared the same session_id.

A container sleeps after 5 minutes idle, and the idle time isn’t configurable. Sleep doesn’t delete the files. When a request with the same id arrives later, a new sandbox starts and loads the saved files first. Open processes, environment variables, and installed system state aren’t restored, so treat a woken container as a fresh machine that has your files on it. Saved files are kept for 30 days after the container was last used. To pull an artifact out, GET /api/v1/containers/{container_id}/files lists what a container produced, and the promote endpoint copies a file into your workspace documents, where it doesn’t expire.

What you can review afterward

Each shell tool result in the response carries the commands the model ran and each command’s stdout, stderr, and outcome, so your application can log them the same way it logs the rest of the response. Input & Output Logging stores your prompts and completions on OpenRouter for review on the Logs page, and guardrail detections appear there too. For production monitoring, Broadcast streams traces to an external observability platform as requests complete.

If you route in-region, the shell tool isn’t available and Input & Output Logging is skipped, even when it’s enabled. Broadcast supports in-region routing, and each destination is configured with the data regions it receives traces from.

What the round trip costs you

A request that runs a sandboxed command takes longer than the same request without one. In a set of single requests we sent on September 22, 2026, a request with no shell tool returned in about 2 seconds, and requests that made one shell call returned in 8 to 21 seconds depending on the model. Those are single samples from one session, not a benchmark. Budget for several seconds of overhead per shell call.

Sandbox time is billed at $0.0001 per second. The clock starts when a request first runs a sandbox command and stops when the response completes. A request that starts a new or sleeping container is billed a minimum of 30 seconds, and later requests that reuse the same warm container pay only their metered time. The request above started a new container, and its usage object reported a server_tool_cost of 0.003, which is the 30-second minimum. A container that is idle between requests isn’t billed.

Change the model without touching your tool code

Swap the model and your tool definition stays the same.

We sent one request body six times on September 22, 2026, changing only the model field, and asked each model to run python3 -c "print(sum(range(1, 101)))" in the sandbox. Each model made one shell call and returned 5050.

All six ran in the same sandbox with the same tool definition, whatever their own provider offers natively, because the sandbox belongs to us and not to the model’s provider. Test the model you plan to use before you build on it, because tool-calling reliability differs between models. Our tool calling guide covers how server tools and your own function tools share one tools array.

When you want a sandbox platform of your own

Choose a dedicated sandbox platform when you need something a hosted tool doesn’t give you.

Modal documents sandboxes built from custom images with a configurable lifetime of up to 24 hours and GPU resources.

Daytona documents sandboxes created from a public container image, including GPU sandboxes.

E2B documents sandboxes that run for up to 24 hours on its Pro plan and 1 hour on its base plan, with pause and resume for longer workloads.

Conclusion

For short, bounded commands inside a model request, use a hosted tool. You add one entry to the tools array and get an isolated container with outbound network access off, and you pay for inference plus the seconds the sandbox runs. Budget several seconds of overhead per shell call and reuse a container where you can.

Move to a sandbox platform you operate when a job needs a custom base image, a GPU, a session that runs for hours, or ownership of the security boundary itself. In either case, check the provider’s current documentation before you commit, because our two tools are in beta and the other three providers’ tools change too.

Frequently asked questions

Is there a hosted sandboxed shell tool that models can call directly during a request?

Yes. Our openrouter:shell server tool gives a model a sandboxed Linux shell that runs on our infrastructure during the request, on both the Responses API and the Messages API. Set engine to openrouter and the commands run in an isolated container, with each command’s stdout, stderr, and exit or timeout outcome returned to the model. OpenAI, Anthropic, and Google each offer a hosted code execution tool for their own models.

Can I give a model a sandboxed shell it can run commands in?

Yes. Add {"type": "openrouter:shell", "parameters": {"engine": "openrouter"}} to the tools array of a Responses or Messages API request. The model can then emit shell calls, and we run the commands in an isolated container and return each command’s output to the model. The container has no outbound network access unless you configure a network_policy allowlist.

What SDKs or platforms provide server-side code execution out of the box?

Our openrouter:shell server tool runs commands for any model on the Responses and Messages APIs, and our openrouter:bash server tool does the same on the Messages API only. OpenAI, Anthropic, and Google each run code for their own models through the Responses API shell tool, the code execution tool, and the Gemini API code execution tool. The OpenAI Agents SDK wraps OpenAI’s hosted tools as CodeInterpreterTool and ShellTool. E2B, Modal, and Daytona are sandbox platforms you integrate and operate yourself rather than tools a provider runs inside the API call.

Which sandbox is best for AI agents?

It depends on how long the work runs and how much control you need over the runtime. For short commands inside a request, a hosted tool such as openrouter:shell means you run no infrastructure. For a custom base image, GPU access, or a session that runs for hours, a sandbox platform you operate, such as Modal or Daytona, gives you those controls.

How do you sandbox an AI agent?

You run the agent’s commands in an environment isolated from your own systems and limit what that environment can reach. With a hosted tool the provider does this. On OpenRouter, containers are isolated from our infrastructure and from your machine, scoped to your account and workspace, outbound network access is off by default, each command is bounded by timeout_ms, and output is capped by max_output_length. Workspace guardrails add prompt-injection detection in front of the model.

What is a sandboxed AI tool?

It’s a tool whose side effects are confined to an isolated environment rather than your production systems. For code execution, the model’s commands run in a container with its own filesystem, restricted network access, and a time limit, and only the command output travels back to the model. Our shell and bash server tools work this way.

References

• Server tools reference
• Shell server tool
• Bash server tool
• Containers
• Guardrails and prompt-injection detection
• Input & Output Logging
• Tool calling guide
• Responses API tool calling
• Model catalog

Original source

Today we’re introducing a new Model Router Benchmarks page for comparing performance across the growing number of router options. Every router was run through a set of benchmarks from varied domains to score them on quality, speed, and cost.

Individual models representing both best-in-class cost efficiency and intelligence are shown alongside routers as a baseline for comparison. Keep an eye on this page over time, as we’ll continually add new routers and update scores as the routers improve.

OpenRouter introduced two of the first model routers with our Auto Router and Free Models Router. They were created so you’d have a dependable option that stays up to date with the latest models, at your preferred level of cost. Since then we’ve continually improved them, including by using the wisdom of the market to inform routing decisions.

In recent months, many routers launched that use the differing strengths across the model landscape to optimize performance.

Cost differences: For example, comparing DeepSeek v4 Flash with GPT-6 Astra, GPT-6 Astra’s average price paid per token is over 48x higher, and it’s 21x more expensive to run an average 10 to 49 turn session in Codex. Routers can switch parts of a task to a lower-cost model to save on cost.

Task diversity: For example, the model that performs best on legal research may not be the same as the model that’s best for coding. Routers can switch between them to leverage each model’s strengths.

Session stage: Different parts of an agentic session may require different levels of intelligence. Routers can switch between models or reasoning efforts to adjust for parts of a session that require complex planning or mundane execution.

In practice, these techniques each have challenges that can sometimes lead to worse performance than using a single model directly:

• Switching between models means requests get more expensive while the input cache is rebuilt.
• Routers often don’t have enough information to assess task complexity by simply looking at the prompt.
• Heuristic signals for session stage may not agree with the model’s next step.
• The more processing you run on the routing decisions, the more latency is introduced.

We offer these Model Router Benchmarks to show which routers overcome these inherent challenges so you can decide when (and if) it’s the right time to put them to use. They’re designed to help you find the router and model combination that matches the performance characteristics you need.

Types of routers

The term “router” is used interchangeably to mean many different things. Let’s break it down.

Provider routing: Models are usually served by several inference providers, so a provider router picks the one that fits your preferences for price, speed, uptime, and data policy, then falls back to another if it fails. This is what OpenRouter has done since day one.

Model routing: Instead of choosing a single model directly, you can send a request to a router such as the Auto Router, and it decides which model should answer. These behave just like models, but may use one or more models to respond to your request.

The model routers we benchmarked take a few different approaches:

Routers that execute a blend of models: Unbiased’s Pareto and Sakana’s Fugu are inference providers that use a blend of models, sometimes with frontier escalation. They don’t disclose the models used or how often switching happens, and they bill at standardized per-token rates.

Routers that select a model: Auto Router and Jev Router select a model for each turn. The model selection is transparent, and you’re billed at the standard rate for whichever model was selected.

Routers that swap between a pre-defined set of models: NVIDIA’s Switchyard pairs a cheaper model with a stronger one and switches between them as an agent works through a task. Several pairs were benchmarked to show which models work best together, but you can run it with any pairing you want.

There are a couple of other styles, such as Pareto Code or the -latest model slugs, that alias to a single model. Pareto Code picks the most cost-optimal model for your desired level of intelligence, and the -latest slugs point to the most recent model within a family. Fusion is another style that runs the same request across a council of models and synthesizes the best answer. These styles weren’t benchmarked because they serve specialized purposes that warrant different comparison methods.

Router Index blends quality, speed, and cost

Benchmarks reveal performance across several dimensions, making it hard to tell which router performed best. The Router Index synthesizes quality, speed, and cost into a 0 to 10 score for each benchmark. By default, it weights quality (benchmark score) at 60%, time per task at 20%, and cost at 20%. Your priorities may differ, so you can drag a slider on the page to adjust the index weights.

Try any of these routers by simply swapping them in for your model

As with any benchmark, our Model Router Benchmarks are only representative of general tasks rather than your own work. All of the routers listed are available on OpenRouter. Swap them in for your model in any app, harness, or through our API.

Original source

Dealing with a leaked key is stressful

The fewer old keys you have, the fewer can leak. That’s why we built Security Center. It’s one place to see every key across your workspaces, spot the risky ones, and disable, archive, or cap hundreds at once.

It’s available on all OpenRouter plan types. Try it under Settings > Security.

1,000+ keys across 85 employees

The obvious defense against leaked keys is to have fewer of them. So we audited our own OpenRouter org, and because we build OpenRouter on OpenRouter, there was a lot to audit.

We found over 1,000 active keys for 85 employees. Many had no spend limit and 168 hadn’t been used in months, belonging to employees who had completely forgotten about them.

If a team our size ended up with this much key sprawl, you probably have some too.

What the Security Center shows you

The Security Center has three tabs: Overview, Key safety, and IP allowlist.

The Overview counts your keys in four groups: no spend limit, never expires, unused or idle for 90+ days, and safe to remove. Below the counts, it lists recommendations for what to fix first:

• Remove unused keys. Ask each owner to confirm, then archive.
• Set a spend limit. A limit caps what a leaked key can spend.
• Review keys without expiration. Replace them with expiring keys, or remove the ones no one needs.
• Turn on key spend alerts. Get an email when a key crosses a share of its spend limit.
• Enforce a maximum key lifetime. Requests from keys that never expire, or outlive the limit, get rejected (existing keys included).

If nothing needs attention, the page says so.

Key safety: every key in one list

Key safety lists every key that can still make requests, across every workspace on your account. Until now, keys lived on each workspace’s own page. The list includes MCP and management keys, with owner, last use, spend limit, and expiration side by side. Admins see every key; members see the keys they created.

Each key gets a risk score based on what a leaked copy could do, which comes down to its spend limit and expiration.

Usage sets a separate status label. A key with no recorded usage, or idle for six months with under $1 of lifetime spend, is marked ‘Safe’ to remove. Recent or meaningful usage is ‘In use’. Keys in between get labeled under ‘Review’ meaning they’ve been used, but not much and not lately. For example, $3 of lifetime spend and no requests in five months. Someone probably wired that key into something once, so check who depends on it before you archive.

Clean up in bulk

You can also take bulk actions. Filter by risk or idle time, then select up to 500 keys and disable, archive, or add a spend limit to all of them. Disabling is reversible, archiving isn’t, and adding a limit never overwrites one someone already set.

Admins can also copy a CSV of any recommendation’s keys and owners to hand to a script or an agent.

The Security Center works from key metadata, spend, limits, and expiry. It never reads your prompts or completions.

Lock keys to your own network

The IP allowlist, which used to live in privacy settings, now has its own tab. It controls where a key works. Add the addresses your servers, office network, and CI runners call from, and OpenRouter rejects any request from an address outside that list with a 403 Forbidden.

The list applies to every key in your account or organization, including the ones you created years ago, and changes take effect immediately. Entries can be a single IPv4 or IPv6 address or a CIDR range, each with a label so you remember what it covers.

The IP allowlist is available to organization admins on Enterprise plans.

How we contain our key risk

Since our audit we’ve implemented some best practices to keep our keys in check:

• Clean up on a schedule. Start with Safe to remove keys, confirm with their owners, then archive.
• Check keys when someone leaves. Filter by owner and archive or auto-revoke (via SCIM) the keys no one took over.
• Give every key a limit and an expiration. If a leak does happen, setting a cap and expiration contains the blast radius.
• Pin production keys to known addresses. If you’re on a Pro or Enterprise plan, put your servers and CI runners on the IP allowlist.
• Rotate keys that stay in use. The key rotation guide walks through the order.

Getting started

The Security Center is available on all plan types. You can find it under Settings > Security.

Check out the docs to learn more.

Original source

Qwen 3.8

Qwen 3.8 is Alibaba’s model generation released in August 2026. Four models share the name, and they differ in ways that matter before you pick one. You can download Qwen3.8 27B under Apache 2.0 and Qwen3.8 2.4T A95B under the Qwen3.8-Max License. Qwen3.8 Flash has no repository of its own, but Qwen3.8-Flash-Next, the model it is based on, is downloadable under the Qwen Community License 1.0. Qwen3.8 Max has no public weights.

Alibaba announced Qwen3.8-Max as the hosted model of the generation. The QwenLM/Qwen3.8 release notes record the Qwen3.8-2.4T-A95B weights arriving on Hugging Face on 12 August 2026 and the 27B weights on 14 August 2026. The 2.4T model card describes this as the first open release of a Qwen-Max-class model.

The model you can download is not the same as the model you can call. Alibaba’s model card describes Qwen3.8-Max as the hosted version of Qwen3.8-2.4T-A95B with added features, including vision input, non-thinking mode, a 1,000,000-token default context, and built-in tools. The same relationship holds for Flash. Alibaba describes Qwen3.8-Flash as the hosted version of Qwen3.8-Flash-Next with a 1,000,000-token default context and built-in tools.

We route all four. This post covers what each model is, which ones you can download, what the licenses require, and what each one costs to run.

The four Qwen 3.8 models

Prices are US dollars per million tokens, prompt then completion, from our model catalog on 11 September 2026. The 27B price is a range because 14 providers serve it at different rates. Six of the seven 2.4T providers charge $2 / $6, and Venice charges $2.50 / $7.50.

Model Weights License Input Context, native / largest hosted Price qwen/qwen3.8-max-0902 API only Proprietary Text, image, video Not published / 1,000,000 $2 / $6 qwen/qwen3.8-2.4t-a95b Hugging Face Qwen3.8-Max License Text 262,144 / 1,048,576 $2 / $6 qwen/qwen3.8-27b Hugging Face Apache 2.0 Text, image, video 262,144 / 1,000,000 $0.15 to $0.45 / $2.00 to $3.20 qwen/qwen3.8-flash Hugging Face, as Qwen3.8-Flash-Next Qwen Community License 1.0, on Qwen3.8-Flash-Next Text, image, video 262,144 on Qwen3.8-Flash-Next / 1,000,000 $0.15 / $0.47

The model ID qwen/qwen3.8-max is an alias. It currently resolves to qwen/qwen3.8-max-0902, the snapshot dated 2 September 2026. Use the dated ID if you want the request to keep hitting the same snapshot after Alibaba publishes a new one.

The context column shows two numbers because they measure different things. The first is the window the model was trained on, which Alibaba publishes for the open-weight models and does not publish for Max. The second is the largest window any provider in our catalog offers, and it is larger because the window can be extended past the trained length. The Qwen3.8-27B model card describes extending the 27B to 1,000,000 tokens with YaRN, a RoPE scaling technique, and notes that static YaRN can reduce performance on shorter inputs. Whether a provider extends the window, and how, is that provider’s decision, so the ceiling you get depends on where we route your request.

If you use prompt caching, the cache rates differ by model and provider. Max charges $0.25 per million tokens to read from the cache and $2.50 to write to it. Flash charges $0.016 to read and, on Alibaba, $0.20 to write. The 2.4T charges $0.25 to read on most providers and lists a write rate only on Alibaba, at $2.50. The 27B cache read rate ranges from $0.032 to $0.18 across the providers that offer it, and only Alibaba lists a write rate, at $0.53.

Is Qwen 3.8 open source?

Partly, and the answer depends on which model you mean.

You cannot download Max. It has no Hugging Face repository. Max runs on Alibaba’s servers and you reach it through an API.

You can download the 2.4T, and its license adds conditions at scale. Qwen/Qwen3.8-2.4T-A95B is the instruction-tuned model with 2.4 trillion total parameters and 95 billion active per token. It does not use Apache 2.0. Qwen wrote the Qwen3.8-Max License for it. The Hugging Face metadata records the license as "other" with the name "qwen3.8-max".

The license grants the rights to use, copy, modify, distribute, sublicense, sell, deploy, host, fine-tune, and create derivative works, subject to two conditions:

1. Attribution: If a commercial product or service built on the model has more than 100,000,000 monthly active users or more than US$20,000,000 in monthly revenue, the model name must be prominently displayed in its user interface.
2. Separate license for large model-as-a-service businesses: If you or your affiliates run a Model as a Service or AI Work Assistant business, and your aggregate revenue exceeds US$50,000,000 in any consecutive twelve months, you need a separate license from Qwen before any commercial use. This condition does not apply to internal use that does not make the model, its outputs, or its capabilities available to a third party.

The license defines Model as a Service as giving a third party access to model inference or fine-tuning in a way that lets them control the inputs, parameters, or training data. It excludes relaying requests to models hosted by others. It defines AI Work Assistant as an independent product primarily designed for AI-assisted coding or office productivity.

You can download the 27B under Apache 2.0. Qwen/Qwen3.8-27B records apache-2.0 in its Hugging Face metadata. Apache 2.0 is on the Open Source Initiative approved list and carries no revenue or user-count conditions.

Flash has open weights under a different name and a third license. There is no repository for Qwen3.8 Flash. There is one for Qwen3.8-Flash-Next, which Alibaba describes as an experimental preview of the architecture that will underpin Qwen4. Its license is the Qwen Community License 1.0. It has the same attribution condition as the Qwen3.8-Max License, and it requires a separate license from Qwen for any Model as a Service or AI Work Assistant business before commercial use, with no revenue threshold.

Two of the three open-weight models therefore ship under licenses that Qwen wrote and that are not OSI-approved. Open weights means you can download and inspect the checkpoint. It does not mean the license is open source.

Qwen 3 is an earlier generation

Qwen 3 is not the version before Qwen 3.8. Qwen 3 was released in April 2025, and Qwen 3.5, Qwen 3.6, and Qwen 3.7 shipped between the two. Qwen 3.8 follows Qwen 3.7. A license term that applied to a Qwen 3 model does not tell you anything about the four models in this post.

Can you run Qwen 3.8 locally?

You can run the 27B on your own hardware. The 2.4T requires multi-GPU infrastructure.

The 2.4T is a mixture-of-experts model. Only 95 billion parameters are active on any one token, but all 2.4 trillion parameters have to be loaded, because the model selects a different set of experts for each token. The number your hardware has to hold is 2.4 trillion. The README in the QwenLM/Qwen3.8 repository shows serving commands for the 27B only.

The 27B is a dense model with 27 billion parameters. At bf16, two bytes per parameter, the weights occupy about 54 GB, and about half that at fp8. Alibaba’s serving examples run it across four GPUs at the full 262,144-token native window using SGLang, vLLM, or TokenSpeed.

Serving it at 1,000,000 tokens requires the YaRN override described in the model card, and the model card advises changing the RoPE parameters only when you need the longer context.

The open weights of the large model only read text. The hosted Max accepts images and video, and so does the open-weight 27B. The open-weight 2.4T does not. When we sent an image to qwen/qwen3.8-2.4t-a95b on 11 September 2026, the request failed with HTTP 404 and the message "No endpoints found that support image input". The same request to qwen/qwen3.8-27b and qwen/qwen3.8-flash succeeded.

On text, the open weights score close to Max.

These scores come from Artificial Analysis, an independent benchmarking organization, and we publish them on each model’s page. We retrieved them on 11 September 2026. Artificial Analysis measures a dated snapshot of each model, so the Max row is the 2 September 2026 snapshot and the 27B row was measured at xhigh reasoning effort. The scores will move if Artificial Analysis remeasures a model or if you run the model at a different reasoning setting.

The 2.4T is 0.3 points behind Max on the Intelligence Index, 0.1 ahead on the Coding Index, and 0.8 ahead on the Agentic Index. The 27B is behind Max by 6.4, 3.7, and 3.1 points respectively.

If you want Max-class weights to self-host a workload that reads screenshots, the 27B is the model that reads images.

Choosing a model

Use these facts to narrow the choice.

• If you need image or video input and want to hold the weights, the 27B is the only open-weight Qwen 3.8 model in our catalog that accepts them. It is Apache 2.0.
• If you need image or video input and want the highest Artificial Analysis scores in the family, Max scores highest on the Intelligence Index.
• If you want the lowest per-token price, Flash is $0.15 per million prompt tokens and $0.47 per million completion tokens, and it accepts images and video. You cannot download Qwen3.8 Flash itself, only Qwen3.8-Flash-Next.
• If you work with text and want to hold the weights, the 2.4T is the open-weight variant of Max. Check the license thresholds above against your own numbers.
• If you work with text and do not want to host anything, Max and the 2.4T cost the same through us on six of the 2.4T’s seven providers, so the difference is whether you need image input.

How to call the Qwen 3.8 models

Change the model ID and the rest of the request stays the same. This example uses the TypeScript SDK.

To send an image, pass an array of content parts instead of a string.

The same request to qwen/qwen3.8-2.4t-a95b fails, because none of its providers accept image input.

What affects the cost

Two things change your bill beyond the listed rates. The models reason before they answer, and you pay for the reasoning tokens. The provider we route you to sets the price.

Reasoning is on by default

All four models reason before answering by default. Reasoning tokens are billed at the completion rate, which is the higher of the two rates. See reasoning tokens for how the reasoning parameter works.

The share of your output that is reasoning can be large. We asked the 27B through Alibaba, three times, to explain in two sentences why fp8 quantization changes a model’s output. Reasoning tokens were 74% of the completion tokens on each run, at 128 of 174, 171 of 231, and 181 of 243. On an image description request to the same model, 87 of 121 completion tokens were reasoning.

Whether you can turn reasoning off depends on the model and the endpoint. On 11 September 2026 we sent reasoning: { effort: 'none' } to each model.

Model Result of effort: 'none' qwen/qwen3.8-max-0902 HTTP 400, Reasoning is mandatory for this endpoint and cannot be disabled. qwen/qwen3.8-2.4t-a95b HTTP 400, same message qwen/qwen3.8-27b HTTP 200, zero reasoning tokens qwen/qwen3.8-flash HTTP 200, zero reasoning tokens

Alibaba’s model card lists non-thinking mode as a feature of the hosted Max, but the Alibaba endpoint we route to rejects the request. The 27B result covers the routes we tested, which were Alibaba and Chutes. The Flash result covers Makora. Other providers can behave differently, so check the response usage field when you rely on reasoning being off.

To disable reasoning on the 27B, add one field.

We ran the fp8 question three times with reasoning on and three times with it off, both through Alibaba. With reasoning on, the runs averaged 216 completion tokens and $0.00058 per call. With it off, they averaged 61 completion tokens and $0.00017 per call. That is roughly 70% less, whether you count tokens or dollars.

Max and the 2.4T still accept the other effort values. Requests with effort set to max, high, and minimal returned HTTP 200 on all four models in the same test run, so you can control how much they reason. You cannot stop them on the current endpoints.

Provider pricing and context limits

Max has one provider in our catalog, Alibaba. Flash has two, Alibaba and Makora. The 2.4T has seven. The 27B has 14, and they do not charge the same. On 11 September 2026, 27B prompt prices ran from $0.15 to $0.45 per million tokens and completion prices from $2.00 to $3.20. The current list is on the model page, and it changes, so treat the figures here as a snapshot.

The 27B providers do not all serve the same context either. Two of the 14 offer the 1,000,000-token window, one stops at 65,536, and the rest stop at 262,144. A request carrying 400,000 tokens of input has to be routed to a provider that accepts it. Providers also quantize differently. Most of the 14 list fp8, one lists fp4, and several list their quantization as unknown. The weights behind one model ID are not identical from provider to provider, and the Artificial Analysis scores above describe the model as measured by Artificial Analysis, not any particular provider’s copy.

No single figure describes what the 27B costs. Pin a provider if the number matters to you. You can pick a provider with the order field and switch off fallbacks. Provider routing has the full set of controls.

Reasoning tokens are not itemized everywhere. In our test on 11 September 2026, Chutes returned the model’s reasoning text but reported zero reasoning tokens in usage, three runs out of three. Alibaba, CoreWeave, and Novita reported non-zero reasoning token counts on every run. You pay for those tokens either way, so a dashboard that sums reasoning_tokens across providers will undercount.

Going direct to the 27B providers would mean 14 accounts and 14 integrations. Through us it is one model ID in the request, and when a provider slows down or returns an error, we route the request to another one.

FAQ

Is Qwen 3.8 open source?

Partly. Qwen3.8 27B is published under Apache 2.0. Qwen3.8 2.4T A95B is published under the Qwen3.8-Max License, which Qwen wrote and which adds conditions at scale. Qwen3.8 Flash has no repository of its own, but the model it is based on, Qwen3.8-Flash-Next, is published under the Qwen Community License 1.0. Alibaba describes the -Next model as an experimental preview of the architecture planned for Qwen4, and Qwen3.8 Flash as its hosted version, so the suffix marks a preview release rather than a second Flash model. Qwen3.8 Max has no public weights.

Is Qwen 3.8 free to use?

You can download Qwen3.8 27B, Qwen3.8 2.4T A95B, and Qwen3.8-Flash-Next from Hugging Face and run them on your own hardware without a per-token fee. Qwen3.8 Max is available only through an API. Calling any of the four through OpenRouter is billed per token.

Can I use Qwen 3.8 commercially?

Qwen3.8 27B is Apache 2.0. Qwen3.8 2.4T A95B allows commercial use with two conditions. If your product exceeds 100,000,000 monthly active users or US$20,000,000 in monthly revenue, you must display the model name in its interface. If you or your affiliates run a Model as a Service or AI Work Assistant business with aggregate revenue over US$50,000,000 in any consecutive twelve months, you need a separate license from Qwen before commercial use. The Qwen Community License 1.0 on Qwen3.8-Flash-Next has the same attribution condition and requires the separate license for any Model as a Service or AI Work Assistant business, with no revenue threshold. The conditions are set out in the Qwen3.8-Max License and the Qwen Community License 1.0 files.

Do the open Qwen 3.8 weights accept images?

Qwen3.8 27B accepts text, images, and video. Qwen3.8-Flash-Next is a causal language model with a vision encoder. Qwen3.8 2.4T A95B is text only. On OpenRouter, an image request to qwen/qwen3.8-2.4t-a95b returns a 404 with the message "No endpoints found that support image input".

Was Qwen 3 the release before Qwen 3.8?

No. Qwen 3 was released in April 2025. Qwen 3.5, Qwen 3.6, and Qwen 3.7 were released between Qwen 3 and Qwen 3.8. Qwen 3.8 follows Qwen 3.7.

Can I turn off reasoning on Qwen 3.8?

It depends on the model and endpoint. On OpenRouter, qwen/qwen3.8-max-0902 and qwen/qwen3.8-2.4t-a95b rejected reasoning: { effort: 'none' } with a 400 error stating that reasoning is mandatory for the endpoint. qwen/qwen3.8-27b and qwen/qwen3.8-flash accepted it on the routes we tested and returned zero reasoning tokens. In our test, disabling reasoning on the 27B cut the cost of the same prompt by roughly 70%.

References

1. Qwen, Qwen3.8-Max announcement.
2. Hugging Face, Qwen/Qwen3.8-2.4T-A95B and its LICENSE.
3. Hugging Face, Qwen/Qwen3.8-27B.
4. Hugging Face, Qwen/Qwen3.8-Flash-Next and its LICENSE.
5. GitHub, QwenLM/Qwen3.8.
6. OpenRouter, Qwen3.8 Max, Qwen3.8 2.4T A95B, Qwen3.8 27B, and Qwen3.8 Flash model pages.
7. OpenRouter, Reasoning tokens, Provider routing, Prompt caching, and Image inputs.
8. Artificial Analysis, independent model benchmarks, retrieved 11 September 2026.

Original source

How the Batch API works

For large batches of work or requests that don’t need to be completed immediately, you can now use the new batch API. When submitting a batch, a provider gets to choose when during a 24 hour window they will complete the request, and in exchange they generally charge 50% (and sometimes less) of their normal per-token price.

It works today on more than 70 models. Learn how to use it in the Batch API docs.

In practice, we’ve observed you rarely wait anywhere near 24 hours. Across 230k+ batches that completed over our two week beta period, the median finished in 7 minutes and 90% finished within an hour.

Batch is an asynchronous API for workloads where you can accept highly variable response times, such as labeling a corpus, back-filling embeddings, scoring an eval set, summarizing a backlog of tickets, or running the same prompt across a few thousand rows overnight.

Call api/v1/batches with your list of requests and specify what endpoint shape to use. Chat completions, responses, messages, and embeddings are all supported. For example:

curl https://openrouter.ai/api/v1/batches \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d '{
  "endpoint": "/v1/chat/completions",
  "model": "google/gemini-3.8-flash",
  "requests": [
    { "custom_id": "ticket-0001", "body": { "messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }] } },
    { "custom_id": "ticket-0002", "body": { "messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }] } }
  ]
}'

After the request, begin polling GET /api/v1/batches/:id until the status is completed, failed, expired, or cancelled. Completed batches return their results inline in the same response.

Most batches finish in minutes

We looked at the batches that completed during our two week beta period and measured the time from acceptance to finished results. The median was 7 minutes, the 90th percentile was 1.0 hour, and the 99th percentile was 10.3 hours. We also found that the time of day you submit matters more than how many requests you send.

Batches submitted between 5am and noon Pacific are significantly slower than other times of day. The slowest tenth take 2 to 4.5 hours. Submit at any other hour and the 90th percentile drops under 1.1 hours, and after 6pm Pacific it’s under 50 minutes.

A single-request batch finishes in 5 to 11 minutes depending on the hour, and a batch of 1,000 or more requests finishes in 12 to 21 minutes. Large batches do take longer to process. The slowest tenth of batches with more than 100 requests submitted between midnight and noon Pacific took as long as 6.8 hours.

What’s supported with batches

• Request shapes: any text request body shape you already send to OpenRouter will be supported, including chat completions, responses, messages, and embeddings.
• Routing: Each batch is executed on a single provider. By default we choose the cheapest batch endpoint for the model after your provider allowlist, data policy, and BYOK settings are applied.
• BYOK: with a provider key configured, batches on providers that support it route through your key and you pay only the BYOK fee.
• Per-request results: every result returns independently, so a few bad rows never fail the rest of the job.
• Retention: inputs and results are kept for 30 days, or until you DELETE the batch.
• Logging: every batch shows up in the Batches tab of your logs with model, provider, status, and cost.
• Pricing: the discount applies to per-token pricing and varies by model. Web search calls bill at standard rates.
• Inputs: images and files must be public URLs. Audio, video, and OpenRouter’s own web search plugin aren’t available in batch (see the limitations section of the docs).

Get started

Pick a batch-capable model, grab an API key, and post your first batch to https://openrouter.ai/api/v1/batches. The Quickstart has the full request and response shapes.

Tell us what you’re batching in #feedback on Discord.

Original source

Nemotron 3.5 Lightning at a glance

Specification Nemotron 3.5 Lightning
Model type Hybrid Mamba-2, attention, and mixture-of-experts language model
Model size 30B total parameters, 3B active
Input and output Text to text
Model context limit Up to 1 million tokens, per NVIDIA’s model card
OpenRouter standard model 262,144-token context on every current provider. Completion limits vary by provider, from 32,768 to 235,929 tokens
OpenRouter free model 1,000,000-token context, up to 65,536 completion tokens
Tool calling Listed for both models. Two of the four current standard providers list it
Structured outputs Listed for the standard model on all four current providers. Not listed for the free model
OpenRouter model IDs nvidia/nemotron-3.5-lightning and nvidia/nemotron-3.5-lightning:free
Weights BF16 reference weights, plus optimized NVFP4 and GGUF releases
License OpenMDW-1.1
Released August 11, 2026

The endpoint limits and features in this table come from our model pages on September 11, 2026. Check the standard model page or the free model page before designing around a specific limit.

What does 30B MoE with 3B active parameters mean?

Nemotron 3.5 Lightning is a sparse mixture-of-experts model. It contains 30 billion parameters in total, but it does not use every expert for every token. A router selects a smaller subset of experts as the model processes each token, which brings the active parameter count to about 3 billion. You will also see this written as 30B-A3B, meaning 30 billion total parameters and about 3 billion active parameters.

This design gives the model more total capacity than a 3B dense model without running all 30 billion parameters for every token. That is how Lightning combines a relatively large checkpoint with the throughput needed for frequent agent calls.

The active count is not a complete compute or memory specification. The full model still has to be stored, and shared layers also run during inference. Hardware requirements depend on the precision, context length, inference engine, batching strategy, and cache configuration.

Lightning also uses a hybrid architecture rather than a pure Transformer stack. The NVIDIA model card describes interleaved Mamba-2 and MoE layers with selected attention layers. It also supports configurable reasoning and ships with multi-token prediction and two speculative-decoding drafters, DSpark and DFlash, for faster generation, along with an NVFP4 checkpoint tuned for inference.

What is Nemotron 3.5 Lightning designed for?

Long-running agents make many model calls. Some calls require difficult planning. Much of the work is narrower, such as choosing a tool, generating arguments, checking a result, editing a file, or deciding what to do next.

NVIDIA positions Lightning for that execution layer. The model is a candidate when your workload has the following characteristics:

• The agent makes many calls, and latency or cost compounds across the run.
• Each call has a clear objective and enough context to complete it.
• The model needs to use tools, follow instructions, or return structured data.
• You want open weights that you can customize for a domain or deployment target.

For example, a coding agent could use a larger reasoning model to inspect a repository and decide on an implementation plan. Lightning could then handle individual steps such as locating a symbol, editing one file, running a tool, and interpreting the result.

Nemotron 3.5 Lightning compared with Nemotron 3 Ultra

NVIDIA distilled Lightning from Nemotron 3 Ultra, but the two models are not interchangeable. Lightning is the smaller model for high-volume agent execution. Ultra is the larger model for complex reasoning and orchestration.

Specification Nemotron 3.5 Lightning Nemotron 3 Ultra
OpenRouter model ID nvidia/nemotron-3.5-lightning nvidia/nemotron-3-ultra-550b-a55b
Total parameters 30B 550B
Active parameters 3B 55B
Primary role High-volume execution and specialized tasks Complex reasoning and orchestration
Context on OpenRouter 262,144 tokens on every current provider 202,800 to 262,144 tokens, depending on provider
Tool calling Listed on two of four current providers Listed on every current provider
Structured outputs Listed on every current provider Listed on one of four current providers
Provider prices on September 11, 2026 $0.065 to $0.10/M input, $0.18 to $0.25/M output $0.50 to $0.625/M input, $2.20 to $3.125/M output

NVIDIA reports that Lightning reaches up to four times the throughput of similarly sized open models. In NVIDIA’s PinchBench testing, it completed 10,000 agent tasks up to 30% faster at comparable accuracy. These are vendor-reported results, so test throughput and task completion on your own workload.

The distinction is the difficulty and consequence of the call. Ultra is the stronger fit when a step requires deep reasoning across an ambiguous problem, produces the plan for a long workflow, or makes a decision that is expensive to reverse. Lightning is the stronger fit when the task is well-scoped and repeated often enough for latency and cost to matter.

You do not need to assign one model to the entire agent. Route complex calls to Ultra and frequent execution calls to Lightning. Then measure whether the mix improves cost per completed task without reducing reliability.

Routing between the two models on OpenRouter

If you would rather hand model selection to us, use Auto Router. Send openrouter/auto as the model ID, and Auto Router classifies the prompt before selecting a model based on the task type, model capabilities, tool support, and cost. The cost_tier setting selects a cost band, from low to max. It does not escalate a conversation from a cheaper model to a more expensive one. If you want Auto Router to choose only between Lightning and Ultra, restrict its choices with allowed_models.

Routing between the two models outside OpenRouter

NVIDIA’s NeMo Switchyard implements this routing pattern as an open-source orchestration layer. Its escalation router starts each conversation with a lower-cost model, and an LLM judge moves the session to a more capable model when it detects sustained difficulty. In LangChain’s evaluation of 145 multi-step agent tasks, routing between Lightning and Claude Opus 4.8 reduced cost by 74% compared with the frontier-only baseline while sending about 7% of calls to the frontier model. Accuracy was about six points lower. This result shows the routing trade-off on that benchmark. It is not a benchmark of a Lightning-and-Ultra combination. Test any routing setup on your own tasks before you rely on the savings.

How long is the context window?

Nemotron 3.5 Lightning supports up to 1 million tokens at the model level. The context available to your application depends on the endpoint that serves it.

On September 11, 2026, every provider behind the standard model ID exposes a 262,144-token context window. Completion limits differ by provider, from 32,768 to 235,929 tokens. The :free model exposes the full 1-million-token context window and caps completions at 65,536 tokens.

If a request needs more than 262,144 tokens, only the free endpoint currently exposes the model’s full 1-million-token context on OpenRouter. NVIDIA’s notice on that endpoint states that use is logged for security purposes and to improve NVIDIA products and services, and asks you not to upload confidential information or personal data. For sensitive or production long-context workloads, choose another endpoint or model.

This difference matters when you switch between the two model IDs. A request that succeeds on the free endpoint may exceed the context limit of the standard endpoint. The free endpoint also lists tool calling but does not list response_format or structured outputs.

Check the model page instead of treating the architecture’s maximum context as a promise from every provider. We show the current limits and supported parameters for each endpoint on the model page.

Open weights and local deployment

NVIDIA publishes the BF16 reference checkpoint under the OpenMDW-1.1 license. The model card describes the BF16 release as a starting point for post-training, domain adaptation, research, and producing optimized variants. NVIDIA also publishes training data and recipes for customization and evaluation, and the model card says the release is ready for commercial use.

For direct inference, NVIDIA recommends its NVFP4 release. It also provides a GGUF checkpoint for supported local systems. The BF16 guidance lists an H100 80GB or A100 80GB for single-GPU deployment. The optimized releases target hardware such as the RTX 5090, RTX PRO 6000, and DGX Spark.

Running the model locally does not mean every machine can serve the full 1-million-token context window. In NVIDIA’s current Ollama guidance, the default context scales with available VRAM. It is 4K below 24GB, 32K from 24GB to below 48GB, and 256K with 48GB or more. The llama.cpp example uses about 40K. You can raise those limits, and doing so may require more memory or CPU offloading.

The weights are publicly available, and “open-weight” is the clearest description. Read the OpenMDW-1.1 license before you redistribute a modified model or make deployment decisions around its terms.

How to call Nemotron 3.5 Lightning on OpenRouter

If you do not want to provision GPUs or manage an inference engine, call Lightning through OpenRouter. The standard model ID keeps the same API while we route requests across the available providers for that model.

Install the OpenRouter TypeScript SDK.

npm install @openrouter/sdk

Set OPENROUTER_API_KEY in your environment, then send a request with the Lightning model ID. The standard model lists structured outputs, so you can ask for a JSON schema and have the response conform to it.

When we ran this request on September 11, 2026, the model returned {"category": "Bug (Payment Checkout)", "priority": "medium"}. Sampled outputs vary between runs, and the schema, not the values, is what the request guarantees.

Provider support for response_format and structured outputs differs within the same model. By default we prefer providers that support the tools and response_format parameters you send, and a provider that does not support a parameter ignores it. Setting require_parameters to true, as the example does, restricts the request to providers that support every parameter in it. See the provider routing docs for the full behavior.

To try the free endpoint, change the model ID to nvidia/nemotron-3.5-lightning:free. The free endpoint does not list response_format, so drop that field there and validate the JSON yourself. It is useful for evaluation and low-volume experiments, and it has different limits and availability from the standard endpoint.

Do not submit confidential information or personal data through the free endpoint. Its model page carries NVIDIA’s notice that use is logged for security purposes and to improve NVIDIA products and services. Review that notice before deciding which prompts to send.

By default we load-balance the standard model across its providers, ordered by price. Two variants change that ordering.

nvidia/nemotron-3.5-lightning:nitro sorts providers by throughput, and nvidia/nemotron-3.5-lightning:exacto prefers providers with stronger tool-calling quality signals. For a model chosen for latency and tool use, Nitro and Exacto are the two variants worth knowing.

Lightning accepts tools and tool_choice, so you can also use it inside an agent loop. See our tool-calling agent loop guide for the full request, tool execution, and iteration flow.

Should you use Nemotron 3.5 Lightning?

Nemotron 3.5 Lightning is worth evaluating when you need a fast, open-weight model for frequent, well-defined agent steps. Its 30B-A3B architecture, tool support, and low listed token price make it a candidate execution model for agents that would otherwise send every call to a much larger reasoning model.

Use the model name as a starting point, not the decision. Build an evaluation set from the tool calls and tasks your agent performs. Measure task success, retries, latency, and total cost across the full run. A cheaper call does not help if the agent repeats it or escalates the result often enough to erase the savings.

Start with nvidia/nemotron-3.5-lightning, or use nvidia/nemotron-3.5-lightning:free to test the model before adding paid traffic.

FAQ

What is Nemotron 3.5 Lightning?

Nemotron 3.5 Lightning is NVIDIA’s open-weight 30B mixture-of-experts language model with about 3B active parameters per token. NVIDIA positions it for high-volume agent execution, tool use, coding, instruction following, and specialized tasks.

Is Nemotron 3.5 Lightning the same as Nemotron 3 Ultra?

No. Nemotron 3.5 Lightning has 30B total parameters and 3B active parameters. Nemotron 3 Ultra has 550B total parameters and 55B active parameters. Lightning targets frequent execution steps. Ultra targets complex reasoning and orchestration.

What does 30B-A3B mean?

30B-A3B means the model has 30 billion parameters in total and activates about 3 billion parameters per token. The mixture-of-experts router selects a subset of the model’s experts for each token instead of running every expert.

Does Nemotron 3.5 Lightning support tool calling and structured outputs?

The standard OpenRouter model, nvidia/nemotron-3.5-lightning, lists tools, tool_choice, response_format, and structured outputs among its supported parameters. Support varies by provider, so set require_parameters to true if your request depends on one of them. The free model lists tools and tool_choice but not response_format or structured outputs.

Is there a free Nemotron 3.5 Lightning API?

Yes. We list nvidia/nemotron-3.5-lightning:free, served by NVIDIA at no token cost. Free models have different rate limits and availability from paid models, and this endpoint carries an NVIDIA data notice. Do not send confidential information or personal data through it.

Can I run Nemotron 3.5 Lightning locally?

Yes. NVIDIA publishes BF16, NVFP4, and GGUF weights under the OpenMDW-1.1 license. The BF16 reference checkpoint targets a single 80GB H100 or A100. The NVFP4 and GGUF releases target supported local hardware such as the RTX 5090, RTX PRO 6000, and DGX Spark. Check NVIDIA’s current model card and license before choosing a deployment.

Original source

What is Jev? And what should you use it for?

It’s TypeSafe’s decision model that takes a passage of text plus a typed question, and returns a typed answer that is one of the choices among your predefined set of answers. So if I have a question about ticket routing, like should this ticket go to billing, technical, or account, Jev looks at the text of the ticket and answers with one of the predefined choices, billing. But it’s even cooler than that because it gives you the calibrated probability of the answer choice as well as the probability of each of the other choices. Oh, and there’s also an overall confidence score (more on that later). TypeSafe calls these System One models, and Jev is the first of them.

But what does Jev actually return? Let’s find out! It returns three primitives, and there are real API responses for each below. Here’s how to read the probabilities without fooling yourself. And here’s how to call Jev with an OpenRouter API key.

What is a decision model (System One model)?

Jev is a decision model. A System One model is one that takes in a piece of state and, given that state, returns a decision. In this case, Jev’s decision is always a typed value you defined in advance, plus an actual probability number from zero up to one. The name comes from what Daniel Kahneman called System One in Thinking, Fast and Slow: fast, pattern-matching thinking, where System Two is slow and deliberate.

So what makes this a decision model and not some oracle? A decision model must return from a small set of values you decided beforehand. There is no free-form text, so nothing you need to parse or worry about hallucination.

Now, TypeSafe calibrated Jev so that when it says 0.8 (an 80% chance), it means that, like an 80% chance of rain, Jev is right about that kind of answer about 80% of the time (System One concepts).

That’s only true when you average across many answers. Any single answer can still be wrong. That makes a practical difference; it’s a point we’ll come back to later.

Jev vs LLM: what each one returns

The difference between Jev and LLMs becomes clear once you understand what both do. Both can read natural language, but the split is entirely on the output side.

Here’s how the two compare side by side:

Generative LLM vs Jev

Output: Tokens: prose, code, JSON you asked for vs A typed answer plus a probability distribution

Shape guaranteed?: Only with structured outputs, and the content can still be wrong vs Always one of your options, levels, or a yes/no probability

Uncertainty: Hidden inside the prose vs Returned as numbers you can threshold on

Input: Text, and often images, audio, or video vs Text only (strings, JSON objects, arrays of text)

Price on OpenRouter: Varies by model, input and output billed vs $0.042 per million input tokens, output free

Whenever you need words as the main output, for example, a reply, summary, or code patch, reach for an LLM. When you need to make a code-actionable decision, use Jev. In just about every real-world case, the two systems work together: Jev routes and verifies things, while the LLM provides the language. And that’s what the companion post Jev vs LLM: when to use each is all about. It benchmarks that split on 140 support cases and ties the two together in TypeScript.

Jev’s three primitives: Choice, Score, and Noul

Every question you ask Jev is one of three types. A request has two parts: state, the text to evaluate, and a questions object containing one or more typed questions. Jev answers all in one pass, each answer returned under the key you gave.

The examples below are real calls I placed on September 21, 2026. They run against typesafe/jev-1.13 through OpenRouter’s Decisions API, and if you set OPENROUTER_API_KEY, you can run the examples as written.

Choice: pick one option from a fixed set

When you get a Choice from the API, it always has three things: the selected option, a probability for every option, and a confidence value.

Score: rate on ordered levels you describe

Score takes an ordered array of level descriptions. It returns a number representing the score for the text and a legend mapping every index in the array to its description, plus Jev’s probability for each level and a confidence value.

The score, 1.15, is the probability-weighted average of the levels, 0.85 times 1 plus 0.15 times 2, as shown in the above response. Your own run will land a few hundredths away because Jev’s probabilities vary slightly between calls. Reading “we can still download JSON exports” as a workaround, Jev lands mostly on level 1. It also gives some weight to “blocking” because finance can’t use the workaround.

Because scores are on an ordinal scale, you must keep the levels in a real order from least to most and describe each level in words. The TypeSafe docs list the Score primitive details, including a cap of 10 levels.

Noul: probability that a yes/no proposition holds

Noul is the simplest primitive of the three. It takes as input a proposition and returns the probability that the answer is yes.

Keep in mind that a Noul answer has no separate confidence field. The probability is the whole answer. The next section explains how to interpret it.

Ask all three in one call

You might even be asking yourself, hey, how many questions can go in that questions object? As many as you want. The state doesn’t have to be a plain string: it can be a JSON object with all the flexibility that implies. Your instructions can then refer to its fields by name in backticks.

Let’s say you want to ask a Choice, a Score, and a Noul all at once from the same ticket domain in a single round trip.

The response shows that if all went smoothly, each answer lands under the key you gave it. In code, you’ll then be able to go answers.team.choice and answers.refund.noul directly.

How to read Jev probabilities and confidence

There are three guidelines to keep in mind when reading Jev’s probabilities and confidence.

When the Noul is near 0.5, that’s Jev saying it doesn’t know what to think. Read it as a state of uncertainty. For example, a refund question on a message returned Noul values near 0.5, indicating uncertainty. When the outcome is in the middle, treat it as a third outcome, one you can take action on. Ask a follow-up question, or hand the ticket over to a person.

Choice and Score confidence is a measure of how concentrated the distribution is. The more all of the distribution’s weight falls on one outcome, the higher the confidence. A choice concentrated around a single option yields a high confidence number, on a scale from 0 to 1. The higher the confidence number, the less hesitation there is between the options. But just because Jev concentrates its attention on an option doesn’t mean the answer is right. TypeSafe draws its confidence numbers from the shape of its probabilities, not their actual correctness (confidence docs). In the extreme case, a model that assigns all of its weight to one outcome will have a confidence of 1.0, while less concentration will diminish that number.

You can verify that with the billing example. This state was muddier: “I upgraded to Pro yesterday but the dashboard still says Free and I got charged. Which one is it?” The response shows Jev still picks billing, although a fifth of its weight sits on technical, and the confidence drops to 0.69. Confidence is a measure of how torn Jev is between options, but whether those options are well chosen is up to you.

Pick your own thresholds from your own labeled data. The operational advice is simple. Calibration holds in aggregate, which means that the right cutoff for you depends on how expensive your mistakes are. TypeSafe’s confidence guide suggests three bands: Act automatically if the confidence passes one threshold. Proceed with caution in the middle, flagging for an additional layer of user confirmation or simply flagging for review. And route to a person or different system if the confidence falls below the other threshold.

How do you set those bands? Label a few hundred examples. For each of them, inspect where Jev’s probabilities fall. Then pick bands that ensure the error rate within the act band is one that you can live with.

What Jev leaves to your code

There are a few things that Jev doesn’t do that you might expect it to do as a model. Those stay in your own code or with an LLM:

• Text generation. Every Jev response is one of the three typed shapes shown earlier, never a completion, never a JSON body, never an explanation. When a pipeline needs a sentence, an LLM produces it and Jev checks it. The verified cascade cookbook shows this pattern.

• Visible reasoning. Jev returns only the distribution over your options, nothing else. No rationale, no chain of thought. Keep the distribution as the signal to log and threshold on. For an audit trail, log the request ID, names of the questions, the probabilities, and the threshold your code applied. Keep the state itself out of the log because tickets and documents contain customer data.

• Tool calls, conversation, or multi-step plans. Jev answers the questions you sent about the state you sent in one round trip. It makes no tool calls, holds no conversation, and takes no steps. It works inside agents as the deciding part, for example gating tool calls.

That’s three! Here are two more constraints to keep in mind:

• Jev accepts text only, strings, JSON objects, or arrays of text. No images, audio, or video.

• Exact arithmetic, date math, and threshold comparisons aren’t Jev’s job. Compute them first and hand Jev the semantic part.

When to use the Jev model

Now that you’ve got the idea of what Jev does, let’s talk about when it makes sense: when the answer your LLM is expected to provide fits into an enum, a boolean, or a number.

When to use Jev

• Routing and triage: you determine queue, owner, priority. One Choice and one Score per ticket.

• Classification and tagging at scale: cheap enough (see pricing below) to run on every row; the output is already a label.

• Gating agent actions: before a destructive tool run, ask a Noul whether the user’s request authorizes deleting these files; block below your threshold.

• Verifying LLM output: draft with an LLM, then ask Jev whether that draft is grounded in the policy text you supplied.

• Ranking and filtering: score candidates against described levels, then sort.

When to skip Jev

When the answer needs to be prose, or when the input is an image, or when the logic is deterministic. An exact rule? A regex or database lookup beats any model. To poke at it before coding, Jev Lab runs several of these patterns in the browser on live calls.

How to access Jev on OpenRouter

You’ll need an OpenRouter API key and a model ID: typesafe/jev-1.13 (pinned) or ~typesafe/jev-latest (alias to track the newest version). The same call can be made three different ways: using the OpenRouter SDK, the TypeSafe SDK, or raw HTTP.

Decisions API with the OpenRouter SDK

The code below shows the same call as the one in the muddier ticket in the confidence section: a POST https://openrouter.ai/api/alpha/decisions call via the OpenRouter TypeScript SDK’s alpha.decisions. The code has been tested with SDK 1.3.11, installed via npm install @openrouter/sdk. Because of the top-level await, run it with bun run or as an ES module.

TypeSafe JavaScript SDK pointed at OpenRouter

The second calling path is the TypeSafe SDK: point the SDK’s base URL at OpenRouter and pass your OpenRouter API key as the API key. The SDK will append /v1/systemone to the base URL. Bare model names like jev-1.13 map to typesafe/jev-1.13. The code below is the same call to Jev, using @typesafe-ai/sdk; the code has been tested with SDK 0.6.0.

TypeSafe Python SDK pointed at OpenRouter

The same idea in Python, with typesafe-sdk 0.7.1 (pip install typesafe-sdk 0.7.1).

Keep your OpenRouter key safe and sound on the server-side. The TypeSafe SDK guide has details about the alias mapping, the response envelope, and error handling.

Jev pricing and context window

From the OpenRouter model page on September 21, 2026:

Input price: $0.042 per million tokens

Output price: $0

Context window: 32,000 tokens

Modality: Text in, decisions out

Provider: TypeSafe

To sum up: the three-question ticket call above used 447 input tokens and cost $0.000019 (about two thousandths of a cent). A million tickets of that size come to about $19. Output is free and the answers are tiny (between 17 and 69 tokens across every call above). Because the key order inside probabilities shifts between calls, I suggest you index entries by name rather than by position.

OpenRouter lists a 32k context window. TypeSafe’s model page lists 64k tokens per request total, with 32k for your state plus the longest question.

Is Jev open source?

Quick question: Is Jev proprietary? The answer is obvious (yes), but let me explain. TypeSafe hasn’t published Jev’s weights or a paper on Jev, and everyone who wants to use Jev calls the same hosted model.

Try it

Now it’s time to see all this with your very own OpenRouter API key! Grab it, paste one of the curl commands above, and change the state to a real message from your own system.

• If you would rather use TypeSafe’s client, read the TypeSafe SDK guide.
• Work through how to gate tool calls with Jev, putting a Noul in front of an agent action.
• Read Jev vs LLM: when to use each for benchmarks, and the route-then-write pattern.
• Follow How to Use Jev to build a full moderation pipeline in TypeScript, with Choice, Score, and Noul questions in one request.
• See how Jev scored against a frontier model in Is Jev as Accurate as Frontier Models at Classification?, and where it fits next to a judge model in Jev vs LLM-as-a-Judge.
• The Jev documentation hub lists every Jev guide and cookbook on OpenRouter.
• And before you budget, check the model page to see what the current pricing looks like for that model!

FAQ

What is Jev?

Jev is a decision model made by TypeSafe. You interact with most models by sending them text. You don’t do only that with Jev. Instead, you send it text plus one or more typed questions, and the answers you get back are typed answers with calibrated probabilities. No text generation. TypeSafe calls this category of models System One models.

Who makes Jev?

Jev is a decision model. It was built by TypeSafe, an AI lab that was founded by Diogo Almeida, Erik Gafni, and Sasha Sheng. Jev was released in early access on September 15, 2026. Right now, the current version is Jev 1.13. You can get it through TypeSafe’s own API or through OpenRouter.

Is Jev an LLM?

No. Jev is a non-generative decision model, while LLMs are generative models: Jev reads your natural language like LLMs do, but then generates no text, no tokens. Jev returns one of three answer shapes, depending on your question: either a choice among a set of pre-determined options, a score on a defined set of levels, or the probability of yes vs no. Because there’s no generated text, there’s nothing to parse and no chance of output hallucination.

What is Jev used for?

Okay, so you’ve got to decide which support ticket gets routed to which team. You’ve got to classify this blog post into a content type. You want to gate an AI agent’s tool call based on a confidence threshold. You’ve got to rank a set of candidates in order. And you’ve got to verify a draft against a policy. Sound familiar? In these situations, you’re the one who would previously ask an LLM a question, then use a regex to parse out a boolean value or an enum value. Meet Jev, the model purpose-built for just these kinds of decisions, replacing that LLM-plus-regex workflow.

How do I access Jev?

So you want to access Jev through OpenRouter? Well, you only need an OpenRouter API key and a POST to the following URL: https://openrouter.ai/api/alpha/decisions, with model typesafe/jev-1.13. Alternatively, you can use the decisions client in OpenRouter’s SDK. You can even use the TypeSafe JavaScript and Python SDKs. Just set the base URL to https://openrouter.ai/api and pass your OpenRouter key.

What does Jev cost?

Right now, Jev 1.13 on OpenRouter costs $0.042 per million input tokens (and output is completely free) as of September 21, 2026. Which means a typical three-question call on a short support ticket used around 450 input tokens, costing you roughly two thousandths of a cent. So always remember to check out the model page to see current pricing.

Is Jev open source?

No. Jev is a proprietary model and as such there are no published weights and no paper. You can, however, call it through the TypeSafe API or OpenRouter.

Original source

Tl;dr

• ZDR prevents an eligible inference provider from retaining your prompts and responses.
• Your data still reaches the provider and is processed by the model.
• On OpenRouter, you can enforce ZDR through account settings, guardrails, or the provider.zdr request field.
• ZDR applies to provider inference. Your logs, enabled tools, plugins, and other storage features require separate controls.

What is zero data retention (ZDR)?

Zero Data Retention is a data-handling policy under which an AI provider processes your request without persisting the prompt or response. ZDR governs provider-side retention. It doesn’t mean your data remained inside your network or that every service involved in the request followed the same policy.

We evaluate data policies at the endpoint level because a provider’s general policy may differ from the policy attached to a particular model endpoint. When we can’t confirm an endpoint’s policy, we take the conservative position and classify it as retaining and training on data.

ZDR answers one of three separate questions:

• Retention at rest: Whether the provider stores prompts and responses after the response returns. ZDR covers this.
• Data in transit: Your request still travels to the provider, and the model still processes it. ZDR doesn’t change that.
• Training on data: Whether a provider uses your inputs to improve its models. That’s a distinct control, though providers often pair it with ZDR.

A “no training” policy doesn’t necessarily mean ZDR. A provider may decline to train on your data while retaining it temporarily for another reason. The reverse relationship is stronger. An endpoint that doesn’t retain your data can’t later use that data for training.

ZDR routing is also separate from our own logging policy. We don’t store prompt or response content unless you opt in to input and output logging. We still retain request metadata such as token counts, latency, model, and cost so you can view usage and activity information.

What ZDR covers and what it doesn’t

ZDR has a specific boundary. It covers provider-side retention for eligible inference endpoints. It doesn’t become a universal privacy policy for every system your request touches.

Data-handling question Covered by ZDR? What it means Provider stores your prompt after inference Yes A ZDR endpoint doesn’t persist the prompt Provider stores the model response Yes A ZDR endpoint doesn’t persist the completion Provider trains on the retained prompt Indirectly prevented There is no retained prompt available for training Data reaches the model No The provider must process the input to generate a response Processing stays in a specific country or region No Use data residency or in-region routing OpenRouter stores request metadata No Metadata can be retained without prompt or response content Your application logs prompts No Your own logging and storage policies still apply A plugin or tool retains data No Tools have their own operators and data policies An endpoint retains data for abuse or legal review No We don’t treat a retaining endpoint as ZDR

“We do not train on your data” may still allow temporary retention for abuse detection or legal obligations. That endpoint doesn’t meet our ZDR definition. You can still use it under a no-training policy, but you shouldn’t describe the request as zero retention.

ZDR enforcement also applies only to inference provider routing. A web search plugin, external tool, or other service you enable can receive request data under its own retention terms. Review those policies separately before using them in a workflow with strict retention requirements.

Your application remains another retention layer. A ZDR request can still leave a complete prompt in an error tracker, an analytics event, a database row, or an application log. Provider-side ZDR doesn’t delete any of those copies.

Check caching separately

We consider provider-side in-memory prompt caching compatible with ZDR because the prompt isn’t written to persistent storage. Cached representations remain in provider memory long enough to reuse the prompt and improve performance.

Our response caching feature has different behavior because it temporarily stores generated responses. Account-level ZDR disables response caching, but the per-request provider.zdr field doesn’t affect response-cache eligibility. A system that requires zero storage across every layer should therefore review its response-caching configuration separately.

ZDR versus related AI privacy controls

ZDR is one of several privacy controls. Each control answers a different question.

ZDR vs. “no training on your data”

These often ship together, but they answer different questions. “No training” governs whether your inputs improve a model. ZDR governs whether the provider stores those inputs after the response returns. A provider can promise not to train on your data while still holding it briefly for abuse checks or legal reasons. If you need both, enforce both.

On OpenRouter, data_collection lets you filter providers based on whether they store user data non-transiently and may train on it. Setting it to "deny" excludes those endpoints. You can combine it with zdr: true when you want both controls to be explicit in the request.

ZDR vs. data residency and region pinning

Region pinning controls where your request is processed, inside a specific region for GDPR, for example, while ZDR controls whether the data is kept afterward. A provider might process and retain a request in the EU, or process a ZDR request somewhere else entirely.

Use both controls when a policy names a permitted processing location and a retention requirement. We provide in-region routing for the US and EU on Business and Enterprise plans through the us.openrouter.ai and eu.openrouter.ai API domains. This is separate from ZDR enforcement.

ZDR vs. self-hosting

Self-hosting keeps inference inside your own infrastructure. Before choosing it, check whether these controls meet your requirements:

1. Region pinning: Controls where the request is processed.
2. ZDR routing: Prevents provider-side storage of prompts and responses.
3. Per-key or per-workspace guardrails: Separate one customer’s traffic and policy from another’s.
4. Your own logging: Records what was sent and where.

Self-hosting is required when policy prohibits all third-party processing, including transient inference.

How to route to ZDR-eligible endpoints on OpenRouter

A provider offering ZDR doesn’t automatically make every request ZDR-compliant. You must enforce the policy at the account, guardrail, or request level.

Account-level enforcement in privacy settings

You can enforce ZDR across every request without touching your code, from your account’s privacy settings. ZDR can be required per model group, covering Anthropic, OpenAI, Google, SpaceXAI, and non-frontier endpoints, so you can require it for some model groups and not others. You can also enforce it through guardrails. See the Zero Data Retention guide.

Data policy filtering is a separate switch. It lets you disable providers that store inputs for training. Opt out of training in your account settings and we won’t route to providers that train on your data.

Per-request enforcement with the zdr routing control

For request-level control, set the zdr field inside the provider block. When zdr is true, the request routes only to endpoints with a Zero Data Retention policy. When it’s false or omitted, it doesn’t affect routing.

The related data_collection control takes "allow", the default, or "deny". Set it to "deny" and routing excludes endpoints that store user data non-transiently and may train on it.

The per-request zdr parameter composes as an OR with your account-wide and guardrail settings. If any one of them turns ZDR on, enforcement applies. The request-level flag can only ensure ZDR is on, it can’t override or loosen an account-wide or guardrail rule.

How to verify a vendor’s ZDR claim

Check these five points.

1. What exact data does ZDR cover? Confirm whether it includes prompts, completions, uploaded files, tool inputs, cached representations, and identifiers.
2. Does the policy apply per provider or per endpoint? Model features and API endpoints can have different storage requirements. A provider-wide statement may hide exclusions.
3. What falls outside the policy? Ask about metadata, plugins, tools, prompt caching, response caching, logging, batch APIs, and stateful features.
4. How is ZDR enforced? Look for an account policy, guardrail, or request-level routing control rather than manual provider selection.
5. How can you verify continued eligibility? Data policies change. We maintain endpoint-level policy information and publish the current list of ZDR endpoints at https://openrouter.ai/api/v1/endpoints/zdr, so routing decisions can follow the current policy rather than a static spreadsheet.

Conclusion

ZDR reduces provider-side storage risk, but only when your request reaches a qualifying endpoint. Treat it as an enforceable routing requirement.

For sensitive inference traffic, combine the controls that match the actual policy. Use ZDR for retention, data_collection: "deny" for storage and training restrictions, and in-region routing when processing location matters. Then review your application logs, enabled tools, and caching configuration so another layer doesn’t recreate the data you removed from the provider.

Building on our API? Restrict your calls to ZDR endpoints, starting with our ZDR docs and provider routing controls. Reviewing AI data handling for your org? See OpenRouter for Enterprise.

FAQ

What is ZDR (zero data retention)?

Zero Data Retention means an AI provider processes your prompt and returns a response without persisting either one afterward. ZDR governs provider-side storage for eligible endpoints. It doesn’t prevent the data from reaching the model, keep the request inside your network, or control copies created by your own systems.

What is a zero data retention policy for AI?

A zero data retention policy states that an AI provider won’t store prompts or responses after processing them. The policy should identify which endpoints and features qualify, how it’s enforced, and what happens to metadata, caches, tools, and logs. A no-training policy alone doesn’t guarantee zero retention.

What is a zero-retention API?

A zero-retention API processes requests without storing the prompt or response after inference. With OpenRouter, you can restrict a request to eligible endpoints by setting provider.zdr to true. Account-level privacy settings and guardrails can enforce the same requirement across broader groups of requests.

Which LLM is best for privacy?

Privacy depends on the endpoint and routing policy, rather than the model name alone. The same model can be available through providers with different retention rules. Filter the model catalog for eligible endpoints, then enforce ZDR, data-collection, and residency requirements through your routing configuration.

Original source

OpenRouter · 9/10/2026

OpenRouter Fusion is our compound model. It takes one prompt and turns it into a short debate among several models. We send your prompt to a panel of expert models at once, and a judge compares every response so the calling model can write a single final answer.

With Fusion, you trade some speed and tokens for quality. This page covers what Fusion does, what it costs, when it beats a single model, and when it doesn’t.

Tl;dr

• What Fusion does. It gives a model access to a multi-model deliberation tool. A panel answers in parallel, a judge maps agreement and disagreement, and the calling model writes the final response.
• What it costs. An invoked Fusion call adds panel and judge completions. The default three-model panel costs roughly four to five times as much as one completion on the same prompt and often takes two to three times longer.
• When to use it. Use Fusion as a selective escalation path for complex research, expert critique, and decisions where an incorrect answer creates more cost than a few additional model calls.

What is OpenRouter Fusion?

Fusion is a compound inference system that gives a model access to multi-model deliberation. When the model invokes Fusion, several models answer the prompt in parallel. A judge compares their responses and produces structured analysis, which is then used to produce the final answer.

That makes Fusion different from a single model call. A single call follows one model’s reasoning path. Fusion can bring several reasoning paths, source selections, and interpretations into the same response.

Fusion and auto-routing serve different jobs. Auto-routing picks one model for your request by classifying the prompt’s task type and choosing the model the OpenRouter community spends the most on for that kind of task. Fusion combines many models and blends their answers, and it can produce a stronger answer than any single panelist would give.

How OpenRouter Fusion works

Fusion adds a deliberation loop inside a normal model request.

The pipeline has four stages:

1. The calling model evaluates the prompt. When you use Fusion, we resolve the alias to a model and attach the Fusion tool. The model can answer directly or invoke Fusion when the task warrants more analysis.
2. The panel works in parallel. Between one and eight participant models independently answer the prompt. Each panelist can use OpenRouter web search and web fetch to find current sources.
3. The judge compares the responses. Our docs call this role the analyst. The judge identifies consensus, contradictions, partial coverage, unique insights, and blind spots. It returns that comparison as structured analysis.
4. The calling model writes the answer. The original model receives the judge’s analysis and uses it to produce the response returned to your application.

The judge’s job is comparison rather than simple voting. Three models repeating the same unsupported claim don’t automatically make that claim correct. The judge can also surface a useful point that appears in only one response or identify a gap that every panelist missed.

Where the quality gain comes from

Fusion benefits from diversity between models and variation between separate runs. Different models may choose different methods, notice different constraints, or retrieve different sources. Even two runs of the same model can follow different reasoning paths and make different tool calls.

We tested that second effect by pairing Claude Opus 4.8 with another Opus 4.8 run and using the same model for synthesis. The fused configuration scored 65.5% on our DRACO benchmark run, compared with 58.8% for a solo Opus 4.8 run. That 6.7-point improvement suggests that the comparison and synthesis process contributes meaningful value even without model diversity.

Multi-model ensembling is an established technique. Fusion’s product value is operational. You can add a panel, judge, tools, and synthesis loop through one model slug or server tool instead of building and maintaining that orchestration yourself.

Fusion vs. a single model

Fusion can improve difficult answers, but it does so by performing more work.

Quality depends on the task

On DRACO, a deep-research benchmark by Perplexity AI, a budget Fusion panel using Gemini 3 Flash, Kimi K2.6, and DeepSeek V4 Pro scored roughly 64.7% against about 65.3% for Claude Fable 5 on its own. Fable-based results reflect 93 of 100 tasks because of content filters, so direct comparisons are slightly uneven.

DRACO measures deep research, not raw coding or general chat. Fusion’s synthesis tends to help most on research and analysis prompts, where several viewpoints genuinely sharpen the answer. Don’t assume the same margin carries to every task.

More tokens, but cost per task can still win

Fusion pays for several model calls plus the judge, so a single request uses more tokens than a single-model call. Cost per call isn’t always the number that matters, though. Cost per correct answer often matters more.

If one Fusion call returns the right answer while a cheaper model needs three attempts, a rerun, and a human to check it, Fusion can come out cheaper across the whole task. Count the total cost of reaching the result, not the price of one request. Our Fusion model page has current costs.

Latency runs two to three times longer

Fusion calls often take two to three times as long as a standard single-model call. The panel runs concurrently, so you’re not waiting on each model in sequence, but you still wait for the slowest panelist and then for the judge. That delay usually rules out chat, autocomplete, and other real-time paths.

Non-deterministic by design

A panel plus a synthesis step can return different results run to run. That’s by design. It’s fine for a one-off research task, but it becomes a problem when you need repeatable output, like evaluation suites, regression tests, or any check that compares today’s result against yesterday’s.

When to use Fusion and when to skip it

The strongest production pattern is selective escalation. Let a model handle routine work directly and invoke Fusion for the smaller set of prompts that deserve additional scrutiny. For per-step escalation to a single stronger model, see the Advisor server tool.

Use it for high-stakes, research-style prompts where being wrong is expensive

Research questions, expert review, comparisons, and due-diligence summaries, anywhere accuracy is the priority and a later correction costs real time or money. In practice, think of summarizing a competitive field from a dozen current sources before you commit to a strategy call.

Use it when you’d otherwise poll several models by hand

If your current workflow is asking three models the same question and comparing the answers yourself, Fusion is doing exactly that job. The judge also compares answers more consistently than a manual review.

Skip it for latency-sensitive or high-QPS interactive paths

When a user is waiting for a response, the two-to-three-times latency is too long. In practice, this means a customer chatbot or inline code completion.

Skip it for reproducibility-sensitive workloads

Evals, regression suites, and anything that needs stable results across runs. Non-determinism makes those comparisons unreliable. In practice, a CI pipeline that checks whether an LLM’s output changed.

Skip it for simple, well-scoped tasks a single mid-tier model already handles

If one mid-tier model returns correct answers today, a panel just adds cost and delay for no real gain. In practice, classification, extraction, short rewrites, and format conversion.

How to use OpenRouter Fusion

You can test Fusion in the web interface or call it through any supported inference endpoint.

The no-code path

Open the Fusion lab, select a preset, and enter a prompt that benefits from multiple perspectives.

Start with a prompt you already know is difficult. Compare the fused result with the answer from your current production model. Look for fewer factual errors, broader coverage, clearer disagreement handling, or less human editing.

The interface also lets you build a custom panel before moving the configuration into your application.

The Fusion API

The simplest API path is to replace your current model slug with openrouter/fusion. With no additional configuration, Fusion uses the default Quality panel and lets the model decide whether deliberation is necessary.

The example below selects the general-budget preset, overrides its judge model, and requires Fusion to run:

import os
from openai import OpenAI
client = OpenAI(
    base_url = "https://openrouter.ai/api/v1",
    api_key = os.environ["OPENROUTER_API_KEY"],
)
response = client.chat.completions.create(
    model = "openrouter/fusion",
    messages = [{
        "role" : "user",
        "content" : "Compare three approaches to multi-tenant data isolation.",
    }],
    tool_choice = "required",
    extra_body = {
        "plugins" : [{
            "id" : "fusion",
            "preset" : "general-budget",
            "model" : "~openai/gpt-latest",
        }]
    },
)
print(response.choices[0].message.content)

The preset chooses a curated panel. The nested model field selects the judge and, when you use the Fusion model alias, the model that writes the final response. Explicit analysis_models or model values override the corresponding preset settings.

tool_choice: "required" forces the model to invoke Fusion instead of letting it decide.

The available general-purpose presets currently include:

• general-high for the strongest all-around panel
• general-budget for cheaper panelists with a frontier judge
• general-fast for a panel optimized around similar response times

You can also attach the openrouter:fusion server tool to your own outer model. That path is useful when the same model needs access to Fusion alongside your application’s other tools.

See the Fusion Router documentation for the full request formats and the Presets guide for reusable configuration management.

Conclusion

Fusion gives difficult prompts more than one attempt and gives your application a structured way to compare those attempts. That can improve research, critique, and high-cost decisions where a fast first answer isn’t enough.

The additional scrutiny has a measurable price. The panel and judge add tokens, cost, latency, and output variance. Those costs are justified when they reduce retries, manual comparison, or the risk of acting on an incomplete answer.

Start with one hard prompt from your real workload. Compare Fusion with your current model using cost per accepted result, not model price alone.

Try it with the Fusion Router, or read the Fusion benchmark announcement for the full DRACO methodology and results.

Frequently asked questions

How does OpenRouter Fusion work?

OpenRouter Fusion lets a calling model send a prompt to several panel models in parallel. A judge compares the responses and returns structured analysis covering consensus, contradictions, partial coverage, unique insights, and blind spots. The calling model uses that analysis to write the final answer.

What is model fusion in AI?

Model fusion can refer to combining model parameters or combining outputs from several models. OpenRouter Fusion uses output-level deliberation during inference. It gathers independent responses, compares them through a judge, and gives the resulting analysis to the model that writes the final response.

Is OpenRouter Fusion better than Fable 5?

Fusion outperformed Fable 5 on its own in several frontier-panel configurations on the DRACO deep-research benchmark by Perplexity AI. A budget panel scored 64.7%, compared with Fable 5 at 65.3%, while a frontier panel scored 69.0%. Fable-based results reflect 93 of 100 tasks because of content filters. These results apply to deep research and don’t establish that Fusion replaces Fable across long-horizon or general workloads.

Does OpenRouter Fusion have an API?

Yes. You can call the OpenRouter Fusion API with the openrouter/fusion model slug or attach Fusion as an openrouter:fusion server tool. The fusion plugin configures the panel and judge for either entry point but does not start a Fusion run on its own. Both entry points use the same underlying panel, judge, and final-answer pipeline.

Original source

Tl;dr

• A preset stores a model or fallback array, a system prompt, provider routing, and sampling parameters under one name.
• Use it in any request as "model": "@preset/your-preset-name".
• Update it in the dashboard, and every app that uses it changes, no need for a code update.

What is an OpenRouter preset?

An OpenRouter preset is a named, versioned config that stores your model choices, a system prompt, provider routing preferences, and sampling parameters. We let you use it in any API request through its slug. This keeps your LLM setup out of your application code.

Think of a preset the way you would think of a .env file or a Terraform module, where a config lives outside your application logic and gets referenced by name rather than copied into it. The difference is where it lives and who can change it. A .env file still ships with your repo and needs a redeploy to update. We store the preset in the OpenRouter dashboard instead, so changing it immediately updates every app that references it.

A single preset can hold:

• Model selection, one model or an ordered list of fallback models.
• System prompt, the instructions added before every request.
• Provider routing, sort by price or latency, and include or exclude providers.
• Sampling parameters, temperature, top_p, and other settings.
• Tools, including OpenRouter server tools such as web search, image generation, advisors, and subagents.

Preset vs. hardcoding your request body

Hardcoded request | @preset/name
Where config lives: In each codebase | Once in the dashboard
Change the system prompt: Edit and redeploy each app | Edit the preset; no redeploy
Model fallbacks: Repeat the array everywhere | Define it once in the preset
Who can change it: Anyone who can ship code | Anyone with dashboard access

How to create your first preset

Step 1: Open the preset editor

Go to openrouter.ai/settings/presets and create a preset. Pick a slug you’ll remember, because you’ll use it as @preset/your-slug.

Step 2: Choose model and routing

Pick one model or add an ordered list. If the first model fails from a rate limit, outage, or too much context, we try the next one. You set the fallback order once in the preset instead of repeating it in every request.
Set provider routing here too, as seen in the screenshot above. You can sort by price or latency, or block a provider from your requests.

Step 3: Add a system prompt and sampling settings

Add your system prompt and set temperature, top_p, and any other sampling parameters. These become the defaults for every request that uses the preset.

Step 4: Reference it from an API call

Set the model field to your preset. Compare these requests.

Before, with every parameter in the request:

from openrouter import OpenRouter
client = OpenRouter(api_key=OPENROUTER_API_KEY)
resp = client.chat.send(
    model="anthropic/claude-opus-4.8",
    provider={"sort": "price"},
    temperature=0.7,
    messages=[
        {"role": "system", "content": "You are a concise technical writer..."},
        {"role": "user", "content": "Explain preset versioning."},
    ],
)

After, with a preset reference:

resp = client.chat.send(
    model="@preset/tech-writer",
    messages=[
        {"role": "user", "content": "Explain preset versioning."},
    ],
)

You can also create a preset through the API with a POST request to the preset endpoint. We save the fields that belong to the preset config and ignore transient fields like messages, stream, and prompt.

You can override preset settings in each request. If the request body includes temperature, that value wins over the preset value. The merge is shallow, so request fields replace matching preset fields, and we keep any preset fields you leave out. tools is the exception: we combine the preset’s tools with the request’s tools, and a request tool replaces a preset tool with the same name. You can also use a separate preset field ("preset": "@preset/tech-writer") or the combined form ("model": "anthropic/claude-opus-4.8@preset/tech-writer"). The preset field needs the full @preset/ prefix. A bare slug is ignored.

Can you set a default preset?

Not for the API. Presets have versions. Saving an existing slug creates a new version and makes it active, but there is no global default preset for the API. Every request must name a preset. The Chatroom does have a Default Preset setting under Settings > Preferences that applies to new chat messages.

For the API, put your default behavior in a preset and use that preset everywhere. If you want short answers, add that rule to the preset’s system prompt (“Answer in three sentences or fewer unless asked to expand.”) and set the sampling parameters. Every service that uses @preset/short-answers will follow those defaults unless a request overrides them.

Using a preset by name gives you steady behavior you can change in one place. The same pattern extends to more complex setups. Here are two worked examples.

Use case 1: An image-prompt-enhancement preset

Image models often do better with detailed prompts. Instead of adding that detail by hand each time, pair a text model with the image-generation tool in a preset. The text model refines each prompt before it goes to the image model.

The preset stores a text model, a system prompt that turns a short request into a full visual brief covering subject, composition, lighting, palette, and style, and the image-generation tool.

Once the preset is saved, test it directly in the Chatroom by opening the preset and selecting “Test in Chat.” Send a short prompt like “a cozy reading nook by a rainy window,” and the preset applies its system prompt and image tool. The text model expands your short prompt into a full visual brief, then calls the image tool automatically.

The preset stores the tools array and tool_choice along with the model, system prompt, and sampling parameters.

Every app that uses @preset/image-enhancer gets the same behavior. The preset-enhanced images cookbook walks through the full process.

Use case 2: Pinning a Fusion configuration as a preset

A Fusion setup runs a panel of models alongside your primary model, and the primary model writes the final answer using the panel’s outputs. Store the entire setup in a preset, then use it via a single reference.

Store the openrouter:fusion tool, with its analysis_models panel and analyst model, in the preset’s tools, then call @preset/fusion-panel from anywhere. Your web app, evaluation tools, and Slack bot all use the same panel. You adjust it in the dashboard instead of editing three codebases. For the exact fields a Fusion configuration uses and how the panel works, see the Fusion docs.

This also gives ML engineers a repeatable evaluation config with the same models, sampling settings, and routing under one stable slug.

Managing presets over time

Each time you save a preset with an existing slug, our system creates a new version and makes it active. API requests that use the preset get the active version.

Change the system prompt in the dashboard, and every app using that slug picks up the change on its next request. No redeploy. If the change hurts quality, restore an earlier version in the dashboard. New requests use it right away.

A product manager can change the system prompt or model in the dashboard without asking an engineer to edit code or ship a new build.

Presets across apps and teams

One preset can serve an API integration, a batch job, and a third-party app that supports the OpenAI-compatible API. Because presets use the standard model field, any client that can set a model string can use one.

For larger teams, a preset gives you one versioned place for model choices, routing, and prompts. It replaces constants scattered across several repos.

If you’re using an organization account, all members can access organization presets, which makes them a good way to share best practices across teams.

Conclusion

Three things to remember:

• A preset stores a model, system prompt, routing rules, and sampling settings under one name. Use it as @preset/name in any request.
• Updating a preset changes every app that uses it, with no redeploy. You can restore an earlier version if needed.
• The same shared config can power image-prompt enhancement and Fusion panels across many apps.

Create your first preset at openrouter.ai/settings/presets. Then try the preset-enhanced images cookbook.

Frequently Asked Questions

How do I change the model in OpenRouter?

Change the model in the preset at openrouter.ai/settings/presets. Every call that uses @preset/your-preset-name will use the new model on its next request. Without a preset, you have to change the model field in each request body.

What is OpenRouter model usage?

We give you one API endpoint for hundreds of models from Anthropic, OpenAI, Google, Meta, and other providers. Pick a model per request or use a preset, then pay each model’s token rate. See the models list for current options and prices.

What is the API limit for OpenRouter?

Rate limits vary by model and provider. A preset doesn’t change them. A preset only stores config, the model, prompt, routing, and sampling settings. The same limits apply whether you use a preset or put the parameters directly in each request. See the limits docs for current figures.

Is OpenRouter compatible with OpenAI?

Yes. We offer an OpenAI-compatible API. Presets use the standard model field, so any OpenAI SDK client can use them by setting "model": "@preset/your-preset-name".

Can a preset include fallback models?

Yes. A preset can hold an ordered list of models. If the first fails from a rate limit, outage, or too much context, we try the next one. You set the order once in the preset instead of in each request.

Can I override a preset’s settings per request?

Yes. Parameters in the request body win over preset values. The merge is shallow, so request fields replace matching preset fields, and we keep any preset fields you leave out. Tools are combined, with request tools replacing preset tools of the same name.

Do presets work through the API or only in the Chatroom?

Presets work through the OpenAI-compatible API. You can use @preset/slug in the model field, a separate preset field set to @preset/slug, or the combined model@preset/slug form.

Original source

Seedance 2.5 has been live on our video API since August 7, 2026, and as of September 3, 2026, the model page lists it from $0.1028 per second of generated video, which is what 480p works out to.

That per-second figure is derived, not fixed. Billing is per video token, and the token count scales with output pixels as well as duration, so a second of 720p costs a little over twice a second of 480p from the same model.

Tl;dr

• Best at long single takes and at work that starts from existing footage. Clips run to 30 seconds, and input_references accepts image, video, and audio assets, so an existing clip can be edited or extended rather than regenerated.
• Live specs as of September 3, 2026: clips of 4 to 30 seconds, 480p or 720p, six aspect ratios, first and last frame control, and audio generated in the same pass.
• Cost runs about $0.103 per second at 480p and $0.231 at 720p, derived from the billing formula of (width x height x fps x duration) / 1024 video tokens at 24 fps, billed at $0.0000107 apiece.
• A request that carries a video reference bills at $0.0000064 per token, about 40% below the base rate, which makes editing and extension cheaper per second than generating from scratch.
• Audio costs nothing extra, since we bill the same rate whether generate_audio is on or off. That isn’t true of Veo 3.1 or Seedance 1.5 Pro, which both charge less for silent output.
• Use a different model when you need 1080p or 4K, when you want the lowest price per second at the same clip length, or when you need frame-exact reproducibility.
• The slug is bytedance/seedance-2.5. Higher resolutions belong to bytedance/seedance-2.0, and bytedance/seedance-2.0-fast and bytedance/seedance-2.0-mini are the cheaper draft slugs.

What Seedance 2.5 is

Seedance 2.5 is ByteDance’s video generation model, and it takes several kinds of input into one video output. You can generate from a text prompt alone, from images that fix the first or last frame of a shot, or from reference assets that steer the result without pinning a specific frame. Our model page describes it as suited to long-form storytelling, reference-based generation, video editing, and video extension, and those four jobs cover most of what it’s used for.

Here’s the current spec snapshot from our video models endpoint, so you can see where those capabilities stop in practice.

• Slug: bytedance/seedance-2.5
• Released: August 7, 2026
• Clip length: 4 to 30 seconds
• Resolutions: 480p, 720p
• Aspect ratios: 16:9, 4:3, 1:1, 3:4, 9:16, 21:9
• Frame control: frame_images accepts first_frame and last_frame
• References: input_references accepts image, video, and audio assets
• Audio: generate_audio, defaults to true
• Seed: Accepted, though determinism isn’t guaranteed
• Provider passthrough keys: watermark, req_key, output_format
• Price: $0.0000107 per video token, or $0.0000064 with a video reference

Last checked September 3, 2026 on the video models endpoint.

The endpoint also lists the exact output sizes, which is useful when you’d rather send size than a resolution and an aspect ratio: 854x480, 752x560, 640x640, 560x752, 480x854, 992x432 at 480p, and 1280x720, 1112x834, 960x960, 834x1112, 720x1280, 1470x630 at 720p.

Five Seedance slugs, and which one you want

The model name is the human-readable label, ByteDance: Seedance 2.5, while the slug is the string you put in the model field, bytedance/seedance-2.5. We carry five, and they aren’t interchangeable. Seedance 2.5 is the newest and the longest. Seedance 2.0 is the higher-resolution model, with clips of 4 to 15 seconds at 480p through 4K for $0.000007 per video token at 480p and 720p. Seedance 2.0 Fast and Seedance 2.0 Mini run to 15 seconds at a 720p ceiling for $0.0000042 and $0.0000035 per token, which makes them the draft slugs. Seedance 1.5 Pro is the older model that generates video and audio in a single pass, stops at 1080p and 12 seconds, and bills $0.0000024 per token with audio or half that without.

What it doesn’t do

Seedance 2.5 lists no 1080p and no 4K output. If you need either, that’s Seedance 2.0 in the same family, or Veo 3.1 outside it. This is the one spec where the newer version is narrower than the older one, and it’s worth checking against your delivery format before you build around 2.5.

What it’s best at

The 4-to-30 second window and the reference inputs describe what the model accepts. The reason to choose it over the rest of the family is narrower than that. It’s the Seedance model for length and for footage you already have.

Four optional controls, each reducing how much the model can change on its own. Frame images and reference assets select different modes rather than combining, and frame images take priority if both are sent.

Thirty seconds in one generation

Thirty seconds is the longest single generation we carry, matched only by Wan 3.0 and Wan 3.0 Prime. Every other model that publishes a duration range stops at 20 seconds or less. That matters because the alternative to a long take is stitching short ones, and stitching is where continuity breaks down. If your deliverable is a 30-second spot or a single continuous scene, this is the shortest path to it.

References in three modalities

input_references accepts image, audio, and video assets on Seedance generation 2 and newer, which includes 2.5. Image references carry a face, a product, or a style. A video reference gives the model existing footage to edit or extend. An audio reference gives it a track to work against. Our model page lists up to 50 reference assets per request, which is a model page figure rather than a limit our endpoint publishes, so treat the exact ceiling as reported rather than measured.

Video and audio references work on every Seedance model from generation 2 onward, so this isn’t unique to 2.5 inside the family. It does separate the family from the rest of the catalog. Wan 3.0, Veo 3.1, and Kling v3.0 Pro all take image references only.

Editing and extension bill at a lower rate

A request that carries a video reference and no frame images bills at $0.0000064 per video token instead of $0.0000107, about 40% less. At 720p that’s $0.138 per second instead of $0.231. So the cheapest way to get 30 seconds of 720p out of this model is to extend footage you already have rather than to generate it cold.

Both ends of a shot, not just the first

frame_images accepts entries typed as first_frame and last_frame, so you can fix where a shot begins and ends and let the model fill in the motion between them. Wan 3.0, the other 30-second model, lists first_frame only. If a request carries both frame images and reference assets, the frame images take priority and the job is treated as image-to-video, so the references will have no visible effect and the request bills at the base rate.

Audio in the same pass, at no extra charge

generate_audio defaults to true, and we price video tokens identically whether audio is on or off, so muting a generation saves nothing. Veo 3.1 charges $0.40 per second with audio against $0.20 without, and Seedance 1.5 Pro halves its own rate for silent output. The model page also lists multilingual audiovisual generation, so a line in a language other than English is worth attempting here before you budget for a separate voice pass.

Where it fits, with prompts you can paste

Length, references, and included audio fit a specific kind of job: one continuous scene, or a change to footage that already exists. Three cases fit that description well.

One long take

This is the strongest fit, since 30 seconds is enough for a full scene and the model doesn’t need you to cut. Write the beats in order and give the camera one instruction per beat.

Extending or editing existing footage

Send the clip as a video reference and describe only what should change or what comes next. This is the request shape that bills at the lower video-input rate.

Dialogue and talking heads

Short lines sync better than long ones, but 30 seconds gives you room for two or three of them in one take instead of one line per generation.

Prompt patterns that transfer

• Order the beats and time them. A 30-second prompt reads more like a shot list than a description.
• Name what must stay fixed. Face, wardrobe, product proportions, lens, and lighting are all worth stating explicitly rather than assuming a reference carries them.
• Name the camera move and the lens. Push-in, handheld, 35mm, and shallow depth of field are instructions the model acts on.
• Draft at 480p and finish at 720p. The two resolutions differ by more than a factor of two in price, but not in how the prompt is written.

Calling it from your own code

Those prompts go into a request body rather than a chat message, because video generation doesn’t run on /chat/completions. It has a dedicated asynchronous endpoint, so you submit a job to POST /api/v1/videos, poll the polling_url we return until the status reads completed, then download the result with your API key. Generation usually takes from 30 seconds to a few minutes, and a 30-second polling interval is a reasonable default. Video models also don’t appear in the plain models list, so use /api/v1/videos/models or the video model collection to find them.

The minimal text-to-video call is a submit and a poll.

Seedance 2.5 accepts a seed, but determinism isn’t guaranteed by every provider. Check the video generation docs before you build a workflow that depends on repeatable output. Provider-specific options travel under provider.options..parameters, where is the provider slug. Seedance runs on one provider, seed, and its allowed passthrough keys are watermark, req_key, and output_format.

What a clip actually costs

Those requests bill on a formula rather than a flat per-second rate, which is the single most useful thing to understand before you scale up. The token count is (width x height x fps x duration) / 1024 at 24 fps, and every token bills at $0.0000107, or $0.0000064 when the request carries a video reference. So the only variables are pixels, seconds, and whether you’re supplying footage, and you can price any clip before you run it.

• 480p (854x480): about 9,600 video tokens per second, $0.103 per second, $1.03 for 10-second clip, $3.08 for 30-second clip, $0.062 per second with a video reference
• 720p (1280x720): 21,600 video tokens per second, $0.231 per second, $2.31 for 10-second clip, $6.93 for 30-second clip, $0.138 per second with a video reference

Last checked September 3, 2026. Figures are calculated from the token formula and the per-token rates on the video models endpoint. Vertical formats cost the same as landscape at equal pixel counts, so 720x1280 prices match 1280x720.

Measured against Veo 3.1 with audio at $0.40 per second, Seedance 2.5 at 480p is about four times cheaper and at 720p about 1.7 times cheaper. Measured against Wan 3.0 at $0.05 per second at 480p and $0.10 at 720p, Seedance 2.5 costs about twice as much at both. It isn’t the cheapest way to generate video. It’s the cheapest way to generate a 30-second clip with last-frame control, video references, or audio references.

Seedance 2.5 against Seedance 2.0, Wan 3.0, and Veo 3.1

Once you can price a clip, the comparison comes down to which requirement you hit first. All four models below are in our catalog, and every figure comes from the same endpoint on the same day.

• Seedance 2.5 (bytedance/seedance-2.5): 4 to 30 seconds, 480p and 720p, 6 aspect ratios, first and last frame control, image, video, and audio references, seed accepted, $0.103 per second at 480p, $0.231 at 720p
• Seedance 2.0 (bytedance/seedance-2.0): 4 to 15 seconds, 480p to 4K, 7 aspect ratios, first and last frame control, image, video, and audio references, seed accepted, $0.067 per second at 480p, $0.151 at 720p, $0.374 at 1080p, $0.778 at 4K
• Wan 3.0 (alibaba/wan-3.0): 2 to 30 seconds, 480p to 1080p, 5 aspect ratios, first frame control only, image references only, seed accepted, $0.05 per second at 480p, $0.10 at 720p, $0.20 at 1080p
• Veo 3.1 (google/veo-3.1): fixed lengths 4, 6, or 8 seconds, 720p to 4K, 2 aspect ratios, first and last frame control, image references only, seed accepted, $0.40 per second at 720p and 1080p, $0.60 at 4K

Last checked September 3, 2026 on the video models endpoint. Seedance per-second figures are derived from the token formula, while Wan 3.0 and Veo 3.1 publish flat per-second SKUs on that same endpoint, so the figures are comparable but reached by different routes.

Seedance 2.5 wins on inputs and on length together. It’s the only model here that takes 30 seconds, both frame types, and video and audio references in the same request. Seedance 2.0 is the one to use for 1080p and 4K, and it’s cheaper per second at every resolution the two share. Wan 3.0 matches the 30-second ceiling at half the price and reaches 1080p, but it lists first-frame control only and no video or audio references, so it’s the better default for long text-to-video and the worse one for editing. Veo 3.1 is the fixed-length option, at $0.60 per second for 4K, and Veo 3.1 Fast generates 4K for $0.30 per second if you want the cheaper route to that resolution.

Because all four sit behind the same POST /api/v1/videos call and the same key, running your own comparison is a one-field change rather than four subscriptions. The video model collection lists everything we carry with the modality filter already applied.

You don’t have to write code to compare them. Our video benchmarks page runs the same prompt through every video model we carry, including Seedance 2.5, Seedance 2.0, Wan 3.0, and Veo 3.1, and shows each output next to what it cost and how long it took to generate. Watch the clips, sort by cost or generation time, then pick the models you want to try in chat. Use it to check the claims in this review before you spend anything.

The verdict

Seedance 2.5 is the right choice when clip length, existing footage, or included audio matter more than resolution. Thirty seconds in one generation, image, video, and audio references, both frame types, and a lower rate for video-referenced requests are a combination nothing else in our catalog offers. Use Seedance 2.0 when you need 1080p or 4K, Wan 3.0 when you want 30 seconds of text-to-video for less, and Veo 3.1 when a fixed 4, 6, or 8-second 4K clip is the deliverable. Teams that need repeatable, approval-ready output should treat its results as material to review rather than as finals.

FAQ

Is Seedance 2.5 actually good for videos?

Yes, for longer single takes and for work that starts from existing footage. Seedance 2.5 generates clips of 4 to 30 seconds, accepts image, video, and audio references, supports first- and last-frame control, and generates audio in the same pass at the same rate. It’s a weaker choice when you need 1080p or 4K output, the lowest price per second, or frame-exact reproducibility.

What can Seedance 2.5 do?

Seedance 2.5 generates video from a text prompt, from images that fix the first or last frame, or from reference assets that carry a subject, a style, or existing footage into a new clip. As of September 3, 2026, it produces clips of 4 to 30 seconds at 480p or 720p, across six aspect ratios, with audio by default and a seed parameter to narrow variance between runs. The model page also lists video editing and video extension, and up to 50 reference assets per request.

How much does Seedance 2.5 cost per video?

Seedance 2.5 bills per video token, and tokens are (width x height x fps x duration) / 1024 at 24 fps, so the price scales with both pixels and seconds. The rate is $0.0000107 per token, which works out to about $0.103 per second at 480p and $0.231 at 720p, so a 10-second 720p clip costs roughly $2.31. Audio is included at the same rate. A request that carries a video reference bills at $0.0000064 per token instead, about 40% less.

Which AI video model is best?

There’s no single best video model, and the right answer depends on which requirement you hit first. Seedance 2.5 is the pick for 30-second clips that need last-frame control or video and audio references. Seedance 2.0 is the pick for 1080p and 4K. Wan 3.0 also runs to 30 seconds, costs less per second, and reaches 1080p, but it lists first-frame control only. Veo 3.1 is the pick for fixed 4, 6, or 8-second clips at 4K. All of them run through the same asynchronous video endpoint, so you can compare them on your own prompt by changing one field. The video model collection lists the full set.

How do I access the Seedance 2.5 API?

Send a POST request to https://openrouter.ai/api/v1/videos with bytedance/seedance-2.5 in the model field and your prompt in prompt, then poll the polling_url from the response until the status is completed and download the result with your API key. Generation is asynchronous and usually takes from 30 seconds to a few minutes. The full request schema, including webhooks, is on the video generation page.

Does Seedance 2.5 do audio?

Yes. Seedance 2.5 generates audio in the same pass as the video, and generate_audio defaults to true. We bill the same per-token rate with audio on or off, so there’s no cost reason to disable it. That isn’t true of Veo 3.1 or Seedance 1.5 Pro, which both charge less for silent output. The model page also lists multilingual audiovisual generation, so it’s worth testing a line in your own target language.

What’s the difference between Seedance 2.5 and Seedance 2.0?

Seedance 2.5 runs twice as long and stops at a lower resolution. It generates 4 to 30 second clips at 480p or 720p, accepts image, video, and audio references, and bills $0.0000107 per video token. Seedance 2.0 generates 4 to 15 second clips at 480p through 4K and bills $0.000007 per token at 480p and 720p, so it’s cheaper per second and the only one of the two that reaches 1080p or 4K. Pick 2.5 for length and for editing existing footage, and 2.0 for resolution.

References

• ByteDance: Seedance 2.5 model page. Entry price, release date, per-second rates, reference asset count, and capability description.
• Video models endpoint (/api/v1/videos/models). Supported durations, resolutions, aspect ratios, output sizes, frame image types, seed support, passthrough keys, and pricing SKUs for every video model cited here.
• Video generation documentation. Request schema, asynchronous job lifecycle, provider passthrough, webhooks, and the Zero Data Retention exclusion.
• Announcing Video Generation. Launch scope for the video endpoint and the models supported on day one.
• Video model collection. The catalog filtered to video output, used for the comparison set.
• Video benchmarks. Side-by-side outputs from every video model on a shared prompt, with cost and generation time per model.
• ByteDance: Seedance 2.0 model page. Resolution range, clip ceiling, and per-token rates for the higher-resolution model in the family.
• Alibaba: Wan 3.0 model page. Clip ceiling, resolutions, frame control, and per-second pricing for one of the other two 30-second models.
• Google: Veo 3.1 model page. Fixed clip lengths and per-second pricing with and without audio.

All product sources above were last checked on September 3, 2026. Creator coverage is cited as framing only, not as a source of specs or prices.

Original source

Today we’re launching US In-Region Routing alongside the EU routing we shipped last October, to give you guaranteed control over where your AI workloads are processed.

Requests to us.openrouter.ai are decrypted inside the US and routed only to US provider endpoints, so your prompts and completions stay in-region for the entire lifecycle of the request. eu.openrouter.ai does the same for the EU.

Companies need stronger guarantees around data residency and processing. OpenRouter’s In-Region Routing makes it easy to set up for the US or EU.

How it works

Send API requests through the region-specific base URL instead of openrouter.ai :

https://us.openrouter.ai/api/v1
https://eu.openrouter.ai/api/v1

Your API key, request body, and model IDs are unchanged, and your provider preferences, fallbacks, and privacy settings carry over from your account. You can point one service at us.openrouter.ai while the rest of your traffic keeps using openrouter.ai.

Requests sent to us.openrouter.ai or eu.openrouter.ai are decrypted and processed inside that region, and only providers running in that region can serve them. If no in-region provider offers the model, the request fails with a 404 (No endpoints found supporting your data region.) rather than being routed outside the region.

The regional endpoints serve a subset of the global catalog under the same model IDs. A model is listed for a region if at least one provider can serve it from inside that region. For the live list, call /api/v1/models through the regional domain, or filter the models page by In-Region Routing (US or EU). We will continue to add models and providers to both regions. Tell us which models you want in-region next in #feedback on Discord.

You can also enforce In-Region Routing for a specific workspace, team, or API key with Guardrails. Set the guardrail’s allowed data regions to one region, and OpenRouter rejects any covered request that arrives on another hostname. Configure it under Guardrails in your settings or through the Management API.

In-Region Routing is available on the Business and Enterprise plans.

US/EU models served in-region

Without In-Region Routing, using a model from a lab based in the US or EU doesn’t guarantee that a data center in that region serves the request. At global endpoints, models from OpenAI, Anthropic, Google, xAI, Meta, NVIDIA, Thinking Machines, or Mistral can run anywhere the provider serving the request operates. A data residency review asks where your request was decrypted and where inference ran, so a US team using GPT-5.6, Claude Opus 5, or Gemini 3.6 Flash on a global endpoint can still fail a compliance review. OpenRouter In-Region Routing ensures the request stays within the region you specify: US or EU.

Chinese open-weight models available in-region

Open-weight models are improving rapidly, and shifting traffic to them can significantly reduce costs. On OpenRouter, the share of tokens going to open-weight models has risen steadily for requests originating in both the US and EU:

Models from US labs, like NVIDIA’s Nemotron 3 Ultra and Thinking Machines’ Inkling, are part of that growth. But models from Chinese labs are still most of the volume, and procurement approval for those models can be difficult.

In-Region Routing allows teams with data residency requirements to get the price and performance gains from Chinese open-weight models. When a US or EU provider hosts a model, requests go to that provider and the lab is not involved. DeepSeek V4 Pro, Kimi K3, and GLM 5.2 are all available for US In-Region Routing because Baseten, Fireworks, and Azure serve them from US data centers. Send a prompt to any of them through us.openrouter.ai and it is decrypted and run in the US. GLM 5.2 is also available at eu.openrouter.ai, served from Mistral’s EU data centers.

Ensuring end-to-end regional routing

When a gateway offers regional routing, it can mean one of two things. The difference is whether your prompt ever exists in plaintext outside the region.

• Inference-only regional routing. The gateway pins the provider’s inference to your region. But the request itself is decrypted and processed wherever the gateway runs before it is forwarded to the provider. For that part of the path, your prompts are in plaintext outside your region. Server tools are often overlooked for regionalization, which can lead to your prompt data leaving the jurisdiction. For example, web search sometimes uses global instances on regional inference endpoints.

• End-to-end regional routing. This is how OpenRouter’s In-Region Routing works. It keeps your data in-region for the full path. Requests to us.openrouter.ai or eu.openrouter.ai are decrypted inside the region and processed there at every step. Only providers running in that region receive them. If no in-region provider can serve the model, the request fails with a 404 (No endpoints found supporting your data region.) It never leaves the region. We comprehensively evaluate server tools and the data jurisdictions they run in before offering them in a specific jurisdiction. Tools that would send data outside the region are disabled rather than falling back to global infrastructure.

When you evaluate any gateway’s data residency claim, ask two questions. Where is the request decrypted and processed? And where do its tools run?

Get started

In-Region Routing is available on our Business and Enterprise plans. Upgrade to Business from your plan settings, or talk to the enterprise team.

Then pick a model from the US or EU list and send your requests to https://us.openrouter.ai/api/v1 or https://eu.openrouter.ai/api/v1.

Read the docs for more info.

Original source

Introducing the openrouter:shell server tool and the Files API

any model on OpenRouter can now run commands in a hosted Linux container. The Files API enables upload of files for models to work with and the download of outputs. Both are available today in beta.

Shell and Files join our growing list of server tools, enabling you to create server-side agentic behaviors you can swap across models. For example, you can ask any model to search the web, write a script that turns the results into a chart, and run it entirely with server-side compute.

Try it in the chatroom by turning on the shell tool, and read the shell, containers, and Files API guides for the API details. Sandbox time costs $0.0001 per second, is billed as part of the request, and includes Files API usage. Details are in the Pricing section.

How shell works

To use openrouter:shell, send it in the tools array for any model that supports tool calling. This lets the model decide when it needs a terminal and when to invoke it:

curl https://openrouter.ai/api/v1/responses \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "deepseek/deepseek-v4-pro-0813",
  "input": "Check the Python version, then write a script that prints the first 20 primes and run it.",
  "tools": [
    { "type": "openrouter:shell", "parameters": { "engine": "openrouter" } }
  ]
}'

We’ve introduced three capabilities that work together to provide server-side command execution and files:

• Shell and Bash: We support the OpenAI-compatible Shell tool, on the Responses API and the Anthropic Messages API, as well as openrouter:bash, the Anthropic-compatible Bash tool, on the Messages API. Both work with any model.
• Files API: workspace storage under /api/v1/files. Upload files, attach them to a container by id, and keep the files a run produces.
• Containers: these are the sandbox where shell commands run. Files written there persist between requests that share a container id. You can access the container’s contents through the /api/v1/containers API.

When the model calls the tool, it emits a batch of commands. They are executed within the container, each in its own invocation, and return stdout, stderr, and an exit code to the model. This allows the model to react to the output it receives. For example, if it writes a script to parse your CSV and the parse fails, it can see the problem on stderr and fix the script before answering.

The generation details view on the Logs page shows the request as a timeline. The model turn and the sandbox run appear as separate rows, each with its own duration and cost.

Shell and bash

We shipped two different tools for sandbox command execution to provide compatibility with both the OpenAI and Anthropic spec. The most notable difference is that the bash tool’s default is to ask your app to run the command locally. On OpenRouter, you can change the engine to override this behavior and execute on the server.

openrouter:shell openrouter:bash Compatible with OpenAI’s shell tool Anthropic’s bash tool APIs Responses, Messages Messages Runs commands by default In the OpenRouter sandbox In your application

engine: "openrouter" on either tool guarantees server-side execution in the OpenRouter sandbox with any model.

Containers

A container is an isolated Linux environment on OpenRouter’s infrastructure scoped to your workspace. Containers can be configured for the needs of your app:

• Network: outbound access is off by default. For jobs like pip3 install, set network_policy to an allowlist such as { "type": "allowlist", "allowed_domains": ["pypi.org", "files.pythonhosted.org"] }, or { "type": "allowlist", "allowed_domains": ["*"] } for unrestricted egress. Allowlisted hosts are reachable on ports 80 and 443; requests to domains outside the allowlist fail with HTTP 520 rather than a connection error. The policy cannot be changed after the container starts.
• Files: only files under the home directory (/workspace/home) are captured. Each shell result also returns a list of ids for the files a command created or changed (prefixed with cfile_). The container files endpoint lists everything saved in the container, and the Files API is for moving files in and out.
• Reuse across requests: by default, a conversation gets a fresh container. If a request includes a session_id or a previous shell result with an identifiable container, that container is reused. To pick a container explicitly, pass { "type": "container_reference", "container_id": "my-project" } in the tool’s environment field.
• Lifetime: a container sleeps after 5 minutes idle. This is not configurable.

Files and shell together

The Files API is workspace storage that sits alongside the container. You upload inputs there for shell to work on, and you move shell’s outputs back into it.

Upload a file for shell

Upload the input with POST /api/v1/files. The response includes a file id that starts with or_file_:

curl https://openrouter.ai/api/v1/files \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-F "file=@data/sales.csv"

Then attach it by id in the tool’s environment:

{
  "type": "openrouter:shell",
  "parameters": {
    "engine": "openrouter",
    "environment": {
      "type": "container_auto",
      "file_ids": ["or_file_011CNha8iCJcU1wXNR6q4V8w"]
    }
  }
}

Attached files appear in the home directory as writable copies, up to 20 per container. Each copy is named with the last 8 characters of the file id plus the original filename, so data/sales.csv attached with the id above becomes ~/NR6q4V8w-sales.csv. Changes inside the container won’t affect the original workspace file. A container starts with only the files you attach to it.

Download a file shell generated

Each shell result lists the files the command touched, with a cfile_ id for each. Download the files with the container file content endpoint:

curl "https://openrouter.ai/api/v1/containers/$CONTAINER_ID/files/$FILE_ID/content" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-o output.txt

Container files are kept for 30 days. To keep one for the long term, promote it:

curl -X POST "https://openrouter.ai/api/v1/containers/$CONTAINER_ID/files/$FILE_ID/promote" \
-H "Authorization: Bearer $OPENROUTER_API_KEY"

Promoting copies the container file into your workspace and returns a new or_file_ id, which you can attach to a later run the same way as an upload. Unlike uploads, promoted files are downloadable through the Files API.

Files API details

You can view all of your files on the workspace files page. Files you upload directly cannot be downloaded, but files promoted from a container can.

Using multiple server tools together

Shell is one of many server tools we offer and are powerful when working together. Here the model uses web search to find material and shell to turn it into a file:

curl https://openrouter.ai/api/v1/responses \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "deepseek/deepseek-v4-pro-0813",
  "input": "Look up the three biggest open-source AI releases this week, then write ~/out/releases.md with one paragraph each and a source link.",
  "tools": [
    { "type": "openrouter:web_search" },
    { "type": "openrouter:shell", "parameters": { "engine": "openrouter" } }
  ]
}'

The resulting ~/out/releases.md shows up in the shell result’s file list, and you download it with the container file content endpoint above.

This combination also matters if you don’t want to give the container network access. Web search runs outside the container, so the model can pull in web content and pass it into its commands while the container stays on the default network policy with no internet access of its own.

In the chatroom, the same combination works with the shell and web search switches turned on. Files the run creates appear in the conversation as downloads.

Pricing

Shell and Bash usage is billed by sandbox time. The price is $0.0001 per active second, metered from the moment a request first runs a sandbox command until the last sandbox command. Time a container spends idle after the request ends is not billed.

We bill a minimum of 30 seconds when a request starts a cold container, either a new one or one that has gone idle. If an agent makes several requests to the same container in succession, only the first pays the minimum.

Billing per request makes it easy to find the cost to run a specific request. A request’s cost is its token cost plus its sandbox time, and the sandbox time appears as its own row in the request’s timeline on the Logs page.

Files API usage has no separate charge, but total storage is limited to 10 GiB.

Disabling tools for a workspace

Server tools are enabled by default. A workspace admin can turn any of them off from the workspace’s Server Tools page, where each tool has a switch showing Available or Blocked. The setting applies to every request the workspace makes, whether through API keys, the chatroom, or presets.

Get started

Shell, Bash, the Files API, and containers are in beta and available now. The API may change during the beta. If something doesn’t work the way you expect, let us know in #feedback on our Discord.

• Shell server tool
• Bash server tool
• Files API
• Containers
• All server tools

Original source