Together AI Release Notes

Together SDK and CLI 3.0

Together Python SDK 3.0, TypeScript SDK 1.0, and Together CLI 3.0 are now available.

3.0 is a major release with breaking changes. The beta paths for dedicated model inference and GPU clusters no longer work, the SDK base URL drops /v1, and whoami returns a new response shape. Read SDK v3 migration before you upgrade.

What’s new:

• Dedicated model inference and GPU clusters out of beta: Manage dedicated model inference and GPU clusters with client.endpoints, client.models, and client.clusters, or with tg endpoints, tg models, and tg clusters in the CLI.
• Faster chat completions: The SDKs send chat completions directly to the inference host, https://api-inference.together.ai, by default.
• Identity for any credential: client.whoami() returns a typed identity for project API keys, organization API keys, and user sessions, including credentials that aren’t scoped to a project.
• Billing usage in the SDK: client.organization.usage.costs.list() returns your organization’s cost line items by day or hour from the billing usage API, which is in beta and enabled per organization.

New serverless models

The following models are now available on serverless:

• MiniMaxAI/MiniMax-M3.1-Flash: 1,048,576 context length, NVFP4 quantization, function calling, and structured outputs. Pricing: $0.15 input / $0.60 output / $0.015 cached input (per 1M tokens).

Applied credit types in billing usage

Billing usage line items now include an applied_credit object. Its type identifies the kind of credit that covers the usage: CREDIT (free credit), PREPAID (credit purchased in advance), or POSTPAID (credit under a paid commitment). Line items for usage that no credit covers omit the object.

Usage covered by different credits now appears as separate line items.

See Get billing usage.

Model deprecations

The following models have been deprecated and are no longer available on serverless:

• meta-llama/Llama-3.3-70B-Instruct-Turbo. Use meta-models/Muse-Glimmer-30B instead.
• deepseek-ai/DeepSeek-V4-Flash-0731. Use deepseek-ai/DeepSeek-V4.1-Flash-1017 instead.

Both models remain available through on-demand dedicated endpoints. For Llama 3.3 70B, deploy meta-llama/Llama-3.3-70B-Instruct. See Deprecations for migration options.

Geo-specific inference

You can now keep serverless inference processing in the United States. Set your client’s base URL to https://us-api.together.ai/v1, using your existing API key and model IDs. There are no GPUs to provision.

US models: moonshotai/Kimi-K3, zai-org/GLM-5.3, zai-org/GLM-5.2, deepseek-ai/DeepSeek-V4.1-Flash, deepseek-ai/DeepSeek-V4.1-Flash-1017, deepseek-ai/DeepSeek-V4-Pro-0813, and MiniMaxAI/MiniMax-M3.

US-only inference costs 1.25× standard serverless rates (a 25% premium) for input, cached input, and output tokens. See Geo-specific inference for details.

New serverless models

The following models are now available on serverless:

• reflection/Beam-1.0-501B-A23B: 1,048,576 context length, NVFP4 quantization, function calling, and structured outputs. Pricing: $0.40 input / $1.20 output / $0.04 cached input (per 1M tokens).

New dedicated endpoint models

The following models are now available for deployment on dedicated endpoints:

• reflection/Beam-1.0-501B-A23B.

New serverless models

The following models are now available on serverless:

• meta-models/Muse-Spark-Open-816B-A42B: 1,048,576 context length, NVFP4 quantization, function calling, and structured outputs. Pricing: $1.25 input / $4.25 output / $0.15 cached input (per 1M tokens).
• nvidia/NVIDIA-Nemotron-3.5-Super-VL: 262,144 context length, NVFP4 quantization, image input, function calling, and structured outputs. Pricing: $0.15 input / $0.50 output / $0.02 cached input (per 1M tokens).

New dedicated endpoint models

The following models are now available for deployment on dedicated endpoints:

• nvidia/NVIDIA-Nemotron-3.5-Super-VL.

Post-training

Post-training lets you write your own training loop and run it against Together AI GPUs. Your code builds each batch and chooses the loss function and learning rate, while the GPUs you provision run the forward and backward passes, optimizer steps, and sampling. It bills per GPU-minute.

Write a loop with the Python SDK (together.post_training, version 2.41.0 or later), or start from a recipe in the post-training cookbook.

What’s included

• Three training methods: Supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning with group relative policy optimization (GRPO). Each method trains a low-rank adapter (LoRA) or the full model weights.
• Custom loss functions: Train with any objective you compute in PyTorch from the model’s log-probabilities.
• Multi-turn environments: Train tool-use agents with GRPO over whole episodes.
• Several adapters on one base model: Attach several LoRA sessions to one model resource, so a hyperparameter sweep costs one base model’s GPU time.
• Multi-LoRA deployment: Attach trained adapters to one dedicated deployment of their base model, then name an adapter in each request to compare them.
• Prepare for FP4 inference: Merge a LoRA adapter into GLM-5.3 or Kimi-K3 and quantize the result to FP4 for dedicated model inference.
• Tinker compatibility: Run a Tinker training script on post-training by changing its import line and, if needed, its base model.
• CLI commands: Together CLI 2.41.0 adds tg training to list model resources, sessions, and checkpoints and to run FP4 preparation jobs. tg beta endpoints deploy adds --attach-adapter and --merge for deploying a trained adapter.

See Post-training for details.

New dedicated endpoint models

The following models are now available for deployment on dedicated endpoints:

• nvidia/parakeet-tdt-0.6b-v3 (speech-to-text).
• openai/whisper-large-v3 (speech-to-text).

Unified together-ai agent skill

A single together-ai agent skill now replaces the per-product Together AI agent skills. It covers every product area, plus Together Link setup. The skill loads the guide for a product area only when a task needs it, so installing it doesn’t fill your agent’s context.

If you installed any of the earlier skills (together-chat-completions, together-images, and so on), delete them after you install together-ai, so your agent doesn’t load outdated guidance alongside the new skill.

Lower maximum batch size for Qwen 27B full fine-tuning

The maximum batch size for full fine-tuning of Qwen/Qwen3.8-27B, Qwen/Qwen3.6-27B, and Qwen/Qwen3.5-27B is now 4, down from 16 for SFT and 8 for DPO.

See Supported models for each model’s limits.

Browser login for the Together CLI

Together CLI 2.41.0 adds tg login, which logs you in through your browser instead of with an API key. If you don’t have a Together AI account yet, you can create one during login.

The release also adds:

• tg switch, which changes the project the CLI works in.
• tg projects ls, which lists the projects your current credentials can access.
• tg logout, which removes your stored credentials.

For CI/CD and scripts, the CLI still accepts an API key.

See Login.

New provisioned throughput models

The following models are now available on provisioned throughput:

• zai-org/GLM-5.3.
• zai-org/GLM-5.3-Flash.
• deepseek-ai/DeepSeek-V4.1-Flash.

Dynamic rate limits for serverless retired

Serverless no longer applies dynamic rate limits that scale with your recent traffic. Most users will no longer encounter rate limits, though requests can still be limited during periods of high demand. If you get a 429 Too Many Requests response, reduce your request rate and spread out bursts. If you get a 503 Service Unavailable response, wait briefly before retrying. Retry both with exponential backoff.

See Rate limits.

Together Link beta

Together Link runs six coding agents on models hosted by Together AI: Claude Code, Codex, OpenCode, and Pi Code in the terminal, plus Claude Desktop (including Cowork) and ChatGPT Desktop. Install it with one command, launch your agent through it, and your normal agent configuration stays untouched. It’s now in beta on macOS and Linux.

# Install Together Link
curl -fsSL https://link.together.ai/install | bash

# Open the interactive launcher
togetherlink

# Or launch an agent directly, pinned to one model
togetherlink --main moonshotai/Kimi-K3 claude

What’s included

• Six agents: Launch Claude Code (tclaude), Codex (tcodex), OpenCode (topencode), or Pi Code (tpi) in your terminal, or switch Claude Desktop and ChatGPT Desktop to a reversible Together Link profile. OpenCode requires OpenCode 2, and Pi Code requires version 0.80.8 or newer.
• Auto router: Sessions default to the auto model, which picks a Together AI model for each request. In Claude Code and Claude Desktop sessions with an Anthropic API key, it sends the most difficult requests to Claude Opus.
• Models: moonshotai/Kimi-K3, zai-org/GLM-5.3, zai-org/GLM-5.3-Flash, and deepseek-ai/DeepSeek-V4.1-Flash, all with 1M context, billed at standard serverless rates. Pin one with --main, or switch with your agent’s /model command.
• Cost tracking: A session receipt on exit, an estimated spend in Claude Code’s status line, and togetherlink usage for spend across sessions.
• Image generation: Claude Code, Claude Desktop, and ChatGPT Desktop sessions include a skill that generates and edits images with Together AI image models.
• Headless mode: Each agent’s own non-interactive flags pass through, so scripts can drive tclaude -p, tcodex exec, topencode run, and tpi -p.

See Together Link for details.

Code sandbox SDK and CLI

The new together-sandbox SDK runs commands and code in isolated runtime environments built from Docker-image snapshots. It ships as a Python SDK, a TypeScript SDK, and a standalone CLI, and is available to organizations on an allowlist (contact us to request access).

What’s changed from the legacy SDK (@codesandbox/sdk)

• Together-native authentication: Clients authenticate with your Together API key (TOGETHER_API_KEY) instead of a CodeSandbox API token.
• Python support: The legacy SDK was TypeScript-only. The new SDK is published on both PyPI and npm, and the CLI installs as a self-contained binary.
• Docker-defined environments: Sandboxes boot from snapshots built from a Docker image or Dockerfile by Together’s remote image builder, replacing templates built with the CodeSandbox CLI. No local Docker is required.
• Snapshot-based persistence: Sandboxes are ephemeral by default and termination is permanent. To maintain state, snapshot the filesystem on termination and start a new sandbox from it. This replaces the legacy hibernate and resume model.

See Code sandbox for the new workflow.

New dedicated endpoint models

The following models are now available for deployment on dedicated endpoints:

• hexgrad/Kokoro-82M (text-to-speech).
• Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice (text-to-speech).

Higher LoRA rank limit for fine-tuning

You can now train low-rank adapters (LoRA) with a rank of up to 128 for the majority of models, up from 64. The default rank for these models stays at 64, so set lora_r to use a higher one.

The model limits response has a new lora_training.default_rank field next to lora_training.max_rank. Run tg fine-tuning model-limits to see both values for a model.

Version 2.36.0 of the Together CLI and Python SDK uses the default rank when you don’t set lora_r. Earlier versions, including the 1.x SDK, use the model’s maximum rank instead, which is now 128 on these models. Upgrade to 2.36.0 or set lora_r yourself, especially if you continue training from a rank-64 adapter, where the rank has to match.

In the console, the rank field now starts at the model’s default rank (64 on most models) instead of 8.

See Supported models for each model’s default and maximum rank.

Longer fine-tuning context for Qwen 27B models

Qwen/Qwen3.8-27B, Qwen/Qwen3.6-27B, and Qwen/Qwen3.5-27B now support a 131,072-token context for SFT (up from 32,768) and 65,536 for DPO (up from 16,384), for both LoRA and full fine-tuning. Batch size limits have also changed: LoRA jobs on these models run at a batch size of 2, down from 16, and the maximum DPO batch size for full fine-tuning drops from 16 to 8.

See Supported models for each model’s limits.

Batch API files are now retained for 7 days

The input file you upload for a batch job, along with the output and error files the job produces, are now retained for 7 days. After that the files are no longer accessible, so download your results before the window closes. Reusing an uploaded input file across batch jobs also works only within that window.

See Batch inference.