Baseten Release Notes

You can start sending requests to Inkling today through our Model APIs by calling the OpenAI-compatible endpoint with your Baseten API key. For larger workloads, dedicated deployments are available.

Inkling is Thinking Machines Lab's open-weights multimodal model, built for breadth: it accepts text, image, and audio inputs, the first model on our Model APIs with audio support.

curl -X POST https://inference.baseten.co/v1/chat/completions \
-H "Authorization: Bearer $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "thinkingmachines/inkling",
  "messages": [{
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "Summarize what is said in this recording."
      },
      {
        "type": "audio_url",
        "audio_url": {
          "url": "https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav"
        }
      }
    ]
  }]
}'

For more information and to get started, see our docs.

Model API Deprecation (GLM 5.1, GLM 5, Kimi K2.5, Nemotron Super 120B)

The GLM 5.1, GLM 5, Kimi K2.5, and Nemotron Super 120B APIs will be deprecated at 5pm PT July 24th.

At that time the model IDs will become inactive and return an error for all requests. As open source models advance rapidly, we prioritize serving the highest quality models and deprecate models when stronger alternatives are available.

We recommend the following models as alternatives, all of which offer superior intelligence for each specific use case. Just swap in the new Model ID(s) prior to the deprecation date.

model alternatives

If you’d like to continue using the previous weights, please contact us about a dedicated deployment of the model.

We're building one CLI for the whole Baseten model workflow: deploy from local, call your models, stream and filter logs, check metrics, and manage deployments, environments, and secrets, all with --output json and --jq for agents and scripting.

To get started, use brew on macOS:

Add and trust the tap

brew tap basetenlabs/baseten
brew trust basetenlabs/baseten

Install the CLI

brew install baseten

Give feedback, file an issue, or PR at github.com/basetenlabs/baseten-cli.

For more information, see our docs.

Connect your coding agent to the Baseten MCP server and install the Baseten skill to manage your workspace from your agent. Your agent can deploy and promote models, tune autoscaling, pull logs, and launch training jobs. Every tool is labeled read-only or mutating, so you control which calls change your account.

Get started by sending this prompt to your agent:

Install the Baseten agent toolkit following the instructions at github.com/basetenlabs/baseten-skills, all global (-g -y): the `baseten` skill, the backend MCP https://api.baseten.co/mcp with header "Authorization: Bearer $BASETEN_MCP_KEY", and the docs MCP https://docs.baseten.co/mcp. Run the commands in a shell where $BASETEN_MCP_KEY is set; don’t print the key. Then tell me how to verify and whether to restart.

For more information, see our docs.

You can now cap how aggressively the autoscaler removes replicas when traffic drops. Set max_scale_down_rate between 1% and 50% (default 50%) to limit the share of excess replicas removed at each scale-down step.

Lower the rate to scale down more gradually and keep more replicas warm when traffic tends to rebound. Raise it toward 50% to release idle capacity faster.

curl -X PATCH \
https://api.baseten.co/v1/models/$MODEL_ID/deployments/production/autoscaling_settings \
-H "Authorization: Api-Key $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"max_scale_down_rate": 20
}'

For more information, see our docs.

truss model-logs now takes filter flags so you can pull exactly the logs you need. Scope the window with --since, --start, and --end, filter by --min-level or message text with --includes and --excludes, or trace a single inference with --request-id.

truss model-logs --model-id abc123 --deployment-id def456 --since 1h --min-level error
[2026-06-22 00:00:00]: (abc12) response failed to return

The same logs, deployment metrics, and model listing are also available via the Management API, which is currently in beta.

Requires Truss 0.18.11, for more information, see the CLI reference.

You can start sending requests to GLM 5.2 today through our Model APIs by calling the OpenAI-compatible endpoint with your Baseten API key. For larger workloads, dedicated deployments are available.

GLM-5.2 is Z.ai's flagship model for agentic engineering is built to perform well on long-horizon coding tasks. GLM-5.2 runs on the Baseten Inference Stack.

curl -X POST https://inference.baseten.co/v1/chat/completions \
-H "Authorization: Api-Key $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "zai-org/GLM-5.2",
"messages": [{"role": "user", "content": "Refactor this function for readability"}]
}'

For more information and to get started, see our docs.

You can start sending requests to Kimi-K2.7-Code today through our Model APIs by calling the OpenAI-compatible endpoint with your Baseten API key. For larger workloads, dedicated deployments are available.

Kimi K2.7 Code is Moonshot AI's coding-focused model, built for long-horizon agentic engineering tasks with a 262K-token context window. Kimi-K2.7-Code runs on the Baseten Inference Stack.

curl -X POST https://inference.baseten.co/v1/chat/completions \
-H "Authorization: Api-Key $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "moonshotai/Kimi-K2.7-Code",
  "messages": [{
    "role": "user",
    "content": "Write a binary search in Rust"
  }]
}'

For more information and to get started, see our docs.

Baseten now surfaces engine-native metrics for models served with vLLM or SGLang directly in the Metrics tab.

Baseten automatically detects the engine through your container's code /metrics endpoint, then graphs metrics such as tokens per second, time to first token, KV cache usage, and requests running or queued, no configuration or redeploy required.

vLLM metrics

You can also export these metrics to your own observability stack alongside Baseten's standard metrics.

For more information, see our docs.