Baseten Release Notes

We just launched the Baseten Delivery Network (BDN), designed to make cold starts 2-3x faster for large models.

BDN solves three root causes of slow cold starts: slow weight pulls from upstream storage, replica stampedes under load, and upstream availability dependencies. On first deployment, BDN mirrors your weights to secure storage. From there, a multi-tier cache (node cluster mirrored origin) serves weights with consistent hashing and single-flight semantics: each file fetched once per cluster, not once per pod. Fine-tunes sharing weights with a base model only pull the delta.

Check out the launch blog to learn more, or see the docs to get started.

You can now query your billing usage programmatically using the new GET /v1/billing/usage_summary endpoint. Pass a date range of up to 31 days to get a breakdown of costs across Dedicated Inference, Training, and Model APIs.

The response includes aggregate totals and a per-resource or per-model breakdown[] array, with daily granularity on each entry. Deleted resources are still returned in breakdown[] with is_deleted: true, so historical cost attribution is preserved.

Example output:

{
  "dedicated_usage": {
    "subtotal": 123,
    "credits_used": 123,
    "total": 123,
    "minutes": 123,
    "breakdown": [{
      "billable_resource": {
        "id": "<string>",
        "kind": "MODEL_DEPLOYMENT",
        "name": "<string>",
        "is_deleted": true,
        "instance_type": "<string>",
        "environment_name": "<string>"
      },
      "subtotal": 123,
      "minutes": 123,
      "inference_requests": 123,
      "daily": [{
        "date": "2023-12-25",
        "subtotal": 123,
        "minutes": 123,
        "inference_requests": 123
      }]
    }]
  },
  "training_usage": {
    "subtotal": 123,
    "credits_used": 123,
    "total": 123,
    "minutes": 123,
    "breakdown": [{ ... }]
  },
  "model_apis_usage": {
    "subtotal": 123,
    "credits_used": 123,
    "total": 123,
    "breakdown": [{
      "model_name": "<string>",
      "model_family": "<string>",
      "subtotal": 123,
      "input_tokens": 123,
      "output_tokens": 123,
      "cached_input_tokens": 123,
      "daily": [{
        "date": "2023-12-25",
        "subtotal": 123,
        "input_tokens": 123,
        "output_tokens": 123
      }]
    }]
  }
}

Check out the billing API reference for the full schema and parameters.

Truss can now upgrade itself directly from the CLI. Use the new truss upgrade command to update to the latest version. Truss will detect your package manager (supports uv, pip, pipx, and anaconda) and ask for confirmation before proceeding.

You can also upgrade or roll back to a specific version by passing it as an argument: truss upgrade 0.14.0.

Truss will also now notify you when a new version is available, displayed right at the start of normal commands like truss push.

Version checks run once daily and can be disabled by setting check_for_updates = false under [preferences] in $HOME/.config/truss/settings.toml.