DigitalOcean Release Notes

Coinbase CEO Brian Armstrong recently posed the question every company scaling AI is asking: how do you keep spend flat while token usage grows exponentially? This isn’t hypothetical. It’s confronting companies across every sector:

• Uber exhausted its annual AI coding budget within the first four months of the year and subsequently introduced a $1,500 monthly limit per employee.
• Walmart placed token limits on its internal Code Puppy agent after employees repeatedly asked it to solve similar problems.
• A Priceline employee reported that a routine Cursor renewal came back 4-5x more expensive.

Usage caps may help control the bill, but they also limit productive work. A better answer is to improve the economics of every request through better defaults, routing, and making caching work for your specific workload scenarios.

Today, we’re making DigitalOcean Inference Router cache-aware. In April, we launched preference-aware routing, so our router could match each request to the model that best fit a developer’s task and priorities. Now, it can also account for the value of context that’s already cached. This advances Inference Router from selecting the right model for each request toward optimizing the entire agentic session across quality, cost, and cache locality. With this release, our Inference Router now offers a comprehensive set of controls for you to build an intelligence layer that fits how your team actually works.

A warm cache can be more valuable than a cheaper model

Caching is a critical consideration when building agents, because they repeatedly send the same large body of context: system instructions, tool definitions, repository context, and an accumulating conversation history.

Here’s how top providers are putting caching to work:

• Z.ai uses a 90.9% cache-hit rate as its average assumption for coding workloads when estimating usage for its coding plans.
• Anthropic shares that Claude Code uses prompt caching to make back-to-back calls cheaper and faster.
• OpenAI reports that cached prompts can reduce latency by up to 80%.

For agents, caching is not a marginal optimization. It shapes the cost and latency of almost every subsequent model call. At DigitalOcean, we are seeing this first-hand as we scale more models on behalf of customers. With the recent release of Kimi K3, we’ve observed an aggregate cache-hit rate of 90%+ across our own workloads as developers use the model for coding and long-horizon tasks; individual workloads will differ.

Cache-aware routing changes the economics of model routing. Consider, as an illustration, an agent with 90,000 input tokens already cached on Claude Sonnet 5 out of a 100,000 token context. At the standard pricing of $2.5 per million input tokens (with cache writing enabled) and $0.2 per million cached tokens, a 90% cache hit makes the next request cost approximately $0.043 in input tokens. (Pricing information is current as of the publication date.)

While GLM‑5.2 appears cheaper at its $0.7 per million uncached input rate, switching models discards the warm cache and forces re-processing of the full 100,000 input token context. On the assumption in this illustration, that request would cost $0.07: approximately 1.6 times more than staying on the nominally more expensive model in this scenario. Sticking with the warm model requires prefilling only the 10,000 uncached tokens; switching requires all 100,000—10x more prompt processing before generation can even begin. That doesn’t translate into a 10x latency increase, since prefill performance varies by model and serving system. But it does explain why a cache-breaking switch can meaningfully increase time to first token, even when the destination model is otherwise faster.

How Inference Router supports cache-aware routing

Before the launch of cache-aware routing, Inference Router evaluated each request independently. It could correctly determine that another model was more affordable or better suited to the context presented, but didn’t recognize that the request belonged to an ongoing agent session with a warm prompt cache.

But the act of switching models can invalidate the existing cache and force the destination model to process the entire prompt again. For agents that repeatedly send large system instructions, tool definitions, repository context, and conversation history, using the “cheaper” model can make the next request more expensive and slower. Another complication is that it can also change model behavior partway through an agent’s loop.

When customers told us they needed more control over that tradeoff, we built cache-aware routing. It introduces two complementary mechanisms: explicit model affinity for applications that already manage sessions, and a routing-budget policy that determines when breaking affinity is worth the additional cost.

Explicitly associate requests with a custom HTTP header: X-Model-Affinity

Applications that already maintain session or task identifiers can pass an explicit affinity key with each request:

import os
import uuid
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MODEL_ACCESS_KEY"],
    base_url="https://inference.do-ai.run/v1/",
)

session_id = str(uuid.uuid4())
messages = []
user_turns = [
    "Help me debug this failing test.",
    "Here's the stack trace, what's causing it?",
    "That fixed it, now can you also add a regression test?",
]

for user_turn in user_turns:
    messages.append({"role": "user", "content": user_turn})
    response = client.chat.completions.create(
        model="router:<your-router-name>",
        messages=messages,
        extra_headers={"X-Model-Affinity": session_id},
    )
    assistant_reply = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_reply})

The first request is routed according to the developer’s configured task, model pool, and routing preferences. Requests with the same X-Model-Affinity value are then treated as part of the same unit of work, allowing Inference Router to preserve the session’s model binding and reuse its cached context. Affinity identifiers should represent meaningful units of work: a coding session, research task, support conversation, or individual agent run. When the application starts a genuinely new task, it can provide a new identifier, allowing the Inference Router to make a fresh preference-aware decision.

For common agentic requests, Inference Router can also infer affinity if an explicit identifier is not available. It derives a stable session key from the request context that remains unchanged across turns, including system and developer instructions, tool definitions, and the first user message. If that stable prefix changes, Inference Router treats the cache as cold and establishes a new binding. This reassigns the session to a model, which then starts accumulating its own warm cache from scratch.

Control cache-breaking switches with a routing budget

Model affinity headers are ideal for applications that already track meaningful units of work—such as research tasks and support conversations—and want deterministic control over which requests share the same model binding. With this release, we’ve also introduced the routing budget: a complementary control that keeps Inference Router evaluating alternative models without requiring any changes to your application code.

When the routing policy proposes switching models, Inference Router calculates the incremental cost of leaving the session’s warm cache. It does this by comparing the cached input cost of staying on the current model with the uncached cost of rebuilding the context on the candidate model, then evaluates that cost against the session’s cumulative switching spend.

Developers can define this trade-off with a maximum switching budget, set relative to what the session would have cost had it stayed on the existing model. For example, X-Routing-Max-Switch-Spend-Pct: 20 limits cumulative switching costs to 20% above that baseline. Model selection and economics remain separate: the router identifies its preferred model, while the routing budget determines whether switching to it is worth the additional input cost.

Together, developers can use these controls to choose the appropriate level of involvement:

• Use X-Model-Affinity when the application already has an authoritative session or task identifier.
• Let Inference Router automatically detect affinity based on stable agent context across related turns.
• Configure X-Routing-Max-Switch-Spend-Pct to control how much additional input cost the router can incur by switching models.

New Analyze Page

We have updated the Analyze page to give you detailed visibility into how your router makes cache-aware decisions.

In the top-line router view, you can quickly get answers to questions like:

• What is overall caching efficiency?
• How many requests switched models?
• How many were held to keep the cache warm and what is the overall latency as a result?

This lets you get a high level snapshot of your router at a glance.

From here, you can drill down further into the behavior of specific models and tasks to get a more detailed understanding of the traffic mix. This makes it easier to identify specific model hotspots, validate routing strategy, and tune router preferences over time.

We’ve also added trend tracking for cache efficiency giving you visibility at both request and token level over time. This makes it easier to spot cache regressions, understand performance trends, and validate the impact of prompt and cache tuning changes.

Together, these views help teams move from high-level monitoring to targeted optimization right from the Inference Router UI.

Routing that starts with developer preferences

When we launched DigitalOcean Inference Engine and Inference Router, we gave developers a way to define tasks, create model pools, and express whether they wanted to optimize for quality, cost, or latency. Inference Router then semantically matches each request to a task and applies those preferences to select a model. Developers can start with DigitalOcean presets—opinionated, routinely updated model selections informed by our evaluations—or define their own tasks, model pools, and priorities. Either path works out of the box: no router training or application-side routing logic required. Early customer LawVo reported reducing inference costs by more than 40% while maintaining the accuracy, speed, and reliability its users expected*.

This approach is grounded in years of research into preference-aware routing. In Arch-Router: Aligning LLM Routing with Human Preferences, our team introduced a compact 1.5-billion-parameter model that maps requests to developer-defined domains and actions and can incorporate new models without retraining. We published the model with open weights—the broader approach remains available through Plano, our Apache-licensed open-source AI proxy and data plane. That research originated from a simple observation: benchmarks are useful, but they are not preferences.

Model benchmarks are maps, not routing tables

Benchmarks let us compare models under controlled, repeatable conditions. They help narrow a large model catalog, identify broad strengths, and bootstrap routing before an application has enough real-world traffic to run its own evaluations. That makes them a valuable starting point for DigitalOcean presets.

But model performance is conditional on the surrounding application: the system prompt, tool definitions, context, output constraints, conversation history, and definition of success. Change the agent harness, and the relative ranking of models changes with it. A model that performs best on an isolated coding benchmark may not be the ideal choice for use within a coding agent operating across a large repository with dozens of tools and a long conversation history.

Relatedly, one developer may prefer a particular model’s visual style for image generation, while another may prioritize instruction following, tool-call reliability, latency, or cost. Neither preference can be inferred from a general-purpose leaderboard. Preselecting a model on benchmark scores alone is not intelligent routing. Routing is only intelligent once it knows what the developer is optimizing for. Over time, it becomes a personalization problem. But even a preference-aware router can make the wrong economic decision if it evaluates every request in isolation.

Better defaults, better routing, and better caching

Across sectors, the knee-jerk response to rapidly growing inference bills has often been to ration access. Yet Coinbase has publicly reported that 91% of Coinbase employees were not reaching their existing usage caps. Lowering those caps would have generated more alerts and friction without addressing what actually drove most of the spend. Coinbase instead moved toward cheaper defaults, task-aware routing, and better caching, which it reports improved LibreChat’s cache hit rate from 5% to 60%.

These three controls reinforce one another:

• Better defaults prevent every request from beginning on the most expensive model.
• Preference-aware routing selects models based on the task and the developer’s values.
• Cache-aware routing preserves the accumulated economic value of an agentic session instead of discarding it between turns.

A cheap default may not meet the quality bar for a complex task. A benchmark-driven router may not reflect an application’s real evaluations. A cache-aware system should not preserve a warm model when it is no longer appropriate for the work. No single technique is sufficient on its own. The objective is not to maximize tokens or blindly minimize their price, but rather to maximize useful intelligence per dollar spent while preserving the quality, latency, and reliability each application requires.

Routing is only intelligent when it understands what you are optimizing for and what switching away from an in-progress task actually costs. The DigitalOcean Inference Router gives you the control and visibility to build an intelligence layer that fits how your team actually works. Use it now to create a preset or custom router and add model affinity to your next agentic workflow. All figures in this post are illustrative and based on the pricing, models, and configurations available as of the publication date; third-party figures are as reported by those parties. Results and savings vary with configuration, implementation, and usage, and are not guaranteed. All marks are the property of their respective owners, and no affiliation or endorsement is implied.

*Disclaimer: This reflects LawVo’s own reported experience in its own environment and is not necessarily representative of results other customers will achieve.

The following open-source models are deprecated from DigitalOcean Inference as of 18 August 2026:

• Llama 3.3 Instruct-70B
• DeepSeek R1 Distill Llama 70B
• Qwen3-32B
• Qwen3 Coder Flash

Migrate Llama 3.3 Instruct-70B to Llama 4 Maverick 17B 128E Instruct (llama-4-maverick), DeepSeek R1 Distill Llama 70B to DeepSeek V4 Flash (deepseek-4-flash), and Qwen3-32B and Qwen3 Coder Flash to Qwen 3.5 397B A17B (qwen3.5-397b-a17b) to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

The following Alibaba model is now available on DigitalOcean Inference for serverless inference, Agent Development Kit, and agents:

• Qwen3.8-2.4T-A95B

For more information, see the Available Models page.

Cloud Security Posture Management (CSPM) paid plans support Managed Rules, which let you enable or disable CSPM rules for your entire team.

To exclude specific resources while keeping a rule enabled for the rest of the team, suppress findings for those resources. For more information, see How to Manage Rules and How to Suppress Findings.

Spot GPU Droplets are now in Public Preview, giving you access to latest-generation GPUs for short-lived, fault-tolerant work like batch training, batch inference, and rendering.

Latest-generation GPUs: NVIDIA HGX™ B300 and AMD Instinct™ MI350X/MI355X GPUs.

Pricing locked at creation: Your Spot rate stays fixed for the life of the Droplet, with no long-term reservation.

Works with the rest of your stack: Vector databases, Spaces object storage, Managed Databases, and CPU Droplets, all on one bill.

Droplets are interruptible and can be reclaimed at any time. We aim to give at least two hours' notice. Terms and Conditions apply.

Provision today

We have launched the Memphis, Tennessee, USA (mem1) datacenter, which supports AMD Instinct MI355X Spot GPU Droplets and many other products. For the full list of supported products, see the regional availability matrix.

• AMD Instinct MI355X GPUs are now available in MEM1 as Spot GPU Droplets in 1- and 8-GPU configurations. Spot prices vary daily based on available capacity. For pricing, see Spot GPU Droplet pricing.
• Spot GPU Droplets are now available in public preview for supported GPUs in single-GPU and 8-GPU configurations. Pricing may change daily based on capacity.

To compare the capacity tiers, see Spot GPU Droplets vs On-Demand GPU Droplets.

• DigitalOcean Container Registry (DOCR) now provides mirror registries: DigitalOcean-curated regional mirrors of popular AI container images, available in 11 datacenter regions. You can pull nvidia/pytorch and rocm/pytorch images from a region-local mirror using standard Docker tooling, or use SOCI lazy loading to start containers in seconds while image layers stream on demand. DOCR also supports pushing your own SOCI-enabled images to your registry.

For details, see How to Pull AI Images from DOCR Mirror Registries and How to Push SOCI-Enabled Images to Your Container Registry.

DeepSeek-V4-Flash-0731, DeepSeek's latest release optimized for agentic coding and autonomous workflows, is now available through DigitalOcean Inference Engine. A 284B-parameter mixture-of-experts model with only 13B parameters active per request, this model delivers fast inference at a fraction of the compute cost of a dense model of comparable size. A full re-post-training pass focused on autonomous task execution pushes its agentic performance ahead of DeepSeek's own V4-Pro, despite the smaller active parameter count. It excels at repo-scale coding agent workflows, multi-step tool use, and production-grade agentic loops.

Access the model now ->

Spend alerts

Spend alerts are now generally available for teams and organizations, replacing billing alerts. You can create multiple alerts with incremental percentage thresholds, scoped to total spend, specific products, or, for organizations, specific teams. Notifications arrive within an hour of your spend crossing a threshold.

Existing billing alerts have been migrated to the Spend alerts page as an alert named Account Spend Alert.

The following Anthropic model is deprecated from DigitalOcean Inference as of 5 August 2026

• Claude Opus 4.1

Migrate to Claude Opus 4.8 (anthropic-claude-opus-4.8) to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

Kansas City (MKC1) is now available as our newest DigitalOcean data center region. MKC1 expands DigitalOcean's footprint to 20 data centers across 11 global regions. MKC1 is a fully liquid-cooled data center and runs NVIDIA B300 GPUs.

Available at launch

• Droplets, custom images, and Droplet Autoscaler
• Kubernetes (DOKS), App Platform, and Functions
• Managed Databases
• Spaces Object Storage (standard and cold tiers), Volumes, NFS, backups and snapshots
• Networking: Load Balancers, VPC, NAT Gateway, Cloud Firewalls, reserved IPs, DNS, and Partner Network Connect
• Cloud Security Posture Management (CSPM), Marketplace, monitoring and alerts, and IAM

MKC1 has achieved an ISO 27001:2022 certification alongside SOC 2 Type II, which is reflected in a SOC 3 Type II report. These certifications can be accessed in our Security Reports and Certifications Center. As with every DigitalOcean region, you get the full platform in one place and on one bill, so your inference, compute, storage, databases, and apps sit together instead of across vendor boundaries.

Deploy in MKC1 now ->

• We have launched the Kansas City, Missouri, USA (mkc1) datacenter, which supports GPU Droplets, Kubernetes, Managed Databases, Spaces object storage, App Platform, Functions, and many other products. For the full list of supported products, see the regional availability matrix.

• Cloud firewall rules now support an action of allow or deny. Deny rules let you block traffic from specific sources, such as known malicious IP addresses, while allowing broader access to your services. Deny rules take precedence over allow rules that match the same traffic. For rule actions and configuration steps, see How to Configure Firewall Rules.