Claude Updates & Release Notes

Catch more bugs before they reach production.

Code Review is out of research preview and available to Claude Team and Enterprise organizations.

Eligible Claude Enterprise organizations can claim at least $10,000 in Code Review credits by January 15, 2027. Credits expire 30 days after you claim them. Terms apply.

What's new in Code Review

Coding agents keep increasing the pace at which teams write code, and the review bottleneck hasn't eased. For general availability, we made Code Review faster, better at finding the bugs that matter, and easier to tune to each of your repositories.

Since we released the research preview, Code Review has gotten sharper, catching more of the bugs that would otherwise reach production. We also run Code Review on pull requests at Anthropic, which shows us where reviews slow our own engineers down. Teams using Code Review during the preview told us the rest: it was catching real issues before a human reviewer looked at the PR, but they wanted reviews that were quieter, sized to each change, and faster. Here’s what we shipped:

• Speed: Most reviews now finish about 3x faster (~20 minutes down to 6-10 minutes) compared to during preview.
• Quality: Find more of the bugs that matter, with clear, concise findings that state the problem, what triggers it, and the fix in a few sentences.
• Flexibility: Admins choose Quick, Standard (default), or Deep for each repository, so routine codebases get a quick pass and the most critical ones get a deep review. Every level delivers high-quality reviews.

Get started

Code Review is now generally available for Team and Enterprise plans.

• For admins: Enable Code Review in your Claude Code settings, install the GitHub App, and select repositories you’d like to run reviews on.
• For developers: Once enabled, reviews run automatically on new PRs. No configuration needed.

Explore the docs for more information.

Original source

Claude Dashboards and Claude Motion are now in beta. Docs, Slides, and Design are out of beta and on every Claude plan, including Free.

Starting today, Claude can turn your data into a dashboard that stays current, and your ideas into an animated explainer for your next presentation. Claude Dashboards connects to a data platform like BigQuery, Databricks or Snowflake, or a CRM tool like Salesforce, and is in beta on paid plans. Claude Motion is in beta on Team and Enterprise. Claude Docs, Slides, and Design are out of beta and available on every plan, including Free.

Asking questions about company data usually means filing a ticket or writing SQL. And ideas that would land better in motion usually end up as another static slide. So people wait, or skip it. Claude could already help with parts of this. Now you can ask for the dashboard or the animation itself, and see how Claude made it. Each chart shows the query behind it, and each animation is built as code you can edit. Claude makes the first version, and you decide what gets shared.

Claude Dashboards: ask quick questions of your company’s data

Connect your company’s data platform, such as Amazon Redshift, BigQuery, ClickHouse, Databricks, or Snowflake, and ask a question in plain language. Claude pulls answers from your data, builds the dashboard, and keeps it current as the data changes. It works with your other connectors too. For example, you can ask for a dashboard of your Salesforce opportunities. Dashboards work well for quick, exploratory questions, like how this week’s signups compare with last month’s. Click any number to see the query behind it, or ask Claude to explain it. Each chart shows when its data was last refreshed.

Dashboards work alongside your BI and analytics tools. When a question needs deeper analysis, send the dashboard from Claude straight to Amplitude, Grafana, Hex, Mixpanel, Omni, Perplexity, PostHog, or Sigma and pick up the analysis from there, with Looker, monday.com, and Tableau coming soon.

Claude Motion: explain your ideas with animation

Ask Claude to turn a quarterly report into a 30-second explainer for an all-hands, add motion to a chart in a board deck, or make a product walkthrough for customer onboarding. Claude builds an animation from your prompt, and it plays like a short video. You can adjust it in the editor or ask Claude to change it, then download it as an MP4 file.

Claude Motion is built for explaining your own work. Claude writes code that animates your text, charts, shapes, and images, so you can change any word, number, or timing. It doesn’t use a video generation model, so there’s no generated footage and no AI-generated people. When you want to take it further, open it in Adobe, Descript, HeyGen, Higgsfield, invideo, Luma AI, or Runway, with Canva and Captions coming soon.

Docs, Slides, and Design are ready for everyday work

Since we introduced Docs, Slides, and Design in every Claude conversation, people have made more than 45 million docs, decks, and designs in Claude. Most of what we’ve shipped since came from your feedback, and it makes them easier to use every day. So we’re taking the beta label off.

• Ready for enterprise. Artifacts now support CMEK, and admins choose which artifact templates their organization uses.
• Edit with your team. Your team and Claude can work on the same doc, deck, design, or dashboard.
• Share beyond your team. Share slides, designs, dashboards, and animations with people outside your organization, or with anyone who has the link, if your admin allows it.
• Exports that keep your formatting. PowerPoint and PDF downloads look like what you made in the editor, and decks go straight to Google Slides as editable files.
• Edit on your phone. Change a doc, deck, or design in the Claude mobile app.

Claude Design has moved into Claude

Claude Design started at its own URL, claude.ai/design. Since September 16, Design also works inside every Claude conversation, where it can use your context, files, and connectors. Running two versions slows down fixes for both, so we’re folding the standalone site into Claude. It stays at claude.ai/design until December 14, so you have time to move over.

• Migrate your design systems today. On the Artifacts page in Claude, click "Migrate team design systems." One migration brings over every design system in your organization, and the originals stay unchanged in the standalone version until it closes.
• Your projects need nothing from you yet. They keep working in the standalone version until it closes.
• Save your chats and comments. Chats with Claude and comments on your projects stay in the standalone version and won't be available after it closes. Public links to standalone projects stop working then too.

Read the migration guide for more information.

Getting started

Open Claude and ask for what you need, like “Build a dashboard of this quarter’s revenue by region,” “Animate how our pricing plans compare for the sales kickoff,” or “Turn these notes into a deck for Monday.” Start with the dashboard you’d wait for, or the report that needs more than a slide.

For Enterprise admins: Dashboards and Motion are off by default, and you can turn them on in Organization settings > Artifacts. Docs, Slides, and Design turn on by default October 15, or you can turn them on today.

Original source

Claude Haiku 5.5 launch

We just launched Claude Haiku 5.5, the cheapest, fastest, and most capable small model we’ve released, designed for high-volume, cost-sensitive tasks. For more information, see our blog post: Claude Haiku 5.5.

Monthly API credits for Max and Team plans

Max and Team plans now include monthly API credits for running your own apps and agents on the Claude Platform. They roll out over a few days. Claim them by linking a Claude Console organization in Settings > Billing (Max) or Organization settings > Billing (Team). Learn more about monthly API credits for Max and Team plans.

Original source

Security teams at Comcast and Booz Allen used Claude Mythos-class models through Project Glasswing to find, validate, and fix vulnerabilities their existing scanners missed.

Earlier this year, Anthropic launched Project Glasswing, a limited access program to give defenders of the world’s critical infrastructure a head start on securing their codebases. This week, we launched a new, expanded version of our Cyber Verification Program (CVP) that makes advanced cyber capabilities and reduced blocking classifiers available to qualifying security professionals.

As part of Project Glasswing, organizations defending critical infrastructure have put Claude Mythos models to work on the systems they defend. Across these organizations, one theme emerged: while existing cybersecurity tools can identify single vulnerabilities in a silo, exploit chains, where an attacker links several individually minor weaknesses into a path to something serious, are much harder to trace. Models like Claude Mythos help piece together code, configuration, and live application behavior so that organizations can catch both ordinary single-file bugs and the sophisticated cross-system vulnerabilities that only appear when components interact.

We spoke with security leaders from Comcast and Booz Allen who participated in the program about what Claude Mythos-class models found in their environments and how it’s changing the way their teams defend against attacks.

Comcast: Separating findings from exploitable vulnerabilities

Comcast, a global media and technology company, used Claude Mythos Preview to identify a critical authentication vulnerability in a public-facing platform during an assessment covering 258 business-critical systems and approximately 170 million lines of code. The issue could have allowed an attacker to bypass authentication controls and was remediated before any evidence of exploitation was observed.

“Discovery is becoming faster,” said Noopur Davis, EVP, Chief Information Security and Product Privacy Officer, Comcast. “Discovery is becoming easier. The volume of findings is enormous. Validation of these volumes of findings is the new bottleneck."

The finding emerged from Claude Mythos Preview's ability to reason across a complex software ecosystem rather than evaluating individual files or components in isolation. The vulnerability was not caused by a single defective component. Instead, it arose from the interaction of multiple systems and assumptions that appeared correct when reviewed independently. By maintaining context across code, configuration, and application behavior, Claude Mythos Preview identified a security gap that emerged across multiple interacting components, making it difficult to detect through conventional approaches.

“Frontier AI models are making it much easier to generate findings at scale,” said Davis. “But a finding becomes a vulnerability only after validation confirms the underlying issue, and a vulnerability becomes exploitable only when it can actually be exploited in the target environment.”

Comcast engineers validated the finding against the running application and confirmed that the underlying security issue could be reached in practice. The team then traced the root cause, implemented remediation, and verified that the vulnerability had not been exploited prior to being fixed. The finding was representative of a broader pattern observed: some of the highest-impact issues identified by Claude Mythos-class models emerged not from individual coding errors, but from subtle interactions across complex software environments.

“Early access probably accelerated our understanding of the security capabilities of frontier AI models by about a year,” Davis said. “We really appreciate the community that Anthropic has built around this program - it allows participants to share lessons learned, thus helping the wider ecosystem.”

Booz Allen: Piecing together exploit chains

Booz Allen builds and deploys technology for defense, civil, national security, and commercial missions, and its cyber practice works to close the speed gap between AI-powered adversaries and traditional cyber defenses. This includes running red-team exercises and building defenses for customers facing AI-enabled adversaries.

Booz Allen’s security assessments run for a fixed period of time, and engineers can read only so much source code, configuration, and compiled software before they end. Wider coverage alone wouldn't close that gap, because real attacks usually work by chaining small weaknesses across code, configuration, identity, and deployment settings, and tracing those connections by hand across a large software portfolio is slow even for a skilled operator.

Booz Allen used Claude Mythos Preview to review far more software than a short engagement window would normally allow, and to identify weaknesses that could be strung together into exploit chains.

“Mythos was surprising in its ability to reason across boundaries rather than only identify isolated code issues,” said Sahil Sanghvi, Vice President of AI Engineering at Booz Allen. “It helped connect application behavior, configuration, identity, permissions, and deployment context into coherent security hypotheses.” The work also moved from discovery to remediation faster than his team expected, with the model helping to draft fixes and assemble evidence for the engineers who owned the affected systems.

In one case, a Claude Mythos-class model found a flaw in the code that runs when a device first powers on, before the operating system loads. That code decides whether the device should lock itself or erase its data, and it makes that call using a security key stored on the machine. The key had been left unprotected, so the operating system it was meant to restrain could swap in a key an attacker controlled and then feed the machine instructions it would trust. An attacker could hold off the lock-or-erase step indefinitely, keeping a lost, stolen, or compromised device fully accessible when it should have shut itself down.

There were several opportunities for the system to lock the key, and each was missed. When the key was first saved, the setting that would have protected it was switched off. The code that wrote it to the hardware passed that setting along without correcting it. A separate safeguard that could have locked the key afterward was never turned on. Claude found the vulnerability by following that one setting through two different programs written in two different languages, then confirming that nothing ever locked the key.

“Mythos can create a large candidate set quickly, but teams still need to validate findings, deduplicate systemic patterns, and route issues to the right owners,” said Brad Medairy, President of National Cybersecurity, Booz Allen. "One analyst reviewed eight production systems across 138 repositories in twelve days. Without Mythos, a portfolio review at that scale would have taken us several months with a larger team."

For Booz Allen, that points to a different way of running security review, one where operators keep the judgment calls while the model extends how much of the portfolio they can keep under watch.

“Our research shows that frontier models have the ability to autonomously execute the full cyber kill chain,” Medairy said. “Organizations must operate continuous cyber security reviews and defenses to keep pace.”

What’s next

As models with the capabilities of Claude Mythos become part of everyday security work, organizations can examine their own systems the way a skilled attacker would, tracing how code, configuration, and legacy components interact and closing the gaps between them before anyone else finds a way through. We built Project Glasswing to give defenders that capability first, and we’ll keep expanding access via our new Cyber Verification Program (CVP) so that more of the organizations running critical software can put it to work.

Learn more about the Cyber Verification Program (CVP).

Original source

Claude for Google Workspace is now in public beta

Teams that run on Google Workspace can now bring Claude into their files or work on their files directly from Claude, with our new add-on and Google Docs, Sheets, and Slides connectors (in beta).

Claude for Google Workspace™ is now in public beta on all paid Claude plans. It adds Claude to Google Docs, Sheets, and Slides, so that you can work with Claude directly in your open files. Additionally, we are releasing new Google Docs, Sheets, and Slides connectors (beta) that Claude uses to create and edit Google files directly from Claude.

This video shows Claude working inside Google Docs, Sheets, and Slides.

Claude works where you work

With the add-on, Claude opens in a sidebar next to your file, so you don’t need to switch between apps or browser tabs when you work. Claude can read the doc, sheet, or deck you have open, see which text, cells, or slides you’ve selected, and make changes directly in the file.

In Docs, for example, Claude can fix a sentence or restyle a heading in place without touching the surrounding formatting. For bigger rewrites, it can propose edits as suggestion cards in the sidebar. Each card highlights the passage it would change, and you can apply or dismiss it. You can ask Claude to tighten the executive summary of a product requirements document and turn the next steps into a table, and Claude will do both directly in the doc.

In Sheets, Claude can write formulas, build pivot tables and native Sheets charts, and add new tabs. For joins or data cleaning, it can pull a range into Python and write the results back into the sheet. You can start from an empty sheet, attach your source files, ask for a Q3 budget vs. actuals report with a tab for each team and a summary chart, and Claude can flag anything that’s over the budget.

In Slides, Claude can build new slides from your deck’s layouts and themes, so they match the rest of the deck. Then it can check its work and flag elements that overlap or run off the slide, or text that’s hard to read.

You decide how much Claude does on its own. In the default “Ask before edits” mode, each change appears as an approval card with a summary and Claude waits for your approval to make the edit. In “Accept all edits” mode, Claude works through the task and applies changes without stopping.

Your connectors, skills, and enterprise controls come with you

When you are signed in with your Claude account, the sidebar has the same models, connectors, and skills you use in Claude. For example, if you ask for a quarterly business review (QBR) deck for your customer, Claude can pull the account history from Salesforce and recent call notes through your Google Drive connector. If you save the QBR format as a skill, your team can build the next deck following the same format and steps.

On Enterprise plans, controls such as the Compliance API, customer-managed encryption keys (CMEK), and OpenTelemetry audit export apply to the add-on too

Edit Google files from Claude

With the Google Docs, Sheets, and Slides connectors, also in beta, you can start in Claude and it will create and edit your Google files from the chat. Paste a Google file link or ask for a Google doc, sheet, or deck, and on supported setups it will open in a pane beside your conversation. Starting from Claude makes most sense when you need a new file or your work spans several files. Claude’s access matches your existing Google sharing permissions.

Getting started

Claude for Google Workspace is in beta on all paid plans.

Install it from the Google Workspace Marketplace, then open a file and go to Extensions > Claude > Open Claude. Admins can deploy it to their domain or selected groups from the Google Admin console. The Help Center guide covers setup.

To edit Google files from Claude, turn on the Google Docs, Sheets, and Slides connectors. On Team and Enterprise plans, an owner or primary owner needs to enable the connectors first.

Original source

Today we're introducing mods, small TypeScript functions that change how Claude Code works. A mod can rewrite a prompt, add new UI, replace a built-in feature, or add entirely new functionality. You can write a mod yourself, or ask Claude Code to write one for you. Mods ship inside plugins, so you install and share them like any plugin. They work in the Claude Code CLI and desktop app.

Mods run with the same access to your machine as Claude Code itself. They aren’t sandboxed, and you should only install mods from sources you trust, the same way you'd install any code on your computer.

To see what mods can do, read our guide to building your first mod.

Why we built mods

Developers have asked for more control over how Claude Code works, without waiting for us to ship a feature. Hooks helped give users some of this control, but hooks can't rewrite events, draw new UI, or replace features. Mods can.

We want Claude Code to feel like yours, so you can shape it to the way you work. We shared the design for mods on GitHub before launch to get feedback from developers. Thank you to everyone who weighed in.

How mods work

Each time Claude Code does something, it emits an event. Examples include calling a tool, asking for permission, and drawing part of the screen. A mod is a function that hooks into one of these events. A mod can run before the event, after it, or instead of it. It can also wrap the event, running code both before and after. With one function, a mod can:

• Rewrite a prompt before it reaches the model.
• Block, rewrite, or retry a tool call.
• Approve or deny a permission request.
• Redact secrets from tool output before Claude reads it.

A mod can also change what you see. It can edit or replace part of the interface Claude Code draws, like a tool result or a question from Claude. It can add buttons and inputs, and other mods can respond when you press them. Today, a mod can target the terminal, the desktop app, or both.

When several mods hook the same event, they run in the order they load. The first mod to load sees the event first and the result last. This lets you stack mods from different authors.

You can also use Claude Code to mod Claude Code. Ask Claude to create a mod, and it can write the TypeScript, install it, and hot reload it in your session.

Swap built-in features for your own

Some built-in features of Claude Code now ship as mods. For example, the built-in /diff feature is now a mod, so you can turn it off (in /plugin) or replace it with your own version. We plan to move more built-in features to mods over time, so you can pare Claude Code down to a small core and add back only what you want.

Mods for teams and enterprises

Mods ship inside plugins, so your existing plugin controls apply. Admins can allow or block plugin marketplaces. On Team and Enterprise plans, an owner sets this in the admin console. On Claude API and third-party API plans, admins push managed settings to users' machines.

On Team and Enterprise plans, and on any machine with managed settings, a built-in mod called sec-default (“security default”) loads first. It stops mods that users install from doing risky things, like overriding your permission deny rules. You can view the source code to see what it restricts. Admins can load their own mods first instead. If you do, add sec-default to your list to keep its restrictions.

Teams can also use mods to build their own controls and functionality. For example:

• CI/CD status: a mod can show your pipeline’s status in a pane beside the conversation, and update it as builds pass or fail.
• Production safeguards: a mod can require confirmation before any command touches production config.
• Audit logging: a mod that loads first can record every call that every other mod makes.

Getting started

Mods are available today in the Claude Code CLI and desktop app. Install plugins that include mods from the Claude directory or by running /plugin in the CLI. To share a mod, package it in a plugin and submit it to the directory.

To build your own mods, read our getting started guide or see the documentation.

Original source

Today we're introducing mods

small TypeScript functions that change how Claude Code works. A mod can rewrite a prompt, add new UI, replace a built-in feature, or add entirely new functionality. You can write a mod yourself, or ask Claude Code to write one for you. Mods ship inside plugins, so you install and share them like any plugin. They work in the Claude Code CLI and desktop app.

Mods run with the same access to your machine as Claude Code itself. They aren’t sandboxed, and you should only install mods from sources you trust, the same way you'd install any code on your computer.

To see what mods can do, read our guide to building your first mod.

Why we built mods

Developers have asked for more control over how Claude Code works, without waiting for us to ship a feature. Hooks helped give users some of this control, but hooks can't rewrite events, draw new UI, or replace features. Mods can.

We want Claude Code to feel like yours, so you can shape it to the way you work. We shared the design for mods on GitHub before launch to get feedback from developers. Thank you to everyone who weighed in.

How mods work

Each time Claude Code does something, it emits an event. Examples include calling a tool, asking for permission, and drawing part of the screen. A mod is a function that hooks into one of these events. A mod can run before the event, after it, or instead of it. It can also wrap the event, running code both before and after. With one function, a mod can:

• Rewrite a prompt before it reaches the model.
• Block, rewrite, or retry a tool call.
• Approve or deny a permission request.
• Redact secrets from tool output before Claude reads it.

A mod can also change what you see. It can edit or replace part of the interface Claude Code draws, like a tool result or a question from Claude. It can add buttons and inputs, and other mods can respond when you press them. Today, a mod can target the terminal, the desktop app, or both.

When several mods hook the same event, they run in the order they load. The first mod to load sees the event first and the result last. This lets you stack mods from different authors.

You can also use Claude Code to mod Claude Code. Ask Claude to create a mod, and it can write the TypeScript, install it, and hot reload it in your session.

Swap built-in features for your own

Some built-in features of Claude Code now ship as mods. For example, the built-in /diff feature is now a mod, so you can turn it off (in /plugin) or replace it with your own version. We plan to move more built-in features to mods over time, so you can pare Claude Code down to a small core and add back only what you want.

Mods for teams and enterprises

Mods ship inside plugins, so your existing plugin controls apply. Admins can allow or block plugin marketplaces. On Team and Enterprise plans, an owner sets this in the admin console. On Claude API and third-party API plans, admins push managed settings to users' machines.

On Team and Enterprise plans, and on any machine with managed settings, a built-in mod called sec-default (“security default”) loads first. It stops mods that users install from doing risky things, like overriding your permission deny rules. You can view the source code to see what it restricts. Admins can load their own mods first instead. If you do, add sec-default to your list to keep its restrictions.

Teams can also use mods to build their own controls and functionality. For example:

• CI/CD status: a mod can show your pipeline’s status in a pane beside the conversation, and update it as builds pass or fail.
• Production safeguards: a mod can require confirmation before any command touches production config.
• Audit logging: a mod that loads first can record every call that every other mod makes.

Getting started

Mods are available today in the Claude Code CLI and desktop app. Install plugins that include mods from the Claude directory or by running /plugin in the CLI. To share a mod, package it in a plugin and submit it to the directory.

To build your own mods, read our getting started guide or see the documentation.

Original source

Today, Claude for Government is generally available for federal and state agencies.

The platform, which delivers Claude's coding and agentic work capabilities through a FedRAMP High authorized environment, has been in public beta since July.

Agencies access capabilities comparable to Anthropic’s commercial customers, without compromising compliance requirements. New capabilities generally arrive on the commercial release cadence.

Claude works directly with files on the desktop, allowing agency staff to use skills, plugins and projects for memo creation, RFP reviews, casework, and other tasks. With Claude Code, public sector teams can build and modernize the software systems that underpin public services.

Claude for Government governance controls are purpose-built for public sector agencies. Administrators can set configuration defaults as well as allocate and control spending across departments. Security teams and authorizing officials get audit logs and documentation that supports the agency ATO process. Procurement officers can contract with Anthropic directly and award on general-availability terms.

The Claude Code command-line interface and Claude for Microsoft 365 are also rolling out in early access through the same environment and with the same administrative controls.

Configuration view in the admin console

Billing, administration, and oversight

No seat fees. Agencies pay for usage in fixed increments with a hard not-to-exceed cap, so spend does not exceed what an agency has obligated. Administrators define user tiers with spend and model limits per group, track usage by user and by model, and get burndown alerts before a balance runs low.

Spend analytics view in the admin console

Administration that matches how departments are organized. Department-level administrators allocate prepaid usage to sub-agencies while each manages its own users. Agencies connect their own identity provider for single sign-on, with self-serve setup in the admin portal. SCIM group mappings set rate limits, dollar caps, and allowed models for each seat tier. Layered configuration sets defaults for sub-agencies, including what Claude can connect to and which features are available.

Oversight by design. Administrative actions are recorded in an audit log that organization administrators can review. Sensitive operations on Anthropic's side require two-person approval. Usage exports are metering data only, so agencies can answer ATO and IG requests without moving sensitive material. Conversation history stays local on the agency-managed device.

Getting started

Claude for Government is generally available to federal and state agencies today. Agencies do not need a separate cloud-provider relationship to get started. Existing customers can move to the desktop application and bring their conversation history with them through an in-app import.

Our FedRAMP Secure Configuration Guide is available through Anthropic's trust center. The application deploys through standard agency MDM platforms.

New agencies can request access at claude.com/solutions/government. To join the early access for Claude Code CLI or Claude for Microsoft 365, contact our public sector team.

Original source

Today

Claude for Government is generally available for federal and state agencies. The platform, which delivers Claude's coding and agentic work capabilities through a FedRAMP High authorized environment, has been in public beta since July.

Agencies access capabilities comparable to Anthropic’s commercial customers, without compromising compliance requirements. New capabilities generally arrive on the commercial release cadence.

Claude works directly with files on the desktop, allowing agency staff to use skills, plugins and projects for memo creation, RFP reviews, casework, and other tasks. With Claude Code, public sector teams can build and modernize the software systems that underpin public services.

Claude for Government governance controls are purpose-built for public sector agencies. Administrators can set configuration defaults as well as allocate and control spending across departments. Security teams and authorizing officials get audit logs and documentation that supports the agency ATO process. Procurement officers can contract with Anthropic directly and award on general-availability terms.

The Claude Code command-line interface and Claude for Microsoft 365 are also rolling out in early access through the same environment and with the same administrative controls.

Billing, administration, and oversight

No seat fees. Agencies pay for usage in fixed increments with a hard not-to-exceed cap, so spend does not exceed what an agency has obligated. Administrators define user tiers with spend and model limits per group, track usage by user and by model, and get burndown alerts before a balance runs low.

Administration that matches how departments are organized. Department-level administrators allocate prepaid usage to sub-agencies while each manages its own users. Agencies connect their own identity provider for single sign-on, with self-serve setup in the admin portal. SCIM group mappings set rate limits, dollar caps, and allowed models for each seat tier. Layered configuration sets defaults for sub-agencies, including what Claude can connect to and which features are available.

Oversight by design. Administrative actions are recorded in an audit log that organization administrators can review. Sensitive operations on Anthropic's side require two-person approval. Usage exports are metering data only, so agencies can answer ATO and IG requests without moving sensitive material. Conversation history stays local on the agency-managed device.

Getting started

Claude for Government is generally available to federal and state agencies today. Agencies do not need a separate cloud-provider relationship to get started. Existing customers can move to the desktop application and bring their conversation history with them through an in-app import.

Our FedRAMP Secure Configuration Guide is available through Anthropic's trust center. The application deploys through standard agency MDM platforms.

New agencies can request access at claude.com/solutions/government. To join the early access for Claude Code CLI or Claude for Microsoft 365, contact our public sector team.

Original source

Principles for designing evals and hillclimbing against them without fooling yourself, and how the claude-api skill's build-eval and hillclimb commands put them to work.

AUTHOR

Lance Martin

PUBLISHED

Sep 28, 2026

READING TIME

12 min

Evaluations provide a signal on how your app or skill is performing on specific tasks. But designing evaluations, and improving performance on them without fooling yourself, is hard. We've added guidance for both to the claude-api skill.

With the skill, you can run /claude-api build-eval to build an evaluation inside your codebase, and run /claude-api hillclimb to improve your application against it, one change at a time, with a held-out set of examples to catch overfitting.

In this article, we highlight the principles of good eval design and hillclimbing first, then show how Claude Code with the claude-api skill applies those principles. We’ll close by showing a few examples of these commands.

EVAL DESIGN

Well designed evaluations have a few common elements (Figure 1):

1. Eval tasks mirror production. Sample tasks that you care about in “production,” or the setting in which the capability or application you are testing will be used. Sometimes tasks are picked because they are easy to generate or they are easy to grade. But it’s important to ensure that the task distribution represents what you actually care about.
2. Performance improves with stronger models and more thinking. More capable models and higher effort levels typically should perform better on an evaluation. If they don’t, ambiguous tasks or a miscalibrated grader often are hobbling performance.
3. There is “passable” headroom at the frontier. The most capable model at the highest effort should be well below 100% on the evaluation, otherwise you can’t reliably judge how changes impact performance. Importantly, the gap should not be explained by impossible or ambiguous tasks: a common tell is that a task fails every evaluation run, regardless of the number of replicates. A good task is one where two domain experts would reach the same verdict and everything the grader checks is stated in the task.
4. Low run-to-run variance. High variance is often due to poorly designed, ambiguous tasks or a grader that produces different verdicts on identical output. Variance can also hide in the configuration. For example, effort may not be applied consistently. Also, the environment can affect the results of the evaluation: leftover state from an earlier trial (a file, a git history) can hand the agent the answer.

Adversarial sampling

Model capability is jagged. If you pick cases because today's model fails them, you are sampling the valleys of one model's capability surface (Figure 2). The evaluation can end up measuring that model's failure fingerprint rather than what is intrinsically hard or valuable for your application to do.

Pick hard cases because a human judged them hard: a useful test is to be able to say why a task is hard before you include it. Include cases that are specific failures in your application derived from production traffic, bug reports, or tickets. However, don’t blindly trust user traffic: users sometimes try what they expect to work, so a task distribution drawn strictly from user traffic may skew easy.

/CLAUDE-API BUILD-EVAL

The build-eval command in the claude-api skill turns these principles into a guided workflow. When you run /claude-api build-eval in Claude Code, Claude interviews you, builds the eval inside your codebase, and pauses for approval at specific points.

Designing examples

Claude helps you sample inputs to build evaluations in this order:

1. Production transcripts, after asking about retention and sensitive data.
2. Bug reports and support tickets.
3. Five to ten cases you write by hand.
4. Cases synthesized from your codebase.

The skill prioritizes production traffic, but it can also generate synthetic data anchored in a few real examples that you provide. The skill instructs Claude to generate a simple page that shows you every input and waits until you confirm them. As an illustration, below we show an example set of inputs for an e-mail router application that the skill may ask the user to review (Figure 3).

Validating the grader

After the inputs, Claude proposes the cheapest grader that fits your application’s output:

• Programmatic verification: If the output possibilities are constrained, it uses a code based check (exact match, a label from a fixed set, JSON that matches a schema, tests that pass).
• LLM-as-judge: It will default to this type of check if the output space is open-ended, with many valid answers but clear quality criteria. In this case, a second model reads the input, the output and a rubric written as checkable claims (not a 1-to-5 scale), and returns a score with its reasoning. If you have a baseline to compare against, the judge instead reads both inputs in random order, without being told which is the baseline, and picks the better one. You pick the judge model, and it should not be the model you are testing.

Claude grades a handful of cases and asks whether you would have scored any of them differently (Figure 4). In general, it is important to read a sample of scored transcripts before believing your evaluator; scoring failures are among the most common ways an evaluation is misconfigured.

When you’ve validated the grader, the skill tells you the size of the evaluation set (cases × repeats × model, and roughly how long it will take), runs the baseline, and prints the score with a confidence interval. What you get back: the cases, the grader, the runner, one JSON line and one full transcript per case, and a plain page that lists each case's score with a link to its transcript. If you want more than that page shows (e.g., a chart), just ask and Claude will build it as an extra page next to it. By default, these extra pages are static files that open locally and load nothing from the network.

Diagnostic checks

During the baseline runs mentioned above, Claude checks a number of things:

• Grader: Claude runs the grader twice on the same output, and reports whether the verdict changed.
• Plumbing: Claude checks for timeouts, API errors, and cut-off answers to ensure infrastructure noise doesn't pass as model variance.
• Headroom: if the baseline already scores about 95% or higher, the skill warns the user and alerts that the hillclimb should aim to explore cost or latency rather than quality.

HILLCLIMBING

Now that you have a reliable means of grading your application’s performance on a task, you can try to improve it. Hillclimbing is an effective way to tune parameters like effort or prompts, which trade-off cost and performance. Some general tips for choosing where to apply it:

• Cheap iteration - It should be inexpensive (in terms of time, cost, and effort) to modify whatever surface you are focused on for hillclimbing. Many internal efforts and customers have focused hillclimbing on text, such as prompts and skills. These are easy to change and revert. In contrast, open-ended modifications to an agent harness during hillclimbing may involve extensive code changes.
• Attributable - Changes in the score on your evaluation should be attributable to the surface you are modifying during hillclimbing. For example, several successful applications of hillclimbing have focused on skill triggering. The evaluation metric (the trigger rate for the skill) is directly coupled to the skill description that is being modified.
• Well-scoped objective - One common failure mode is an open-ended request to improve performance without careful consideration of the headroom available in the evaluation; an evaluation that’s near saturation or a poorly scoped surface (e.g., an open-ended request to update the harness) is more likely to stall. One generally strong objective across various efforts is cost: even if an evaluation is saturated, you can ask Claude to find ways to reduce cost while keeping performance at parity.

Overfitting

Even a well-designed evaluation rarely matches the exact task distribution you care about in production. As a result, "overfitting" to an evaluation is a common problem and results in a system that performs better on an evaluation than on production traffic.

There are many ways an evaluation can "leak" into your harness (the code around the model, including prompts, tools, and loop that calls Claude). For example, consider an evaluation task that benefits from OCR, but OCR is rarely beneficial in your production tasks. The evaluation harness might add an OCR tool to your application, which improves on the benchmark without any impact on production. More broadly, hillclimbing may add features to that harness that address edge cases in the particular evaluation examples you’ve chosen. These harness additions improve your evaluation score, but don’t translate to improvements in production (Figure 5).

Three things can help address this:

• Split the cases. Use a train set that the hillclimber may read and a test set that is never seen. If the train set scores improve while the test set scores stay flat, then that is a common overfitting warning sign.
• Never paste failures into the prompt. If the hillclimber reads the failing transcripts, it should never paste the failure content into the prompt.
• Keep the answers structurally out of the model's reach. Models can sometimes “reward hack” by directly finding answers to evaluations.

As discussed below, the claude-api skill applies these principles for you.

/CLAUDE-API HILLCLIMB

The hillclimb command in the claude-api skill turns these principles into a guided workflow. When you run /claude-api hillclimb in Claude Code, Claude iterates to improve against a given evaluation. You choose what changes it can make including:

• Your system prompt
• Skills or instruction files
• Tool descriptions
• Model choice, effort level, and other API parameters
• Your harness code

Before it starts, Claude asks what you want to optimize (e.g., performance, or cost while performance holds) and then splits the evaluation set at random into test and train. With a cost goal, it considers a few common cost drivers, including prompt caching, auditing the prompt for compatibility with the selected model, and picking the model and effort setting.

Before the first round, Claude checks that the eval's noise (how far the score can move by chance alone) is smaller than the smallest improvement you'd act on; if it isn't, it says so and suggests more repetitions or cases.

Each round, Claude reads the previous round’s train transcripts and proposes one change as a patch. It aims each round at a change whose effect can show above the eval's noise: it fixes the failing behavior at its root (e.g., rewrites the section that causes it or adds a missing rule) rather than rewording a line. It then runs evaluation with the patched change. At this point, Claude applies a check: if the train set improves but the test set is flat, Claude suspects overfitting and reverts the patch. If there is a regression, Claude reverts. If train and test sets improve, it keeps the patch (Figure 6).

When the score stalls for two or three rounds, Claude reads each remaining train failure and sorts it by cause. It does the same early if no single fix could gain more than the eval's noise, and suggests more repetitions or cases, rather than spending rounds on changes too small to measure. This step can catch ambiguous evaluation cases, harness errors, or run-to-run variance.

Only legitimate failures are included in more hillclimbing rounds.

When hillclimbing completes, Claude leaves your code at the version that did best on the test set for your goal. It reports the test result against the baseline with confidence intervals (Figure 7). If the gain is within noise, it says so and recommends against merging.

EXAMPLES

Hillclimbing for cost reduction

We ran /claude-api hillclimb on an internal customer support benchmark with the goal of reducing cost and improving performance. The benchmark included 44 tickets, with 30 used for the search and 14 held out. It started on Opus 4.8 at default (high) effort settings with 74.4% decision accuracy on the search tickets and a token cost of 4.6 cents per ticket.

The hillclimb first audited the prompt, removing mandatory tool-call rituals, a scratchpad step, and contradictory rules. Then it tried Opus 5.5 on low effort. This cleared the baseline accuracy bar at 87.8% and cut cost to 1.9 cents per ticket, less than half the starting cost.

Part of that saving comes from Opus 5.5's pricing: input and output tokens cost 20% less than on Opus 4.8, and cache reads cost 60% less. Because Opus 5.5 cleared the bar, the hillclimb then stepped down a tier to check whether a cheaper model could clear it too. Sonnet 5 on low effort scored about the same, 88.9%, at about half the cost, 1 cent per ticket (Figure 8).

Finally, Claude improved the prompt with routing rules and a refund-cap cross-reference, bringing Sonnet 5 to 98.9% at about the same cost. On the 14 held-out tickets that the search never saw, the final configuration scored 90.5% against the original setup's 78.6%, at about one fifth of the cost.

Hillclimbing for performance improvement

Another example is our claude-api skill, which provides guidance on using our APIs and general tips for working with Claude (including the sub-commands discussed in this article). We want to ensure our skill can correctly implement code that uses our APIs, and we built an evaluation set derived from our documentation to test the skill.

On our evaluation, the skill started at 66%. We gave the hillclimber access to documentation and our SDKs, allowing Claude to identify errors and self-correct them (Figure 9). Claude found that the skill was missing coverage of eight features.

Adding sections for them in the skill improved performance to 74%. It then found errors in C# and Java type tables, boosting performance to 77%.

After the score stalled for two rounds, Claude analyzed the remaining failures and bucketed them by root-cause. A normal round makes one edit for the most common failure. This step makes no edit; it only sorts every remaining failure by cause. This reflection step was useful in a few ways:

• Reflecting across a collection of failures, the hillclimber found that the skill content was present but Claude was simply writing older API shapes (e.g., from its trained priors). To address, the hillclimber added a table near the top of the skill that guided Claude from the forms it remembered to the current ones: for example, from extended thinking with a fixed token budget, which the API now rejects on recent Opus models, to adaptive thinking, and from older versions of the web search and web fetch tools to the current ones. It also moved the C# and Java warnings against fixed-budget thinking above their adaptive-thinking examples. This improved performance to 80%.
• Tasks that never improved in performance despite addressing obvious content gaps are tells that the example or grader is flawed. One task asked for code that catches one error type, while its grader wanted a chain of at least three. Claude reworded the task. Another grader's instructions contradicted our docs, and testing the real API showed the docs were right. Addressing these, along with more skill edits, brought performance to ~88%.

GETTING STARTED

Note: Run claude update first. The claude-api skill ships inside Claude Code, so updating gets you the latest version of these commands.

Then, in Claude Code:

/claude-api build-eval
/claude-api hillclimb

These sub-commands can be used directly in Claude Code via the claude-api skill:

Run /claude-api build-eval if you want to generate an evaluation set for a particular problem. You can steer it by providing access to examples (e.g., traces). Claude will employ the guidance shared in this article to design the examples and grader, and ensure you approve the examples and the grader.

Run /claude-api hillclimb if you have an evaluation and want Claude to improve on this, guided by your goal (e.g., better performance, or lower cost while performance holds). Claude will employ the guidance shared in this article to check for overfitting while climbing and check for bugs in the eval itself, such as a grader that marks a correct-looking answer wrong or a harness error, both before the first round and whenever the score stalls.

With special thanks to Misha Khalman for skill development. With thanks to Misha Khalman, Michael Segner, Matt Bell, Matt Thanabalan, and Punit Shah for reviews, contributions, and product support.

Original source

Protection in layers

Protection starts with safeguards inside the model. Managed Agents and NVIDIA Open Shell add limits that sit outside the model and apply to what the agent does. Each layer is designed to enforce its limits independently, so protection doesn't depend on any single layer. The layers are modular, so companies can adopt the ones that fit their setup.

Claude Managed Agents does the work and holds the credentials

With Managed Agents, the agent loop runs on a separate server from the sandbox, the isolated environment where the work happens. Credentials, meaning passwords and access keys, are held in a separate vault, so the agent never sees them.

Managed Agents also provides audit trails, which record what each agent did, and integration with a company's existing access controls. Companies can bring their own sandbox setup and choose where and how it runs.

NVIDIA OpenShell sets what an agent can reach

OpenShell is open source secure runtime software from NVIDIA. It governs and monitors all AI agent behavior and enforces policies for every action. OpenShell blocks everything unless a rule allows it. It checks each tool an agent tries to use and applies rules to the files, network connections and data the agent accesses. The rules are enforced outside the agent, and OpenShell logs every decision it allows or blocks.

Teams can start with narrow permissions, review the log, and use Claude to tighten the rules toward the least access a task needs. OpenShell's policy prover then uses mathematical proof to confirm what the agent can reach under the rules the team wrote.

What Claude Managed Agents includes

• Production-grade agents with secure sandboxing, authentication, and tool execution handled for you.
• Long-running sessions that operate autonomously for hours, with progress and outputs that persist even through disconnections.
• Multi-agent orchestration, where agents can spin up and direct other agents to parallelize complex work.
• Trusted governance, giving agents access to real systems with scoped permissions, identity management, and execution tracing built in.

How teams use Managed Agents

• Notion lets teams hand work to Claude inside their workspace. Engineers use it to ship code, and other employees use it to produce websites and presentations. Dozens of tasks can run in parallel while the team works on the results together.
• Rakuten runs specialist agents across engineering, product, sales, marketing, and finance, each deployed within a week.
• Asana built AI Teammates, agents that work alongside people in Asana projects, take on tasks and draft deliverables. Using Managed Agents, the team added advanced features faster than it could have otherwise.

Availability

Managed Agents is available today. It can operate in a sandbox you control either running on your own infrastructure, or with a managed provider. NVIDIA OpenShell is open source under the Apache 2.0 license and available on GitHub and NVIDIA's developer resources page.

Original source

You can now submit plugins to the Claude directory through a new developer portal, track them through review, and see usage analytics once they’re live.

Every day, millions of people connect Claude to their apps, work tools, and data. Today, we're making it easier for developers to reach them.

Plugins package MCP connectors, Agent Skills, or both, and are the main way to build third-party extensions for Claude. Build a plugin, submit it through the new directory submission portal, and once approved, it's listed in the Claude directory.

Submit and track plugins in the directory submission portal

The directory submission portal is open to developers on paid Claude plans. There are two ways to submit and get your plugin published in the Claude directory:

• Single MCP connector: point to your remote MCP server.
• Plugin bundle: combine MCP servers and skills, host them on GitHub, and submit the repo. In Claude Code, plugins can also include LSPs, commands, hooks, and agents.

Whichever path you choose, the portal will guide you from submission to launch. You can:

• Auto-validate your plugin. Each submission is checked and safety-scanned as soon as you submit, so you catch issues early.
• Review status and feedback. See where your plugin is in the review process, results from the safety scan, and recommended changes.
• Publish when you’re ready. Once approved, you decide when to publish your plugin in Claude.

Stylized view of review status and recommended changes for a plugin. Data is illustrative.

Monitor and improve your plugin once it's published

Once your plugin is live, usage analytics show installs by product surface and version, so you can prioritize fixes and features for your users. On the discovery side, you’ll see how often your listing is viewed and which searches lead people to it, so you can refine it to reach new users.

Stylized view of usage metrics for a published plugin. Data is illustrative.

Bring a rich experience to users with MCP 2.0 and its extensions

Claude supports the latest MCP spec, commonly referred to as MCP 2.0, which includes a stateless core. You can improve the experience of using your plugin with two MCP extensions: MCP Apps, for interactive UI inside chat, and Enterprise Managed Auth, for zero-touch OAuth for enterprise users. Support for more MCP features and extensions is coming soon.

Start building plugins for Claude

Plugins are the main way for third-party developers to create extensions for Claude. Over the coming weeks, one discovery experience will roll out across Claude and Claude Code.

Skills and MCP connectors stay as building blocks, and the Claude directory will continue to list them. In the future, developers will be able to turn their connector listing into a plugin. If you have an existing skill, connector, or plugin on the Claude directory, you do not need to make any changes.

To get started with building plugins, read our docs for how to build a plugin and submit your plugin here.

Original source

Every day, millions of people connect Claude to their apps, work tools, and data. Today, we're making it easier for developers to reach them.

Plugins package MCP connectors, Agent Skills, or both, and are the main way to build third-party extensions for Claude. Build a plugin, submit it through the new directory submission portal, and once approved, it's listed in the Claude directory.

Submit and track plugins in the directory submission portal

The directory submission portal is open to developers on paid Claude plans. There are two ways to submit and get your plugin published in the Claude directory:

• Single MCP connector: point to your remote MCP server.
• Plugin bundle: combine MCP servers and skills, host them on GitHub, and submit the repo. In Claude Code, plugins can also include LSPs, commands, hooks, and agents.

Whichever path you choose, the portal will guide you from submission to launch. You can:

• Auto-validate your plugin. Each submission is checked and safety-scanned as soon as you submit, so you catch issues early.
• Review status and feedback. See where your plugin is in the review process, results from the safety scan, and recommended changes.
• Publish when you’re ready. Once approved, you decide when to publish your plugin in Claude.

Stylized view of review status and recommended changes for a plugin. Data is illustrative.

Monitor and improve your plugin once it's published

Once your plugin is live, usage analytics show installs by product surface and version, so you can prioritize fixes and features for your users. On the discovery side, you’ll see how often your listing is viewed and which searches lead people to it, so you can refine it to reach new users.

Stylized view of usage metrics for a published plugin. Data is illustrative.

Bring a rich experience to users with MCP 2.0 and its extensions

Claude supports the latest MCP spec, commonly referred to as MCP 2.0, which includes a stateless core. You can improve the experience of using your plugin with two MCP extensions: MCP Apps, for interactive UI inside chat, and Enterprise Managed Auth, for zero-touch OAuth for enterprise users. Support for more MCP features and extensions is coming soon.

Start building plugins for Claude

Plugins are the main way for third-party developers to create extensions for Claude. Over the coming weeks, one discovery experience will roll out across Claude and Claude Code.

Skills and MCP connectors stay as building blocks, and the Claude directory will continue to list them. In the future, developers will be able to turn their connector listing into a plugin. If you have an existing skill, connector, or plugin on the Claude directory, you do not need to make any changes.

To get started with building plugins, read our docs for how to build a plugin and submit your plugin here.

Original source