Keyterm prompting: boost transcription accuracy on difficult entities

Brands, drug names, and industry terms are naturally difficult to transcribe, even for the best STT models.

With keyterm prompting, pass up to 100 keyterms (1,200 characters total) with the keyterm query parameter when you open a connection. Ink-2 biases toward transcribing these terms correctly with no extra latency.

Here’s a clinician walking a patient through a prescription change, with Ink-2 and Deepgram Flux each given the same three keyterms.

Keyterms provided to both models:

Paracetamol

Ibuprofen

Aspirin

ink-2

Returns all three keyterms as given

flux-general-en

Transcribes "Paracetamol" incorrectly

Keyterm prompting on Ink-2 is also strong on getting names correct.

Keyterms provided to both models:

Sarita

Singh

ink-2

Returns both keyterms as given

flux-general-en

Misses both keyterms

We measure keyword recall on Earnings22, a benchmark built from real quarterly earnings calls: company names, tickers, acronyms, the kind of vocabulary general-purpose STT tends to find tricky.

Keyterm prompting takes Ink-2, the most accurate STT model, and boosts its recall by 20%. With prompting enabled, Deepgram Flux misses 13.5% of keywords and Ink-2 only misses 6.2%, so Ink-2 misses less than half as many.

Keyterm prompting works well for:

• Product and brand names (“Cartesia”, “Ink-2”)
• Drug names and clinical terms (“semaglutide”, “atorvastatin”)
• Names or addresses on a caller’s account
• Industry acronyms and internal jargon

Ink-2 has improved measurably on our purpose-built enterprise AI benchmark, EVA-Bench, in our evaluation. It holds up on the domain-specific vocabulary that shows up in real enterprise conversations (airline, IT service, HR), and the gains we saw between runs came from Cartesia’s own iteration, not from anything we tuned.

Joyce Li
AI Product Manager

Configurable turn detection: tune for faster latency or more accurate endpointing

Configurable turn detection allows you to tune latency and endpointing accuracy to optimize your agent’s conversational flow for your use case.

Lower latency

Closes the turn sooner

Ends a turn at each pause, splitting the request into three, so the agent can start replying earlier.

Higher turn-taking accuracy

Holds the turn longer

Waits through the caller's pauses and keeps the whole request as one turn.

Ink-2 ships with a balanced default, but now you can configure turn detection yourself: tune toward lower latency for snappy conversation, or toward higher turn-taking accuracy when capturing information correctly matters most. Set it when you open the connection, or adjust it mid-stream as the call unfolds.

Try it today

Keyterm prompting and configurable turn detection are live now for all Ink-2 users. Check out the docs for keyterm prompting and turn detection, and try Ink-2 at play.cartesia.ai.

We’re excited to release Ink-2: a speech-to-text model built for real-time voice agents.

It’s ranked #1 on Artificial Analysis’s streaming leaderboard for lowest word error rate†, with the most accurate built-in turn detection of any provider, so the model knows precisely when to listen and when to respond.

For voice agents, speech-to-text has to get three things right: accuracy, turn detection, and latency. If any one of these falls short, the experience breaks down. The agent may misunderstand the user, interrupt at the wrong time, respond too slowly, or make the conversation feel unnatural. Ink-2 was built to lead on all three.

Accuracy: Getting every word right

We’ve done extensive work on structured entity recognition like phone numbers, email addresses, alphanumerics, and dates. Ink-2 understands when it’s mid-entity and waits for the full sequence before committing, with no special prompting needed.

We built Ink-2 to be robust across a range of accents, which reflects real voice agent calls. On AppTek, a multi-accent benchmark spanning 14 English accents on real call-center dialogue, Ink-2 is the strongest streaming STT provider at 8% WER, vs. 10% for Deepgram Flux and 12% for ElevenLabs Scribe v2.

Accuracy also holds under production conditions, not just clean reference audio. Our internal benchmark samples audio directly from live voice agent calls covering non-native English speakers, background noise, and degraded audio from poor network conditions. Ink-2 achieves 6.5% WER, compared to 9.2% for ElevenLabs Scribe v2 and 9.4% for Deepgram Flux.

Both structured entities and real-world production audio are where accuracy actually gets tested in a live voice agent, not just in a clean benchmark.

Turn detection: Knowing when to listen and when to respond

Accuracy shows up clearly in benchmarks, but turn detection is where most voice agents fall apart in production.

Most voice agents decide a turn is over based on silence. If someone pauses long enough, the turn abruptly ends. This works in a test environment, but on a real call it means cutting a customer off mid-address, or jumping in right after “and my email is…”

We created Ink-2 with built-in semantic endpointing: the model reads meaning, not silence, to decide when a turn is over. It knows when an incomplete address is still being given or when a trailing thought isn’t a stopping point. The turn stays open until the model is confident the speaker is done.

Ink-2 emits three events natively, with no external VAD needed:

• turn.start — the user has begun speaking
• turn.eager_end — the model predicts the turn is wrapping up; your LLM can start generating early
• turn.end — turn confirmed complete

We measure turn detection against a human-labeled reference. Precision is how often the model is right when it calls a turn over (low precision means cutting people off), and recall is how often it catches every real end-of-turn (low recall means awkward dead air while the model keeps listening). F1 balances the two into a single score.

Ink-2 holds both precision and recall high at once, while the alternatives each trade one off for the other.

Latency: Keeping the conversation flowing

Similar to TTS latency, transcription latency has a direct impact on how natural a conversation feels. The metric we care about is Time-to-Final-Transcript (TTFT): how long it takes to get a final transcript from the moment the user finishes speaking. This is what determines whether your agent feels like it’s paying attention or like it’s buffering.

Ink-2’s latency is lightning fast, with a TTFT of 0.1s. And because turn.eager_end lets your LLM get a head start before the turn is fully confirmed, your agent responds fast enough to feel like it’s actually in the conversation.

Join the teams powered by Ink-2

Ink-2 is live at play.cartesia.ai, available directly via API and across the platforms voice teams build on, including LiveKit, Vapi, and Pipecat.

We’re excited to keep building alongside the best teams pushing voice AI forward and running Ink-2 in production. We’ve already pushed out a world class English model, and multilingual support is on the way, so Ink-2 fits the way your users speak, wherever they are.

Try Ink-2

Test Ink-2 out for yourself today

Speech-to-Text

Keyterm prompting for better accuracy — Set domain-specific terms, brand or product names, and rare or invented words so the model transcribes them correctly. See the keyterms guide, and try it out on the Playground or API.

Improved Playground features — Watch transcription in real time on a sample clip or your own audio. Configure keyterms and adjust turn detection sensitivity right in the Speech-to-Text Playground.

Text-to-Speech

Access control for voices and pronunciation dictionaries — Manage access to your custom voices and pronunciation dictionaries via the Playground or API. Set access to public to use them on third-party platforms.

Voice Agents

Live agent experiences via API — Power live transcripts and keep your systems in sync with calls in real time via turn events, word-level assistant text, interruption state, tool calls, and call IDs. See the Agents WebSocket API.

Tune noise suppression for each agent — Adjust how much noise suppression to apply to the caller’s audio. 0 is off and 100 is max, configurable via API and in the Playground.

Track call activity in the Usage dashboard — View daily call volume and minutes and spot usage spikes alongside spend, with date range and agent filters.

Voices

Expanded multilingual voices — 42+ voices in the voice library now speak up to 15 languages natively. Try out your favorite voice in a new language in the voice library, or retrieve its supported languages with the Get Voice API.

Developer Tools

Manage your organization via API — Programmatically invite and manage users in your organization using your Admin API key.

Credit usage breakdown — View and export usage by capability, model, voice, or API key on the credit usage dashboard, or pull the same data programmatically with the Get Credit Usage API.

Deprecations

Sonic-2, Sonic-turbo, and Sonic-3-2025-10-27 will be sunsetted after October 20, 2026. Migrate to the newest stable TTS snapshot before then.

The quickest way to get started with Sonic is by using an integration. If you are using the API directly, check out Cartesia’s client libraries and AI tools.

The references section contains complete documentation for all API endpoints and best practices.

Prompting tips

Get natural-sounding output from Sonic with minimal prompt engineering.

Custom Pronunciations

Specify custom pronunciations for words that are hard to get right, like proper nouns or domain-specific terms.

Volume, Speed, and Emotion

Control the speed, volume, and emotion of generated speech.

SSML Tags

Laughter, pauses, and mid-transcript controls

Compare TTS Endpoints

How bytes, SSE, and WebSocket differ for text-to-speech, and when to use each.

Output Format

How to find the right parameters for output audio

Migrating from Sonic 3 to Sonic 3.5

What's new in Sonic 3.5 and what to know before you switch.

Voices

Instant Voice Clone

Learn how to get the best voice clones from your audio clips

Pro Voice Clone

Create near replica voices by leveraging more of your data

Multilingual Voices

Make a voice sound native in new languages and accents.

Streaming Transcripts

Contexts and Continuations

Generating audio from partial transcripts

Context Flushing and Flush IDs

Separating a continuation into multiple parts

Buffering

Control how text is buffered before speech generation to balance prosody and latency

Learn how to stream input text to Sonic TTS

In many real-time use cases, you don’t have input text available upfront—like when you’re generating it on the fly using a language model. For these cases, we support input streaming through a feature we call continuations.

This guide will cover how input streaming works from the perspective of the TTS model. If you just want to implement input streaming, see the WebSocket API reference, which implements continuations using contexts. The Python and TypeScript SDKs handle the continue flag for you: ctx.push() sends each chunk with continue: true, and ctx.no_more_inputs() sends continue: false. See the WebSocket continuations example for working code.

Continuations are generations that extend already generated speech. They’re called continuations because you’re continuing the generation from where the last one left off, maintaining the prosody of the previous generation.

If you don’t use continuations, you get sudden changes in prosody that create seams in the audio.

Prosody refers to the rhythm, intonation, and stress in speech. It’s what makes speech flow naturally and sound human-like.

Let’s say we’re using an LLM and it generates a transcript in three parts, with a one second delay between each part:

1. Hello, my name is Sonic.
2. It's very nice
3. to meet you.

To generate speech for the whole transcript, we might think to generate speech for each part independently and stitch the audios together:

Figure 1: Generate transcripts independently & stitch them together.

Unfortunately, we end up with speech that has sudden changes in prosody and strange pacing:

Now, let’s try the same transcripts, but using continuations. The setup looks like this:

Figure 2: Generate transcripts using continuations.

Here’s what we get:

As you can hear, this output sounds seamless and natural.

You can scale up continuations to any number of inputs. There is no limit.

Caveat: Streamed inputs should form a valid transcript when joined

This means that "Hello, world!" can be followed by " How are you?" (note the leading space) but not "How are you?", since when joined they form the invalid transcript "Hello, world!How are you?".

In practice, this means you should maintain spacing and punctuation in your streamed inputs.

End complete sentences with closing punctuation (for example ., ?, or !).

If a streamed chunk does not end with sentence-ending punctuation, the model often treats it as an incomplete sentence. That can cause:

• Extra latency: Text may stay in the automatic input buffer until the model sees a clearer boundary or until max_buffer_delay_ms elapses (3000ms by default), so audio starts later than you expect.
• Audio artifacts: The model expects natural sentence endings; without closing punctuation, the generated audio sometimes ends with odd or distorted sounds.

When a user-facing utterance is finished, put terminal punctuation on the final segment (and signal that no more text is coming on the context when appropriate, for example no_more_inputs() in the SDK or continue: false over the WebSocket).

Automatic buffering with max_buffer_delay_ms

When streaming inputs from LLMs word-by-word or token-by-token, we buffer text until the optimal transcript length for our model. The default buffer is 3000ms, if you wish to modify this you can use the max_buffer_delay_ms parameter, though we do not recommend making this change.

If you plan on using speed or volume SSML tags with buffering, make sure decimal values are not split up. Submitting 1.0 as 1, ., 0 will result in unintended failure modes.

How it works

When set, the model will buffer incoming text chunks until it’s confident it has enough context to generate high-quality speech, or the buffer delay elapses, whichever comes first.
Without this buffer, the model would immediately start generating with each input, which could result in choppy audio or unnatural prosody if inputs are very small (like single words or tokens).

Configuration

• Range: Values between 0-5000ms are supported
• Default: 3000ms

Use this only if

• you have custom buffering client side, in which case you can set this to 0
• you have choppiness even at 3000ms, in which case you can try a higher value

// Example WebSocket request with `max_buffer_delay_ms`
{
  "model_id": "sonic-3.5",
  "transcript": "Hello", // First word/token
  "voice": {
    "mode": "id",
    "id": "a0e99841-438c-4a64-b679-ae501e7d6091"
  },
  "context_id": "my-conversation-123",
  "continue": true,
  "max_buffer_delay_ms": 3000 // Buffer up to 3000ms
}

Let’s try the following transcripts with continuations and the default max_buffer_delay_ms=3000:

['Hello', 'my name', 'is Sonic.', "It's ", 'very ', 'nice ', 'to ', 'meet ', 'you.']

Turn detection controls

Adjust turn-start, turn-end, and eager-end thresholds to balance response speed against detection accuracy. See the Turn Detection guide for more details.

Professional Voice Clones now available on Sonic 3.5

They deliver better speaker similarity and more stable generation than on Sonic 3, especially for rare or non-native accents.

Test sample rates

Set any sample rate from 8 to 44.8 kHz based on your intended use case on the TTS Playground.

Upload knowledge bases

Let agents access domain-specific information via docs including FAQs, pricing, and guides. Set it up with the Knowledge Base guide.

Batch outbound calling

Send up to 5,000 calls with a single API request. Control how many run at once, track status, retry failed calls, and schedule batches for later. See the batch calling guide.

Zero data retention

Enable ZDR so transcripts, audio recordings, and logs from agent calls are never stored. Now available for Enterprise customers.

Call Redaction API

Delete the transcript, audio recordings, and logs for any call on demand. The call record itself is preserved with non-sensitive operational metadata. Available via the Delete Call API.

Smoother preview calls

Preview calls on the Playground now have lower latency and fewer dropped connections. Try calling one of your voice agents or a Cartesia voice in the voice library.

Your most-used voices, now multilingual

10 of our most popular voices, including Brooke and Blake, now support ~10 new languages with native-sounding accents, so you can serve global audiences with a consistent voice.

Better localization

Localized voices now have even more natural-sounding accents across supported languages with recent updates to the localization model.

Ink-2

our state-of-the-art streaming STT model — Build responsive real-time voice experiences with built-in turn detection and accurate transcription even in noisy environments. It currently supports only English, with additional languages coming later.

• Try it on the Cartesia Playground.
• Integrate via API, Python, TypeScript/JavaScript, LiveKit, and PipeCat.
• Switching from Deepgram Flux? See the migration guide.

Sonic 3.5 is now generally available

Our most natural, expressive TTS model is out of preview and production-ready. Use the sonic-3.5 alias for the latest stable snapshot. See the Sonic 3.5 model overview.

• Switching from Sonic 3? See Migrating from Sonic 3 to Sonic 3.5 for what’s new and what to check before moving production traffic.

Speed and volume controls

Dial speed and volume up or down so voices sound the way you want. See the speed and volume guide.

Line / Agents

• More natural conversations — Eligible Line agents run on Sonic 3.5 (TTS) and Ink 2 (STT) by default, improving naturalness, pacing, latency, and turn-taking. No config change needed.
• Bring your own Twilio account — Connect your Twilio account and import your existing phone numbers. You can still use the free Cartesia-provisioned numbers included in your plan. See the Twilio integration guide.
• SIP trunking (Beta) — Connect your existing phone system directly to Cartesia’s voice agents using SIP (Session Initiation Protocol) trunking. Reach out at [email protected] for early access.
• Phone number and provider APIs — Provision, import, and configure phone numbers and providers via API. See the phone numbers API.

Voices

• Filter voices by locale — Find voices with the right accent by passing a locale (e.g. en-GB) into the language field when listing voices. The API response now includes a country field (e.g. GB) to make each voice’s regional accent easier to identify. See the voices API reference.
• 57 new voices across 11 locales — Added 57 new voices in the voice library, including ar-AE, de-DE, en-CA, en-GB, en-NZ, en-US, en-ZA, es-MX, fr-CA, he-IL, and th-TH.

Sonic 3.5

Sonic 3.5 is now available on sonic-3-latest. We’d love for you to try it and tell us what you think.

Why you should try it

• More natural speech, pacing, and emotional expression, especially noticeable on expressive, conversational, and support-style transcripts.
• Cleaner audio quality across all languages and voices.
• Better alphanumeric read-out — confirmation codes, order numbers, phone numbers, IDs, and emails sound meaningfully more natural, in all supported languages.
• Step-change multilingual performance, particularly Hebrew, Japanese, Spanish, Hindi, German, Korean, and French.
• English heteronyms — tricky English heteronyms like “read,” “bass,” and “bow” now pronounce correctly in context.

How to try it

1. Point your API call or Playground request to the model ID sonic-3-latest.
2. Keep your existing voice IDs, request shape, and prompting — no code changes required for most customers.
3. Send us feedback on any voice or transcript that behaves differently than you expect.

As with any -latest alias, sonic-3-latest can be updated without notice and is not recommended for production. Pin to a dated snapshot (e.g. sonic-3) for production traffic.

What to know to be successful

• Spell tags still work the same way. If you already wrap alphanumerics in <spell>...</spell>, you don’t need to change anything — you’ll just get better-sounding output. See Prompting Tips for more details.
• If you use custom delimiters (commas/periods between characters or groups) to control pacing, our recommended format has changed. Use spaces between characters and commas between groups, e.g. A B C, 1 2 3 instead of A, B, C. 1, 2, 3. See Prompting Tips for more details.
• Speed and volume controls are temporarily disabled on sonic-3-latest. If you rely on speed or volume augmentation (including via SSML), stay on sonic-3 for now. We believe that Sonic 3.5 has more natural pacing and you may find that you don’t need to use speed control as much when using this model.
• Timestamps behave slightly differently. If you use end-of-word timestamps for interruption handling, you should not see a meaningful change. If you depend on beginning-of-word timestamps, please test carefully and reach out if you see regressions for your use case.
• Existing Professional Voice Clones (PVCs) do not carry over to sonic-3-latest. Professional Voice Clones are pinned to the base model they were trained on (e.g. sonic-3) and will function as a standard voice clone for this model. For more information, see Pro Voice Clone.
• Providing proper context to the model improves naturalness. Please see our buffering guide here for more details.

Where to look for help

• Sonic 3.5 model overview
• Prompting tips for Sonic 3.5
• Model aliases and snapshots

API

• Usage and API keys — New HTTP APIs for usage and API keys.
• Speech-to-text (STT) — Improved documentation. See STT streaming.

Playground

• Improved call details experience — Click on a transcript to seek audio when reviewing calls.
• Cancel call — You can now cancel active calls from the Playground, for example, if you mistakenly made outbound calls.
• Keys — One Keys screen with Standard and Admin tabs when your org has access.
• Pronunciation dictionaries — In-app list and detail views for dictionaries tied to your organization.

Line / Agents

• LLM provider — Agent inference paths standardize on Anthropic; setup copy and defaults no longer point voice agents at Gemini keys.
• OpenAI WebSocket mode — We now support OpenAI’s WebSocket mode, which offers low latency for agent inference.
• Transfer and end call interruption — In the Line SDK, you can set transfer and end call as uninterruptible.

Models / Voices

• Voice Library — 34 new voices across 10 locales (ar-001, de-DE, en-US, en-AU, he-IL, hi-IN, ko-KR, tl-PH, ta-IN, te-IN).
• Voice cloning — More reliable uploads for M4A (and similar) source clips when creating clones.

Self-hosted

• Playground — Add voices to your on-prem deployment.
• Pronunciation dictionaries — POST /onprem/add-pdict to import dictionaries from cloud into self-hosted stacks.
• STT — Optional streaming STT via your configured provider integration in self-hosted environments.

A note from Albert, Chief Scientist at Cartesia.

A lot of our work at Cartesia starts from a simple constraint: intelligence should be natural enough to interact with in real time. That sounds like a product goal, but quickly turns into a modeling one. If you want systems that can listen, reason, and respond quickly, inference efficiency is not a nice-to-have. It is part of the model quality bar.

We’ve spent years pushing state space models forward, from the original Mamba line to the ideas that are now showing up across the field. Mamba-3 is the next step in that progression. Where Mamba-2 was shaped around training efficiency, Mamba-3 is built for the world we are actually living in now: one where post-training, deployment, and real-time workloads are increasingly dominated by inference.

This post is cross-posted from Goomba Lab, which is led by Cartesia Chief Scientist Albert Gu.

Since the release of Mamba-2 in mid-2024, most architectures have switched from Mamba-1. Why? Mamba-2 made the bet that training efficiency was the largest bottleneck for state space models (SSMs), and thus simplified the underlying SSM mechanism to deliver 2−8× faster training compared to its predecessor, leading to wider adoption.

Since then, the LLM landscape has started to shift. While pretraining is still super important, more attention has been focused on post-training and deployment, both of which are extremely inference-heavy. The scaling of post-training methods, especially with reinforcement learning with verifiable rewards (RLVR) for coding or math, requires huge amounts of generated rollouts, and most recently, agentic workflows, such as Codex, Claude Code, or even OpenClaw, have pushed inference demand through the roof.

Despite the clear, growing importance of inference, many linear architectures (including Mamba-2) were developed from a training-first perspective. To accelerate pretraining, the underlying SSM was progressively simplified (e.g., the diagonal transition was reduced to a scalar times identity). While this brought training speed, it left the inference step “too simple” and squarely memory-bound — the GPUs aren’t brr-ing but moving memory most of the time.

In this new age of inference, we care a lot about pushing the boundaries of the quality-efficiency frontier: we want the better models to run faster.

A natural question arises:

What would an SSM designed with inference in mind look like?

The Mamba-3 Model

What’s missing? The main appeal of linear models is in their name: compute scales linearly with sequence length because of a fixed-size state. Unfortunately, there is no free lunch. The same fixed state size that enables efficient computation forces the model to compress all past information into one representation, the exact opposite of a Transformer, which stores all past information through a continuously growing state (the KV cache) — a fundamental difference. So, if we can’t grow the state, how do we make that fixed state do more work?

We see that earlier designs simplified the recurrence and the transition matrix to make training fast. However, the change also reduced the richness of the dynamics and left decoding memory-bound: each token update performs very little computation relative to memory movement. This provides us with three levers we can pull: (1) make the recurrence itself more expressive, (2) use a richer transition matrix, and (3) add more parallel (and almost free) work inside each update.

From these insights, we improve upon Mamba-2 in three core ways that:

1. increase the expressivity of the SSM mechanism through a more general recurrence derived from our exponential-trapezoidal discretization scheme,
2. expand the state-tracking capabilities by modeling a complex-valued SSM system, and
3. improve the model’s general performance with little impact on decode latency by using multi-input, multi-output (MIMO) SSMs, which model multiple SSMs in parallel, instead of the current single-input, single-output (SISO) SSMs.

Through these three changes, Mamba-3 pushes the frontier of performance while maintaining similar inference latency.

Notably, all three of these changes are inspired by the more “classical” control theory and state space model literature.

Our work goes against the grain of many modern linear architectures, which use alternative interpretations of recurrence (such as linear attention or test-time training) that don’t easily capture these concepts.

Architecture

What has changed in the Mamba-2 layer? Beyond the three methodological upgrades to the core SSM discussed above, we’ve revamped the architecture a bit to make it more in line with conventional modern language models.

Based on the diagram, you’ll notice we’ve changed a couple of things. On a high level,

Norms. We added in QKNorm 1, which empirically stabilizes the training of Mamba-3 models. The addition of this norm brings Mamba-3 in line with contemporary Transformer and Gated DeltaNet (GDN) models. With QKNorm, the RMSNorm from Mamba-2 becomes optional. However, we empirically find that it may still be worth keeping in hybrid models due to helping length extrapolation capabilities. More on this later.

Goodbye Short Conv. We’ve been able to get rid of the pesky short causal convolution of Mamba-1/2 by combining (1) simple biases on B and C after BCNorm with (2) our new discretization-based recurrence. The new recurrence implicitly applies a convolution on the input to the hidden state, and we show how this is the case in Part 2 of our blog.

Finally, you’ll notice a couple of new components, namely RoPE and MIMO projections. The RoPE module expresses complex-valued SSMs via the interpretation of complex transitions as rotations, forgoing the costly reimplementation of kernels. The MIMO projections expand the B and C matrices to the appropriate representation needed for MIMO SSMs.

We dig into the motivation and exact implementation of these two in greater detail in the second part of our blog (lots of goodies there 🎁), so for now, just think of them as standalone, fundamental improvements that individually contribute to improving the model’s performance and/or capabilities.

Finally, our overall architecture now adopts interleaved MLP layers following the standard convention of Transformers and other linear models.

Empirical Results

We evaluate our final Mamba-3 model against other popular linear alternatives and the Transformer baseline.

Language Modeling

We find that our new Mamba-3 model outperforms the prior Mamba-2 model and strong linear attention alternatives, such as GDN, on language modeling across various pretrained model scales. Mamba-3-SISO is directly comparable to prior linear models; for example, it matches Mamba-2 exactly in architecture shapes (model dimensions, state size, etc.) and has comparable training time. Our MIMO variant of Mamba-3 further boosts accuracy on our downstream tasks by more than 1 percentage point over the regular Mamba-3 at the 1B scale, with the caveat that MIMO requires longer training times but not longer decoding latencies!

Retrieval Tasks

Linear models, with their fixed-size state, naturally underperform their Transformer counterparts on retrieval-based tasks. As expected, within pure models, the Transformer is superior on retrieval tasks, but Mamba-3 performs well within the class of sub-quadratic alternatives. Interestingly, the addition of MIMO further improves retrieval performance without increasing the state size.

Given this innate deficit but overall strong modeling performance, we predict that linear layers will be predominantly used in conjunction with global self-attention layers in the future.*

*at least for language modeling

Hybrid models that combine the general memory-like nature of linear layers with the exact database-like storage of self-attention’s KV cache have been shown empirically to outperform pure models while enabling significant memory and compute savings, and we do find here that the combination of linear layers with self-attention enables better retrieval compared to a vanilla Transformer.

However, we highlight that the exact way that these linear models interact with self-attention is not fully understood. For instance, we find that the use of the optional pre-output projection for Mamba-3 improves the length generalization performance on the synthetic NIAH tasks at the slight cost of in-context real-world retrieval tasks. Furthermore, even the details of the returned norm such as placement, e.g., pre-gate vs post-gate, and type, grouped vs regular, have non-negligible effects on accuracy on tasks composed of semi-structured and unstructured data, such as FDA and SWDE.

Kernels Here, There, and Everywhere

We’re excited to see what people build with Mamba-3. To help facilitate this, we are open-sourcing our kernels, which are on par in terms of speed with the original Mamba-2 Triton kernels.

Benchmarking Latencies

Prefill Latency

Prefill+Decode Latency

Prefill and prefill+decode (same token count for both prefill and decode) latencies across sequence lengths for a 1.5B model on a single H100-SXM 80GB GPU. A batch size of 128 was used for all sequence lengths, and wall-clock times in seconds are reported over three repetitions.

When comparing models at the 1.5B scale, Mamba-3 (SISO variant) achieves the fastest prefill + decode latency across all sequence lengths, outperforming Mamba-2, Gated DeltaNet, and even the Transformer with its highly optimized vLLM ecosystem. Furthermore, Mamba-3 MIMO is comparable to Mamba-2 in terms of speed but has much stronger performance.

Mamba-3 SISO’s Triton-based prefill maintains nearly identical performance to Mamba-2, demonstrating that the new discretization and data-dependent RoPE embeddings do not introduce additional overhead, while Mamba-3 MIMO only incurs a moderate slowdown for prefill due to its efficient TileLang implementation. The strong decode performance for both Mamba-3 variants can be partially attributed to the CuTe DSL implementation, which was made significantly easier by the simplicity of Mamba-3 components.

Design Choices

We spent a lot of time thinking about how to make the kernels as fast as possible without compromising on ease-of-use. We ended up using the following stack: Triton, TileLang, and CuTe DSL.

The use of Triton was quite an easy choice. It’s pretty much standard for architecture development (the great flash linear attention repo is purely in PyTorch and Triton) for good reason, as it enables better performance than standard PyTorch by enabling controlled tiling and kernel fusion while being a platform-agnostic language. Triton also has some pretty nifty features, like PTX (a GPU-oriented assembly language) injection and its Tensor Memory Accelerator support (on Hopper GPUs) for bulk, asynchronous transfers from global to shared memory.

Our MIMO prefill kernels were developed with TileLang instead. The additional projections corresponding with the variant present an opportunity where we can reduce memory IO via strategic manipulation across a GPU’s memory hierarchy. Unfortunately, Triton didn’t provide the granularity of memory control we desired, so we opted for TileLang, which allows us to explicitly declare and control shared-memory tiles and create register fragments, reusing memory more efficiently while still being high-level enough for us to develop the kernels quickly.

Since we’ve been hammering the importance of inference and decode, we decided to use CuTe DSL for our decode kernels. Through its Python interface, we’re able to generate low-level kernels using high-level abstractions from CUTLASS. Here, we practically have CUDA-level control, enabling us to develop highly-performant kernels tailored to the specifications of our hardware (Hopper GPUs, in this case). With fine-grained control over tensor layouts and warp specialization, we built a kernel that takes advantage of all the bells and whistles in the GPU.

Importantly, these implementations across varying levels of GPU abstraction are made possible by the underlying algorithmic design of Mamba-3’s simple, lightweight additions and their clever instantiations. We discuss details such as the exact fusion structure and kernel DSL in more depth in our full release.

Next Up

Glad you made it to the end of Part 1! There were a lot of details regarding our kernels and experimental results and ablations we didn’t have time to cover in this post, but don’t fret! Everything can be found in our paper, and the kernels have been open-sourced at mamba-ssm!

Up next, the second (and final) part of the series delves into the three core improvements to Mamba-3 and their SSM foundations, and gives some directions we’re especially interested in.

Breaking

• Text-to-Agent (T2A) API — Text-to-Agent workflow for Line is deprecated.

API

• Error responses — For Cartesia-Version: 2026-03-01, we now return structured JSON. See API Errors. API versions before 2026-03-01 continue to return legacy error formats (for example HTTP Title: Message).
• Voices — PATCH /voices/{id}: voice owners can now update accent and gender. Voice creation validates language. Invalid voice UUIDs and pronunciation-dictionary IDs return 404 instead of ambiguous errors.
• PVC model routing — PVC voices require a dated model ID (e.g. sonic-3-2026-01-12) instead of sonic-3. See Pro Voice Clone.
• Voice search — Name and metadata search is diacritics-insensitive.

Playground

• Pro voice clones
  • Clearer language mismatch messaging
  • Background noise removal is now a simple on/off control
  • Fine-tuning model support:
    • Removed support for older models
    • Now only sonic-3-2026-01-12 is supported
• Multilingual agents — Multilingual agent configuration is now supported in the Playground.
• Agents UI — Search by call ID and agent ID.

Billing

• Concurrency — Organizations can receive notifications when concurrency nears configured limits.

Model / voice

• Professional Voice Clones — Backend updates improve stability of the professional voice cloning workflow.
• Accents & filters — Additional accent options (e.g. Irish, New Zealand, South African, Belgian) and locale aliases for accent filtering in APIs and Playground.
• Voice Library — 94 new voices across 17 locales (including Arabic, German, English variants, Spanish, Finnish, French, Hebrew, Hindi, Japanese, Korean, Polish, Portuguese, Swedish, Telugu, Thai, and more).

Self-hosted

• On-premises — API for managing voices on self-hosted deployments.

Cartesia SDK

• cartesia-js v3.0.0 (Mar 2) — Major updates:
  • New features: flush_id included in chunk and voice changer binary responses; output_format and infill support; inline WebSocket response types; byte endpoint returns ArrayBuffer; improved WebPlayer and client export.
  • Fixes: memory leak and timing issues with abort signals/listeners, handling of empty Content-Length, and TimeoutError now includes a message.

See cartesia-js releases for full details.

Line

• History Management API: You can add or replace the history provided to your agent, for example, to summarize a long conversation.
• Custom User Events: You can send bidirectional custom events between your client and the agent. You could use this, for example, if you have a web application with UI interactions.
• Uninterruptible Messages: You can set messages as uninterruptible. A common use case is a legal disclaimer at the beginning of a call.
• End Tool Call Improvements: The default end call tool call is more conservative to prevent calls from ending prematurely.

API

• Increased reliability of API connections

Cartesia SDK

• cartesia-python v3.0.0 (Feb 9). See full details in cartesia-python releases.

Playground

• Shipped a new TTS page
• Shipped a new Voice Creation page
• Shipped a new Agents page

Model changes

• Improved pronunciation of real-world text patterns across languages
  • Enhanced support for structured and formatted speech patterns: numbers, dates, times, currency, phone numbers, IDs, percentages, and amounts/measurements.
  • Support for various date formats (YYYY-MM-DD, YYYY/MM/DD, 年月日).
  • Support for measurement units (meters, kg, tablespoon, gigabytes, etc.) with locale awareness.
  • Support for domestic and international phone number formats with locale-specific chunking for French, Italian, German, Portuguese, Korean, and more.
  • Improved alphanumeric ID handling with katakana/hiragana readings and Latin acronym transliteration to katakana for Japanese.
  • Improves all languages except English, Hindi & other Indic languages, Arabic, Hebrew, Chinese, Swedish, Georgian, Bulgarian, and Tagalog (targeted for future updates).
• Support for regional and locale-specific pronunciation within languages
  • Regional voices use region-specific terms in addition to accent (e.g. Belgian and Swiss French “nonante” vs. Canadian and French “quatre-vingt-dix”).
  • Region-specific number terminology, currency symbols, date formats, and measurement units.
  • Locale-aware date and time formatting (e.g. Russian year suffixes, French/Spanish time conventions).
  • Locale-aware currency symbol handling (e.g. $ as “dollars” in en_US and “pesos” in es_MX).
  • Locale pronunciation falls back to the primary country for that language (e.g. US for English, Brazil for Portuguese). We will continue to expand locale-aware support.
  • Improves all languages except English, Hindi & other Indic languages, Arabic, Hebrew, Chinese, Swedish, Georgian, Bulgarian, and Tagalog (targeted for future updates). Existing regional pronunciation for English voices (e.g. British) is unaffected.

Voice changes

• Voice Library: 39 new voices across 21 locales

Breaking changes effective June 1, 2026

The following model snapshots and languages are discontinued effective June 1, 2026:

• sonic: All snapshots and languages
• sonic-english: All languages
• sonic-multilingual: All languages
• sonic-2: snapshots sonic-2-2025-04-16, sonic-2-2025-05-08, sonic-2-2025-06-11; languages it, nl, pl, ru, sv, tr, hi
• sonic-2: snapshot sonic-2-2025-03-07; all languages
• sonic-turbo: snapshot sonic-turbo-2025-06-04; languages it, nl, pl, ru, sv, tr
• sonic-turbo: snapshot sonic-turbo-2025-03-07; all languages

The following endpoints are discontinued effective June 1, 2026:

• Voice Embedding: POST /voices/clone/clip (replacement: Clone Voice)
• Mix Voices: POST /voices/mix (no replacement)
• Create Voice: POST /voices (replacement: Clone Voice)

The following endpoints stop accepting voice embeddings effective June 1, 2026:

• TTS (bytes): POST /tts/bytes (replacement: Voice ID)
• TTS (SSE): POST /tts/sse (replacement: Voice ID)
• TTS (WebSocket): WSS /tts/websocket (replacement: Voice ID)

API

Regionalization — Calls routed to US, EU, APAC by origin.

Parameterized outbound calls — Docs

Pronunciation dictionaries — Docs

Model changes

• Sonic-3 model versioning scheme introduced
  • New preview track: sonic-3-latest (continuous updates for early access and feedback).
  • Stable track: sonic-3 always points to the most recent stable release.
  • Immutable dated snapshots: sonic-3-YYYY-MM-DD never change.
  • Details: Continuous updates and model snapshots
  • Promotion to stable checkpoint:
    • sonic-3-2026-01-12
      Included improvements: consistent speed & volume, custom IPA pronunciations with stronger adherence, Hindi prosody improvements, Korean prosody/intonation improvements.

Voice changes

• Featured Voices launched — Curated set of 30+ best-performing voices (e.g. Cathy, Henry).
• Voice Library — December: 25 new voices across 6 languages.
• Voice Library — January: 9 Spanish voices (Mexican, Colombian, Castilian).

Playground

• Voice library usability improvements (test with your own scripts, call an agent per voice).
• One-click Report Issue on TTS Playground.
• Mini voice picker (recently used + saved) on TTS page.
• PVC UI + reliability (loading skeletons, error messages, better behavior with large datasets and silence).

Line

• Line SDK v0.2 — Repo. Improved DX, long-running tool-call handling, committed turns, better turn-taking and transcription.

Reimagine your voice with our AI voice changer

Let our AI re-deliver your words in another voice, exactly the way you want them to sound.

TRY IT OUT

TALK TO SALES

TRUSTED BY 50K+ CUSTOMERS

Transform your voice today
Experience precise control over the speech you generate on our platform. Show us how you want something to be said, and our voices will deliver it perfectly, every time.

EXTENSIVE VOICE LIBRARY

Discover a diverse collection of unique voices in our voice library to bring your content to life.

CUSTOM VOICES

Mix multiple voices, customize speed and emotions to design your custom voice.

TRANSLATION AND LOCALIZATION

Our AI voice changer excels at localization, preserving the original voices and emotions.

Instantly change your voice from a 3 second clip Scale up to hours of content with Fine-Tuning

• SOURCE
• ORACLE
• BRIGHTON
• SOURCE
• HERO VOICE
• ROBOTIC MALE
• SOURCE
• PIPPA
• OVERLORD

"Cartesia’s Sonic model is a game-changer for our Conversational Video Interface. Its ultra-low latency of 90ms and high-quality voice generation have enabled us to create truly immersive real-time conversations with AI digital twins. The natural voices and voice design capabilities have elevated our product to new heights."
— Hassaan Raza, Co-Founder and CEO, Tavus

Make your content accessible to a global audience
Sonic supports seamless speech in 15 languages, with more added every release.

15 LANGUAGES

From Japanese to German—any language you need, we’ve got it.

LOCALIZATION

Localize a given voice to any accent or language.

• German
• English
• Spanish
• French
• Japanese
• Portuguese
• Chinese
• Italian

What our customers say
Join the growing list of companies opting for Sonic.

TRY IT NOW

TALK TO SALES

"Cartesia’s voice API power dynamic and empathetic conversational experiences that are consistently dependable. What really stands out to me is how natural and considerate the responses feel—especially the empathetic tone in statements like ‘I’m sorry, that must be frustrating.’"
Sami Ghoche, CEO of Forethought

"In 1999, Salesforce brought software to the cloud. In 2025, 11x is killing software as we know it and unleashing the era of digital workers. To realise this vision, we needed AI voice technology that feels truly human. Cartesia’s technology gives our AI digital workers reps the speed, reliability, and natural expressiveness required to engage customers at scale.
It's the only solution fit for our relentless drive toward innovation.”
Keith Fearon, Head of Product & Growth, 11x

"Before conversational voice models like Cartesia, Thoughtly relied on legacy text-to-speech APIs from major cloud providers. Nearly two years later, the evolution of this technology is staggering—customers can clone their voice and hear it speaking autonomously over the phone in just 60 seconds.”
Torrey Leonard, CEO, Thoughtly

Lifelike, expressive voices for every use case
Support
Power support experiences that delight your customers.

Gaming
Bring your storytelling to life with immersive voices

Content
Create content that engages viewers and drives clicks.

Media
Narrate content for podcasts, news, and publishing.

Healthcare
Empower healthcare with voices that patients trust.

Sales
Scale sales with lifelike voices that lead to conversions.

Voice Agents
Build responsive AI voice agents for any use case.

Dubbing
Go global with localized voices and accents for every language.

Avatars
Create expressive, relatable AI avatars for any use case.

Logistics
Automate complex logistics with voice-enabled systems.

Recruiting
Screen candidates with AI-powered voice interviews.

Accessibility
Make your content accessible to anyone, anywhere.

How to Use Our AI Voice Changer

STEP ONE

Try Cartesia's AI voice changer on our website. Simply create a free account and upload your original recording.

STEP TWO

Choose your preferred voice and language settings. Transform your voice with our lifelike voice changer.

STEP THREE

Apply voice changes to your content to hear it in a different voice, with complete control over the delivery.

Frequently asked questions

• How does the free AI voice changer work?
• What is a realtime voice changer client?
• Can I use the voice changer for gaming?
• Is the voice changer free to use?
• How do I access the voice changer?
• What makes our voice changer unique?