Deepgram Release Notes

Flux TTS is generally available on /v2/speak: 36 voices, a new Voice Agent default, and what changes in your code, cost, and error handling.

TL;DR: Flux TTS is generally available today on /v2/speak, over a streaming WebSocket and a batch REST endpoint. It's turn-based, and that one change moves real work off your side of the line. Your /v1/speak Aura code is untouched.

Flux TTS is our text-to-speech API for developers building voice agents. It went generally available on a new endpoint, /v2/speak. This post is the orientation: what it is, what changes in your code, where it plugs in, and what it costs to run.

To start with the why, /v1/speak is request-shaped. You hand it text, and it hands you audio. That model is correct for a lot of things and it's the wrong shape for a conversation because a conversation has turns, and a turn is the unit that actually matters. A turn is what gets interrupted. A turn is what carries tone into the next one.

So /v2/speak is turn-based, and almost everything else follows from that. In practice a turn looks like this:

[Python code example omitted]

There is no sentence detection or buffering, and audio comes back while you're still sending.

Best of all, your existing Aura code keeps working exactly as it does today. Every Aura voice is still there. You can move when you have a reason to.

Two transports, one Flux TTS contract

Flux TTS ships on both a streaming WebSocket and a batch REST call at the same /v2/speak path.

The WebSocket is the one built for agents. You open a connection, stream tokens in as your LLM produces them, and audio starts coming back before you've sent the last one. The batch endpoint is a single request that returns a finished audio file, and it's the right call for anything that is not happening live: voicemail, notifications, pre-rendered prompts.

They deliberately share a contract. model and the media settings behave the same on both, so moving a voice between your real-time path and your batch path doesn't mean re-learning anything. The conversational pieces, Flush and Interrupt and the turn lifecycle, only exist on the socket because they only mean something there. Compressed formats work the other way: mp3, opus, flac, and aac are batch-only, since the streaming socket sends raw samples with no container.

If you’re interested in learning more, take a look at the docs' batch versus streaming page if you're genuinely unsure which side a workload belongs on.

Thirty-six Flux TTS voices today, more languages later

Voices are model strings shaped flux-{voice}-{language}, like flux-alexis-en. The launch catalog is thirty-six English voices across seven accents: American, British, Irish, Australian, Indian, Singaporean, and Filipino. Twelve featured all-rounders out of a catalog of thirty-six is the subset you'll see surfaced first.

That English-only catalog matters for one thing beyond accent choice: a couple of the runtime controls are language-dependent. So today they work everywhere and that stops being automatically true when more languages arrive. The voice catalog has the full list with descriptions.

What Flux TTS lets you delete

This is the part I'd lead with if someone gave me thirty seconds. Adopting Flux TTS is mostly a subtraction.

Your text chunker. Flux TTS never needs you to find sentence boundaries or hold text back. It streams a turn's audio as tokens arrive. If you built sentence detection to feed another TTS API, it comes out.

Your reconnect-between-turns hack. The model carries its sense of how it has been speaking across the whole connection. So turn twenty sounds like turn one without you doing anything. Plenty of people reconnect between turns to stop voices drifting. There's nothing left for that code to do, and only a new connection resets it.

Your character counting. Every turn reports billable_character_count, and SessionMetadata totals the session when it ends. The server already knows whatever you were tallying locally to predict a bill.

Your own guess at what the caller heard. When somebody talks over your agent, sending Interrupt with a playback_offset gets you back text_spoken and text_remaining, so the history you hand your LLM matches what actually reached the caller's ears. That's a whole class of bug, the agent referencing a sentence nobody heard, that stops being your problem. You still track one playback counter, but what you delete is the timestamp bisection and string slicing that used to turn that counter into a cut point.

If you've solved this before, you solved it with timestamps. The standard toolkit is word or character-level timings: you bisect them against your own playback clock to find the cut point, slice the string, and hope the two stay in agreement when your audio path changes. It works and it's yours to maintain. Flux TTS does the slice server-side and hands you both halves as plain strings, which is the difference between a feature you own and a field you read.

This is the big one, so big that it will have its own post.

What's changed in Deepgram Flux TTS since Early Access

If you were on Early Access, the surface just grew considerably. At EA, /v2/speak understood three messages: Speak, Flush, and Close. Everything below landed today.

Interrupt, with the spoken-text split described above.

Configure, so you can change speaking rate mid-call without dropping the connection. speed takes seven values from 0.85 to 1.15, and the change lands at the next natural break so it doesn't sound abrupt. speed arrives on the batch endpoint at the same time.

Markup stripping. If your prompt template still emits SSML, or audio tags from ElevenLabs or Cartesia, Flux TTS strips them, speaks the clean text, and warns. It's a closed list, so your Markdown and HTML still pass through and get read aloud.

expressivity, in beta. A whole number from -2 to 2 that moves a voice off the delivery we tuned it to, set at connect time on either transport, and fixed for the life of the session. 0, the default, is the only value validated for production; moving off it can introduce hallucinations and pronunciation errors, more so the further you push it. Audition non-default values through the API or at talk.deepgram.com before you ship them.

Where Deepgram Flux TTS plugs in

The Voice Agent API. Flux TTS is now the default speak provider. Omit agent.speak from your Settings message and you get Flux TTS on flux-kit-en, where you used to get Aura's aura-asteria-en. That default flip comes with a breaking change: a session that omits agent.speak while requesting mp3, opus, flac, or aac now gets INVALID_SETTINGS instead of audio, because Flux TTS streams raw samples and has no container. Name an aura-* model explicitly if you need that config to keep working as-is. To opt into Flux TTS on purpose, set version to v2 and use a flux-* model. Mid-call, UpdateSpeak switches voices, and a turn already in flight finishes in the voice it started with. If you want a voice agent without assembling the pipeline yourself, this is the shortest path there is.

Your own pipeline. The Python, JavaScript, and Java SDKs all ship a speak.v2 client. Pair it with a streaming STT and your LLM and you have the standard three-box agent.

Frameworks. LiveKit and Pipecat both ship Flux TTS support today. LiveKit's TTSv2 class covers streaming and batch, documented at the LiveKit integration guide. Pipecat's DeepgramFluxTTSService tracks the playback position Interrupt wants, documented at the Pipecat integration guide.

Starter apps for Node, Flask, FastAPI, Django, and Java, if you'd rather read something that runs.

What Flux TTS costs you to run

Two operational facts to plan around, and the first one surprises people.

Concurrency is per plan and per region, and the regional gap is wide. On streaming, pay-as-you-go runs up to 45 concurrent connections in North America and 5 in the EU and Australia, Growth moves North America to 60 and leaves the EU and Australia at 5, and Enterprise starts at 100 in North America and 50 in each of the others. Batch REST has its own, smaller ceiling: 15 in North America on both pay-as-you-go and Growth, 5 in the EU and Australia on both, and Enterprise starts at 25 across every region. If you're deploying into Europe, check that ceiling before you size anything, because it's the number most likely to bite you in week one.

Sessions have two clocks. A connection closes after 60 seconds with no inbound client message, and any session closes at the one-hour mark. Send a WebSocket Ping between turns if your agent might sit quiet, and treat the hour as a reconnect you schedule.

Billing is per character on the text that actually gets synthesized, reported per turn and totaled per session. Markup that gets stripped is never billed, since stripping runs before the count.

How Flux TTS fails

Flux TTS is deliberately hard for developers to knock over. Most problems arrive as a Warning on a connection that stays open, which is the right design for something carrying a phone call and also means the failures you hit are quiet ones. Error always closes the connection, Warning never does, and that distinction is most of what you need.

One error worth naming here since it's new at GA: INVALID_SETTINGS fires when a Voice Agent session omits agent.speak while asking for a compressed format like mp3, opus, flac, or aac. Flux TTS is now the default and it only streams raw samples, so that combination fails instead of falling back to Aura the way it used to. Name an aura-* model explicitly if you need the old behavior (more in "Where Deepgram Flux TTS plugs in").

The ten that matter, with what each looks like in your logs, are in a future post coming soon.

Start building with Deepgram Flux TTS

Create a Deepgram account for an API key and $200 in free credit, point a WebSocket at /v2/speak, and use flux-alexis-en. The streaming quickstart is about fifteen lines.

If you’re coming from Aura, migrating from /v1/speak is mostly deleting things, which is the best kind of migration.

And if you build something with this, I'd genuinely like to see it. Flux TTS is new enough today that nobody has built the interesting things yet, and we have a way in for developers who want to go deeper than the docs. Join the community and share what you’ve built.

Original source

Introduction

Last week, we launched Flux TTS, the second model in the Flux family and the first conversation-native text-to-speech model built for real-time voice agents. Where Flux STT reimagined speech recognition for live conversation, Flux TTS does the same for the speaking half: it holds the whole conversation in memory as it speaks instead of starting fresh on every line, so tone, pacing, and pronunciation stay consistent from the first turn to the tenth. No SSML, no style tags.

That launch post covers why we built it. This one is about wiring it up — Flux STT on listen, Flux TTS on speak, your LLM of choice in the middle. Both models are generally available now, and Flux TTS is free to build with through September 12, 2026.

Check out the Pipecat docs here

Introduction

Building a production voice agent means choosing an orchestration framework and wiring in speech models for both directions of the conversation. A Pipecat voice agent can run Deepgram Flux for speech-to-text and Flux TTS for the voice.

A standard Pipecat quickstart needs a Silero VAD, a smart-turn analyzer, an STT service, and a third-party TTS. Flux folds transcription and turn detection into one model, so two of those components disappear. You configure fewer services and thresholds, with fewer hand-offs to debug between parts that were never designed to agree with each other.

Both halves of the stack are production-ready today. Flux STT and Flux TTS are generally available, and both support self-hosted and on-prem deployment alongside the cloud API. This guide builds a real-time voice agent that replaces two separate pipeline components with a single speech model.

Key takeaways

Building on that, the Flux stack removes two components from the standard Pipecat pipeline before you write any application code.

• Flux handles transcription and turn detection in one model; Pipecat auto-requests ExternalUserTurnStrategies, so you skip manual turn-strategy configuration.
• Silero VAD becomes optional with Flux; keep it only if you want STT metrics.
• Deepgram's latency docs report lower agent response latency versus STT-plus-VAD pipelines.
• Pipecat includes DeepgramFluxTTSService; the default voice is flux-alexis-en.
• Flux STT and Flux TTS are both generally available, and both deploy in the cloud, self-hosted, or on-prem.

Why Pipecat and Deepgram Flux fit together for production voice agents

Flux combines transcription with model-native turn detection, which makes the VAD and the separate turn analyzer optional. Most quickstarts put VAD and turn-analysis logic outside the transcription service, so STT handles the words while something else handles the turn boundary.

What Pipecat handles in the pipeline

Originally built inside Daily as internal tooling for real-time conversational AI, Pipecat is an open-source Python framework for voice and multimodal agents. Its vendor-neutral architecture lets you choose providers across the pipeline, including text-to-speech, and Daily itself is only an optional transport.

At its core, it handles orchestration across services, moving audio frames between your transport, your STT service, your LLM, and your TTS. In its default setup, it also runs a SileroVADAnalyzer plus a local smart-turn model to decide when the user has finished speaking. Those are the two pieces Flux makes redundant.

What Flux adds that a standard STT service doesn't

Deepgram's latency docs report lower agent response latency than traditional STT-plus-VAD setups, because the model detects turn endings earlier and more accurately. Deepgram's launch post positions Flux as conversational speech recognition built to understand when a speaker is done.

It reduces false interruptions while keeping latency low. If you set eager_eot_threshold, EagerEndOfTurn can fire before EndOfTurn at the cost of more LLM calls. Instead of a speech_final flag inferred from silence, Flux emits structured turn events from the same model that produces the transcript.

The architecture changes because the thing deciding whether you're done talking has actually understood what you said.

Where Flux TTS fits as the voice

Once the LLM produces text, something has to say it without blowing the latency budget. Flux TTS is Deepgram's Speak v2 synthesis model, exposed in the framework as DeepgramFluxTTSService. Voices use the flux-{voice}-{lang} naming pattern.

The cascade route described here gives you component-level control over the LLM and turn thresholds while exposing per-service metrics.

Setting up your Pipecat project and Deepgram credentials

Three API keys and one CLI command take you from an empty directory to a scaffolded agent. Deepgram's official Pipecat integration guide uses the same setup path.

Installing the Pipecat CLI and scaffolding a project

Before you run the CLI commands below, make sure you have Python 3.11+ and uv installed. Then install the CLI and scaffold:

uv tool install "pipecat-ai[cli]"
pipecat init
cd pipecat-bot
pipecat create --name pipecat-deepgram --bot-type web --transport daily --mode cascade \
--stt deepgram_flux_stt --llm openai_llm --tts deepgram_tts --no-deploy-to-cloud

The create command wires Flux STT, an OpenAI LLM, and Deepgram TTS into a cascade over a Daily WebRTC transport. Finish with cd pipecat-deepgram/server and uv sync to install dependencies. It's a rare thing in voice AI: a scaffold command that actually leaves you with a working demo instead of a pile of stub files.

Getting your Deepgram API key

Both Flux STT and Flux TTS read the same DEEPGRAM_API_KEY, created in your Deepgram dashboard. You'll also need an OpenAI API key for the LLM and a Daily API key for the scaffolded WebRTC transport.

Managing three keys for a first run is simpler than it might seem at first. Thankfully, though, each one maps to exactly one service, so a 401 tells you immediately which key to check. The Daily key belongs to the transport. For more information, check out the pipecat integration docs here.

Configuring environment variables

With all three keys in hand, copy the template with cp .env.example .env, then fill in four values:

• DEEPGRAM_API_KEY: your Deepgram key
• DEEPGRAM_VOICE_ID: the scaffold's variable for the legacy DeepgramTTSService, using the older Aura-2-era voice format; leave a value in it, because an empty string may return a 400 error
• OPENAI_API_KEY: from your OpenAI dashboard
• DAILY_API_KEY: from your Daily dashboard

Wiring Flux speech-to-text into the pipeline

With your project scaffolded and credentials in place, the STT side of the pipeline needs just one change. Swap in one constructor and the external turn-control pieces drop out.

Replacing a generic STT service with DeepgramFluxSTTService

Inside bot.py, the scaffold already uses the Flux service because you passed --stt deepgram_flux_stt. If you're migrating an existing bot, the swap looks like this:

from pipecat.services.deepgram.flux.stt import DeepgramFluxSTTService
stt = DeepgramFluxSTTService(
    api_key=os.getenv("DEEPGRAM_API_KEY"),
    settings=DeepgramFluxSTTService.Settings(
        min_confidence=0.3,
    ),
)

Use the settings= object, not the flat model and params constructor arguments. Both were deprecated in 0.0.105, with removal targeted at 2.0.0, so code written against them will break on that major version. Defaults are sensible: model flux-general-en, and server-side end-of-turn detection keeps the pipeline simpler.

Handling turn events: start, end, and eager end of turn

Four handlers give you hooks into the conversation: on_start_of_turn, on_end_of_turn, on_eager_end_of_turn, and on_turn_resumed. StartOfTurn is your barge-in signal; Deepgram's Flux API reference describes it as more reliable than external VAD and guaranteed to carry a non-empty transcript.

The eager pair only fires when you set eager_eot_threshold. Use it only when reducing response latency is worth the extra LLM work. TurnResumed then tells you the user kept talking.

The service source shows that the framework doesn't yet ship built-in gating to start LLM or TTS work early on EagerEndOfTurn or cancel it on TurnResumed. Eager handling is yours to build for now.

Skipping external VAD and turn detectors

You can drop the Silero import for turn control. The framework's service docs say Flux "automatically requests ExternalUserTurnStrategies at start," so you don't configure turn strategies by hand.

The same docs mark VAD as optional when Flux drives turn detection; include a SileroVADAnalyzer only if you want the STT metrics it feeds. Per the v1.6.0 changelog, STT services with server-side end-of-turn detection recommend ExternalUserTurnStrategies automatically, and your own setting still wins if you set one.

Adding Flux TTS and an LLM to complete the loop

The v1.6.0 release notes say the framework ships a dedicated DeepgramFluxTTSService, distinct from the older DeepgramTTSService that targets legacy Aura-2 voices. Check which one you're importing.

Configuring DeepgramFluxTTSService and voice selection

The voice catalog is English-only for now, all in the flux-{voice}-{lang} format per Deepgram's voice docs. Configure the service like this:

from pipecat.services.deepgram.flux.tts import DeepgramFluxTTSService

tts = DeepgramFluxTTSService(
    api_key=os.getenv("DEEPGRAM_API_KEY"),
    settings=DeepgramFluxTTSService.Settings(
        voice="flux-alexis-en"
    ),
)

Here, flux-alexis-en is the default voice, and the settings= object is where the Flux voice lives; DEEPGRAM_VOICE_ID can be used to override the voice from your .env settings otherwise it uses the default.

The service streams from the Speak v2 WebSocket at https://api.deepgram.com/v2/speak, which produces raw audio only. The Flux TTS quickstart lists linear16 (the default), mulaw, and alaw encodings, and the streaming endpoint rejects batch-only parameters like container.

Connecting an LLM for response generation

You already wired in OpenAI with the --llm openai_llm flag, so there's nothing to add for a first run. The framework's modular service architecture lets you swap LLM providers without changing pipeline code, so your Flux configuration survives a later model change.

At runtime, Flux emits a final transcript when it detects the end of the user's turn, the LLM streams tokens, and Flux TTS speaks them as they arrive.

Testing the full cascade locally

Run uv run python bot.py --transport daily to start the agent. Interrupt it mid-sentence while you test, since should_interrupt defaults to True and the bot should stop the moment you speak.

It's a small, satisfying moment when you talk over the bot and it actually shuts up. One catch: standard interruption_strategies such as MinWordsInterruptionStrategy don't work as expected with Flux, because the service pushes an InterruptionTaskFrame upstream instead of letting the transport control interruption (issue #2988).

Deploying and scaling your Pipecat voice agent

For production, pick a transport, decide whether Flux should run in your own infrastructure, and watch turn detection under real traffic. It's also the moment to re-check whether a cascade or speech-to-speech architecture fits your call volume.

Choosing a transport for production

If Daily's WebRTC infrastructure doesn't fit, alternatives include: FastAPI WebSocket, LiveKit WebRTC, Vonage WebRTC, and the keyless peer-to-peer SmallWebRTCTransport.

Telephony runs through WebSocket serializers for Twilio, Telnyx, Plivo, Exotel, Genesys, and Vonage. For managed hosting, Pipecat Cloud reached general availability on January 8, 2026, and Daily states that anything you run there can be self-hosted exactly the same way.

Self-hosted and VPC options for compliance-sensitive teams

Audio never has to leave your network. Deepgram documents self-hosted Flux with two variants, flux-general-en and flux-general-multi, per the deployment environments docs.

The variants run in private infrastructure, including VPCs and dedicated cloud or bare-metal deployments. Self-hosting requires a Deepgram Enterprise Plan. For AWS-native teams, Flux is a supported model family on Amazon SageMaker, where the container runs network-isolated in your own AWS VPC with no connection to the Deepgram Cloud.

Capacity planning runs off the engine_flux_max_streams and engine_flux_used_streams metrics the self-hosted Engine exposes, and the SageMaker container tunes concurrency with flux.max_streams=25.

Flux TTS follows the same path. It deploys in the cloud, self-hosted, or on-prem, with the same model and the same benchmarks across all three, so the speaking half of the pipeline can run under the same HIPAA, data-residency, and enterprise compliance rules as the listening half.

Monitoring latency and turn detection in production

Use enable_metrics=True in PipelineParams to collect per-service TTFB and processing time, plus time-to-first-audio for TTS (metrics docs). Add a UserBotLatencyObserver to measure how long the user waits between going quiet and hearing the bot respond, which is the number callers actually feel.

On the Deepgram side, track percentiles rather than averages. A mean hides the tail, and the tail is where turn detection fails. Deepgram recommends tracking percentile latency at p50, p95, and p99.

It also recommends watching how often EagerEndOfTurn resolves to TurnResumed versus EndOfTurn to judge your eager threshold. Use the Flux baseline as a starting point, then compare it with your own traffic.

Test turn detection against your own callers' audio. Create a free account, grab your $200 free credits, and test Flux with your own callers' audio.

Check out the Pipecat docs here

FAQ

What's the difference between Deepgram Flux and Nova-3 for voice agents?

Choose Flux when turn detection is part of the speech model; Nova-3 fits transcription-heavy workflows where your pipeline already owns turn boundaries. Deepgram's models overview recommends Flux for real-time agents and interactive, turn-based experiences.

It recommends Nova-3 for meetings and captioning, along with workflows that involve multi-speaker or noisy audio. Flux streams over /v2/listen; Nova-3 uses /v1/listen. For migrations, treat the endpoint change as a pipeline behavior change. Flux is built for real-time voice agents while keeping word error rate low.

Does Pipecat support Deepgram Flux out of the box?

Yes. Deepgram's Flux Multilingual announcement confirms partner integrations with Twilio, Vapi, LiveKit, Pipecat, and Jambonz, so the same model travels with you if you later switch frameworks.

Can I skip external VAD entirely when using Flux in Pipecat?

For turn control, yes, but test the handoff before deleting every VAD-related line. Log on_start_of_turn, on_end_of_turn, and on_turn_resumed, then compare those events with the actual audio.

If the bot starts listening, stops listening, and handles resumed speech correctly, Flux is owning turn management. Keep Silero only when you need VAD-derived STT latency measurements; maintainer guidance in issue #4279 confirms that role.

Check out the Pipecat docs here

What Flux TTS voice options work with the Flux TTS service?

Treat voice IDs as configuration, not code constants. The catalog will keep growing — more languages and voice cloning are on the roadmap — so keep the selected voice in an environment variable or config file and pass it into DeepgramFluxTTSService.Settings(voice=...).

Use flux-alexis-en as your local default, and make an unsupported voice fail loudly during startup rather than mid-call.

Can Flux run in a self-hosted or VPC environment?

Yes, on both sides of the pipeline. Treat the STT side as a capacity-planning project before you ship: choose a supported GPU, avoid NVIDIA T4, and keep Flux on its own Engine node. If the container starts but streams fail under load, check Engine stream capacity before tuning turn thresholds. Flux TTS deploys in the cloud, self-hosted, or on-prem as well, so both halves of the agent can run inside the same compliance boundary.

Original source

Text to speech that reads the whole conversation, not just the current line. Handles interruptions, holds context, and ships natural voice agents faster.

The voice sounds warm when it picks up, then the warmth drains out by minute three. The customer cuts in and the agent doesn't know where it left off, the account number comes out wrong, and the tone that felt right in the demo feels scripted on a real call.

Every team building a voice agent has run into this. The TTS layer works, right up until it meets a real conversation.

Today we're launching Flux TTS, the second model in the Flux family. Flux STT reimagined speech-to-text for live conversations. Flux TTS does the same for text-to-speech: the first conversation-native TTS built for real-time voice agents.

And, it's free until September 12.

[Start building →]

[Try the demo →]

Why voice agents break the Text-to-Speech layer

Today's TTS was built for narration: audiobooks, voiceover, IVR prompts. But voice agents need a model that can keep up with a live conversation as it unfolds.

When you take a model built for the first job and drop it into the second, three things fall apart.

Tone resets every turn because the model doesn't know it's in a conversation. Every line is a fresh request, so an empathetic tone can't hold across a long call, and a shift in the customer's mood can't carry into the next response.

You end up doing the model's job for it. To get the delivery right, developers wrap everything in SSML, style tags, and prompt engineering, hand-feeding the model the context it should be picking up from the conversation itself.

And once a line starts generating, you can't change it, there is no slowing it down mid-sentence, no fixing pronunciation, no adjusting when the caller's mood shifts. Whatever the model started, it finishes, whether or not it still fits.

This is why we built Flux TTS to hold the whole conversation in memory as it speaks, instead of starting fresh on every line. Once the model can hear what's happening around it, a lot of things get easier:

• Automatic expressive delivery. No SSML, style tags, or prompt engineering.
• Cross-turn consistency. Tone, pacing, and pronunciation hold across the whole call.
• Native interruption handling. The server reports what the caller actually heard.
• Time to first audio as low as 80ms.
• Production-grade entity accuracy on the terms that break voice agents.
• Deploy anywhere: cloud, self-hosted, or on-prem.

“We work with voice agent developers every day, and they consistently run into the gap between TTS that sounds impressive in a demo and TTS that holds up across a real, multi-turn call. Deepgram Flux TTS takes a different approach by tracking across turns instead of treating each line as a standalone request. Deepgram is making TTS part of the agent’s conversational architecture rather than a layer developers have to tune line by line, and we’re excited to bring that capability into the Pipecat ecosystem.”

Kwindla Hultman Kramer
CEO, Daily

[Try the demo →]

Voices that read the room

What makes Flux TTS voices expressive by default, even without style tags?

A big piece of it comes down to training data. Instead of studio audio with style markup layered on top, Flux TTS learned from real human conversation, with all the disfluencies and the shifts in tone that happen when people actually talk to each other. So the voice comes out empathetic when a customer is frustrated and precise when the details matter, without anyone having to write a style tag to make it happen. The model picks up what's going on in the conversation and adjusts tone, pacing, and emotional register on its own.

The other key part is that the model keeps track of the whole conversation while it's speaking. That means the voice doesn't reset every turn like other TTS does. With most models, turn one comes out warm and slow, turn three sounds bright and clipped, and by turn five you're getting a flat scripted read. Flux TTS keeps one voice going the whole session, so the tone, pacing, and pronunciation you got at the start are still there at minute ten.

"At jambonz, we know how important it is to build voice agents that hold up in production, especially when connecting AI applications to live phone calls. Deepgram Flux TTS tackles a problem every voice AI developer encounters: speech that flattens over the course of a conversation and forces teams into endless SSML tuning. By combining conversation-aware speech with low-latency performance and deployment flexibility, Deepgram gives teams a way to improve the caller experience without having to hand-tune every turn or give up control of their stack."

Dave Horton
Founder

Accurate when precision matters

For businesses in healthcare, finance, and insurance, a voice agent lives or dies on the details: the drug name, the account number, the policy ID, the dollar amount. Those are the moments the customer is actually listening for, and the moments where getting it wrong costs the most.

The inputs that cause the most trouble in production are the specific ones: alphanumerics, drug names, technical strings, IVR patterns, currency, dates. These rarely surface during vendor demos, but they come up constantly in real calls. Flux TTS is built to handle them, with reliable delivery on the account numbers, order confirmations, and dosages voice agents actually have to say out loud.

The benchmarks back that up. Flux TTS has the lowest word error rate of any read-aloud TTS we tested, with a median WER of 2.2%, roughly half of ElevenLabs and a third of Cartesia. On hard prompts, the kind of inputs that come up in production, Flux TTS comes in at 3.4% and beats the next-best model by 47%.

Built for the shape of a conversation

Most TTS is one-way: text goes in, audio comes out. That works fine for narration, but it falls apart the moment a caller cuts in, a turn ends unexpectedly, or the agent needs to change speed or pronunciation while it's still speaking.

Flux TTS is built to participate in the conversation. Every utterance has an explicit lifecycle (Idle, Generating, Completing) with events the agent can act on. When a caller cuts in, the server reports exactly what the caller heard, so the agent resumes cleanly instead of guessing or repeating itself. Speed, pronunciation, and pacing can change while the model is speaking and server-side flushing lets the model decide when a phrase is ready, so developers don't have to write chunking logic to balance latency and naturalness themselves.

Three architectural choices make that work at real-time speed without sacrificing quality.

A high-fidelity neural codec. Most TTS uses a lossy codec that restricts the model's expressive range to avoid audio artifacts. Flux TTS uses a high-fidelity neural codec, the same family as recent research from Kyutai, Meta, and Google, so the full expressive range comes through without a fidelity cap.

Interleaved text-to-audio generation. Standard TTS processes the entire input before starting to generate audio, so first-audio latency grows with the length of the response. Flux TTS generates text and audio together, so audio starts streaming on the first token and first-audio latency stays under 200ms regardless of response length.

That matters because in a voice agent, the TTS shares a budget with the STT and the LLM, and a caller starts noticing lag around 800ms of total round-trip time.

A Mamba state-space backbone. Other models run on transformers, where holding conversational context gets expensive fast, so the model resets every turn. Flux TTS uses a Mamba state-space backbone with fixed-size memory, so it can hold context across the whole session without the quadratic cost.

Every other TTS on the market forces a tradeoff between expressiveness, stability, and latency: pick two, live without the third. These three choices are why Flux TTS doesn't. All the orchestration developers wire up themselves today, VAD, barge-in logic, state management, re-feeding conversation history to fake context, now lives in the model.

And it runs wherever your data has to live. Flux TTS deploys in the cloud, self-hosted, or on-prem, with the same model and same benchmarks across all three. HIPAA, data residency, and enterprise compliance are built in, so a voice agent in healthcare, finance, or government can run under the same rules the rest of the business does.

The full conversation loop: Flux STT + Flux TTS

You can wire up the best STT and the best TTS on the market, and the agent still ends up feeling stitched together. The two models don't share context, and the caller hears it as robotic: the tone doesn't shift when they get frustrated, slowing down has no impact on pace, and there are weird lags while they wait for a response.

Flux STT and Flux TTS are both built by Deepgram, purpose-built to work together. Today, they run as a paired stack behind one API, so wiring both into an agent is a single config. Next on the roadmap, the two models will share state directly. That's when the agent starts responding to how the caller actually sounds: matching their pace, shifting tone with their mood, pronouncing names back the way they were spoken.

That's the entire configuration. Flux STT on listen, Flux TTS on speak, your LLM of choice in the middle, and the agent runs. Together, Flux STT and Flux TTS make voice agents work in the environments that break most stacks.

Proven in restaurants

Take a restaurant. Between engine noise at a drive-thru, the person ordering at the kiosk next to you, a fryer hiss, and multilingual environments, it might be the hardest environment there is for a voice agent.

Restaurants also come with their own unique vocabulary and rhythm:

• Brand names: Dr. Pepper, Frappuccino
• Prices: $4.99, 2 for $10
• Non-English food names: açaí, queso, jalapeño
• Hours and locations: 8:00am to 8:00pm, San Diego, CA
• Confirmation pacing: repeating the order back at the speed the customer is thinking, not a scripted read

This is exactly the type of situation Flux TTS was designed to handle. In restaurant scenarios, it beats every model we tested – not just overall, but also on ability to maintain a consistent tone within the voice agent experience.

View the demo here.

What's next

Flux TTS ships at GA with expressive, conversational English voices. Coming next:

• More languages: dozens of them, with voice identity preserved across them
• Voice cloning: brand voices, customer voices, personalized agents, no engineering required
• Emotional controls: speaking style, non-verbal cues, dynamic range, without retraining
• Flux Shared State: Flux STT and Flux TTS sharing acoustic and semantic context, so what one model perceives conditions on what the other just produced

Our research team will publish technical deep dives on the pieces that make Flux TTS work, including the state-space architecture, interleaved generation, and cross-turn context modeling, in a companion series.

"Voice agents are moving from stitched-together demos to production systems, where every part of the pipeline has to run together in real time. Deepgram Flux TTS delivers speech that's tuned to the conversation and stays low-latency. Making it available on LiveKit's agent platform gives developers a way to add conversation-aware voice to the same real-time infrastructure they already build and deploy on.”

David Zhao
CTO, Livekit

Try Flux TTS today

Flux TTS is available now in the Deepgram API, and free through September 12.

Deepgram's stance on the voice economy is simple: we meet builders where they already are. Flux TTS is available through REST and WebSocket APIs directly, and through the platforms most voice agent teams already use, including Pipecat, LiveKit, jambonz, and Twilio.

Get started:

• Try the demo
• Get started with Flux TTS
• Sign up for an API key
• Try Flux TTS for Restaurants

Whether you're integrating Flux TTS into an existing voice agent stack or building fresh, the path is the same. Drop it in, wire up the conversational events, and ship.

Original source

Flux TTS, the newest model in Deepgram’s Flux family, brings conversation-native speech to enterprise voice agents in production as voice becomes the preferred interface for AI and the company surpasses $100M in ARR.

SAN FRANCISCO (August 12, 2026) – As voice agents move from impressive demos into business-critical interactions, enterprises need systems that can do more than sound natural.

They must maintain context, adapt when conversations take an unexpected turn, and reliably complete critical tasks at production scale without requiring extensive human intervention. Deepgram, the real-time AI infrastructure company underpinning the Voice AI economy, today released Flux TTS, a conversation-native text-to-speech (TTS) model purpose-built as the conversation engine for enterprise voice agents. To experience Flux TTS firsthand, listen to voice samples and explore the developer documentation at https://deepgram.com/product/text-to-speech/flux

Flux TTS joins Flux speech-to-text (STT) within Deepgram’s broader Voice AI platform and is backed by the same enterprise-grade runtime and deployment flexibility organizations already use to process speech with Deepgram at scale. Through Deepgram’s Voice Agent API, enterprises can orchestrate speech recognition, agent reasoning, and speech generation through a single API, reducing integration complexity, latency, and potential failure points that come with stitching together separate speech models and vendors.

Companies including Decagon, Sierra, Vapi, and Granola already use Deepgram’s voice infrastructure to power conversational AI experiences at scale. As Deepgram surpasses $100 million in annual recurring revenue, Flux TTS builds on that production foundation and Deepgram’s existing TTS capabilities with more expressive, conversation-native speech generation designed specifically for live interactions.

Built for Conversations that Don’t Follow a Script

Traditional TTS models were not built to sustain an ongoing conversation. Each request is treated as static output: text comes in, speech goes out, and the model resets. Real conversations are stateful: people pause, change direction, refer back to earlier turns, share complex information, and interrupt one another. When a voice agent loses that continuity–forgets a newly requested time midway through rescheduling an appointment, loses track of which troubleshooting steps a customer has already tried, or forces the customer to repeat themselves–it leads to failed tasks, human escalation, and an inability to deploy the agent in higher-stakes workflows.

Enterprises also face a broader systems challenge. Building a voice agent often requires connecting separate models for listening and speaking, and then managing the latency, orchestration, and potential failure points between them. Flux TTS is designed to reduce that burden.

“The market has spoken–literally–and voice is becoming the preferred interface for AI. But the next phase of this market will not be won by the model that sounds best in a demo. It will be won by systems enterprises can trust to complete business-critical work when conversations become unpredictable. Flux TTS was built for that standard: it begins responding in as low as 80 milliseconds while maintaining context and adapting to interruptions, moving text-to-speech from an audio feature to business-critical infrastructure,” said Scott Stephenson, CEO and Co-Founder of Deepgram.

Designed for Production

Flux TTS brings three core system behaviors to the speaking layer:

• Stays consistent from one turn to the next: Rather than treating each response as a standalone line of speech, Flux TTS carries the conversation forward, bringing context along for the entire conversation. Prior turns inform how the next response is spoken, helping the voice maintain consistent tone, pacing, and emotional register throughout the exchange. This helps the agent stay aligned with the exchange without additional prompt engineering, SSML, or style tags.
• Keeps pace with a live conversation: Flux TTS begins responding in as low as 80ms, even under production load. Persistent state, native interruption handling, and an explicit turn lifecycle allow the agent to adapt when someone interrupts or changes direction.
• Holds up under enterprise conditions: Flux TTS accurately communicates consequential information such as drug names, account numbers, and alphanumerics. Deployment options in a customer’s cloud or on premises also support scale, security, compliance, and data-residency requirements.

How Flux TTS Delivers Value for Customers

IBM and Coval are two examples of how Flux TTS is being used across the Voice AI ecosystem, from powering enterprise-grade agent experiences to helping teams evaluate how voice agents perform in real-world conversations. These examples underscore what production Voice AI increasingly demands: infrastructure that can maintain consistency, handle complexity, and perform reliably as agents take on more consequential customer interactions.

“Enterprises are moving from scripted automation to AI agents that can handle more complex customer interactions. Deepgram’s Flux TTS gives watsonx Orchestrate customers access to a text-to-speech model designed to maintain context and consistency across an entire conversation,” said Suzanne Livingston, VP of watsonx Orchestrate at IBM. “Flux TTS strengthens voice as a core part of the enterprise agent stack, and ultimately helps enterprises build authentic voice experiences better suited for real-world deployment across industries.”

“We evaluate voice agents every day at Coval, and the failure modes on the TTS side are remarkably consistent: tone that flattens by minute three, expressiveness that requires heavy SSML tuning, and inconsistency across turns that breaks the sense of a real conversation,” said Brooke Hopkins, Founder & CEO of Coval. “Deepgram Flux TTS is the first model we’ve seen that tackles those issues directly, treating the conversation as the unit rather than the individual line. That gives voice agent teams a TTS model that can hold up under the same rigorous, scenario-based evaluation we apply to the rest of the stack, and greater confidence in how their agents will perform in production.”

This reliability is critical across interactions like account management, where an agent needs to accurately communicate alphanumeric codes; sales and scheduling, where calls might be interrupted by new requests; restaurant ordering, where changes happen midway through the interaction; and technical assistance, where losing context or accuracy can force a customer to start over, abandon the task, or escalate to a human.

Deepgram Flux TTS is now generally available. Through September 12, 2026, developers can build with Flux TTS free with up to 45 concurrent streaming connections globally (5 in EU/AU). Standard pricing applies beginning September 13, 2026.

About Deepgram

Deepgram is the real-time AI infrastructure company underpinning the Voice AI economy. Today, more than 200,000 developers and 1,400 organizations are Powered by Deepgram. Its Voice AI platform offers speech-to-text (STT), text-to-speech (TTS), and full speech-to-speech (STS) capabilities, all powered by an enterprise-grade runtime. Deepgram’s voice-native foundation models, accessed through cloud APIs or as self-hosted/on-premises APIs, deliver unmatched accuracy, low latency, and competitive pricing. Customers include technology ISVs building voice products or platforms, co-sell partners working with large enterprises, and enterprises solving internal use cases. Having processed over 50,000 years of audio and transcribed over 1 trillion words, there is no organization in the world that understands voice better than Deepgram. To learn more, please visit www.deepgram.com, read its developer docs, or follow @DeepgramAI on X and LinkedIn.

Original source

Clear the FIPS compliance gate on your own infrastructure with no accuracy tradeoff. The same speech models you'd evaluate anywhere, now deployable across federal, defense, and public safety.

For many federal and regulated buyers, the evaluation ends before it starts. The models are accurate enough. The latency is fine. Then a security reviewer asks one question: is the cryptography FIPS validated? If the answer is no, the deployment is over, no matter how good everything else is.

This shuts out emergency dispatch, defense, and other federal agencies handling sensitive citizen data. The organizations running some of the highest-stakes voice workloads in the world are also the ones held to the strictest rules about how software handles cryptography. For them, FIPS 140-3-validated cryptography is a procurement gate.

That changes with our latest self-hosted release. Deepgram now supports a FIPS 140-3-compliant configuration, so passing FIPS review no longer forces a choice between compliance and the strongest speech models.

Inside FIPS Compliance

The cryptography is built on modules validated under NIST's Cryptographic Module Validation Program, and it covers the full deployment path, connected or air-gapped. Your security team reviews a known, documented boundary backed by the Federal Information Processing Standards (FIPS).

Crucially, FIPS compliance maintains full feature parity. FIPS and standard environments share identical APIs, deployment workflows, and Nova-2/Nova-3 models, with no difference in performance. Because FIPS is an inherent deployment property, it requires no model downgrades, separate SKUs, or compliance surcharges.

Turning it on is a config change. Set FIPS mode in both your API and Engine config:

[fips]
mode = "enabled"

The deployment comes up with OpenSSL in FIPS mode:

engine [fips] mode=enabled   -> openssl_fips_enabled=true, has_fips_encryption=true

Two things are worth confirming before you cut over, because in FIPS mode the deployment enforces both strictly:

• Everything that connects to the API must support TLS 1.3. The FIPS API accepts TLS 1.3 only and rejects TLS 1.2 and non-FIPS ciphers. That's enforced by the FIPS build we ship (TLS 1.2 is disabled in our FIPS OpenSSL configuration), so any SDK, load balancer, or upstream service in your client path that can't negotiate TLS 1.3 will fail to connect. Verify the end-to-end route from client to API and confirm it before you switch over.
• Your models have to be the FIPS-encrypted builds. The FIPS Engine loads FIPS-encrypted models with file format .dgv2 and will not read the standard model files you already have on disk. Before you cut over, pull the FIPS versions of the models you run so the Engine has something to load. Both Nova-2 and Nova-3 are presently supported model architectures.

The FIPS-compliant deployment guide walks through the full setup end to end.

A first for self-hosted speech

Until now, running a high-accuracy speech model fully self-hosted on your own cloud or bare-metal environment, with FIPS-validated cryptography, hasn't been available off the shelf, leaving teams to self-integrate open models and own the compliance work themselves.

So the buyer who needs FIPS-compliant Voice AI in their own air-gapped or FedRAMP environment had nowhere to go.

We closed this gap because self-hosting has been a first-class Deepgram product for years. The deployment is a set of components we build and operate ourselves: the API, the inference engine, and the licensing path between them, all running in containers on infrastructure you control. Because we own that whole surface, we could route its cryptography through a FIPS-validated module and make FIPS part of the product itself. A vendor that only ships model weights, or only runs in its own cloud, has no comparable surface to validate.

Voice AI, wherever it has to run

This release is one step toward a larger goal: Deepgram's voice AI running wherever it has to, with no compromise to get there. The same models, deployable in the managed cloud, in your own cloud, or entirely on your own infrastructure, each with the security and compliance posture that supports your user demands.

FIPS-compliant self-hosted Deepgram Voice AI is a concrete move in that direction. It runs with or without outbound connectivity: connected deployments reach Deepgram's License Proxy for licensing, and fully air-gapped deployments use the FIPS Billing image for offline license validation. What's next: Flux STT joining Nova-2 and Nova-3 in FIPS scope.

Get started

FIPS-compliant self-hosted is part of the enterprise self-hosted tier.

• Read the self-hosted deployment docs →
• Read the FIPS-compliant deployment guide →
• Talk to our team about federal and regulated deployments →

If voice is part of a regulated system you're building, the cryptography question no longer ends the conversation. You can deploy the models you actually want, on your own infrastructure, and clear the gate.

Original source