Inworld Release Notes

We rebuilt how open models are served for consumer apps, from the attention kernels to the cache router. Today it is open to every developer.

Key results

• up to 7.8× faster first token than the median provider on OpenRouter
• up to 48% lower input price than the median provider, and the lowest of any that doesn't train on your data
• 1.8× more requests served on the same hardware, while median time to first token improves
• up to 90% of each prompt served from cache, at up to 99% off
• 99.999% of requests completed without an error

AI's first wave was built for companies. Developers have felt it, and so have finance teams, but most people haven't yet seen AI change their lives. The bigger wave is built for people: a tutor on the commute, a coach mid-run, a language partner who waits for you to finish your sentence, someone to talk to after a hard day. Today only 2% of US households, and 3% of US consumers, spend anything on AI. Consumer AI is where the internet was in 1995.

Building for people is a different engineering problem. In consumer AI, latency is revenue: when a reply is slow, people go back to tapping buttons. Every turn resends the whole history, so prompts run 13 to 36 times longer than replies. One interaction fans out into many model calls, from moderation and memory to ranking and the reply itself. People pay $5 to $20 a month, or nothing, so each of those calls has to cost almost nothing. And when a consumer app goes down for ten minutes, its users leave for another one, while an enterprise workflow just waits.

The big labs and most inference providers have doubled down on enterprise and coding. Consumer AI needs the opposite on every count. So we built Inworld Inference, the first inference service designed for consumer AI, and today it is open to every developer.

Rather than list thousands of models, we go deep on the open models consumer apps choose. Each one, from Gemma 4 to GLM, DeepSeek and MiMo, gets its own serving build: new attention kernels, realtime decoding where a small draft writes ahead and the full model checks it in one pass, lower-precision weights wherever the evals hold, and a router that sends every turn back to the server that already holds its context.

The same research gives lower latency, lower cost and higher reliability. On Gemma 4 26B, first tokens arrive up to 7.8 times faster than from the median provider on OpenRouter, and the same hardware serves 1.8 times the requests it did in September, with a faster median first token. Input prices are the lowest of any provider that doesn't train on your data, on three of the five models we feature. Up to 90% of each prompt is served from cache, and cached input costs up to 99% less. On GLM 5.3 Flash, 99.999% of more than 20 million requests in 30 days completed without an error. Every build ships only if it matches the eval scores of the build it replaces.

Consumer AI is all we do, so being good at this is existential for us. Almost every token we serve comes from an app people use in their own lives, tens of trillions of tokens a month, 25 times what we served in March. Use the hosted models with no rate limits and a service tier per request, route across them with automatic failover, or bring your own model for a dedicated deployment.

01 Consumer vs. enterprise inference

For an enterprise task that replaces expensive manual work, a single result can justify a lot of compute, and the result can wait. A consumer app has to deliver interaction after interaction inside one subscription or one free session, while a person waits for each reply. So the number to optimize is the cost of one completed interaction, at the quality and response time the product needs.

Our goal is to get this down to a few cents per user per day, and that's something that you can only do with true experts and an ML team that really understands the trade-offs and techniques for accomplishing that.

The tightest constraint is time. In conversation, people leave 0 to 300 ms between turns. The model's first token is only part of that window, because the network and, in a voice app, speech synthesis use the rest. So the first token has to arrive well inside 300 ms, at production load. On Gemma 4 26B, the median first token now lands in 117 ms, which leaves most of the window for the voice.

The second difference is the size of each request. A consumer turn sends the whole conversation as the prompt and gets one short reply back. Across the apps we served in the 28 days to 23 September, median prompts were 13 to 36 times the length of replies. Enterprise work leans the other way, toward long outputs such as code and documents.

That workload already runs on Inworld at scale. On the most-used Gemma 4 models, we process up to 1.8 times the daily tokens that OpenRouter publicly reports across every provider on its platform combined.

We also serve those models faster. OpenRouter publishes first-token latency for every provider of a model. Against those measurements:

• Gemma 4 26B: the lowest median (0.17 s) and p95 (0.31 s) first token of the providers compared. That is 3.6 times faster at the median and 7.8 times faster at p95 than the median provider, at an input price nearly 40% below the median provider's.
• MiMo 2.6 Pro: the lowest input price of the providers on OpenRouter, with the lowest median (0.39 s) and p95 (0.91 s) first token. That is 2.6 times faster at the median and 4.6 times faster at p95 than the next-fastest provider.

02 Consumer inference tasks, and how we serve each

A consumer app doesn't make one kind of model call. A single turn can check the message for safety, classify what the person wants, read their memory, write the reply, call a tool and update the memory afterwards. Each of those calls has a different size, a different deadline and a different budget, and a provider built around one of them leaves the rest slow or expensive.

The same work sits underneath every row: kernels and decoding built for each model, a router that keeps each conversation's cache warm, precision checked against the evals, and a service tier per request so each call gets the speed or price it needs.

03 The serving challenge: long context in, short reply out

Consumer apps have a distinctive traffic pattern. They send very long context to get a short reply, so processing and reusing the input matters as much as generation speed. And the calls one user action triggers each have their own size and deadline, as the timeline below shows.

A dialogue turn with 3,500 tokens of history and a 200-token reply puts most of the work on the prompt. Long conversations pull the average far above the median: on GLM 5.3 Flash, the average request is 13,433 tokens in for 309 out.

A story with a longer answer shifts the weight to decoding. The two phases hit different limits. Prefill, the work of processing the input, is bound by compute and gains from cache reuse and batching. Decode, the work of generating the output, is bound by memory bandwidth and gets faster when each step is cheaper or produces more than one token.

So we work on both sides of the turn. On the prompt side, we avoid recomputing context the model has already seen, and chunked prefill processes a long prompt in pieces so it never holds up the replies streaming to other users. On the output side, we make every decode step cheaper and let each step produce several tokens. And because a background summary can wait while a reply cannot, every request carries a service tier. The next two sections cover the prompt side first, then the output side.

04 KV-aware routing for long conversations at scale

Each turn resends the conversation so far. Prefix caching keeps the attention keys and values (the KV cache) for context the model has already processed, so the engine computes only the new part of the prompt. Every reply is still generated fresh.

At fleet scale, the hard part is placement. A conversation's cache lives on the replica that served its last turn, while another replica may have more free capacity. Our router is KV-aware. It hashes each block of the prompt, tracks which replica holds which blocks, and scores every replica on two things: how much of this request's prefix it already holds, and how busy it is. Sending every turn to the warmest cache would overload it, so the score trades reuse against load, tuned on representative traffic. KV that doesn't fit in accelerator memory spills to host memory instead of being evicted and recomputed.

Built into each engine, model by model

Cache-hit rate is one of the numbers we watch most closely, because it sets both how soon a reply starts and what the turn costs. Getting it high on real traffic takes more than turning prefix caching on. Different models and engines represent cached context differently: draft modes for realtime decoding, hybrid attention layouts, recurrent state and data-parallel ranks each change how prompt tokens become cache blocks, and the router finds a match only when its view agrees with the engine's.

So we integrate the router with every model's serving path and measure the hit rate on production traffic. That work roughly doubled cache hits on GLM 5.3 Flash, from 25% to 47% on a replay of 5,000 production requests, and tripled them on DeepSeek V4.1 Flash. Our stack holds high hit rates on mixed traffic from many apps at once, and on one customer's dedicated deployment, cache hits reach 80% in production.

For your app. MiMo 2.6 Pro lists cached input at $0.0036 per million tokens and uncached input at $0.425, so every cached token costs 99% less. When 90% of a prompt comes from cache, the input cost of that turn falls by about 89%, and only a tenth of the prompt has to be processed before the reply can start.

Under the hood. Each cached block skips its share of prefill. With more of every prompt coming from cache, each replica can carry more conversations at once, and time to first token grows far more slowly as a conversation gets longer.

05 Faster replies through model-specific decoding and precision

People experience latency twice: while waiting for a response to begin, and while waiting for it to finish. Time to first token (TTFT) measures the first wait. Inter-token latency (ITL) sets the pace of the stream. A serving configuration has to handle both at the app's production concurrency.

Draft-and-verify, tuned for each model

Realtime decoding is our draft-and-verify serving approach, based on speculative decoding. A lightweight draft model proposes several tokens, and the target model verifies them together in one forward pass, accepting consecutive tokens until the first rejection. With exact acceptance and correction, generation can advance by multiple tokens per target-model step while preserving the target model's output distribution, with the same quality.

We optimize the drafts and the verify step for each model. Gemma 4 uses the draft models Google publishes for it, and GLM and DeepSeek use their own multi-token prediction layers. On a dedicated deployment, with the customer's approval, we train the draft on a sample of that app's traffic. A draft that predicts that app's outputs more accurately gets more tokens accepted per verification step, and we measure the gain at production load, because draft overhead, acceptance rate and concurrency all affect it.

Verify attention on split-KV: up to 7.5× faster

On Gemma 4 26B, a mixture-of-experts model, the verify step turned out to be the bottleneck. Verifying a 7-token draft sends 8 query tokens per sequence through attention, and the stock kernel only uses its faster split-KV path for single-token decoding. Verify fell back to a 2D kernel that split the work into about 60 blocks and left most of the chip's compute units idle.

We changed the kernel so verify batches of up to 16 query tokens take the split-KV path. On the model's global (full-context) attention layers, verify attention ran 3.4 to 7.5 times faster, and attention fell from 27.5% to 16.5% of each decode step. The full build, which also runs the whole verify step as one captured graph and sets the draft length to 7 tokens, took ITL p50 from 6.4 to 5.2 ms and TTFT p50 from 154 to 117 ms on replayed production traffic, with the same math and reasoning scores.

For your app. ITL sets how fast a reply streams: at 5.2 ms per token, a 200-token reply finishes in about a second after it starts. Faster decode steps also let each accelerator serve more requests, which is what keeps prices low.

Under the hood. Decode is bound by memory bandwidth: every step reads all of the weights to produce one token per sequence. A draft lets one read of the weights produce several tokens, so the speed of the verify step sets the gain. When attention dropped from 27.5% to 16.5% of the step, every request in the batch got faster.

Same eval scores, fewer bytes per token

A cheaper build ships only if it scores the same. Every build has to match the production build it replaces on our evals, including MATH, MMLU-Pro and GSM8K. Within that gate, each layer runs at the lowest precision its evals allow. On Gemma 4 31B, the MLP, which holds most of the weights, runs in 4-bit floating point (NVFP4), while attention and the KV cache stay at FP8.

The gate catches regressions before they ship. It found a scaling bug in FP8 attention, and the fix brought MATH back from 87.4 to 89.6. It also blocked a build that was 80% cheaper to serve, because that build lost 5 points on MATH.

For your app. Fewer bytes per weight keeps inference affordable as your app grows, at the same latency and quality. That is part of how Gemma 4 26B lists at $0.07 per million input tokens and $0.34 output.

06 Continuous optimization, model by model

Our work doesn't stop when a model goes live. We continually profile production traffic to improve response times, serving efficiency and reliability, from new attention kernels that accelerate generation to resolving memory leaks that require periodic restarts. Each change is evaluated against our quality benchmarks before release. The table below shows a selection of these improvements and their measured results.

The gains hold in production. On Gemma 4 26B, each serving instance now handles about 1.8 times the requests it did in September, while 7-day median TTFT fell from 178 to 161 ms. On Gemma 4 31B, one week of optimization made median TTFT 2.8 times faster on replayed production prompts, from 943 to 336 ms at the same eval scores, and in production the daily median fell from 365 to 271 ms even as traffic rose 26%. In a replay, cache-aware routing on DeepSeek V4 Flash made the slowest 1% of first tokens about 10 times faster, from 9.4 s to 0.87 s.

07 Flexible service tiers and strong reliability

A reply someone is waiting for needs a fast response; a background summary can wait and prioritize lower cost. Our four serving tiers let you choose the right balance for each request, using the same model weights. Set service_tier per request, or choose a default for your API key or workspace.

Reliable serving is a priority across every tier. Over 30 days, GLM 5.3 Flash served more than 20 million requests with 98 failures, a 99.9995% success rate, counting server errors, timeouts and capacity rejections. Across all hosted models, more than 99.99% of requests completed without a server error; that fleet-wide metric counts server errors only.

At about 4.8 failures per million, someone making 100 requests a day would see about one failure every six years, counted by the final status returned to your app. When a serving pool is saturated or unavailable, the router can redirect requests to another pool, helping keep your app responsive without requiring you to manage recovery between serving pools.

08 Routing: failover, conditions and A/B splits

Configure a primary model and compatible fallbacks once, so your app can recover without building its own model-switching logic. List fallback models in models and set fallback.ttft_timeout to specify how long to wait for the primary model's first token. If it misses that deadline, the router tries the next model on the list. A saved router can also route on the request's metadata or messages with conditions written in CEL (Common Expression Language), split traffic between variants by weight for an A/B test, and keep each user on one variant with user. When the models in a route are ones we host, each model tried keeps the caching and realtime decoding described above.

Every response says what happened. metadata.attempts lists each model tried, its time to first token and whether it succeeded, and metadata.reasoning says why the route went where it did, so you can debug a routing decision from the response alone.

Because the router treats a slow first token as a failure, a stalled model costs the turn the timeout plus the fallback's first token (300 + 157 ms in the trace above) instead of however long the first model would have taken, and your app needs no retry logic.

09 Your model, optimized for your app

Start with hosted open models and choose a serving tier for each task. When your app needs its own model, the goal is a better agent for your users at lower cost and lower latency, and we build it with you on a dedicated deployment.

Bring your own weights, or work with our engineers on post-training with an approved dataset and app-specific evaluations. We measure quality against the model you run today, then tune for efficiency until the new model meets your quality targets for less. Post-training changes what the model does; a draft trained on your traffic changes only how fast we serve it, so we evaluate the two separately.

The deployment can also include custom kernels, cache routing tuned to your traffic and capacity matched to it. On Gemma 4 31B, our own attention kernel serves about 4 times the tokens a second of the stock kernel at full load, and one customer's 31B fine-tune gained 35% throughput when we converted it to 4-bit weights. After launch, we keep profiling the deployment on your traffic, and a change ships only when it holds its gains at production load.

10 Customers: Status and Latitude

For Status by Wishroll, we worked with the team on targeted prompts, app-specific evaluators and production load testing, and the new setup cut their AI costs by more than 95% while keeping or improving content quality. The saving came from that work as well as from a lower token price.

The real upside with Inworld is that it's a continued relationship that allows us to keep optimizing performance and also keep lowering the costs because now we have this team of ML experts working with us.

Latitude's Voyage makes many calls per turn. Roughly 20 AI tasks run each turn alongside story generation, which added up to tens of millions of inference calls in Latitude's first six weeks on Inworld. We serve the World Engine tasks, the calls that apply the story's rules each turn, and the story models, including Latitude's own fine-tunes. The team moved its first production inference task in a day, half of its engine traffic within a week, and all of its story-model traffic in about a month. A user waits on the combined workflow, so every supporting call has to finish inside the turn.

11 Get started with the OpenAI-compatible API

For Chat Completions, set the Inworld base URL, your API key and a model ID. Streaming works the same way it does with OpenAI.

Gemma 4 26B lists at $0.07 per million input tokens, $0.035 cached and $0.34 output, nearly 40% below the median provider that doesn't train on your data. These are Default rates; lower rates are available as you scale. Every hosted model, with its price and cached-input rate, is on the models page. You can try any model in the browser first, with no API key, in the playground.

Original source

Why we're coming together

Inworld has acquired Ultravox, the platform developers use to build real-time voice agents. Ultravox team members who have joined Inworld will continue developing the platform.

We've worked closely with Ultravox through our text-to-speech partnership, sharing a focus on delivering Realtime AI at scale. Through that work, we saw how our technologies complement each other, and how bringing them together could help developers build better voice experiences.

Ultravox brings experience in speech understanding, reasoning, and task completion, alongside tools for managing turn-taking and interruptions in live conversations. Inworld brings top-of-benchmarks speech models (Realtime TTS-2 and Realtime STT-1), model serving, and real-time inference infrastructure for consumer-facing AI applications. Together, we can improve the full conversation: understanding what a user says, helping them accomplish a task, and responding in a voice that fits the moment.

For developers, this means better speech-to-speech experiences ahead through Inworld, alongside continued improvements to the voices and capabilities available on Ultravox. The first improvement is available today.

“I'm incredibly proud of what the Ultravox team built: real-time voice AI that customers depend on every day. When we partnered with Inworld on text-to-speech, we saw a team that moves fast and shares our commitment to delivering real-time AI at scale. I believe Inworld is the right home for Ultravox and the right team to take it further.”
ZACH KOCH · CO-FOUNDER AND FORMER CEO, ULTRAVOX

What you get today

Built-in Inworld voices on Ultravox now run on Realtime TTS-2. If your agent already uses one of these voices, the upgrade is in place. Your existing voice IDs continue to work, with no code changes required. TTS-2 is a top-ranked voice model on Artificial Analysis.

More control over how your agent speaks. TTS-2 introduces natural-language steering, giving developers control over an agent's pace, tone, and emotional delivery. You can ask a voice to slow down when explaining a complicated step, sound warmer when reassuring a customer, or bring more energy to a character, all through inline instructions.

That control helps developers shape how an agent speaks around the conversation it's having. The speaker identity stays the same, though pacing and intonation may differ with the updated model. Steering is optional, so you can introduce it where it helps your application.

“At Equal AI, we're building a personal AI for aspiring Indians that knows you, speaks your language, and helps manage your calls every day. We were already working with Inworld's voice technology before the acquisition, so we're excited to see Inworld's voices and the Ultravox platform come together. Bringing them under one roof is a natural next step for the voice experiences we're building for our customers.”
AKHILESH DAMARAJU · CEO, EQUAL AI

What this means for existing Ultravox customers

Your agents keep running as usual. There is no migration or code change required as part of this announcement, and the TTS-2 upgrade comes at no additional cost. If you use a voice from another provider, nothing changes today.

You can continue using Ultravox through your existing account, with continued support from Inworld and members of the team who built the platform. For questions about your account or agents, reach us at [email protected].

What we're working on next

This acquisition advances our work on speech-to-speech experiences that bring speech understanding, reasoning, and expressive voice generation closer together. We'll share more about those capabilities and further improvements for Ultravox customers in the coming weeks.

Original source

Realtime TTS now serves POST /v1/audio/speech in OpenAI's format, so apps built on OpenAI's text-to-speech can switch to Inworld by pointing the SDK at Inworld and choosing an Inworld model and voice — see OpenAI Compatibility.

Works with the OpenAI SDKs

• The official Python and Node.js SDKs work unchanged, including streaming responses.
• Formats: mp3, opus, flac, wav, and pcm, plus server-sent events with stream_format: "sse".
• One base URL: https://api.inworld.ai/v1 also serves the Inworld Router, so one client configuration covers LLM and TTS.

Original source

An open customizable framework for reproducible, inspectable TTS evaluation.

We are open-sourcing the Inworld TTS Open Evaluation Toolkit. It samples TTS systems, evaluates the audio offline, and builds comparable reports. Each run stores its configuration beside the per-sample measurements. Users can trace a score to the source text and audio, inspect the ASR transcript, and check the applied thresholds. The first release includes common metrics and a specially crafted compact English dialogue stress set.

Motivation

TTS evaluations can be hard. Two teams can run the same TTS model on the same dataset and get different WER numbers: the choice of an ASR model and text normalization matter. Voice choice and synthesis settings do too. So does the decision to include or exclude a failed sample. We saw it a lot in practice. And once we move from basic metrics to subjective ones, like naturalness or expressiveness, things become even worse.

Modern systems mostly handle simple prompts well. Implementation details and failures on harder cases therefore have more influence on the result. A mean score without its protocol is rarely enough to reproduce the claim. Our goal is to provide an extensible and customizable tool for trustful and reproducible evaluations.

What we are releasing

The Inworld TTS Open Evaluation Toolkit keeps a run in one inspectable directory. The workflow is simple:

1. Sample audio with fixed models and voices, recording the synthesis settings. Built-in adapters cover Inworld, ElevenLabs and Hume; any other system can provide audio through a manifest.
2. Evaluate the clips offline with local ASR and explicit normalization. Compute audio metrics, then apply configurable thresholds.
3. Report both the aggregate and the samples behind it.

The HTML report has no CDN or hosted runtime. Cross-run reports provide metrics by semantic groups (accuracy-related, subjective ones, failure modes) with confidence intervals and model-health summaries.

If a sample violates a threshold, the detailed version of the report places the expected text beside the ASR transcript. It also includes the audio when available.

The measurement cache is content-addressed. If the decoded audio and source text still match the metric configuration, the toolkit reuses its ASR and audio measurements. Thresholds and reports are still rebuilt, which makes tuning a decision rule much cheaper than transcribing the same batch again.

Metrics

The initial metric set covers several common things plus different failure surfaces:

• word- and character-level error rates, including insertion/deletion/substitution breakdowns;
• clipping and silence ratios, plus loudness and tail-click signals;
• duration and speaking-rate measurements;
• optional NISQAv2 dimensions and ECAPA speaker similarity;
• vowel-prolongation detection.

They should not be collapsed into one universal ranking. WER is also affected by mistakes made by the ASR evaluator. Low WER says little about whether a voice sounds natural.

We also provide lightweight arousal and expressiveness proxies. We discussed this split in Beyond Quality: Emotionality and Expressiveness. For a more direct look at those acoustic properties, the open-source Voice Lens and its browser demo calculate arousal and expressiveness locally from pitch/loudness/rhythm features. Uploaded audio stays in the browser.

Dialogue stress set

The repository includes a small English dialogue dataset built to surface failures quickly. Its 100 prompts include fragments and interruptions, along with emphasis markup. Repeated words, nested quotations and long dramatic lines make the small run more demanding than a list of simple benchmark sentences.

It is intentionally a stress set, not a representative sample of all speech. A deployment decision still needs production prompts in every relevant language. It also needs the voices the product will actually use. The bundled set is useful for finding obvious trouble before paying for the larger run.

Open by design

Even if reproducibility does not guarantee a correct evaluation, it makes the assumptions inspectable. We want teams to be able to rerun this framework and control its defaults. They can extend it with new metrics or datasets when the evaluation calls for them. The repository also includes agent instructions covering installation and the first end-to-end report from the command line.

Code and examples are in the repository README, together with the setup notes.

Original source

Launched Realtime TTS-2 Flash (inworld-tts-2-flash), the fastest member of the TTS-2 family — see Models:

• Our lowest latency: 20 ms time to first audio (server-side P90 TTFB, excluding network latency) — 5× faster than inworld-tts-2 at 100 ms, making it the best choice for latency-critical real-time agents.
• Our lowest cost: The most cost-efficient model per character, ideal for high-volume workloads.
• Full TTS-2 language coverage: The same 200+ languages and locales as inworld-tts-2, plus instant voice cloning and timestamp alignment.

Steering instructions and Professional Voice Cloning are supported on inworld-tts-2 only — use it when you need directed, contextually aware delivery. Non-verbal tags like [laugh] work on both models.

Original source

Steering on inworld-tts-2 follows one rule: a [tag] applies from where you write it until you change it. See the Steering guide.

• [reset]: New reserved tag that ends a styled passage and returns the voice to its own character for the rest of the text. [shouting] We need to leave now! [reset] Do you understand me? shouts only the first sentence.
• Instructions survive pauses: A <break/> no longer clears the active instruction. A pause is a pause and never changes delivery.

Behavior change

An inline [tag] previously affected only the text immediately after it and delivery could revert on its own partway through longer text. A tag now stays in force until you change it. If you relied on an instruction wearing off — for example [shout] Hi. Normal text. expecting the second sentence unstyled — add [reset] where normal delivery should resume. Requests that use no inline tags are unaffected, as are inworld-tts-1.5-max and inworld-tts-1.5-mini.

Original source

Realtime TTS-2 is live, and we cut prices in half or more for most developers, across the whole stack.

@mastra/voice-inworld-realtime collapses STT, LLM, and expressive, realtime TTS into one WebSocket, with semantic VAD, barge-in, and tool calling out of the box.

Most voice agents are cascaded pipelines: mic → STT → LLM → TTS → speaker. Each box is a different vendor, each hop adds latency, and the audio channel goes silent every time a tool call fires. The user notices.

Inworld's Realtime API folds the cascade into a single WebSocket session. The new @mastra/voice-inworld-realtime package binds that session to a Mastra Agent: semantic VAD ends the user's turn when they actually stop, barge-in cuts the assistant mid-sentence, and the LLM calls your Mastra tools without breaking the audio stream.

This reference CLI does all of this in less than 100 lines of TypeScript, and our example voice design agent shows how this can be integrated into a real application.

Watch the demo here.

One WebSocket, one session

Inworld's Realtime API takes the STT/LLM/TTS cascade and ships it as a single session over WebSocket. You pick an LLM (OpenAI, Anthropic, Google, and others; the routing string is provider-agnostic), pick an Inworld voice, and the pipeline runs end-to-end on Inworld's side.

Swap providers without touching your integration. Choose your STT engine independently of the LLM. And because the LLM emits Inworld's [steering] tags inline, like [whisper softly like you're sharing a secret], the TTS renders prosody and non-verbal cues in the same response stream, with no second model call and no SSML preprocessor.

For a Mastra developer, the interesting question is: how do you bind that to an Agent?

The voice package

@mastra/voice-inworld-realtime implements Mastra's MastraVoice interface against the Realtime API. It registers as an Agent's voice, surfaces the realtime stream as a typed event emitter, and routes the LLM's tool calls back to your Mastra createTool definitions.

The minimum config:

import { InworldRealtimeVoice } from '@mastra/voice-inworld-realtime';
const voice = new InworldRealtimeVoice({
  model: 'openai/gpt-5.4-nano',
  speaker: 'Jason',
});

model is provider/model, the same routing string the Realtime API uses on the wire.
speaker is any voice in Inworld's built-in library, or your cloned custom voices. Attach it to an Agent the same way you'd attach any other Mastra voice:

new Agent({
  id: 'voice-demo',
  name: 'Voice Demo',
  instructions: 'You are a concise voice assistant. Reply in one or two short sentences. Use the get-current-time tool when asked the time.',
  tools: { getCurrentTime },
  voice,
});

Tool calls happen mid-conversation

Tool definitions look like any other Mastra tool:

const getCurrentTime = createTool({
  id: 'get-current-time',
  description: 'Returns the current local time.',
  inputSchema: z.object({}),
  outputSchema: z.object({ time: z.string() }),
  execute: async () => ({
    time: new Date().toLocaleTimeString(),
  }),
});

When the user asks for the time, the LLM inside the Realtime session emits a tool call, the voice package routes it to execute(), and the result streams back into the session as a tool result. The audio channel stays open the whole time. No "let me look that up for you" stall while a cascaded pipeline tears down and rebuilds around the tool call.

The same contract works for anything you'd register on a non-voice agent: an HTTP call, a database query, an MCP tool. Mastra's tool surface doesn't change because the voice is realtime.

Semantic VAD, barge-in, and the interrupted event

Two things voice agents get wrong: they keep talking when the user starts talking, and they cut the user off because they think a pause means the turn is over.

Inworld's semantic VAD targets both. It detects intent boundaries, "did this user actually finish their thought?", rather than silence thresholds, and the eagerness is tunable on the session config. The default settings give natural conversational cadence; you can lean cautious or aggressive depending on your domain.

Barge-in is wired through the interrupted event:

voice.on('interrupted', ({ response_id }) => {
  players.get(response_id)?.kill('SIGTERM')
});

When the user starts speaking over the assistant, Inworld emits interrupted with the response ID of the assistant turn that should stop. The demo kills the matching play process; in a browser, you'd cancel the AudioBufferSourceNode for that response.

The combination is what makes the conversation feel live: the assistant yields when interrupted, the user's turn ends when they actually finish, and tool calls don't break either side of that contract.

Streaming transcript

For debugging or for a UI transcript pane, the writing event streams text from both sides:

voice.on('writing', ({ text, role }) => {
  // role is 'user' | 'assistant'
});

User text comes from STT, assistant text from the LLM, both incremental.

tool-call-start and error round out the event surface:

voice.on('tool-call-start', ({ toolName }) => console.log(`\n[tool] ${toolName}\n`));
voice.on('error', err => console.error('\n[error]', err));

tool-call-start fires before execute() runs; error surfaces anything the WebSocket or the pipeline raises. Same shape as the rest of the surface, just typed events, no string parsing.

Start building

A Mastra Agent that can have an engaging conversation. The Realtime API handles the speech-to-speech cascade and @mastra/voice-inworld-realtime translates between that pipeline and Mastra's Agent and tool primitives.

From here, the places to extend are the usual Mastra ones. Add memory so the agent carries context across sessions. Add scorers and evals to grade voice behavior over time. Add more tools. Anything that fits createTool works. Swap the LLM by changing one string. Swap the voice the same way.

Learn more in Inworld's Realtime API docs. For the agent layer, see the Mastra docs.

Original source

Cost is the wall in front of consumer AI,

and we are taking it down.

A consumer app pays for AI on every session, but most of its users never pay, and the ones who do spend only a few dollars a month. That math puts a wall in front of every consumer product, and it decides whether you scale or stall. We are taking that wall down for the people who serve everyone, by cutting prices in half or more for most developers across the whole stack. That includes text-to-speech, speech-to-text, the LLM, and compute, with prices dropping further as you scale.

Here is the problem and the fix in one picture: on a flat rate your bill climbs in lockstep with engagement, so the more people use your app the harder cost pushes back.

MONTHLY SPEND GROWS SLOWER WITH INWORLD

Total monthly cost as you scale

OTHER PROVIDERS ($30/1M)

INWORLD (TIERED)

YOUR SAVINGS

10M

20M

120M

500M

1B

MONTHLY TTS VOLUME (CHARACTERS)

$0

$8K

$15K

$23K

$30K

Monthly spend ($)

Save $20K/mo

Source: Illustrative, based on Inworld plan-tier pricing (Developer $15, Growth $12.50, Enterprise $10 per 1M characters) vs a flat $30/1M competitor rate.

Lower the unit cost at every tier and the whole spend curve bends down with it, so growth stops being the thing that breaks your economics.

Consumer AI is hard for three reasons.

Cost is at the root of all of them.

A consumer app's costs start climbing the moment it works. The three: it has to scale, the people it serves are the buyers, and most of its cost is the AI itself. Each is hard on its own, and all three get harder because cost rises with the very engagement the product is built to create.

1

Scale is a necessity

Apps can reach millions of users in weeks.

Reliability is non-negotiable; an outage loses users and their habits.

A few people run infrastructure built for millions.

Most models were tuned for enterprise, not consumer.

When an app takes off, spending does not rise in proportion, because one line of the bill grows faster than every other cost you carry, and faster than anything a small team can offset.

MONTHLY SPEND AS A CONSUMER APP SCALES. AI INFERENCE GROWS FASTEST.

Costs grow with every user

AI INFERENCE ($/MO)

PAID ACQUISITION ($/MO)

OPERATIONS ($/MO)

DAILY USERS

0

1

2

3

4

5

6

7

8

9

10

11

12

MONTHS SINCE LAUNCH

$0K

$55K

$110K

$165K

$220K

Monthly cost ($K)

0K

150K

300K

450K

600K

Daily users

Illustrative growth model for a scaling consumer AI app. Cost in $K / month; users in thousands of DAU.

AS YOU SCALE, LOWER COST KEEPS YOUR MARGIN INSTEAD OF ERODING IT · ILLUSTRATIVE

Margin vs scale

WITHOUT THE PRICE CUT

WITH LOWER COST

1K

10K

50K

100K

500K

1M

DAILY ACTIVE USERS

0%

20%

40%

60%

80%

100%

Gross margin (%)

Starting margin 52%

22%

80%

Source: Illustrative gross-margin model at fixed price points. Both lines start at the same point to isolate the effect of scale.

Running the AI is that runaway line, so cutting its cost is the difference between scale that compounds your margin and scale that quietly drains it, which raises the obvious question of who is even paying.

2

Consumers are the buyers

Most users pay nothing; those who do pay $5 to $20 a month.

They expect a natural, realtime response, not a slow workflow.

Cost keeps most teams in English-only markets.

Acquisition is paid, so thin revenue has to cover serving everyone.

A consumer app and an enterprise tool can look identical on screen, yet they run on opposite economics, and the split shows up the moment you ask who actually sends money.

ABOUT 3 IN 100 USERS EVER PAY

Who pays for consumer AI

PAYING USER

FREE USER

~3%

OF USERS EVER PAY

Source: Menlo Ventures, 2025 State of Consumer AI

REVENUE PER USER, PER MONTH

What a single user is worth

CONSUMER APP

ENTERPRISE SEAT

$0

$100

$200

$300

$ PER USER / MONTH

Enterprise seat

Consumer app

$200

($100 to $300)

$10

($5 to $20)

Source: a16z, Menlo Ventures, 2025 to 2026

A small minority of payers, each worth a sliver of one enterprise seat, has to fund the compute every free session burns, so the price of each unit is what quietly decides survival.

3

AI is most of the cost

AI is the largest cost line, and it grows with every session.

Cheaper tokens do not help when you use far more of them.

Cutting unit cost means locking into a SKU and losing flexibility.

Many models and configs make billing its own problem.

You already know this one because you pay it every month: hunting for slightly cheaper tokens does not save you when usage grows faster than the discount; the bill tracks how much people engage.

EVEN AFTER THE VOLUME DISCOUNTS YOU EARN AT SCALE

Your AI bill grows with every user

LIST RATE

WITH VOLUME DISCOUNTS

10K

100K

500K

2M

DAILY ACTIVE USERS

$0K

$40K

$80K

$120K

$160K

Total AI bill ($K / mo)

$3K

$19K

$56K

$98K

Illustrative app-spend model; total monthly AI inference cost by daily active users, list rate vs a typical scaling volume discount.

Volume discounts soften the slope without ever flattening it, since growing means using more, so the only lever that holds is a structurally lower cost. Here is what happens when teams get one.

Lower the cost,

and each of these eases.

None of these problems vanish with a cheaper bill, but every one of them gets easier. Scale becomes affordable, more markets come into reach, and a small team can run a real product without infrastructure eating the margin. Teams already building on Inworld show what changes, and the reason the lever works is that the cost lives almost entirely in three places.

~95%

LOWER ON LLM COST

Wishroll / Status

500K+ daily users

~85%

LOWER ON TTS COST

Bible Chat

800K+ daily users

~40%

LOWER ON TTS COST

Talkpal

10M+ learners

250K+

USERS SERVED AT CONSUMER COST

Luvu

realtime AI coaching

"We scaled from prototype to 1 million users in 19 days with over 20× cost reduction."

FAI NUR · CEO, STATUS

"Inworld reduced our TTS costs by about 10x, while remaining neutral or better on all the metrics we care about, which is pretty incredible."

CRESTON BROOKS · CO-FOUNDER & CTO, LUVU

"We chose Inworld because of its low latency, high-quality output, multilingual support and competitive pricing."

DIMITRI DEKANOZISHVILI · CO-FOUNDER, TALKPAL

The biggest of these cuts is not a discount: Wishroll split one giant prompt into small, task-tuned models through the Router, and engagement held. The full mechanics are in the Router section below. Beyond these, some of the largest consumer apps we serve, including one processing more than 600 billion tokens a day, are moving their LLM workloads onto our realtime inference. Customer figures are self-reported.

FOLLOW THE MONEY

The voice AI stack was built for the enterprise.

Consumer was left to carry the cost.

The biggest labs make most of their money from businesses and developers, and it's not too hard to see why. Enterprises pay for productivity: a company will spend a hundred to a few hundred dollars per seat to save an employee time. On the other hand, only about 3% of consumer AI users ever pay, and those who do pay five to twenty dollars a month for something they enjoy.

So the models were built for that first buyer. The voice infrastructure especially was priced for enterprise and tuned for narrow jobs like support calls, IVR, and transcription, not for the range of things consumers actually do. And for a consumer, almost the entire cost of a session is the AI itself, split across the LLM, text-to-speech, and speech-to-text. Pricing that works for a per-seat tool doesn't work for an app that pays for every minute someone uses it.

Follow the money inside one live conversation and nearly every cent lands in the same place: the models talking, listening, and thinking, second by second, for everyone you are lucky enough to keep engaged.

WHAT A LIVE SESSION'S SERVING COST IS MADE OF

Most of the cost is the AI itself

LLM

TEXT-TO-SPEECH

SPEECH-TO-TEXT

55%

30%

15%

100%

AI INFERENCE

Illustrative cost mix for a typical conversational, voice AI session. The whole serving cost is AI inference, split across three layers.

Whoever owns the LLM, the text-to-speech, and the speech-to-text owns almost the whole cost of a session, which means they, not you, decide whether an app built for the full breadth of human experience can afford to grow.

WHY WE ARE DOING THIS

AI should reach everyone,

not only the enterprise.

We are a research lab, and we measure this technology by the breadth of human experience it reaches, not a handful of business tasks. If only enterprise economics work, only enterprise problems get built. We would rather a wider range of developers serve a wider range of people: the companion at midnight, the tutor, the coach, the daily check-in.

We can lower the wall because our research delivers both quality and efficiency, and we own the layers where cost adds up. We pass those savings on instead of marking them up.

Enterprise voice covers one narrow band of what people actually do with their voice. Here is the whole spectrum a consumer developer has to serve.

ENTERPRISE VOICE IS TUNED FOR A NARROW SET OF JOBS, EVEN WHERE ITS VOLUME IS LARGE

AI should reach the breadth of human experience

THE FULL BREADTH (CONSUMER)

ENTERPRISE VOICE (ONE SLICE)

Support / IVR

Accessibility

Language learning

Education

Coaching

Health

Journaling

Companions

Social

Meditation

Kids

SPECTRUM OF CONSUMER USE CASES

Diversity of needs served (illustrative)

The full breadth of human experience

CONSUMER VOICE AI

Enterprise voice

SUPPORT, IVR, TRANSCRIPTION

Illustrative

That whole curve is who this price cut is for.

WHAT WE DID

Our API prices drop by half or more for most developers,

across the whole stack.

New pricing is live today for every developer. For most developers it drops by half or more across every layer of a voice app, and it falls further as you scale. Here is each layer, what it does for a consumer app, and where it lands.

$ PER 1M CHARACTERS · INWORLD SHOWN ON THE GROWTH PLAN ($25 TO $12.50, ~50% OFF)

Text-to-speech price vs the market

OTHER PROVIDERS

INWORLD

$0

$55

$110

$201.60

$ PER 1M CHARACTERS

Google Gemini 3.1

Flash TTS*

ElevenLabs

Cartesia Sonic 3.5†

OpenAI TTS

Realtime TTS-2

(Growth plan)

$180

$100

$35

$30

$12.50

Provider API rates, June 2026. Inworld on the Growth plan; $10 at enterprise scale. *Gemini 3.1 Flash TTS is billed by audio output tokens; ~$180/1M is the effective rate on a typical query. †Cartesia estimated from published tier pricing.

TEXT-TO-SPEECH

Top-ranked voice, a fraction of the cost

Realtime TTS-2 is top-ranked on independent speech benchmarks. It supports steering and conversational context, and a single voice works across languages, so you can open new markets without switching models. It answers in realtime, and it costs a fraction of the comparable premium voices.

DOWN TO ~$10 / 1M CHARS

$ PER HOUR, STREAMING

Speech-to-text price vs the market

OTHER PROVIDERS

INWORLD (WAS)

INWORLD (NOW)

$0

$0.30

$0.60

$1.15

$ PER HOUR (STREAMING)

Azure

Google STT

Deepgram Nova-3

AssemblyAI Universal-3

Pro

Inworld STT-1 (was)

Inworld STT-1 now (at

scale)

$1

$0.96

$0.46

$0.45

$0.35

$0.10

Source: provider pricing, June 2026; Inworld new pricing. On-demand $0.15/hr, down to $0.10 at scale.

SPEECH-TO-TEXT

Fast, accurate, among the lowest cost

Our speech-to-text ranks top-4 in quality and reads emotion, accent, and intent as the user speaks, so the app responds to a person. It handles long, continuous sessions that call-center transcribers can't, and it now costs less than every major streaming rate we benchmark.

DOWN TO ~$0.10 / HOUR

WHAT A TYPICAL 5% GATEWAY MARKUP COSTS PER YEAR. THE INWORLD ROUTER: $0.

A gateway charges you more

5% GATEWAY MARKUP (PER YEAR)

INWORLD ROUTER: $0

$100K / mo

$500K / mo

$1M / mo

MONTHLY LLM SPEND

$0

$200K

$400K

$600K

Markup paid per year

+$60K/yr

+$300K/yr

+$600K/yr

A typical gateway adds ~5% on credit purchases. The Inworld Router passes routed models through at no markup.

THE LLM, THROUGH THE ROUTER

Hundreds of models, no markup

The LLM is a major line. Through the Router you reach 220+ models from every major lab and route to the best one for each job with a config change, not a migration. Routed third-party models carry no markup, where a typical gateway adds about 5%, which compounds into real money at scale. And the biggest wins come from decomposition: we helped one social app split one giant prompt into small, task-tuned models, cutting AI cost about 95% while engagement held.

NO MARKUP · UP TO ~95% LOWER

THE SAME TOP OPEN MODELS, UP TO 50% BELOW THE PUBLIC THIRD-PARTY RATE

Up to half the cost for your LLMs

YOU PAY

YOU SAVE

0%

25%

50%

75%

100%

Public third-party rate

Inworld realtime

inference

100%

50%

save up to 50%

Inworld realtime inference serves the top open models up to 50% below what you would pay elsewhere, with better latency and reliability.

REALTIME INFERENCE, A NEW PRODUCT

Optimized open models, up to 50% below the public rate

Realtime inference is a new product. The same team that tuned our voice models now hosts the top open models consumer apps run in production and serves them up to 50% below the public rate, with better latency and reliability, accessed through the Router. Three of the ten highest-volume consumer apps we work with are already moving their LLM workloads onto it.

UP TO 50% OFF, BETTER LATENCY

VARIABLE TOKEN COST VS DEDICATED GPUS AS VOLUME GROWS (ILLUSTRATIVE)

When fixed compute wins

PER-TOKEN (VARIABLE)

DEDICATED GPU (FIXED, FROM $5/GPU-HR)

0x

3x

6x

10x

MONTHLY VOLUME (RELATIVE)

$0

$3K

$6K

$11K

Monthly cost ($)

Crossover: fixed GPUs win

Source: Illustrative; Inworld $5/GPU-hr, GCP H100 ~$11/GPU-hr.

COMPUTE

Fixed GPUs when you reach real scale

At the largest scale, per-token billing is not always the right shape. You can move to dedicated GPUs from $5 per GPU-hour, less than half the on-demand rate of a hyperscaler, and transition at the crossover where fixed compute beats variable.

FROM $5 / GPU-HOUR

AND IT COMPOUNDS

Your unit price gets cheaper the more

you scale.

Today's prices are the ceiling, not the floor. As your total spend grows, the unit price of every layer falls. We own the layers where cost adds up, and scale makes us more efficient, not less. Your unit price drops by tier, all the way to enterprise. Shown here for Realtime TTS-2; speech-to-text and the LLM follow the same shape.

REALTIME TTS-2, $ PER 1M CHARACTERS BY MONTHLY SPEND

Your unit price drops as you scale

REALTIME TTS-2 UNIT PRICE

ELEVENLABS API ~$100/1M, FLAT (OFF SCALE)

Up to $300

$300

$1.5K

$5K

$25K

$100K

MONTHLY SPEND

$0

$10

$20

$30

$ per 1M characters

$25

$15

$12.50

$10

$7.50

$5

Inworld pricing, June 2026. The discounts arrive at modest spend: a $300/mo subscription covers about 20M characters at $15/1M, $1,500/mo covers about 120M at $12.50, and enterprise commitments step down from $10 toward $5 at the largest spend levels (illustrative). For comparison, the ElevenLabs API standard rate is ~$100 / 1M characters (standard tier, off this scale).

You saw the shape of this curve at the top of the post. Here it is with your spend on it: the total grows far slower than on a flat rate. You choose how to buy, trading flexibility for a lower, more predictable rate, up to about 90% off with fixed compute for the largest workloads.

TOTAL MONTHLY COST AT INCREASING VOLUME

What you actually spend

INWORLD (TIERED)

ILLUSTRATIVE FLAT RATE ($30 / 1M)

10M

20M

120M

500M

1B

MONTHLY VOLUME (CHARACTERS)

$0

$7.5K

$15K

$22.5K

$30K

Monthly spend ($)

SAVE $20K/MO

Illustrative. Inworld spend-tier pricing ($15 at $300/mo, $12.50 at $1,500/mo, $10 at enterprise scale, per 1M characters) vs a flat $30 / 1M rate.

MAXIMUM DISCOUNT BY HOW YOU BUY

More commitment, bigger discount

MAX DISCOUNT VS ON-DEMAND

Pay-as-you-go

Subscription

Commit

Fixed compute

PURCHASING MODEL

0%

25%

50%

75%

100%

Max discount (%)

0%

50%

80%

90%

Inworld pricing tiers: subscriptions reach 50% off list (TTS-2 $25 to $12.50 on Growth), enterprise commits reach 80% (TTS-1.5 Mini $25 to $5), fixed compute up to ~90% (illustrative, at full utilization vs per-token public rates).

Discounts on top of a high base never fixed the math. Discounts on top of a structurally lower base do, and fixed compute is the point where the line finally flattens. Pick the shape that matches your stage; the rate only moves one way as you grow.

Lower cost is the front door.

The rest is built for consumer too.

Human feel

Voice profiling reads emotion and intent as the user speaks, and steerable, audio-native TTS-2 answers in kind, so a session adapts to the person.

Modularity

Reach any model you need through one API, and swap it with a config change instead of a migration.

Unified billing and management

One bill across every layer, volume discounts that apply across the stack, and per-team and per-user controls.

Experiment freely

A/B test models and variants on live traffic to find what improves both the experience and the margin.

OUR COMMITMENT

Building the stack

consumer AI deserves.

The price cut is the visible part. Underneath it is a stack we build and own end to end, where each layer rests on the one below. Read it from the surface down.

SURFACE

FOUNDATION

LAYER 01 · APIS

CLOSEST TO THE USER

Simple, reliable APIs

VOICE THAT FEELS HUMAN

REALTIME LATENCY

ONE CROSS-LINGUAL VOICE

THE CONTROL DEVELOPERS NEED

LAYER 02 · INFERENCE

WHERE THE PRICE CUT COMES FROM

Optimized inference and model management

DEEP OPTIMIZATION ON TOP OPEN MODELS

PERFORMANCE, COST, RELIABILITY

UNIVERSAL ACCESS VIA THE ROUTER

UNIFIED BILLING AND COST CONTROL

LAYER 03 · RESEARCH

THE BEDROCK

Research to serve the breadth of human experience

WORLD-CLASS RESEARCH TEAM

PROPRIETARY TRAINING DATA

YEARS OF R&D

BUILT FOR THE DIVERSITY OF HUMAN EXPERIENCE

Voice profiling reads the user, and steering makes responses expressive and empathetic. Realtime latency is the bar for engaging a consumer, and one cross-lingual voice reaches users across languages and markets, with the control developers need made cost-effective.

Deep optimization applied to the top open models consumer apps run in production means better performance, cost, and reliability, with universal access through the Router and unified billing across every model.

Where others narrow their focus, we want to enable the diversity of users and use cases that consumer represents, across voice, language models, and code. The breadth is the point, and it is the layer everything above is built to serve.

No single layer is the moat.

The integrated whole is.

We are taking on the cost so consumer developers can win.

New pricing is live today for every developer. We build the best realtime models for voice and speech, optimize the inference that serves the LLMs you run, and keep lowering the cost, so a wider diversity of developers can bring AI to everyone.

Original source

Realtime TTS-2 is live. Built for realtime conversation that feels human.

Realtime interactions are becoming the primary way people interact with agents. To showcase what's possible with Inworld's latest voice model, Realtime TTS-2, Inworld partnered with Stream to build a flagship reference implementation using their open-source Vision Agents framework. The 'Crashout Buddy' watches your face, hears your words, and shapes its delivery in real time based on how the end user actually feels in the moment.

This reference example can be adapted to many more use cases: from professional coaching with realtime guidance, companion apps that notice context and environment, patient intake with verbal and non-verbal cues, 1:1 personalized customer experiences, and beyond.

What this means for realtime voice agents

Realtime TTS-2 + Vision Agents only requires a few lines of code to unlock the ability for AI to make users feel heard and stay engaged:

• Emotional context and conversational awareness become first-class inputs.
• Whatever your agent knows about the user from sentiment in transcript, signals from a vision model, sensor data, etc. can be turned into a steering directive that shapes how the voice delivers the response.
• Multilingual deployment prevents a fragmented voice identity.
• One voice across 100+ languages means a single agent persona for a global user base. No need for model swaps mid-conversation when a user switches languages.
• Non-verbal sounds become part of the script.
• Laughs, sighs, breaths, and pauses live inline alongside the words. The agent sounds like it's actually listening before responding.
• Long conversations build on themselves.
• Because Realtime TTS-2 carries conversational context forward, sustained interactions stop feeling like a series of disconnected scripts.
• Sub-200ms latency unlocks interruption and barge-in patterns suitable for realtime agent loops.

“We've had early access to Inworld's Realtime TTS-2 for a few days and we're all blown away. The expressiveness, language steering and multi-lingual support are genuinely impressive. The subtle details like natural pausing make it hard to differentiate between AI and human.”

NEEVASH RAMDIAL · VISION AGENTS LEAD, STREAM

Voice steering, live

The same steering capability powering Crashout Buddy. Pick a delivery tag and hear how the voice changes with the same line.

Start building

Any use case can follow a simple pattern: take the user's video feed, run a lightweight perception model on it, use the results as context, and let an expressive voice model render the response with appropriate delivery. Vision Agents makes the orchestration simple. Inworld Realtime TTS-2 makes the voice interaction believable.

• Vision Agents — the framework
• Inworld Realtime TTS-2 — expressive voice
• GitHub repo — the code

The next generation of agents doesn't read scripts. It understands the full context of the user and makes them feel truly understood.

Original source

Realtime TTS-2 from Inworld AI

Realtime TTS-2 from Inworld AI is a new generation of voice model built for realtime conversation. It hears the full audio of the exchange, picks up the user's tone, pacing and emotional state, then takes voice direction in plain English the way developers prompt an LLM. It holds one voice identity across over 100 languages. Available today via the Inworld API and the Inworld Realtime API as a research preview.

WHAT LAUNCH PARTNERS AND CUSTOMERS ARE SAYING

Voice AI that actually feels human.

"Inworld's TTS-2 marks a real step forward in emotionally expressive voice synthesis. When combined with the conversational intelligence of LiveKit agents, it enables interactions that feel genuinely human — responsive, nuanced, and alive in ways that feel natural."
DAVID ZHAO · CO-FOUNDER & CTO, LIVEKIT

"I've never seen steering work like this before TTS-2. The output is extremely natural and faithful to the steering prompt, even when it's hyper-specific. The biggest battle you fight with TTS is feeling bland, stale, and robotic — this level of steering unlocks a whole new axis to keep the experience fresh."
CRESTON BROOKS · CO-FOUNDER & CTO, LUVU

"We've always believed language learning should have no borders. TTS 2.0 just made that a lot more real."
DIMITRI DEKANOZISHVILI · CO-FOUNDER, TALKPAL

"We've been chasing the uncanny valley of voice AI for years — Inworld is finally closing the gap between 'impressive' and 'actually believable' with TTS 2.0. When your character speaks and you forget it's AI, that's when the story becomes real."
LOUIS MUK · CEO, ISEKAI ZERO

"We've had early access to Inworld TTS-2 for a few days and we're all blown away. The expressiveness, language steering and multi-lingual support are genuinely impressive. The subtle details like natural pausing make it hard to differentiate between AI and human."
NASH RAMDIAL · DEVELOPER RELATIONS, STREAM

"Inworld just made voice AI feel genuinely human across 100+ languages. Partnering with them means we can help bring that experience to kids around the world, safely and compliantly."
KIERAN DONOVAN · CEO, K-ID

"AI Native games need characters you can deeply connect with. Voice models that offer full control and emotional complexity to make characters feel real is one of the biggest pieces missing. TTS 2 is a significant advance in helping make that future a reality."
NICK WALTON · CEO, LATITUDE

"Inworld was already at the top of the Artificial Analysis TTS Arena and Realtime TTS-2 pushes further on a dimension VoiceRun customers care about: directability. Style, pacing, emphasis, emotion, and delivery can be shaped in ways that matter for real enterprise deployments."
NICK LEONARD · CEO & CO-FOUNDER, VOICERUN

Voice AI was built for audiobooks. We rebuilt it for conversation.

Realtime TTS 1.5 already ranks #1 on the Artificial Analysis Speech Arena, ahead of Google and ElevenLabs. Quality is solved. So we asked the next question: what does voice AI sound like when it is built for the way humans actually talk to each other? In realtime, mutual, alive to the moment?

Voice AI was shaped by the static stuff: audiobooks, narration, voiceover. A sentence in, audio out, the model never hearing the person on the other end.

Realtime TTS-2 is built from the ground up for realtime conversation. It listens to the prior turns of the exchange, so your tone and pacing carry forward. It takes voice direction in plain English, so you steer the read the way a director would. It holds one voice identity across over 100 languages, so the speaker stays the same person mid-switch. And Advanced Voice Design lets you build a saved voice from prose. Four capabilities that work together, in one model, on the same realtime connection.

Capabilities include:

1. Voice Direction: Natural-language descriptions of how a line should be delivered, passed inline at the start of your text. Not fixed preset emotions or sliders. Write prompts like stage directions.
2. Conversational Awareness: The model takes actual audio of prior turns, not just transcripts, so tone, pacing, and emotional state carry forward automatically.
3. Crosslingual: One voice identity preserved across 100+ languages, including mid-utterance language switches.
4. Advanced Voice Design: Create a new voice from a written prompt, no reference audio needed.

Additional features:

• Non-verbal markers like laughs, sighs, breaths inline in text.
• Disfluencies like uh and um in natural places.
• Voice cloning from a short audio sample.
• Stability modes to dial expressiveness up or down (Expressive, Balanced, Stable).

Built for natural realtime conversation: Listening, Thinking, Expressing stages all on the same persistent connection, passing full audio context, user's state, and conversation history.

Available today through the Inworld API and the Inworld Realtime API. Customers on Realtime TTS 1.5 upgrade by changing the model identifier, no other code changes needed. SDKs available in Node and Python.

"We are obsessed with how Voice AI feels, not just how it sounds." - KYLAN GIBBS, CEO, INWORLD AI

Pricing and documentation available at docs.inworld.ai and inworld.ai/pricing.

Original source

Launched Realtime TTS-2 (inworld-tts-2), our most powerful and expressive TTS model:

• Natural Language Steering: Direct any voice with bracketed instructions like [say excitedly], [whisper in a hushed style], or free-form directions like [speak as if barely holding back rage]. Covers articulation, intonation, volume, pitch, range, speed, vocal style, and non-verbals ([laugh], [sigh], etc.). See the Steering guide.
• Stronger Multilingual Support: Production-quality synthesis across 15 languages, plus experimental support for 90+ additional languages. See Languages.
• Cross-Lingual Voice Synthesis: Reuse the same voice across multiple languages. For best results, specify the language field.
• Voice Localization: Localize your voice for the most consistent, native-sounding speech in a target language. See Voice Localization.
• Delivery Mode: New deliveryMode field (STABLE, BALANCED, EXPRESSIVE) controls the trade-off between consistency and emotional range.
• Updated Voice Design: Released an updated version of Voice Design with improved generations. See Voice Design.

Original source

How Do We Evaluate TTS?

When evaluating TTS systems, we usually focus on the obvious stuff: efficiency and accuracy. But once you nail those basics, things get interesting. What makes one voice more engaging than another? We decided to check it out.

This post will show how, while working on the new Inworld TTS models, we developed metrics for emotionality and expressiveness, and then used them to create voices that better match user needs.

First, let's compare the basic things: cost, speed, scalability, and ease of use. This approach assumes that the TTS works flawlessly.

When it doesn't (and, well, it often doesn't), there are well-established metrics to evaluate accuracy issues. We look at generation errors, pronunciation deviations, tone inconsistencies, unnatural sounds, and so on.

But what comes next?

Let's say we have a fast, cheap TTS that generates speech without errors. This is where things get interesting. Depending on our task, we might have some specific requirements. For a virtual AI companion, empathy and emotionality might be important, while for an AI support specialist, calmness and professionalism matter more.

How do we define such properties? Can we learn to work with them objectively? We started with emotionality, since it's the most researched aspect of expressive speech.

Emotionality

Evaluating the emotionality of text or speech is a fairly well-studied problem.

On one hand, theory tells us that the degree of emotionality in speech directly or indirectly affects various acoustic parameters: fundamental frequency (F0, pitch), loudness, speed, and so on. [1], [2].

On the other hand, there are plenty of ready-made solutions out there. Currently, HuggingFace hosts more than 20 models for emotion classification from speech audio.

We tried a bunch of them, and, unfortunately, most of them don't work quite the way we'd like. Some of them seem to be overfitted on specific domains, others are just unstable on our tests. So, we built our own "arousal" metric that evaluates overall speech emotionality regardless of the specific emotion.

Here are a few generation examples from the same speakers but with different levels of emotionality:

• HADES
• ALEX
  • Low 0:02 0:01
  • Medium 0:03 0:02
  • High 0:02 0:02

Emotionality vs Expressiveness

Now, if you think about it, emotionality isn't everything we want from expressive speech. Speech can be emotionally neutral yet very expressive.

Think of your favorite audiobook narrated by a professional voice actor. Such a narrator uses various techniques (speed, volume, tone, rhythm, pauses, semantic emphasis) to make the flow of speech less monotonous and to highlight key words and phrases.

Can we automatically evaluate expressiveness beyond emotionality? It turns out we can. We managed to develop an "expressivity" metric that, with reasonable accuracy, determines the level of expressiveness in speech. While arousal captures intensity, expressivity measures variation and dynamic range across prosodic features.

Here are a few examples where we can hear high expressiveness with roughly similar emotionality:

• PHILIP
• ELIZABETH
• AGNES
  • Low 0:12 0:02
  • 0:03
  • High 0:14 0:03
  • 0:03

From Metrics to Training

With these metrics in hand, we can train models in a controlled way using reinforcement learning approaches, either increasing or decreasing emotionality and expressiveness depending on our preferences. We can also use them for data collection and training controlled generation models with explicit tags.

In our recent TTS 1.5 release, we used this approach to achieve more natural results that better match user needs. For instance, our AI companion voices now score 30% higher on expressivity while maintaining natural speech patterns.

Next Steps

Our current results are just the beginning. There's plenty of work ahead and quite a few tricky nuances to figure out: different languages and cultures use different intonation patterns to express emotions; non-standard voices (voices of non-human characters like robots, aliens, monsters, and cartoon characters) have unusual acoustic characteristics and may require special attention.

One way or another, we're planning to share the code for our metrics with the community. We are also going to create special benchmarks for developing and calibrating new ones.

If you're working on similar problems, we'd love to hear from you. The intersection of perceptual quality and measurable metrics is a fascinating space, and there's room for everyone to contribute.

Original source

Launched Inworld TTS 1.5, our newest generation of realtime TTS models featuring:

• Two New Models: Our flagship model inworld-tts-1.5-max is ideal for most use cases, with the best balance of quality and speed. For use cases where latency is the top priority, we also offer inworld-tts-1.5-mini.
• Latency Improvements: Our new TTS-1.5 models achieve P90 latency for first audio chunk delivery under 250ms for our Max model and under 130ms for our Mini model, a 4x improvement compared to TTS-1.
• More Expressive and More Stable: TTS-1.5 is 30% more expressive than prior generations and demonstrates a 40% reduction in word error rates.
• Additional Languages: We've added support for additional languages, including Hindi, Arabic, and Hebrew, bringing total languages supported to 15.

Original source

Announcing Inworld TTS-1.5. The world's best realtime text-to-speech. <200ms latency. #1 on benchmarks.

We’re releasing Inworld TTS-1.5, the fastest, highest-quality realtime voice AI models available. With time-to-first-audio P90 latency of <250ms for 1.5 Max and <130ms for 1.5 Mini (4x faster than prior generations) and top rankings on independent leaderboards, this release sets a new standard for developers building voice-enabled applications at scale. TTS-1.5 improves on the Inworld models already #1 on leaderboards with 30% greater expressiveness, 40% reduction in word error rates, and enhanced multilingual support. It is also more than 25x lower cost than alternatives. Whether you're powering conversational AI agents, live translation, or interactive media experiences, TTS 1.5 gives you the world’s best text-to-speech without compromise.

Inworld TTS-1.5 Max is recommended for most applications, while TTS-1.5 Mini is optimized for hyper-latency sensitive applications.

Production-grade realtime latency: Professional voice actor quality at human-native speeds

For realtime applications latency isn't just a metric. It's the difference between a natural conversation and an awkward delay. TTS-1.5 delivers breakthrough speed improvements that unlock new categories of realtime experiences.

Our new TTS-1.5 models achieve time-to-first-audio P90 latency under 250ms for our Max model and under 130ms for our Mini model. This is a 4x improvement from prior generations. The Max model now delivers quality previously only achievable at much higher latencies, running nearly as fast as the Mini model while producing richer, more expressive speech.

Engagement-optimized quality: Upgrade every user experience with leading expression and stability

Speed means nothing if quality suffers. TTS 1.5 delivers both. Our models rank #1 on the Artificial Analysis TTS Leaderboard. What makes this ranking particularly meaningful is that it reflects blind comparisons by thousands of real users evaluating which outputs sound more natural and human. When developers and end users consistently choose Inworld TTS over alternatives, that's validation that matters.

Beyond third-party validation, TTS 1.5 is 30% more expressive than prior generations and demonstrates a 40% reduction in word error rate, reducing hallucinations, cutoffs, and artifacts. The result is speech that's virtually indistinguishable from human speaking: emotionally nuanced, contextually aware, and reliably accurate.

An expanded range of expression means support for many new consumer-facing use cases and applications, where the personality of voice really matters to engage, retain and convert every user. Ultimately meaning better business outcomes across the next wave of AI applications.

Unlocked consumer-scale: Enhanced multilingual support, still 25x lower cost than alternatives

State-of-the-art voice AI should be accessible to every developer, from indie hackers building their first voice app to enterprises scaling to millions of users.

Language support now spans 15 languages, with the addition of Hindi and expanded coverage across major world languages. Combined with on-prem deployment options, TTS 1.5 serves global enterprises with diverse requirements for data residency, compliance, and customization.

Most significantly, TTS 1.5 remains 25x more affordable than the next best model, a gap that's only widened as competitors have raised prices. At $0.005 per minute for 1.5 Mini and $0.01 per minute for 1.5 Max, we're keeping our commitment to radically accessible pricing that doesn't force developers to choose between quality and budget.

Model Price (in characters) Price (per minute) Best For
inworld-tts-1.5-max $10/M characters $0.01/min Most applications. Great balance of optimal quality and latency.
inworld-tts-1.5-mini $5/M characters $0.005/min Extremely latency-sensitive applications.

What Inworld TTS 1.5 unlocks: use-case inspiration

Bible Chat, Particle, Luvu, Talkpal, Astrobeam, and many others are proving what is possible when developers have access to consumer-grade voice AI. The combination of sub-200ms latency, benchmark-leading quality, and accessible pricing opens new possibilities:

• Conversational AI agents: Build voice assistants that respond naturally, without the awkward pauses that break immersion. TTS-1.5's speed makes multi-turn conversations feel genuinely fluid.
• Real-time translation and dubbing: Live interpretation requires voice synthesis that keeps pace with speakers. TTS-1.5 delivers the latency profile that makes real-time language bridging viable at scale.
• Interactive entertainment: From AI companions to narrative experiences, TTS-1.5 enables characters that speak with emotional range and contextual awareness, responding in real-time to user input.
• Accessibility applications: Screen readers, navigation aids, and assistive technologies benefit from natural-sounding speech that doesn't fatigue listeners or create cognitive load.

"We're blown away with Inworld’s latest models which achieve unmatched voice realism at a fraction of the cost. We’re excited to bring these models to Layercode where developers can create and deploy realtime latency, life-like voice agents with them." - Damien Tanner, CEO, Layercode

Enterprise-ready deployment options, now supporting On-Prem

TTS 1.5 supports the deployment flexibility enterprises require:

• Cloud API: Immediate access via our standard API with global availability
• On-prem deployment: Full model hosting on your infrastructure
• Custom solutions: Contact our enterprise team for volume pricing, SLAs, and tailored deployment architectures

For organizations with strict data residency requirements or regulatory constraints, on-premise deployment provides complete control over voice synthesis without sacrificing capability.

TTS 1.5 is also available now via Layercode, LiveKit, NLX, Pipecat, Stream Vision Agents, Ultravox, Vapi, and Voximplant.

"I see an inflection in the not-so-distant future where conversational voice becomes the primary interface. The exciting thing is a lot of the technologies, like Inworld's realtime TTS, that need to come together to make this a reality are already here. And they're only getting better. So it's super exciting to operate in this space with partners like Inworld setting the pace for this innovation." - Andrei Papancea, CEO & Co-Founder, NLX

Get started today

TTS 1.5 is available now:

• 1. Try the TTS Playground: Hear TTS 1.5 in action with your own text or clone with a voice sample
• 1. Read the documentation: API reference, SDKs, and integration guides
• 1. Contact enterprise sales: Volume pricing, on-premise options, custom voice development

We're just getting started. TTS 1.5 represents our most significant voice AI release yet, and the foundation for what's coming next. We can't wait to see what you build.

Questions? Reach out to our team today.

Original source

Build faster, smarter realtime agents - instant streaming, lower latency, and smart interruption handling

1. Runtime V0.8

Built for any use case: companions, language tutors, customer support, fitness trainers, games, and more.

• Lower Latency Agents : Core runtime optimizations reduce latency significantly, making live multimodal agents feel snappier even under heavy LLM and TTS loads.
• Instant streaming responses: Graph start is asynchronous now, enabling agents to begin streaming tokens or audio as soon as a run kicks off, eliminating awkward silence at the start of each interaction.
• Smart early stopping: Cancel an agent run mid‑response for barge‑in, safety, or cost control so agents stop talking the moment the user or policy requires it.
  For new users: Get Started
  For returning users: Follow the 0.6 -> 0.8 migration guide

2. Template Library

Production‑ready in one click

• Launch from full example projects: Clone production‑ready multimodal templates directly into your project and go from idea to running agent in minutes.
• Find the right template fast: Filter by input/output modality, SDK, and use case to jump straight to the patterns that match your product.
  Start Building with Templates on Portal
  Start Building with Templates on Website
  Don’t find the template you’re looking for? Talk to our team to request one

3. Overview Page

API keys, onboarding, and application health in one place

• Personalised onboarding by SDK: See a tailored “getting started” flow for your chosen SDK, so you only follow the steps that matter to your stack.
• Everything you need, front and center: Jump straight to your API key, llms.txt, and templates from the home view, cutting setup time from minutes to seconds.
• Holistic observability at a glance: View key traces and logs so you can catch regressions, debug incidents, and keep AI experiences reliable.
  Personalized Onboarding
  Observability at a Glance
  Get Started
  Talk to our team

Original source