Hume Release Notes

Follow

55 release notes curated from 61 sources by the Releasebot Team. Last updated: Nov 17, 2026

Get this feed:
  • Nov 16, 2026
    • Date parsed from source:
      Nov 16, 2026
    • First seen by Releasebot:
      Nov 17, 2026
    • Modified by Releasebot:
      Nov 17, 2026
    Hume logo

    Hume

    Faster responses

    Hume improves response speed for a faster experience.

    Faster responses

    Original source
  • Nov 13, 2026
    • Date parsed from source:
      Nov 13, 2026
    • First seen by Releasebot:
      Nov 14, 2026
    Hume logo

    Hume

    Hume Audio Decision Benchmarks

    Hume releases Audio Decision, an API model that predicts how listeners would rate audio and returns a distribution of likely answers without needing a transcript. It leads the reported human-preference benchmarks and combines calibrated predictions with low latency and cost.

    Predicting a listening test in one API call

    Before shipping a voice, codec, or speech pipeline, teams need to know how human the voice sounds. That usually means running a listening test: recruiting raters, collecting judgments, and waiting for results. Even a straightforward question like “Does this voice sound natural?” takes time and money.

    Our new decision model, Hume Audio Decision, estimates how people would judge an audio clip without needing a transcript. Send a clip and a question, and the model predicts the share of human raters who would choose each answer, such as each rating on a five-point naturalness scale. The output is a distribution over answers, showing how listeners split when they disagree.

    In our tests on four public human-preference benchmarks (AudioMOS-AES, NISQA, SOMOS, and DataPoint), Hume Audio Decision scored highest across 11 models. The comparison includes Google’s newest general model, Gemini 4 Hydrogen, and Google’s decision model, Gemini Decision 1, which also returns a distribution of answers.

    We also compared Hume Audio Decision with individual human raters. In six of seven comparisons, the model agreed with the rest of the crowd more closely than a single held-out rater did.

    Hume Audio Decision had the lowest mean latency we measured, at 0.22 seconds per question, and it costs less than every model that scored 0.73 or higher overall. On general audio understanding, it comes within 0.043 of the leading model on MMSU and within 0.057 on MMAU, and it leads ParaPair and EmphAssess.

    Comparing predictions with human ratings

    Two examples show what the output looks like. In the first, raters judged speech quality. In the second, they judged whether a speaker emphasized one word. Each example compares the model’s predicted distribution with the raters’ votes, and the model outputs were recorded in advance.

    How good does this speech sound?

    Rating Model Human votes Bad 63.39% 6 / 8 Poor 29.02% 2 / 8 Fair 5.99% 0 / 8 Good 1.34% 0 / 8 Excellent 0.27% 0 / 8

    Recorded output for “What overall quality rating would you give this speech sample?” Audio: URGENT 2026 SQA, CC BY 4.0; MP3 re-encoding. Percentages rounded.

    Is the word “stifle” emphasized?

    “And in return for all this you stifle me.”

    Answer Model Human votes Yes 95.66% 8 / 8 No 4.34% 0 / 8

    Recorded output for “Decide whether ‘stifle’ is emphasized in this clip.” Audio: LibriTTS, Zen et al. (2019). Prominence votes: Morrison et al. (2023). CC BY 4.0; MP3 re-encoding.

    Benchmark results

    We tested Hume Audio Decision and ten other models on eleven public benchmarks in three groups: human preference, audio understanding, and tool calling. Every model received the same audio and questions. Most models return a single answer. Hume Audio Decision and Gemini Decision 1 return a distribution, and we score their top answer for accuracy.

    Hume Audio Decision leads all four human-preference benchmarks, ParaPair, and EmphAssess. Gemini 4 Hydrogen leads MMSU, MMAR, and Auto-SLURP. Qwen3.8-Omni-Flash leads MMAU, and Gemini 3.8 Flash leads Audio2Tool.

    On the overall score, the mean of the three group means, Hume Audio Decision ranks first at 0.800. Gemini 3.8 Flash follows at 0.770 and Gemini 4 Hydrogen at 0.755. Highlighted values mark the leader in each column.

    Model Overall AES NISQA SOMOS DataPoint MMSU MMAU ParaPair EmphAssess MMAR Auto-SLURP Audio2Tool Hume Audio Decision 0.800 0.784 0.762 0.526 0.781 0.820 0.762 0.930 0.987 0.660 0.936 0.774 Gemini 3.8 Flash 0.770 0.689 0.517 0.384 0.756 0.839 0.788 0.876 0.726 0.799 0.961 0.877 Gemini 4 Hydrogen 0.755 0.629 0.537 0.383 0.756 0.863 0.802 0.893 0.579 0.828 0.966 0.825 Qwen3.8-Omni-Flash 0.730 0.743 0.583 0.135 0.719 0.811 0.819 0.900 0.900 0.383 0.790 0.955 Clef-Omni 0.672 0.632 0.434 0.238 0.665 0.708 0.771 0.587 0.379 0.696 0.962 0.830 Gemini Decision 1 0.656 0.606 0.440 0.327 0.751 0.679 0.742 0.661 0.327 0.634 0.897 0.761 gpt-audio-1.5 0.654 0.648 0.283 0.243 0.711 0.677 0.747 0.754 0.332 0.654 0.910 0.806 Inkling-Small 0.650 0.647 0.406 0.160 0.696 0.744 0.788 0.822 0.318 0.651 0.840 0.776 Gemini 3.5 Flash Lite 0.613 0.598 0.324 0.297 0.701 0.678 0.719 0.573 0.302 0.603 0.861 0.709 Nemotron 3 Nano Omni 0.584 0.433 0.199 0.095 0.634 0.651 0.761 0.504 0.261 0.586 0.931 0.790 Kimi-Audio-7B 0.562 0.114 0.278 0.281 0.715 0.630 0.697 0.536 0.336 0.564 0.855 0.715

    The four human-preference columns show Spearman correlation with the crowd’s mean rating. DataPoint instead shows how often a model picks the crowd’s majority answer. The other seven columns show accuracy. Overall averages the three group scores (human preference, understanding, and tool calling), and each group score is the mean of its benchmarks.

    Human preference

    NISQA measures audio quality using mean opinion scores (MOS). SOMOS is a large-scale MOS dataset for text-to-speech (TTS), covering 200 systems built on one voice. DataPoint collects A/B judgments on modern TTS systems. AudioMOS-AES rates production complexity, production quality, content usefulness, and content enjoyment.

    Hume Audio Decision leads all four, by margins from 0.025 on DataPoint to 0.179 on NISQA.

    BENCHMARK QUESTIONS HUME AUDIO DECISION BEST OTHER MODEL RANK OF 11 AudioMOS-AES, 4 axes 12,240 0.784 Qwen3.8-Omni-Flash 0.743 1 NISQA, 5 dimensions 3,560 0.762 Qwen3.8-Omni-Flash 0.583 1 SOMOS, per clip 2,047 0.526 Gemini 3.8 Flash 0.384 1 DataPoint naturalness 6,300 0.781 Gemini 3.8 Flash and Gemini 4 Hydrogen 0.756 1

    Understanding

    The understanding benchmarks evaluate what is in the audio and how it is spoken. MMSU tests spoken-language understanding and reasoning. MMAU covers questions about speech, environmental sounds, and music, while MMAR tests deeper reasoning across those audio types. ParaPair compares speaking rate, emphasis, and style between clips, and EmphAssess tests whether particular words are emphasized.

    Hume Audio Decision leads ParaPair and EmphAssess. It comes within 0.043 of the leader on MMSU and within 0.057 on MMAU. Gemini 4 Hydrogen leads MMSU and MMAR, and Qwen3.8-Omni-Flash leads MMAU.

    BENCHMARK QUESTIONS HUME AUDIO DECISION LEADER, OR BEST OTHER MODEL RANK OF 11 MMSU 5,000 0.820 Gemini 4 Hydrogen 0.863 3 MMAU 1,000 0.762 Qwen3.8-Omni-Flash 0.819 6 ParaPair, non-tie 10,350 0.930 Qwen3.8-Omni-Flash 0.900 1 EmphAssess 1,716 0.987 Gemini 3.8 Flash 0.726 1 MMAR 997 0.660 Gemini 4 Hydrogen 0.828 5

    Tool calling: more to come

    We’re continuing to develop tool-calling capabilities and will share more soon. Current results are included in the overall benchmark table.

    Comparisons with human raters

    The benchmarks above compare models with one another. The next three tests compare Hume Audio Decision with people: a single held-out rater, panels of several raters, and the crowd’s agreement at each confidence level.

    Hume Audio Decision vs. a single rater

    We start with a single rater. We hold out one rater and measure how often that person agrees with the rest of the crowd. Then we measure Hume Audio Decision against the same crowd. We repeat this for every rater on every question, using public sets that publish each rater’s votes.

    BENCHMARK AND AXIS ONE RATER HUME AUDIO DECISION DIFFERENCE DataPoint naturalness, hit rate 0.567 0.686 +0.119 [+0.103, +0.133] AudioMOS-AES content enjoyment 0.632 0.783 +0.150 [+0.138, +0.164] AudioMOS-AES content usefulness 0.625 0.745 +0.120 [+0.107, +0.134] AudioMOS-AES production quality 0.582 0.809 +0.227 [+0.214, +0.239] AudioMOS-AES production complexity 0.738 0.435 −0.302 [−0.336, −0.269] NISQA LIVETALK speech quality, 5 dimensions 0.607 0.721 +0.114 [+0.073, +0.152] SOMOS naturalness 0.191 0.520 +0.329 [+0.299, +0.361]

    Agreement with the rest of the crowd. Differences show 95% bootstrap intervals.

    On DataPoint, a single held-out rater picks the crowd’s majority answer 56.7% of the time, and Hume Audio Decision picks it 68.6% of the time. SOMOS raters agree poorly with one another because ratings vary little between clips. The one axis where a single rater scores higher is AudioMOS-AES production complexity (0.738 vs. 0.435).

    How many human raters match Hume Audio Decision?

    Listening tests usually average several raters, so we also asked how many raters it takes to match Hume Audio Decision. We build panels of randomly selected raters, average each panel’s votes, and compare that average with the held-out crowd. Larger panels agree more closely. The panel size at which human agreement equals Hume Audio Decision’s is the model’s rater equivalent.

    Across all axes, Hume Audio Decision matches about 2.0 raters on NISQA and 1.7 raters on AudioMOS-AES. On DataPoint and SOMOS, the model agrees with the crowd more closely than the largest panels we could build: 5 raters on DataPoint and 4 on SOMOS.

    Results vary by axis. On NISQA loudness, Hume Audio Decision matches about 6.3 raters. On AudioMOS-AES production complexity, it matches fewer than one.

    BENCHMARK AND AXIS HUME AUDIO DECISION ONE RATER EQUIVALENT RATERS NISQA LIVETALK loudness 0.686 0.408 about 6.3 AudioMOS-AES production quality 0.767 0.552 about 4.6 NISQA LIVETALK coloration 0.742 0.513 about 4.1 AudioMOS-AES content enjoyment 0.761 0.616 about 2.8 NISQA LIVETALK discontinuity 0.696 0.561 about 2.4 AudioMOS-AES content usefulness 0.712 0.599 about 2.1 NISQA LIVETALK noisiness 0.682 0.562 about 2.0 NISQA LIVETALK overall quality 0.803 0.729 about 1.8 AudioMOS-AES production complexity 0.412 0.723 under 1 DataPoint naturalness, hit rate 0.653 0.560 more than 5 SOMOS naturalness 0.376 0.139 more than 4

    Agreement is Spearman rank correlation for rating scales and agreement with the held-out crowd’s majority choice for DataPoint.

    The panel sizes have a limit. Each dataset publishes a fixed number of votes per item, so the largest panel we can build is small. On DataPoint, the largest panel is 5 raters, so “more than 5” is as far as we can measure.

    SOMOS clips have 8 to 19 votes, with a median of 11. We hold out 4 votes per clip and cap panels at 4 raters, so every clip appears at every panel size.

    How confidence compares with crowd agreement

    Because Hume Audio Decision returns a distribution, each answer comes with a confidence. That confidence is useful for routing requests or flagging clips for human review only if it is accurate. When the model says 75%, the crowd should agree about 75% of the time.

    To check calibration, we group predictions by how confident the model is in its top answer. For each group, we measure how often the crowd’s top answer matches.

    On Humeval dev, our internal held-out set of 6,849 questions, confidence and agreement line up. When Hume Audio Decision is 75% confident, the crowd agrees 74% of the time. In one confidence group, Qwen3-Omni-30B-A3B-Instruct, which also returns a distribution, is 97% confident but matches the crowd only 46% of the time.

    Expected calibration error (ECE) is the gap between confidence and agreement, averaged over the groups and weighted by their size. Lower is better. The table compares ECE on our internal set and four public sets.

    SET QUESTIONS HUME AUDIO DECISION CLEF-OMNI KIMI-AUDIO-7B QWEN3-OMNI-30B-A3B-INSTRUCT Humeval dev, internal held-out 6,849 0.010 not scored not scored 0.421 NISQA 3,560 0.044 0.352 0.309 0.602 SOMOS 2,047 0.043 0.182 0.341 0.243 AudioMOS-AES 12,240 0.112 0.081 0.369 0.198 DataPoint 6,300 0.113 0.202 0.199 0.224

    Expected calibration error by set. Lower is better, and highlighted values are the lowest in each row.

    Cost and latency

    We also measured latency and cost through each model’s API. Hume Audio Decision returns its distribution without generating text, and its mean latency was 0.22 seconds per question, the lowest we measured. Gemini Decision 1 was next at 0.42 seconds. The other models took 0.68 to 10.23 seconds, including 5.22 seconds for Gemini 4 Hydrogen.

    At $0.28 per 1,000 questions, Hume Audio Decision costs less than every model that scores 0.73 or higher overall: Qwen3.8-Omni-Flash ($0.34), Gemini 3.8 Flash ($1.00), and Gemini 4 Hydrogen ($4.43). Gemini Decision 1 and Gemini 3.5 Flash Lite cost less, and both score lower on all four human-preference benchmarks.

    MODEL MEAN LATENCY PER QUESTION MEAN $ PER 1,000 QUESTIONS Hume Audio Decision 0.22 s $0.28 Gemini Decision 1 0.42 s $0.05 gpt-audio-1.5 0.68 s $3.37 Gemini 3.5 Flash Lite 0.88 s $0.11 Inkling-Small 1.13 s $0.38 Gemini 3.8 Flash 3.33 s $1.00 Gemini 4 Hydrogen 5.22 s $4.43 Qwen3.8-Omni-Flash 10.23 s $0.34

    Latency is server-side processing time, averaged over 200 requests per benchmark sent one at a time. Hume Audio Decision was measured on 8 of the 10 benchmarks because the API does not yet support paired audio.

    Cost is the billed API cost. Gemini Decision 1 is estimated from input tokens at list price. Clef-Omni, Kimi-Audio-7B, and Nemotron 3 Nano Omni are not included.

    Try it

    Hume Audio Decision is available through the Hume API at $0.47 per million Hume tokens.

    FAQ

    How does Hume Audio Decision differ from a mean opinion score (MOS) predictor? A MOS predictor learns one score for one kind of rating. Hume Audio Decision takes the question as input, so one model handles rating scales, A vs B comparisons, multiple-choice questions, and yes/no questions. On a single dataset with its own training split, a dedicated predictor can score higher.

    Do you train on these benchmarks? DataPoint and AudioMOS-AES are used only for evaluation. NISQA and SOMOS test files are excluded from training.

    Why does Hume Audio Decision predict a distribution instead of one answer? A distribution estimates the share of listeners who would choose each answer, including when preferences are closely divided.

    How is calibration calculated for models that return only one answer? For API models that do not return logits, this evaluation assigns 100% probability to the predicted choice when calculating calibration error. Their ECE on the four public sets is 0.42 to 0.85.

    Methodology

    • Benchmarks. AudioMOS-AES (12,240 questions, 4 axes), NISQA test sets (3,560 questions, 5 quality dimensions), SOMOS clean test (2,047 clips), and DataPoint naturalness (6,300 pairs). The comparisons with single raters use NISQA LIVETALK only, the one NISQA set that publishes each rater’s votes. Those comparisons use panels of 10 raters for DataPoint and AudioMOS-AES, 24 for NISQA LIVETALK, and a median of 11 for SOMOS.
    • Metrics. Rating scales use Spearman r between the model’s score and the crowd mean. A vs B uses the share of pairs where the model picks the crowd’s majority. SOMOS per-clip scores are correlations over clips. Benchmark scores use the full evaluation sets, and the 200-request samples per benchmark are used only to measure latency and cost. ParaPair is the one exception: it is scored on non-tie pairs only. When we reviewed pairs that raters marked as ties, many still showed a clear difference between the two clips, and the ParaPair authors also report non-tie accuracy separately.
    • Gemini models. Gemini Decision 1 was evaluated on November 11, 2026, through the Gemini Developer API. It returns a full answer distribution, and accuracy uses its top answer. Gemini 4 Hydrogen was evaluated on November 12, 2026, on Vertex AI for accuracy and through OpenRouter, with Google as the provider, for latency and cost. It ran at its default thinking level, and its single answer is scored.
    • Latency and cost. Means over a fixed sample of 200 requests from each launch benchmark, sent one at a time. The other models were measured on 10 benchmarks and Hume Audio Decision on 8, through the Hume API. The two pairwise benchmarks are excluded for Hume Audio Decision because the current API does not support paired audio inputs. Latency is server-side processing time for every model: provider time reported by OpenRouter, OpenAI, and Google, and gateway server time for the Hume API. API cost is the billed cost. Gemini Decision 1’s cost is estimated from its input tokens at the list price of $0.10 per million input tokens, recorded on November 11, 2026.
    • Confidence intervals. We report 95% bootstrap confidence intervals for the comparisons with human raters but not for the main benchmark scores. Single-rater comparisons use 1,000 bootstrap resamples over questions. Panel-curve bands are 95% bootstrap intervals over requests.
    Original source
  • All of your release notes in one feed

    Join Releasebot and get updates from Hume and hundreds of other software products.

    Create account
  • Nov 7, 2026
    • Date parsed from source:
      Nov 7, 2026
    • First seen by Releasebot:
      Nov 11, 2026
    Hume logo

    Hume

    NOV 7, 2026

    Hume introduces shorter audio, making audio experiences more concise.

    Shorter audio

    Original source
  • Nov 6, 2026
    • Date parsed from source:
      Nov 6, 2026
    • First seen by Releasebot:
      Nov 7, 2026
    Hume logo

    Hume

    Higher rate limit

    Hume increases its rate limit.

    Higher rate limit

    Original source
  • Nov 2, 2026
    • Date parsed from source:
      Nov 2, 2026
    • First seen by Releasebot:
      Nov 3, 2026
    Hume logo

    Hume

    Audio Decisions API launch

    Hume launches the Audio Decisions API.

    Audio Decisions API launch

    Original source
  • Similar to Hume with recent updates:

  • Nov 2, 2026
    • Date parsed from source:
      Nov 2, 2026
    • First seen by Releasebot:
      Nov 3, 2026
    Hume logo

    Hume

    Introducing Hume's Audio Decision API: A decision layer for speech and sound

    Hume introduces its Audio Decision API, which turns speech and sound into structured answers with confidence scores. Teams can ask custom questions about emotion, speakers, background sounds and more, then use the results to evaluate voice models, analyze calls and build more perceptive voice agents.

    Contact centers score every call against a rubric. Changing what gets scored in the call is a project: new rubric, new pipeline, new evaluations, days if not weeks before the first answer.

    And the transcript this all runs on is only the words. Not the pause, not the sigh, not the two people talking over each other. It misses the reasons a call went wrong. That information is in the audio, sitting in call recordings or actively getting ignored on real time calls. Until now, there has been no way to extract this information in a form you can act on.

    Today we're introducing Hume's Audio Decision API, the first audio-native model that answers questions about speech and sound. You choose the questions and the possible answers, anything from whether a voice is AI generated to identifying if the caller is frustrated or if there is a siren in the background noise.

    This is a different architecture. Raw audio goes in, an answer with a confidence score comes out, structured and ready for your code or your agent to act on. Fast enough to ask dozens of questions every few seconds on a live call and fast enough to make an archive with 40,000 audio recordings searchable in minutes. There is no system today where you can make that query.

    It is built on years of Hume's emotion science research and millions of human judgments about how people perceive speech and sound. What used to take a composition of human listeners and distinct model judgements, now takes a single API call.

    Ask your audio a question

    Send raw audio and ask a question. The API returns probabilities over the answers you defined.

    You write the question as a sentence along with the possible answers. The Audio Decision API returns probabilities for the answers you provide, whether they're yes/no, multiple choice, rating scales or spans across a clip:

    • Yes/No: "Does the customer sound frustrated?" returns yes or no
    • Choice: "Which best describes the agent's tone?" returns the option from your list
    • Scale: "How patient does the agent sound?" returns a score on your scale
    • Span: "Where does someone laugh, sigh or gasp?" returns start and end times

    You can ask multiple questions and update them in real time. Adding a question is a line of JSON. No labels, no retraining, no added complexity. The same recording can be asked about background sound, language, accent and emotional expression at once, and you can set confidence thresholds that link responses to business rules.

    You can also add additional context to each request, which can further optimize performance.

    Questions can cover:

    • Speech & languages
    • Speakers & voices
    • Emotion & expression
    • Acoustic environment
    • Non-speech events
    • Conversation dynamics
    • Communication quality
    • Intent & outcomes
    • Social & behavioural signals
    • Temporal & contextual signals

    We’ve included a guide on how to ask questions along with example questions across dimensions and industries in our Docs.

    To walk through an example: For a retail customer-service voice agent, you might check whether the speaker sounds uncertain or frustrated, identify how many speakers are on the call, whether there was excessive background noise and ask what user intent was: return an item, cancel a subscription, etc. In a business workflow you could pair those judgments with transcription and give both as inputs to the LLM so that it can take action on business rules such as whether to wait, clarify or escalate.

    Teams can also run questions in bulk across audio libraries, track expression over successive segments and pinpoint the moments when particular sounds or vocal events occur. The speed of processing also unlocks an audio search engine capability: find every call where the customer sounded frustrated. There is no system today where you can query audio so quickly without added pre-processing, labeling, indexing or transcription.

    Grounded in human judgment

    Counting speakers or detecting a siren involves relatively objective facts. Whether a voice sounds frustrated, empathetic or natural is a question of perception, and listeners don't always agree.

    For years, Hume has asked panels of listeners to rate emotional expression, naturalness, vocal delivery and conversational fit. Tens of millions of these judgments have informed Real-World VoiceEQ and helped train our Audio Decision Model to reflect how people perceive speech and sound.

    In an initial evaluation, Hume's Audio Decision model achieved 70.7% agreement with a human panel's average judgment, compared with 56.5% for the individual-rater baseline. The result shows how human-level perceptual judgment can be brought to speech and sound at scale, without asking a panel to review every recording. And more importantly, it unlocks human-level judgment for live voice AI interactions.

    When benchmarked against existing methods for audio processing and understanding, the Audio Decision Model outperforms on three key vectors: accuracy, speed and cost.

    Get started

    Explore the Audio Decision API and request API access by joining the waitlist. Try out the realtime and upload file experience in the playground, where you can add your own questions and get immediate responses to them, with no added configuration.

    Check out our Documentation to learn more about supported questions, experimental features and more.

    What you can build

    Evaluate and improve voice models

    A voice model can be intelligible and still sound awkward. Teams can ask whether generated speech sounds natural, conveys the intended emotion or fits the situation, then compare the answers across models and checkpoints.

    These judgments can guide data selection and training feedback, including reinforcement learning. Teams should validate their evaluations against representative human ratings.

    Build more perceptive voice agents

    A caller may say they're fine while sounding frustrated, or pause because they're thinking rather than because they've finished speaking. Those cues rarely appear in a transcript.

    Audio Decision API gives a cascaded voice agent access to those cues. It can weigh an audio judgment alongside the transcript and its business rules before responding. Live and ambient applications remain a longer-term opportunity as real-time capabilities develop.

    Analyze and search audio at scale

    A contact center could ask which calls include interruptions, whether agents sound reassuring and which conversations deserve review. Teams can apply those questions across an entire library, then change the criteria as new patterns emerge.

    They can also track how expression changes during a call and jump to the moments when specific sounds or vocal events occur.

    Built for quick decisions

    Many audio tasks need a judgment, not a description. Generative models are useful for open-ended explanations, but generating text adds unnecessary work when all an application needs is a structured answer. Fixed classifiers tend to be limited to predefined tasks.

    Audio Decision API returns probabilities for the choices developers define. It's designed for fast, cost-efficient judgments that can be repeated across many recordings.

    Try out the Audio Decision API today and unlock every call.

    Original source
  • Oct 6, 2026
    • Date parsed from source:
      Oct 6, 2026
    • First seen by Releasebot:
      Oct 7, 2026
    Hume logo

    Hume

    Introducing Expression APIs: Give your AI more of what people mean

    Hume introduces Expression APIs that turn voice and facial cues into structured scores for live interactions, recordings, and research. The release adds rich audio and video expression measurement to help developers capture context beyond transcripts and build more expressive AI experiences.

    Human communication carries meaning beyond words. A change in tone, a hesitant delivery, or a surprised expression can change how we understand what someone says, and how we respond.

    Consider “I can’t believe it.” After good news, those words might convey excitement. After a setback, they might express frustration. During a playful exchange, they might communicate amusement. When AI applications hear these words in any of these contexts, the words they transcribe stay exactly the same.

    For applications built to interact with people, that missing context matters. A voice agent may understand a customer’s request while missing frustration in their delivery. A researcher may know what participants said while losing the expressive detail that helps explain their reactions.

    Today, we’re introducing Hume Expression APIs to make those signals available to developers and researchers. Our models turn vocal and facial expression into structured scores that estimate how people are likely to perceive what an expression conveys, with voice expression also measuring speaking styles and vocal qualities. Expression API supports live interactions as well as analysis and tagging of recordings after the fact.

    Our models were trained on over 260,000 speech recordings and 370,000 human-rated images of facial expressions. Evaluated against human judgment (the gold standard for expression measurement), our models match or exceed state-of-the-art models on tested measures of vocal and facial expression.

    Audio Expression API: Measure vocal expression, delivery, and vocal qualities

    Expression API turns a person’s emotional expression, speaking styles, and vocal qualities into structured scores. Developers can use those signals to inform an agent’s next response, identify difficult moments for training and review, or find recordings with the characteristics their model needs to learn from.

    Our vocal-expression model measures expression across 414 tags, comprising 23 higher-level emotion categories and 391 finer-grained descriptions directly from audio. These span amusement, anger, interest, joy, sadness, and surprise, alongside more specific expressions such as admiration, frustration, nervous laughter, and relief.

    Our voice descriptor model adds 190 characteristics, including raspy, breathy, fast, monotone, whispering, and theatrical. Together, the models describe combinations such as quietly expressed frustration or amusement in a deep, raspy voice, giving developers richer annotations for analysis and training.

    Available in 50+ languages, including Arabic, German, English, Spanish, French, Indonesian, Italian, and Japanese.

    Video Expression API: Bring facial expression into the picture

    Video Expression makes visible signals that people communicate via facial expression available as structured measurements. Developers can use them to inform live interactions, while researchers can tag recordings, locate reactions worth reviewing, and compare expressive responses alongside what participants say.

    Our facial expression model measures 48 emotional-expression categories, including admiration, amusement, anger, anxiety, concentration, confusion, excitement, joy, and sympathy. It also measures 27 visible facial descriptions, such as Smile, Frown, and Jaw drop.

    Our facial-expression model draws on a human-rated collection with participants from the United States, South Africa, India, China, Venezuela, and Ethiopia.

    Expression APIs in action: put expression measurement to work

    Use voice and facial-expression scores during live interactions, or analyze and tag recordings afterward. Timestamped results connect scores to the source material, helping teams review relevant moments and select data for evaluation and training. Batch processing is coming soon.

    Help AI agents resolve difficult calls and improve customer satisfaction

    An AI agent may understand a customer’s request but miss frustration in their delivery. Real-time vocal-expression scores provide context for acknowledging the difficulty, clarifying an answer, or offering a handoff—with the goal of reducing unnecessary escalations and improving customer satisfaction.

    After calls, expression tags can help contact-center teams find moments expressing frustration or anger, prioritize QA review, and select examples for coaching human agents or improving AI behavior.

    Make market research faster and capture reactions more fully

    Transcripts can flatten participants’ reactions, while reviewing hours of recordings takes time. Voice and Visual Expression APIs help researchers locate and compare expressive responses to a product concept, advertisement, or message alongside participants’ stated feedback.

    This focuses manual review and preserves context beyond the transcript. Researchers can investigate where words and delivery differ, interpreting those moments through the recording, follow-up questions, and broader study.

    Help robots adapt when an interaction breaks down

    A robot may continue its instructions when a person appears confused or expresses frustration. Vocal and facial-expression measurements add context for deciding when to pause, repeat, simplify, or request human assistance.

    In health and aged-care settings, developers can use these measurements to build and test interactions designed to reduce misunderstandings and help people complete tasks more comfortably, alongside their words and circumstances.

    Find targeted data and test whether your model improves

    Finding training examples to address an evaluation weakness can require substantial manual review. Expression and voice tags help researchers retrieve recordings with characteristics such as quietly expressed frustration, whispered speech, or a raspy voice. Facial-expression scores similarly organize visual data.

    Teams can preprocess data for evaluation by tagging recordings and building targeted test sets. The same annotations help identify underrepresented expressions and select fine-tuning and post-training examples, reducing search time and supporting progress measurement.

    Start building with Hume’s Expression API

    Talk to Hume about API access and how voice and visual expression can fit your application, research, or model-development workflow.

    Original source
  • Oct 6, 2026
    • Date parsed from source:
      Oct 6, 2026
    • First seen by Releasebot:
      Oct 7, 2026
    Hume logo

    Hume

    Behind Hume’s Expression Measurement Models for Face and Voice

    Hume introduces its Expression API for real-time voice and facial expression measurement, adding structured scores for emotion, voice traits, and facial cues to help developers annotate data, evaluate AI outputs, and analyze expressive media across multiple languages.

    Editor's note

    This post covers the science, development and evaluation of Hume’s state of the art Expression API, that enables developers to enrich their applications or research pipelines with real-time measurement of voice and facial expressions.

    Facial expressions and vocal tone are central to how we communicate. They convey emotions such as amusement, interest, frustration, and surprise, often without naming those feelings in words. Understanding these signals is part of how we connect with one another and respond to what others express.

    Expression measurement turns these signals into structured scores that can be used to annotate data, evaluate AI outputs, and explore expressive audio and images. It gives us a way to describe the emotional meaning people perceive, compare expressions across recordings, and follow how an interaction changes over time.

    Consider someone saying, “I can’t believe it.” After good news, the phrase might sound excited. After a setback, it might convey frustration. During a playful exchange, it might express amusement. The words stay the same, while the voice and facial expression contribute different meanings.

    At Hume, our work on expression measurement is grounded in emotion science. Our research has explored the range of meanings people associate with facial and vocal expressions, including nuanced and blended expressions. This scientific perspective guides our approach: measure expression through a detailed vocabulary and evaluate those measurements against human judgments. Read more about the science behind Hume’s expression measurements.

    An expression can communicate several emotional qualities at once, such as amusement blended with disbelief. Our speech emotion model captures this detail through separate scores for 414 tags: 23 core emotions and 391 finer-grained descriptions. These include amusement, anger, interest, joy, sadness, and surprise, alongside more specific expressions such as admiration, frustration, nervous laughter, and relief.

    The voice descriptor model measures 190 characteristics across 27 families, describing qualities such as raspy, breathy, fast, monotone, whispering, and theatrical. These measurements complement emotional expression, allowing us to describe combinations such as frustration expressed quietly or amusement in a deep, raspy voice. Our speech models support Arabic, Bengali, German, English, Spanish, French, Hebrew, Indonesian, Italian, Japanese, Korean, Polish, Portuguese, Russian, Turkish, and Vietnamese.

    Our facial expression model recognizes emotional expression across 48 Hume categories, including admiration, amusement, anger, anxiety, concentration, confusion, excitement, joy, and sympathy. It captures nuanced expressions and distinguishes positive from negative surprise. The model also measures 27 visible facial descriptions, such as Smile, Frown, and Jaw drop, providing a complementary view of an expression’s visible features.

    Data and human ratings behind expression measurement

    For facial expression, our training data includes more than 370,000 images from our human-rated FaceMimic collection. Across the collection, more than 13,000 raters contributed more than one million emotion ratings, describing both the emotional meaning of an expression and its intensity. Participants photograph themselves mimicking a reference expression, then rate their own and others’ expressions. The collection includes participants from the United States, South Africa, India, China, Venezuela, and Ethiopia.

    Our speech emotion and voice descriptor models share a speech training resource with more than 260,000 recordings. Its emotion-annotated portion includes more than 210,000 recordings, representing more than 270 hours of mostly performed speech, with multiple emotion annotations for most clips. The resource captures both emotional expression and vocal characteristics, supporting measurements of what a voice conveys and how it sounds.

    Evaluating agreement with human perception

    We evaluated our voice models using our Hume Feedback API, collecting structured judgments from listeners, including native speakers for multilingual studies. We asked whether the models’ descriptions agreed with human ratings and whether higher scores identified stronger examples of a particular expression or vocal characteristic.

    For each model, we begin with evaluations on our internal human-rated data, then examine results on public datasets. These studies assess how well the models recognize expression and capture the strength of the qualities people perceive.

    Within each study, the compared systems were assessed on the same evaluation samples.

    Speech Emotion Recognition Results

    We compared 11 systems on the same set of more than 9,000 recordings from 32 public datasets, including Gemini 3.8 Flash and specialized systems such as EmotionThinker, EmpathicInsight, and VoiceCLAP. All systems made zero-shot predictions: we applied the models directly, without task-specific fine-tuning for this benchmark. Our speech emotion model achieved the highest overall AUC, with statistically significant advantages over each of the other systems shown. Its advantage over Gemini 3.8 Flash also remained statistically significant when we evaluated the full test sets of more than 43,000 recordings and checked different combinations of datasets.

    We use one metric throughout the public benchmark tables: emotion-ranking score, also called AUC. Imagine taking two recordings: one labeled sad and one without sadness. The score measures how reliably the model gives the sad recording a higher sadness score, averaged across the evaluated emotions. Higher is better: 50 represents chance-level ranking and 100 represents perfect ranking. This measures how well recordings are ordered, rather than the percentage of labels guessed correctly. The overall table shows all 11 systems ordered by this score. EmpathicInsight and VoiceCLAP share the same rounded score.

    The table below groups all 32 public datasets by spoken language and uses the same emotion-ranking score as the overall comparison. Each dataset receives equal weight within its language group, and every model is evaluated on the same recordings. The final row gives the overall mean across all 32 datasets, with each dataset weighted equally. Bold values mark the highest displayed score in each row, including ties at the shown precision.

    Scores are averaged across the public datasets for each language, with each dataset given equal weight.

    We then compared our speech emotion model with Gemini on internal human ratings collected through our Hume Feedback API. This evaluation covers finer-grained emotional expressions across 16 languages, excluding the broad parent emotion categories. We use the same emotion-ranking metric as in the public benchmark: AUC, shown on a 0-100 scale, with higher scores indicating better ranking. The mean gives each language equal weight.

    On our internal human-rated evaluation of finer-grained emotional expressions across 16 languages, our model performed comparably to Gemini.

    Internal human-rated evaluation of finer-grained emotional expressions across 16 languages, with each language weighted equally.

    On an English evaluation of emotion labels named by multiple annotators, our model achieved a statistically significant emotion-ranking advantage over Gemini 3.8 Flash.

    Voice Descriptor Characteristics Results

    Voice characteristics give us another way to describe and organize speech. Someone searching an audio collection might want to find raspy voices, fast delivery, or theatrical expression. Our Hume Expression API assigns scores to these qualities, helping bring relevant recordings to the top.

    To evaluate how well this works, we use a descriptor-ranking score, measured by average precision. For example, when recordings are sorted by their raspiness scores, those that listeners described as raspy should appear near the top. We average this measure across the evaluated characteristics and report it on a 0–100 scale, where higher scores indicate better ranking.

    Using human ratings collected through our Hume Feedback API, we compared our voice descriptor model with Gemini 3.8 Flash, GPT-Audio-1.5, Grok Voice Think Fast 2.0, and specialist models on an evaluation set of more than 2,000 recordings across 16 languages. Listeners assessed whether each vocal characteristic was present and how strongly it came across. Alongside the descriptor-ranking score, we report intensity-ranking agreement: how closely a model follows listeners’ judgments of which recordings express a characteristic more strongly. This is measured by Spearman correlation, with higher scores indicating closer agreement.

    Nine systems, ranked by descriptor-ranking score. Bold marks the highest score in each column.

    Facial Expression Recognition Results

    Facial expressions can convey several emotional qualities at once. A smile might communicate amusement, relief, or excitement, depending on the expression. Our facial expression model assigns scores across 48 emotional categories, alongside descriptions of visible features such as a smile, frown, or dropped jaw.

    We use the same expression-ranking score, AUC, as in the voice sections. For amusement, for example, a strong model gives higher amusement scores to faces viewers labeled amused than to faces they did not. We average this comparison across the evaluated emotions and report it on the same 0–100 scale.

    We first compared our model with 25 other models on EmoNet-Face HQ, a public benchmark of synthetic faces rated by psychology experts. All models were evaluated on the same set of about 1,200 images across 40 emotions. The comparison included Gemini, GPT, Grok, Qwen, and specialist facial emotion models.

    Hume achieved the highest expression-ranking score, with statistically significant advantages over all 25 comparators, including Gemini 3.8 Flash. The table shows Hume and the four strongest comparator families, with the highest-scoring model selected from each family.

    Highest-scoring model from each comparator family. Bold marks the highest score.

    Gemini 3.8 Flash scored 68.0 in the same evaluation. These results measure how reliably the models distinguish faces with a human-rated emotional quality from faces without it.

    We then examined how the models agreed with human judgments on our internal FaceMimic evaluation. This includes about 2,100 photographs of individual expressions, each rated by at least two people, across the 48 Hume emotions. Hume, Gemini 3.8 Flash, and Qwen3.8-27B were evaluated on the same images using the same expression-ranking metric.

    Internal FaceMimic evaluation: ~2,100 photographs of individual expressions, each rated by at least two people across the 48 Hume emotions. Bold marks the highest score.

    Our model achieved statistically significantly higher expression-ranking scores than both Gemini and Qwen in this internal evaluation. Together, the public benchmark and internal human ratings show strong recognition of the emotional qualities people perceive in facial expressions.

    We also compared Hume with dedicated facial emotion recognition models on the same held-out FaceMimic images. To make this comparison fair at the evaluation level, each pair received identical images and was scored against the same averaged human ratings, using only emotions both models could represent. The emotion mappings and image crop rules were fixed before scoring, and emotions without an aligned counterpart were excluded from both models.

    Expression-ranking score (AUC; higher is better) on shared emotions

    Expression-ranking score (AUC; higher is better) on shared emotions. Each row uses its own shared emotion vocabulary, so scores should be compared within a row rather than against the 40- or 48-emotion tables above. Bold marks the higher score in each row.

    Put expression measurement to work

    These measurements can become part of an existing workflow: adding expression labels to a dataset, comparing the delivery of generated speech, or helping reviewers find relevant moments in recorded interactions. Our human-rated evaluations provide evidence for choosing the measurements that fit each task.

    Contact Hume for Expression Measurement API access and to discuss how it can support your data annotation, model evaluation, or product workflow.

    Original source
  • October 2026
    • No date parsed from source.
    • First seen by Releasebot:
      Oct 5, 2026
    Hume logo

    Hume

    EVI API improvements

    Hume releases EVI version 4-mini with Octave 2 TTS and optional supplemental LLM support for multilingual capabilities.

    EVI version 4-mini is now available.

    This version enables the use of Octave 2 for TTS alongside a supplemental LLM of your choosing, bringing Octave 2’s multilingual capabilities to EVI. Specify "version": "4-mini" in your EVI config version to use it.

    Original source
  • October 2026
    • No date parsed from source.
    • First seen by Releasebot:
      Oct 5, 2026
    Hume logo

    Hume

    EVI API improvements

    Hume now lets users change voices during an active session with a voice_id in session settings messages.

    You can now change the voice within an active session by specifying a voice_id in a session settings message.

    Original source
  • Sep 10, 2026
    • Date parsed from source:
      Sep 10, 2026
    • First seen by Releasebot:
      Sep 11, 2026
    Hume logo

    Hume

    Introducing the Hume Voice Replication Leaderboard

    Hume launches the Voice Replication Leaderboard as part of its Real World VoiceEQ Benchmark, giving teams a live way to compare text-to-speech models on speaker identity, naturalness, audio quality, and accent and emotion handling.

    A new VoiceEQ benchmark for voice replication

    Today we are publishing the Voice Replication Leaderboard, a new component of Hume’s Real World VoiceEQ Benchmark. It measures how well leading text-to-speech models reproduce a specific voice across varied real-world conditions.

    Like Real-World VoiceEQ, the leaderboard combines human judgments with objective measures rather than reducing performance to a single score.

    How we evaluated voice replication

    We evaluated eleven leading models, asking each to reproduce the same 25 reference voices. To better reflect real-world use, the reference set spans three categories designed to stress different aspects of replication:

    • Five standard references drawn from archival recordings of public figures and conversational US speech, not specifically selected for accent or emotional variation;
    • Five expressive references spanning anger, fear, happy, sad, and calm
    • 15 references selected for accent variation, split into 6 native English varieties and 9 non-native accents, drawn from a mix of read and conversational speech.

    Every tested model was asked to replicate these reference voices with the same seven prompts, written to avoid prescribing a particular emotion or delivery style. Each generated clip was evaluated across four dimensions:

    Same speaker: A 1–5 human rating of whether the generated clip sounded like the reference speaker.

    Quality: A 1–5 human rating of whether the audio was clean and free of artifacts, independent of speaker identity.

    Naturalness: A 1–5 human rating of how human the delivery sounded, including pacing and prosody, independent of speaker identity.

    Objective similarity: The cosine similarity between the TitaNet speaker embeddings of the reference and generated clips, scored from 0 to 1.

    For the three human-rated dimensions, each clip was independently assessed by three paid raters via the Hume Human Feedback API. Raters were blind to the model that produced it and heard the reference recording first, followed by the generated clip. When we report a model's score across all references, we weight the three reference categories equally, one third each, rather than averaging over samples.

    Results

    Overall, no model was strongest across every dimension or reference type. Performance depended both on what was measured and on the voice being replicated. The results reinforce a core principle of VoiceEQ: voice systems should be evaluated as a profile of capabilities, not reduced to a single score.

    Voice Replication Leaderboard

    Human raters were asked to score 1–5 of whether a clip generated by a TTS model sounded like the reference speaker provided. Best value per column in accent color. Hover a score for the full row.

    Ranked by same speaker. No model leads every column: identity, quality, naturalness and objective similarity are led by four different models.

    hume.ai/rw-voice-eq

    Sounding natural is not the same as sounding like the right person

    A voice can sound convincingly human without sounding like the person it is intended to replicate. Cartesia sonic-3.6-beta received the highest naturalness score (4.36), but ranked eighth on same-speaker similarity (3.63). OpenBMB VoxCPM2 ranked first on same-speaker similarity (4.21), but placed fifth on naturalness (3.96).

    This is not evidence of an unavoidable trade-off; for example some models perform well on both. But it shows why the dimensions must be measured separately.

    Performance changes with the voice being replicated

    Replicating speech from expressive reference voices created the wide gap in performance: Higgs Audio v3 scored 3.38 on emotional references, while VoxCPM2 scored 4.38 on the same category. However no model performed equally well across standard, expressive, and accented references. Notably, Higgs Audio v3 tied with VoxCPM2 for the lead on accented voices, but also had the widest spread across the three reference categories.

    Same-speaker score by reference type

    Human rating out of 5. The 25 reference voices split into 5 standard, 5 emotional and 15 accented, and the accented set splits again into 6 native English varieties and 9 non-native accents. Darker is higher. Spread is best category minus worst. Best value per column in accent.

    The two models at the top score higher on non-native accents than native. Every other model but one does the reverse. Higgs Audio v3 leads accents at 4.28 and drops to 3.38 on emotional references, the widest spread on the board.

    hume.ai/rw-voice-eq

    Clean audio is increasingly becoming table stakes among the models tested

    Quality scores were high and tightly grouped, ranging from 3.85 to 4.61, with an average of approximately 4.30. Same-speaker ratings varied more widely, from 2.91 to 4.21. For example, Inworld TTS-2 received the highest quality score (4.61), but ranked ninth on same-speaker similarity (3.62).

    Objective and human-rated similarity measures are complementary, not interchangeable

    The objective embedding score broadly supported what human listeners heard but the rankings diverged in meaningful cases. VoxCPM2 and LongCat took the top two places under both approaches, in opposite order, and Fish Audio s2-pro and Higgs Audio v3 held third and fourth under both. ElevenLabs Eleven v3 ranked last under both.

    Cartesia sonic-3.6-beta ranked eighth by human same-speaker rating and fifth by objective similarity. Speaker embeddings provide a useful, scalable signal, but they do not capture everything people hear when deciding whether two clips sound like the same person. For evaluation and model improvement, objective measures and human judgments are most useful together.

    What this means for teams building and choosing voice models

    For product teams: test the voice you actually need - given there is no single model that is best across speaker identity, naturalness, quality, and every reference type. The right choice depends on the application. A model suited to neutral narration may not be the best fit for emotionally varied customer service; one that performs well for one speaker may be less reliable for speakers with different accents.

    Teams should test models using voices, prompts, languages, emotional conditions, and recording environments that resemble their actual deployment rather than relying on a general score alone.

    For model developers: separate the failure modes to drive improvement.

    Improving voice replication requires measuring identity, naturalness, quality, and objective similarity independently. Researchers should break results down by reference type and, where possible, by individual emotion and accent. This helps determine whether a weakness reflects emotional speech broadly, a particular expressive condition, or a specific type of reference recording.

    For example, our leaderboard suggests that emotionally expressive speech remains a challenge. To diagnose where the failure occurs, researchers can test two conditions separately (i) replicating a voice from an emotionally expressive reference while keeping the target text neutral, and (ii) asking the model to deliver emotionally charged text from a neutral reference. Comparing same-speaker and naturalness scores across these conditions can help distinguish identity extraction from expressive generation. If speaker identity remains stable but naturalness falls when emotional delivery is required, the weakness is likely in how the model renders expressive speech. If identity falls when the reference itself is emotional, even with neutral target text, the model may be struggling to recover a consistent representation of the speaker from expressive audio.

    Explore the leaderboard

    The Voice Replication Leaderboard is live on Hugging Face, with per-category breakdowns and curated reference and replica audio samples. It extends the Real World VoiceEQ Benchmark’s effort to evaluate voice AI the way people actually experience it.

    Public leaderboards provide a shared baseline. Hume also works with frontier labs and product teams to design evaluations around their own speakers, languages, emotional contexts, and production failure modes—combining human ratings and objective measures to identify weaknesses and generate the preference data needed to improve them. If you’re evaluating a model or developing one yourself, get in touch.

    Original source
  • Aug 21, 2026
    • Date parsed from source:
      Aug 21, 2026
    • First seen by Releasebot:
      Sep 11, 2026
    Hume logo

    Hume

    Measuring benchmark optimization in speech recognition

    Hume adds a Benchmark fitting tab to the Open ASR Leaderboard and new tests to expose benchmark optimization in speech models. The update highlights reference errors, masked numbers, and orthographic switching across public and held-out audio to better reflect real-world transcription quality.

    Public voice AI benchmarks increasingly suggest that models are performing at human levels.

    Yet those scores don't always reflect how models work in the real-world. Since public benchmarks are open and widely used, models can also become optimized for the tests themselves. Their scores may improve because they have learned benchmark-specific patterns and not because they have become better at the underlying task.

    One reason is that traditional benchmarks overlook many of the conditions and qualities that make voice systems reliable, natural, contextually appropriate, and effective in practice. That's why we recently introduced held-out sets in Real World VoiceEQ, the Open-ASR Leaderboard, and the Far-field ASR Leaderboard: to measure more of what matters in real-world use.

    However, broader measurement alone does not solve the problem. This phenomenon, sometimes called benchmark optimization or "benchmaxxing," is often discussed around machine learning, however, it has been difficult to measure in speech recognition.

    Our latest research introduces three tests to help quantify it. We evaluated 11 widely used open-source ASR models and found that several of the highest-scoring systems reproduced benchmark transcripts from the VoxPopuli English and LibriSpeech (clean, other) datasets – even when the audio contradicted them, relevant words had been silenced, or the audio equally supported two different written forms.

    In some cases, models appeared to rely not only on what was said, but also on subtle acoustic cues that indicated which benchmark they were being tested on. As a result, their scores overstated how well they could transcribe speech more generally.

    Reference disagreement (VoxPopuli case study)

    VoxPopuli is known to contain a high number of transcription errors (which is why Artificial Analysis released a cleaned version). Our consensus disagreement probe tests what happens when leading ASR models encounter these errors: Do they accurately transcribe what the audio says, or reproduce the benchmark's incorrect reference transcript?

    To test this at scale, we use an ensemble of independent models selected for their low phoneme error rate (PER). PER measures how closely a written transcription matches the sounds in the audio, making it a useful proxy for how faithfully a model transcribes what it hears. The ensemble results can be used to flag cases in which the models unanimously disagree with the benchmark's reference transcript. We then compare a sample of those flagged cases against human annotations to validate the corrected transcripts.

    For example, one VoxPopuli clip audibly includes the phrase "Thank you, Mr. President," but the reference transcript omits "Thank you." Six of the 11 models we tested reproduced the benchmark's erroneous transcript—giving the "expected" answer even though it contradicted the audio. On the real clip, the formatting follows the same pattern: models that omit "Thank you" also reproduce the benchmark's punctuation style, writing "Mr" without a period, while models that include the audible phrase tend to write "Mr." with the period.

    When we present the same content in newly collected voices from EU parliamentary recordings or generic voices, this behavior often weakens or disappears. In the below samples, all but one model flips back to transcribing the audio-faithful transcript for a clone of a new parliamentary recording. This suggests that the models are responding to acoustic cues that help them identify the benchmark membership and thus produce the expected transcript even if it contradicts the audio.

    The reference transcript for this clip reads "Mr President, I have another complaint about this procedure, which is that it is not secret." The audio in all three clips below actually says the same thing, preceded by an audible "Thank you,"—the clones are text-to-speech renditions of that true sentence, so the courtesy is audible in all three. Green highlighting and ✅ mark a transcript that includes the audible "Thank you"; red highlighting and ❌ mark a transcript that reproduces the benchmark's erroneous omission. All transcripts are raw model output, prior to any normalization—casing and punctuation are preserved exactly as generated, including lowercase output from some models.

    Parakeet is the only model that flips between reproducing the benchmark on the real clip and getting it right on the same-speaker clone. Phi-4 is the only model still dropping the courtesy on the ep-fresh clone. When we instead resynthesize the sentence in a generic TTS voice unconnected to any parliamentary recording, all eleven models restore the courtesy.

    The results suggest that this problem is both widespread and meaningful. Our methodology flagged potential reference errors in 40% of the VoxPopuli test clips we analyzed, affecting roughly 3% of all reference words.

    Models exhibiting benchmark-optimized behavior reproduced erroneous reference transcripts 18–30% of the time. The scatterplot below compares VoxPopuli word error rate (WER) on the x-axis with the rate at which each model reproduces the benchmark's incorrect reference instead of the consensus correction. The models with the lowest WER—and therefore the strongest reported benchmark performance—are also the most likely to reproduce these errors.

    Masked Entity Retrieval

    To build on the consensus disagreement probe, we deliberately silence numbers in the audio samples of test datasets and ask the models to transcribe what it hears. The number is literally absent from the audio, so models should not output any number, much less the exact number in the text.

    Some of these numbers are semi-predictable (although still unlikely for a model to predict), yet others are quite surprising. The following clip combines both probes, showing both how models recreate reference transcript errors including an incorrect number and one model even autocompletes a relatively random year (2011) despite it being silenced. In each model's row below:

    • green highlighting with strikethrough marks reference-transcript words the model correctly did not reproduce (audio-faithful);
    • green highlighting with underline marks a correct, audio-faithful insertion in place of the reference's erroneous wording;
    • red highlighting (plain text) reproduces the reference transcript's erroneous, audio-unsupported content: keeping "Mr President", writing "more than 1 amendments" where the audio says "one thousand six hundred", supplying the silenced year "2011", or ending on "plenary".

    Recovery rates were highest on the public benchmarks and lower on held-out or newly collected audio (ep-fresh and libri-fresh below). On LibriSpeech, some of the strongest benchmark-performing models reproduced masked numbers in roughly 30–40% of examples, even though the number itself had been removed. The effect weakened on freshly collected data for several models, suggesting that the surrounding benchmark-associated audio—not only textual autocomplete—helped the models recover the reference.

    Orthographic Switching

    Our orthographic switching probe tests whether models reproduce the exact spelling used in a benchmark's reference transcript despite it not being clear in the audio. Orthographic variants are words that are semantically and phonetically identical but can be spelled different ways (1 vs one, Mr. vs mister, John vs Jon, Honor vs Honour, etc). In theory, models should consistently prefer one spelling over another, or alternate between them at roughly random rates. If models systematically switch to match what is in each benchmark's reference transcript, that suggests the models are picking up on which spelling the test expects.

    Transcription: "I URGED ON THE BOYS THAT WHATEVER HAPPENED WE SHOULD NOT SHOOT ANY ONE" — models using "any one": 6/11, models using "anyone": 5/11

    Transcription: "CAMOUFLAGE WAS NOT A WORD THE CAPTAIN OR ANYONE ELSE OF HIS TIME YET UNDERSTOOD" — models using "any one": 2/11, models using "anyone": 9/11

    Within LibriSpeech, we test one intra-dataset switch involving an older spacing convention: some reference transcripts use "any one", while others use "anyone." We measure the minimum accuracy for a given variant, which we call "switch rate". If a model only uses one variant it would have a 0% switch rate; a model which picks randomly would be expected to have a 50% switch rate. A model which knows which variant to use in every test sample would earn a 100% switch rate.

    Our second probe tests an inter-dataset switch, in which each benchmark uses a different spelling convention consistently across its test corpus. For example, VoxPopuli uses the abbreviation "Mr.", while LibriSpeech spells out "Mister."

    Multiple models exceed the 50% random-choice baseline, with some reaching roughly 90% switch accuracy. This suggests that the models can identify which dataset an audio sample comes from and select the spelling convention that benchmark expects, even though both forms sound identical.

    Localizing the switches

    To test whether these behaviors generalize beyond the public benchmarks, we also collected fresh data from the same source domains but after the models' training cutoffs: recent European Parliament recordings for VoxPopuli and recordings from newly active LibriVox narrators for LibriSpeech. However, when presented with recently collected data from the same domain, many models stop matching the reference transcript and revert to more audio faithful transcriptions.

    Other interventions point to the same conclusion. Phrases which are present in the audio but are omitted in the reference transcript can reappear when a model is asked to translate the audio or when its attention is restricted to the relevant frames. Trimming away surrounding benchmark context, or appending ordinary conversational audio, can also restore the faithful transcript. Appending VoxPopuli audio can have the opposite effect, making otherwise faithful synthetic or mined samples more likely to match the benchmark reference.

    Together, these results suggest that models are able to faithfully transcribe the literal spoken words, but are using surrounding acoustic context to decide whether to follow the audio or a benchmark-specific transcription policy.

    Conclusion

    Our findings suggest that, on two major open-source datasets, some models detect dataset-associated acoustic cues and adjust their transcription behavior accordingly. Specifically, models may reproduce words that are absent from the audio but present in the reference transcript, recover silenced numbers at elevated rates, or use surrounding acoustic context to select the written variant expected by a particular benchmark.

    For people selecting models, these findings underscore the importance of using fully held-out evaluation sets, as RW-Voice-EQ Bench and the Open ASR Leaderboard do, and of looking beyond word error rate on a single public benchmark. To this end, a "Benchmark fitting" tab has been added to the Open ASR Leaderboard, which includes two of the above analyses across all models: quantifying (1) reference error rates from VoxPopuli and (2) orthographic switching across all public datasets. The relevant scripts are open-sourced on GitHub as well as the un-normalized model outputs.

    Our findings also suggest that benchmark developers should avoid simple independent and identically distributed test splits in favor of temporal, speaker, or other metadata-based separation. Greater transparency around training data and model-selection procedures would also help researchers understand how these behaviors arise.

    Public benchmarks remain valuable: they are transparent, repeatable, easy to run, and well understood by the research community. But they are most useful when we can distinguish genuine transcription improvements from benchmark-specific gains that do not generalize to new audio.

    For more information, we encourage you to read our full report.

    Original source
  • May 2026
    • No date parsed from source.
    • First seen by Releasebot:
      May 16, 2026
    Hume logo

    Hume

    TTS API bug fixes

    Hume fixes a bug that caused duplicate interleaved TTS audio and distorted output.

    Fixed a bug where duplicate interleaved audio was included in TTS audio output.

    This resolves an issue where audio chunks could be duplicated and interleaved, resulting in distorted output.

    Original source
  • May 15, 2026
    • Date parsed from source:
      May 15, 2026
    • First seen by Releasebot:
      May 16, 2026
    • Modified by Releasebot:
      Oct 5, 2026
    Hume logo

    Hume

    May 15, 2026

    Hume adds an experimental temperature parameter to TTS endpoints for more varied or consistent speech generation.

    Added an experimental temperature parameter to TTS endpoints.

    Controls sampling temperature for speech generation.

    Higher values increase variation; lower values increase consistency.

    Original source
  • Apr 10, 2026
    • Date parsed from source:
      Apr 10, 2026
    • First seen by Releasebot:
      Apr 11, 2026
    Hume logo

    Hume

    April 10, 2026

    Hume adds configurable turn detection and interruption settings to EVI configs, giving users finer control over turn-taking, speech detection, and interruption behavior on a per-config basis.

    EVI API additions

    Added configurable turn detection and interruption settings to EVI configs. You can now control how EVI handles turn-taking and interruptions on a per-config basis.

    • turn_detection.end_of_turn_silence_ms: How long EVI waits after speech ends before committing a turn (500-3000ms, default 800ms).
    • turn_detection.speech_detection_threshold: Sensitivity of voice activity detection (0.0-1.0, default 0.5).
    • turn_detection.prefix_padding_ms: Audio padding before detected speech (default 300ms).
    • interruption.min_interruption_ms: Minimum speech duration before EVI can be interrupted (50-2000ms, default 800ms).
    Original source
Releasebot

Curated by the Releasebot team

Releasebot is an aggregator of official release notes from hundreds of software vendors and thousands of sources.

Our editorial process involves the manual review and audit of release notes procured with the help of automated systems.