New Models

Mistral Large 4 (Le Chonk)

Mistral Large 4 is Mistral's largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.

Key Features

Mistral Large 4 includes the following architectural choices:

• ~1T total / ~52B active parameters: granular Mixture-of-Experts with 128 routed experts (4 active per token) and 1 shared expert.
• 1M context length, for long-context workloads.
• Multimodal input: accepts text and image input, with text output. Pixtral-style vision encoder trained to handle variable image sizes and aspect ratios.
• Instruct and Reasoning functionalities with function calls, reasoning effort configurable per request.

Mistral Large 4 offers the following capabilities:

• Reasoning Mode: three reasoning efforts — low, medium, high (default) — so the same model can answer a quick chat reply or work through a complex agentic run.
• Vision: analyzes images, charts, technical drawings, and documents, with state-of-the-art visual grounding among open weights.
• Multilingual: natively fluent in 160+ languages, including all official languages of the European Union.
• System Prompt: strong adherence and support for system prompts.
• Agentic: native function calling, JSON output, and strict JSON schema enforcement.
• Cybersecurity: state of the art among open-weight models for security work, while refusing malicious requests.
• Large Context Window: supports a 1M context window.

The model is released under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.

Recommended Settings

Reasoning Effort:

'low' / 'medium' / 'high' — default is 'high'.

Use reasoning_effort="high" for complex prompts and agentic usage, 'low' for fast chat replies and 'medium' for day to day usage.

Sampling: for all reasoning efforts, we recommend the following sampling parameters:

• temperature=1.0
• top_p=0.95

System Prompt: define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.

Tools: keep the set of tools well-defined and limit their number to the minimum required for the use case.

Context: the checkpoint supports a 1M context window.

Breaking changes

• 🚨 Separate EP logic into its own file (#49679) by @3outeille

Quantization

FP8 quantized models can now be used for adapter training (such as LoRA with PEFT). Backward passes were added to the dense and grouped FP8 matmuls, and is_trainable is now True. Backward also dequantizes experts in chunks and skips autograd at inference, which cuts peak memory in the MoE layer backward from about 28.9 GiB to 2.8 GiB at 8k tokens. Separately, the AMD CI image now install

• FP8 backward: chunked expert dequant, skip autograd.Function at inference (#50117) by @ArthurZucker in [#50117]
• Train adapters on top of FP8 weights (#50031) by @qgallouedec in [#50031]
• [AMD CI] Install onnxscript, bitsandbytes and peft in the AMD image (#50046) by @Abdennacer-Badaoui in [#50046]

Bugfixes and improvements

• Processor unnecessary warnings (#49812) by @zucchini-nlp in [#49812]
• Fix boolean padding in ClapFeatureExtractor (#23648) (#49960) by @rudy-07 in [#49960]
• [Fix] Count cropping of an uninitialized layer as a no-op (#50104) by @remi-or in [#50104]
• [loading] Allow PermuteForRope to act on single tensors while renaming (#50092) by @Cyrilvallez in [#50092]
• Add Nemotron 3.5 Transcribe (#50084) by @eustlb in [#50084]
• [Nemotron3Diarization] add left context support (#49283) by @eustlb in [#49283]
• [Janus] Fix processor iterating a string prompt character by character (#50044) by @tarekziade in [#50044]
• fix(config): forward token kwarg to get_config_dict in PreTrainedConfig.from_pretrained (#50081) by @ydshieh in [#50081]
• Fix rope_scaling setter for configs that assign it during init (#50080) by @hmellor in [#50080]
• [timesfm] Fix fp16 overflow on pad_val in _forward_transform (#50021) by @tarekziade in [#50021]
• Update dev version (#50068) by @albertvillanova in [#50068]
• Pin the shared Hub MIC fleet dataset (#49802) by @tarekziade in [#49802]
• [docs] NVIDIA/Intel/CPU (#49650) by @stevhliu in [#49650]
• Document how to handle non-persistent buffer initialization in custom models (#49880) by @Tialo in [#49880]
• Remove legacy benchmark/ and disable automatic benchmark runs (#50040) by @tarekziade in [#50040]
• Remove deprecation cycle for candidate generators (#50034) by @Cyrilvallez in [#50034]
• Fix test pil import leak (#49915) by @guarin in [#49915]
• fix: preserve NumPy integer label dtypes in default data collators (#49900) by @BlazeSR10hawk in [#49900]
• [ESMFold2] Don't build the MSA encoder for checkpoints without one (#49890) by @jimburtoft in [#49890]
• [minimax_m3_vl] Gate the integration tests on total accelerator memory (#50025) by @tarekziade in [#50025]
• Fix: Add kwargs in LFM2.5 conv layers (#50022) by @younesbelkada in [#50022]
• Forward code_revision to every pipeline loader (#49832) by @Rocketknight1 in [#49832]
• [moshi] Skip Moshiko integration tests whose checkpoint is gone (#50017) by @tarekziade in [#50017]
• Keep a capturing group's escapes when reversing a conversion pattern (#49772) by @IlyasMoutawwakil in [#49772]
• perf(ci): skip the lazy blob fetch in git blame on a partial clone (#50007) by @nrenvo in [#50007]
• fix(canary): set layerdrop=0.0 in test tester to prevent flaky test (#50012) by @ydshieh in [#50012]
• add a WA to Dockerfile for XPU (#50008) by @kaixuanliu in [#50008]
• pipelines: defer eager model imports in text_to_audio and token_classification (#50001) by @ydshieh in [#50001]
• [exporters] resolve @register_patch paths lazily to avoid eager model imports (#50000) by @ydshieh in [#50000]
• [Fix] Remove useless kwargs (#49966) by @remi-or in [#49966]
• Fix model's config attr naming (#49898) by @zucchini-nlp in [#49898]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

• @tarekziade
  • Fix _init_weights gaps in 7 audio models and enable test_can_init_all_missing_weights for them (#49386)
  • Fix image backend equivalence tests for Pillow 13 (#50105)
  • [Janus] Fix processor iterating a string prompt character by character (#50044)
  • [timesfm] Fix fp16 overflow on pad_val in _forward_transform (#50021)
  • Pin the shared Hub MIC fleet dataset (#49802)
  • Remove legacy benchmark/ and disable automatic benchmark runs (#50040)
  • [minimax_m3_vl] Gate the integration tests on total accelerator memory (#50025)
  • [moshi] Skip Moshiko integration tests whose checkpoint is gone (#50017)
• @eustlb
  • Add Nemotron 3.5 Transcribe (#50084)
  • [Nemotron3Diarization] add left context support (#49283)

Original source

Release v5.21.0
New Model additions
MiniCPM-OCR
MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family. It keeps the MiniCPM-V 4.6 architecture, which pairs a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts and tables.
Links: Documentation | Paper
Feat test (#49920) by @guarin in #49920
Flint
Flint is Recursive's mixture-of-experts language model. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, it uses hybrid attention (sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers). It also has learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, and the first two layers are dense.
Links: Documentation
Add Flint model support. (#49824) by @lovishmadaan in #49824
UltraVBert
UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view. The connector merges each 4 x 4 block of vision patches into one token, so a tile becomes 64 image tokens, and RMS-normalizes them to the scale of UltraBERT's token embeddings. The max_image_tiles setting (27 by default) caps the number of tiles per page.
Links: Documentation
Add UltraVBERT (#49685) by @athrael-soju in #49685
Gradium Phonon TTS
Gradium Phonon TTS is Gradium's family of on-device text-to-speech models, trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and the text, then emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends. The latents are decoded to 24 kHz audio by a Pocket TTS audio tokenizer. Unlike Pocket TTS, voice conditioning uses 512-dimensional latents from a separate, unreleased speaker codec, so voices come as precomputed latents and cannot be cloned from a recording.
Links: Documentation
Add GradiumPhononTTS (#49791) by @eustlb in #49791
Pocket TTS
Pocket TTS is Kyutai's 100M-parameter text-to-speech model, built as a Continuous Audio Language Model that runs faster than real time on CPU. A causal transformer emits one continuous audio latent per frame (12.5 Hz) through a one-step flow head, and an end-of-speech head decides when to stop. A Mimi-based audio tokenizer encodes voice prompts and decodes the generated latents to 24 kHz audio. Two flow heads are released, Lagrangian self-distillation (default) and drifting, and voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states.
Links: Documentation | [Paper](https://huggingface.co/papers/2509.06926 (the Continuous Audio Language Models paper, arXiv 2509.06926). It is the only paper the documentation links, and it describes the architecture family Pocket TTS belongs to. The docs do not cite a separate Pocket TTS technical report.) | [Blog Post](NONE. The documentation links no official blog post. It points only to the original code repository at https://github.com/kyutai-labs/pocket-tts.)
add PocketTTS (#49589) by @eustlb in #49589
Pocket TTS Audio Tokenizer
The Pocket TTS audio tokenizer is the codec used by Pocket TTS. It has Mimi's architecture (SEANet convolutions plus a small causal transformer on each side), but its residual vector quantizer is replaced by a continuous 32-dimensional bottleneck, the VAE-GAN from Continuous Audio Language Models. It converts 24 kHz audio into latents at 12.5 Hz and decodes them back to audio. It serves as the audio_tokenizer of PocketTTSProcessor, encoding voice prompts and decoding generated latents, and its encoder and decoder can also be loaded on their own.
Links: Documentation | Paper
add PocketTTSAudioTokenizer (#49588) by @eustlb in #49588
Breaking changes
Fixed Gemma4 processor imports (the Gemma4, Gemma4 unified and EmbeddingGemma2 processors) so from transformers import * no longer fails in environments that have PIL but not torch; no user action is needed.
🚨 Fix gemma4 cross imports resulting in import failing without torch but with PIL (#49748) by @Sai-Suraj-27
Added an "fp8" backend to QuantizedCache, alongside quanto and hqq, that stores KV states in torch.float8_e4m3fn with a single scale per tensor; enable it with cache_implementation="quantized" and cache_config={"backend": "fp8"}.
🚨Add an FP8 backend to QuantizedCache (#48677) by @jiqing-feng
Tensor parallelism now has a dedicated embedding style, shards every model's input embeddings by default, and supports widths that aren't divisible by the TP size, so ColwiseParallel and RowwiseParallel now apply only to linear layers; custom TP plans that applied them to embeddings must switch to the new embedding style.
🚨 TP: give embeddings their own style, shard input embeddings by default, and shard widths that don't divide by the TP size (#49570) by @IlyasMoutawwakil
Every MoE TP and EP plan now shards its experts, so models no longer keep all experts on every rank, and a new test_moe_parallel_plans_shard_experts test enforces this; custom or downstream MoE parallel plans must add expert sharding to pass it and to avoid wasted memory.
🚨 MoE: shard the experts in every TP/EP plan, and test for it (#49206) by @IlyasMoutawwakil
Cache
This release improves cache handling across the library. Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, chunked prefill supports cache continuation, and the Qwen4-Exp indexer's position history stays aligned with its cached keys. Linear-attention caches also gain new getters (is_recording_past, get_recurrent_state, conv-state access) that stop exposing cache internals and prepare for paged-c
[FA] Fix FA with static cache (#49998) by @Cyrilvallez in [#49998]
[docs] CB cache behavior (#49025) by @stevhliu in [#49025]
[CB] Reduce warmup time by not computing attention over full cache (#49853) by @remi-or in [#49853]
Fix Qwen4-Exp indexer position cache lifecycle (#49840) by @taking-lying-flat in [#49840]
[Cache] Linear attn cache: use new getters (#49375) by @remi-or in [#49375]
[Cache] Get conv states (#49309) by @remi-or in [#49309]
[Cache] Add get_recurrent_state (#49306) by @remi-or in [#49306]
[Cache] Add is_recording_past (#49305) by @remi-or in [#49305]
Fix BLOOM crash with static cache / compiled generate (#49481) by @kaixuanliu in [#49481]
Allow Cache continuation for chunked prefill (#49662) by @Cyrilvallez in [#49662]
Vision
Vision image processors now composite PNG transparency onto white instead of exposing black pixels, and background_color is preserved correctly across the DeepSeek-VL and Janus processors. Model fixes cover SmolVLM's inputs_merger when image tokens aren't a multiple of the patch size, HunYuan-VL image assignment during batched beam search, and Ernie4.5-VL-MoE expert sharding under TP/EP. The DETR
Composite transparency in shared RGB conversion and image processors (#49079) by @Nikhi00718 in [#49079]
[Fetcher] Fix missing test deps for dynamically-resolved image processors (#49826) by @ydshieh in [#49826]
fix(vision): Route the DETR conversion scripts through from_pretrained (#49830) by @harshaljanjani in [#49830]
Fix background_color handling across DeepSeek-VL and Janus image processors (#49767) by @tritsystem in [#49767]
Move the MI300 docker image to ROCm 10.1 and update test expectations (#49460) by @Abdennacer-Badaoui in [#49460]
docs: fix 'allows to' grammar in Vision Encoder Decoder docs (#48730) by @mrchatam in [#48730]
Fix SSH runner: add /opt/venv/bin to PATH for Python 3.11 images (#49759) by @ydshieh in [#49759]
Fix SmolVLM inputs_merger when image tokens are not a multiple of patch_size (#49725) by @albertvillanova in [#49725]
Ernie4.5-VL-MoE: shard the text and vision experts under TP and EP (#49462) by @IlyasMoutawwakil in [#49462]
Fix HunYuan-VL image assignment during batched generation (#49557) by @Arnav66692 in [#49557]
Generation
This release fixes a range of generation bugs: BLOOM crashes with static cache or compiled generate, generate dropping the type of dict-like inputs such as ModelOutput, and incorrect image assignment or placeholder handling in batched or beam-search generation for HunYuan-VL and Idefics2/3. It also fixes Qwen3-Omni audio generation mode handling, SAM mask-generation multi-zoom crops, empty frame s
fix: prevent empty frame sampling for short videos (#49908) by @BlazeSR10hawk in [#49908]
Fix Windows path separators breaking relative imports of Hub custom-generate modules (#48951) by @dltsum in [#48951]
Fix multi-zoom crops in the SAM mask-generation pipeline and related bugs (Fixes #35530) (#49689) by @stlukyanenko in [#49689]
update speculative decoding docs (#49697) by @merveenoyan in [#49697]
fix idefics3 generation issues (#49738) by @molbap in [#49738]
fix tests for generation (#49740) by @molbap in [#49740]
Fix inputs dict casting in generate (#49731) by @Cyrilvallez in [#49731]
[fix][Qwen3-Omni] Respect generation_mode in audio generation (#49505) by @Sunt-ing in [#49505]
Parallelization
This release extends parallelism support and fixes several related bugs. It adds a tensor-parallel plan for Nemotron and makes Ernie4.5-VL-MoE shard its text and vision experts under TP and EP. It also restores expert-parallel token-dispatch gradients on torch 2.7–2.10, fixes a Molmo2 model-parallel device mismatch, and corrects base_model_prefix in several model families (PP-OCR detectors
Add tensor parallel plan for Nemotron (#48780) by @Siddhant231xyz in [#48780]
fix model parallel bug for molmo2 model (#49816) by @kaixuanliu in [#49816]
Fix base_model_prefix in the PP-OCR detectors, CLIPSeg, TimesFM, CLIP and VideoPrism heads so base checkpoints load (#48871) by @blipbyte in [#48871]
Fix expert-parallel token dispatch gradients on torch 2.7 to 2.10 (#49710) by @digit50 in [#49710]
Skip Muse TP (#49717) by @remi-or in [#49717]
Quantization
Quantization now handles DeepSeek-V4.1's FP4 experts correctly: FineGrainedFP8Config gains an expert_dtype option ("fp8" by default, or "fp4") that the FP8 experts read instead of allocating them as FP8. TimesFM 2.5 no longer casts float inputs to an integer weight dtype when loaded with quantization such as bitsandbytes 4/8-bit. A broader cleanup
Fine-grained FP8: DeepSeek-V4's expert_dtype in the quantization config (#49848) by @IlyasMoutawwakil in [#49848]
Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights (#46942) by @JOhnsonKC201 in [#46942]
Quantization cleanpup (#49548) by @ArthurZucker in [#49548]
Bugfixes and improvements
[Fix] Fix CSM codebook embedding tying (#49197) by @orrangetabby17 in [#49197]
[tests_fetcher] fix missing .as_posix() in preprocessor_patterns path keys (#49994) by @ydshieh in [#49994]
fix(ci): skip bot suggestion if a maintainer already posted run-slow: (#49992) by @ydshieh in [#49992]
[loading] Allow Renaming and Converter to work on the sume subkeys (#49928) by @Cyrilvallez in [#49928]
[Fix] Fix NUMA and cpuset detection in CB (#49980) by @remi-or in [#49980]
[Refactor] Make CB utils depend on no CB import (#49968) by @remi-or in [#49968]
chore: improve typing in modeling_rope_utils.py (#49831) by @RyanMullins in [#49831]
is_fouroversix_available() returned an (available, version) tuple. Ev… (#49894) by @sywangyi in [#49894]
Match a skip-list entry from the start only up to a module boundary (#49771) by @IlyasMoutawwakil in [#49771]
[pocket_tts, gradium_phonon_tts] use sub_configs_defaults (#49868) by @eustlb in [#49868]
Fixes a dtype mismatch in flash_attn test (#49912) by @kaixuanliu in [#49912]
[Fix] Remove duplicate assignment in DeepseekV3.2 config (#49896) by @remi-or in [#49896]
add XPU Expectations for 4 new model tests (#49815) by @kaixuanliu in [#49815]
Make sub-config-defaults easier (#49906) by @zucchini-nlp in [#49906]
Fix out-of-bounds index bug in musicgen_melody model (#49842) by @kaixuanliu in [#49842]
test: add parity check to the CPU-only continuous batching test (#49728) by @nrenvo in [#49728]
[CB] Reduce the max_memory_percent in case of a CPU model (#49844) by @remi-or in [#49844]
Improve clarity and grammar in Auto Classes documentation (#44550) by @himani27301 in [#44550]
[gradium_phonon_tts] point to the released Hub repos (#49867) by @eustlb in [#49867]
[EmbeddingGemma2] update integration tests (#49576) by @eustlb in [#49576]
Fix a comment (#44872) by @IvanFan-Van in [#44872]
Fix hardcoded torch.float64 crashing on MPS (#46723) (#46730) by @Abhijeet1815 in [#46730]
fixup metadata workflow (#49854) by @vasqu in [#49854]
docs(whisper): cast input features to model dtype in AutoModel example (#49837) by @Kayvan-Zahiri in [#49837]
Fix torch_float casting to int instead of float (#47468) by @rahulrshetty45 in [#47468]
Upgrade Python Min version to 3.11 (#49251) by @Sai-Suraj-27 in [#49251]
skip sdpa_can_dispatch_on_flash tests for 3 models (#49809) by @kaixuanliu in [#49809]
[LED][Longformer] Replace for-loop with unfold in _chunk ONNX-export path (#46169) by @guinik in [#46169]
logits_to_keep bool mask for all ForXXX classes (#48825) by @jp1924 in [#48825]
Fix several object detection models/loader fix for module tying (#49827) by @molbap in [#49827]
Fix encoder logits processors when precomputed encoder_outputs lack token IDs (#49696) by @kunalsanga in [#49696]
Defer some imports (#49663) by @jayzuccarelli in [#49663]
refactor CTC loss (#49833) by @ebezzam in [#49833]
Add Intel XPU CI (#49254) by @regisss in [#49254]
fix(mask2former): register input_projections as nn.ModuleList (fixes #49698) (#49703) by @Omarinho12345 in [#49703]
Only tie MM Grounding DINO decoder heads when the config shares them (#49552) by @ in [#49552]
[Docs] Add Marathi (मराठी) translation of README (#48746) by @shreeyadav06 in [#48746]
Fix CLVP c_proj residual scaled-init silently never applied (#47478) by @ShamSaleem in [#47478]
fix failed test cases for unlimited_ocr model (#49806) by @kaixuanliu in [#49806]
Fix transformers env when requests is not installed (#48412) by @panshularora in [#48412]
Apply EXIF orientation in load_image_as_tensor with torchvision (#49794) by @digit50 in [#49794]
Fix Fuyu processor test (#49821) by @zucchini-nlp in [#49821]
use torchcodec-xpu and upgrade torch to 2.14.1 (#49814) by @sywangyi in [#49814]
update some case to add xpu expectation (#49807) by @sywangyi in [#49807]
[serge] Fix 2 integration tests regressed by commit c48824e (PR #49404) (#49705) by @sergereview[bot] in [#49705]
docs: fix OOM/hang for Qwen dynamic resolution models by setting min_pixels and max_pixels (#46007) by @poojansatani in [#46007]
docs: fix 'allows to' grammar in DPT docs (#48728) by @mrchatam in [#48728]
docs: fix 'allows to' grammar in LiLT docs (#48731) by @mrchatam in [#48731]
docs: fix prompt formatting and code alignment in Gemma model doc (#48775) by @Rattan344 in [#48775]
docs: fix grammar typo in ALBERT model card (#48774) by @Rattan344 in [#48774]
Deprecate old input names (#49424) by @zucchini-nlp in [#49424]
Fix NameError in CI comment workflows: missing section marker variables (#49801) by @ydshieh in [#49801]
[CI] Improve run-slow PR comments and remove CI recap (#49782) by @ydshieh in [#49782]
Show the actual accelerate version for transformers env (#49668) by @Sai-Suraj-27 in [#49668]
[CI] fix perceriver old regression (#49792) by @molbap in [#49792]
mimi moshi output capturing (#49670) by @eustlb in [#49670]
Remove the resolved prototype Whisper MIC expected failure (#49785) by @tarekziade in [#49785]
Fix CohereCompass text RoPE to use Qwen3-VL interleaved M-RoPE (#49715) by @mmangkad in [#49715]
Alias thinking -> reasoning_content (#49535) by @Rocketknight1 in [#49535]
[vLLM] Add Qwen-Omni Thinker base models and Gemma 3n per-layer inputs (#49437) by @harshaljanjani in [#49437]
Update MIC fleet references and documentation (#49768) by @tarekziade in [#49768]
[CI] no split modules for SAM (#49766) by @molbap in [#49766]
Enable tests forgotten in idefics3 (#49758) by @molbap in [#49758]
Fix tests_fetcher backbone step (never worked since introduction) (#49742) by @ydshieh in [#49742]
[modular] Extend the "del self.xxx" pattern to match typed assignments (#49735) by @remi-or in [#49735]
Xpu add (#49757) by @sywangyi in [#49757]
Deprecate rope_scaling assignment and normalize legacy RoPE keys (#49515) by @jinwukong in [#49515]
Make longrope compileable (#49555) by @jiqing-feng in [#49555]
Fix stack_frames > 1 wiring bug in MiniCPMV4_6VideoProcessor (#49752) by @airlsyn in [#49752]
fix moshi failing tests (#49750) by @eustlb in [#49750]
[docs] Gaudi (#49618) by @stevhliu in [#49618]
[serge] Fix 2 integration tests for model convnextv2 failing with output_mismatch (tensor values differ (2)) (#48838) by @sergereview[bot] in [#48838]
[serge] Fix 2 integration tests for model eomt_dinov3 failing with output_mismatch (tensor values differ (2)) (#48882) by @sergereview[bot] in [#48882]
[serge] Fix 4 integration tests for model lw_detr failing with output_mismatch (tensor values differ (4)) (#49040) by @sergereview[bot] in [#49040]
[serge] Fix 2 integration tests for model nemotron failing with output_mismatch (list output differs (2)) (#48945) by @sergereview[bot] in [#48945]
multi device fix for molmo2 (#49736) by @molbap in [#49736]
Fix the rest of VLMs (#49720) by @zucchini-nlp in [#49720]
Fix ValueError when loading a fusion-enabled checkpoint twice (#49412) by @kaixuanliu in [#49412]
Avoid a host-to-device copy in eager_mask (#49684) by @hmellor in [#49684]
is_tracing() now recognizes XPU stream capture, alongside the exist… (#49722) by @sywangyi in [#49722]
Document how to avoid the Metaspace prefix after added tokens (#45822) by @qflen in [#45822]
Declare the input modalities of the VLMs that report text only (#49677) by @qgallouedec in [#49677]
Fix zero expert gradients in bfloat16 with expert parallelism and ep_size < fsdp_size (#49688) by @qgallouedec in [#49688]
Fix test selection for virtual module imports (#49690) by @tarekziade in [#49690]
[docs] add Google TPU hardware guide (#49642) by @tengomucho in [#49642]
Copy default response parser args (#49370) by @Rocketknight1 in [#49370]
Make DtensorShardOperation work on torch 2.6 and 2.7 (#49657) by @hazelxue in [#49657]
Fix from_pretrained on torch 2.7: require torch>=2.8 for distributed checkpointing (#49652) by @naveecode in [#49652]
Fix DeepGEMM Mega MoE under expert parallelism (#49428) by @IlyasMoutawwakil in [#49428]
Fix flaky Evolla tests producing NaN outputs (#49660) by @tarekziade in [#49660]
[rocm] Add ROCm expectations to Kolibri-1 logits test (#49661) by @ericschreiber in [#49661]
[CB] Various small fixes for MPS (#49654) by @remi-or in [#49654]
Aligns test expectations with the accelerator path under test: (#49653) by @sywangyi in [#49653]
[docs] Processor config (#49643) by @stevhliu in [#49643]
post release stuff (#49646) by @vasqu in [#49646]
Significant community contributions
The following contributors have made significant changes to the library over the last release:
@ydshieh
[tests_fetcher] fix missing .as_posix() in preprocessor_patterns path keys (#49994)
fix(ci): skip bot suggestion if a maintainer already posted run-slow: (#49992)
[Fetcher] Fix missing test deps for dynamically-resolved image processors (#49826)
Fix NameError in CI comment workflows: missing section marker variables (#49801)
[CI] Improve run-slow PR comments and remove CI recap (#49782)
Fix tests_fetcher backbone step (never worked since introduction) (#49742)
Fix SSH runner: add /opt/venv/bin to PATH for Python 3.11 images (#49759)
@remi-or
[Fix] Fix NUMA and cpuset detection in CB (#49980)
[Refactor] Make CB utils depend on no CB import (#49968)
[Fix] Remove duplicate assignment in DeepseekV3.2 config (#49896)
[CB] Reduce warmup time by not computing attention over full cache (#49853)
[CB] Reduce the max_memory_percent in case of a CPU model (#49844)
[modular] Extend the "del self.xxx" pattern to match typed assignments (#49735)
[Cache] Linear attn cache: use new getters (#49375)
[Cache] Get conv states (#49309)
[Cache] Add get_recurrent_state (#49306)
[Cache] Add is_recording_past (#49305)
[Kernels] Add MPS to supported device types (#49655)
Skip Muse TP (#49717)
[CB] Various small fixes for MPS (#49654)
@Nikhi00718
Composite transparency in shared RGB conversion and image processors (#49079)
@eustlb
[pocket_tts, gradium_phonon_tts] use sub_configs_defaults (#49868)
[gradium_phonon_tts] point to the released Hub repos (#49867)
[EmbeddingGemma2] update integration tests (#49576)
Add GradiumPhononTTS (#49791)
add PocketTTS (#49589)
add PocketTTSAudioTokenizer (#49588)
🟡 [mimi] separate encoder and decoder (#49577)
mimi moshi output capturing (#49670)
fix moshi failing tests (#49750)
@jp1924
logits_to_keep bool mask for all ForXXX classes (#48825)
@molbap
Fix several object detection models/loader fix for module tying (#49827)
[CI] fix perceriver old regression (#49792)
[CI] no split modules for SAM (#49766)
Enable tests forgotten in idefics3 (#49758)
fix idefics3 generation issues (#49738)
fix tests for generation (#49740)
multi device fix for molmo2 (#49736)
@athrael-soju
Add UltraVBERT (#49685)
@harshaljanjani
fix(vision): Route the DETR conversion scripts through from_pretrained (#49830)
fix(audio): Route the ALM conversion scripts through from_pretrained (#48525)
[vLLM] Add Qwen-Omni Thinker base models and Gemma 3n per-layer inputs (#49437)
@lovishmadaan
Add Flint model support. (#49824)
@tarekziade
Remove the resolved prototype Whisper MIC expected failure (#49785)
Update MIC fleet references and documentation (#49768)
Cleaned up tokenizer tests (#49669)
Fix test selection for virtual module imports (#49690)
Fix flaky Evolla tests producing NaN outputs (#49660)
@yonigozlan
Fix small bug in auto mappings (#49664)
@ArthurZucker
Quantization cleanpup (#49548)

Original source

Patch release: v5.20.2

This patch notably solves an issue where some inputs could loose their original type inside prepare_inputs_for_generation in generate, leading to problems with multimodal models. It also fixes some other issues listed on the below commits.

It contains the following commits:

• Fix inputs dict casting in generate (#49731) by @Cyrilvallez
• Fix zero expert gradients in bfloat16 with expert parallelism and ep_size < fsdp_size (#49688) by @qgallouedec
• Declare the input modalities of the VLMs that report text only (#49677) by @qgallouedec
• Avoid a host-to-device copy in eager_mask (#49684) by @hmellor

Original source

Release v5.20.1

Special surprise including the new model Beam from Reflection AI 🥳

Beam is a token-efficient, frontier-scale model focused on coding and agentic use cases. It is built for teams that want to run a powerful model on their own hardware, infrastructure, or cloud, and customize it to their specific needs.

Links: Documentation

Fix small bug in auto mappings (#49664) by @yonigozlan in #49664

Some real patch fixes

We broke loading models on older torch versions 😢 should be fixed now. But here is the full list of fixes

• Fix HunYuan-VL image assignment during batched generation (#49557) by @Arnav66692
• Fix DeepGEMM Mega MoE under expert parallelism(#49428) by @IlyasMoutawwakil
• Fix from_pretrained on torch 2.7: require torch>=2.8 for distributed checkpointing (#49652) by @naveecode
• Make DtensorShardOperation work on torch 2.6 and 2.7 (#49657) by @hazelxue

Full Changelog: v5.20.0...v5.20.1

Original source

Release v5.20.0

New Model additions

Muse Spark Open

Muse Spark Open is a multimodal mixture-of-experts model from Meta that takes text, images, audio, and video as input and generates text. It has 816B total parameters (42B active per token) and a context window of up to 1,048,576 tokens shared across modalities. It is trained for agentic coding tasks such as multi-file editing, debugging, and tool use. Checkpoints are available in BF16 and NVFP4 formats.

Links: Documentation | Blog Post

Add Muse Spark Open (#49644) by @guarin in #49644

Ling 3.0

Ling 3.0 Flash is a hybrid Mixture-of-Experts language model from InclusionAI. Its decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer. The MLA blocks use interleaved rotary embeddings and a head-wise output gate, and the feed-forward blocks combine routed and shared experts. It is natively supported in Transformers as BailingHybridForCausalLM, so it loads without trust_remote_code=True.

Links: Documentation

[Model] Add native Ling 3.0 Flash support (#49446) by @icavan in #49446

Kolibri1

Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model, focused on German and English. Each of its 50 layers routes every token to 6 of 384 experts using sigmoid routing, and adds one shared expert. Four sliding-window attention layers alternate with one full-attention layer, and RoPE is applied only in the sliding-window layers.

Links: Documentation

Add Kolibri 1 (#49463) by @lucamotz in #49463

MiniCPM-V 4.7

MiniCPM-V is a series of efficient multimodal large language models developed by OpenBMB. Like MiniCPM-V 4.6, the MiniCPM-V 4.7 architecture pairs a SigLIP vision encoder that has a window-attention merger with a Qwen3.5 language model backbone, and supports both 4x and 16x visual downsampling modes.

The main addition over 4.6 is canvas M-RoPE: instead of numbering visual tokens along a single 1-D sequence, the model lays every image out on a 2-D canvas and assigns each visual token a (temporal, height, width) position, so slices of the same image keep their spatial relationship and video frames keep their temporal order.

Fix more stuff (#48979) by @zucchini-nlp in #48979

UnlimitedOcr

UnlimitedOcr is a single 3B-parameter OCR model from Baidu with a standard 32,768-token context length. It builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention (R-SWA). This keeps the KV cache constant during decoding, so the model can transcribe dozens of pages in a single forward pass. It supports "gundam" mode (640x640 tiles with dynamic cropping) and "base" mode (a single 1024x1024 view), and it can output region detections with bounding boxes.

Links: Documentation | Paper

Add Unlimited OCR (#46836) by @guarin in #46836

UltraBERT

UltraBERT is a family of bidirectional text encoders trained with a combination of masking ratios, pruning, and distillation for classification and retrieval tasks. The released encoders support sequences of up to 32,768 tokens. They use rotary position embeddings, RMSNorm, SwiGLU feed-forward layers, and alternating local and global attention (every third layer is global). Checkpoints include UltraBERT-150M and UltraBERT-380M for masked language modeling and fine-tuning, and UltraBERT-Embed-150M for text embeddings.

Links: Documentation | Paper

Integrate the UltraBERT architecture (#49596) by @tomaarsen in #49596

MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, speaker diarization, and timestamps. It combines a Whisper-style encoder, a 4× frame merge step, a multi-modal projector, and a Qwen3 language model. It produces joint transcription and diarization in a single pass, with segments formatted as [start][S01]text[end]. It supports long-form audio through Whisper-window chunking in the processor, promptable transcription with hotword biasing, and 50+ languages.

Links: Documentation

Add MOSS-Transcribe-Diarize model (#48386) by @itazap in #48386

Breaking changes

🚨 General breaking notice that inference with EP has regressions in speed; we are working on fixing this ASAP! 🚨

Deprecated checks and methods have been removed from cache_utils.py (affecting models such as minimax_m3_vl and pi0), so any code still calling those deprecated cache methods must switch to the current cache API.

🚨 Remove deprecated cache utils methods (#49605) by @guarin

Accidental imports from other models were removed from several model files (e.g. embedding_gemma2, ijepa, lightglue), and the modular converter now avoids adding such cross-model imports. If you imported these names through those model modules, import them from their original modules instead.

🚨 Remove imports from other models (#49473) by @guarin

The minimum kernels version is now 0.17.2, which allows use_fallback=KernelizeFallback.ALL so that a failed kernel load falls back to the original implementation instead of raising an error. Upgrade kernels to 0.17.2 or later.

🚨 Bump min kernels to 0.17.2 for better fallback (#49235) by @SunMarc

A new SubConfigSpec centralizes the initialization of sub-configs, so default sub-configs are collected from dataclass fields instead of by instantiating the class, and this touches the configuration utilities and many model configs. Custom or existing nested configs may need to be updated to declare their sub-configs through this mechanism.

🚨 SubConfigSpec for centralized init of subconfigs (#48282) by @zucchini-nlp

The LUKE and mLUKE tokenizers now treat entity special tokens such as <ent> as special tokens (not split, honored by skip_special_tokens, normalized) while keeping the same spacing as regular vocabulary tokens when decoding. Tokenization and decoding output involving these tokens may therefore differ from before, so check any code or saved outputs that depend on the old behavior.

🚨🚨🚨 luke fixe (#48408) by @itazap

Exporters

Transformers exporters now support quantization directly during export across Dynamo, ONNX, OpenVINO, and ExecuTorch. The new unified quantization API integrates PT2E, ONNX Runtime, and NNCF quantizers, enabling static and dynamic quantization as well as weight-only compression (INT8/INT4), with configurable calibration workflows. This makes it easier to export optimized models for efficient inference across different backends.

For more information see the docs: https://huggingface.co/docs/transformers/exporters_quantization

Parallelization

This release improves distributed training and inference. FSDP × TP model and optimizer checkpoints can now be saved and loaded across different parallel configurations, and expert-parallel dispatch, tensor-parallel plans, static-cache generate(), and the caching allocator's byte count were fixed. Tied LM heads in composite models now get their own TP plan entry, enable_expert_parallel applies the EP plan alone again, TPU loading uses the `tor

• FSDP x TP Saving / Loading, Optimizer (#49256) by @michaelbenayoun in [#49256]
• FSDP x TP Saving / Loading (#48802) by @michaelbenayoun in [#48802]
• Don't set a device per rank for gloo tensor-parallel tests (#49636) by @IlyasMoutawwakil in [#49636]
• fix(distributed): keep enable_expert_parallel as EP alone (#49633) by @dacorvo in [#49633]
• Fix a port allocation race in FSDP tests (#49455) by @icavan in [#49455]
• Expert-parallel dispatch hands the experts the received rows, not the all-to-all's pending result (#49421) by @IlyasMoutawwakil in [#49421]
• Fix static-cache generate() with host-side inputs and under TP (#49317) by @dacorvo in [#49317]
• Composite lm head tp plan (#49080) by @IlyasMoutawwakil in [#49080]
• Fix: use torch.accelerator device APIs so TP loading works again on TPU (#49502) by @tengomucho in [#49502]
• Count ep and fsdp in caching_allocator_warmup byte calculation (#49384) by @3outeille in [#49384]
• Do not wrap the model in FSDP for expert-parallel token dispatch (#49383) by @3outeille in [#49383]

Generation

This release fixes static-cache generate() bugs with host-side inputs and under tensor parallelism. It also speeds up Whisper generation on TPU by cutting device syncs and recompilation, and fixes Whisper batched timestamp decoding for tensor and NumPy inputs. Pipelines get a cleanup of generation-argument handling that removes spurious warnings.

• Add job to automatically generate Intel CPU Docker image (#49560) by @regisss in [#49560]
• Image decoding moves to torchcodec (#47963) by @zucchini-nlp in [#47963]
• Optimize Whisper generation for TPU and device sync (#49426) by @tengomucho in [#49426]
• Generation cleanup in pipelines (#49269) by @Rocketknight1 in [#49269]
• Fix Whisper batch timestamp decoding for tensors (#49419) by @Kingofbooks in [#49419]

Vision

Vision processing was updated in several places. Image decoding now uses torchcodec, Molmo2's image and video kwargs follow the library's standard conventions, and Pixtral Flash Attention no longer detects false image boundaries after the axial RoPE refactor, which had caused CUDA illegal memory access errors and incorrect output. Several CI and Docker changes also shipped: GPU images moved to Python 3.11, a numpy/pyarrow conflict

• Fix numpy/pyarrow conflict in the XPU Docker image (#49582) by @jiqing-feng in [#49582]
• CI: Move GPU images to python 3.11 (#49352) by @tarekziade in [#49352]
• Update molmo2 processing images/videos kwargs (#49563) by @guarin in [#49563]
• Fix Pixtral Flash Attention image boundaries after axial RoPE refactor (#49373) by @oliverholworthy in [#49373]

Kernels

This release improves kernel support across hardware. flash_attention_2 now falls back to the AITER kernel on ROCm when flash-attn isn't installed, and T5-style models get a contiguous SDPA position-bias mask so they use fused kernels again. The Hub rotary kernel is now limited to CUDA inference because its backward pass gave incorrect gradients. The Atlas Inference Gated DeltaNet kernels for Strix Halo and

• Fall back to the AITER kernel for flash_attention_2 on ROCm (#49608) by @Abdennacer-Badaoui in [#49608]
• Make the SDPA position_bias mask contiguous so T5-style models use fused kernels (#49517) by @IvenHsu01 in [#49517]
• Use the Hub rotary kernel for inference only on CUDA (#49439) by @qgallouedec in [#49439]
• [Atlas Inference] Enable torch.compile for the Strix Halo Gated DeltaNet hub kernel (#49453) by @AzeezIsh in [#49453]
• [Atlas Inference] Fix GB10 Gated DeltaNet hub kernel for transformers 5.17+ and enable torch.compile (#49452) by @AzeezIsh in [#49452]

Cache

This release fixes several cache-related bugs: static-cache generate() now handles host-side nested inputs and tensor parallelism correctly, the offloaded cache prefetch no longer races with the offloading copy, and Phi3 LongRoPE cache rebuilds at the context boundary now receive the full prefix. It also counts the query token in MiMo-V2-Flash's sliding window and corrects the quantized cache (hqq/quanto) documentation exam

• Fix offloaded cache prefetch racing with the offloading copy (#49363) by @qgallouedec in [#49363]
• docs: update quantized cache hqq/quanto examples and documentation (#47212) by @DavidJohnQuinlan in [#47212]
• [MiMo-V2-Flash] Count the query token in the sliding window (#49170) by @guapisolo in [#49170]
• Fix Phi3 LongRoPE cache rebuild at the context boundary (#49361) by @LE0-Lin in [#49361]

Bugfixes and improvements

• [docs] Hardware docs template (#49575) by @stevhliu in [#49575]
• Add MIC contract lock files for PR CI integration (#49631) by @ydshieh in [#49631]
• Use aiter-rope version 3 on ROCm (#49595) by @Abdennacer-Badaoui in [#49595]
• Remove missing methods from docs (#49637) by @guarin in [#49637]
• Translate torch.mm(..., out_dtype=...) to ONNX (#49635) by @IlyasMoutawwakil in [#49635]
• [docs] Hardware section (#49541) by @michaelbenayoun in [#49541]
• Raise when a torch fallback would drop packed-sequence boundaries (#49474) by @qgallouedec in [#49474]
• Fa packed sequence decode (#49610) by @SunMarc in [#49610]
• fix nested set_attn_implementation (#49138) by @eustlb in [#49138]
• feat: Add MTP support to Mellum2/2.1 (#49531) by @shadeMe in [#49531]
• extend onnx exporters tests to non-cuda devices (#49627) by @kaixuanliu in [#49627]
• Document video processor backends (#49470) by @DimensionSTP in [#49470]
• Make sinusoidal position embeddings compileable (#49482) by @jiqing-feng in [#49482]
• Save the PEFT adapter of a load-time sharded model from its own parameters (#49500) by @qgallouedec in [#49500]
• fix: Pass the MTP layer type to per-layer-type rotary embeddings (#49468) by @shadeMe in [#49468]
• Add LFM2 to the GGUF loading path (#49400) by @francip in [#49400]
• Add MtpModel helpers for frameworks that own the drafting loop (#49544) by @hmellor in [#49544]
• Build docs with the Python version from .python-version (#49603) by @tarekziade in [#49603]
• Use mlp_layer_types for AXK1 so the MoE layer layout honours moe_layer_freq (#49518) by @hmellor in [#49518]
• Combine Nvidia+AMD run-slow CI comments into one (#49598) by @ydshieh in [#49598]
• Fix flaky streamer timeout tests (#49583) by @tarekziade in [#49583]
• Fix TPU related tests issues (#49568) by @tengomucho in [#49568]
• fix(distributed): bind TPU ranks to device index 0, not LOCAL_RANK (#49543) by @tengomucho in [#49543]
• Use transformers init everywhere (#49573) by @guarin in [#49573]
• [docs] Context parallelism during eval (#48663) by @stevhliu in [#48663]
• Remove examples (#49357) by @SunMarc in [#49357]
• Accept whole-number floats for integer tool arguments in parse_response (#49547) by @yonigozlan in [#49547]
• update Intel CPU Dockerfile (#49408) by @jiqing-feng in [#49408]
• Add missing video processor auto docstrings (#49569) by @guarin in [#49569]
• Add chat input sanitization (#47386) by @Rocketknight1 in [#47386]
• Fix create_chunked_causal_mask() crashing with block_sequence_ids (#49457) by @kaixuanliu in [#49457]
• Fix SinglePositionMultiTokenCandidateGenerator signature (#49413) by @jiqing-feng in [#49413]
• Minicpm 4.7 update integration tests (#49565) by @zucchini-nlp in [#49565]
• Stop treating HunYuan's DynamicNTKAlpha RoPE as dynamic (#49445) by @jiqing-feng in [#49445]
• Fix Gemma3nVision2TextModelTest::test_model_parallelism split (#49562) by @tarekziade in [#49562]
• Fix UnboundLocalError in SuperPoint forward under torch.compile (#49409) by @kaixuanliu in [#49409]
• Preserve resolved remote backend references in AutoVideoProcessor (#49511) by @DimensionSTP in [#49511]
• Fix PR slow CI suggestion: install transformers in a venv (#49558) by @tarekziade in [#49558]
• Fix PR slow CI suggestion: install transformers, fail on script errors (#49549) by @ArthurZucker in [#49549]
• Remove deprecated data helpers (#49360) by @SunMarc in [#49360]
• Fix test_all_params_have_gradient for intentionally frozen parameters (#49264) by @thillai-c in [#49264]
• Flex attn fix for molmo2 test (#49530) by @molbap in [#49530]
• fix bridgetower cross-attn mask shape bug (#49447) by @kaixuanliu in [#49447]
• Use Siglip2Tokenizer when the Hub tokenizer_class is GemmaTokenizer (#49245) by @jayzuccarelli in [#49245]
• Fix mixed BOS prefix validation in WatermarkDetector (#49496) by @dangruiyuan in [#49496]
• Deprecate unused get_uniform_frame_indices (#49464) by @archiep867-sys in [#49464]
• [tests] Some fixes for test suite failures on TPU (#49425) by @tengomucho in [#49425]
• Second phase of torch 2.6 cleanups (#49396) by @Rocketknight1 in [#49396]
• Guard the checks for current_accelerator() and bump Torch version (#49395) by @Rocketknight1 in [#49395]
• Use one static Transformers CI badge in PR descriptions (#49471) by @tarekziade in [#49471]
• fix: treat every declared field of the config class as a default fixes #49456 (#49466) by @tarekziade in [#49466]
• Keep AutoVideoProcessor class lookup lazy (#49458) by @DimensionSTP in [#49458]
• Add Molmo2 (#43451) by @SangbumChoi in [#43451]
• Ci fetches multimodal models recursively (#49389) by @zucchini-nlp in [#49389]
• Recurse for all subconfigs (#49461) by @zucchini-nlp in [#49461]
• Aria, Ernie4.5-VL-MoE, Gemma4, Inkling, MiniMax-M3-VL, Qwen3.5-MoE: run the model tests on the VLM/ALM testers (#49404) by @IlyasMoutawwakil in [#49404]
• Fix CI slack reporting: surface crashed/missing model jobs in model_results.json (#49427) by @ydshieh in [#49427]
• [Qwen3VLMoe] Update expected text in test_small_model_integration_test_expand (#49301) by @ydshieh in [#49301]
• Fix backend attribute in documentation and default backend in MistralCommonBackend (#49149) by @samyuktahegde in [#49149]
• More RoPE simplification, TF32 fixes (#49328) by @Rocketknight1 in [#49328]
• Complete PR #49228: remove unused device=None from rope init in 11 model families (#49285) by @ydshieh in [#49285]
• Revert "[CI] Temporarily disable AMD scheduled CI trigger" (#49433) by @ydshieh in [#49433]
• Fix QuantizedCache residual flushing for multi-token updates (#49418) by @LE0-Lin in [#49418]
• Fix data-dependent branch in _ignore_causal_mask_sdpa under compile (#49354) by @dacorvo in [#49354]
• Fix Inkling moe_intermediate_size when dense_intermediate_size is present (#49416) by @Talhax55z in [#49416]
• fix bug for DebertaV2ForMaskedLM non-legacy load issue (#49152) by @kaixuanliu in [#49152]
• This PR fixes Gemma3n fp16 loading when missing weights need to be in… (#49147) by @sywangyi in [#49147]
• PR CI: grant actions: read to the reusable workflow (#49417) by @tarekziade in [#49417]
• [Bugfix] [Modeling] Fix DSV4 norm (#47486) by @kylesayrs in [#47486]
• [docs] Update Spaces (#49242) by @stevhliu in [#49242]
• handle empty precompiled charsmap in spm converter (#49211) by @itazap in [#49211]
• [CI] Temporarily disable AMD scheduled CI trigger (#49394) by @ydshieh in [#49394]
• [tests] Fix flaky CMVN tests in Speech2TextFeatureExtractionWithoutTorchaudioTest (#49374) by @ydshieh in [#49374]
• Let token dispatch run with ep_size != tp_size (#49368) by @qgallouedec in [#49368]
• [tests] Skip onnx.exactness for BitForImageClassification (ORT fp32 drift) (#49366) by @ydshieh in [#49366]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

• @itazap
  • Add MOSS-Transcribe-Diarize model (#48386)
  • handle empty precompiled charsmap in spm converter (#49211)
  • 🚨🚨🚨 luke fixe (#48408)
• @ydshieh
  • Add MIC contract lock files for PR CI integration (#49631)
  • Combine Nvidia+AMD run-slow CI comments into one (#49598)
  • Fix CI slack reporting: surface crashed/missing model jobs in model_results.json (#49427)
  • [Qwen3VLMoe] Update expected text in test_small_model_integration_test_expand (#49301)
  • Complete PR #49228: remove unused device=None from rope init in 11 model families (#49285)
  • Revert "[CI] Temporarily disable AMD scheduled CI trigger" (#49433)
  • [CI] Temporarily disable AMD scheduled CI trigger (#49394)
  • [tests] Fix flaky CMVN tests in Speech2TextFeatureExtractionWithoutTorchaudioTest (#49374)
  • [Docker] Upgrade to torch 2.14/CUDA 13.0 and optimize FA2 build from source (#49381)
  • [tests] Skip onnx.exactness for BitForImageClassification (ORT fp32 drift) (#49366)
• @icavan
  • Fix a port allocation race in FSDP tests (#49455)
  • [Model] Add native Ling 3.0 Flash support (#49446)
• @tarekziade
  • Build docs with the Python version from .python-version (#49603)
  • Fix flaky streamer timeout tests (#49583)
  • CI: Move GPU images to python 3.11 (#49352)
  • Fix Gemma3nVision2TextModelTest::test_model_parallelism split (#49562)
  • Fix PR slow CI suggestion: install transformers in a venv (#49558)
  • Use one static Transformers CI badge in PR descriptions (#49471)
  • fix: treat every declared field of the config class as a default fixes #49456 (#49466)
  • PR CI: grant actions: read to the reusable workflow (#49417)
• @lucamotz
  • Add Kolibri 1 (#49463)
• @SangbumChoi
  • Add Molmo2 (#43451)
• @maxiaolong03
  • [Model] Add PP-DocLayoutV4 Model Support (#48387)

Original source

Release v5.19.0

New Model additions

EmbeddingGemma2

EmbeddingGemma 2 is a multimodal embedding model from Google built on the Gemma 4 architecture. It encodes text, images, audio, and video, individually or combined in one input, into a shared 768-dimensional vector space for cross-modal retrieval, semantic similarity, clustering, and classification. It uses Matryoshka Representation Learning, so embeddings can be truncated to 512, 256, or 128 dimensions. It also offers configurable visual and video token budgets, and unused vision or audio towers can be disabled at load time to save memory.

Links: Documentation

Smthn smthn (#49364) by @vasqu in #49364

Breaking changes

All MoE models whose routers compute logits now return them when output_router_logits=True, following the Qwen3-MoE pattern (a router_logits recorder on the base model, MoeModelOutputWithPast from the backbone, and a MoE causal LM output from the head), so code that relied on the previous outputs or their absence should read the router logits from these output classes.

🚨 Return router logits from every MoE model that computes them (#48920) by @qgallouedec

Owlv2ForObjectDetection.embed_image_query now selects the query box with the highest objectness score, as in the original OWLv2 notebook, instead of the OWL-ViT heuristic, so image-guided query embeddings and detections may differ from earlier releases.

🚨 Select the OWLv2 image query by objectness (#49200) by @qgallouedec

The "paged|" prefix for SDPA and flash attention implementations is deprecated, so users should set the regular attention implementation (e.g. sdpa or flash_attention_2) for continuous batching instead of paged|sdpa or paged|flash_attention_2.

🚨 Attention 🚨 Deprecate "paged|" prefix for SDPA and flash (#49112) by @remi-or

The regular flash and SDPA attention functions (flash_attention.py, sdpa_attention.py) now support continuous batching directly, and "paged|..." implementations for these are redirected to them, while eager still requires the "paged|eager" prefix.

🚨 Attention 🚨 Make regular attention support CB (#49101) by @remi-or

In continuous batching, the cache update for the index-based and block-table paths is now fused into a single call, which slightly changes the cache update function's behavior and affects any custom code that calls the separate update paths.

🚨 [CB] 🚨 Fuse update for index and block table path (#49088) by @remi-or

Continuous batching internals changed in preparation for removing "paged": `max

🚨 [CB] 🚨 Little fixes before removing "paged" (#49069) by @remi-or

Parallelization

Expert parallelism gains a token-dispatch implementation, selected via the new ep_dispatch_experts plan rule and now the default for Qwen3 MoE and Mellum, which removes the requirement that EP size equal TP size. The Trainer was also adapted to work with expert parallelism, and the docs now note that PEFT adapters support tensor parallelism. A CI-related fix for pipeline-parallel chart2table inference was also included.

[distributed]: adapt Trainer to work Expert Parallel (#48873) by @3outeille in [#48873]

[distributed] Add expert-parallel token dispatch, default for Qwen3 MoE (#48865) by @3outeille in [#48865]

Fix pp chart2table inference (#49225) by @zucchini-nlp in [#49225]

[docs] TP for PEFT adapters (#49060) by @stevhliu in [#49060]

Cache

This release fixes quantized cache handling: generate no longer mutates the user's cache_config, and QuantizedLayer.reorder_cache is repaired. It also adds per-layer cache configuration, so DynamicCache and StaticCache initialize each layer from its own config (sliding window, attention chunk size, conv states, and attention head counts) to better support heterogeneous models. Separately, the deprecation cycle on mask and cache

Fix quantized cache (#48700) by @jiqing-feng in [#48700]

[CI] check_bad_commit: use EFS cache to avoid Xet FUSE OOM (exit 137) (#49273) by @ydshieh in [#49273]

Support per-layer cache configuration (#48178) by @eladsegal in [#48178]

Remove deprecation cycle on mask and cache (#49231) by @Cyrilvallez in [#49231]

Bugfixes and improvements

Revert "[CI] Temporarily disable AMD scheduled CI trigger" (#49271) (#49362) by @ydshieh in [#49362]

ALM testx fIxing... (#49355) by @zucchini-nlp in [#49355]

Fix DataCollatorForLanguageModeling ignoring seed=0 (#49349) by @akanyaani in [#49349]

Fix explicit kernelization modes for evaluation models (#49339) by @DimensionSTP in [#49339]

fix(cohere_compass): add video to input_modalities (#49320) (#49323) by @destopianpirate in [#49323]

GPT-OSS: read the clamped SwiGLU's alpha and limit from the config (#49346) by @IlyasMoutawwakil in [#49346]

[distributed] Default MoE ep_plans to token dispatch (#49160) by @3outeille in [#49160]

Move CI to Python 3.11, with the version set in .python-version (#49311) by @tarekziade in [#49311]

Bump doc-builder main docs workflow pin (#49350) by @paulinebm in [#49350]

[distributed] decoupled tp_plan and ep_plan (#48859) by @3outeille in [#48859]

[distributed] Add dense and expert device mesh (#48857) by @3outeille in [#48857]

Bump doc-builder workflow pin (#49342) by @paulinebm in [#49342]

Allow registering new processing backends (#48984) by @zucchini-nlp in [#48984]

Fix WatermarkDetector repeated-ngram counting (#49315) by @LE0-Lin in [#49315]

Fix minicpm 4.6V video batching and CI (#49259) by @zucchini-nlp in [#49259]

remove colwise_gather_output for lm_head for DeepseekV4 (#49314) by @3outeille in [#49314]

Fix the labels docstring of CLIPSegForImageSegmentation (#49001) by @qgallouedec in [#49001]

[CI] Temporarily disable AMD scheduled CI trigger (#49271) by @ydshieh in [#49271]

Remove the unused pandas entry from setup.py deps (#49329) by @tarekziade in [#49329]

Fix run_glue.py label ids becoming strings with pandas>=3 (#49326) by @tarekziade in [#49326]

[Fix] Remove old test file with two ancient tests (#49280) by @remi-or in [#49280]

Fix chunked prefill with multi-axis position ids (Qwen3-VL) (#49319) by @dacorvo in [#49319]

Why override when you can fix in Mixin (#49105) by @zucchini-nlp in [#49105]

Do RoPE with mul instead of matmul (#49265) by @Rocketknight1 in [#49265]

Deprecated pipelines should redirect; removed pipelines should raise (#49308) by @LysandreJik in [#49308]

Enable model test on upcoming torch_tpu backend (#49207) by @tengomucho in [#49207]

Fix encoder repetition penalty source rows after batch expansion (#49262) by @gitedmond in [#49262]

Preserve masked EOS scores in exponential decay length penalty (#49260) by @gitedmond in [#49260]

[CI] Avoid fetching all artifacts when only specific ones are needed (#49282) by @ydshieh in [#49282]

Revert "[CI] ssh-runner: add optional cache_type input to switch between bucket and EFS runners (#49159)" (#49270) by @ydshieh in [#49270]

Add Trainer.loss_is_scaled_for_ga to declare whether compute_loss already scales for gradient accumulation (#49240) by @qgallouedec in [#49240]

fix precomputed flash kwargs clashing (#47928) by @IlyasMoutawwakil in [#47928]

Reapply modular examples (#49230) by @Cyrilvallez in [#49230]

Remove some stuff that does not exist (#49232) by @Cyrilvallez in [#49232]

Remove rotaries deprecation cycle (#49228) by @Cyrilvallez in [#49228]

[GTE] Add memory mixin (#49234) by @vasqu in [#49234]

[CB] Make CB more device agnostic and add support for XPU (#49156) by @remi-or in [#49156]

correct final metrics due to resuming from checkpoint (#49208) by @SunMarc in [#49208]

[ESM/Persimmon/NllbMoe/ESMFold] use safetensors repos to fix Xet FUSE SIGBUS/IOError on bucket CI runners (#49194) by @ydshieh in [#49194]

Dev update (#49212) by @vasqu in [#49212]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@vasqu

Smthn smthn (#49364)

[GTE] Add memory mixin (#49234)

Dev update (#49212)

@remi-or

[Fix] Remove old test file with two ancient tests (#49280)

[CB] Make CB more device agnostic and add support for XPU (#49156)

🚨 Attention 🚨 Deprecate "paged|" prefix for SDPA and flash (#49112)

🚨 Attention 🚨 Make regular attention support CB (#49101)

🚨 [CB] 🚨 Fuse update for index and block table path (#49088)

[Refactor] Make some flash-attention utils more readable (#49071)

🚨 [CB] 🚨 Little fixes before removing "paged" (#49069)

Original source

New Model additions

Nemotron 3 Diarization

Nemotron 3 Diarization is an open-weight streaming speaker diarization model designed to determine "who spoke when" in real-world audio. It supports both streaming and offline inference, handles up to eight speakers, and orders speaker outputs by each speaker's first arrival in the input audio.

The model uses the Arrival-Order Speaker Cache (AOSC) 1 and FIFO queue introduced for Streaming Sortformer 1, 2. A single checkpoint supports configurable latency profiles, from an 80 ms input buffer to a 30.4 s offline-style buffer, and configurable output frame resolution in multiples of 10 ms. With chunked inference, the maximum audio duration is not limited.

Links: Documentation

Add Nemotron3Diarization (#49056) by @eustlb in #49056

NemotronH Omni

NemotronH Omni is a multimodal reasoning model from NVIDIA that pairs the NemotronH hybrid Mamba-Transformer language model with a RADIO vision encoder and an optional Parakeet-based sound encoder. Image (and video) patches are projected through a RADIO tower and a pixel-shuffle MLP into the language model's embedding space at the <image> / <video> context-token positions; audio clips are projected in the same way at <audio> positions. The result is a single autoregressive model that reasons jointly over text, images, video and sound.

Links: Documentation

Add support for Nemotron Omni (#46509) by @meatybobby in #46509

HyperCLOVAX Vision V2

HyperCLOVAX Vision V2 is a multimodal vision-language model developed by NAVER. It combines the HyperClovaX language model backbone with a Qwen2.5-VL vision encoder. The model supports text, image, and video inputs and is capable of chain-of-thought reasoning via built-in thinking tokens (<think>...</think>).

Links: Documentation

add HyperClovaX Vision (#44314) by @jp1924 in #44314

GTE

GTE was proposed in mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval by Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li and Min Zhang.

GTE is a BERT-style bidirectional encoder that replaces absolute position embeddings with RoPE, uses a gated MLP, and applies layer normalization after each residual connection. The same architecture backs Alibaba's gte--v1.5, gte-multilingual- and gte-en-mlm-* checkpoints as well as Snowflake's snowflake-arctic-embed-m-v2.0.

Links: Documentation

model: Add GTE to Transformers (#48416) by @harshaljanjani in #48416

Breaking changes

• 🚨 [ROCm] gpt-oss: route FA3 to aiter-flash-attn, generate ROCm fixtures (#46837) by @Abdennacer-Badaoui
• 🚨 Speed up detr image processing (#48066) by @guarin
• 🚨 Remap indexers layer_type (#48974) by @Cyrilvallez
• 🚨 [vLLM] Fix video token counting for Transformers backend video inputs (Part 1) (#48894) by @harshaljanjani
• 🚨 🚨Bring some dinos to modern standards (#46266) by @molbap
• 🚨 [Kernels] Bump version (#48714) by @vasqu

Bugfixes and improvements

• fix incorrect hub tokenizer class (#48641) by @itazap
• [CI] Deduplicate Nvidia/AMD CI reply comments and add headers (#48655) by @ydshieh
• Update dev (#48654) by @vasqu
• [tests] Fix NougatModelIntegrationTest: pin artifact revision and update golden values (#48638) by @ydshieh
• [fix] Fix GlmOcr integration tests: wrong token IDs and image token decode bug (#48650) by @ydshieh
• [GLM 5.3 Flash] Preserve original names when saving text checkpoints (#48676) by @Dovis01
• Register activation kernel layers on XPU (#47858) by @jiqing-feng
• Mirror the registered pytree flatten when building automatic dynamic shapes (#48578) by @IlyasMoutawwakil
• Fix AutoImageProcessor requiring torchvision when only Pillow is installed (#48616) by @blipbyte
• Assert cached decode matches recomputing without a cache (#48289) by @IlyasMoutawwakil
• [higgs_audio_v2] Fix: use config.num_codebooks in audio labels tensor (Simple Fix!) (#48560) by @b-re-w
• [MoE] Fix eager EP (#48653) by @vasqu
• [MusicgenMelody] Fix conditioning silently dropped at generation step 0 (#48679) by @ydshieh
• Fix deepspeed ci (#48640) by @SunMarc
• Synthetic test assets (#48589) by @tarekziade
• Skip the expert-parallel sentinel masking when expert parallelism is off (#48201) by @qgallouedec
• [Fix] yolos offload issue (#48688) by @molbap
• [serge] Fix 2 integration tests for model minimax failing with output_mismatch (tensor values differ (2)) (#48515) by @sergereview[bot]
• [serge] Fix 2 integration tests for model mistral failing with other (other (2)) (#48429) by @sergereview[bot]
• Fix Qwen2.5-VL temporal RoPE for fractional video intervals (#48669) by @yeyeyeping
• Fix slow integration tests on XPU (#48611) by @jiqing-feng
• QA: Added a MemoryCleanupMixin class for tests (#48681) by @tarekziade
• [serge] Fix 1 integration test for model flex_olmo failing with other (other (1)) (#48668) by @sergereview[bot]
• Fix D-FINE / RT-DETR main loss being computed over the denoising queries (#48528) by @stefan-it
• [CI] Replace hardcoded username allowlists with author_association check in workflow triggers (#48712) by @ydshieh
• Add flash_attention_4 in attn_implementation AutoModel docstring (#48684) by @3manifold
• Keep the expert-parallel sentinel slots out of the router gradient (#48689) by @qgallouedec
• [DeepseekV3] OOM cascade root-cause investigation (generator ref leak in conversion_mapping) (#48720) by @ydshieh
• Use huggingface_hub httpx export (#48685) by @Wauplin
• unifying device_mesh init to enable PP + TP inference (#48155) by @3outeille
• Enable FSDP2 + expert parallelism via a 2-D (fsdp, tp) device mesh (#48516) by @qgallouedec
• Switch daily CI to torch 2.14 — update expected outputs (#48750) by @ydshieh
• fix videomae load error (#48675) by @sywangyi
• Fix silently random-initializing RTDetrModel/SEWDForCTC loads (wrong base_model_prefix) (#48744) by @
• Fix Glm4vMoeIntegrationTest: offload_folder + MemoryCleanupMixin (#48776) by @ydshieh
• [InternVL] Normalize num_patches before np.cumsum (#48469) by @lorenzozanee
• Kernel api doc, part 2 (#46889) by @michaelbenayoun
• Fix hidden state selection on Gemma4 assistant's first prefill step (#48704) by @glistening
• Fix inkling embedding norm (#48786) by @Cyrilvallez
• Fix TextToAudioPipeline crash for tokenizer-only models (#48505) by @jiqing-feng
• Cast pixel values to the patch embedding dtype in DeepSeek-OCR-2 (#48632) by @jiqing-feng
• add kernel mapping entry for RMSNormGated, KDA, Conv1D on XPU (#48702) by @kaixuanliu
• Bump peft version requirement (#48716) by @shniubobo
• [serge] Fix 12 integration tests for model edgetam failing with import_or_config (other (12)) (#48322) by @sergereview[bot]
• Restore legacy tensor-parallel initialization for compatibility (#48797) by @3outeille
• Fix kosmos flaky test (#48800) by @IlyasMoutawwakil
• Add integration tests for MuseGlimmerAssistantModel (#48796) by @ydshieh
• Fix AutoModel.from_pretrained not restoring modules_to_save weights (#48595) by @shniubobo
• [docs] mrope and axial rope (#48717) by @stevhliu
• Fix decoder only path for older bert variants (#48785) by @Cyrilvallez
• [Fix] Clean-up ternaries in the DeepSeek family (#48447) by @remi-or
• [utils] Add MUSA support for Flash Attention 2 (#48612) by @XiaomingFun233
• [generate] Drop attention mask early without padding (#48814) by @Cyrilvallez
• Pin utf-8 in test_can_init_all_missing_weights source read (Windows non-UTF-8 locale fix) (#48819) by @dltsum
• Relax static-cache tolerance in test_generate_with_static_cache (1e-5 → 5e-5) (#48815) by @ydshieh
• Detect nested rope_parameters without relying on layer_types (#48798) by @hmellor
• Support for MoE in the GGUF integration (#48529) by @SunMarc
• [docs] gguf (#46357) by @stevhliu
• [apply_chat_template] pass sampling_rate to call (#48794) by @eustlb
• [CI] Add link checker (#48160) by @stevhliu
• Qwen3.8 GGUF (#48660) by @SunMarc
• docs: fix broken #combining-with-fsdp2 anchor in expert_parallelism.md (#48854) by @ydshieh
• [GPTNeoXJapanese] Fix RoPE ignoring partial_rotary_factor (#48652) by @blipbyte
• QA: deactivate rule 41 (#48852) by @tarekziade
• Fix reset on the dynamic cache layers (#48809) by @jiqing-feng
• [generate] Make all methods and logits processors agnostic to lm_head output size (#48846) by @Cyrilvallez
• Fix kosmos (#48853) by @Cyrilvallez
• Add test for causal only variant of some encoder-decoder models (#48760) by @nandan2003
• Fix ESMFold2 ligand iPLDDT weighting: mol_type non-polymer code is 3, not 4 (#48831) by @faustomilletari
• Fix flaky RfDetr test_save_load (#48869) by @ydshieh
• fix(ci): harden GitHub Actions workflows (#48160) (#48834) by @hf-security-analysis[bot]
• Fix Glm4MoeIntegrationTest: split into 3 tests, switch to GLM-4.5-Air (#48820) by @ydshieh
• Fix StaticCache for Mllama and enable torch.compile (#48141) by @jiqing-feng
• Fix Minimax M2 partial_rotary_factor by remapping legacy rotary_dim (#48486) by @dajiaohuang
• Fix Pixtral image processor do_pad option (#48872) by @jzakrzew
• [CB] Add pause mechanism (#48462) by @remi-or
• Fix DFlash sampled candidates losing the batch dimension (#48281) by @VaggelisGian
• Fix sliding window cache when assistant model calls generate (#48280) by @VaggelisGian
• Fix GLM rope_parameters dict shared mutation across test instances (#48895) by @ydshieh
• [feat] Allow untying hidden_states[-1] from last_hidden_state via the model config (#48087) by @tomaarsen
• Fix Idefics2 padding when the first example has no image (#48753) by @Arnavsharma2
• Don't fail model loading when the accelerator can't report free memory (#48136) by @studioego
• [Improvement] Rework the docstring and comments of mHC (#48888) by @remi-or
• [serge] Fix 3 integration tests for model deepseek_vl failing with other (other (3)) (#48536) by @sergereview[bot]
• Add auto_docstring task model overrides (#47815) by @guarin
• Make LongcatFlashConfig self-consistent without building the model (#48899) by @hmellor
• [DSA] Only save latents on dsa with indexer as well (#48876) by @vasqu
• [Chat Parsing] Coerce oneOf tool arguments (#48719) by @yonigozlan
• Always materialize the causal mask in Doge so sdpa stays causal (#48821) by @blipbyte
• Fix docstring argument names that don't match signatures (#48474) by @IshaanPotle
• Mark test_generate_with_static_cache as flaky for olmo and bigbird_pegasus (#48856) by @ydshieh
• Clean up some MoE models' integration tests (#48833) by @ydshieh
• Fix T5 tied weights order for lm_head (#48238) by @vaibhavmashal
• Size the device_map buffer from the largest leaf module (#47211) by @dhruv7477
• Fix compile cache tests (#48923) by @Cyrilvallez
• Update arXiv citation in NeoMME model doc (#48926) by @tonywu71
• Standardise Aria's MoE onto the experts interface (#48907) by @hmellor
• Fix possessive typo in Whisper long-form warning (#48877) by @erikdw
• Modular conversion small fix (#48934) by @zucchini-nlp
• fix(vibevoice-asr): use integer ceiling division for audio token count (#48864) by @ege-arhan
• Improve lazy import error messages (#48602) by @karatarassul4-max
• Fix tests due to dropping attn mask (#48903) by @SunMarc
• docs: fix docstring parameters that do not match signatures (#48904) by @simpleqt
• DOC: Improve documentation for remap_legacy_layer_types function (#48651) by @mathewOracle
• Align special tokens on the text config (#48847) by @qgallouedec
• docs: fix dead doc links in i18n READMEs and the xlnet docstring (#48905) by @simpleqt
• Fix AXK2 integration test: update CUDA expected text and rename class (#48941) by @ydshieh
• Resolve the Hub revision once per load instead of passing a private _commit_hash around (#47611) by @Wauplin
• Keep image processor backends in sync on keys and dtypes (#48739) by @yupengtang
• Fix infeasible cost matrix errors in hugarian matcher losses (#47730) by @guarin
• Fix more stuff (#48979) by @zucchini-nlp
• Skip an unnecessary image copy in the torchvision image normalization path (#48897) by @jzakrzew
• Better attn default for gguf (#48935) by @SunMarc
• Add pose estimation keypoint preprocessing to Sapiens2ImageProcessor (#47199) by @Sainava
• Fix image processor class-level size mutation and min_pixels handling (#48916) by @Gracy769
• Contract the mamba2 chunk scan with einsum instead of broadcast-then-sum (#48978) by @tarekziade
• Fix typos in DeepseekV4 comments (#48953) by @Janiarafath
• Reduce peak memory in examples_torch CI job (OOM fix) (#48983) by @ydshieh
• Fix vibevoice TTS batched audio index (#48902) by @ebezzam
• [CB] [Major] Upgrade the cache to support different attention types (#47809) by @remi-or
• Fix SwitchTransformers Top1 router: raw logits, expert capacity accounting, and router losses (#48421) by @yurekami
• Fix Qwen3OmniMoeIntegrationTest OOM (#48987) by @ydshieh
• Let prefix_allowed_tokens_fn override model -inf and raise an exception on unsatisfiable generation constraints. (#48927) by @ksh108405
• [generate] Simplify candidate generators by removing required update_candidate_strategy (#48982) by @Cyrilvallez
• [generate] Always correctly restrict assisted decoding with max length/eos token (#48981) by @Cyrilvallez
• QA: applied ruff rule PLW1514 (#48990) by @tarekziade
• Update tokenizer gguf support (#48656) by @SunMarc
• [vLLM] Fix video token counting for Transformers backend video inputs (Part 2) (#48900) by @harshaljanjani
• Update ggml kernels path (#48991) by @SunMarc
• Enable compressed-tensors FP8 kernels on MPS (torch >= 2.15) (#48985) by @Isalia20
• Fix mps autocast handling in rotary embeddings (#49006) by @Isalia20
• Fix static cache per layer head shapes (#48619) by @dacorvo
• Summarization examples: download NLTK punkt_tab, not punkt (#49014) by @davanstrien
• [docs] Fix legacy hf CLI references (transformers) (#48988) by @Wauplin
• Fix RecurrentGemma compiled generation with StaticCache (#48961) by @sywangyi
• updated GraniteMoeHybrid expectations (CPU) (#49016) by @tarekziade
• OpenVINO HF Exporter (#47003) by @IlyasMoutawwakil
• fix noisy comments (#49013) by @tarekziade
• Fix assisted decoding for VLM due to dropping attn mask (#49019) by @SunMarc
• Deprecate min-max pixels (#49021) by @zucchini-nlp
• Keep special token ids the tokenizer does not define (#48708) by @albertvillanova
• Added more usage of MemoryCleanupMixin (#49011) by @tarekziade
• Shorten noisy OpenVINO SDPA comment (#49033) by @tarekziade
• PEFT x Dtensor-based TP integration (#48485) by @michaelbenayoun
• fix(zamba): add use_associative_scan config flag to avoid torch.compile slowdown (#48331) by @msnliu
• Scope GITHUB_TOKEN permissions per job (#49046) by @hf-security-analysis[bot]
• Fix mask creation not being skipped under torch.compile (#48975) by @jiqing-feng
• [Chat] Loading GGUF models served with the Chat CLI (#49031) by @ariG23498
• Pin GitHub Actions to commit SHAs (#49049) by @hf-security-analysis[bot]
• Only seed numpy in BigBird block-sparse attention during training (#49037) by @jiqing-feng
• QA: Fix llama4 leak (#49042) by @tarekziade
• Fix startup failures: drop permissions reusable-workflow callers cannot grant (#49055) by @paulinebm
• Fix startup failures: drop pull-requests: read from check_failed_tests.yml (#49057) by @paulinebm
• Register the remaining mamba-ssm kernel layers on XPU (#49035) by @jiqing-feng
• Add workflow for building XPU CI Docker images (#49039) by @regisss
• Fix assistant masks for processors (#48793) by @Rocketknight1
• Add MPS maintainer (#49052) by @Rocketknight1
• [docs] Loading behavior (#49023) by @stevhliu
• [Nemotron3Diarization] nit: hub pr merged to main (#49065) by @eustlb
• fix(ci): harden GitHub Actions workflows (#49057) (#49059) by @hf-security-analysis[bot]
• Honor config.output_router_logits in the MoE VLM wrappers (#48885) by @qgallouedec
• Fix XPU Docker image build (#49073) by @regisss
• Fix assisted eos token condition (#49075) by @Cyrilvallez
• [serge] Fix OOM in Moshi integration tests with MemoryCleanupMixin (#48839) by @sergereview[bot]
• Fix odd head_dim validation for RoPE configurations (#48524) by @somuai
• Keep already-decoded array arguments unchanged (#49062) by @yonigozlan
• Fix NaN in Parakeet eager attention with padded batches (#49070) by @ArthurZucker
• Bump huggingface_hub upper bound to <3.0 (transformers) (#49083) by @Wauplin
• Keep attention_mask as None in OPT's causal mask creation (#49002) by @jiqing-feng
• Add Trainer.end (#48875) by @qgallouedec
• Add image processing tester init (#48829) by @guarin
• [Executorch] Add MLX recipe (#48910) by @metascroy
• [Zamba] Fix associative scan breaking ONNX export and OOM in integration test (#49092) by @ydshieh
• Fix RGB early-return skipping PNG tRNS compositing (#49005) by @cs-fisha
• Use namespaced dataset ids in docs and PyTorch examples (#49017) by @davanstrien
• Fix MaskFormerSwin attention mask dtype to follow hidden states (#49032) by @kaixuanliu
• [NemotronH-Omni] Fix device mismatch in test tensor creation (#49102) by @ydshieh
• [serge] Fix 2 integration tests for model cvt failing with output_mismatch (tensor values differ (2)) (#49076) by @sergereview[bot]
• [serge] Fix 2 integration tests for model hy_v3 failing with output_mismatch (tensor values differ (2)) (#49068) by @sergereview[bot]
• [serge] Fix 2 integration tests for model pvt_v2 failing with other (#49067) by @sergereview[bot]
• Keep the eos ids the config declares (#49082) by @albertvillanova
• Use grouped_mm on TPU devices under torch.compile (#49097) by @salkan0
• [serge] Fix 2 integration tests for model jamba failing with other (other (2)) (#49044) by @sergereview[bot]
• Remap the legacy Gemma 1 hidden_act in the config post-init (#49084) by @PCfVW
• Fix exporters import on torch < 2.9 (is_contiguous_or_false) (#49124) by @ydshieh
• [gemma3n] fix audio test fixture: use hf_hub_download instead of load_dataset (#49128) by @ydshieh
• qwen3 models map to wrong tokenizer class on the hub (#49116) by @itazap
• Restore the Unicode whitespace set in the GPT-SW3 tokenizer (#48912) by @David-Wu1119
• [AMD] Fix some integration tests (#49153) by @Abdennacer-Badaoui
• [Gemma] Fix test_model_7b_fp16_static_cache expected value for cuda 8 after #49084 (#49133) by @ydshieh
• Fix UMT5 decoder self-attention not being causal (#49135) by @the-cross-art
• Skip flash tests that fall back to a hub kernel when kernels is missing (#49129) by @Abdennacer-Badaoui
• Auto generate model inits (#47829) by @guarin
• Fix get_json_schema dropping items/enum for unions of list/dict/Literal types (#49136) by @JoeyTan21
• Pick the default flash implementation based on the current hardware (#49109) by @Abdennacer-Badaoui
• Deprecate the use_mamba_kernels config flag that no longer has any effect (#49155) by @albertvillanova
• [CI] ssh-runner: add optional cache_type input to switch between bucket and EFS runners (#49159) by @ydshieh
• [generation] Encode multimodal data only once (#45783) by @zucchini-nlp
• No more -hf repo names for ESMC (#49158) by @Rocketknight1
• [MPS] Remove cu_seqlens_k clone workaround for metal-flash-sdpa (#49091) by @Isalia20
• [PerceptionLM] Restore test_inputs_embeds overrides to fix flaky test (#49165) by @ydshieh
• [CB] Fix failing tests discovered when using the B200 (#49171) by @remi-or
• [imagegpt/vilt/trocr] fix cache fixture tests: use hf_hub_download instead of load_dataset (#49178) by @ydshieh
• Fix lost call (#49163) by @molbap
• Document image_like_kwargs (#49180) by @guarin
• [CacheHardIntegrationTest] use dedicated safetensors repo to fix Xet bucket cache corruption (#49182) by @ydshieh
• Document running the example scripts on Hugging Face Jobs (#49050) by @davanstrien
• Fix backslash handling in generate flag values in transformers chat (#48709) by @JHC56
• Finish removing the MPS autocast workaround (#49157) by @rubenG1009
• Fix Trainer checkpoint resume crashing on CPU with multiple processes (#49123) by @neevmodh
• Fix command syntax for optimum-cli export (#46451) by @Ahwar
• QA: fix noisy comment checker (#49184) by @tarekziade
• update to torch 2.14 (#49199) by @sywangyi
• Video processors - general maintenance (#48251) by @zucchini-nlp
• [Parakeet] Convert NeMo's stochastic depth to layerdrop (#49191) by @Deep-unlearning
• [MPS] Let mps sdpa handle grouped query attention directly (#49187) by @Isalia20
• Replace datasets that no longer load with maintained uploads (#49018) by @davanstrien
• Fail fast on eval OOM under auto_find_batch_size (#49198) by @qgallouedec
• Fix SequenceBiasLogitsProcessor edge cases: token id 0 and prefix equal to context (#49117) by @lucaluo925
• Map bare list, tuple and dict annotations to the right JSON schema type in get_json_schema (#49145) by @825pranav
• [Kernels] Sync mamba version (#49205) by @vasqu
• gguf user defined tokens (#49008) by @SunMarc
• QA: restore masking_utils export comment with a noqa (#49209) by @tarekziade
• Fix deepstack features for mixed-input (#49177) by @zucchini-nlp
• Fix stale _added_tokens_encoder entries in cpmant and wav2vec2 (#47440) by @ishan-1010
• Fix additional_special_tokens data loss with extra_special_tokens (#47848) by @erichanwang
• Fix MPS GQA version gating (#49210) by @Isalia20
• Add Strix Halo (gfx1151) Atlas Inference Hub-kernel path for Qwen3.5/3.6/3.8 Gated DeltaNet (#49127) by @AzeezIsh
• Fix MiniMax M3 partial 3D vision rotary embeddings (#49164) by @cuichenx
• Fix missing router_logits in Qwen3.5-MoE and other MoE models (#49179) by @zucchini-nlp
• [Nemotron3Diarization] fix streaming last stft frame dropped (#49167) by @eustlb

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@harshaljanjani

• model: Add GTE to Transformers (#48416)
• [vLLM] Fix video token counting for Transformers backend video inputs (Part 2) (#48900)
• 🚨 [vLLM] Fix video token counting for Transformers backend video inputs (Part 1) (#48894)

@eustlb

• [Nemotron3Diarization] fix streaming last stft frame dropped (#49167)
• [Nemotron3Diarization] nit: hub pr merged to main (#49065)
• Add Nemotron3Diarization (#49056)
• [apply_chat_template] pass sampling_rate to call (#48794)

@tarekziade

• QA: restore masking_utils export comment with a noqa (#49209)
• QA: fix noisy comment checker (#49184)
• QA: Fix llama4 leak (#49042)
• Shorten noisy OpenVINO SDPA comment (#49033)
• Added more usage of MemoryCleanupMixin (#49011)
• fix noisy comments (#49013)
• updated GraniteMoeHybrid expectations (CPU) (#49016)
• QA: applied ruff rule PLW1514 (#48990)
• Contract the mamba2 chunk scan with einsum instead of broadcast-then-sum (#48978)
• QA: deactivate rule 41 (#48852)
• QA: Added a MemoryCleanupMixin class for tests (#48681)
• Synthetic test assets (#48589)

@ydshieh

• [CacheHardIntegrationTest] use dedicated safetensors repo to fix Xet bucket cache corruption (#49182)
• [imagegpt/vilt/trocr] fix cache fixture tests: use hf_hub_download instead of load_dataset (#49178)
• [PerceptionLM] Restore test_inputs_embeds overrides to fix flaky test (#49165)
• [CI] ssh-runner: add optional cache_type input to switch between bucket and EFS runners (#49159)
• [Gemma] Fix test_model_7b_fp16_static_cache expected value for cuda 8 after #49084 (#49133)
• [gemma3n] fix audio test fixture: use hf_hub_download instead of load_dataset (#49128)
• Fix exporters import on torch < 2.9 (is_contiguous_or_false) (#49124)
• [NemotronH-Omni] Fix device mismatch in test tensor creation (#49102)
• [Zamba] Fix associative scan breaking ONNX export and OOM in integration test (#49092)
• Fix Qwen3OmniMoeIntegrationTest OOM (#48987)
• Reduce peak memory in examples_torch CI job (OOM fix) (#48983)
• Fix AXK2 integration test: update CUDA expected text and rename class (#48941)
• Clean up some MoE models' integration tests (#48833)
• Mark test_generate_with_static_cache as flaky for olmo and bigbird_pegasus (#48856)
• Fix GLM rope_parameters dict shared mutation across test instances (#48895)
• Fix Glm4MoeIntegrationTest: split into 3 tests, switch to GLM-4.5-Air (#48820)
• Fix flaky RfDetr test_save_load (#48869)
• docs: fix broken #combining-with-fsdp2 anchor in expert_parallelism.md (#48854)
• Relax static-cache tolerance in test_generate_with_static_cache (1e-5 → 5e-5) (#48815)
• Add integration tests for MuseGlimmerAssistantModel (#48796)
• Fix Glm4vMoeIntegrationTest: offload_folder + MemoryCleanupMixin (#48776)
• Switch daily CI to torch 2.14 — update expected outputs (#48750)
• [DeepseekV3] OOM cascade root-cause investigation (generator ref leak in conversion_mapping) (#48720)
• [CI] Replace hardcoded username allowlists with author_association check in workflow triggers (#48712)
• [MusicgenMelody] Fix conditioning silently dropped at generation step 0 (#48679)
• [fix] Fix GlmOcr integration tests: wrong token IDs and image token decode bug (#48650)
• [tests] Fix NougatModelIntegrationTest: pin artifact revision and update golden values (#48638)
• [CI] Deduplicate Nvidia/AMD CI reply comments and add headers (#48655)

@molbap

• Fix lost call (#49163)
• 🚨 🚨Bring some dinos to modern standards (#46266)
• [Fix] yolos offload issue (#48688)

@remi-or

• [CB] Fix failing tests discovered when using the B200 (#49171)
• [CB] [Major] Upgrade the cache to support different attention types (#47809)
• [Improvement] Rework the docstring and comments of mHC (#48888)
• [CB] Add pause mechanism (#48462)
• [Fix] Clean-up ternaries in the DeepSeek family (#48447)

@jiqing-feng

• Keep attention_mask as None in OPT's causal mask creation (#49002)
• Register the remaining mamba-ssm kernel layers on XPU (#49035)
• Only seed numpy in BigBird block-sparse attention during training (#49037)
• Fix mask creation not being skipped under torch.compile (#48975)
• Fix StaticCache for Mllama and enable torch.compile (#48141)
• Fix reset on the dynamic cache layers (#48809)
• Cast pixel values to the patch embedding dtype in DeepSeek-OCR-2 (#48632)
• Fix TextToAudioPipeline crash for tokenizer-only models (#48505)
• Fix slow integration tests on XPU (#48611)
• Register activation kernel layers on XPU (#47858)

@Wauplin

• Bump huggingface_hub upper bound to <3.0 (transformers) (#49083)
• [docs] Fix legacy hf CLI references (transformers) (#48988)
• Resolve the Hub revision once per load instead of passing a private _commit_hash around (#47611)
• Use huggingface_hub httpx export (#48685)

@meatybobby

• Add support for Nemotron Omni (#46509)

@Sainava

• Add pose estimation keypoint preprocessing to Sapiens2ImageProcessor (#47199)

@jp1924

• add HyperClovaX Vision (#44314)

Original source

Release v5.17.0

New Model additions

HYV4

Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every token to 8 of them. The context window is 1M tokens.

The architecture combines four features:

Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent (kv_lora_rank) that kv_b_proj expands back to one key/value per query head.

DeepSeek Sparse Attention (DSA) selects index_topk keys per query with a lightweight indexer.

Following IndexShare, only the layers marked "full" in indexer_types run an indexer; "shared" layers reuse the previous full layer's selection.

Gated MLA with learnable attention sinks, where each head owns a sink logit that participates in the softmax and contributes no value, as in GPT-OSS.

Independent Hyper-Connections (iHC) replace the plain residual path with hc_mult parallel residual streams that are collapsed before, and redistributed after, every sublayer.

The implementation does not execute the multi-token prediction (MTP) layers. Released checkpoints keep those weights so that other runtimes can use them for speculative decoding; they are ignored at load time.

Links: Documentation

Add h4 (#48473) by @ArthurZucker in #48473

VibeVoice

VibeVoice is a novel framework for synthesizing high-fidelity, long-form speech with multiple speakers by employing a next-token diffusion approach within a Large Language Model (LLM) structure. It's designed to capture the authentic conversational "vibe" and is particularly suited for generating audio content like podcasts and multi-participant audiobooks.

Links: Documentation

Implement VibeVoice (#40546) by @pengzhiliang in #40546

NeoMME

NeoMME is a family of efficient 260M and 800M parameter multimodal-native multilingual foundation encoders from H Company. It processes multilingual text tokens and raw image patches in a single bidirectional Transformer encoder, without a separately pretrained vision tower or causal language model.

NeoMME-Retriever is a model fine-tuned from the NeoMME backbone for visual document retrieval with joint late-interaction and dense objectives. It takes text queries and documents (text or page screenshots) and produces multi-vector embeddings for MeanMaxSim scoring (late-interaction) and mean-pooled embeddings for cosine similarity (dense).

Links: Documentation

Add NeoMME and NeoMME-Retriever (#47992) by @tonywu71 in #47992

Fun-ASR-Nano

Fun-ASR-Nano is an 800M-parameter end-to-end speech recognition model developed by Alibaba DAMO Academy's FunAudioLLM team. It achieves state-of-the-art performance on Chinese, English, and Japanese ASR benchmarks while being significantly smaller than comparable models.

Key features are

Chinese, English, and Japanese, including 7 Chinese dialects and 26 regional accents

Hotword customization for domain-specific vocabulary

Native punctuation output (no separate punctuation model needed)

Links: Documentation

Add Fun-ASR-Nano model (#46180) by @LauraGPT in #46180

KimiLinear

Kimi Linear is a hybrid linear attention architecture from Moonshot AI, introduced in Kimi Linear: An Expressive, Efficient Attention Architecture.

At its core is Kimi Delta Attention (KDA), a refinement of Gated DeltaNet that gives each key channel its own forget gate, so the recurrent state decays per channel instead of per head. KDA is used in most layers; every fourth layer keeps a full-attention block that reuses DeepSeek-V3's Multi-head Latent Attention (MLA), and the feed-forward blocks are DeepSeek-V3-style MoE with a shared expert.

Links: Documentation

Kimi linear (#48250) by @remi-or in #48250

Canary

Canary-1B-v2, a fast, robust multilingual model for Automatic Speech Recognition (ASR) and Speech-to-Text Translation (AST):

Canary reuses the Fast Conformer encoder from Parakeet (loaded through [ParakeetEncoder] / [ParakeetEncoderConfig]) and pairs it with a Transformer decoder that uses fixed sinusoidal positional embeddings, cross-attention to the encoder outputs and tied input/output embeddings. The task is selected through a decoder prompt prefix built by [CanaryProcessor] of the form <|startofcontext|> <|startoftranscript|> <|emo:undefined|> <source_lang> <target_lang> <pnc|nopnc> <|noitn|> <|notimestamp|> <|nodiarize|>, where source_lang == target_lang selects transcription and otherwise selects translation.

Links: Documentation

model: Add NVIDIA Canary-1B-v2 to Transformers (#46825) by @harshaljanjani in #46825

NeuCodec

The NeuCodec model was proposed in Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates.

NeuCodec is a neural audio codec extending on XCodec2. It takes advantage of the following features:

Finite Scalar Quantization (FSQ) quantisation resulting in a single codebook, making it ideal for downstream modeling with Speech Language Models.

Trained with CC data such that there are no Non-Commercial data restrictions.

At 50 tokens/sec and 16 bits per token, the overall bit-rate is 0.8kbps.

The codec takes in 16kHz input and outputs 24kHz using an upsampling decoder.

The FSQ encoding scheme allows for bit-level error resistance suitable for unreliable and noisy channels.

Links: Documentation

Add support for NeuCodec (#47143) by @harryjulian in #47143

Breaking changes

Vision rotary embeddings (2D/3D) have been standardized into a unified RoPE frequency computation module, so users with custom vision models relying on attention-layer-level or model-specific RoPE grid interleaving logic must migrate to the new centralized modeling_rope_utils.py implementation.

🚨 Vision (2d/3d) rotary embeddings (#48105) by @zucchini-nlp

Generation

Generation improvements include a performance optimization that avoids unnecessary accelerator synchronization on every decode step (reducing per-step overhead), and a fix to prevent unconditional downloading of remote hub files during generation. Several correctness fixes were also applied, including enforcing auto-compile cache checks for encoder-decoder models, standardizing past_key_values naming in AfMoE, and resolving flaky export and integration test failures.

[Generate] Avoid unconditionally downloading remote hub file (#48620) by @vasqu in [#48620]

[generate] stop synchronizing the accelerator on every decode step (#47975) by @SunMarc in [#47975]

[AfMoE] Standardize past_key_values argument naming across forward and generate (#48430) by @shenhuaqingshi in [#48430]

Fix MTP generation test regex gate for escaped layer ignore keys (#48003) (#48262) by @Noxtimo in [#48262]

fix(generation): Enforce the auto-compile cache check for encoder-decoder models (#48364) by @harshaljanjani in [#48364]

[serge] Fix 4 integration tests for model generation failing with output_mismatch (list output differs (4)) (#48133) by @sergereview[bot] in [#48133]

[VibeVoice] Skip generate export tests (flaky) (#48396) by @ydshieh in [#48396]

Cache

Fixed several cache-related bugs, including a quantized cache issue in VibeVoice, incorrect rejection of non-static cache implementations in VoxtralRealtime, missing auto-compile cache checks for encoder-decoder models, and a silent failure when paged attention is called without a cache. Documentation was also updated to clarify ContinuousBatchingConfig usage and sliding window model limitations.

vibevoice: fix bug for quant cache (#48487) by @kaixuanliu in [#48487]

Fix VoxtralRealtime rejecting non-static cache implementations (#48082) by @jiqing-feng in [#48082]

Raise when a paged attention forward is called with no cache (#48297) by @qgallouedec in [#48297]

[docs] Pass ContinuousBatchingConfig and sliding window models (#48381) by @stevhliu in [#48381]

Retry get_daily_ci_runs on stale GitHub API cache (#48374) by @ydshieh in [#48374]

Kernels

Kernel support was improved with fixes for nested FLA kernel imports when only fla-core is installed, a warning when hub-kernel functions silently fall back to slower pure-PyTorch reference implementations, and the ability to register standalone functions (e.g., RoPE) in KernelConfig with optional non-inheritance of default mappings. Additional fixes include corrected repository paths for ESMFold2 kernels and updated documentation for KernelConfig customization.

Support nested FLA kernel imports for fla-core (#48221) by @DimensionSTP in [#48221]

Warn once when a hub-kernel function falls back to its reference PyTorch path (#48185) by @qgallouedec in [#48185]

[docs] Kernel updates (#48465) by @stevhliu in [#48465]

[Kernels] Enable functions into kernels registry and allow non inheritance (#48443) by @vasqu in [#48443]

Fix kernel commit and repo paths for ESMFold2 (#48186) by @Rocketknight1 in [#48186]

Quantization

Fixed several quantization bugs, including a quant cache issue in VibeVoice, incorrect FP8 embedding handling for Qwen models, missing FP8 tensor parallelism layer overrides, and unnecessary MXFP4 weight dequantization on XPU devices.

fix qwen4exp-fp8 ple embedding (#48368) by @JJJYmmm in [#48368]

Keep MXFP4 weights quantized on XPU when use_kernels is set (#47923) by @jiqing-feng in [#47923]

Fix missing FP8 TP layer overrides (#48343) by @changwangss in [#48343]

Bugfixes and improvements

MRoPE continued (#48594) by @zucchini-nlp in [#48594]

[fix] Update stale expected strings in HunYuanVL integration tests (#48646) by @ydshieh in [#48646]

[Quantizaiton]support 5/6/7 bits in AutoRound (#48481) by @wenhuach21 in [#48481]

[fix] Update stale golden values and fix expected_logits shape in FlavaForPreTraining integration tests (#48639) by @ydshieh in [#48639]

Fix YOLOS device mismatch with device_map="auto" (#46886) by @swankystark in [#46886]

Honor shift_labels in decoder-only LLM/VLM losses (#48493) by @qgallouedec in [#48493]

[KimiLinear] Fix test_cpu_offload: set num_local_experts=4 in model tester (#48624) by @ydshieh in [#48624]

[docs] Per-layer config (#48601) by @stevhliu in [#48601]

Fix generate_flags parsing in transformers chat (#48597) by @SunMarc in [#48597]

[fix] Fix how we read package versions - triggered by torch 2.14+ (#48615) by @ydshieh in [#48615]

[Docker] Upgrade CPU torch to <=2.14.0, torchcodec to <=0.16.0 (#48614) by @ydshieh in [#48614]

Fix AttributeError in gradient_checkpointing_enable(offload=True) (#48590) by @tarekziade in [#48590]

Another day fixing CI (#48591) by @zucchini-nlp in [#48591]

esmfold2: keep distogram_head in fp32 as well (#48488) by @kaixuanliu in [#48488]

Add supports_context_parallel to PreTrainedModel (#48442) by @qgallouedec in [#48442]

[docs] mlinter reference (#48460) by @stevhliu in [#48460]

[docs] Add a LiteRT page under community integrations (#48540) by @john-rocky in [#48540]

[GLM 5.3 Flash] Fix NaN gradients in chunked KDA (#48455) by @imvladikon in [#48455]

[docs] Fix [[autodoc]] directives in ALBERT model documentation (#48593) by @samyuktahegde in [#48593]

[Fix] Fix A10 expectations for a test (#48454) by @remi-or in [#48454]

Add PR comment CI for AMD (MI300) (#48065) by @ydshieh in [#48065]

docs: fix docstring parameter names that do not match signatures (#48575) by @simpleqt in [#48575]

docs: remove phantom parameters from docstrings (#48576) by @simpleqt in [#48576]

extend some case to xpu as well (#48502) by @sywangyi in [#48502]

[serge] Fix 2 integration tests for model glm4_moe failing with OOM (other (2)) (#48551) by @sergereview[bot] in [#48551]

[serge] Fix 2 integration tests for model nemotron failing with import_or_config (other (2)) (#48582) by @sergereview[bot] in [#48582]

Compress the agent conventions file and document two modular pitfalls (#48586) by @tarekziade in [#48586]

Guard against a None video processor class when the backend is unavailable (#48557) by @caiotheodoro in [#48557]

[serge] Fix 2 integration tests regressed by commit 83d46aa (PR #47625) (#48580) by @sergereview[bot] in [#48580]

[nit] use requires_backends (#47576) by @eustlb in [#47576]

[serge] Fix 2 integration tests for model kosmos2 failing with import_or_config (other (2)) (#48552) by @sergereview[bot] in [#48552]

Fix failing tests for cohere_compass (#48005) by @kaixuanliu in [#48005]

[Fix] Use dedicated helpers for DeepGEMM and SonicMoE tests (#48523) by @remi-or in [#48523]

CI: Point test fixtures at hf-internal-testing copies we already host (#48521) by @tarekziade in [#48521]

fix failed test cases for glm5_next (#48497) by @kaixuanliu in [#48497]

Pass kwargs to the Mamba2 mixer in Nemotron-H, Falcon-H1 and Mamba2 (#48490) by @kfastino in [#48490]

Retire test_multi_gpu_data_parallel_forward (#48508) by @tarekziade in [#48508]

Processing tests [part 2] (#47922) by @zucchini-nlp in [#47922]

QA: Add noisy comment checker (#48484) by @tarekziade in [#48484]

Allow nested rope params for tiny models (#48435) by @zucchini-nlp in [#48435]

Fix sliding-window mask layer_idx in Gemma3/Gemma4 create_masks_for_vision_model (#48482) by @jiqing-feng in [#48482]

add xpu expectations for hunyuan_vl model tests (#48504) by @kaixuanliu in [#48504]

[Qwen 3.5 Moe] Fix decorators (#48436) by @vasqu in [#48436]

Infinite loop in dependency search (#48393) by @zucchini-nlp in [#48393]

[serge] Fix 2 integration tests for model fsmt failing with output_mismatch (tensor values differ (2)) (#48496) by @sergereview[bot] in [#48496]

Fix some tests by removing the deprecation cycle (#48503) by @Cyrilvallez in [#48503]

Remove deprecation (#48500) by @Cyrilvallez in [#48500]

Fix Pix2StructTextAttention init using hidden_size instead of d_kv (#47558) by @ in [#47558]

Fix pre patch release utility (#48499) by @Cyrilvallez in [#48499]

Update dev version (#48498) by @Cyrilvallez in [#48498]

Fix Inkling inputs_embeds and add more tests (#47827) by @Cyrilvallez in [#47827]

Simplify and fix qwen4 tests (#48340) by @Cyrilvallez in [#48340]

[docs] Partial checkpointing and group_by_length (#48463) by @stevhliu in [#48463]

doc: fix syntax error and typos in VibeVoice documentation (#48489) by @VimalN2005 in [#48489]

[Qwen4 Exp] Use partial to avoid skipping mask more easily (#48456) by @vasqu in [#48456]

Add support for NeuCodec (#47143) by @harryjulian in [#47143]

[serge] Fix 1 integration tests regressed by commit bd95093 (PR #47493) (#48426) by @sergereview[bot] in [#48426]

Fix rotary embedding regression (#48477) by @Cyrilvallez in [#48477]

Remove deprecated mask functions (#48476) by @Cyrilvallez in [#48476]

[MTP] Save memory by only capturing the last layer's hidden_states (#48475) by @Cyrilvallez in [#48475]

Allow capturing only necessary hidden_states with capture_outputs (#48081) by @sywangyi in [#48081]

Support per-layer MTP configuration (#48264) by @eladsegal in [#48264]

[docs] Fix code snippets (#47772) by @stevhliu in [#47772]

[Fix] Sparse TikToken tokenizers silently fail (#48446) by @remi-or in [#48446]

No inherit decorator for NeoMME (#48457) by @zucchini-nlp in [#48457]

Batch Rebalance Data Sampler (#47340) by @delock in [#47340]

[serge] Fix 2 integration tests for model cwm failing with import_or_config (other (2)) (#48414) by @sergereview[bot] in [#48414]

fix: Add DEIMv2 attribution (#48448) by @harshaljanjani in [#48448]

[fix] inkling: mps + cuda mel spec extraction (#47432) by @eustlb in [#47432]

fix: decode() batch path respects self.clean_up_tokenization_spaces (#47793) by @lorenzozanee in [#47793]

Grounding dino fp16 dtype [backlog] (#48438) by @molbap in [#48438]

Init the process group with a load-scaled timeout for sharded loading (#48228) by @qgallouedec in [#48228]

Raise a clear error when a token is both forced and suppressed (#47511) by @qgallouedec in [#47511]

Clarify device placement in pipelines (#47367) by @LysandreJik in [#47367]

Add offload to gradient checkpointing (#48444) by @qgallouedec in [#48444]

[MiniCPMV4_6] Update test_small_model_vision_generation_batch expected output (value drift) (#48406) by @ydshieh in [#48406]

[serge] Fix 2 integration tests for model hyperclovax failing with other (other (2)) (#48440) by @sergereview[bot] in [#48440]

fix(models): Drop the position-indexed token type lookup in RoPE encoders (#48407) by @harshaljanjani in [#48407]

Document image_hidden_states/pixel_values mutual exclusivity for SmolVLM/Idefics2/Idefics3 (#47714) by @verma8076 in [#47714]

Re-order a bit for easier navigation (#48434) by @zucchini-nlp in [#48434]

Deprecated stuff gone (#48367) by @zucchini-nlp in [#48367]

[serge] Fix 6 integration tests for model seamless_m4t_v2 failing with other (other (6)) (#48425) by @sergereview[bot] in [#48425]

[Docs]: Update GLM 5.3 (#48401) by @Dovis01 in [#48401]

Avoid print to stdout that fails the job check_failed_tests job (#48391) by @ydshieh in [#48391]

Fix incorrect tuple return annotations on forward methods returning a Tensor (#48359) by @Gronoxx in [#48359]

Fix interval merge invariant in _find_disjoint (#47860) by @sharmax-vikas in [#47860]

skip mtp slow tests for now (#48328) (#48329) by @tarekziade in [#48329]

Update Tailscale action version in workflow (#48394) by @glegendre01 in [#48394]

fix some failure in xpu (#48252) by @sywangyi in [#48252]

[Improvement] Make gated delta rule more explicit (#47625) by @remi-or in [#47625]

[CB] Fix wrong device scoping (#48370) by @remi-or in [#48370]

Bump transformers-mlinter to 0.1.5 and clear the new findings (#48259) by @tarekziade in [#48259]

fix: flash-attn fallback failing on torch2.13 (#48388) by @NanoCode012 in [#48388]

[LongcatFlash] Fix test_longcat_generation_cpu: use device_map="cpu" to avoid MoE disk offload issue (#48377) by @ydshieh in [#48377]

[Qwen3VLMoe] Update test_small_model_integration_test_batch expected output (value drift) (#48376) by @ydshieh in [#48376]

[ONNX] Skip affected models on torch 2.13 (two dynamo regressions) (#48191) by @ydshieh in [#48191]

Fix safe_open mmap memory exhaustion on Windows by using pread backend (#48341) by @eryk-roch in [#48341]

Fix Zamba2 construction for num_mem_blocks > 1 checkpoints (#48325) by @john-rocky in [#48325]

[Docs] Change 5.3 Flash pos in toc (#48366) by @Dovis01 in [#48366]

[conftest] Use get_cpu_ram_total_gib for psutil patch (cgroup-aware) (#48290) by @ydshieh in [#48290]

Fix flaky test_training_gradient_checkpointing for BigBirdPegasus (fp noise filter) (#48332) by @ydshieh in [#48332]

Quiet continuous batching at default verbosity (#48314) by @qgallouedec in [#48314]

Wait for the first request in the async continuous batching bootstrap (#48304) by @qgallouedec in [#48304]

Ignore a stale best checkpoint recorded in a resumed trainer state (#48319) by @VaggelisGian in [#48319]

[docs] Fix links and remove TokenizerFast (#47748) by @stevhliu in [#47748]

Fix incorrect token classification prefix for ESMC (#48348) by @Rocketknight1 in [#48348]

Create the continuous batching CPU group with local synchronization (#48302) by @qgallouedec in [#48302]

Resolve continuous batching config against the text config for composite models (#48299) by @qgallouedec in [#48299]

[CI] Unblock fast CI for now (failing tests) (#48344) by @vasqu in [#48344]

[qwen4_exp] disable torch/onnx export tests due to data-dependent control flow (#48345) by @ydshieh in [#48345]

[debug] Trace previous CI run selection in get_previous_daily_ci.py (#48338) by @ydshieh in [#48338]

Normalize HunYuanVL's legacy field aliases via attribute_map (#48261) by @hmellor in [#48261]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@ydshieh

[fix] Update stale expected strings in HunYuanVL integration tests (#48646)

[fix] Update stale golden values and fix expected_logits shape in FlavaForPreTraining integration tests (#48639)

[tests] Fix integration test golden values broken by fast image processor default (PR #41388) (#48637)

[KimiLinear] Fix test_cpu_offload: set num_local_experts=4 in model tester (#48624)

Fix GPU memory teardown in CLI serve tests (#48618)

[fix] Fix how we read package versions - triggered by torch 2.14+ (#48615)

[Docker] Upgrade CPU torch to <=2.14.0, torchcodec to <=0.16.0 (#48614)

Add PR comment CI for AMD (MI300) (#48065)

[MiniCPMV4_6] Update test_small_model_vision_generation_batch expected output (value drift) (#48406)

Avoid print to stdout that fails the job check_failed_tests job (#48391)

[VibeVoice] Skip generate export tests (flaky) (#48396)

[LongcatFlash] Fix test_longcat_generation_cpu: use device_map="cpu" to avoid MoE disk offload issue (#48377)

[Qwen3VLMoe] Update test_small_model_integration_test_batch expected output (value drift) (#48376)

[ONNX] Skip affected models on torch 2.13 (two dynamo regressions) (#48191)

Retry get_daily_ci_runs on stale GitHub API cache (#48374)

[conftest] Use get_cpu_ram_total_gib for psutil patch (cgroup-aware) (#48290)

Fix flaky test_training_gradient_checkpointing for BigBirdPegasus (fp noise filter) (#48332)

[qwen4_exp] disable torch/onnx export tests due to data-dependent control flow (#48345)

[debug] Trace previous CI run selection in get_previous_daily_ci.py (#48338)

@LauraGPT

Add Fun-ASR-Nano model (#46180)

@tarekziade

Fix AttributeError in gradient_checkpointing_enable(offload=True) (#48590)

Compress the agent conventions file and document two modular pitfalls (#48586)

CI: Point test fixtures at hf-internal-testing copies we already host (#48521)

Retire test_multi_gpu_data_parallel_forward (#48508)

QA: Add noisy comment checker (#48484)

skip mtp slow tests for now (#48328) (#48329)

Bump transformers-mlinter to 0.1.5 and clear the new findings (#48259)

@remi-or

[Fix] Fix A10 expectations for a test (#48454)

Kimi linear (#48250)

[Fix] Use dedicated helpers for DeepGEMM and SonicMoE tests (#48523)

[Fix] Sparse TikToken tokenizers silently fail (#48446)

[Improvement] Make gated delta rule more explicit (#47625)

[CB] Fix wrong device scoping (#48370)

[CB] Fail faster (#48334)

@ArthurZucker

Add h4 (#48473)

@harryjulian

Add support for NeuCodec (#47143)

@delock

Batch Rebalance Data Sampler (#47340)

@harshaljanjani

fix: Add DEIMv2 attribution (#48448)

model: Add NVIDIA Canary-1B-v2 to Transformers (#46825)

fix(generation): Enforce the auto-compile cache check for encoder-decoder models (#48364)

fix(models): Drop the position-indexed token type lookup in RoPE encoders (#48407)

@tonywu71

Add NeoMME and NeoMME-Retriever (#47992)

@Dovis01

[Docs]: Update GLM 5.3 (#48401)

[Docs] Change 5.3 Flash pos in toc (#48366)

[Glm 5.3 Flash] GLM 5.3 Flash Support (#48342)

@pengzhiliang

Implement VibeVoice (#40546)

Original source

Release v5.16.1

This is a special release as we include GLM! (and a few small fixes)

GLM-5.3-Flash

GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.

GLM-5.3-Flash starts from a newly trained base model, with its architecture and training recipe redesigned around capability and efficiency. For the first time in the GLM series, we introduce a hybrid architecture combining sparse and linear attention, sharply reducing long-context serving costs while preserving precise long-context capabilities. The model also adopts Manifold-Constrained Hyper-Connections (mHC) to further improve scaling efficiency. Together with our latest 30T-token multimodal pre-training corpus, these changes enable GLM-5.3-Flash to deliver more intelligence with less compute.

Links: Documentation

[Glm 5.3 Flash] GLM 5.3 Flash Support (#48342) by @Dovis01 in #48342

Small patch fixes

Mainly BC behavior for TP and pinning a hf kernel for security reasons 🤗

Restore BC for the tensor-parallel API (#48300) by @ArthurZucker

Fix kernel commit and repo paths for ESMFold2 (#48186) by @Rocketknight1

Full Changelog: v5.16.0...v5.16.1

Original source

Release v5.16.0

New Model additions

Qwen4-Exp

Qwen4-Exp builds on Qwen3.5's hybrid text and multimodal architecture with three key components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE).

GR is a Qwen-developed residual architecture that combines Hyper-Connection with GatedNorm. It mixes multiple residual streams with fine-grained elementwise gating before each attention and Mixture-of-Experts (MoE) block, then controls how much of the block output is injected back into each stream.

QSA uses multiple query heads to score compressed key blocks, selects the most relevant contiguous token blocks, and keeps the incomplete trailing block uncompressed. This block-level selection reduces indexing overhead and improves memory locality for long sequences. Combined with Gated DeltaNet, QSA makes Qwen4-Exp the first hybrid architecture to integrate linear and sparse attention, substantially improving inference efficiency for long-context workloads.

PLE enriches selected decoder layers with layer-specific lexical features derived from hashed token n-grams and a dilated depthwise convolution.

Links: Documentation

Add Qwen4Exp model (#48337) by @Cyrilvallez in #48337

GraniteSpeech5

Granite Speech 5.0 Turbo CTC is a lightweight (~470M parameters) conformer encoder for automatic speech recognition, trained with Connectionist Temporal Classification (CTC) on BPE targets. It is a fast, encoder-only member of the Granite Speech family: transcription requires a single forward pass followed by greedy CTC decoding, with no autoregressive decoder.

Architecturally, it extends the Granite Speech conformer CTC encoder with:

• Frame stacking + block-wise time subsampling: the feature extractor stacks pairs of log-mel(+delta) frames (2x), and the first two conformer blocks each subsample time by 2 through a stride-2 depthwise convolution (with a mean-pooled residual), for a total 8x time reduction at 10 ms mel hop.
• Block attention with Shaw's relative positional embeddings: attention is computed over fixed-size blocks (the sequence is right-padded to a whole number of blocks, with padded frames masked out), using separate bias-free query/key/value projections.
• Self-conditioned CTC: the CTC posteriors of the middle layer are projected and fed back into the hidden states, and the CTC head is shared between this mid-layer self-conditioning and the final prediction.

Links: Documentation

Add Granite Speech 5.0 - (#48288) by @eustlb in #48288

Step3p7

Step-3.7-Flash was proposed in Step 3.7 Flash by StepFun. It is a 198B-parameter sparse Mixture-of-Experts vision-language model, pairing a 196B-parameter MoE language backbone with a 1.8B-parameter vision encoder for native image understanding.

StepFun hasn't published a technical report for Step-3.7-Flash, so the details below are drawn from the released checkpoint's configuration rather than a paper.

• Sparse MoE decoder: all but the first 3 decoder layers route through a MoE block of 288 routed experts (top-8 per token) plus a single shared expert. The router scores experts with a sigmoid and a learned per-expert bias instead of an auxiliary load-balancing loss, the same strategy as DeepSeek-V3.
• Gated attention: each attention layer adds an extra projection whose sigmoid output gates the attention output per head, before the output projection — the same Gated Attention mechanism used in Qwen3-Next. A subset of layers use fewer heads and a sliding window instead of full attention.
• Multi-token prediction: some checkpoints ship extra decoder layers trained for multi-token prediction, which [~GenerationMixin.generate] can use for speculative decoding via use_mtp=True.
• Vision encoder: a SigLIP-style ViT with 2-D rotary position embeddings and a learned per-layer scale on the attention and MLP branches. Its output is downsampled 4x by two stride-2 convolutions before a linear projector maps it into the text model's hidden size.
• Dynamic image tiling: instead of a fixed tile grid, the image processor picks its tiling window from each image's own aspect ratio, producing one downscaled global view plus zero or more local high-resolution crops per image.

Links: Documentation

[new model] step 3.7 (#46658) by @itazap in #46658

CohereCompass

CohereCompass is the base architecture for small, specialized (vision-)language models trained by Cohere.

Links: Documentation

Add CohereCompass modeling (#47878) by @calpt in #47878

ESMC and ESMFold2

ESMC and ESMFold2 are new state-of-the-art protein language and folding models from BioHub. ESMC is trained with a masked language modeling objective, and it can be easily transferred to sequence and token classification tasks for proteins. Checkpoints exist in various sizes, from 300M parameters up to 6B parameters. It works as a drop-in replacement for older ESM-2 and ESM-3 models, with significantly higher accuracy.

ESMFold2 is a state-of-the-art protein folding model which produces high accuracy predictions. It uses an iterated diffusion approach that is significantly different from the original ESMFold, offering huge improvements in accuracy for more complex structures.

Links: Documentation ESMC, Documentation ESMFold2

Port ESMC and ESMFold2 to Transformers (#46419) by @Rocketknight1 in #46419

Breaking changes

The legacy tensor-parallel implementation has been replaced with a DTensor-native backend, so users relying on the previous TP API for inference or training must migrate to the new DTensor-based interface.

🚨 TP dtensor API inference + training (#47579) by @3outeille

attn_implementation="sdpa" dispatch is now properly supported for wav2vec2_conformer, wav2vec2-bert, and SeamlessM4T/v2 models, which may change initialization behavior for users who previously worked around this limitation.

🚨[wav2vec2] Support attn_implementation=sdpa dispatch (#46196) by @YangKai0616

FuyuProcessor no longer returns the image_patch_indices output, so any code that depends on this field must be updated to remove references to it.

🚨 Leftover processors (#47924) by @zucchini-nlp

Cache

Several cache-related bugs were fixed in this release, including an off-by-one error in the sliding window cache, Whisper speculative decoding cache corruption, CpmAnt use-cache failures, Qwen2.5-Omni/Qwen3-Omni-MoE generation with compilable caches, and compressed-tensors loading for KV-cache-only quantized models. Documentation was also added for cache token removal using negative values, and per-layer cache configuration support (allowing models to use different cache settings per layer) was introduced.

Cpmant fix use cache (#48013) by @jiqing-feng in [#48013]

[docs] Cache crop (#47950) by @stevhliu in [#47950]

Revert "Support per-layer cache configuration and attention-mask selection" (#48175) by @Cyrilvallez in [#48175]

Support per-layer cache configuration and attention-mask selection (#47901) by @eladsegal in [#47901]

Fix Qwen2.5-Omni / Qwen3-Omni-MoE generation with a compilable cache (#47872) by @jiqing-feng in [#47872]

Fix sliding window cache index off-by-one on wraparound (#47708) by @hameedibrh in [#47708]

[Whisper] Fix speculative decoding: UnboundLocalError, cache corruption, and speed regression (#48000) by @ydshieh in [#48000]

Fix compressed-tensors loading for KV-cache-only quantized models (#47904) by @kylesayrs in [#47904]

Generation

This release fixes several generation bugs across multiple models, including Whisper speculative decoding issues (UnboundLocalError, cache corruption, speed regression, and left-padded batch position IDs), broken image generation in Emu3, garbage output in OLMo/GPTNeoX, and Qwen2.5-Omni/Qwen3-Omni-MoE generation with compilable caches. Additionally, logit distributions for candidate generators using sampling are now aligned by returning logits after applying logit processors.

[Whisper] Fix speculative decoding: preserve cleared suppress tokens through super().generate() (#48108) by @ydshieh in [#48108]

[Whisper] Fix decoder position IDs for left-padded batches in longform generation (#48028) by @ydshieh in [#48028]

[serge] Fix 2 integration tests for model generation failing with import_or_config (other (2)) (#48061) by @sergereview[bot] in [#48061]

Align logit distributions for CandidateGenerators using sampling (#48007) by @Cyrilvallez in [#48007]

[GPTNeoX] Fix post_processor not overridden when loading from pretrained (OLMo garbage generation) (#47988) by @ydshieh in [#47988]

[emu3] 🦮 Black Labrador is back! Fix image generation broken since #37033 (#47948) by @ydshieh in [#47948]

Attention

Several attention-related bug fixes were made in this release, including correcting a SigLIP2 documentation typo, fixing Flash/SDPA attention dispatch tests for xcodec2 and ROCm RDNA GPUs, resolving a GPT2 cross-attention mask being silently discarded, and enabling SDPA support declaration in TimmWrapper. Per-layer cache configuration and attention-mask selection support was also introduced, allowing models with heterogeneous layer configurations to use distinct sliding_window, attention_chunk_size, and number_of_conv_states values per layer.

doc: Fix typo in SigLIP2 Flash Attention code example (#48197) by @VimalN2005 in [#48197]

[xcodec2] Fix flex attention and flash dispatch tests (#48244) by @jiqing-feng in [#48244]

Fix ROCm SDPA-flash skip guard that crashes on RDNA GPUs (#47965) by @Abdennacer-Badaoui in [#47965]

[GPT2] Fix encoder_attention_mask being silently discarded in cross-attention (#47946) by @DavidJohnQuinlan in [#47946]

Declare sdpa support in TimmWrapper (#47939) by @jiqing-feng in [#47939]

Quantization

Quantization improvements include adding NVFP4 quantization support via HF kernels (enabling on-the-fly BF16 weight quantization with ~50% memory reduction), and fixing several bugs: reverting a regression in is_quantization_compressed that caused incorrect module layouts for packed-format checkpoints, fixing CLIP weight initialization failures with quantized checkpoints, and restoring KV-cache quantization setup for KV-cache-only quantized models.

Revert "[Quantization]: Refactor is_quantization_compressed for format-based detection" (#48072) by @subin9 in [#48072]

feat: add nvfp4 quantization (#47883) by @drbh in [#47883]

[DeepSeekV2] Fix integration tests OOM: use device_map=auto instead of 8-bit quantization (#47991) by @ydshieh in [#47991]

Fix CLIP _init_weights when a child module carries quantized weights (#47921) by @Bluear7878 in [#47921]

Parallelization

Introduced a naive pipeline parallel inference engine supporting tied/untied weight embeddings with seamless generate() integration, while restoring backward compatibility for the tensor-parallel API with a deprecation cycle for tp_plan in from_pretrained(). Additionally fixed a model parallel bug in the BLT model affecting beam search.

Restore BC for the tensor-parallel API (#48300) by @ArthurZucker in [#48300]

fix bug for blt model parallel bug (#48327) by @kaixuanliu in [#48327]

Pipeline parallel naive inference (#47289) by @3outeille in [#47289]

Kernels

Kernel support was improved with documentation updates highlighting supported models, a fix for export crashes on kernel-decorated functions by adding a is_torchdynamo_exporting guard, and the default Flash Attention 2 hub kernel version was bumped to v3 to resolve compatibility issues with newer PyTorch versions.

[docs] Kernel supported models (#48258) by @stevhliu in [#48258]

[Fix] Export crashes on kernel-decorated function (#47808) by @remi-or in [#47808]

Bump default flash-attn2 hub kernel version to v3 (#47863) by @jiqing-feng in [#47863]

Bugfixes and improvements

Fix video-llama modular conversion (#48336) by @zucchini-nlp in [#48336]

CI: gate the hunyuan-moe slow test (#48330) by @tarekziade in [#48330]

Add a regression test for force_accelerate_hooks signature preservation (#48260) by @wtdcode in [#48260]

Add an opt-in per-frame pixel cap (cap_pixels_per_frame) to the Qwen3-VL video processor (#48071) by @dkrisman in [#48071]

Fix scores type in stopping criteria docstrings (#47676) by @qgallouedec in [#47676]

Docstring check didn't match some file - fix it (#48121) by @zucchini-nlp in [#48121]

Add shared ImageProcessingTester (#47745) by @guarin in [#47745]

Fix AutoTokenizer returning TokenizersBackend for DeepSeek-R1-Distill-Qwen models (#48211) by @ydshieh in [#48211]

[docs] Fix failing doctests (#47687) by @stevhliu in [#47687]

Fix build_2d_sinusoidal_position_embedding on MPS (#47897) by @guarin in [#47897]

[Gemma4] Investigate flaky test_generation_beyond_sliding_window_1_eager (#48236) by @ydshieh in [#48236]

Let gradient checkpointing skip layers with every_n_layers (#48200) by @qgallouedec in [#48200]

Disable daily nightly CI (#48292) by @remi-or in [#48292]

gs (#48288) by @eustlb in [#48288]

CI: fix muse OOMs (#48284) by @tarekziade in [#48284]

Fix BayesianDetectorModel.from_pretrained() by calling post_init() (#48254) by @woojinpaik in [#48254]

[Fix] Avoid duplicating tests in CI (#48287) by @remi-or in [#48287]

[GDN] Fix recurrent FLA fallback (#48266) by @vasqu in [#48266]

Fix tie_word_embeddings not lifted from text_config for some VLM configs (BC regression) (#45857) by @qgallouedec in [#45857]

replace xpu-smi subprocess call in benchmark_v2 (#48083) by @kaixuanliu in [#48083]

ignore mlinter ci file (#48267) by @tarekziade in [#48267]

Fix dtype mismatch in grouped_mm_fallback for LoRA training on Mamba+… (#47933) by @adh-aakriti in [#47933]

Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131) by @qgallouedec in [#48131]

deterministic layer_types buffer registration in multiple models (#48162) by @mowoe in [#48162]

Fix nemotron_h save_pretrained emitting singular backbone.embedding.weight (#48075) by @yuekaizhang in [#48075]

force_accelerate_hooks should not hide the signature it wraps (#48156) by @SunMarc in [#48156]

fix(data_collator): align TokenClassification numpy_call with torch_call (#48212) by @ in [#48212]

Fix a typo in a use of a local variable field_ in a test (#48184) by @AleksMat in [#48184]

[serge] Fix 2 integration tests regressed by commit 16780c8 (PR #47622) (#48134) by @sergereview[bot] in [#48134]

[Gemma4] Fix stale expected values in integration tests (#48233) by @ydshieh in [#48233]

[TableTransformer, PI0] Fix stale expected values and OOM in integration tests (#48198) by @ydshieh in [#48198]

Post two CI badges on a PR: CPU PR CI and GPU run-slow (#48190) by @tarekziade in [#48190]

Fix stale expected values in integration tests (cuda sm_86 / Aug04 regressions) (#48171) by @ydshieh in [#48171]

Assign a reviewer even when a codeowner has left, and route models by modality (#48085) by @tarekziade in [#48085]

[VITS] Un-skip test_model_forward (#46375) by @blipbyte in [#46375]

Apply context parallelism to the evaluation path (#48167) by @qgallouedec in [#48167]

Fix gpt_oss runs on GPU (#48118) by @tarekziade in [#48118]

[EsmFold2] Fix stale expected distogram logit values (#48182) by @ydshieh in [#48182]

Fix Apr 05 integration test regressions (cuda sm_86) (#48170) by @ydshieh in [#48170]

Add MLU support to is_flash_linear_attention_available (#46995) by @atri2549 in [#46995]

Fix integration test expected values for cuda sm_86 (Mar 15 regressions) (#48168) by @ydshieh in [#48168]

Fix DynamicCache reconstruction during ExecuTorch export (#47900) by @eladsegal in [#47900]

[LLaVA] Fix pixtral integration tests for cuda sm_86 (#48166) by @ydshieh in [#48166]

Port ESMC and ESMFold2 to Transformers (#46419) by @Rocketknight1 in [#46419]

[Qwen2.5-Omni] Update stale expected values for cuda sm_86 (#48164) by @ydshieh in [#48164]

[Mistral3] Fix batched integration tests: padding_side=left + update expected values (#48161) by @ydshieh in [#48161]

Fix DeepSeek V2 default vocab size (#48159) by @hmellor in [#48159]

[InternVL] Fix stale expected values for Llama integration tests (cuda sm_80) (#48153) by @ydshieh in [#48153]

Retry transient network errors (RemoteDisconnected) in github_utils (#48124) by @ydshieh in [#48124]

fix bugs for clvp model (#47127) by @kaixuanliu in [#47127]

Always tie embeddings for LongT5 and Pop2Piano (#47620) by @jiqing-feng in [#47620]

Use generator with seed for LengthGroupedSampler in Trainer._get_eval_sampler for deterministic eval order with per_device_eval_batch_size > 1 (#48025) by @philipshurpik in [#48025]

Enable mlinter findings artifact for inline PR reviews (#48117) by @ydshieh in [#48117]

Fix CpmAnt loading: size lm_head to vocab_size (#48012) by @jiqing-feng in [#48012]

Delete old mlinter review comments before posting new ones (#48107) by @ydshieh in [#48107]

[Video] Warn and return all frames when num_frames exceeds total_num_frames (#48074) by @carlszk in [#48074]

Accept artifact dir as argument in post_mlinter_review.py (#48106) by @ydshieh in [#48106]

Fix mlinter artifact path (#48088) by @ydshieh in [#48088]

[Video] Fix convert_to_rgb channel slicing and alpha blending for RGBA videos (#48053) by @ in [#48053]

[PE] Skip test_sdpa_can_dispatch_on_flash for TimmWrapper-backed models (#48064) by @ydshieh in [#48064]

fix(pipeline): preserve model.generation_config precedence over pipeline defaults (#47752) (#47953) by @nithin42 in [#47953]

[Gemma3] Update integration test expected values for A10G (#48036) by @ydshieh in [#48036]

Fallback from 'lanczos' to 'bicubic' when on cuda (#48026) by @zucchini-nlp in [#48026]

[serge] Fix 2 integration tests regressed by commit b9090ae (PR #47096) (#48060) by @sergereview[bot] in [#48060]

[Fix] Small FA-related test failures in CB (#47341) by @remi-or in [#47341]

fix: honor empty processor_kwargs={} in multimodal pipelines (#48044) by @ in [#48044]

[CircleCI] Enable CI for private forks, no-op for public repo (#48056) by @ydshieh in [#48056]

Support BatchFeature in length-grouped samplers (#48034) by @qgallouedec in [#48034]

Fix EOS for candidate generators (#47931) by @ in [#47931]

[Gemma3n] Update integration test expected values for A10G + torch 2.13 (#48035) by @ydshieh in [#48035]

fix failed test cases for muse_glimmer (#48011) by @kaixuanliu in [#48011]

Cohere compass tests (#47895) by @zucchini-nlp in [#47895]

docs: use relative paths for README language menus and add fa/ro entries (#47777) by @Priyans-Lathiya in [#47777]

Moving mlinter to 0.1.4 (#47918) by @tarekziade in [#47918]

[MoE] Fix Blackwell GPU crash with torch._grouped_mm on torch <= 2.8 (#48014) by @ in [#48014]

[docs] Muse Glimmer (#47882) by @stevhliu in [#47882]

[Florence2] Fix two integration test failures caused by torch 2.13 and auto-dtype (#48031) by @ydshieh in [#48031]

Proper separation of tests (#47943) by @zucchini-nlp in [#47943]

unpin pytest in the examples_torch deps (#48023) by @tarekziade in [#48023]

[CI] Fix startup failure in pr_build_doc_with_comment workflow by adding missing get-pr-number dependency (#47971) by @ in [#47971]

Let the GPU verify caller turn on the memory probe (#48001) by @tarekziade in [#48001]

Fix MTP config when mlp_layer_types is absent (#48015) by @Cyrilvallez in [#48015]

Remove duplicate block_sparse_moe assignment in GraniteMoeDecoderLayer (#47876) by @Aman2394 in [#47876]

[ModernVBERT] Fix integration test checkpoint (404 since April) (#48009) by @ydshieh in [#48009]

Fix cropping (#48006) by @Cyrilvallez in [#48006]

Fix DFlash candidate token device mismatch with device_map="auto" (#47877) by @sywangyi in [#47877]

🔴 Allow tokenizers 0.23.1 (#46381) by @ArthurZucker in [#46381]

[Whisper] Fix batch decode_with_timestamps in WhisperTokenizer.decode() (#47997) by @ydshieh in [#47997]

[OLMoE] Update expected logits for A10G and add torch.no_grad() (#47989) by @ydshieh in [#47989]

[OLMo] Fix OOM in logits tests by adding torch.no_grad() (#47986) by @ydshieh in [#47986]

[AXK1] Fix expected logits for CUDA A10G (#47980) by @ydshieh in [#47980]

[Gemma] Update expected values for A10G (#47976) by @ydshieh in [#47976]

Fix gemma4 video to device (#47896) by @guarin in [#47896]

Remove stale (None, None) fallback in qwen2_5_vl batch_different_resolutions test (#47972) by @ydshieh in [#47972]

Potential fix for code scanning alert no. 267: Artifact poisoning (#47949) by @tarekziade in [#47949]

Fix GatedDeltaNet A_log dtype to prevent -inf under bfloat16 init (#47944) by @Nkluge-correa in [#47944]

[serge] Fix 2 integration tests for model got_ocr2 failing with other (other (2)) (#47937) by @sergereview[bot] in [#47937]

Fix Jinja block endings in CHAT WITH MODELS' Writing a chat template … (#47960) by @ak1for2business-prog in [#47960]

[tests] Fix expected output for Qwen2.5-VL batch_wo_image on CUDA (#47968) by @ydshieh in [#47968]

[serge] Fix 2 integration tests for model opt failing with other (other (2)) (#47909) by @sergereview[bot] in [#47909]

Scan a diff in trufflehog, not the whole repo history (#47945) by @tarekziade in [#47945]

[serge] Fix 2 integration tests for model vivit failing with output_mismatch (tensor values differ (2)) (#47566) by @sergereview[bot] in [#47566]

Fix Gemma sliding_window being halved on every config save/reload (#47940) by @Bluear7878 in [#47940]

docs: fix incorrect PEFT anchor link in fine-tuning section (#47927) by @dsulot in [#47927]

CI: add vllm-test-init and vllm-test-transformers jobs on dedicated runners (#47934) by @ydshieh in [#47934]

Make muse glimmer exportable (#47871) by @IlyasMoutawwakil in [#47871]

docs: add installation instructions for NVIDIA Spark (ARM64) devices (#47906) by @mfuntowicz in [#47906]

[CohereCompass] Minor docs fixes (#47903) by @calpt in [#47903]

fix: correct checkpoints, config annotations, and create_dummy_models improvements (#47902) by @ydshieh in [#47902]

Transform paths and repeat joining for response parsing (#47648) by @Rocketknight1 in [#47648]

[docs] Update toctree (#47781) by @stevhliu in [#47781]

Add CI_CPU_MEMORY_LIMIT_GB to check_failed_tests workflow (#47884) by @ydshieh in [#47884]

Use tiny Hub checkpoint in Qwen3ASR processor test (#47833) by @ydshieh in [#47833]

Update AutoRound XPU/CPU backend (#47826) by @yiliu30 in [#47826]

docs(tests): fix typos in test comments (#47859) by @zhaoxinyi02 in [#47859]

Update version post release (#47870) by @Cyrilvallez in [#47870]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@tarekziade

CI: gate the hunyuan-moe slow test (#48330)

CI: fix muse OOMs (#48284)

ignore mlinter ci file (#48267)

Post two CI badges on a PR: CPU PR CI and GPU run-slow (#48190)

Assign a reviewer even when a codeowner has left, and route models by modality (#48085)

Fix gpt_oss runs on GPU (#48118)

Moving mlinter to 0.1.4 (#47918)

unpin pytest in the examples_torch deps (#48023)

Let the GPU verify caller turn on the memory probe (#48001)

Potential fix for code scanning alert no. 267: Artifact poisoning (#47949)

Scan a diff in trufflehog, not the whole repo history (#47945)

@dkrisman

Add an opt-in per-frame pixel cap (cap_pixels_per_frame) to the Qwen3-VL video processor (#48071)

@jiqing-feng

Cpmant fix use cache (#48013)

[xcodec2] Fix flex attention and flash dispatch tests (#48244)

Always tie embeddings for LongT5 and Pop2Piano (#47620)

Fix CpmAnt loading: size lm_head to vocab_size (#48012)

Fix Qwen2.5-Omni / Qwen3-Omni-MoE generation with a compilable cache (#47872)

Bump default flash-attn2 hub kernel version to v3 (#47863)

Declare sdpa support in TimmWrapper (#47939)

@ydshieh

Fix AutoTokenizer returning TokenizersBackend for DeepSeek-R1-Distill-Qwen models (#48211)

[Gemma4] Investigate flaky test_generation_beyond_sliding_window_1_eager (#48236)

[Gemma4] Fix stale expected values in integration tests (#48233)

[TableTransformer, PI0] Fix stale expected values and OOM in integration tests (#48198)

Fix stale expected values in integration tests (cuda sm_86 / Aug04 regressions) (#48171)

[EsmFold2] Fix stale expected distogram logit values (#48182)

Fix Apr 05 integration test regressions (cuda sm_86) (#48170)

Fix integration test expected values for cuda sm_86 (Mar 15 regressions) (#48168)

[LLaVA] Fix pixtral integration tests for cuda sm_86 (#48166)

[Qwen2.5-Omni] Update stale expected values for cuda sm_86 (#48164)

[Mistral3] Fix batched integration tests: padding_side=left + update expected values (#48161)

[InternVL] Fix stale expected values for Llama integration tests (cuda sm_80) (#48153)

Retry transient network errors (RemoteDisconnected) in github_utils (#48124)

[Whisper] Fix speculative decoding: preserve cleared suppress tokens through super().generate() (#48108)

Enable mlinter findings artifact for inline PR reviews (#48117)

Delete old mlinter review comments before posting new ones (#48107)

Accept artifact dir as argument in post_mlinter_review.py (#48106)

Fix mlinter artifact path (#48088)

[Whisper] Fix decoder position IDs for left-padded batches in longform generation (#48028)

[PE] Skip test_sdpa_can_dispatch_on_flash for TimmWrapper-backed models (#48064)

[Gemma3] Update integration test expected values for A10G (#48036)

[CircleCI] Enable CI for private forks, no-op for public repo (#48056)

[Gemma3n] Update integration test expected values for A10G + torch 2.13 (#48035)

[Florence2] Fix two integration test failures caused by torch 2.13 and auto-dtype (#48031)

[ModernVBERT] Fix integration test checkpoint (404 since April) (#48009)

[Whisper] Fix speculative decoding: UnboundLocalError, cache corruption, and speed regression (#48000)

[Whisper] Fix batch decode_with_timestamps in WhisperTokenizer.decode() (#47997)

[Whisper] Fix integration test failures on A10G (dtype, stale values, API changes) (#47995)

[DeepSeekV2] Fix integration tests OOM: use device_map=auto instead of 8-bit quantization (#47991)

[OLMoE] Update expected logits for A10G and add torch.no_grad() (#47989)

[OLMo] Fix OOM in logits tests by adding torch.no_grad() (#47986)

[GPTNeoX] Fix post_processor not overridden when loading from pretrained (OLMo garbage generation) (#47988)

[AXK1] Fix expected logits for CUDA A10G (#47980)

[Gemma] Update expected values for A10G (#47976)

[emu3] 🦮 Black Labrador is back! Fix image generation broken since #37033 (#47948)

Remove stale (None, None) fallback in qwen2_5_vl batch_different_resolutions test (#47972)

[tests] Fix expected output for Qwen2.5-VL batch_wo_image on CUDA (#47968)

CI: add vllm-test-init and vllm-test-transformers jobs on dedicated runners (#47934)

fix: correct checkpoints, config annotations, and create_dummy_models improvements (#47902)

Add CI_CPU_MEMORY_LIMIT_GB to check_failed_tests workflow (#47884)

Use tiny Hub checkpoint in Qwen3ASR processor test (#47833)

@eustlb

gs (#48288)

@eladsegal

Fix DynamicCache reconstruction during ExecuTorch export (#47900)

Support per-layer cache configuration and attention-mask selection (#47901)

@YangKai0616

🚨[wav2vec2] Support attn_implementation=sdpa dispatch (#46196)

@drbh

feat: add nvfp4 quantization (#47883)

@Priyans-Lathiya

docs: use relative paths for README language menus and add fa/ro entries (#47777)

@itazap

[new model] step 3.7 (#46658)

@calpt

[CohereCompass] Minor docs fixes (#47903)

Add CohereCompass modeling (#47878)

Original source

Patch release v5.15.1

This patch most notably solves a few issues with DFlash and MTP candidate generators, as well as an issue where images could sometimes not be processed on accelerator if using Lanczos filter.

It contains the following commits:

• Fix DFlash candidate token device mismatch with device_map="auto" (#47877) by @sywangyi and @Cyrilvallez
• Align logit distributions for CandidateGenerators using sampling (#48007) by @Cyrilvallez
• Fix MTP config when mlp_layer_types is absent (#48015) by @Cyrilvallez
• Fallback from 'lanczos' to 'bicubic' when on cuda (#48026) by @zucchini-nlp
• Fix gemma4 video to device (#47896) by @guarin

Original source

Release v5.15.0

New Model additions

Meta Muse Glimmer

Muse Glimmer, released today, is Meta’s new multimodal model, especially designed for agentic use cases. Distilled from Muse to 30B parameters, and released under the Apache 2.0 license, it can be deployed to local setups for privacy-aware applications such as coding, document analysis, personal assistants, Claw- or Hermes-like setups.

Muse Glimmer is a dense 30B parameter model consisting of:

• 2B ViT-style encoder for vision (Perception Encoder)
• 28B parameter text decoder

We're covering it in the following blogpost: http://hf.co/blog/muse-glimmer

GraniteMoeSWA & GraniteSWA

Links: Documentation

Add Granite-swa and Granitemoe-swa model support (#47179) by @daviswer in #47179

Links: Documentation

Add Granite-swa and Granitemoe-swa model support (#47179) by @daviswer in #47179

A.X-K1 & A.X-K2

Links: Documentation

Add AXK2 from SKT (#47528) by @vasqu in #47528

Links: Documentation

add_axk1 (#46867) by @kmswin1 in #46867

Cosmos3 Edge

Links: Documentation

Add Cosmos3 Edge model support (#47181) by @atharvajoshi10 in #47181

Breaking changes

Kernels are now opt-in rather than mandatory for linear attention models (Mamba, GDN, Conv-only, etc.), so users who relied on automatic kernel selection must explicitly enable kernels to maintain previous behavior.

🚨 [Kernels] Refactor all linear attn models & native kernels fallback (#47630) by @vasqu

The cache cropping API now only accepts negative values (relative offsets) instead of absolute sizes, so users calling crop methods directly must update their code to pass negative values accordingly.

🚨 [cache] Cropping can only be done with negative values (#47720) by @Cyrilvallez

T5 and its model family (MT5, LongT5, etc.) now support SDPA and other attention backends via ALL_ATTENTION_FUNCTIONS, meaning the default attention implementation may change and users relying on the previous eager-only path should explicitly set attn_implementation="eager" if needed.

🚨 Enable SDPA (and other attention backends) for T5 and propagate to the T5 family (#47014) by @jiqing-feng

Several small private helper functions (e.g., _is_url, _build_image_tokens) have been removed from multimodal processor files, so users or downstream libraries that imported these private functions directly must remove or replace those references.

🚨 Processors update the rest (#46556) by @zucchini-nlp

Attention

This release includes several attention fixes and improvements, including correcting Multi-Head Latent Attention (MLA) cache compression, optimizing Flash Attention max sequence length computation in vision models, and fixing bugs in CTRL flex-attention and SDPA prefill with position bias. Additional changes refactor linear attention models for better maintainability, make Gemma 4's heterogeneous attention config explicit, and improve MPS support via metal-flash-sdpa integration.

[Fix] Fix multi-head latent attention (MLA) (#47761) by @remi-or in [#47761]

Refactor all linear attention models to latest best standards for convolution (#47452) by @Cyrilvallez in [#47452]

Allow metal-flash-sdpa for OpenAIPrivacyFilter on MPS (#46740) by @ArthurZucker in [#46740]

Use new per_layer_config for Gemma 4 so that heterogeneous attention config is explicit (#47384) by @hmellor in [#47384]

add paged attention tests support for XPU (#47163) by @kaixuanliu in [#47163]

Move value padding into the attention interfaces that need it (#47451) by @hmellor in [#47451]

Simplify function dispatch for linear attention (#47450) by @Cyrilvallez in [#47450]

Optimize flash attention max seqlen computation in vision attention (#47170) by @ShareLer in [#47170]

Fix BlockMask crash in CTRL flex-attention generation (#46854) by @jiqing-feng in [#46854]

[CB] Automatically switch attention implementation to flash (#47330) by @remi-or in [#47330]

Fix sdpa prefill with position_bias (#47359) by @Cyrilvallez in [#47359]

Vision

Vision improvements in this release include performance optimizations such as faster image preprocessing for vision-language models (GLM4V, MiniMaxM3-VL, and others) by eliminating redundant tensor copies, and more efficient Flash Attention variable-length paths by precomputing maximum sequence lengths once per forward pass. Several bug fixes were also applied, including correcting dtype alignment in Kosmos2/Kosmos2_5 embedding merges, fixing a position-embedding initialization fallback in Phi4Multimodal, resolving PIL resize parity in Hunyuan-VL, and patching stop-sequence handling in the image-text-to-text pipeline.

Modularize qwen-format vision processors (#47573) by @zucchini-nlp in [#47573]

Update daily CI Docker image to torch 2.13.0 / CUDA 13.0 (#47738) by @ydshieh in [#47738]

Align image feature dtype in kosmos2 and kosmos2_5 embedding merge (#47691) by @ in [#47691]

Speed up image preprocessing for vision-language models (#47453) by @labAxiaoming in [#47453]

Fix vision position-embedding init width fallback in Phi4Multimodal (#47509) by @ in [#47509]

Fix Hunyuan-VL PIL image resize parity with reference preprocessing (#47233) by @IMvision12 in [#47233]

Fix image-text-to-text stop_sequence handling (#47032) by @Sunt-ing in [#47032]

Refactor image loading in tests to use load_test_image helper (#47218) by @LevelVoid in [#47218]

Generation

Several generation improvements and bug fixes were made, including enabling batched audio generation for Qwen2.5/3-Omni, allowing sliding window cache layers to work with speculative decoding, and fixing memory overhead from static cache persistence across generate() calls. Multiple model-specific bugs were also resolved, including crashes in KyutaiSpeechToText, MusicgenForCausalLM, CTRL flex-attention, and assisted decoding for EncoderDecoder cache and OlmoHybrid models.

Align OlmoHybrid to use a native cache in generate (#47604) by @Cyrilvallez in [#47604]

[generate] Stop setting the static cache as an attribute to save memory (#47731) by @Cyrilvallez in [#47731]

Add support for batched Qwen2.5/3-Omni audio generation (#47186) by @IMvision12 in [#47186]

[cache] Allow sliding window layers to be roll-backed for speculative decoding (#47447) by @Cyrilvallez in [#47447]

Fix shape mismatch in KyutaiSpeechToText generate() last window (#46952) by @jiqing-feng in [#46952]

Fix typo in MusicgenForCausalLM.generate() (#46974) by @jiqing-feng in [#46974]

Fix assisted decoding for models with EncoderDecoder cache & OlmoHybrid (#47361) by @Cyrilvallez in [#47361]

Cache

Several cache-related bugs were fixed, including correcting NemotronH's missing "mlp" layer-type mapping, resolving recurrent-layer padding masks being skipped during chunked prefill and cache continuation for hybrid models, and fixing assisted decoding for models with EncoderDecoderCache and OlmoHybrid. Additional improvements include aligning OlmoHybrid to use a native cache, enabling sliding window layers to support speculative decoding rollback, and stopping the static cache from being stored as a model attribute to reduce unexpected memory overhead.

[docs] MPS graph cache (#47304) by @stevhliu in [#47304]

Fix NemotronH: Register "mlp" in the cache layer-type mappings (#47535) by @qgallouedec in [#47535]

Fix recurrent-layer padding mask being skipped on continued forwards (chunked prefill, cache continuation) (#47087) by @abcgco in [#47087]

Kernels

⚠️ The kernels python package will very likely be a required dependency for transformers[torch] in the near future. This will help us deliver maximum performance to all users; kernels will only be downloaded from trusted publishers manually approved by the HF team. Please let us know of any issues you're facing beforehands so that we may solidify our integration.

Improved robustness of the kernels integration by refactoring function handling to use layer repos, fixing CI EROFS fallback patches for kernel downloads via HfApi, resolving a positional argument collision in causal_conv1d_fn, and bumping the FP8 kernels version to prevent NaNs.

[conftest] Fix EROFS fallback for kernel downloads (correct interception point) (#47794) by @ydshieh in [#47794]

[conftest] Fix EROFS fallback for kernel downloads via HfApi (#47791) by @ydshieh in [#47791]

[Kernels] Refactor function handling (#46883) by @vasqu in [#46883]

Kernels and loaders robustification (#47334) by @IlyasMoutawwakil in [#47334]

Fix causal_conv1d_fn positional activation colliding with hub kernel's seq_idx (#47527) by @qgallouedec in [#47527]

[FP8] Bump kernels version (#47344) by @vasqu in [#47344]

[docs] FlashAttention kernel fallback (#47345) by @stevhliu in [#47345]

Quantization

Quantization support was expanded with FP8 kernels for compressed-tensors models, fixes for FP8 module normalization and format-based compression detection, and a multi-device MXFP4 dequantization race condition fix. GPTQ and MXFP4 tests were also extended to cover Intel XPU devices.

extend tests/quantization/gptq/test_gptq.py::GPTQTestCUDA and tests/q… (#47166) by @sywangyi in [#47166]

Compressed tensors fp8 (#47216) by @SunMarc in [#47216]

Fix A.X-K2 fp8 modules_to_not_convert normalization for the gated-norm MLP (#47578) by @kmswin1 in [#47578]

[Quantization]: Refactor is_quantization_compressed for format-based detection (#47152) by @rigen1048 in [#47152]

Fix multi-device mxfp4 dequantization race in _convert_moe_packed_tensors (#47423) by @kaixuanliu in [#47423]

Audio

Batched audio generation is now supported for Qwen2.5/3-Omni, and several bug fixes were applied across audio models, including a dtype mismatch in Gemma4 audio feature merging, a bfloat16 positional embedding error in AudioFlamingo3, and missing backend requirement guards for Voxtral. The VibeVoice ASR processor was also updated to make audio input optional and support multiple audios per prompt.

feat[vLLM x v5]: Make audio optional and support multiple audios in VibeVoice ASR processor (#47483) by @harshaljanjani in [#47483]

Fix Gemma4 audio feature dtype mismatch in masked_scatter (#47482) by @danielhanchen in [#47482]

[fix] fix requirements audio feature and proc (#47113) by @eustlb in [#47113]

[AudioFlamingo3] Fix bfloat16 dtype mismatch in audio encoder positional embedding (#47258) by @snkii in [#47258]

Parallelization

Expanded FSDP support across 94 ForCausalLM model classes with auto-generated FSDP plans, added end-to-end FSDP tests including distributed checkpoint save/load and generation, and introduced a dedicated FSDP CI job. Additionally, fixed a device mismatch bug in create_bidirectional_sliding_window_mask under model parallelism and resolved a tensor parallel inference issue for models with tied embeddings.

skip fsdp tests when backend is mps (#47601) by @3outeille in [#47601]

Fix model parallel device mismatch in create_bidirectional_sliding_window_mask (#47560) by @abcgco in [#47560]

Add FSDP plans to all models (#47165) by @3outeille in [#47165]

Fix TP inference for tied embedding (#47503) by @3outeille in [#47503]

Add FSDP CI and end-to-end FSDP tests + save fsdp (#47357) by @3outeille in [#47357]

Tokenization

This release adds native support for Mistral's "tekken" tokenizer format via AutoTokenizer, fixes a CodeLlama tokenizer bug where leading whitespace was incorrectly dropped during decode, and patches a potential ReDoS vulnerability caused by unescaped tokenizer filenames being used as regex patterns in from_pretrained.

[Mistral] Add native tekken tokenizer support to AutoTokenizer (#47507) by @juliendenize in [#47507]

Fix CodeLlama tokenizer dropping leading whitespace on decode (#47488) by @SuryanshSS1011 in [#47488]

Fix potential ReDoS by escaping tokenizer filename used as regex pattern (#47498) by @hameedibrh in [#47498]

Serve

Improved the serve chat parsing to unify streaming and non-streaming paths under a single response parser that handles tool calls, reasoning, and content, simplifying the addition of new model support. Additionally, hardened daily CI reporting by fixing GitHub API diagnostic output being captured in Slack payloads and adding rate-limit resilience to prevent report failures when paginating large job matrices.

CI: Log GitHub API diagnostics to stderr (#47635) by @tarekziade in [#47635]

Update serve chat parsing (#46267) by @SunMarc in [#46267]

ci: harden daily CI reporting against GitHub API rate limits (#47382) by @tarekziade in [#47382]

Bugfixes and improvements

Fix cached_files silently returning stale file on read-only filesystem (EROFS) (#47852) by @ydshieh in [#47852]

Fix PhimoeIntegrationTest (#46539) by @ydshieh in [#46539]

make examples under doc device agnostic (#47812) by @kaixuanliu in [#47812]

cancel deterministic for XPU in gemma4 tests (#47790) by @kaixuanliu in [#47790]

Serialize post-mlinter-review after post-link to avoid PR description race (#47832) by @ydshieh in [#47832]

Use content hash for mlinter review deduplication (#47830) by @ydshieh in [#47830]

Add new args in auto-docstring (#47737) by @zucchini-nlp in [#47737]

Add check_model_inits.py (#47656) by @guarin in [#47656]

add xpu in installation guide (#47785) by @sywangyi in [#47785]

Fix mlinter review job: checkout before artifact download (#47820) by @ydshieh in [#47820]

Post mlinter findings as inline PR review comments (#47819) by @ydshieh in [#47819]

Fix ci style (#47818) by @vasqu in [#47818]

Hotfix axk2 indexer norm (#47810) by @kmswin1 in [#47810]

open fla support for XPU to benefit from the acceleration (#47799) by @kaixuanliu in [#47799]

[docs] Update BatchEncoding.to() type annotation and docstring (#47789) by @samyuktahegde in [#47789]

Fix linting (#47807) by @Cyrilvallez in [#47807]

Fix patching in some models (#47798) by @zucchini-nlp in [#47798]

Add post-mlinter-review job to post-dashboard-link workflow (#47800) by @ydshieh in [#47800]

Migrate torchao integration off deleted torchao.dtypes (#47797) by @vkuzo in [#47797]

[conftest] Also wrap snapshot_download for EROFS fallback (#47796) by @ydshieh in [#47796]

Fix MI355 CI: bump hf-workflows pin to NUM_SLICES=4 (#47792) by @Abdennacer-Badaoui in [#47792]

[Fix] Wrong type hint in get_number_of_image_patches (#47788) by @remi-or in [#47788]

Fix Dac offload tests (#47775) by @guarin in [#47775]

[Fix] Swapped height and width in KimiK25 (#47786) by @remi-or in [#47786]

Remove dangling files and folders (#47764) by @Cyrilvallez in [#47764]

Fix AI-written conversion mappings (#47755) by @Cyrilvallez in [#47755]

update mistral common version for PR 47507 (#47677) by @itazap in [#47677]

Fix spelling/grammar in model files (batch 3/3) (#47684) by @Rocketknight1 in [#47684]

Fix spelling/grammar in core library, examples, and utils (#47685) by @Rocketknight1 in [#47685]

Fix spelling/grammar in model files (batch 2/3) (#47683) by @Rocketknight1 in [#47683]

update sonicmoe versions (#47769) by @IlyasMoutawwakil in [#47769]

clean up reverse_op fixme in compressed_tensors (#47701) by @DhanushPillay in [#47701]

PR CI with torch 2.13 (#47767) by @ydshieh in [#47767]

Import utils compilation fixes (#47726) by @IlyasMoutawwakil in [#47726]

Fix DBRX MoE hidden size and expert GLU transposes (#47671) by @kaixuanliu in [#47671]

Fix multi token decode merging (#47762) by @IlyasMoutawwakil in [#47762]

Fix: Remove redundant @can_return_tuple conflicting with @capture_out… (#47733) by @guarin in [#47733]

fix processor config nested key fallback (#47628) by @YunzhuLu in [#47628]

[CI - Debug] Skip /transformers-dependent steps for CPU runner (#47759) by @ydshieh in [#47759]

[CI] Add CPU runner support to ssh-runner workflow (#47757) by @ydshieh in [#47757]

Simplify reverse weight conversion (#47725) by @Cyrilvallez in [#47725]

Remove useless linting for inv_freq (#47753) by @Cyrilvallez in [#47753]

Use explicit nn.Buffer everywhere for modular (#47722) by @Cyrilvallez in [#47722]

Remove stale and redundant _no_split_modules entries (#47645) by @guarin in [#47645]

Executorch exporter fixes (#47243) by @IlyasMoutawwakil in [#47243]

Acc fix in xpu (#47500) by @sywangyi in [#47500]

update Dockerfile for xpu torch2.13 (#47502) by @sywangyi in [#47502]

feat[vLLM]: Support text replacement offsets in the remaining old-format processors (#47614) by @harshaljanjani in [#47614]

Fix ImportError in transformers.exporters on torch < 2.8 (#47711) by @Neal006 in [#47711]

Fix failing tests for axk1 and axk2 (#47727) by @kaixuanliu in [#47727]

Fix failing tests for granite_swa and granitemoe_swa (#47723) by @kaixuanliu in [#47723]

skip invalid test cases for inkling tests (#47493) by @kaixuanliu in [#47493]

[docs] Fix BatchEncoding documentation inconsistencies (#47647) by @samyuktahegde in [#47647]

Fix spelling/grammar in model files (batch 1/3) (#47682) by @Rocketknight1 in [#47682]

Fix spelling/grammar in English docs (q → z) (#47681) by @Rocketknight1 in [#47681]

Fix spelling/grammar in English docs (h → p) (#47680) by @Rocketknight1 in [#47680]

Fix spelling/grammar in English docs (a → g) (#47679) by @Rocketknight1 in [#47679]

fix npu check (#47587) by @DhanushPillay in [#47587]

fix: correct text input validation logic in 8 multimodal processors (and → or) (#47663) by @AbdullahRasheed45 in [#47663]

Fix feature dtype mismatch in masked_scatter for seven multimodal models (#47673) by @ in [#47673]

[docs] storing and loading chat templates (#47650) by @stevhliu in [#47650]

adding amd quark config class changes (#47322) by @debasisdwivedy in [#47322]

Fix compressed tensors impl (#47652) by @SunMarc in [#47652]

Hoist special-token lookups in wav2vec2 decode paths and drop a dead filter in wav2vec2_phoneme (#47557) by @ishan-1010 in [#47557]

silencing elastic warning by import distributed lib inside functions (#47665) by @3outeille in [#47665]

Fix missing github_utils.py download in PR CI dashboard workflow (#47668) by @ydshieh in [#47668]

Exportable kimi (#47096) by @IlyasMoutawwakil in [#47096]

better guarding to handle torch compiled with USE_DISTRIBUTED=0 (#47619) by @3outeille in [#47619]

Remove gemma4 warnings (#47664) by @Cyrilvallez in [#47664]

[Chat Parsing] Type inline tool-call arguments from the calling tool's JSON Schema (#47529) by @yonigozlan in [#47529]

Improve Trainer DataLoader Controls for Streaming and Multiprocessing (#47164) by @muyihao in [#47164]

Update maintainer list (#47644) by @Rocketknight1 in [#47644]

Allow position_ids_start=2 on DataCollatorWithFlattening for RoBERTa etc. (#47525) by @tomaarsen in [#47525]

Remove Rotary warning (#47642) by @Cyrilvallez in [#47642]

Drop multimodal inputs natively in prepare_inputs_for_generation if not in prefill (#47622) by @Cyrilvallez in [#47622]

Simplify all Rotary modules (#47598) by @Cyrilvallez in [#47598]

[docs] response_template when serving (#47626) by @stevhliu in [#47626]

[docs] MTP support (#47301) by @stevhliu in [#47301]

Fix GPT-2 c_proj depth scaling initialization (#47459) by @DavidJohnQuinlan in [#47459]

Fix fp8_linear compilability (#47623) by @IlyasMoutawwakil in [#47623]

byebye torch 2.4 (#47609) by @ydshieh in [#47609]

Vectorize NoRepeatNGramLogitsProcessor and remove its host sync (#47571) by @hameedibrh in [#47571]

Fix CUDA Graph breaking host to device copy from scalar tensor allocation (#47547) by @hmellor in [#47547]

Fix some processors (#47608) by @zucchini-nlp in [#47608]

CI: Add serge review relay workflow and review rules (#47610) by @tarekziade in [#47610]

[docs] Exporters (#47374) by @stevhliu in [#47374]

CI: use a single function for GH calls (#47474) by @tarekziade in [#47474]

Remove redundant guarding for distributed (#47570) by @3outeille in [#47570]

Fix modular for mamba packages (#47494) by @Cyrilvallez in [#47494]

Remove deprecated conversion in Kimi (#47581) by @zucchini-nlp in [#47581]

CI: use transformers-ci daily workflow with OTEL (#47360) by @tarekziade in [#47360]

Fix slow tensor path in _check_special_mm_tokens (#47580) by @guan404ming in [#47580]

CI: let's run integration failure cron at 10pm (#47537) by @tarekziade in [#47537]

Better and more extensive tests for RoPE (#46912) by @zucchini-nlp in [#46912]

Fix mamba2 family decode and simplify all reshape ops (#47569) by @Cyrilvallez in [#47569]

[DiffusionGemma] Cast the decoder padding mask to bool (#47295) by @kashif in [#47295]

General maintenance (#47517) by @zucchini-nlp in [#47517]

fixed the benchmark script with DistributedConfig (#47568) by @tarekziade in [#47568]

Fix failing tests for mimo_v2_flash (#47284) by @kaixuanliu in [#47284]

Make tokenization_mistral_common importable without mistral_common installed (#47397) by @juliendenize in [#47397]

Use --flake-runs=1 in check_bad_commit.py for PR comment CI (#47522) by @ydshieh in [#47522]

Deprecate the old response_schema (#47320) by @Rocketknight1 in [#47320]

fix: add pickle support to _LazyConfigMapping for spawn multiprocessing (#46026) by @kfojcik-intel in [#46026]

Delete old deprecations (#47518) by @zucchini-nlp in [#47518]

Run only @slow tests in PR comment CI (#47521) by @ydshieh in [#47521]

Fix incorrect type hint (#47519) by @hmellor in [#47519]

Deprecate CB config in gen configuration (#47291) by @remi-or in [#47291]

[Offloading] [Bugfix] Fix fully offloaded model saving (#47336) by @kylesayrs in [#47336]

CI: fix torchaudio pinning +proper break in rnnt (#47422) by @tarekziade in [#47422]

Fix Qwen2.5-Omni Token2Wav DiT rotary embedding layout (interleaved cos/sin) (#47403) by @HenryVarro666 in [#47403]

CI: add reproduce mode to serge verify caller (#47492) by @tarekziade in [#47492]

[fix][whisper]: fix max_new_tokens handling (#46795) by @eustlb in [#46795]

Isolate MLA KV expansion to make it easier to bypass (#47460) by @hmellor in [#47460]

Fix loss alignment and Trainer token counting for encoder decoder models (#46903) by @OmkumarSolanki in [#46903]

Fix the HunyuanVL's torchvision backend (#47499) by @Mi-Jiazhi in [#47499]

[DiffusionGemma] Support gradient checkpointing (#46572) by @kashif in [#46572]

Fix failing tests for zaya (#47268) by @kaixuanliu in [#47268]

tipsv2_dpt: fix failing tests for XPU (#47292) by @kaixuanliu in [#47292]

Fix some failed test cases related with XPU Expectations (#47173) by @kaixuanliu in [#47173]

fix: guard DTensor import in sharding_utils.py for PyTorch < 2.5 (#47481) by @ in [#47481]

CPU can incur a slow path on non-contiguous magnitudes (#47351) by @vbayanag in [#47351]

Route chat management calls to the service root (#47138) (#47303) by @dhruv7477 in [#47303]

fix: liger unnecessarily materializes logits in VRAM during eval, causing OOM (#45273) by @excepshenal in [#45273]

Fix gradient inflation when combining label smoothing with gradient accumulation (#47261) by @Incheonkirin in [#47261]

Fix MoE expert decompression for non-32-divisible bit widths (#47315) by @KKothuri in [#47315]

[Qwen3ASR] Add hotword parsing, and fix language parsing and training. (#47111) by @ebezzam in [#47111]

Remove deprecated training args and is_fast property (#46917) by @cyyever in [#46917]

Consistent output shape from get_image_features (#46405) by @zucchini-nlp in [#46405]

fix failed test cases for qwen3_omni_moe model (#47449) by @kaixuanliu in [#47449]

Fix double-shifted training loss in GitForCausalLM (#47395) by @ in [#47395]

Fix CohereASR training-loss double-shift (same as Moonshine fix #46784) (#46895) by @sharmax-vikas in [#46895]

Warn when group_by_length is silently ignored for iterable datasets (#47379) by @qgallouedec in [#47379]

Update bug report list (#46607) by @molbap in [#46607]

fix: remove unreachable return in special token builder (#47420) by @hai1222 in [#47420]

Add Harry to slow CI (#47454) by @vasqu in [#47454]

BLT: vectorize patch length processing (#47385) by @sj0618 in [#47385]

Fix TrackioCallback fails to log evaluation metrics after training ends (#46935) by @lewtun in [#46935]

[Kimi] add integration tests (#47383) by @zucchini-nlp in [#47383]

Add distributed runtime utils and DistributedMixin (#47352) by @3outeille in [#47352]

Fix Cosmos 3 Edge Patch packing order (#47399) by @atharvajoshi10 in [#47399]

Fix yarn mscale_all_dim for DeepSeek v2 and Mistral 4 (#47435) by @hmellor in [#47435]

Hoist special-token lookups out of per-token loops in six slow tokenizers (#47425) by @ishan-1010 in [#47425]

fix typos and variable naming in quicktour.md (#47418) by @yashasvi-srivastava21 in [#47418]

Normalize multimodal input keys in AnyToAnyPipeline (#47074) by @Sunt-ing in [#47074]

Fix Aria checkpoint key conversion mapping (#47151) by @sywangyi in [#47151]

extend tests/models/qwen3_next/test_modeling_qwen3_next.py::Qwen3Next… (#47184) by @sywangyi in [#47184]

CI: add serge verify (GPU) caller workflow (#47381) by @tarekziade in [#47381]

Fix orthogonal_ init for low-precision dtypes (bf16/fp16) (#47252) by @janbernloehr in [#47252]

[docs] Fix decode examples and expected output in fast_tokenizers (#47369) by @samyuktahegde in [#47369]

[serge] Fix 20 integration tests for model whisper failing with output_mismatch (list output differs (10), other (6) (#47150) by @sergereview[bot] in [#47150]

[Mistral] Move MistralConverter into integrations/mistral/ package (#46603) by @juliendenize in [#46603]

Fix GLM video frame padding for temporal patches (#47141) by @labAxiaoming in [#47141]

[docs] Inkling (#47350) by @stevhliu in [#47350]

Fix Daily CI reporting issues (#47364) by @tarekziade in [#47364]

[peft] Support key_mapping with PEFT models (#46766) by @tomaarsen in [#46766]

Fix model tests for tipsv2 (#47356) by @kaixuanliu in [#47356]

Fix TimesFM 2.5 window_size AttributeError (#47363) by @kashif in [#47363]

fix: allow num_labels property to return None when id2label is unset (#47069) by @SebTardif in [#47069]

[Tests] Fix slow video tensor creation from list of numpy arrays in SmolVLM (#44731) by @Defalt-Meh in [#44731]

Update dev version on main (#47366) by @vasqu in [#47366]

Fix deepgemm on multiple devices (#47323) by @IlyasMoutawwakil in [#47323]

ci: Handle empty GitHub token in CI run lookup (#47362) by @tarekziade in [#47362]

Fix inkling feature extractor (#47349) by @ArthurZucker in [#47349]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@ydshieh

Fix cached_files silently returning stale file on read-only filesystem (EROFS) (#47852)

Fix PhimoeIntegrationTest (#46539)

Serialize post-mlinter-review after post-link to avoid PR description race (#47832)

Use content hash for mlinter review deduplication (#47830)

Fix mlinter review job: checkout before artifact download (#47820)

Post mlinter findings as inline PR review comments (#47819)

Add post-mlinter-review job to post-dashboard-link workflow (#47800)

[conftest] Also wrap snapshot_download for EROFS fallback (#47796)

[conftest] Fix EROFS fallback for kernel downloads (correct interception point) (#47794)

[conftest] Fix EROFS fallback for kernel downloads via HfApi (#47791)

PR CI with torch 2.13 (#47767)

[CI - Debug] Skip /transformers-dependent steps for CPU runner (#47759)

[CI] Add CPU runner support to ssh-runner workflow (#47757)

Update daily CI Docker image to torch 2.13.0 / CUDA 13.0 (#47738)

Fix missing github_utils.py download in PR CI dashboard workflow (#47668)

byebye torch 2.4 (#47609)

Use --flake-runs=1 in check_bad_commit.py for PR comment CI (#47522)

Run only @slow tests in PR comment CI (#47521)

@kaixuanliu

make examples under doc device agnostic (#47812)

cancel deterministic for XPU in gemma4 tests (#47790)

open fla support for XPU to benefit from the acceleration (#47799)

Fix DBRX MoE hidden size and expert GLU transposes (#47671)

Fix failing tests for axk1 and axk2 (#47727)

Fix failing tests for granite_swa and granitemoe_swa (#47723)

skip invalid test cases for inkling tests (#47493)

Fix failing tests for mimo_v2_flash (#47284)

Fix failing tests for zaya (#47268)

tipsv2_dpt: fix failing tests for XPU (#47292)

Fix some failed test cases related with XPU Expectations (#47173)

add paged attention tests support for XPU (#47163)

Fix multi-device mxfp4 dequantization race in _convert_moe_packed_tensors (#47423)

fix failed test cases for qwen3_omni_moe model (#47449)

Fix model tests for tipsv2 (#47356)

@vasqu

Fix ci style (#47818)

🚨 [Kernels] Refactor all linear attn models & native kernels fallback (#47630)

[Kernels] Refactor function handling (#46883)

Add AXK2 from SKT (#47528)

Add Harry to slow CI (#47454)

Update dev version on main (#47366)

[FP8] Bump kernels version (#47344)

@kmswin1

Hotfix axk2 indexer norm (#47810)

Fix A.X-K2 fp8 modules_to_not_convert normalization for the gated-norm MLP (#47578)

add_axk1 (#46867)

@remi-or

[Fix] Fix multi-head latent attention (MLA) (#47761)

[Fix] Wrong type hint in get_number_of_image_patches (#47788)

[Fix] Swapped height and width in KimiK25 (#47786)

Deprecate CB config in gen configuration (#47291)

[CB] Automatically switch attention implementation to flash (#47330)

@juliendenize

[Mistral] Add native tekken tokenizer support to AutoTokenizer (#47507)

Make tokenization_mistral_common importable without mistral_common installed (#47397)

[Mistral] Move MistralConverter into integrations/mistral/ package (#46603)

@IMvision12

Add support for batched Qwen2.5/3-Omni audio generation (#47186)

Fix Hunyuan-VL PIL image resize parity with reference preprocessing (#47233)

@jiqing-feng

🚨 Enable SDPA (and other attention backends) for T5 and propagate to the T5 family (#47014)

Fix shape mismatch in KyutaiSpeechToText generate() last window (#46952)

Fix typo in MusicgenForCausalLM.generate() (#46974)

Fix BlockMask crash in CTRL flex-attention generation (#46854)

@tarekziade

CI: Log GitHub API diagnostics to stderr (#47635)

CI: Add serge review relay workflow and review rules (#47610)

CI: use a single function for GH calls (#47474)

CI: use transformers-ci daily workflow with OTEL (#47360)

CI: let's run integration failure cron at 10pm (#47537)

fixed the benchmark script with DistributedConfig (#47568)

CI: fix torchaudio pinning +proper break in rnnt (#47422)

CI: add reproduce mode to serge verify caller (#47492)

CI: add serge verify (GPU) caller workflow (#47381)

ci: harden daily CI reporting against GitHub API rate limits (#47382)

Fix Daily CI reporting issues (#47364)

ci: Handle empty GitHub token in CI run lookup (#47362)

@daviswer

Add Granite-swa and Granitemoe-swa model support (#47179)

@ShareLer

Optimize flash attention max seqlen computation in vision attention (#47170)

@atharvajoshi10

Fix Cosmos 3 Edge Patch packing order (#47399)

Add Cosmos3 Edge model support (#47181)

Original source

Patch release v5.14.1

This patch solves a few issues which appeared when integrating Inkling model, most notably an issue affecting models using EncoderDecoderCache during assisted generation. It also fixes an issue that could appear during prefill with StaticCache and sdpa without padding for Inkling which uses a position_bias.

It contains the following commits:

• Fix sdpa prefill with position_bias (#47359) by @Cyrilvallez
• Fix assisted decoding for models with EncoderDecoder cache & OlmoHybrid (#47361) by @Cyrilvallez
• [FP8] Bump kernels version (#47344) by @vasqu
• Fix deepgemm on multiple devices (#47323) by @IlyasMoutawwakil

Original source

Release v5.14.0

New Model additions

Inkling (fresh from Thinking Machines): 975B total, 41B active

Add Inkling model #47347 by @molbap @Cyrilvallez @eustlb and @zucchini-nlp

Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers.

TIPSv2

Links: Documentation

Add TIPSv2 (#46347) by @Ternura143 in #46347

TIPSv2 DPT

Links: Documentation

Add TIPSv2 (#46347) by @Ternura143 in #46347

🚨 Breaking changes

GPTNeoX now remaps embed_out to lm_head and GPTBigCode has _supports_attention_backend = True enabled for vLLM compatibility; users relying on the previous weight naming or attention backend behavior for these models should update their code accordingly.

🚨 Fix GPTBigCode and GPTNeoX for the Transformers modelling backend for vLLM (#47198) by @hmellor

Kernels

Several kernel-related fixes and improvements were made, including pinning the kernels dependency to a compatible version in the benchmark workflow, removing a deprecated package_name argument from LocalLayerRepository, and making the DeepGEMM Triton fallback more robust when CUDA_HOME is unset or misconfigured. Additionally, SDPA prefill was updated to leverage the FlashAttention kernel with StaticCache, yielding significant performance gains (up to 260% faster for large input sizes).

Pin kernels to compatible version in benchmark workflow (#47339) by @tarekziade in [#47339]

[Fix] Remove deprecated argument from kernels call (#47100) by @remi-or in [#47100]

[Fix] Make DeepGEMM triton fallback more robust (#47126) by @remi-or in [#47126]

[sdpa] Allow prefill to use FA kernel with StaticCache (#47094) by @Cyrilvallez in [#47094]

Generation

Generation improvements include adding Multi-Token Prediction (MTP) decoding support, static ensemble verification for speculative decoding to improve draft token acceptance rates, and a fix for crashes in greedy assisted generation with different tokenizers. A misleading double-negative warning message for synced_gpus in continuous batching mode was also corrected.

[generation] Fix misleading synced_gpus warning in continuous batching (#47158) by @Partha-Shankar in [#47158]

[generate] Add proper MTP support (#46229) by @Cyrilvallez in [#46229]

Fix crash in greedy assisted generation with different tokenizers (#46936) by @Sunt-ing in [#46936]

[Generation] Add static ensemble verification for lossy speculative decoding (#45979) by @kasakh in [#45979]

Performance

Fixed a Flash Attention performance regression affecting models like Qwen3-VL and resolved a MoE decode optimization bug where the grouped-to-batched matrix multiplication switch was not applied to experts residing in submodels (e.g., VLMs with a nested text config).

Fix FA performance regression (#47134) by @andreasgoulas in [#47134]

Fix MoE decode optimization for experts living in a submodel (#47107) by @IlyasMoutawwakil in [#47107]

Make doc builds faster (#47099) by @mishig25 in [#47099]

Cache

Cache dispatch logic was simplified by introducing explicit layer-type mappings for sliding and static layers, reducing complexity in cache routing. Additionally, fixes were made for read-only cache failures in CPU CI environments and for MPS graph cache growth during variable-length batch training on Apple Silicon.

Fix CI read-only cache failures by patching cached_files in conftest (#47043) by @ydshieh in [#47043]

trainer: clear MPS graph cache via torch_empty_cache_steps (#45818) by @anagnorisis2peripeteia in [#45818]

[cache] Simplify cache dispatch based on layer_types (#47118) by @Cyrilvallez in [#47118]

Bugfixes and improvements

ci: cover xet as well (runtime error) (#47338) by @tarekziade in [#47338]

[docs] TokenizersBackend fallback (#47302) by @stevhliu in [#47302]

Resolve continuous batching XPU availability checks at runtime (#47185) by @kaixuanliu in [#47185]

[Nit] Add kernels_fallback_ok kwarg to is_flash_attn_N_available (#47318) by @remi-or in [#47318]

[Nit] Add expectations for gemma4 tests on H100 (#47311) by @remi-or in [#47311]

[docs] DeepGEMM requirements (#47324) by @stevhliu in [#47324]

DeepGEMM shouldn't pad on SM90 (#47313) by @IlyasMoutawwakil in [#47313]

Fix half-precision torch.compile crash in DETR-family sine position embeddings (#47238) by @David-Wu1119 in [#47238]

Fix hardcoded paths in siglip checkpoint/vocab loading (#47178) by @XanxusCrypto in [#47178]

Update AMD CI runner groups to amd-mi300 (#47307) by @Abdennacer-Badaoui in [#47307]

Point to Gemma 4 model in Gemma4ForCausalLM docstring example (#47255) by @lefft in [#47255]

Fix Qwen Omni batched text postprocessing (#47197) by @Sunt-ing in [#47197]

Fix AqlmConfig error messages to say "int" instead of "float" (#47089) by @Sreekant13 in [#47089]

Fix check for interactive stdout in _style function (#47283) by @smart8986 in [#47283]

Fix get_json_schema crash on non-string docstring choices (#47072) by @Sreekant13 in [#47072]

Make MODEL_IDS_TO_TOKENIZERS_BACKEND capture all DeepSeek R1 distills (#47296) by @hmellor in [#47296]

Update doc preprocessing regex to prevent ReDoS (#47187) by @WilliamRoyNelson in [#47187]

Shard on read Dtensor aware (#46717) by @3outeille in [#46717]

Switch AMD daily CI to mi300 runners (#47259) by @Abdennacer-Badaoui in [#47259]

tests: reduce processor test memory usage by using tiny Hub checkpoints (#47213) by @ydshieh in [#47213]

Torch compile backend defaults to "neuron" (#47035) by @michaelbenayoun in [#47035]

Fix flash-attn Docker build broken by setuptools 83 removing pkg_resources (#47251) by @ydshieh in [#47251]

Add heterogeneous config support (per-layer configuration) (#45333) by @eladsegal in [#45333]

[fix] update integration test values (#47146) by @eustlb in [#47146]

Fix DeepSpeed SP loss aggregation and LocalLayerRepository kwargs (#47073) by @sshivampeta in [#47073]

tests only for the top 10 download models (#47244) by @3outeille in [#47244]

Fix InputTokensDetails missing cache_write_tokens for openai>=2.34.0 (#47248) by @ydshieh in [#47248]

Revert "Trigger a scheduled run" (#47249) by @ydshieh in [#47249]

Remove Executorch from CI until latest version is supported and fully tested on CI env (#47242) by @IlyasMoutawwakil in [#47242]

Be more defensive with remap_legacy_layer_types for custom models (#47245) by @hmellor in [#47245]

Fix DistributedConfig docstring for unimplemented sp_plan (#47237) by @3outeille in [#47237]

Switch mlinter to 0.1.2 (#47172) by @tarekziade in [#47172]

Trigger a scheduled run (#47209) by @ydshieh in [#47209]

Make executorch exporter tests always use xnnpack backend (#47201) by @tarekziade in [#47201]

No agent PR descriptions (#45790) by @Rocketknight1 in [#45790]

Clarify that max_steps is required for datasets without len (#47155) by @albertvillanova in [#47155]

Cleanup pipelines, stop materializing generators (#47142) by @Rocketknight1 in [#47142]

Fix device_map computation when the no_split_modules have different sizes (#47203) by @Cyrilvallez in [#47203]

Add native FSDP2 module + migration (#46707) by @3outeille in [#46707]

Fix experts implementation in two spots (#47097) by @remi-or in [#47097]

[Fix] Remove old automatic cross attn pattern from output recorders (#47117) by @remi-or in [#47117]

🌐 [i18n-KO] Translate accelerator_selection.md to Korean (#47157) by @kkwjk2718 in [#47157]

[i18n-KO] Translate optimum.md to Korean and fix Furiosa typo (#47156) by @kkwjk2718 in [#47156]

[docs] fix curly quotes rendering to straight quotes (#47135) by @clijo in [#47135]

Fix custom code which doesn't know about the new linear layer type names (#47174) by @hmellor in [#47174]

Reject path traversal in the transformers_weights config field (#46890) by @LinZiyuu in [#46890]

[docs] Custom code conversion mapping (#47114) by @stevhliu in [#47114]

Add exporters min version requirements and test skip (#47161) by @IlyasMoutawwakil in [#47161]

tests: reduce processor test memory usage and use tiny test assets (#47168) by @ydshieh in [#47168]

Clarify input device placement in the Quicktour inference example (#47136) by @samyuktahegde in [#47136]

Extend continuous batching memory prediction test to XPU (#47159) by @sywangyi in [#47159]

Fix case where _LazyAutoMapping.register is passed a str key (#47148) by @hmellor in [#47148]

[docs] MoE decode switching (#47149) by @stevhliu in [#47149]

add XPU output expectations for minicpm3 tests (#47092) by @kaixuanliu in [#47092]

Diffusion gemma: fix failed test cases (#47025) by @kaixuanliu in [#47025]

add XPU Expectation for cosmos3_omni tests (#46880) by @kaixuanliu in [#46880]

Fix IndexError Bug in XLMRoberta/Camembert ForMultipleChoice by restoring the pooler (#47147) by @pariidanDKE in [#47147]

Skip caching_allocator_warmup on Neuron (no reuse pool to warm; currently OOMs) (#47029) by @dacorvo in [#47029]

[docs] continuous batching (offloading behavior, max batch tokens, block size minimum) (#46925) by @stevhliu in [#46925]

[docs] fix autolinks (#46968) by @stevhliu in [#46968]

revert #47121 (#47144) by @eustlb in [#47144]

Fix output labels for AudioFlamingo3 (and related) models (#47112) by @ebezzam in [#47112]

Fix false len claims in Trainer docstrings (#47131) by @albertvillanova in [#47131]

processor tests: use tiny Hub repos to reduce CI memory (#47115) by @ydshieh in [#47115]

[serge] Fix 12 integration tests for model dac failing with output_mismatch (tensor values differ (6), other (6)) (#47121) by @sergereview[bot] in [#47121]

Fix CLI compatibility with huggingface_hub 1.22 (#47059) (#47064) by @dhruv7477 in [#47064]

we want to run the CI in the release branches (#47125) by @tarekziade in [#47125]

Small improvement (#47128) by @Cyrilvallez in [#47128]

[Model] Support use_cache=False for DeepSeek V4 (#46965) by @kylesayrs in [#46965]

docs-fix: IMDb dataset link in sequence classification guide (#47062) by @abhishekkapoorx in [#47062]

Fix AltCLIP text embedding resize test (#47079) by @IMvision12 in [#47079]

fix mask return-type contract regression and add correctness guard for (#47019) by @kaixuanliu in [#47019]

Fix save_pretrained with offloading and weight conversions (#47018) by @Cyrilvallez in [#47018]

Update dev (#47044) by @vasqu in [#47044]

[Gemma4] Update 1 integration test (#47042) by @vasqu in [#47042]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@ArthurZucker

v5.14.0

@tarekziade

ci: cover xet as well (runtime error) (#47338)

Pin kernels to compatible version in benchmark workflow (#47339)

Switch mlinter to 0.1.2 (#47172)

Make executorch exporter tests always use xnnpack backend (#47201)

Remove executorch from all-latest-gpu image + add torch smoke test (#47196)

we want to run the CI in the release branches (#47125)

@remi-or

[Nit] Add kernels_fallback_ok kwarg to is_flash_attn_N_available (#47318)

[Nit] Add expectations for gemma4 tests on H100 (#47311)

[Fix] Remove deprecated argument from kernels call (#47100)

[Fix] Make DeepGEMM triton fallback more robust (#47126)

Fix experts implementation in two spots (#47097)

[Fix] Remove old automatic cross attn pattern from output recorders (#47117)

@ydshieh

tests: reduce processor test memory usage by using tiny Hub checkpoints (#47213)

Fix flash-attn Docker build broken by setuptools 83 removing pkg_resources (#47251)

Fix InputTokensDetails missing cache_write_tokens for openai>=2.34.0 (#47248)

Revert "Trigger a scheduled run" (#47249)

Fix CI read-only cache failures by patching cached_files in conftest (#47043)

Trigger a scheduled run (#47209)

tests: reduce processor test memory usage and use tiny test assets (#47168)

processor tests: use tiny Hub repos to reduce CI memory (#47115)

@eladsegal

Add heterogeneous config support (per-layer configuration) (#45333)

@eustlb

[fix] update integration test values (#47146)

revert #47121 (#47144)

@Ternura143

Add TIPSv2 (#46347)

Original source