NewYour coding agent can read the release notes before it upgrades.Set up the MCP server โ
PyPI, #269 most downloaded on PyPI
Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Latest stable
5.24.0
Last release today
21 Nov 2026
Ships fairly regularly
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
6 versions withdrawn
withdrawn after publishing
10 years old
254 releases ยท first in 2016
Drag across the ruler to compare, or tap a release to open it.
past_key_values and cache_params are renamed to cache across the library. The old names still work with a deprecation warning; see the dedicated sectiโฆ
Inkling 2 Mini is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs. It is a smaller sibling of Inkling 2 Small, added in v5.23.0. It has 24 decoder layers instead of 48, and compensates with a looping mechanism in which layers 6 to 17 run three times. A learned message-passing mechanism mixes the residuals of the previous pass into the attention and mixture-of-experts inputs of each loop layer. The decoder has a hidden size of 2304 and routes each token to 8 of 128 experts. The image encoder, tokenizer and processor are the same as Inkling 2 Small's.
A research checkpoint (TML-d24-varloop-research) has the same architecture but supports a variable number of loop passes. Fewer passes are faster, and more passes improve quality. Both checkpoints contain NVFP4 and MXFP8 quantized layers, which are dequantized at load time.
Links: Documentation | Blog Post | HF Blog Post
Apertus 1.5 is a multimodal model from the Swiss AI Initiative that accepts text, images and audio and generates text. It extends the Apertus language model with the EMU3.5 vision tokenizer for images and WavTokenizer for audio. The EMU3.5 tokenizer is an image-only VQGAN with the same encoder as Chameleon's, and it assigns codes by the largest inner product with the codebook. The media tokenizers turn images and audio into discrete tokens, which the language backbone processes alongside text in a shared input vocabulary. The output vocabulary contains only text tokens, so image and audio generation are not supported.
Links: Documentation
WavTokenizer encodes 24 kHz mono audio into discrete tokens using a single codebook, at 40 or 75 tokens per second depending on the checkpoint, and reconstructs audio from those tokens. WavTokenizerModel encodes and decodes, and WavTokenizerEncoderModel only encodes. Speech, music/audio and unified checkpoints are available. The 40-token-per-second variant is the audio tokenizer of Apertus 1.5. Training the codec is not supported.
Links: Documentation | Paper
Dory is a looped (latent-recurrent) language model from NVIDIA. It uses a normalized-GPT architecture with interleaved sliding-window and full attention layers. A shared block of layers is applied repeatedly, so Dory can spend more compute per token. The loop count can be changed at inference time on the same checkpoint to trade speed against quality.
Links: Documentation | Paper (nGPT)
HY4-VL (Hunyuan V4 Vision-Language) combines the HYV4 mixture-of-experts text backbone with a vision encoder for image-text and video-text generation. It supports dynamic-resolution visual inputs and uses multimodal rotary position embeddings (mRoPE) to represent temporal and spatial positions. It builds on the Qwen2-VL design and shares its vision patch merger with HunYuan-VL.
Links: Documentation
Omnilingual ASR is a suite of multilingual speech-recognition models from Meta FAIR that covers more than 1,600 languages. Each model pairs a Wav2Vec2-style audio encoder with one of two heads:
OmniASRForCTC): a linear projection to the vocabulary, decoded non-autoregressively with greedy CTC.OmniASRForConditionalGeneration): audio embeddings are projected into a Llama decoder that generates the transcription. It optionally takes a language code (e.g. "eng_Latn"), which generally improves transcription quality.Both variants come in 300M, 1B, 3B and 7B sizes.
Links: Documentation | CTC Documentation | Paper
Lara Mini is a multilingual base language model from Translated with 7.2B total and 0.6B active parameters per token. It is a hybrid of linear-time recurrent layers, sliding-window attention and a few global attention layers, each followed by a fine-grained mixture of experts. The weights are released under the CC BY-NC-SA 4.0 license (non-commercial use) in a gated repository.
Links: Documentation | Paper
SarvamMLA is the architecture of Sarvam-105B, a 105B-parameter mixture-of-experts language model from Sarvam AI. It combines Multi-head Latent Attention (MLA) with sparse MoE routing: 128 routed experts with 8 active per token, plus 1 shared expert. It uses sigmoid routing with group-based top-k selection and YaRN rotary embeddings for contexts up to 131K tokens. The architecture is the same as DeepSeek-V3, with a custom configuration.
Links: Documentation
SarvamMoE is the architecture of Sarvam-30B, Sarvam AI's 30B-parameter mixture-of-experts model. It uses grouped-query attention with QK normalization, and 128 routed experts with 6 active per token, plus 1 shared expert. The architecture is the same as GLM-4-MoE, so the auto classes load it with the GLM-4-MoE implementation and only SarvamMoEConfig is specific to SarvamMoE.
Links: Documentation
EXAONE covers the EXAONE 3.0, EXAONE 3.5 and EXAONE Deep language models from LG AI Research. Their architecture is the same as Llama, so the auto classes load them with LlamaModel and LlamaForCausalLM, and only ExaoneConfig is specific to EXAONE. The released checkpoints are renamed to the Llama names when loading and back when saving, so they no longer need trust_remote_code=True.
Links: Documentation
BailingHybrid, like Ling 3.0 in v5.20.0 (#49737) by @icavanBeamForTokenClassification head (#50634) by @AymenKallalapast_key_values / cache_params โ cache (#50585, #50730)past_key_values and cache_params are renamed to cache across the library. The old names still work with a deprecation warning; see the dedicated section below.
past_key_values/cache_params to cache (#50585) by @CyrilvallezAttention models used to call their cache past_key_values, while Mamba-style and linear-attention models called it cache_params. Hybrid models therefore had to juggle two names for the same object. There is now a single name, cache, everywhere: model forward signatures, generate, prepare_inputs_for_generation, the masking utilities, and every ModelOutput.
# Before
outputs = model(input_ids, past_key_values=past_key_values, use_cache=True)
next_cache = outputs.past_key_values
# Now
outputs = model(input_ids, cache=cache, use_cache=True)
next_cache = outputs.cachepast_key_values= / cache_params= are renamed to cache automatically, with a one-time deprecation warning. Model forward and prepare_inputs_for_generation accept the old names until v5.40.0, and generate accepts them until v6.0.0. Passing both an old name and cache raises a ValueError.outputs.past_key_values, outputs.cache_params and outputs["past_key_values"] still return the cache. Output classes can also still be built with the old field names (e.g. CausalLMOutputWithPast(past_key_values=...)), so most remote-code models keep working unchanged.cache. Layers now call cache.update(key_states, value_states, layer_idx, kwargs=kwargs), so custom Cache subclasses that override update should accept the extra kwargs argument.๐จ peft>=0.22.0 is now required.
๐ Docs: https://huggingface.co/docs/transformers/main/en/kv_cache
Generative multimodal models now return shared output classes (MultimodalBaseModelOutputWithPast, MultimodalCausalLMOutputWithPast, and their MoE variants) instead of one output class per model. Code that checks for a model-specific output class should switch to these.
modeling_attn_mask_utils.py has been removed. Every function in it had been deprecated in favor of transformers.masking_utils. Remote-code models that still import _prepare_4d_causal_attention_mask or AttentionMaskConverter from it must move to transformers.masking_utils.
LayoutLMv3 (and the PP-DocLayoutV2 reading-order model that shares its code) now go through the attention interface, so SDPA and flex attention are supported. The relative position bias is built once per forward pass and passed to attention as position_bias.
Beyond the rename, this release adds static-cache support to assisted decoding and fixes generate with flex attention and a static cache. StaticCache now allocates values with the value states' dtype, and the cache is kept under activation checkpointing when gradients are off. The is_linear / is_croppable properties are fixed for hybrid linear/full-attention models, BigBird block-sparse decoding works with a KV cache, and KV sharing is restored for the Gemma 4 assistants.
test_bc_cache_name_{forward,generate} (#50757) by @ydshiehGeneration fixes cover sampled beam search (candidates are sorted again before finalization), constrained sampling (batch IDs are preserved), generate on MPS with encoder-decoder models, and TextStreamer, which no longer drops spaces or lags on text without spaces such as Chinese, Japanese and Korean. StopStringCriteria initializes faster, and its cache now actually hits.
Continuous batching now works under torch.inference_mode() and finds sharded key/value projections under any decoder path. The per-token memory estimate scales with the TP size, warmup handles async and static inputs correctly, and an out-of-bounds error in the top-k processor is fixed.
Chat parsing gets a new python-call-list content parser, better handling of bare keys and duplicate XML arguments, and three further parser fixes. apply_chat_template accepts message objects, and assistant masks are correct with left truncation. transformers serve handles a missing Hub cache, lists loaded models in /v1/models, keeps local paths containing @, and keeps assistant content when tool_calls is empty.
python-call-list content parser (#50619) by @nico-martinunquoted_keys (#50917) by @gagan-baghelMultimodal pipelines now accept media-only inputs and string assistant prefills, keep Mistral prefills in Voxtral, stop mutating generation kwargs, and return chat messages as input_text when returning tensors. pipeline also respects a device_map that targets a single device.
pipeline ignoring device_map when it targets a single device (#50645) by @wanhakimVision fixes cover Muse Glimmer's embedding order, GotOCR2 and Emu3 image sizes, Kimi video inputs, NemotronH Omni video forwarding, SAM3 RoPE precision in bf16, and DETR-family configs with dilation=None. SigLIP2 and LFM2-VL now process all images at once with tensor ops. Flash Attention is turned off for multimodal models whose image mask it would drop.
Audio fixes cover the OmniASR CTC blank token, Qwen3 Omni audio kwargs, Qwen3-ASR's truncation=False default, PeAudioVideo loading with timm 1.0.31, and the audio training examples.
truncation=False default for Qwen3-ASR audio (#50740) by @hmellorDequantized models are now saved without the quantizer's weight conversions, and the unquantized weights of a dequantized FP8 checkpoint are cast to the model's dtype. The quantization docs also get a standard template for each method.
FSDP2 modules are resharded before the optimizer is created, pipeline parallelism spreads layers more evenly, and TP sharding of tied weights is fixed.
Exported ONNX, OpenVINO and ExecuTorch programs load without trust_remote_code. Pickled .pt2 programs, which can run arbitrary code when loaded, now require trust_remote_code=True.
test_generate_from_multimodal_encoder_outputs (#51049) by Note truncated.
Your coding agent can read these notes before it upgrades. Set up the MCP server โ
๐จ Deprecate Wav2Vec2ProcessorWithLM and pyctcdecode/kenlm LM decoding (#50408) by @eustlb
<img width="1200" height="675" alt="image" src="https://github.com/user-attachments/assets/9115e142-1be3-4f79-a51d-ea0e5c25b152" />
Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for use in English and other languages and across multiple coding languages. Its language backbone has 48 mixture-of-experts decoder layers, each routing tokens to 8 of 384 experts, for 594B total and 17B active parameters. It keeps the Inkling decoder (learned relative attention, short causal convolutions, mixture-of-experts layers) and adds block attention residuals, gated attention with a learned sink, and dynamic key/value convolutions. Images are encoded by a Perception Encoder-based vision transformer and projected into the language model embeddings, one embedding per 40x40 pixel patch.
Links: Documentation | Blog Post
JEV is a decision head that scores single-token labels with a linear layer at the last token of the prompt, soft-capped like the logits of a language model head. JevForDecision is the model of the checkpoint and its head, and it reads the head files shipped next to the weights (head.safetensors and judge_config.json for JEV). The same head is also used by GEV and reads the 255 answer codes of pplx-decider and d3, and it can be run through the experimental decision pipeline or directly with decide.
Note: JEV and the decision pipeline are currently experimental. Their APIs, configuration, and behavior may change as development continues. They should be used with caution in production environments.
Links: Documentation
<img width="732" height="815" alt="image" src="https://github.com/user-attachments/assets/5a67fe26-2cee-4bbf-bc51-65e8f843b756" />
LFM3 is the third generation of Liquid Foundation Models from Liquid AI. Like LFM2, it is a hybrid architecture that combines gated short convolutions with grouped-query attention, with the attention layers split between full-attention and sliding-window layers. The attention layers apply RMSNorm to the query and key heads and gate their output with a per-head sigmoid. Every layer applies a sandwich norm to the operator and feed-forward outputs before the residual addition.
Links: Documentation
Dots 3 Note Preview is a mixture-of-experts causal language model that accepts native text, image, video, and audio inputs. It uses a shared vision encoder for images and videos and a Whisper-style audio encoder, and both encoders project their outputs into the language model's hidden space before autoregressive text generation. BF16 and FP8 inference are supported.
Links: Documentation
LFM3-VL is a vision-language model family from Liquid AI, built on the LFM3 language model. It takes images and videos at their native resolution together with text. It combines a hybrid vision encoder (a linear patchifier, gated depthwise 2D convolution layers, and bidirectional grouped-query attention layers with 2D rotary embeddings) with a projector and an LFM3 language model. The projector merges each 2x2 block of patch features into one language token through a layer norm and a two-layer MLP.
Links: Documentation
IQuest-Coder-V1 is a family of code language models developed by IQuest Lab, available in 7B, 14B and 40B sizes with base, instruct and thinking variants. Its architecture is the same as Llama, so the Transformers auto classes load it with LlamaModel and LlamaForCausalLM, and only IQuestCoderConfig is specific to IQuestCoder. The looped IQuest-Coder-V1-*-Loop-* checkpoints use a different architecture and aren't supported.
Links: Documentation
InternLM3 is an 8B instruction-tuned language model developed by Shanghai AI Laboratory (checkpoint: internlm/internlm3-8b-instruct). Its architecture is the same as Llama, so the auto classes load it with the Llama implementation (LlamaModel and LlamaForCausalLM), and only InternLM3Config is specific to InternLM3. The config names the attention biases qkv_bias and the output projection and MLP biases bias. InternLM3Config loads these as attention_bias and mlp_bias and requires them to be equal, as they are in the released checkpoints.
Links: Documentation
ep_router expert-parallel style (#49615) by @IlyasMoutawwakilep_dispatch_experts (#49613) by @IlyasMoutawwakildeepgemm_megamoe EP implementation for DeepGEMM Mega MoE (#49606) by @IlyasMoutawwakil[audio] to torchaudio + soundfile + soxr (drop librosa and phonemizer) (#50610) by @eustlbExportedModel, Hub save/load, one backend registry (#50029) by @IlyasMoutawwakilkernels] Bump version (#50307) by @vasquhttps://github.com/user-attachments/assets/2b17b2ab-b2b8-4e34-a983-4a7177330e4e
Every export now returns an ExportedModel. You call it like the original model (same kwargs, same output class), push it to the Hub, and load it back on the backend's own runtime. The same API covers Dynamo, ONNX Runtime, OpenVINO and ExecuTorch.
from transformers import AutoModel, AutoTokenizer
from transformers.exporters import ExportedModel, OnnxConfig, OnnxExporter
model = AutoModel.from_pretrained("google/embeddinggemma-2")
tokenizer = AutoTokenizer.from_pretrained("google/embeddinggemma-2")
inputs = tokenizer(["Exporters are here!"], return_tensors="pt")
exported = OnnxExporter().export(model, inputs, OnnxConfig(dynamic=True, ort_fusions="cuda"))
exported.push_to_hub("my-org/embeddinggemma-2-onnx")
exported_model = ExportedModel.from_pretrained("my-org/embeddinggemma-2-onnx", device="cuda")
outputs = exported_model(**inputs) # same output class as model(**inputs)
OnnxConfig(ort_fusions="cpu" | "cuda" | "webgpu" | "dml") rewrites the graph into ONNX Runtime's fused transformer ops at export time: attention, GroupQueryAttention over the KV cache, rotary, norms and MoE. OpenVINO exports now hit OpenVINO's fused CPU kernels for attention, RoPE, MoE experts and compressed int8/int4 weights, which makes decode 2.4โ3.9ร faster than before.torch.compile: on EmbeddingGemma2 (8 ร 128 tokens), ONNX Runtime with the TensorRT EP is 7.6ร faster than eager PyTorch and 1.4ร faster than torch.compile on an A100. OpenVINO is 4.1ร faster than eager and 2.2ร faster than torch.compile on a Xeon Platinum 8570 (4.9ร and 2.6ร with int8 weights).subfolder of the model repo. Its export.json records the config, input shapes, package versions and I/O names. Quantized exports (PT2E, NNCF, ONNX Runtime) save and run the same way.๐จ export() returns an ExportedModel, and an ONNX program is an onnx_ir.Model. OnnxConfig's and OpenVINOConfig's output_path options are gone in favour of save_pretrained. Custom backends register with register_backend.
๐ Docs: https://huggingface.co/docs/transformers/main/en/exporters
This release fixes a range of vision-model bugs: classification pipelines now respect problem_type, and several models get corrected behavior (Step3.7 RoPE axis order, MiniMax-M3-VL packed attention isolation, Qwen VL positions with reused caches, PaddleOCR-VL interpolation alignment, Kosmos2 SDPA masking, and Emu3 float image handling). Chunked prefill now keeps image inputs on the
This release improves distributed training and inference. Expert parallelism gains a selectable ep_implementation (all-reduce for CUDA-graph-friendly inference, all-to-all for training). The Trainer now reduces FSDP/HSDP gradients only on the last micro-batch, and the token-count normalization is fixed to use true division. Tensor-parallel fixes cover attention sinks (including Beam attention) and tied-weight comparisons on TPU
ep_dispatch_experts: all-reduce at inference, all-to-all in training (#49429) by @IlyasMoutawwakil in [#49429]distributed] Trainer: reduce FSDP/HSDP gradients on the last micro-batch only (#50286) by @3outeille in [#50286]Beam] Fix TP on attn (#50302) by @vasqu in [#50302]This release fixes several cache correctness issues: static-cache prefill attention in Gemma4's KV-shared layers, Qwen VL positions when a cache is reused with a new image or video, and DeepSeek-V4 compression-state reordering during beam search. It also improves cache sharing for Mistral Large 4 and adds OpenVINO export fusion for the KV cache and attention. Smaller changes cover CI and test fixes, including vLLM CI memory
This release fixes several hub kernel issues: the MPS RMSNorm kernel repo id was corrected, the MegaBlocksMoeMLP mapping was restored for MXFP4 GPT-OSS, and a failed flash-attention kernel load is now retried instead of leaving models unregistered. It also raises the minimum kernels version (fixing ROCm issues and a multi-GPU bug), bumps mamba-ssm kernels to v4
GptOss] Fix the kernels autodispatch test (#50630) by @vasqu in [#50630]kernels] Bump min version (#50509) by @vasqu in [#50509]lazy_import_flash_attention (#50391) by @albertvillanova in [#50391]MegaBlocksMoeMLP hub kernel mapping for MXFP4 GPT-OSS (#50150) by @jiqing-feng in [#50150]This release fixes a range of attention bugs: vision-attention isolation in MiniMax-M3-VL, SDPA decode under torch.compile, eager NaNs in Qwen4-Exp with left padding, an unintended causal mask in Kosmos-2's SDPA projection, and default full-attention mask selection that wrongly picked linear layers. It also retries failed attention kernel loads, excludes padding from the repetition penalty, and lets the O
SDPA] Fix decode under compile (#50388) by @vasqu in [#50388]This release fixes several generation bugs: padded generation in RecurrentGemma, batched decoding in Falcon-H1, beam search cache reordering in DeepSeek-V4, and batched metadata and beam scores in TextGenerationPipeline. It also lets an unchanged GenerationConfig be saved without strict-validation errors, avoids copying the KV cache when decoding with grouped-query attention (GQA), and fixes an `Attribut
layer_types (#50577) by @ydshieh in [#50577]This release fixes several quantization bugs: FP8 modules_to_not_convert entries are now renamed like weight names, so modules such as f_a_proj and f_b_proj are no longer wrongly quantized. MXFP4 experts now dequantize to the requested dtype instead of always using bfloat16. The MegaBlocksMoeMLP hub kernel mapping is restored for MXFP4 GPT-OSS, which fix
modules_to_not_convert entries like weight names (#50354) by @albertvillanova in [#50354]This release makes audio handling more correct and robust. AudioClassificationPipeline now respects the model's problem type (raw outputs for regression, sigmoid for multi-label, softmax otherwise), and VibeVoice and VibeVoice-ASR avoid GPU syncs when removing audio padding and gain a sample argument to disable VAE noise for deterministic outputs. Chat templates can also take audio decoded from video via `{"type": "aud
sample argument to disable VAE noise in audio features (#50179) by @hmellor in [#50179]Gemma4] Fix FA slicing (#50681) by @vasqu in [#50681]ep_dispatch_experts (#49607) by @IlyasMoutawwakil in [#49607]make run-mic easier to read and to target (#50652) by @tarekziade in [#50652]HfArgumentParser on Python 3.15 (#50651) by @qgallouedec in [#50651]Offloading] Fix tests for some models (#50631) by @vasqu in [#50631]FA] Fix FA tests for unique models (#50632) by @vasqu in [#50632]ModelRunner) (#50032) by @IlyasMoutawwakil in [#50032]OnnxConfig(ort_fusions=...)) (#50309) by @IlyasMoutawwakil in [#50309]distributed] Checkpoint the layer's forward, not __call__, so FSDP2 hooks stay outside the checkpoint (#50490) by @3outeille in [#50490]transcription input arg to apply_transcription_request (#49869)" (#50587) by @ebezzam in [#50587]ForPreTraining models (argmax near-tie) (#50575) by @ydshieh in [#50575]grounding_dino failing with output_mismatch (tensor values differ (6), oth (#49886) by @sergereview[bot] in [#49886]test_causal_lm_loss_honors_shift_labels (#50533) by @ydshieh in [#50533]per_layer_config (#50486) by @zucchini-nlp in [#50486]apply_chat_template (#45422) by @qgallouedec in [#45422]test_eager_matches_batched_and_grouped_inference in fp16 / bf16 (#50500) by @ydshieh in [#50500]pegasus failing with other (other (2)) (#49507) by @sergereview[bot] in [#49507]Colpali] Fix offload tests (#50432) by @vasqu in [#50432]distributed] Support HSDP with expert parallelism (#50293) by @3outeille in [#50293]distributed] Add HSDP (dp_replicate_size) to DistributedConfig (#50285) by @3outeille in [#50285]Gemma4Audio2TextModelTest::test_training_gradient_checkpointing_use_reentrant_true (#50410) by @ydshieh in [#50410]T5GemmaModelTest::test_decoder_model_past_with_large_inputs (#50409) by @ydshieh in [#50409]Owlv2ModelTest::test_eager_matches_sdpa_inference_01_fp16_pad_left (#50403) by @ydshieh in [#50403]PocketTTSAudioTokenizerModelTest::test_batching_equivalence on CPU (#50402) by @ydshieh in [#50402]distributed] Divide the expert gradients once when efsdp_size == 1 (#50292) by @3outeille in [#50292]Rope] Fix per layer rope in head dim manipulation (#49864) by @vasqu in [#49864]FA] Fix head dim manipulation for more special models (#49862) by @vasqu in [#49862]transcription input arg to apply_transcription_request (#49869) by @ebezzam in [#49869]FA] Fix old models that do not use a proper interface (#49859) by @vasqu in [#49859]Moe] Fix grouped mm fallback (#49857) by @vasqu in [#49857]device_id only for NCCL (#50281) by @michaelbenayoun in [#50281]bboxes being overwritten before use in prepare_example (#50218) by @Jose-Marcial-GF in [#50218]X as X re-export syntax and duplicate module entries (#50253) by @ydshieh in [#50253]trainer.push_to_hub (#49874) by @not-lain in [#49874]save_pretrained on every rank for PEFT model (#49316) by @michaelbenayoun in [#49316]apply_transcription_request (#49944) by @hmellor in [#49944]TRANSFORMERS_SDPA_FORCE_GQA envvar in sdpa_attention_forward (#49459) by @jakobknauer in [#49459]WatermarkDetector p-value computation (#50123) by @jonasschaefer in [#50123]FA] Less flaky FA tests (#49861) by @vasqu (direct commit on main)The following contributors have made significant changes to the library over the last release:
make run-mic easier to read and to target (#50652)Gemma4] Fix FA slicing (#50681)Offloading] Fix tests for some models (#50631)FA] Fix FA tests for unique models (#50632)GptOss] Fix the kernels autodispatch test (#50630)kernels] Bump min version (#50509)Colpali] Fix offload tests (#50432)SDPA] Fix decode under compile (#50388)Rope] Fix per layer rope in head dim manipulation (#49864)FA] Fix head dim manipulation for more special models (#49862)kernels] Bump version (#50307)Beam] Fix TP on attn (#50302)FA] Less flaky FA tests (#49861)FA] Fix old models that do not use a proper interface (#49859)Moe] Fix grouped mm fallback (#49857)layer_types (#50577)ForPreTraining models (argmax near-tie) (#50575)test_causal_lm_loss_honors_shift_labels (#50533)test_eager_matches_batched_and_grouped_inference in fp16 / bf16 (#50500)Gemma4Audio2TextModelTest::test_training_gradient_checkpointing_use_reentrant_true (#50410)T5GemmaModelTest::test_decoder_model_past_with_large_inputs (#50409)Owlv2ModelTest::test_eager_matches_sdpa_inference_01_fp16_pad_left (#50403)PocketTTSAudioTokenizerModelTest::test_batching_equivalence on CPU (#50402)X as X re-export syntax and duplicate module entries (#50253)[audio] to torchaudio + soundfile + soxr (drop librosa and phonemizer) (#50610)transcription input arg to apply_transcription_request (#49869)" (#50587)transcription input arg to apply_transcription_request (#49869)Some in between patch for security and distributed fixes. See the included commits :hugs:
Some in between patch for security and distributed fixes. See the included commits :hugs:
Full Changelog: https://github.com/huggingface/transformers/compare/v5.22.1...v5.22.2
Some in between patch for security and distributed fixes. See the included commits ๐ค
Full Changelog: v5.22.1...v5.22.2
We just had to fix the documentation!
We just had to fix the documentation!
** [Fix] Remove WIP doc for Ml4 (https://github.com/huggingface/transformers/pull/50124) from @remi-or
Remove deprecation cycle for candidate generators (#50034) by @Cyrilvallez in [#50034]
<img width="1329" height="780" alt="image" src="https://github.com/user-attachments/assets/1cff9ca7-19e3-4622-8db6-fb1cfa842579" />
Mistral Large 4 is Mistral's largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.
Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.
Mistral Large 4 includes the following architectural choices:
Mistral Large 4 offers the following capabilities:
low, medium, high (default) โ so the same model can answer a quick chat reply or work through a complex agentic run.The model is released under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use โ operating a Model-as-a-Service business requires a separate commercial license with Mistral.
'low' / 'medium' / 'high' โ default is 'high'.reasoning_effort="high" for complex prompts and agentic usage, 'low' for fast chat replies and 'medium' for day to day usage.temperature=1.0top_p=0.95FP8 quantized models can now be used for adapter training (such as LoRA with PEFT). Backward passes were added to the dense and grouped FP8 matmuls, and is_trainable is now True. Backward also dequantizes experts in chunks and skips autograd at inference, which cuts peak memory in the MoE layer backward from about 28.9 GiB to 2.8 GiB at 8k tokens. Separately, the AMD CI image now install
PermuteForRope to act on single tensors while renaming (#50092) by @Cyrilvallez in [#50092]rope_scaling setter for configs that assign it during init (#50080) by @hmellor in [#50080]git blame on a partial clone (#50007) by @nrenvo in [#50007]The following contributors have made significant changes to the library over the last release:
_init_weights gaps in 7 audio models and enable test_can_init_all_missing_weights for them (#49386)Fix rope_scaling setter for configs that assign it during init
** Fix rope_scaling setter for configs that assign it during init (#50080 )
We broke remote code loading in vllm, this fixes it sorry everyone!
One column per quarter.