PyPI · #225 most downloaded on PyPI

transformers

5.17.0180M downloads/mohuggingface/transformers ↗New releases in your daily digest, changelog included.

Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Last release 10 days ago
09 Sep 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
6 versions withdrawn
withdrawn after publishing
10 years old
242 releases · first in 2016

Release timeline

242 releases since 2016

Releases

5.0.026 Jan 2026Breaking
Biggest breaking change is in transformers chat. This command starts a terminal UI to interact with a chat model. It used to also be able to start a C…
Transformers v5 release notes

<img width="1800" height="1013" alt="image" src="https://github.com/user-attachments/assets/7b5187d7-6945-4108-a546-6d1d7bfb55e3" />

Highlights

Significant API changes: dynamic weight loading, tokenization

Backwards Incompatible Changes

Bugfixes and improvements

We have a migration guide that will be continuously updated available on the main branch, please check it out in case you're facing issues: migration guide.

Highlights

We are excited to announce the initial release of Transformers v5. This is the first major release in five years, and the release is significant: 1200 commits have been pushed to main since the latest minor release. This release removes a lot of long-due deprecations, introduces several refactors that significantly simplify our APIs and internals, and comes with a large number of bug fixes.

We give an overview of our focus for this release in the following blogpost. In these release notes, we'll focus directly on the refactors and new APIs coming with v5.

This release is the full V5 release. It sets in motion something bigger: going forward, starting with v5, we'll now release minor releases every week, rather than every 5 weeks. Expect v5.1 to follow next week, then v5.2 the week that follows, etc.

We're moving forward with this change to ensure you have access to models as soon as they're supported in the library, rather than a few weeks after.

In order to install this release, please do so with the following:

pip install transformers

For us to deliver the best package possible, it is imperative that we have feedback on how the toolkit is currently working for you. Please try it out, and open an issue in case you're facing something inconsistent/a bug.

Transformers version 5 is a community endeavor, and we couldn't have shipped such a massive release without the help of the entire community.

Significant API changes

Dynamic weight loading

We introduce a new weight loading API in transformers, which significantly improves on the previous API. This weight loading API is designed to apply operations to the checkpoints loaded by transformers.

Instead of loading the checkpoint exactly as it is serialized within the model, these operations can reshape, merge, and split the layers according to how they're defined in this new API. These operations are often a necessity when working with quantization or parallelism algorithms.

This new API is centered around the new WeightConverter class:

class WeightConverter(WeightTransform): operations: list[ConversionOps] source_keys: Union[str, list[str]] target_keys: Union[str, list[str]]

The weight converter is designed to apply a list of operations on the source keys, resulting in target keys. A common operation done on the attention layers is to fuse the query, key, values layers. Doing so with this API would amount to defining the following conversion:

conversion = WeightConverter( ["self_attn.q_proj", "self_attn.k_proj", "self_attn.v_proj"], # The input layers "self_attn.qkv_proj", # The single layer as output operations=[Concatenate(dim=0)], )

In this situation, we apply the Concatenate operation, which accepts a list of layers as input and returns a single layer.

This allows us to define a mapping from architecture to a list of weight conversions. Applying those weight conversions can apply arbitrary transformations to the layers themselves. This significantly simplified the from_pretrained method and helped us remove a lot of technical debt that we accumulated over the past few years.

This results in several improvements:

Much cleaner definition of transformations applied to the checkpoint

Reversible transformations, so loading and saving a checkpoint should result in the same checkpoint

Faster model loading thanks to scheduling of tensor materialization

Enables complex mix of transformations that wouldn't otherwise be possible (such as quantization + MoEs, or TP + MoEs)

Linked PR: https://github.com/huggingface/transformers/pull/41580

Tokenization

Just as we moved towards a single backend library for model definition, we want our tokenizers, and the Tokenizer object to be a lot more intuitive. With v5, tokenizer definition is much simpler; one can now initialize an empty LlamaTokenizer and train it directly on your corpus.

Defining a new tokenizer object should be as simple as this:

from transformers import TokenizersBackend, generate_merges from tokenizers import pre_tokenizers, Tokenizer from tokenizers.model import BPE class Llama5Tokenizer(TokenizersBackend): def __init__(self, unk_token="<unk>",bos_token="<s>", eos_token="</s>", vocab=None, merges=None ): if vocab is None: self._vocab = { str(unk_token): 0, str(bos_token): 1, str(eos_token): 2, } else: self._vocab = vocab self._merges = merges self._tokenizer = Tokenizer( BPE(vocab=self._vocab, merges=self._merges, fuse_unk=True) ) self._tokenizer.pre_tokenizer = pre_tokenizers.Metaspace( replacement="▁", prepend_scheme=_get_prepend_scheme(self.add_prefix_space, self), split=False ) super().__init__( tokenizer_object=self._tokenizer, unk_token=unk_token, bos_token=bos_token, eos_token=eos_token, )

Once the tokenizer is defined as above, you can load it with the following: Llama5Tokenizer(). Doing this returns you an empty, trainable tokenizer that follows the definition of the authors of Llama5 (it does not exist yet :wink:).

The above is the main motivation towards refactoring tokenization: we want tokenizers to behave similarly to models: trained or empty, and with exactly what is defined in their class definition.

Backend Architecture Changes: moving away from the slow/fast tokenizer separation

Up to now, transformers maintained two parallel implementations for many tokenizers:

"Slow" tokenizers (tokenization_<model>.py) - Python-based implementations, often using SentencePiece as the backend.

"Fast" tokenizers (tokenization_<model>_fast.py) - Rust-based implementations using the 🤗 tokenizers library.

In v5, we consolidate to a single tokenizer file per model: tokenization_<model>.py. This file will use the most appropriate backend available:

TokenizersBackend (preferred): Rust-based tokenizers from the 🤗 tokenizers library. In general it provides optimal performance, but it also offers a lot more features that are commonly adopted across the ecosystem:

handling additional tokens

a full python API for setting and updating

automatic parallelization,

automatic offsets

customization

training

SentencePieceBackend: for tokenizers requiring the sentencepiece library. It inherits from PythonBackend.

PythonBackend: a Python implementations of the features provided by tokenizers. Basically allows adding tokens.

MistralCommonBackend: relies on MistralCommon's tokenization library. (Previously known as the MistralCommonTokenizer)

The AutoTokenizer automatically selects the appropriate backend based on available files and dependencies. This is transparent, you continue to use AutoTokenizer.from_pretrained() as before. This allows transformers to be future-proof and modular to easily support future backends.

Defining a tokenizers outside of the existing backends

We enable users and tokenizer builders to define their own tokenizers from top to bottom. Tokenizers are usually defined using a backend such as tokenizers, sentencepiece or mistral-common, but we offer the possibility to design the tokenizer at a higher-level, without relying on those backends.

To do so, you can import the PythonBackend (which was previously known as PreTrainedTokenizer). This class encapsulates all the logic related to added tokens, encoding, and decoding.

If you want something even higher up the stack, then PreTrainedTokenizerBase is what PythonBackend inherits from. It contains the very basic tokenizer API features:

encode

decode

vocab_size

get_vocab

convert_tokens_to_ids

convert_ids_to_tokens

from_pretrained

save_pretrained

among a few others

API Changes

1. Direct tokenizer initialization with vocab and merges

Starting with v5, we now enable initializing blank, untrained tokenizers-backed tokenizers:

from transformers import LlamaTokenizer tokenizer = LlamaTokenizer()

This tokenizer will therefore follow the definition of the LlamaTokenizer as defined in its class definition. It can then be trained on a corpus as can be seen in the tokenizers documentation.

These tokenizers can also be initialized from vocab and merges (if necessary), like the previous "slow" tokenizers:

from transformers import LlamaTokenizer vocab = {"<unk>": 0, "<s>": 1, "</s>": 2, "hello": 3, "world": 4} merges = [("h", "e"), ("l", "l"), ("o", " ")] tokenizer = LlamaTokenizer(vocab=vocab, merges=merges)

This tokenizer will behave as a Llama-like tokenizer, with an updated vocabulary. This allows comparing different tokenizer classes with the same vocab; therefore enabling the comparison of different pre-tokenizers, normalizers, etc.

⚠️ The vocab_file (as in, a path towards a file containing the vocabulary) cannot be used to initialize the LlamaTokenizer as loading from files is reserved to the from_pretrained method.

2. Simplified decoding API

The batch_decode and decode methods have been unified to reflect behavior of the encode method. Both single and batch decoding now use the same decode method. See an example of the new behavior below:

from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("t5-small") inputs = ["hey how are you?", "fine"] tokenizer.decode(tokenizer.encode(inputs))

Gives:

- 'hey how are you?</s> fine</s>' + ['hey how are you?</s>', 'fine</s>']

We expect encode and decode to behave, as two sides of the same coin: encode, process, decode, should work.

[!NOTE] A common use-case would be: encode, model.generate, decode. However, using generate would return list[list[int]], which would then be incompatible with decode.

3. Unified encoding API

The encode_plus method is deprecated in favor of the single __call__ method.

4. apply_chat_template returns BatchEncoding

Previously, apply_chat_template returned input_ids for backward compatibility. Starting with v5, it now consistently returns a BatchEncoding dict like other tokenizer methods.

# v5 messages = [ {"role": "user", "content": "Hello!"}, {"role": "assistant", "content": "Hi there!"} ] # Now returns BatchEncoding with input_ids, attention_mask, etc. outputs = tokenizer.apply_chat_template(messages, return_tensors="pt") print(outputs.keys()) # dict_keys(['input_ids', 'attention_mask'])

5. Removed legacy configuration file saving:

We simplify the serialization of tokenization attributes:

special_tokens_map.json - special tokens are now stored in tokenizer_config.json.

added_tokens.json - added tokens are now stored in tokenizer.json.

added_tokens_decoder is only stored when there is no tokenizer.json.

When loading older tokenizers, these files are still read for backward compatibility, but new saves use the consolidated format. We're gradually moving towards consolidating attributes to fewer files so that other libraries and implementations may depend on them more reliably.

6. Model-Specific Changes

Several models that had identical tokenizers now import from their base implementation:

LayoutLM → uses BertTokenizer

LED → uses BartTokenizer

Longformer → uses RobertaTokenizer

LXMert → uses BertTokenizer

MT5 → uses T5Tokenizer

MVP → uses BartTokenizer

These modules will eventually be removed altogether.

Removed T5-specific workarounds

The internal _eventually_correct_t5_max_length method has been removed. T5 tokenizers now handle max length consistently with other models.

Testing Changes

A few testing changes specific to tokenizers have been applied:

Model-specific tokenization test files now focus on integration tests.

Common tokenization API tests (e.g., add_tokens, encode, decode) are now centralized and automatically applied across all tokenizers. This reduces test duplication and ensures consistent behavior

For legacy implementations, the original BERT Python tokenizer code (including WhitespaceTokenizer, BasicTokenizer, etc.) is preserved in bert_legacy.py for reference purposes.

7. Deprecated / Modified Features

Special Tokens Structure:

SpecialTokensMixin: Merged into PreTrainedTokenizerBase to simplify the tokenizer architecture.

special_tokens_map: Now only stores named special token attributes (e.g., bos_token, eos_token). Use extra_special_tokens for additional special tokens (formerly additional_special_tokens). all_special_tokens includes both named and extra tokens.

# v4 tokenizer.special_tokens_map # Included 'additional_special_tokens' # v5 tokenizer.special_tokens_map # Only named tokens tokenizer.extra_special_tokens # Additional tokens

special_tokens_map_extended and all_special_tokens_extended: Removed. Access AddedToken objects directly from _special_tokens_map or _extra_special_tokens if needed.

additional_special_tokens: Still accepted for backward compatibility but is automatically converted to extra_special_tokens.

Deprecated Methods:

sanitize_special_tokens(): Already deprecated in v4, removed in v5.

prepare_seq2seq_batch(): Deprecated; use __call__() with text_target parameter instead.

# v4 model_inputs = tokenizer.prepare_seq2seq_batch(src_texts, tgt_texts, max_length=128) # v5 model_inputs = tokenizer(src_texts, text_target=tgt_texts, max_length=128, return_tensors="pt") model_inputs["labels"] = model_inputs.pop("input_ids_target")

BatchEncoding.words(): Deprecated; use word_ids() instead.

Removed Methods:

create_token_type_ids_from_sequences(): Removed from base class. Subclasses that need custom token type ID creation should implement this method directly.

prepare_for_model(), build_inputs_with_special_tokens(), truncate_sequences(): Moved from tokenization_utils_base.py to tokenization_python.py for PythonBackend tokenizers. TokenizersBackend provides model-ready input via tokenize() and encode(), so these methods are no longer needed in the base class.

_switch_to_input_mode(), _switch_to_target_mode(), as_target_tokenizer(): Removed from base class. Use __call__() with text_target parameter instead.

# v4 with tokenizer.as_target_tokenizer(): labels = tokenizer(tgt_texts, ...) # v5 labels = tokenizer(text_target=tgt_texts, ...)

parse_response(): Removed from base class.

Performance

MoE Performance

The v5 release significantly improves the performance of the MoE models, as can be seen in the graphs below. We improve and optimize MoE performance through batched and grouped experts implementations, and we optimize them for decoding using batched_mm.

<img width="2048" height="1451" alt="image" src="https://github.com/user-attachments/assets/c3f2e59f-3026-4f56-9a56-36e4eb0fcf73" />

Core performance

We focus on improving the performance of loading weights on device (which gives speedups up to 6x in tensor parallel situations); this is preliminary work that we'll continue to work on in the coming weeks. Some notable improvements:

[saving] Simplify general logic by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42766

Do not rely on config for inferring model dtype by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42838

Improve BatchFeature: stack list and lists of torch tensors by @yonigozlan in https://github.com/huggingface/transformers/pull/42750

Remove tied weights from internal attribute if they are not tied by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42871

Enforce call to post_init and fix all of them by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42873

Simplify tie weights logic by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42895

Add buffers to _init_weights for ALL models by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42309

[loading] Really initialize on meta device for huge perf gains by @Cyrilvallez in https://github.com/huggingface/transformers/pull/42941

Do not use accelerate hooks if the device_map has only 1 device by @Cyrilvallez in https://github.com/huggingface/transformers/pull/43019

Move missing weights and non-persistent buffers to correct device earlier by @Cyrilvallez in https://github.com/huggingface/transformers/pull/43021

Library-wide changes with lesser impact

Default dtype update

We have updated the default dtype for all models loaded with from_pretrained to be auto. This will lead to model instantiations respecting the dtype in which the model was saved, rather than forcing it to load in float 32.

You can, of course, still specify the dtype in which you want to load your model by specifying it as an argument to the from_pretrained method.

Shard size

The Hugging Face Hub infrastructure has gradually moved to a XET backend. This will significantly simplify uploads and downloads, with higher download and upload speeds, partial uploads, and, most notably, a higher threshold for accepted file sizes on the Hugging Face Hub.

To reflect this, we're increasing the default shard size of models serialized on the Hub to 50GB (up from 5GB).

use_auth_token

The use_auth_token argument/parameter is deprecated in favor of token everywhere. You should be able to search and replace use_auth_token with token and get the same logic.

Linked PR: https://github.com/huggingface/transformers/pull/41666

Attention-related features

We decided to remove some features for the upcoming v5 as they are currently only supported in a few old models and no longer integrated in current model additions. It's recommended to stick to v4.x in case you need them. Following features are affected:

No more head masking, see #41076. This feature allowed to turn off certain heads during the attention calculation and only worked for eager.

No more relative positional biases in Bert-like models, see #41170. This feature was introduced to allow relative position scores within attention calculations (similar to T5). However, this feature is barely used in official models and a lot of complexity instead. It also only worked with eager.

No more head pruning, see #41417 by @gante. As the name suggests, it allowed to prune heads within your attention layers.

Updates to supported torch APIs

We dropped support for two torch APIs:

torchscript in https://github.com/huggingface/transformers/pull/41688

torch.fx in https://github.com/huggingface/transformers/pull/41683

Those APIs were deprecated by the PyTorch team, and we're instead focusing on the supported APIs dynamo and export.

Quantization changes

We clean up the quantization API in transformers, and significantly refactor the weight loading as highlighted above.

We drop support for two quantization arguments that have been deprecated for some time:

load_in_4bit

load_in_8bit

We remove them in favor of the quantization_config argument which is much more complete. As an example, here is how you would load a 4-bit bitsandbytes model using this argument:

from transformers import AutoModelForCausalLM, BitsAndBytesConfig quantization_config = BitsAndBytesConfig(load_in_4bit=True) model_4bit = AutoModelForCausalLM.from_pretrained( "meta-llama/Llama-3.2-3B", device_map="auto", quantization_config=quantization_config )

Configuration

Methods to init a nested config such as from_xxx_config are deleted. Configs can be init from the __init__ method in the same way. See #41314.

It is no longer possible to load a config class from a URL file. Configs must be loaded from either a local path or a repo on the Hub. See #42383.

All parameters for configuring model's rotary embedding are now stored under mode.rope_parameters, including the rope_theta and rope_type. Model's config.rope_parameters is a simple dictionaty in most cases, and can also be a nested dict in special cases (i.e. Gemma3 and ModernBert) with different rope parameterization for each layer type. Trying to get config.rope_theta will throw an attribute error from now on. See #39847 and #42255

Qwen-VL family configuration is in a nested format and trying to access keys directly will throw an error (e.g. config.vocab_size). Users are expected to access keys from their respective sub-configs (config.text_config.vocab_size).

Configurations of non-generative models (any model that doesn't call model.generate()) will no longer have a generation_config and model.config.generation_config will throw an attribute error.

Processing

Tokenization

Slow tokenizer files (aka: tokenization_<model>.py ) will be removed in favor of using fast tokenizer files tokenization_<model>_fast.py --> will be renamed to tokenization_<model>.py. As fast tokenizers are :hugs:tokenizers - backend, they include a wider range of features that are maintainable and reliable.

Other backends (sentence piece, tokenizers, etc.) will be supported with a light layer if loading a fast tokenizer fails

Remove legacy files like special_tokens_map.json and added_tokens.json

Remove _eventually_correct_t5_max_length

encode_plus --> __call__

batch_decode --> decode

apply_chat_template by default returns naked input_ids rather than a BatchEncoding dict. This was inconvenient - it should return a BatchEncoding dict like tokenizer.__call__(), but we were stuck with it for backward compatibility. The method now returns a BatchEncoding.

Linked PRs:

https://github.com/huggingface/transformers/issues/40938

https://github.com/huggingface/transformers/pull/40936

https://github.com/huggingface/transformers/pull/41626

Processing classes

In processing classes each attribute will be serialized under processor_config.json as a nested dict, instead of serializing attributes in their own config files. Loading will be supported for all old format processors (https://github.com/huggingface/transformers/pull/41474)

XXXFeatureExtractors classes are completely removed in favor of XXXImageProcessor class for all vision models (https://github.com/huggingface/transformers/pull/41174)

Minor change: XXXFastImageProcessorKwargs is removed in favor of XXXImageProcessorKwargs which will be shared between fast and slow processors (https://github.com/huggingface/transformers/pull/40931)

Modeling

Some RotaryEmbeddings layers will start returning a dict of tuples, in case the model uses several RoPE configurations (Gemma2, ModernBert). Each value will be a tuple of "cos, sin" per RoPE type.

Config attribute for RotaryEmbeddings layer will be unified and accessed via config.rope_parameters. Config attr for rope_theta might not be accessible anymore for some models, and instead will be in config.rope_parameters['rope_theta']. BC will be supported for a while as much as possible, and in the near future we'll gradually move to the new RoPE format (https://github.com/huggingface/transformers/pull/39847)

Vision Language models will not have a shortcut access to its language and vision component from the generative model via model.language_model. It is recommended to either access the module with model.model.language_model or model.get_decoder(). See #42156

All models now accept kwargs in their forward methods

Generate

Old, deprecated output type aliases were removed (e.g. GreedySearchEncoderDecoderOutput). We now only have 4 output classes built from the following matrix: decoder-only vs encoder-decoder, uses beams vs doesn't use beams (https://github.com/huggingface/transformers/pull/40998)

Removed deprecated classes regarding decoding methods that were moved to the Hub due to low usage (constraints and beam scores) (https://github.com/huggingface/transformers/pull/41223)

If generate doesn't receive any KV Cache argument, the default cache class used is now defined by the model (as opposed to always being DynamicCache) (https://github.com/huggingface/transformers/pull/41505)

Generation parameters are no longer accessible via model's config. If generation paramaters are serialized in config.json for any old model, it will be loaded back into model's generation config. Users are expected to access or modify generation parameters only with model.generation_config.do_sample = True.

Trainer

New Features

ALST/Ulysses Sequence Parallelism Integration

Added sequence parallelism support via HF Accelerate for training with longer sequences. Enables splitting sequences across devices using ALST (All-to-All Long Sequence Training) and Ulysses algorithms with DeepSpeed.

Improved compute_loss_func Handling

compute_loss_func now always takes priority over the model's built-in loss computation, giving users consistent control over custom loss functions.

num_items_in_batch in Prediction Step

The num_items_in_batch argument is now passed to compute_loss during prediction_step, enabling proper loss scaling during evaluation.

Breaking Changes

report_to now defaults to "none"

Logging integrations are no longer auto-detected by default; users must explicitly specify which reporting backends to use.

Removing arguments without deprecation cycle in TrainingArguments due to low usage

mp_parameters -> legacy param that was later on added to the Sagemaker trainer

_n_gpu -> not intended for users to set, we will initialize it correctly instead of putting it in the TrainingArguments

overwrite_output_dir - > replaced by resume_from_checkpoint, and it was only used in the examples script, no impact on Trainer.

logging_dir -> only used for tensorboard, set TENSORBOARD_LOGGING_DIR env var instead

jit_mode_eval -> use use_torch_compile instead, as torchscript is not recommended anymore

tpu_num_cores-> It is actually better to remove it, as it is not recommended to set the number of cores. By default, all TPU cores are used . Set TPU_NUM_CORES env var instead

past_index -> it was only used for a very small number of models that have special architecture like transformersxl + it was not documented at all how to train those models

ray_scope -> only for a minor arg for ray integration. Set RAY_SCOPE var env instead

warmup_ratio -> use warmup_step instead. We combined both args together by allowing passing float values in warmup_step.

Removing deprecated arguments in TrainingArguments

fsdp_min_num_params and fsdp_transformer_layer_cls_to_wrap -> use fsdp_config

tpu_metrics_debug -> debug

push_to_hub_token -> hub_token

push_to_hub_model_id and push_to_hub_organization -> hub_model_id

include_inputs_for_metrics -> include_for_metrics

per_gpu_train_batch_size -> per_device_train_batch_size

per_gpu_eval_batch_size -> per_device_eval_batch_size

use_mps_device -> mps will be used by default if detected

fp16_backend and half_precision_backend -> we will only rely on torch.amp as everything has been upstreamed to torch

no_cuda -> use_cpu

include_tokens_per_second -> include_num_input_tokens_seen

use_legacy_prediction_loop -> we only use evaluation_loop function from now on

Removing deprecated arguments in Trainer

tokenizer in initialization -> processing_class

model_path in train() -> resume_from_checkpoint

Removed features for Trainer

sigpot integration for hp search was removed as the library was archived + the api stopped working

drop support for sagemaker API <1.10

bump accelerate minimum version to 1.1.0

bump peft minimum version to 0.18.0

bump bitsandbytes minimum version to 0.46.1

New defaults for Trainer

use_cache in the model config will be set to False. You can still change the cache value through TrainingArguments usel_cache argument if needed.

Pipeline

Image text to text pipelines will no longer accept images as a separate argument along with conversation chats. Image data has to be embedded in the chat's "content" field. See #42359

PushToHubMixin

removed deprecated organization and repo_url from PushToHubMixin. You must pass a repo_id instead.

removed ignore_metadata_errors from PushToMixin. In practice if we ignore errors while loading the model card, we won't be able to push the card back to the Hub so it's better to fail early and not provide the option to fail later.

push_to_hub do not accept **kwargs anymore. All accepted parameters are explicitly documented.

arguments of push_to_hub are now keyword-only to avoid confusion. Only repo_id can be positional since it's the main arg.

removed use_temp_dir argument from push_to_hub. We now use a tmp dir in all cases.

Linked PR: https://github.com/huggingface/transformers/pull/42391.

CLI

The deprecated transformers-cli ... command was deprecated, transformers ... is now the only CLI entry point.

transformers CLI has been migrated to Typer, making it easier to maintain + adding some nice features out of the box (improved --help section, autocompletion).

Biggest breaking change is in transformers chat. This command starts a terminal UI to interact with a chat model. It used to also be able to start a Chat Completion server powered by transformers and chat with it. In this revamped version, this feature has been removed in favor of transformers serve. The goal of splitting transformers chat and transformers serve is to define clear boundaries between client and server code. It helps with maintenance but also makes the commands less bloated. The new signature of transformers chat is:

Usage: transformers chat [OPTIONS] BASE_URL MODEL_ID [GENERATE_FLAGS]... Chat with a model from the command line.

It works hand in hand with transformers serve, which means that if transformers serve is running on its default endpoint, transformers chat can be launched as follows:

transformers chat HuggingFaceTB/SmolLM3-3B

It can however use any OpenAI API compatible HTTP endpoint:

transformers chat HuggingFaceTB/SmolLM3-3B https://router.huggingface.co/v1

Linked PRs:

https://github.com/huggingface/transformers/pull/40997

https://github.com/huggingface/transformers/pull/41487

Removal of the run method

The transformers run (previously transformers-cli run) is an artefact of the past, was not documented nor tested, and isn't part of any public documentation. We're removing it for now and ask you to please let us know in case this is a method you are using; in which case we should bring it back with better support.

Linked PR: https://github.com/huggingface/transformers/pull/42447

Environment variables

Legacy environment variables like TRANSFORMERS_CACHE, PYTORCH_TRANSFORMERS_CACHE, and PYTORCH_PRETRAINED_BERT_CACHE have been removed. Please use HF_HOME instead.

Constants HUGGINGFACE_CO_EXAMPLES_TELEMETRY, HUGGINGFACE_CO_EXAMPLES_TELEMETRY, HUGGINGFACE_CO_PREFIX, and HUGGINGFACE_CO_RESOLVE_ENDPOINT have been removed. Please use huggingface_hub.constants.ENDPOINT instead.

Linked PR: https://github.com/huggingface/transformers/pull/42391.

Requirements update

transformers v5 pins the huggingface_hub version to >=1.0.0. See this migration guide to learn more about this major release. Here are to main aspects to know about:

switched the HTTP backend from requests to httpx. This change was made to improve performance and to support both synchronous and asynchronous requests the same way. If you are currently catching requests.HTTPError errors in your codebase, you'll need to switch to httpx.HTTPError.

related to 1., it is not possible to set proxies from your script. To handle proxies, you must set the HTTP_PROXY / HTTPS_PROXY environment variables

hf_transfer and therefore HF_HUB_ENABLE_HF_TRANSFER have been completed dropped in favor of hf_xet. This should be transparent for most users. Please let us know if you notice any downside!

typer-slim has been added as required dependency, used to implement both hf and transformers CLIs.

New model additions in v5

CWM

<img width="809" height="471" alt="image" src="https://github.com/user-attachments/assets/58bb9c70-d481-48ed-ab8f-6553be7c240f" />

The Code World Model (CWM) model was proposed in CWM: An Open-Weights LLM for Research on Code Generation with World Models by Meta FAIR CodeGen Team. CWM is an LLM for code generation and reasoning about code that has, in particular, been trained to better represent and reason about how code and commands affect the state of a program or system. Specifically, we mid-trained CWM on a large number of observation-action trajectories from Python execution traces and agentic interactions in containerized environments. We post-trained with extensive multi-task RL in verifiable coding, math, and multi-turn software engineering environments.

Add Code World Model (CWM) by @jacobkahn in #41199

SAM3

<img width="1505" height="915" alt="image" src="https://github.com/user-attachments/assets/eec48633-f02b-464a-ae5c-c65473387e53" />

SAM3 (Segment Anything Model 3) was introduced in SAM 3: Segment Anything with Concepts.

The SAM3 addition adds four new architectures:

Sam3

Sam3Tracker

Sam3TrackerVideo

Sam3Video

SAM3 performs Promptable Concept Segmentation (PCS) on images. PCS takes text and/or image exemplars as input (e.g., "yellow school bus"), and predicts instance and semantic masks for every single object matching the concept.

Sam3Tracker and Sam3TrackerVideo perform Promptable Visual Segmentation (PVS) on images. PVS takes interactive visual prompts (points, boxes, masks) or text inputs to segment a specific object instance per prompt. This is the task that SAM 1 and SAM 2 focused on, and SAM 3 improves upon it. Sam3Tracker and Sam3TrackerVideo are updated versions of SAM2 Video that maintain the same API while providing improved performance and capabilities.

SAM3 Video performs Promptable Concept Segmentation (PCS) on videos. PCS takes text as input (e.g., "yellow school bus"), and predicts instance and semantic masks for every single object matching the concept, while preserving object identities across video frames. The model combines a detection module (SAM3) with a tracking module (SAM2-style tracker) to enable robust object tracking across video frames using text prompts.

Add SAM3 to 🤗 Transformers by @yonigozlan in #42285

LFM2 MoE

<img width="1080" height="849" alt="image" src="https://github.com/user-attachments/assets/a9fa1b81-114d-4054-9699-5083ac69d830" />

LFM2-MoE is a Mixture-of-Experts (MoE) variant of LFM2. The LFM2 family is optimized for on-device inference by combining short‑range, input‑aware gated convolutions with grouped‑query attention (GQA) in a layout tuned to maximize quality under strict speed and memory constraints.

LFM2‑MoE keeps this fast backbone and introduces sparse MoE feed‑forward networks to add representational capacity without significantly increasing the active compute path. The first LFM2-MoE release is LFM2-8B-A1B, with 8.3B total parameters and 1.5B active parameters. The model excels in quality (comparable to 3-4B dense models) and speed (faster than other 1.5B class models).

[Model] Lfm2Moe by @paulpak58 in #41401

VideoLlama 3

<img width="812" height="366" alt="image" src="https://github.com/user-attachments/assets/21c82c6e-cf0a-4d6c-a707-b9e57663ca85" />

The VideoLLaMA3 model is a major update to VideoLLaMA2 from Alibaba DAMO Academy.

[model] Add VideoLLaMA3 implementation by @lkhl in #40499

AudioFlamingo 3

<img width="621" height="475" alt="image" src="https://github.com/user-attachments/assets/c9616758-b3aa-41d0-bd58-695966ba146d" />

Audio Flamingo 3 (AF3) is a fully open large audio–language model designed for robust understanding and reasoning over speech, environmental sounds, and music. AF3 pairs a Whisper-style audio encoder with a causal language model and performs replace-in-place audio–text fusion: the processor aligns post-pool audio frames to a dedicated placeholder token and the model replaces those token slots with projected audio embeddings during the forward pass.

The model checkpoint is available at: nvidia/audio-flamingo-3-hf

Highlights:

Unified audio encoder across speech, sound, and music.

Long-audio support via windowing and post-pool alignment (up to 10 minutes maximum). The model processes audio in 30-second windows with a hard limit of 20 windows (10 minutes total). Audio longer than 10 minutes will be truncated.

Deterministic fusion that preserves sequence length by replacing audio placeholder tokens with audio embeddings.

[models] Add AudioFlamingo3 integration by @lashahub in #40290

Nanochat

NanoChat is a compact decoder-only transformer model designed for educational purposes and efficient training. The model features several fundamental architectural innovations which are common in modern transformer models. Therefore, it is a good model to use as a starting point to understand the principles of modern transformer models. NanoChat is a variant of the Llama architecture, with simplified attention mechanism and normalization layers.

[MODEL] Nanochat implementation by @burtenshaw in #41634

FastVLM

<img width="868" height="331" alt="image" src="https://github.com/user-attachments/assets/cd8b82cf-10de-49b0-af2a-28ffbeac6fa7" />

FastVLM is an open-source vision-language model featuring a novel hybrid vision encoder, FastViTHD. Leveraging reparameterizable convolutional layers, scaled input resolution, and a reduced number of visual tokens, FastVLM delivers high accuracy with exceptional efficiency. Its optimized architecture enables deployment even on edge devices, achieving ultra-low TTFT (time to first token) without sacrificing performance.

Add FastVLM by @camilla-deckard in https://github.com/huggingface/transformers/pull/41112

PaddleOCR-VL

<img width="3840" height="2160" alt="image" src="https://github.com/user-attachments/assets/712fbe57-a4b6-4bf1-acf6-3ef803f75f0e" />

PaddleOCR-VL is a SOTA and resource-efficient model tailored for document parsing. Its core component is PaddleOCR-VL-0.9B, a compact yet powerful vision-language model (VLM) that integrates a NaViT-style dynamic resolution visual encoder with the ERNIE-4.5-0.3B language model to enable accurate element recognition. This innovative model efficiently supports 109 languages and excels in recognizing complex elements (e.g., text, tables, formulas, and charts), while maintaining minimal resource consumption. Through comprehensive evaluations on widely used public benchmarks and in-house benchmarks, PaddleOCR-VL achieves SOTA performance in both page-level document parsing and element-level recognition. It significantly outperforms existing solutions, exhibits strong competitiveness against top-tier VLMs, and delivers fast inference speeds. These strengths make it highly suitable for practical deployment in real-world scenarios.

[Model] Add PaddleOCR-VL Model Support by @zhang-prog in https://github.com/huggingface/transformers/pull/42178

SAM: Perception Encoder Audiovisual

<img width="719" height="541" alt="image" src="https://github.com/user-attachments/assets/2fc5ee26-3c15-451f-bc7f-c779c2a78919" />

PE Audio (Perception Encoder Audio) is a state-of-the-art multimodal model that embeds audio and text into a shared (joint) embedding space. The model enables cross-modal retrieval and understanding between audio and text.

Text input

Produces a single embedding representing the full text.

Audio input

PeAudioFrameLevelModel

Produces a sequence of embeddings, one every 40 ms of audio.

Suitable for audio event localization and fine-grained temporal analysis.

PeAudioModel

Produces a single embedding for the entire audio clip.

Suitable for global audio-text retrieval tasks.

The resulting embeddings can be used for:

Audio event localization

Cross-modal (audio–text) retrieval and matching

Sam: Perception Encoder Audiovisual by @eustlb in https://github.com/huggingface/transformers/pull/42905

Jais2

<img width="2100" height="1154" alt="image" src="https://github.com/user-attachments/assets/a9343e81-903b-4445-ba0c-61c87830776a" />

Jais2 a next-generation Arabic open-weight LLM trained on the richest Arabic-first dataset to date. Built from the ground up with 8B and 70B parameters, Jais 2 understands Arabic the way it's truly spoken across dialects, cuulutre, and modern expression. It is developed by MBZUAI, Inception and Cerebras Systems and based on the transformer architecture with modifications including:

LayerNorm instead of RMSNorm

ReLU² activation function

Rotary Position Embeddings (RoPE)

adds jais2 model support by @sarathc-cerebras in https://github.com/huggingface/transformers/pull/42684

Pixio

<img width="5478" height="2102" alt="image" src="https://github.com/user-attachments/assets/abbd93d4-8c4c-4fd9-8fab-58d969d8b296" />

Pixio is a vision foundation model that uses ViT as a feature extractor for multiple downstream tasks like depth estimation, semantic segmentation, feed-forward 3D reconstruction, robotics, and image classification. It is built on the Masked Autoencoder (MAE) pre-training framework, with four minimal yet critical updates: 1) deeper decoder, 2) larger masking granularity, 3) more class tokens, and 4) web-scale curated training data.

Add Pixio pre-trained models by @LiheYoung in https://github.com/huggingface/transformers/pull/42795

Ernie 4.5 VL MoE

<img width="848" height="853" alt="image" src="https://github.com/user-attachments/assets/02817ead-7560-4eea-8f2b-0b959553a3cd" />

The Ernie 4.5 VL MoE model was released in the Ernie 4.5 Model Family release by baidu. This family of models contains multiple different architectures and model sizes. The Vision-Language series in specific is composed of a novel multimodal heterogeneous structure, sharing paremeters across modalities and dedicating parameters to specific modalities. This becomes especially apparent in the Mixture of Expert (MoE) which is composed of

Dedicated Text Experts

Dedicated Vision Experts

Shared Experts

This architecture has the advantage to enhance multimodal understanding without compromising, and even improving, performance on text-related tasks. An more detailed breakdown is given in the Technical Report.

[Ernie 4.5] Ernie VL models by @vasqu in https://github.com/huggingface/transformers/pull/39585

GLM-ASR

<img width="1600" height="1029" alt="image" src="https://github.com/user-attachments/assets/d630a900-9ef5-467c-93ab-34f064501b8d" />

GLM-ASR-Nano-2512 is a robust, open-source speech recognition model with 1.5B parameters. Designed for real-world complexity, it outperforms OpenAI Whisper V3 on multiple benchmarks while maintaining a compact size.

Key capabilities include:

Exceptional Dialect Support Beyond standard Mandarin and English, the model is highly optimized for Cantonese (粤语) and other dialects, effectively bridging the gap in dialectal speech recognition.

Low-Volume Speech Robustness Specifically trained for "Whisper/Quiet Speech" scenarios. It captures and accurately transcribes extremely low-volume audio that traditional models often miss.

SOTA Performance Achieves the lowest average error rate (4.10) among comparable open-source models, showing significant advantages in Chinese benchmarks (Wenet Meeting, Aishell-1, etc..).

This model was contributed by Eustache Le Bihan and Yuxuan Zhang. you can check the model card for more details and our github repo.

GLM-ASR Support by @zRzRzRzRzRzRzR in https://github.com/huggingface/transformers/pull/42875

GLM 4.7 Flash

<img width="5038" height="5860" alt="image" src="https://github.com/user-attachments/assets/af0f2821-3831-4afd-a7dc-26a91f17afd0" />

GLM-4.7-Flash offers a new option for lightweight deployment that balances performance and efficiency.

[GLM-4.7] GLM-Lite Support by @zRzRzRzRzRzRzR in https://github.com/huggingface/transformers/pull/43031

GLM Image

<img width="10101" height="7371" alt="image" src="https://github.com/user-attachments/assets/e2de76ab-35a9-4e42-9252-6b3dbf75e991" />

We present GLM-4.1V-Thinking, GLM-4.5V, and GLM-4.6V, a family of vision-language models (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the reasoning-centric training framework. We first develop a capable vision foundation model with significant potential through large-scale pre-training, which arguably sets the upper bound for the final performance. We then propose Reinforcement Learning with Curriculum Sampling (RLCS) to unlock the full potential of the model, leading to comprehensive capability enhancement across a diverse range of tasks, including STEM problem solving, video understanding, content recognition, coding, grounding, GUI-based agents, and long document interpretation. In a comprehensive evaluation across 42 public benchmarks, GLM-4.5V achieves state-of-the-art performance on nearly all tasks among open-source models of similar size, and demonstrates competitive or even superior results compared to closed-source models such as Gemini-2.5-Flash on challenging tasks including Coding and GUI Agents. Meanwhile, the smaller GLM-4.1V-9B-Thinking remains highly competitive-achieving superior results to the much larger Qwen2.5-VL-72B on 29 benchmarks. We open-source both GLM-4.1V-9B-Thinking and GLM-4.5V. We further introduce the GLM-4.6V series, open-source multimodal models with native tool use and a 128K context window. A brief overview is available at this https URL. Code, models and more information are released at https://github.com/zai-org/GLM-V

[GLM-Image] AR Model Support for GLM-Image by @zRzRzRzRzRzRzR in https://github.com/huggingface/transformers/pull/43100

LWDetr

<img width="706" height="242" alt="image" src="https://github.com/user-attachments/assets/2c4770f4-5d48-4d87-a454-b574b648e5ae" />

LW-DETR proposes a light-weight Detection Transformer (DETR) architecture designed to compete with and surpass the dominant YOLO series for real-time object detection. It achieves a new state-of-the-art balance between speed (latency) and accuracy (mAP) by combining recent transformer advances with efficient design choices.

The LW-DETR architecture is characterized by its simple and efficient structure: a plain ViT Encoder, a Projector, and a shallow DETR Decoder. It enhances the DETR architecture for efficiency and speed using the following core modifications:

Efficient ViT Encoder: Uses a plain ViT with interleaved window/global attention and a window-major organization to drastically reduce attention complexity and latency. Richer Input: Aggregates multi-level features from the encoder and uses a C2f Projector (YOLOv8) to pass two-scale features ( 1 / 8 and 1 / 32 ). Faster Decoder: Employs a shallow 3-layer DETR decoder with deformable cross-attention for lower latency and faster convergence. Optimized Queries: Uses a mixed-query scheme combining learnable content queries and generated spatial queries.

Add LWDetr model by @sbucaille in https://github.com/huggingface/transformers/pull/40991

LightOnOCR

<img width="1172" height="661" alt="image" src="https://github.com/user-attachments/assets/cee98e82-b3d0-42a6-b820-3061752ad4a8" />

LightOnOcr combines a Vision Transformer encoder (Pixtral-based) with a lightweight text decoder (Qwen3-based) distilled from high-quality open VLMs. It is optimized for document parsing tasks, producing accurate, layout-aware text extraction from high-resolution pages.

Add LightOnOCR model implementation by @baptiste-aubertin in https://github.com/huggingface/transformers/pull/41621

Bugfixes and improvements

JetMoe Fix jetmoe after #40132 by @ArthurZucker in #41324

Fixed tiny incorrect import in gemma3 by @Sai-Suraj-27 in #41354

Rope for Qwen2--5-vl by @zucchini-nlp in #41173

🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence by @Cyrilvallez in #41268

Standardize PretrainedConfig to PreTrainedConfig by @Cyrilvallez in #41300

Fix trainer for py3.9 by @SunMarc in #41359

Check model inputs - hidden states by @zucchini-nlp in #40994

[ModularChecker] QOL for the modular checker by @ArthurZucker in #41361

Fixing a typo for BLT model by @Narsil in #41325

:rotating_light: [v5] Remove relative position embeddings (for bert like models) by @vasqu in #41170

Fix typo in model proposal template by @Ombucha in #41352

Better typehints for apply_chat_template by @Samoed in #41355

🚨 Remove BetterTransformer by @Cyrilvallez in #41367

[testing] update test_longcat_generation_cpu by @ydshieh in #41368

Fix flash_attention.py: wrong argument passing for attn_implementation by @TKONIY in #41347

Use canonical get_size_with_aspect_ratio (with max_size) from transformers.image_transforms to fix #37939 by @sonianuj287 in #41284

Fixes in check_model_inputs, GPTBigCodeModel and ImageGPTModel by @IlyasMoutawwakil in #40811

Remove unnecessary list comprehension by @cyyever in #41305

make some ut cases pass on xpu w/ latest torch by @yao-matrix in #41337

Remove unused function patameters by @cyyever in #41358

[CB] Refactors the way we access paged by @ArthurZucker in #41370

serve: add non-streaming mode to /v1/responses; stream event parity; remove placeholder logprobs by @antznette1 in #41353

Update from pretrained error when loading by @ArthurZucker in #33380

[v5] Sync Bert and Bart eager attention by @vasqu in #41248

fix asr ut failures by @yao-matrix in #41332

fix resample in asr pipeline by @yhzx233 in #41298

Correct numerical regression in vision embeddings by @i3hz in #41374

[kernels] Kernel Config by @MekkCyber in #41232

[Cache] lfm2 cache: allocate empty kv layers during init by @paulpak58 in #41396

Fix test for model with dotted name and relative imports by @st81 in #41343

Prefer raising TypeError exception for invalid type by @Sai-Suraj-27 in #41346

[v5] Bump accelerate to 1.1.0 by @SunMarc in #41234

Fix incorrect assignment in update_device_map for GPTQ quantizer by @Sai-Suraj-27 in #41328

[v5] Delete left traces of feature extractor by @zucchini-nlp in #41321

Remove deprecation warning by @Cyrilvallez in #41425

Fix overriding common_kwargs defaults in processor calls by @yonigozlan in #41381

v5 dev version by @LysandreJik in #41436

Tiny Cleanup - Removed duplicate class field definition's by @Sai-Suraj-27 in #41293

🚨🚨 Remove all traces of legacy cache format by @Cyrilvallez in #41378

🚨 [v5] Prune prune_heads by @gante in #41417

[v5] Bump min version of bitsandbytes to 0.46.1 by @SunMarc in #41283

Fixing comments in init file by @MekkCyber in #41414

Use accelerator API to free device memory by @cyyever in #41195

enable new model uts to xpu and fix some failures on xpu by @yao-matrix in #41386

[torchao] Add regex support for ModuleFqnToConfig by @jerryzh168 in #41242

:facepalm: CB nit! by @ArthurZucker in #41413

Remove Python 3.9 classifier by @cyyever in #41410

[JetMoe] Fix KV head repetition and padding free by @vasqu in #41423

[testing] Fix JetMoeIntegrationTest by @ydshieh in #41377

Add Top-H decoding (entropy-bounded truncation) as a LogitsWarper for text generation by @ErfanBaghaei in #40837

Validate processing kwargs with @strict from huggingface_hub by @zucchini-nlp in #40793

Update hqq.md by @prathamesh-chavan-22 in #41452

enable some falcon-mamba uts on xpu by @yao-matrix in #41428

Fix generate outputs and simplify cache tests by @Cyrilvallez in #41440

Fix doc by @Cyrilvallez in #41457

🚨 [v5] Rename left traces of past_key_value in BERT-like models by @zucchini-nlp in #41448

Subconfig is a class attribute by @zucchini-nlp in #41308

[v5] rm utils/tf_ops/ by @gante in #41402

Update GLM-4.1V MMRope implementation by @zRzRzRzRzRzRzR in #41182

[kernels] Cleanup deta kernel by @MekkCyber in #41470

🚨 [v5] Rendundant code in nested configs by @zucchini-nlp in #41314

Remove KERAS_NLP_IMPORT_ERROR by @cyyever in #41468

Fix auto model configuration for encoder of perceptionlm by @fschlatt in #41464

Fix tests fsdp by @SunMarc in #41422

Import Callable from collections.abc by @cyyever in #41130

Pickle - part 2 by @ydshieh in #41476

Remove infer_device by @cyyever in #41088

Change RT-Detr docs to reflect fixed 640x640 input size by @konstantinos-p in #41364

Cleaning hub kernels by @MekkCyber in #41477

[v5] remove load_in_4bit and load_in_8bit by @SunMarc in #41287

:rotating_light: [Attention Masks] Bidirectional masks for encoder and encoder-decoder models by @vasqu in #41265

[Fix] Fix test file error by @YangKai0616 in #40973

enhance patched_tearDown to support python 3.11+ by @yao-matrix in #41429

RT-Detr correct 2d positional embeddings for non-square images by @konstantinos-p in #41380

Fix bnb fsdp loading for pre-quantized checkpoint by @SunMarc in #41415

Remove SigOpt by @SunMarc in #41479

Remove past_index by @SunMarc in #41384

Remove deprecated args in Trainer for v5 by @SunMarc in #41404

Update GLM-4.6 doc by @zRzRzRzRzRzRzR in #41471

report_to default changed to "none" + cleaning deprecated env var by @SunMarc in #41375

deprecate overwrite_output_dir by @SunMarc in #41323

[CI] Fix copies on main by @vasqu in #41486

[Trainer] deprecate ray scope by @SunMarc in #41403

deprecate jit_mode_eval by @SunMarc in #41376

Remove local_rank arg from TrainingArguments by @SunMarc in #41382

Update philosophy by @molbap in #41438

Remove DISABLE_KERNEL_MAPPING flag by @MekkCyber in #41475

Streaming should be handled at the request-level rather than at the istance level by @LysandreJik in #41444

fix bnb model loading by @jiqing-feng in #41499

[kernels] Remove RWKV kernel finally ! by @MekkCyber in #41493

[kernels] rm yoso kernel by @MekkCyber in #41495

Try to remove pickle - BloomTokenizerFast by @ydshieh in #41466

Fixed tiny incorrect imports in glm4v by @Sai-Suraj-27 in #41483

[Parakeet] unnecessary warning & auto mapping by @eustlb in #41412

[causallm tester] automate pipeline mappings + bloom tests by @gante in #41318

Fix some tests by @Cyrilvallez in #41503

fix gemma3n case failure by @yao-matrix in #41426

[voxtral] language detection + skipping lang:xx by @eustlb in #41225

Set truncation to False in Qwen3Omni to avoid default truncation by @BakerBunker in #41473

[QoL] modular conversion shows LoC saved by @molbap in #41500

More trainer cleaning by @SunMarc in #41489

Bump to hfh 1.0.0.rc5 to fix test by @Wauplin in #41508

Revert local_rank deletion and some cleaning by @SunMarc in #41504

Fix detectron2 import by @Cyrilvallez in #41510

add Trainer import to .md in appropriate cell block for training.ipynb transformers_doc by @benkeene in #41484

Remove outdated flags by @Cyrilvallez in #41512

remove tpu_num_cores by @SunMarc in #41383

Allow optuna's catch kwargs passthrough by @nicha-api in #41496

Fix Latex typesetting in documentation by @cyyever in #41177

[testing] reduce runtime of HunYuanMoEV1IntegrationTest:test_model_generation by @ydshieh in #41373

[Qwen3VL] fix: hidden_states in place modification error by @HollowMan6 in #41535

Add MLlama fast image processor by @yonigozlan in #41391

Fixed Type-hints in function defintions by @Sai-Suraj-27 in #41525

[SAM] Fix typing hints by @zucchini-nlp in #41506

Restore cuda graphs to continuous batching by @remi-or in #41421

Add AMD developer cloud support by @fan-amd in #41126

Enable modular files from other libraries by @regisss in #41372

🚨 [v5] generate delegates default cache initialization to the model by @gante in #41505

Fixed typos and formatting by @julian-st in #34215

Add VideoMAE video processor by @Aki-07 in #41534

[from_pretrained] Small refactor from_pretrained: move around unrelated stuff by @ArthurZucker in #41445

Remove references to AutoModelForVision2Seq by @Rocketknight1 in #41513

[Qwen3VL] fix device mismatch error for FSDP2 training by @HollowMan6 in #41536

Patch MistralCommonTokenizer by @juliendenize in #41439

Fix an import error with PreTrainModel by @remi-or in #41571

[Qwen3VLMoe] Fixed: Expected self.dtype to be equal to src.dtype - routing_weights casting by @danielquintas8 in #41420

[kernels] rm mra kernels by @MekkCyber in #41507

delete some tokenizer tests using pickle by @ydshieh in #41514

Add DINOv3Backbone for ConvNext variant by @merveenoyan in #40651

Add conditional checks to _check_and_adjust_attn_implementation() by @zheliuyu in #41542

add rmsnorm kernels support for Intel XPU by @kaixuanliu in #41563

Revert "add rmsnorm kernels support for Intel XPU" by @MekkCyber in #41579

[VisionEncoderDecoderModel] Update loss function by @NielsRogge in #40863

Add iter to DynamicCache by @remi-or in #41569

Revert some breaking changes bnb by @SunMarc in #41581

Fix typsetting and content of llm_tutorial_optimization.md by @cyyever in #41172

Gemma3 fixes by @remi-or in #41572

Benchmark overhaul by @remi-or in #41408

Enable non-streaming mode in transformers serve by @LysandreJik in #41446

[device_map] Accelerate loading by computing device_map much faster by @Cyrilvallez in #41548

Add logits_to_keep to many older CausalLM models by @philiproeleveld in #41335

fix some case failures lead by "torch.compile recompiled part of th… by @sywangyi in #41558

remove ray_scope and check_quantized_param by @SunMarc in #41587

Update issue template by @SunMarc in #41573

[Docs] Fix changed references by @vasqu in #41614

Import expand_device_map instead of redefining it by @Cyrilvallez in #41608

Fix trainer simple tests by @SunMarc in #41449

More markdown file fixes by @cyyever in #41599

torch 2.9 don't ❤️ torchcodec 💔 by @ydshieh in #41610

Update a dataset reop link by @ydshieh in #41618

Add fast path for bidirectional mask creation to fix regression by @i3hz in #41586

enable sdpa enable gqa logic for Ascend NPU by @FightingZhen in #41601

Fix video processing channel format by @zucchini-nlp in #41603

[chat template] update when "push_to_hub" by @zucchini-nlp in #39815

Remove the head masking block in some vision models by @ydshieh in #41620

Remove deprecated code by @SunMarc in #41616

Fix quantization base class by @SunMarc in #41613

[docs] Duplicate entry by @stevhliu in #41591

Update executorch.md by @jackzhxng in #41582

Add Backbone API fine-tuning tutorial by @merveenoyan in #41590

🚨 [v5] Toggle the serialization format in processors by @zucchini-nlp in #41474

Add aux loss for GLM-4.5V by @zRzRzRzRzRzRzR in #41564

Allow passing tp_plan in from_pretrained directly by @Cyrilvallez in #41435

Fix tokenization test by @Cyrilvallez in #41649

Remove randomly added script by @Cyrilvallez in #41650

Add missing dates to docs by @yonigozlan in #41576

Migrate transformers cli to Typer by @Wauplin in #41487

Fix FP-Quant quantization fallback CPU dispatch. by @BlackSamorez in #41619

fix check inputs for text2text pipeline by @jiqing-feng in #41556

[Executorch] Simplify for encoder models by @vasqu in #41627

[Ernie 4.5 Moe] Fix Moe and offloading by @vasqu in #41385

[CI] Build translated docs by @stevhliu in #41632

Fix fp32_ln for various models by @remi-or in #41605

Adjust device logging level and add minor fixes by @mario-koddenbrock in #41636

Fix EncoderDecoder cache by @remi-or in #41612

Format MarkDown documentation and tiny fixes by @cyyever in #41638

Fix typos in documentation by @cyyever in #41641

Fix confusing cls assignment by @cyyever in #41642

Double router compute? by @molbap in #41653

[kernels] refactor function kernel calling by @MekkCyber in #41577

[Fix] Deepseek V3 expert bias routing by @fjosw in #41647

purge HF_HUB_ENABLE_HF_TRANSFER; promote Xet by @Vaibhavs10 in #41656

[Masks] Fix mask handling in eager for vision models by @vasqu in #41625

Use | for Optional and Union typing by @cyyever in #41646

Switch to CB if cache_implementation == paged by @remi-or in #41655

Add in-out modalities as class attribute per model by @zucchini-nlp in #41366

Fix dtype casting with quantization by @Cyrilvallez in #41665

Fix serving continuous batching by @SunMarc in #41624

Small changes to benchmarking script by @remi-or in #41662

Improve package version check by @Cyrilvallez in #41661

improve utils/check_bad_commit.py by @ydshieh in #41658

Erroring when KernelConfig is passed without use_kernels = True by @MekkCyber in #41657

[Trainer] [Breaking change] use_cache default to False by @SunMarc in #41585

🌐 [i18n-KO] Translated chat_extras.md to Korean by @Judy-Choi in #39863

🌐 [i18n-KO] Translated sam_hq.md to Korean by @HyunZ118 in #41340

[i18n-KO] Translated big_bird.md to Korean by @ssum21 in #40445

🌐 [i18n-KO] Translated code_llama.md to Korean by @Judy-Choi in #40558

🌐 [i18n-KO] Translated llama4.md to Korean by @TaskerJang in #40396

:globe_with_meridians: [i18n-KO] Translated ko-LFM2.md to Korean by @ssum21 in #41502

Adding superglue fast image processing by @AlphaOrOmega in #41394

Fix ckpt in docs by @zucchini-nlp in #41659

torch 2.9 still don't ❤️ torchcodec 0.8 💔 by @ydshieh in #41686

Remove deprecated use_auth_token parameter by @Wauplin in #41666

Remove require_torch_bf16_gpu by @cyyever in #40979

path validation for security reason by @ydshieh in #41256

🚨 Remove torchscript support by @Cyrilvallez in #41688

Fix MarkDown syntax by @cyyever in #41676

Use | for Optional and Union typing by @cyyever in #41675

🚨 [v5] Refactor RoPE for layer types by @zucchini-nlp in #39847

Enable faiss-cpu on Windows by @cyyever in #41678

Fix Pylint warnings by @cyyever in #41644

🚨 Remove torch.fx support by @Cyrilvallez in #41683

Remove skipped tests without parents by @Cyrilvallez in #41691

Enable FURB rules in ruff by @cyyever in #41395

Remove upper version bound of pandas by @cyyever in #41677

[Attn] Allow dynamic causality in SDPA via Kwargs by @vasqu in #41692

Simplify GQA conditions in sdpa_attention.py by @justinchuby in #41699

[docs] Manual tp-plan by @stevhliu in #41674

🌐 [i18n-KO] Translated gemma3n.md to Korean by @HyunZ118 in #40873

pin torchcodec on CI docker image by @ydshieh in #41703

Update run_name docs in TrainingArguments by @tobiasofsn in #41705

further improve utils/check_bad_commit.py by @ydshieh in #41658)

feat: add benchmark v2 ci with results pushed to dataset by @McPatate in #41672

Gemma3 conversion script maintenance by @RyanMullins in #41704

Fix Qwen3-Omni inference when mixing video and image inputs in one batch by @BakerBunker in #41741

Fix typo in LFM-VL by @zucchini-nlp in #41742

Revert "Remove upper version bound of pandas" by @ydshieh in #41744

[doc] remove broken notebooks on AMD Dev Cloud by @pagezyhf in #41743

Update type hints in tokenization_utils.py to use | syntax by @faizan842 in #41713

Fix documentation issues by @cyyever in #41726

Apply RUFF PIE rules by @cyyever in #41727

Small Fix for imports by @MekkCyber in #41411

Docs(zh-hans): Refine wording for professionalism in README by @Ri-Nai in #40943

Add vision contribution guide by @molbap in #41456

upgrade xpu docker file to torch 2.8 by @yao-matrix in #41551

[v5] Delete videos from image processing classes by @zucchini-nlp in #41607

Fixed incorrect model_type for qwen2vl and qwen2.5vl when config is saved and loaded again by @i3hz in #41758

[kernels] Add version to function mapping by @MekkCyber in #41685

Reduce warning noise caused by Tensor.new_tensor by @st81 in #41748

Fix graphormer model compilation with Cython 3.1.4 by @alexmalyshev in #41671

Update type hints in modeling_rope_utils.py to use | syntax by @faizan842 in #41714

[v5] Remove deprecated tranformers.onnx by @echarlaix in #41700

Modernize CLIP modeling code by @molbap in #41546

Simplify pipeline padding logic by @Rocketknight1 in #41667

Chat response parsing by @Rocketknight1 in #40894

Add LightGlue fast image processor by @yonigozlan in #41670

Fix bark after #41445 by @ydshieh in #41645

Remove invalid @staticmethod from module-level get_device_and_memory_breakdown by @albertvillanova in #41747

Fix CUDA index out of bounds for q_idx in VLM token type masking for Gemma3, PaliGemma, and example modular by @albertvillanova in #41757

fix: Gemma 3 weights conversion vision and multimodal projector paths by @RyanMullins in #41767

[v5] Delete legacy chat template saving by @zucchini-nlp in #41648

[quantization] fix compressed_tensors tests by @MekkCyber in #41780

[quantization] Skip Fp8 tests when hardware capability < 8.9 by @MekkCyber in #41785

Swap columns and rows of the grid layout in LFM2-VL by @ankke in #41755

fix type annotation typo in docstring by @johntheprime in #41788

Fix chat schema tests by @Rocketknight1 in #41793

Fix attention mask in mamba layers by @zucchini-nlp in #41790

[quantization] fix torchao tests after 0.14.0 release by @MekkCyber in #41777

[Onnx docs] Remove some traces by @vasqu in #41791

flash attn pytest marker by @ydshieh in #41781

Bump AMD docker by @remi-or in #41792

make apollo test case pass by @yao-matrix in #41805

Add a safeguard around a flaky test in gemma2 by @remi-or in #41811

Fix Qwen3Next dtype API usage by @SrijanUpadhyay in #41735

[Trainer] remove env vars by @SunMarc in #41697

Fixed grammar mistakes by @FrogWarlord in #41799

Fixed some grammar mistakes by @FrogWarlord in #41802

transformers cli default flag fix by @ArjunPimpale in #41761

Deprecate warmup_ratio by @SunMarc in #41326

transformers serve quantization docs + some api fixes for bitsandbytes by @SunMarc in #41253

[Parakeet] add output_attention_mask by @eustlb in #41694

unpin torch/torchcodec for CircleCI by @ydshieh in #41839

extend bitnet cases to xpu, all 8 cases pass by @yao-matrix in #41831

extend 2 trainer test cases to xpu by @yao-matrix in #41829

extend 2 blip2 and falcon_h1 test cases to xpu by @yao-matrix in #41825

further reducing flakiness in utils/check_bad_commit.py by @ydshieh in #41658)

Remove redundant code from Qwen3VLProcessor by @Xqle in #41836

Fix MXFP4 quantizer to support variable num_local_experts and hidden_size by @marksverdhei in #41795

Fix Qwen2Audio flash attention mask format for generation by @Abdennacer-Badaoui in #41843

Fix const parsing for dict inputs in chat schemas by @Rocketknight1 in #41824

Share embedding modules in BART, not only weights by @githubnemo in #41821

Fix TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path' by @albertvillanova in #41604

:rotating_light: [Clip] Fix masking and enable flash attention on all model types by @vasqu in #41750

CI workflow for Flash Attn by @ydshieh in #41857

Fix torch.no_grad decorator in VLMS by @yaswanth19 in #41888

Fix installation cmds in docs by @yaswanth19 in #41887

revert changes in _is_package_available by @MekkCyber in #41891

make lfm2_moe integration test pass on XPU by @yao-matrix in #41796

Fix: avoid duplicate token in maybe_load_adapters by @luaenrique in #41903

speed up loading checkpoints for zero stage 3 by @ri938 in #41850

evaluate>=0.4.6 is needed by @stas00 in #41920

Add 6 huggingface notebooks on AMD dev cloud by @fan-amd in #41883

Fix invalid examples in QwenVL model docstrings and add Qwen3VL example by @Xqle in #41812

Allow parse_response to accept token IDs by @Rocketknight1 in #41849

Fix Florence2 conversion script model_type KeyError by @i3hz in #41866

Update some workflow files by @ydshieh in #41892

fix some ut failures on XPU w/ torch 2.9 by @yao-matrix in #41923

Cache latest pytorch amd image locally on mi325 CI runner cluster by @jitesh-gupta in #41926

Minor fix in docker image build workflow by @ydshieh in #41949

fix some ut failures on XPU w/ torch 2.9 by @yao-matrix in #41941

Fix rope_parameters for gemma3 weights conversion script by @douglas-reid in #41922

Fix: Gemma3TextConfig rope scaling assignments by @RyanMullins in #41934

fix prepare_config_and_inputs_for_common bug in llava test by @yao-matrix in #41942

Fix: prevent .gitignore truncation in run_clm_no_trainer.py by @luaenrique in #41957

V4.57.1 training ci: Refactor test_tensor_parallel.py by @3outeille in #41918

[v5] Return a BatchEncoding dict from apply_chat_template by default by @Rocketknight1 in #41626

make recurrent_gemma and voxtral cases pass on xpu by @yao-matrix in #41958

Fix typo in image_processing_lfm2_vl_fast by @yonigozlan in #41940

Run slow v2 by @ydshieh in #41914

Fix detectron2 installation in docker files by @ydshieh in #41975

Fix autoawq[kernels] installation in quantization docker file by @ydshieh in #41978

add support for saving encoder only so any parakeet model can be loaded for inference by @nithinraok in #41969

Use indices as position_ids in modernebert by @remi-or in #41789

test tensor parallel: make tests for dense model more robust by @3outeille in #41968

fix: dict[RopeParameters] to dict[str, RopeParameters] by @RyanMullins in #41963

docs: add continuous batching page by @McPatate in #41847

Fix torchcodec version in quantization docker file by @ydshieh in #41988

[kernels] Add Tests & CI for kernels by @MekkCyber in #41765

Move the Mi355 to regular docker by @remi-or in #41989

More data in benchmarking by @remi-or in #41848

fix (CI): Refactor SSH runners by @glegendre01 in #41991

fix 3 failed test cases for video_llama_3 model on Intel XPU by @kaixuanliu in #41931

Integrate colqwen2.5 using colqwen2 modelling code by @sahil-kabir in #40600

Fixed wrong padding value in OWLv2 by @gjamesgoenawan in #41938

Fix run slow v2: empty report when there is only one model by @ydshieh in #42002

[kernels] change import time in KernelConfig by @MekkCyber in #42004

DOC Fix typo in argument name: pseudoquant by @BenjaminBossan in #41994

Fix torch+deepspeed docker file by @ydshieh in #41985

Correct syntax error in trainer.md by @Yacklin in #42001

Reduce the number of benchmark in the CI by @remi-or in #42008

Fix continuous batching tests by @Rocketknight1 in #42012

add back logging_dir by @SunMarc in #42013

Fix issue with from pretrained and kwargs in image processors by @yonigozlan in #41997

Fix default image_rows and image_cols initialization in Idefics3 and SmolVLM processors by @MilkClouds in #41871

Add GLPNImageProcessorFast by @Aravind-11 in #41725

add fuyu fast image processors by @DeXtAr47-oss in #41817

[kernels] Fix XPU layernorm kernel by @MekkCyber in #41583

[v5] Deprecate Text2Text and related pipelines by @Rocketknight1 in #41996

[FPQuant] MXFP8 and MXFP4 backwards support by @BlackSamorez in #41897

fix deeepspeed in AMD docker file by @ydshieh in #42025

CodeQL workflow for security analysis by @paulinebm in #42015

[tests] Add Context-parallel CI tests by @kashif in #41860

extend fp_quant cases to xpu by @yao-matrix in #41833

Change trigger time for AMD CI by @ydshieh in #42034

Fix the order of methods in processor loading by @zucchini-nlp in #42031

🔴 Isolate prefill from generation loops by @manueldeprada in #40652

update huggingface_hub dependency version by @hanouticelina in #42033

Remove some custom datasets defined in codebase by @ydshieh in #41511

Cleanup workflow - part 1 by @ydshieh in #42023

Fix pr_slow_ci_suggestion.yml after #42023 by @ydshieh in #42049

Fix AutoImageProcessor.register and documentation in auto processing modules by @MilkClouds in #41864

Fix Qwen3-Omni RoPE by @zucchini-nlp in #41778

Avoid explicit checkout in workflow by @ydshieh in #42057

Annoying typo in attention error message by @manueldeprada in #42037

Be careful at explicit checkout actions by @ydshieh in #42060

Fix another Argument list too long in pr_slow_ci_suggestion.yml by @ydshieh in #42061

Fix KeyError in GPT-OSS weight conversion script by @Aznix07 in #42007

Fix KeyError in _is_package_available for packages with dotted names by @yashwantbezawada in #42050

Revert back to use GitHub context by @ydshieh in #42066

Fix missing arg in check_docstring by @yonigozlan in #42054

[deepspeed tests fixes] by @stas00 in #41925

Fix logic in setting self.fsdp when it is False by @roychan in #41974

fix tensor device placement issue of 2 UT cases by @yao-matrix in #41921

add workflow to check permissions and advise a set of permissions req… by @paulinebm in #42071

Fix security issue 5 by @paulinebm in #42072

Fix inconsistency of commit sha during the workflow run by @ydshieh in #42074

QwenVL: add skipped keys in setattr as well by @zucchini-nlp in #41808

permissions worflows fix by @paulinebm in #42080

4.1V Model and GLM-4.5V Model Conversion Code Updates by @zRzRzRzRzRzRzR in #41784

feat(ci): add continuous batching to benchmarks by @McPatate in #41916

Fix modular docstring for Mixtral by @diegoakel in #42041

Fix Auto classes to support dynamically registered processors by @MilkClouds in #41865

Reinstate self.scaling in Gemma3nTextAttention by @RyanMullins in #41751

[v5] 🚨Refactor subprocessors handling in processors by @yonigozlan in #41633

add xpu support in test_modeling_janus.py::JanusIntegrationTest::test… by @sywangyi in #41986

Revert "permissions worflows fix" by @ydshieh in #42110

Fix return metadata checking logic by @Xqle in #42108

Correctly handle unbatched audio inputs in Gemma3nAudioFeatureExtractor by @kho in #42076

[Bugfix] fix qwen3vl expand generation with video by @JJJYmmm in #42089

Fix base model prefix in VLMs by @zucchini-nlp in #42059

fix continuous batching issues, extend ut cases to xpu by @yao-matrix in #41830

📝 docs(smolvlm): fix variable name in batch inference example by @gorkachea in #42123

fix qwen2vl/qwen3vl video processor temporal padding when num_frames%temporal_patch_size!=1 by @yaogang2060 in #42083

[Attn Masks] Non-vmap default for attention masks by @vasqu in #41852

Fix GPT-2 Flash Attention 2 generation with left-padding by @Abdennacer-Badaoui in #41966

Fix model name test for compressed tensors by @SunMarc in #42128

Fix MaskFormer/Mask2Former fast image processors by @yonigozlan in #41393

Remove unused functions in image_transforms.py by @yaswanth19 in #42044

update deps table by @ArthurZucker in #42120

fix: improve video processing fps assignment logic by @Xqle in #42009

Fix T5Gemma module structure by @Cyrilvallez in #42145

DataCollatorForLanguageModeling warning error fixed by @mjaliz in #42144

Bugfix/remove emojis from print by @7amim in #42091

Avoid mutating user-provided arguments in preprocessing utils by @LeonardoEmili in #42126

Enforce check_auto_docstring by @yonigozlan in #41635

Add dinov3 autobackbone by @vijayabhaskar-ev in #41276

Fix logic error in prepare_inputs_for_generation cache slicing condition by @a

Note truncated.

5.0.0rc01 Dec 2025pre-releaseBreaking

Biggest breaking change is in transformers chat. This command starts a terminal UI to interact with a chat model. It used to also be able to start a C…

Transformers v5 release notes

Highlights
Significant API changes: dynamic weight loading, tokenization
Backwards Incompatible Changes
Bugfixes and improvements

Highlights

We are excited to announce the initial release of Transformers v5. This is the first major release in five years, and the release is significant: 800 commits have been pushed to main since the latest minor release. This release removes a lot of long-due deprecations, introduces several refactors that significantly simplify our APIs and internals, and comes with a large number of bug fixes.

We give an overview of our focus for this release in the following blogpost. In these release notes, we'll focus directly on the refactors and new APIs coming with v5.

This release is a release candidate (RC). It is not the final v5 release, and we will push on pypi as a pre-release. This means that the current release is purely opt-in, as installing transformers without specifying this exact release will install the latest version instead (v4.57.3 as of writing).

In order to install this release, please do so with the following:

pip install transformers --pre

For us to deliver the best package possible, it is imperative that we have feedback on how the toolkit is currently working for you. Please try it out, and open an issue in case you're facing something inconsistent/a bug.

Transformers version 5 is a community endeavor, and this is the last mile. Let's ship this together!

Significant API changes

[!NOTE] 👀 Nothing is final and things are still actively in movement. We have a section dedicated to what is planned for future release candidates, yet is known not to work in the RC0. Look for "Disclaimers for the RC0".

We'll be eagerly awaiting your feedback in our GitHub issues!

Dynamic weight loading

We introduce a new weight loading API in transformers, which significantly improves on the previous API. This weight loading API is designed to apply operations to the checkpoints loaded by transformers.

Instead of loading the checkpoint exactly as it is serialized within the model, these operations can reshape, merge, and split the layers according to how they're defined in this new API. These operations are often a necessity when working with quantization or parallelism algorithms.

This new API is centered around the new WeightConverter class:

class WeightConverter(WeightTransform):
    operations: list[ConversionOps]
    source_keys: Union[str, list[str]]
    target_keys: Union[str, list[str]]

The weight converter is designed to apply a list of operations on the source keys, resulting in target keys. A common operation done on the attention layers is to fuse the query, key, values layers. Doing so with this API would amount to defining the following conversion:

conversion = WeightConverter(
    ["self_attn.q_proj", "self_attn.k_proj", "self_attn.v_proj"],  # The input layers
    "self_attn.qkv_proj",  # The single layer as output
    operations=[Concatenate(dim=0)],
)

In this situation, we apply the Concatenate operation, which accepts a list of layers as input and returns a single layer.

This allows us to define a mapping from architecture to a list of weight conversions. Applying those weight conversions can apply arbitrary transformations to the layers themselves. This significantly simplified the from_pretrained method and helped us remove a lot of technical debt that we accumulated over the past few years.

This results in several improvements:

Much cleaner definition of transformations applied to the checkpoint
Reversible transformations, so loading and saving a checkpoint should result in the same checkpoint
Faster model loading thanks to scheduling of tensor materialization
Enables complex mix of transformations that wouldn't otherwise be possible (such as quantization + MoEs, or TP + MoEs)

While this is being implemented, expect varying levels of support across different release candidates.

Linked PR: https://github.com/huggingface/transformers/pull/41580

Tokenization

Just as we moved towards a single backend library for model definition, we want our tokenizers, and the Tokenizer object to be a lot more intuitive. With v5, tokenizer definition is much simpler; one can now initialize an empty LlamaTokenizer and train it directly on your corpus.

Defining a new tokenizer object should be as simple as this:

from transformers import TokenizersBackend, generate_merges
from tokenizers import pre_tokenizers, Tokenizer
from tokenizers.model import BPE

class Llama5Tokenizer(TokenizersBackend):
    def __init__(self, unk_token="<unk>",bos_token="<s>", eos_token="</s>", vocab=None, merges=None ):
        if vocab is None:
            self._vocab = {
                str(unk_token): 0,
                str(bos_token): 1,
                str(eos_token): 2,
            }

        else:
            self._vocab = vocab

        if merges is not None:
            self._merges = merges
        else:
            self._merges = generate_merges(filtered_vocab)

        self._tokenizer = Tokenizer(
            BPE(vocab=self._vocab, merges=self._merges, fuse_unk=True)
        )
        self._tokenizer.pre_tokenizer = pre_tokenizers.Metaspace(
            replacement="▁", prepend_scheme=_get_prepend_scheme(self.add_prefix_space, self), split=False
        )
        super().__init__(
            tokenizer_object=self._tokenizer,
            unk_token=unk_token,
            bos_token=bos_token,
            eos_token=eos_token,
        )

Once the tokenizer is defined as above, you can load it with the following: Llama5Tokenizer(). Doing this returns you an empty, trainable tokenizer that follows the definition of the authors of Llama5 (it does not exist yet :wink:).

The above is the main motivation towards refactoring tokenization: we want tokenizers to behave similarly to models: trained or empty, and with exactly what is defined in their class definition.

Backend Architecture Changes: moving away from the slow/fast tokenizer separation

Up to now, transformers maintained two parallel implementations for many tokenizers:

"Slow" tokenizers (tokenization_<model>.py) - Python-based implementations, often using SentencePiece as the backend.
"Fast" tokenizers (tokenization_<model>_fast.py) - Rust-based implementations using the 🤗 tokenizers library.

In v5, we consolidate to a single tokenizer file per model: tokenization_<model>.py. This file will use the most appropriate backend available:

TokenizersBackend (preferred): Rust-based tokenizers from the 🤗 tokenizers library. In general it provides optimal performance, but it also offers a lot more features that are commonly adopted across the ecosystem:

handling additional tokens
a full python API for setting and updating
automatic parallelization,
automatic offsets
customization
training

SentencePieceBackend: for tokenizers requiring the sentencepiece library. It inherits from PythonBackend.
PythonBackend: a Python implementations of the features provided by tokenizers. Basically allows adding tokens.
MistralCommonBackend: relies on MistralCommon's tokenization library. (Previously known as the MistralCommonTokenizer)

The AutoTokenizer automatically selects the appropriate backend based on available files and dependencies. This is transparent, you continue to use AutoTokenizer.from_pretrained() as before. This allows transformers to be future-proof and modular to easily support future backends.

Defining a tokenizers outside of the existing backends

We enable users and tokenizer builders to define their own tokenizers from top to bottom. Tokenizers are usually defined using a backend such as tokenizers, sentencepiece or mistral-common, but we offer the possibility to design the tokenizer at a higher-level, without relying on those backends.

To do so, you can import the PythonBackend (which was previously known as PreTrainedTokenizer). This class encapsulates all the logic related to added tokens, encoding, and decoding.

If you want something even higher up the stack, then PreTrainedTokenizerBase is what PythonBackend inherits from. It contains the very basic tokenizer API features:

encode
decode
vocab_size
get_vocab
convert_tokens_to_ids
convert_ids_to_tokens
from_pretrained
save_pretrained
among a few others

API Changes

1. Direct tokenizer initialization with vocab and merges

Starting with v5, we now enable initializing blank, untrained tokenizers-backed tokenizers:

from transformers import LlamaTokenizer

tokenizer = LlamaTokenizer()

This tokenizer will therefore follow the definition of the LlamaTokenizer as defined in its class definition. It can then be trained on a corpus as can be seen in the tokenizers documentation.

These tokenizers can also be initialized from vocab and merges (if necessary), like the previous "slow" tokenizers:

from transformers import LlamaTokenizer

vocab = {"<unk>": 0, "<s>": 1, "</s>": 2, "hello": 3, "world": 4}
merges = [("h", "e"), ("l", "l"), ("o", " ")]

tokenizer = LlamaTokenizer(vocab=vocab, merges=merges)

This tokenizer will behave as a Llama-like tokenizer, with an updated vocabulary. This allows comparing different tokenizer classes with the same vocab; therefore enabling the comparison of different pre-tokenizers, normalizers, etc.

⚠️ The vocab_file (as in, a path towards a file containing the vocabulary) cannot be used to initialize the LlamaTokenizer as loading from files is reserved to the from_pretrained method.

2. Simplified decoding API

The batch_decode and decode methods have been unified to reflect behavior of the encode method. Both single and batch decoding now use the same decode method. See an example of the new behavior below:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("t5-small") 
inputs = ["hey how are you?", "fine"]
tokenizer.decode(tokenizer.encode(inputs))

Gives:

- 'hey how are you?</s> fine</s>'
+ ['hey how are you?</s>', 'fine</s>']

We expect encode and decode to behave, as two sides of the same coin: encode, process, decode, should work.

[!NOTE] A common use-case would be: encode, model.generate, decode. However, using generate would return list[list[int]], which would then be incompatible with decode.

3. Unified encoding API

The encode_plus method is deprecated in favor of the single __call__ method.

4. `apply_chat_template` returns `BatchEncoding`

Previously, apply_chat_template returned input_ids for backward compatibility. Starting with v5, it now consistently returns a BatchEncoding dict like other tokenizer methods.

# v5
messages = [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi there!"}
]

# Now returns BatchEncoding with input_ids, attention_mask, etc.
outputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
print(outputs.keys())  # dict_keys(['input_ids', 'attention_mask'])

5. Removed legacy configuration file saving:

We simplify the serialization of tokenization attributes:

special_tokens_map.json - special tokens are now stored in tokenizer_config.json.
added_tokens.json - added tokens are now stored in tokenizer.json.
added_tokens_decoder is only stored when there is no tokenizer.json.

When loading older tokenizers, these files are still read for backward compatibility, but new saves use the consolidated format. We're gradually moving towards consolidating attributes to fewer files so that other libraries and implementations may depend on them more reliably.

6. Model-Specific Changes

Several models that had identical tokenizers now import from their base implementation:

LayoutLM → uses BertTokenizer
LED → uses BartTokenizer
Longformer → uses RobertaTokenizer
LXMert → uses BertTokenizer
MT5 → uses T5Tokenizer
MVP → uses BartTokenizer

These modules will eventually be removed altogether.

Removed T5-specific workarounds

The internal _eventually_correct_t5_max_length method has been removed. T5 tokenizers now handle max length consistently with other models.

Testing Changes

A few testing changes specific to tokenizers have been applied:

Model-specific tokenization test files now focus on integration tests.
Common tokenization API tests (e.g., add_tokens, encode, decode) are now centralized and automatically applied across all tokenizers. This reduces test duplication and ensures consistent behavior

For legacy implementations, the original BERT Python tokenizer code (including WhitespaceTokenizer, BasicTokenizer, etc.) is preserved in bert_legacy.py for reference purposes.

7. Deprecated / Modified Features

Special Tokens Structure:

SpecialTokensMixin: Merged into PreTrainedTokenizerBase to simplify the tokenizer architecture.
special_tokens_map: Now only stores named special token attributes (e.g., bos_token, eos_token). Use extra_special_tokens for additional special tokens (formerly additional_special_tokens). all_special_tokens includes both named and extra tokens.

# v4
tokenizer.special_tokens_map  # Included 'additional_special_tokens'

# v5
tokenizer.special_tokens_map  # Only named tokens
tokenizer.extra_special_tokens  # Additional tokens

special_tokens_map_extended and all_special_tokens_extended: Removed. Access AddedToken objects directly from _special_tokens_map or _extra_special_tokens if needed.
additional_special_tokens: Still accepted for backward compatibility but is automatically converted to extra_special_tokens.

Deprecated Methods:

sanitize_special_tokens(): Already deprecated in v4, removed in v5.
prepare_seq2seq_batch(): Deprecated; use __call__() with text_target parameter instead.

# v4
model_inputs = tokenizer.prepare_seq2seq_batch(src_texts, tgt_texts, max_length=128)

# v5
model_inputs = tokenizer(src_texts, text_target=tgt_texts, max_length=128, return_tensors="pt")
model_inputs["labels"] = model_inputs.pop("input_ids_target")

BatchEncoding.words(): Deprecated; use word_ids() instead.

Removed Methods:

create_token_type_ids_from_sequences(): Removed from base class. Subclasses that need custom token type ID creation should implement this method directly.
clean_up_tokenization(): Removed from base class. Now defined at model class level for models that need it (e.g., PLBart, CLVP, Wav2Vec2).
prepare_for_model(), build_inputs_with_special_tokens(), truncate_sequences(): Moved from tokenization_utils_base.py to tokenization_python.py for PythonBackend tokenizers. TokenizersBackend provides model-ready input via tokenize() and encode(), so these methods are no longer needed in the base class.
_switch_to_input_mode(), _switch_to_target_mode(), as_target_tokenizer(): Removed from base class. Use __call__() with text_target parameter instead.

# v4
with tokenizer.as_target_tokenizer():
    labels = tokenizer(tgt_texts, ...)

# v5
labels = tokenizer(text_target=tgt_texts, ...)

parse_response(): Removed from base class.

Disclaimers for the RC0

PEFT + MoE:

Because we are switching from the naive MOE (nn.ModuleList for experts) we currently have an issue with MoEs that have adapters. For more details see https://github.com/huggingface/transformers/issues/42491#issuecomment-3591485649.

We aim for this to be fixed and released in a following release candidate in the week that follows RC0.

Tensor parallel and Expert parallel + MoE

We are streamlining the MoE support with vLLM; while this is being implemented, tensor parallelism and expert parallelism aren't working as expected. This is known and actively being worked on.

We aim for this to be fixed and released in a following release candidate in the week that follows RC0.

Custom pretrained models:

For anyone inheriting from a transformers PreTrainedModel, the weights are automatically initialized with the common scheme:


    @torch.no_grad()
    def _init_weights(self, module):
        """
        Initialize the weights. This is quite general on purpose, in the spirit of what we usually do. For more complex
        initialization scheme, it should be overridden by the derived `PreTrainedModel` class. In case a model adds an explicit
        `nn.Parameter`, this method should also be overridden in order to initialize it correctly.
        """
        if hasattr(self.config, "initializer_range"):
            std = self.config.initializer_range or 0.02
        elif hasattr(self.config, "init_std"):
            std = self.config.init_std
        elif hasattr(self.config, "initializer_factor"):
            std = self.config.initializer_factor
        else:
            # 0.02 is the standard default value across the library
            std = getattr(self.config.get_text_config(), "initializer_range", 0.02)

        if isinstance(module, (nn.Linear, nn.Conv1d, nn.Conv2d, nn.Conv3d, nn.ConvTranspose1d, nn.ConvTranspose2d)):
            if getattr(module, "weight", None) is not None:
                init.normal_(module.weight, mean=0.0, std=std)
            if getattr(module, "bias", None) is not None:
                init.zeros_(module.bias)
        elif isinstance(module, nn.Embedding):
            if getattr(module, "weight", None) is not None:
                init.normal_(module.weight, mean=0.0, std=std)
                # Here we need the check explicitly, as we slice the weight in the `zeros_` call, so it looses the flag
                if module.padding_idx is not None and not getattr(module.weight, "_is_hf_initialized", False):
                    init.zeros_(module.weight[module.padding_idx])
        elif isinstance(module, nn.MultiheadAttention):
            # This uses torch's original init
            module._reset_parameters()
        # We cannot use `isinstance` on the RMSNorms or LayerNorms, as they usually are custom modules which change names
        # between modelings (because they are prefixed with the model name)
        elif (
            isinstance(module, (nn.GroupNorm, nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d))
            or "LayerNorm" in module.__class__.__name__
            or "RMSNorm" in module.__class__.__name__
        ):
            # Norms can exist without weights (in which case they are None from torch primitives)
            if hasattr(module, "weight") and module.weight is not None:
                init.ones_(module.weight)
            if hasattr(module, "bias") and module.bias is not None:
                init.zeros_(module.bias)

If you want to avoid that, for now you should just do:

class CustomModel(Qwen3VLForConditionalGeneration):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.action_head = nn.Linear(1024, 7)
        self.positional_embedding = nn.Parameter(torch.randn(16, 1152))
        self.post_init()
    
    def _init_weights(self, module):
        pass

There is a tracker for that here: https://github.com/huggingface/transformers/issues/42418.

Library-wide changes with lesser impact

`use_auth_token`

The use_auth_token argument/parameter is deprecated in favor of token everywhere. You should be able to search and replace use_auth_token with token and get the same logic.

Linked PR: https://github.com/huggingface/transformers/pull/41666

Attention-related features

We decided to remove some features for the upcoming v5 as they are currently only supported in a few old models and no longer integrated in current model additions. It's recommended to stick to v4.x in case you need them. Following features are affected:

No more head masking, see #41076. This feature allowed to turn off certain heads during the attention calculation and only worked for eager.
No more relative positional biases in Bert-like models, see #41170. This feature was introduced to allow relative position scores within attention calculations (similar to T5). However, this feature is barely used in official models and a lot of complexity instead. It also only worked with eager.
No more head pruning, see #41417 by @gante. As the name suggests, it allowed to prune heads within your attention layers.

Updates to supported torch APIs

We dropped support for two torch APIs:

torchscript in https://github.com/huggingface/transformers/pull/41688
torch.fx in https://github.com/huggingface/transformers/pull/41683

Those APIs were deprecated by the PyTorch team, and we're instead focusing on the supported APIs dynamo and export.

Quantization changes

We clean up the quantization API in transformers, and significantly refactor the weight loading as highlighted above.

We drop support for two quantization arguments that have been deprecated for some time:

load_in_4bit
load_in_8bit

We remove them in favor of the quantization_config argument which is much more complete. As an example, here is how you would load a 4-bit bitsandbytes model using this argument:

from transformers import AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_4bit=True)

model_4bit = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B",
    device_map="auto",
    quantization_config=quantization_config
)

Configuration

Methods to init a nested config such as from_xxx_config are deleted. Configs can be init from the __init__ method in the same way. See #41314.
It is no longer possible to load a config class from a URL file. Configs must be loaded from either a local path or a repo on the Hub. See #42383.
All parameters for configuring model's rotary embedding are now stored under mode.rope_parameters, including the rope_theta and rope_type. Model's config.rope_parameters is a simple dictionaty in most cases, and can also be a nested dict in special cases (i.e. Gemma3 and ModernBert) with different rope parameterization for each layer type. Trying to get config.rope_theta will throw an attribute error from now on. See #39847 and #42255
Qwen-VL family configuration is in a nested format and trying to access keys directly will throw an error (e.g. config.vocab_size). Users are expected to access keys from their respective sub-configs (config.text_config.vocab_size).
Configurations of non-generative models (any model that doesn't call model.generate()) will no longer have a generation_config and model.config.generation_config will throw an attribute error.

Processing

Tokenization

Slow tokenizer files (aka: tokenization_<model>.py ) will be removed in favor of using fast tokenizer files tokenization_<model>_fast.py --> will be renamed to tokenization_<model>.py. As fast tokenizers are :hugs:tokenizers - backend, they include a wider range of features that are maintainable and reliable.
Other backends (sentence piece, tokenizers, etc.) will be supported with a light layer if loading a fast tokenizer fails
Remove legacy files like special_tokens_map.json and added_tokens.json
Remove _eventually_correct_t5_max_length
encode_plus --> __call__
batch_decode --> decode

apply_chat_template by default returns naked input_ids rather than a BatchEncoding dict. This was inconvenient - it should return a BatchEncoding dict like tokenizer.__call__(), but we were stuck with it for backward compatibility. The method now returns a BatchEncoding.

Linked PRs:

https://github.com/huggingface/transformers/issues/40938
https://github.com/huggingface/transformers/pull/40936
https://github.com/huggingface/transformers/pull/41626

Processing classes

In processing classes each attribute will be serialized under processor_config.json as a nested dict, instead of serializing attributes in their own config files. Loading will be supported for all old format processors (https://github.com/huggingface/transformers/pull/41474)
XXXFeatureExtractors classes are completely removed in favor of XXXImageProcessor class for all vision models (https://github.com/huggingface/transformers/pull/41174)
Minor change: XXXFastImageProcessorKwargs is removed in favor of XXXImageProcessorKwargs which will be shared between fast and slow processors (https://github.com/huggingface/transformers/pull/40931)

Modeling

Some RotaryEmbeddings layers will start returning a dict of tuples, in case the model uses several RoPE configurations (Gemma2, ModernBert). Each value will be a tuple of "cos, sin" per RoPE type.
Config attribute for RotaryEmbeddings layer will be unified and accessed via config.rope_parameters. Config attr for rope_theta might not be accessible anymore for some models, and instead will be in config.rope_parameters['rope_theta']. BC will be supported for a while as much as possible, and in the near future we'll gradually move to the new RoPE format (https://github.com/huggingface/transformers/pull/39847)
Vision Language models will not have a shortcut access to its language and vision component from the generative model via model.language_model. It is recommended to either access the module with model.model.language_model or model.get_decoder(). See #42156

Generate

Old, deprecated output type aliases were removed (e.g. GreedySearchEncoderDecoderOutput). We now only have 4 output classes built from the following matrix: decoder-only vs encoder-decoder, uses beams vs doesn't use beams (https://github.com/huggingface/transformers/pull/40998)
Removed deprecated classes regarding decoding methods that were moved to the Hub due to low usage (constraints and beam scores) (https://github.com/huggingface/transformers/pull/41223)
If generate doesn't receive any KV Cache argument, the default cache class used is now defined by the model (as opposed to always being DynamicCache) (https://github.com/huggingface/transformers/pull/41505)
Generation parameters are no longer accessible via model's config. If generation paramaters are serialized in config.json for any old model, it will be loaded back into model's generation config. Users are expected to access or modify generation parameters only with model.generation_config.do_sample = True.

Trainer

New Features

ALST/Ulysses Sequence Parallelism Integration
- Added sequence parallelism support via HF Accelerate for training with longer sequences. Enables splitting sequences across devices using ALST (All-to-All Long Sequence Training) and Ulysses algorithms with DeepSpeed.
Improved compute_loss_func Handling
- compute_loss_func now always takes priority over the model's built-in loss computation, giving users consistent control over custom loss functions.
num_items_in_batch in Prediction Step
- The num_items_in_batch argument is now passed to compute_loss during prediction_step, enabling proper loss scaling during evaluation.

Breaking Changes

report_to now defaults to "none"
- Logging integrations are no longer auto-detected by default; users must explicitly specify which reporting backends to use.

Removing arguments without deprecation cycle in `TrainingArguments` due to low usage

mp_parameters -> legacy param that was later on added to the Sagemaker trainer
_n_gpu -> not intended for users to set, we will initialize it correctly instead of putting it in the TrainingArguments
overwrite_output_dir - > replaced by resume_from_checkpoint, and it was only used in the examples script, no impact on Trainer.
logging_dir -> only used for tensorboard, set TENSORBOARD_LOGGING_DIR env var instead
jit_mode_eval -> use use_torch_compile instead, as torchscript is not recommended anymore
tpu_num_cores-> It is actually better to remove it, as it is not recommended to set the number of cores. By default, all TPU cores are used . Set TPU_NUM_CORES env var instead
past_index -> it was only used for a very small number of models that have special architecture like transformersxl + it was not documented at all how to train those models
ray_scope -> only for a minor arg for ray integration. Set RAY_SCOPE var env instead
warmup_ratio -> use warmup_step instead. We combined both args together by allowing passing float values in warmup_step.

Removing deprecated arguments in `TrainingArguments`

fsdp_min_num_params and fsdp_transformer_layer_cls_to_wrap -> use fsdp_config
tpu_metrics_debug -> debug
push_to_hub_token -> hub_token
push_to_hub_model_id and push_to_hub_organization -> hub_model_id
include_inputs_for_metrics -> include_for_metrics
per_gpu_train_batch_size -> per_device_train_batch_size
per_gpu_eval_batch_size -> per_device_eval_batch_size
use_mps_device -> mps will be used by default if detected
fp16_backend and half_precision_backend -> we will only rely on torch.amp as everything has been upstreamed to torch
no_cuda -> use_cpu
include_tokens_per_second -> include_num_input_tokens_seen
use_legacy_prediction_loop -> we only use evaluation_loop function from now on

Removing deprecated arguments in `Trainer`

tokenizer in initialization -> processing_class
model_path in train() -> resume_from_checkpoint

Removed features for `Trainer`

sigpot integration for hp search was removed as the library was archived + the api stopped working
drop support for sagemaker API <1.10
bump accelerate minimum version to 1.1.0
bump peft minimum version to 0.18.0
bump bitsandbytes minimum version to 0.46.1

New defaults for `Trainer`

use_cache in the model config will be set to False. You can still change the cache value through TrainingArguments usel_cache argument if needed.

Pipeline

Image text to text pipelines will no longer accept images as a separate argument along with conversation chats. Image data has to be embedded in the chat's "content" field. See #42359

PushToHubMixin

removed deprecated organization and repo_url from PushToHubMixin. You must pass a repo_id instead.
removed ignore_metadata_errors from PushToMixin. In practice if we ignore errors while loading the model card, we won't be able to push the card back to the Hub so it's better to fail early and not provide the option to fail later.
push_to_hub do not accept **kwargs anymore. All accepted parameters are explicitly documented.
arguments of push_to_hub are now keyword-only to avoid confusion. Only repo_id can be positional since it's the main arg.
removed use_temp_dir argument from push_to_hub. We now use a tmp dir in all cases.

Linked PR: https://github.com/huggingface/transformers/pull/42391.

CLI

The deprecated transformers-cli ... command was deprecated, transformers ... is now the only CLI entry point.

transformers CLI has been migrated to Typer, making it easier to maintain + adding some nice features out of the box (improved --help section, autocompletion).

Biggest breaking change is in transformers chat. This command starts a terminal UI to interact with a chat model. It used to also be able to start a Chat Completion server powered by transformers and chat with it. In this revamped version, this feature has been removed in favor of transformers serve. The goal of splitting transformers chat and transformers serve is to define clear boundaries between client and server code. It helps with maintenance but also makes the commands less bloated. The new signature of transformers chat is:

Usage: transformers chat [OPTIONS] BASE_URL MODEL_ID [GENERATE_FLAGS]...

Chat with a model from the command line.

It works hand in hand with transformers serve, which means that if transformers serve is running on its default endpoint, transformers chat can be launched as follows:

transformers chat HuggingFaceTB/SmolLM3-3B

It can however use any OpenAI API compatible HTTP endpoint:

transformers chat HuggingFaceTB/SmolLM3-3B https://router.huggingface.co/v1

Linked PRs:

https://github.com/huggingface/transformers/pull/40997
https://github.com/huggingface/transformers/pull/41487

Removal of the `run` method

The transformers run (previously transformers-cli run) is an artefact of the past, was not documented nor tested, and isn't part of any public documentation. We're removing it for now and ask you to please let us know in case this is a method you are using; in which case we should bring it back with better support.

Linked PR: https://github.com/huggingface/transformers/pull/42447

Environment variables

Legacy environment variables like TRANSFORMERS_CACHE, PYTORCH_TRANSFORMERS_CACHE, and PYTORCH_PRETRAINED_BERT_CACHE have been removed. Please use HF_HOME instead.
Constants HUGGINGFACE_CO_EXAMPLES_TELEMETRY, HUGGINGFACE_CO_EXAMPLES_TELEMETRY, HUGGINGFACE_CO_PREFIX, and HUGGINGFACE_CO_RESOLVE_ENDPOINT have been removed. Please use huggingface_hub.constants.ENDPOINT instead.

Linked PR: https://github.com/huggingface/transformers/pull/42391.

Requirements update

transformers v5 pins the huggingface_hub version to >=1.0.0. See this migration guide to learn more about this major release. Here are to main aspects to know about:

switched the HTTP backend from requests to httpx. This change was made to improve performance and to support both synchronous and asynchronous requests the same way. If you are currently catching requests.HTTPError errors in your codebase, you'll need to switch to httpx.HTTPError.
related to 1., it is not possible to set proxies from your script. To handle proxies, you must set the HTTP_PROXY / HTTPS_PROXY environment variables
hf_transfer and therefore HF_HUB_ENABLE_HF_TRANSFER have been completed dropped in favor of hf_xet. This should be transparent for most users. Please let us know if you notice any downside!

typer-slim has been added as required dependency, used to implement both hf and transformers CLIs.

New model additions in v5

CWM

The Code World Model (CWM) model was proposed in CWM: An Open-Weights LLM for Research on Code Generation with World Models by Meta FAIR CodeGen Team. CWM is an LLM for code generation and reasoning about code that has, in particular, been trained to better represent and reason about how code and commands affect the state of a program or system. Specifically, we mid-trained CWM on a large number of observation-action trajectories from Python execution traces and agentic interactions in containerized environments. We post-trained with extensive multi-task RL in verifiable coding, math, and multi-turn software engineering environments.

Add Code World Model (CWM) by @jacobkahn in #41199

SAM3

SAM3 (Segment Anything Model 3) was introduced in SAM 3: Segment Anything with Concepts.

The SAM3 addition adds four new architectures:

Sam3
Sam3Tracker
Sam3TrackerVideo
Sam3Video

SAM3 performs Promptable Concept Segmentation (PCS) on images. PCS takes text and/or image exemplars as input (e.g., "yellow school bus"), and predicts instance and semantic masks for every single object matching the concept.

Sam3Tracker and Sam3TrackerVideo perform Promptable Visual Segmentation (PVS) on images. PVS takes interactive visual prompts (points, boxes, masks) or text inputs to segment a specific object instance per prompt. This is the task that SAM 1 and SAM 2 focused on, and SAM 3 improves upon it. Sam3Tracker and Sam3TrackerVideo are updated versions of SAM2 Video that maintain the same API while providing improved performance and capabilities.

SAM3 Video performs Promptable Concept Segmentation (PCS) on videos. PCS takes text as input (e.g., "yellow school bus"), and predicts instance and semantic masks for every single object matching the concept, while preserving object identities across video frames. The model combines a detection module (SAM3) with a tracking module (SAM2-style tracker) to enable robust object tracking across video frames using text prompts.

Add SAM3 to 🤗 Transformers by @yonigozlan in #42285

LFM2 MoE

LFM2-MoE is a Mixture-of-Experts (MoE) variant of LFM2. The LFM2 family is optimized for on-device inference by combining short‑range, input‑aware gated convolutions with grouped‑query attention (GQA) in a layout tuned to maximize quality under strict speed and memory constraints.

LFM2‑MoE keeps this fast backbone and introduces sparse MoE feed‑forward networks to add representational capacity without significantly increasing the active compute path. The first LFM2-MoE release is LFM2-8B-A1B, with 8.3B total parameters and 1.5B active parameters. The model excels in quality (comparable to 3-4B dense models) and speed (faster than other 1.5B class models).

[Model] Lfm2Moe by @paulpak58 in #41401

VideoLlama 3

The VideoLLaMA3 model is a major update to VideoLLaMA2 from Alibaba DAMO Academy.

[model] Add VideoLLaMA3 implementation by @lkhl in #40499

AudioFlamingo 3

Audio Flamingo 3 (AF3) is a fully open large audio–language model designed for robust understanding and reasoning over speech, environmental sounds, and music. AF3 pairs a Whisper-style audio encoder with a causal language model and performs replace-in-place audio–text fusion: the processor aligns post-pool audio frames to a dedicated placeholder token and the model replaces those token slots with projected audio embeddings during the forward pass.

The model checkpoint is available at: nvidia/audio-flamingo-3-hf

Highlights:

Unified audio encoder across speech, sound, and music.
Long-audio support via windowing and post-pool alignment (up to 10 minutes maximum). The model processes audio in 30-second windows with a hard limit of 20 windows (10 minutes total). Audio longer than 10 minutes will be truncated.
Deterministic fusion that preserves sequence length by replacing audio placeholder tokens with audio embeddings.

[models] Add AudioFlamingo3 integration by @lashahub in #40290

Nanochat

NanoChat is a compact decoder-only transformer model designed for educational purposes and efficient training. The model features several fundamental architectural innovations which are common in modern transformer models. Therefore, it is a good model to use as a starting point to understand the principles of modern transformer models. NanoChat is a variant of the Llama architecture, with simplified attention mechanism and normalization layers.

[MODEL] Nanochat implementation by @burtenshaw in #41634

Bugfixes and improvements

JetMoe Fix jetmoe after #40132 by @ArthurZucker in #41324
Fixed tiny incorrect import in gemma3 by @Sai-Suraj-27 in #41354
Rope for Qwen2--5-vl by @zucchini-nlp in #41173
🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence by @Cyrilvallez in #41268
Standardize PretrainedConfig to PreTrainedConfig by @Cyrilvallez in #41300
Fix trainer for py3.9 by @SunMarc in #41359
Check model inputs - hidden states by @zucchini-nlp in #40994
[ModularChecker] QOL for the modular checker by @ArthurZucker in #41361
Fixing a typo for BLT model by @Narsil in #41325
:rotating_light: [v5] Remove relative position embeddings (for bert like models) by @vasqu in #41170
Fix typo in model proposal template by @Ombucha in #41352
Better typehints for apply_chat_template by @Samoed in #41355
🚨 Remove BetterTransformer by @Cyrilvallez in #41367
[testing] update test_longcat_generation_cpu by @ydshieh in #41368
Fix flash_attention.py: wrong argument passing for attn_implementation by @TKONIY in #41347
Use canonical get_size_with_aspect_ratio (with max_size) from transformers.image_transforms to fix #37939 by @sonianuj287 in #41284
Fixes in check_model_inputs, GPTBigCodeModel and ImageGPTModel by @IlyasMoutawwakil in #40811
Remove unnecessary list comprehension by @cyyever in #41305
make some ut cases pass on xpu w/ latest torch by @yao-matrix in #41337
Remove unused function patameters by @cyyever in #41358
[CB] Refactors the way we access paged by @ArthurZucker in #41370
serve: add non-streaming mode to /v1/responses; stream event parity; remove placeholder logprobs by @antznette1 in #41353
Update from pretrained error when loading by @ArthurZucker in #33380
[v5] Sync Bert and Bart eager attention by @vasqu in #41248
fix asr ut failures by @yao-matrix in #41332
fix resample in asr pipeline by @yhzx233 in #41298
Correct numerical regression in vision embeddings by @i3hz in #41374
[kernels] Kernel Config by @MekkCyber in #41232
[Cache] lfm2 cache: allocate empty kv layers during init by @paulpak58 in #41396
Fix test for model with dotted name and relative imports by @st81 in #41343
Prefer raising TypeError exception for invalid type by @Sai-Suraj-27 in #41346
[v5] Bump accelerate to 1.1.0 by @SunMarc in #41234
Fix incorrect assignment in update_device_map for GPTQ quantizer by @Sai-Suraj-27 in #41328
[v5] Delete left traces of feature extractor by @zucchini-nlp in #41321
Remove deprecation warning by @Cyrilvallez in #41425
Fix overriding common_kwargs defaults in processor calls by @yonigozlan in #41381
v5 dev version by @LysandreJik in #41436
Tiny Cleanup - Removed duplicate class field definition's by @Sai-Suraj-27 in #41293
🚨🚨 Remove all traces of legacy cache format by @Cyrilvallez in #41378
🚨 [v5] Prune prune_heads by @gante in #41417
[v5] Bump min version of bitsandbytes to 0.46.1 by @SunMarc in #41283
Fixing comments in init file by @MekkCyber in #41414
Use accelerator API to free device memory by @cyyever in #41195
enable new model uts to xpu and fix some failures on xpu by @yao-matrix in #41386
[torchao] Add regex support for ModuleFqnToConfig by @jerryzh168 in #41242
:facepalm: CB nit! by @ArthurZucker in #41413
Remove Python 3.9 classifier by @cyyever in #41410
[JetMoe] Fix KV head repetition and padding free by @vasqu in #41423
[testing] Fix JetMoeIntegrationTest by @ydshieh in #41377
Add Top-H decoding (entropy-bounded truncation) as a LogitsWarper for text generation by @ErfanBaghaei in #40837
Validate processing kwargs with @strict from huggingface_hub by @zucchini-nlp in #40793
Update hqq.md by @prathamesh-chavan-22 in #41452
enable some falcon-mamba uts on xpu by @yao-matrix in #41428
Fix generate outputs and simplify cache tests by @Cyrilvallez in #41440
Fix doc by @Cyrilvallez in #41457
🚨 [v5] Rename left traces of past_key_value in BERT-like models by @zucchini-nlp in #41448
Subconfig is a class attribute by @zucchini-nlp in #41308
[v5] rm utils/tf_ops/ by @gante in #41402
Update GLM-4.1V MMRope implementation by @zRzRzRzRzRzRzR in #41182
[kernels] Cleanup deta kernel by @MekkCyber in #41470
🚨 [v5] Rendundant code in nested configs by @zucchini-nlp in #41314
Remove KERAS_NLP_IMPORT_ERROR by @cyyever in #41468
Fix auto model configuration for encoder of perceptionlm by @fschlatt in #41464
Fix tests fsdp by @SunMarc in #41422
Import Callable from collections.abc by @cyyever in #41130
Pickle - part 2 by @ydshieh in #41476
Remove infer_device by @cyyever in #41088
Change RT-Detr docs to reflect fixed 640x640 input size by @konstantinos-p in #41364
Cleaning hub kernels by @MekkCyber in #41477
[v5] remove load_in_4bit and load_in_8bit by @SunMarc in #41287
:rotating_light: [Attention Masks] Bidirectional masks for encoder and encoder-decoder models by @vasqu in #41265
[Fix] Fix test file error by @YangKai0616 in #40973
enhance patched_tearDown to support python 3.11+ by @yao-matrix in #41429
RT-Detr correct 2d positional embeddings for non-square images by @konstantinos-p in #41380
Fix bnb fsdp loading for pre-quantized checkpoint by @SunMarc in #41415
Remove SigOpt by @SunMarc in #41479
Remove past_index by @SunMarc in #41384
Remove deprecated args in Trainer for v5 by @SunMarc in #41404
Update GLM-4.6 doc by @zRzRzRzRzRzRzR in #41471
report_to default changed to "none" + cleaning deprecated env var by @SunMarc in #41375
deprecate overwrite_output_dir by @SunMarc in #41323
[CI] Fix copies on main by @vasqu in #41486
[Trainer] deprecate ray scope by @SunMarc in #41403
deprecate jit_mode_eval by @SunMarc in #41376
Remove local_rank arg from TrainingArguments by @SunMarc in #41382
Update philosophy by @molbap in #41438
Remove DISABLE_KERNEL_MAPPING flag by @MekkCyber in #41475
Streaming should be handled at the request-level rather than at the istance level by @LysandreJik in #41444
fix bnb model loading by @jiqing-feng in #41499
[kernels] Remove RWKV kernel finally ! by @MekkCyber in #41493
[kernels] rm yoso kernel by @MekkCyber in #41495
Try to remove pickle - BloomTokenizerFast by @ydshieh in #41466
Fixed tiny incorrect imports in glm4v by @Sai-Suraj-27 in #41483
[Parakeet] unnecessary warning & auto mapping by @eustlb in #41412
[causallm tester] automate pipeline mappings + bloom tests by @gante in #41318
Fix some tests by @Cyrilvallez in #41503
fix gemma3n case failure by @yao-matrix in #41426
[voxtral] language detection + skipping lang:xx by @eustlb in #41225
Set truncation to False in Qwen3Omni to avoid default truncation by @BakerBunker in #41473
[QoL] modular conversion shows LoC saved by @molbap in #41500
More trainer cleaning by @SunMarc in #41489
Bump to hfh 1.0.0.rc5 to fix test by @Wauplin in #41508
Revert local_rank deletion and some cleaning by @SunMarc in #41504
Fix detectron2 import by @Cyrilvallez in #41510
add Trainer import to .md in appropriate cell block for training.ipynb transformers_doc by @benkeene in #41484
Remove outdated flags by @Cyrilvallez in #41512
remove tpu_num_cores by @SunMarc in #41383
Allow optuna's catch kwargs passthrough by @nicha-api in #41496
Fix Latex typesetting in documentation by @cyyever in #41177
[testing] reduce runtime of HunYuanMoEV1IntegrationTest:test_model_generation by @ydshieh in #41373
[Qwen3VL] fix: hidden_states in place modification error by @HollowMan6 in #41535
Add MLlama fast image processor by @yonigozlan in #41391
Fixed Type-hints in function defintions by @Sai-Suraj-27 in #41525
[SAM] Fix typing hints by @zucchini-nlp in #41506
Restore cuda graphs to continuous batching by @remi-or in #41421
Add AMD developer cloud support by @fan-amd in #41126
Enable modular files from other libraries by @regisss in #41372
🚨 [v5] generate delegates default cache initialization to the model by @gante in #41505
Fixed typos and formatting by @julian-st in #34215
Add VideoMAE video processor by @Aki-07 in #41534
[from_pretrained] Small refactor from_pretrained: move around unrelated stuff by @ArthurZucker in #41445
Remove references to AutoModelForVision2Seq by @Rocketknight1 in #41513
[Qwen3VL] fix device mismatch error for FSDP2 training by @HollowMan6 in #41536
Patch MistralCommonTokenizer by @juliendenize in #41439
Fix an import error with PreTrainModel by @remi-or in #41571
[Qwen3VLMoe] Fixed: Expected self.dtype to be equal to src.dtype - routing_weights casting by @danielquintas8 in #41420
[kernels] rm mra kernels by @MekkCyber in #41507
delete some tokenizer tests using pickle by @ydshieh in #41514
Add DINOv3Backbone for ConvNext variant by @merveenoyan in #40651
Add conditional checks to _check_and_adjust_attn_implementation() by @zheliuyu in #41542
add rmsnorm kernels support for Intel XPU by @kaixuanliu in #41563
Revert "add rmsnorm kernels support for Intel XPU" by @MekkCyber in #41579
[VisionEncoderDecoderModel] Update loss function by @NielsRogge in #40863
Add iter to DynamicCache by @remi-or in #41569
Revert some breaking changes bnb by @SunMarc in #41581
Fix typsetting and content of llm_tutorial_optimization.md by @cyyever in #41172
Gemma3 fixes by @remi-or in #41572
Benchmark overhaul by @remi-or in #41408
Enable non-streaming mode in transformers serve by @LysandreJik in #41446
[device_map] Accelerate loading by computing device_map much faster by @Cyrilvallez in #41548
Add logits_to_keep to many older CausalLM models by @philiproeleveld in #41335
fix some case failures lead by "torch.compile recompiled part of th… by @sywangyi in #41558
remove ray_scope and check_quantized_param by @SunMarc in #41587
Update issue template by @SunMarc in #41573
[Docs] Fix changed references by @vasqu in #41614
Import expand_device_map instead of redefining it by @Cyrilvallez in #41608
Fix trainer simple tests by @SunMarc in #41449
More markdown file fixes by @cyyever in #41599
torch 2.9 don't ❤️ torchcodec 💔 by @ydshieh in #41610
Update a dataset reop link by @ydshieh in #41618
Add fast path for bidirectional mask creation to fix regression by @i3hz in #41586
enable sdpa enable gqa logic for Ascend NPU by @FightingZhen in #41601
Fix video processing channel format by @zucchini-nlp in #41603
[chat template] update when "push_to_hub" by @zucchini-nlp in #39815
Remove the head masking block in some vision models by @ydshieh in #41620
Remove deprecated code by @SunMarc in #41616
Fix quantization base class by @SunMarc in #41613
[docs] Duplicate entry by @stevhliu in #41591
Update executorch.md by @jackzhxng in #41582
Add Backbone API fine-tuning tutorial by @merveenoyan in #41590
🚨 [v5] Toggle the serialization format in processors by @zucchini-nlp in #41474
Add aux loss for GLM-4.5V by @zRzRzRzRzRzRzR in #41564
Allow passing tp_plan in from_pretrained directly by @Cyrilvallez in #41435
Fix tokenization test by @Cyrilvallez in #41649
Remove randomly added script by @Cyrilvallez in #41650
Add missing dates to docs by @yonigozlan in #41576
Migrate transformers cli to Typer by @Wauplin in #41487
Fix FP-Quant quantization fallback CPU dispatch. by @BlackSamorez in #41619
fix check inputs for text2text pipeline by @jiqing-feng in #41556
[Executorch] Simplify for encoder models by @vasqu in #41627
[Ernie 4.5 Moe] Fix Moe and offloading by @vasqu in #41385
[CI] Build translated docs by @stevhliu in #41632
Fix fp32_ln for various models by @remi-or in #41605
Adjust device logging level and add minor fixes by @mario-koddenbrock in #41636
Fix EncoderDecoder cache by @remi-or in #41612
Format MarkDown documentation and tiny fixes by @cyyever in #41638
Fix typos in documentation by @cyyever in #41641
Fix confusing cls assignment by @cyyever in #41642
Double router compute? by @molbap in #41653
[kernels] refactor function kernel calling by @MekkCyber in #41577
[Fix] Deepseek V3 expert bias routing by @fjosw in #41647
purge HF_HUB_ENABLE_HF_TRANSFER; promote Xet by @Vaibhavs10 in #41656
[Masks] Fix mask handling in eager for vision models by @vasqu in #41625
Use | for Optional and Union typing by @cyyever in #41646
Switch to CB if cache_implementation == paged by @remi-or in #41655
Add in-out modalities as class attribute per model by @zucchini-nlp in #41366
Fix dtype casting with quantization by @Cyrilvallez in #41665
Fix serving continuous batching by @SunMarc in #41624
Small changes to benchmarking script by @remi-or in #41662
Improve package version check by @Cyrilvallez in #41661
improve utils/check_bad_commit.py by @ydshieh in #41658
Erroring when KernelConfig is passed without use_kernels = True by @MekkCyber in #41657
[Trainer] [Breaking change] use_cache default to False by @SunMarc in #41585
🌐 [i18n-KO] Translated chat_extras.md to Korean by @Judy-Choi in #39863
🌐 [i18n-KO] Translated sam_hq.md to Korean by @HyunZ118 in #41340
[i18n-KO] Translated big_bird.md to Korean by @ssum21 in #40445
🌐 [i18n-KO] Translated code_llama.md to Korean by @Judy-Choi in #40558
🌐 [i18n-KO] Translated llama4.md to Korean by @TaskerJang in #40396
:globe_with_meridians: [i18n-KO] Translated ko-LFM2.md to Korean by @ssum21 in #41502
Adding superglue fast image processing by @AlphaOrOmega in #41394
Fix ckpt in docs by @zucchini-nlp in #41659
torch 2.9 still don't ❤️ torchcodec 0.8 💔 by @ydshieh in #41686
Remove deprecated use_auth_token parameter by @Wauplin in #41666
Remove require_torch_bf16_gpu by @cyyever in #40979
path validation for security reason by @ydshieh in #41256
🚨 Remove torchscript support by @Cyrilvallez in #41688
Fix MarkDown syntax by @cyyever in #41676
Use | for Optional and Union typing by @cyyever in #41675
🚨 [v5] Refactor RoPE for layer types by @zucchini-nlp in #39847
Enable faiss-cpu on Windows by @cyyever in #41678
Fix Pylint warnings by @cyyever in #41644
🚨 Remove torch.fx support by @Cyrilvallez in #41683
Remove skipped tests without parents by @Cyrilvallez in #41691
Enable FURB rules in ruff by @cyyever in #41395
Remove upper version bound of pandas by @cyyever in #41677
[Attn] Allow dynamic causality in SDPA via Kwargs by @vasqu in #41692
Simplify GQA conditions in sdpa_attention.py by @justinchuby in #41699
[docs] Manual tp-plan by @stevhliu in #41674
🌐 [i18n-KO] Translated gemma3n.md to Korean by @HyunZ118 in #40873
pin torchcodec on CI docker image by @ydshieh in #41703
Update run_name docs in TrainingArguments by @tobiasofsn in #41705
further improve utils/check_bad_commit.py by @ydshieh in #41658)
feat: add benchmark v2 ci with results pushed to dataset by @McPatate in #41672
Gemma3 conversion script maintenance by @RyanMullins in #41704
Fix Qwen3-Omni inference when mixing video and image inputs in one batch by @BakerBunker in #41741
Fix typo in LFM-VL by @zucchini-nlp in #41742
Revert "Remove upper version bound of pandas" by @ydshieh in #41744
[doc] remove broken notebooks on AMD Dev Cloud by @pagezyhf in #41743
Update type hints in tokenization_utils.py to use | syntax by @faizan842 in #41713
Fix documentation issues by @cyyever in #41726
Apply RUFF PIE rules by @cyyever in #41727
Small Fix for imports by @MekkCyber in #41411
Docs(zh-hans): Refine wording for professionalism in README by @Ri-Nai in #40943
Add vision contribution guide by @molbap in #41456
upgrade xpu docker file to torch 2.8 by @yao-matrix in #41551
[v5] Delete videos from image processing classes by @zucchini-nlp in #41607
Fixed incorrect model_type for qwen2vl and qwen2.5vl when config is saved and loaded again by @i3hz in #41758
[kernels] Add version to function mapping by @MekkCyber in #41685
Reduce warning noise caused by Tensor.new_tensor by @st81 in #41748
Fix graphormer model compilation with Cython 3.1.4 by @alexmalyshev in #41671
Update type hints in modeling_rope_utils.py to use | syntax by @faizan842 in #41714
[v5] Remove deprecated tranformers.onnx by @echarlaix in #41700
Modernize CLIP modeling code by @molbap in #41546
Simplify pipeline padding logic by @Rocketknight1 in #41667
Chat response parsing by @Rocketknight1 in #40894
Add LightGlue fast image processor by @yonigozlan in #41670
Fix bark after #41445 by @ydshieh in #41645
Remove invalid @staticmethod from module-level get_device_and_memory_breakdown by @albertvillanova in #41747
Fix CUDA index out of bounds for q_idx in VLM token type masking for Gemma3, PaliGemma, and example modular by @albertvillanova in #41757
fix: Gemma 3 weights conversion vision and multimodal projector paths by @RyanMullins in #41767
[v5] Delete legacy chat template saving by @zucchini-nlp in #41648
[quantization] fix compressed_tensors tests by @MekkCyber in #41780
[quantization] Skip Fp8 tests when hardware capability < 8.9 by @MekkCyber in #41785
Swap columns and rows of the grid layout in LFM2-VL by @ankke in #41755
fix type annotation typo in docstring by @johntheprime in #41788
Fix chat schema tests by @Rocketknight1 in #41793
Fix attention mask in mamba layers by @zucchini-nlp in #41790
[quantization] fix torchao tests after 0.14.0 release by @MekkCyber in #41777
[Onnx docs] Remove some traces by @vasqu in #41791
flash attn pytest marker by @ydshieh in #41781
Bump AMD docker by @remi-or in #41792
make apollo test case pass by @yao-matrix in #41805
Add a safeguard around a flaky test in gemma2 by @remi-or in #41811
Fix Qwen3Next dtype API usage by @SrijanUpadhyay in #41735
[Trainer] remove env vars by @SunMarc in #41697
Fixed grammar mistakes by @FrogWarlord in #41799
Fixed some grammar mistakes by @FrogWarlord in #41802
transformers cli default flag fix by @ArjunPimpale in #41761
Deprecate warmup_ratio by @SunMarc in #41326
transformers serve quantization docs + some api fixes for bitsandbytes by @SunMarc in #41253
[Parakeet] add output_attention_mask by @eustlb in #41694
unpin torch/torchcodec for CircleCI by @ydshieh in #41839
extend bitnet cases to xpu, all 8 cases pass by @yao-matrix in #41831
extend 2 trainer test cases to xpu by @yao-matrix in #41829
extend 2 blip2 and falcon_h1 test cases to xpu by @yao-matrix in #41825
further reducing flakiness in utils/check_bad_commit.py by @ydshieh in #41658)
Remove redundant code from Qwen3VLProcessor by @Xqle in #41836
Fix MXFP4 quantizer to support variable num_local_experts and hidden_size by @marksverdhei in #41795
Fix Qwen2Audio flash attention mask format for generation by @Abdennacer-Badaoui in #41843
Fix const parsing for dict inputs in chat schemas by @Rocketknight1 in #41824
Share embedding modules in BART, not only weights by @githubnemo in #41821
Fix TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path' by @albertvillanova in #41604
:rotating_light: [Clip] Fix masking and enable flash attention on all model types by @vasqu in #41750
CI workflow for Flash Attn by @ydshieh in #41857
Fix torch.no_grad decorator in VLMS by @yaswanth19 in #41888
Fix installation cmds in docs by @yaswanth19 in #41887
revert changes in _is_package_available by @MekkCyber in #41891
make lfm2_moe integration test pass on XPU by @yao-matrix in #41796
Fix: avoid duplicate token in maybe_load_adapters by @luaenrique in #41903
speed up loading checkpoints for zero stage 3 by @ri938 in #41850
evaluate>=0.4.6 is needed by @stas00 in #41920
Add 6 huggingface notebooks on AMD dev cloud by @fan-amd in #41883
Fix invalid examples in QwenVL model docstrings and add Qwen3VL example by @Xqle in #41812
Allow parse_response to accept token IDs by @Rocketknight1 in #41849
Fix Florence2 conversion script model_type KeyError by @i3hz in #41866
Update some workflow files by @ydshieh in #41892
fix some ut failures on XPU w/ torch 2.9 by @yao-matrix in #41923
Cache latest pytorch amd image locally on mi325 CI runner cluster by @jitesh-gupta in #41926
Minor fix in docker image build workflow by @ydshieh in #41949
fix some ut failures on XPU w/ torch 2.9 by @yao-matrix in #41941
Fix rope_parameters for gemma3 weights conversion script by @douglas-reid in #41922
Fix: Gemma3TextConfig rope scaling assignments by @RyanMullins in #41934
fix prepare_config_and_inputs_for_common bug in llava test by @yao-matrix in #41942
Fix: prevent .gitignore truncation in run_clm_no_trainer.py by @luaenrique in #41957
V4.57.1 training ci: Refactor test_tensor_parallel.py by @3outeille in #41918
[v5] Return a BatchEncoding dict from apply_chat_template by default by @Rocketknight1 in #41626
make recurrent_gemma and voxtral cases pass on xpu by @yao-matrix in #41958
Fix typo in image_processing_lfm2_vl_fast by @yonigozlan in #41940
Run slow v2 by @ydshieh in #41914
Fix detectron2 installation in docker files by @ydshieh in #41975
Fix autoawq[kernels] installation in quantization docker file by @ydshieh in #41978
add support for saving encoder only so any parakeet model can be loaded for inference by @nithinraok in #41969
Use indices as position_ids in modernebert by @remi-or in #41789
test tensor parallel: make tests for dense model more robust by @3outeille in #41968
fix: dict[RopeParameters] to dict[str, RopeParameters] by @RyanMullins in #41963
docs: add continuous batching page by @McPatate in #41847
Fix torchcodec version in quantization docker file by @ydshieh in #41988
[kernels] Add Tests & CI for kernels by @MekkCyber in #41765
Move the Mi355 to regular docker by @remi-or in #41989
More data in benchmarking by @remi-or in #41848
fix (CI): Refactor SSH runners by @glegendre01 in #41991
fix 3 failed test cases for video_llama_3 model on Intel XPU by @kaixuanliu in #41931
Integrate colqwen2.5 using colqwen2 modelling code by @sahil-kabir in #40600
Fixed wrong padding value in OWLv2 by @gjamesgoenawan in #41938
Fix run slow v2: empty report when there is only one model by @ydshieh in #42002
[kernels] change import time in KernelConfig by @MekkCyber in #42004
DOC Fix typo in argument name: pseudoquant by @BenjaminBossan in #41994
Fix torch+deepspeed docker file by @ydshieh in #41985
Correct syntax error in trainer.md by @Yacklin in #42001
Reduce the number of benchmark in the CI by @remi-or in #42008
Fix continuous batching tests by @Rocketknight1 in #42012
add back logging_dir by @SunMarc in #42013
Fix issue with from pretrained and kwargs in image processors by @yonigozlan in #41997
Fix default image_rows and image_cols initialization in Idefics3 and SmolVLM processors by @MilkClouds in #41871
Add GLPNImageProcessorFast by @Aravind-11 in #41725
add fuyu fast image processors by @DeXtAr47-oss in #41817
[kernels] Fix XPU layernorm kernel by @MekkCyber in #41583
[v5] Deprecate Text2Text and related pipelines by @Rocketknight1 in #41996
[FPQuant] MXFP8 and MXFP4 backwards support by @BlackSamorez in #41897
fix deeepspeed in AMD docker file by @ydshieh in #42025
CodeQL workflow for security analysis by @paulinebm in #42015
[tests] Add Context-parallel CI tests by @kashif in #41860
extend fp_quant cases to xpu by @yao-matrix in #41833
Change trigger time for AMD CI by @ydshieh in #42034
Fix the order of methods in processor loading by @zucchini-nlp in #42031
🔴 Isolate prefill from generation loops by @manueldeprada in #40652
update huggingface_hub dependency version by @hanouticelina in #42033
Remove some custom datasets defined in codebase by @ydshieh in #41511
Cleanup workflow - part 1 by @ydshieh in #42023
Fix pr_slow_ci_suggestion.yml after #42023 by @ydshieh in #42049
Fix AutoImageProcessor.register and documentation in auto processing modules by @MilkClouds in #41864
Fix Qwen3-Omni RoPE by @zucchini-nlp in #41778
Avoid explicit checkout in workflow by @ydshieh in #42057
Annoying typo in attention error message by @manueldeprada in #42037
Be careful at explicit checkout actions by @ydshieh in #42060
Fix another Argument list too long in pr_slow_ci_suggestion.yml by @ydshieh in #42061
Fix KeyError in GPT-OSS weight conversion script by @Aznix07 in #42007
Fix KeyError in _is_package_available for packages with dotted names by @yashwantbezawada in #42050
Revert back to use GitHub context by @ydshieh in #42066
Fix missing arg in check_docstring by @yonigozlan in #42054
[deepspeed tests fixes] by @stas00 in #41925
Fix logic in setting self.fsdp when it is False by @roychan in #41974
fix tensor device placement issue of 2 UT cases by @yao-matrix in #41921
add workflow to check permissions and advise a set of permissions req… by @paulinebm in #42071
Fix security issue 5 by @paulinebm in #42072
Fix inconsistency of commit sha during the workflow run by @ydshieh in #42074
QwenVL: add skipped keys in setattr as well by @zucchini-nlp in #41808
permissions worflows fix by @paulinebm in #42080
4.1V Model and GLM-4.5V Model Conversion Code Updates by @zRzRzRzRzRzRzR in #41784
feat(ci): add continuous batching to benchmarks by @McPatate in #41916
Fix modular docstring for Mixtral by @diegoakel in #42041
Fix Auto classes to support dynamically registered processors by @MilkClouds in #41865
Reinstate self.scaling in Gemma3nTextAttention by @RyanMullins in #41751
[v5] 🚨Refactor subprocessors handling in processors by @yonigozlan in #41633
add xpu support in test_modeling_janus.py::JanusIntegrationTest::test… by @sywangyi in #41986
Revert "permissions worflows fix" by @ydshieh in #42110
Fix return metadata checking logic by @Xqle in #42108
Correctly handle unbatched audio inputs in Gemma3nAudioFeatureExtractor by @kho in #42076
[Bugfix] fix qwen3vl expand generation with video by @JJJYmmm in #42089
Fix base model prefix in VLMs by @zucchini-nlp in #42059
fix continuous batching issues, extend ut cases to xpu by @yao-matrix in #41830
📝 docs(smolvlm): fix variable name in batch inference example by @gorkachea in #42123
fix qwen2vl/qwen3vl video processor temporal padding when num_frames%temporal_patch_size!=1 by @yaogang2060 in #42083
[Attn Masks] Non-vmap default for attention masks by @vasqu in #41852
Fix GPT-2 Flash Attention 2 generation with left-padding by @Abdennacer-Badaoui in #41966
Fix model name test for compressed tensors by @SunMarc in #42128
Fix MaskFormer/Mask2Former fast image processors by @yonigozlan in #41393
Remove unused functions in image_transforms.py by @yaswanth19 in #42044
update deps table by @ArthurZucker in #42120
fix: improve video processing fps assignment logic by @Xqle in #42009
Fix T5Gemma module structure by @Cyrilvallez in #42145
DataCollatorForLanguageModeling warning error fixed by @mjaliz in #42144
Bugfix/remove emojis from print by @7amim in #42091
Avoid mutating user-provided arguments in preprocessing utils by @LeonardoEmili in #42126
Enforce check_auto_docstring by @yonigozlan in #41635
Add dinov3 autobackbone by @vijayabhaskar-ev in #41276
Fix logic error in prepare_inputs_for_generation cache slicing condition by @albertvillanova in #41764
:rotating_light: Fix gradient checkpointing for several models and improve test robustness by @githubnemo in #41818
[T5Gemma] Fix cross attention cache by @vasqu in #41890
T5 migration to new masking interface by @Aravind-11 in #41804
fix: improve visibility of ValueError root causes in model config loading by @scottzh8 in #41972
add xpu to valid hardware for torch.compile by @sywangyi in #42079
extend test_beam_search_early_stop_heuristic case to other device by @sywangyi in #42078
fix failure of tests/models/shieldgemma2/test_modeling_shieldgemma2.p… by @sywangyi in #42022
Fixes Flash Attention implementation for models by @i3hz in #42149
fix test failure of speculative_generation on xpu by @sywangyi in #42052
add rmsnorm kernels support for npu by @zheliuyu in #42106
update torchao doc by @jiqing-feng in #42139
feat(kernels): add opt-out flag to disable kernels hub usage through the lib by @mfuntowicz in #41990
handle inputs from Siglip/Siglip2 non-automapped encoder layers by @molbap in #41930
Add slow to some examples tests by @SunMarc in #42164
fix(ci): unexpected keyword argument streaming by @McPatate in #42102
pin pytest<9 for now by @ydshieh in #42162
Docs/i18n updates by @lilin-1 in #42006
Fix in-place modification of user-input in SAM2 embed boxes by @xenova in #42173
[Pop2Piano] Fix cache usage by @vasqu in #42170
Fix helper fn for new processor config format by @zucchini-nlp in #42085
Remove unnecessary slicing in sdpa_attention_forward by @justinchuby in #41900
[PEFT] Fix prefix tuning by @vasqu in #41696
[typo] fix mrope-interleave annotation to avoid ambiguity by @JJJYmmm in #42177
Update transformers to support FqnToConfig by @jcaip in #41894
[PEFT] Fix the general test for prefix tuning by @vasqu in #42185
[TP] Fix parameter detection issue and some invalid TP-plans by @Cyrilvallez in #42129
Refactor weight loading by @ArthurZucker in #41580
🚨 Delete deprecations with end-cycle in v4.xx and v5.0 by @zucchini-nlp in #41681
Add AutoTokenizer mapping for mistral3 and ministral by @patrickvonplaten in #42198
Fix checkpoint loading with DeepSpeed ZeRO3 by @tohtana in #42201
[Pop2Piano] Fix tied weights by @vasqu in #42193
New docker from AMD by @remi-or in #42208
Add cross links for model contribution by @zucchini-nlp in #42207
Stop inheriting tests! by @Rocketknight1 in #42192
Refactor check_auto_docstring using AST by @yonigozlan in #41432
[BLT] Fix cache usage by @vasqu in #42188
Update test_dynamic_cache_exportability_multiple_run (failing on torch 2.10 nightly) by @ydshieh in #42212
Much more efficient and clear weight initialization and tie weights by @Cyrilvallez in #42191
GLM-V update with new processor by @zRzRzRzRzRzRzR in #42122
Fix initialization guard for pytest by @Cyrilvallez in #42234
Fix TP plans for MoE models by @Cyrilvallez in #42236
Add prefix sharing to continuous batching by @remi-or in #42094
Loading optimization by @Cyrilvallez in #42239
calls AttentionMaskConverter._unmask_unattended for xpu device before by @kaixuanliu in #42230
FIX Broken PEFT adapter loading by @BenjaminBossan in #42187
Fix processor test for glm by @molbap in #42233
Fix UnboundLocalError in RT-DETR loss computation by @yashwantbezawada in #42224
Stop inheriting tests (again) by @Rocketknight1 in #42247
[loading] Fix device when source and target are different by @Cyrilvallez in #42246
Reduce timing on CircleCI - part 1 (Use @slow for IntegrationTests) by @ydshieh in #42206
🚨 Delete generation params from model config by @zucchini-nlp in #41695
Allow VLMs to have a correct base_model by @zucchini-nlp in #41589
Make tests run in less time by reducing batch_size by @ydshieh in #42213
Revert "Make tests run in less time by reducing batch_size" by @ydshieh in #42258
Cleanup reference to TFBertTokenizer and TFGPT2Tokenizer by @Rocketknight1 in #42182
delete already deprecated models by @ydshieh in #42235
Fix bnb for the weights refactor by @SunMarc in #42043
Fix looping in torch guard decorator by @Cyrilvallez in #42260
🚨 Generalize get_decoder() for multimodal and delete redundant code 🔪 by @zucchini-nlp in #42156
Audio Flamingo3 - fix attention masking by @zucchini-nlp in #42278
Add support for torch device objects in device validator by @yonigozlan in #42267
Remove doc files of other langs for deleted models by @ydshieh in #42276
[testing] fix cwm by @ydshieh in #42261
fix a typo: pbd -> pdb by @jaeminoh in #42268
Enable glm46v UTs on XPU by @YangKai0616 in #42274
[testing] fix some cases in xpu by @sywangyi in #42273
Remove random flag by @Cyrilvallez in #42282
Fix accelerate integration by @Cyrilvallez in #42264
Fix validation checks order in benchmark_v2 by @Abdennacer-Badaoui in #42280
Update torchcodec to match torchaudio version by @remi-or in #42288
Use torch.get_autocast_dtype instead of torch.get_autocast_gpu_dtype by @qgallouedec in #42055
perf: Optimization for Min-p sampling implementation by @casinca in #42248
Fix device_map computation part 2 by @Cyrilvallez in #42290
Fixed the docstring for WhisperFeatureExtractor by @TopCoder2K in #42286
avoiding conditional indexing in positionalencoding to avoid possibil… by @ppadjinTT in #42090
ENH: Add support for LoRA hotswapping by @BenjaminBossan in #41297
Fix Break change of AWQ FusedModules due to Attention Refactor by @fanqiNO1 in #41909
Remove error string test that was failing by @Rocketknight1 in #42301
Properly protect the is_compiling checks by @Cyrilvallez in #42304
Remove outdated methods in modeling_utils.py by @Cyrilvallez in #42302
Fix Mac mps dataloader_num_workers > 1 causes RuntimeError: share_filename: only available on CPU by @AmitMY in #38819
Fix the init_weights for the MoE models by @Cyrilvallez in #42306
Update link to generation strategies documentation by @omkar-334 in #42252
Update conversion mapping to separate renaming from converting by @ArthurZucker in #42254
fix(granitemoe*): Only create block_sparse_moe if num_local_experts > 0 by @gabe-l-hart in #42036
[SAM3 Video] Add support for multi prompts by @yonigozlan in #42293
Add Pix2Struct fast image processor by @yonigozlan in #42020
Fix post processing methods in keypoints matching models by @yonigozlan in #42018
fix tests/models/xcodec/test_modeling_xcodec.py::XcodecIntegrationTest by @sywangyi in #42272
[loading] Fix device detection by @Cyrilvallez in #42323
Fix typo from side_dict to size_dict by @nihui in #42319
HF Trainer: ALST/Ulysses sequence parallelism integration via HF Accelerate by @stas00 in #41832
Fix gpt2 modeling tests by @Abdennacer-Badaoui in #42321
[loading] Use fewer threads by default for much better performances by @Cyrilvallez in #42324
Allow LayoutLMV3Processor to accept rescale_factor by @Rocketknight1 in #42305
Correctly create tied key mapping in post_init, and dynamic tie weight by @Cyrilvallez in #42270
[CI] Skip EfficientLoFTR test by @vasqu in #42327
[XPU] Add flash_attn2 support for XPU by @YangKai0616 in #41956
[Attn Masks] Lift bidirectional mask restriction on eager by @vasqu in #42325
fix bug when gemma3n model run on multiple device by @kaixuanliu in #42303
Fix ChineseCLIPModel.get_text_features by @JiangJQ2000 in #42351
Gemma3 hybrid fix by @remi-or in #42287
fix(benchmarks): correct sdpa_backend inconsistency and attn_implementation for continuous batching by @engmohamedsalah in #42339
Auto convert tekken.json by @ArthurZucker in #42299
[loading] Re-add and improve disk offloading support by @Cyrilvallez in #42242
Fix typo - indentation in JSON dump example by @anthropikos in #42332
Fix tied weight for Bart (for BC) by @Cyrilvallez in #42355
Fix reference to yelp dataset by @JuanFKurucz in #42349
Fix documentation reference to pytorch max memory allocated by @JuanFKurucz in #42350
Fix reference to imagenet 1k dataset by @JuanFKurucz in #42348
Fix typos by @omahs in #42354
Protect torch.distributed imports by @Cyrilvallez in #42361
Expand npu device for KernelConfig by @zheliuyu in #42358
Replace Optional and Union typing with | in some source files by @cyyever in #42294
Fix code examples to load gpt 1 openai community model by @JuanFKurucz in #42347
fix tekken pattern matching by @ArthurZucker in #42363
Fixed-wrong-ZeRO3-json-snippet-found-in-deepspeed-markdown-file by @Yacklin in #42346
Make benchmarking lighter: clean-up result files and remove non-needed arguments by @remi-or in #42357
Add image processor fast vitpose by @yonigozlan in #42021
Small tp fix by @ArthurZucker in #42366
Remove test inheritance for EfficientLoftr, rename KeypointMatchingOutput to model specific name by @yonigozlan in #42365
Tiny doc fix by @molbap in #42296
Fix TimesFM patch normalization instability by @AnMakc in #42099
[core] Fix torchao by @MekkCyber in #42289
Fix tp by @ArthurZucker in #42368
[Attn Masks] Add skip option for non-packed sequences by @vasqu in #42367
📚 docs(granite-speech): add comprehensive usage examples by @gorkachea in #42125
Xcodec fix by @eustlb in #42095
Replace Optional and Union typing with | in some source files by @cyyever in #42372
[Mistral Tokenizers] Fix tokenizer detection by @vasqu in #42389
misc don't recreate it by @ArthurZucker in #42394
[SAM3] Fix precompute vision_embeds or text_embeds for inference by @yonigozlan in #42407
🚨 Image-text pipeline expects correctly formatted chat by @zucchini-nl

Note truncated.

Open source →

4.57.03 Oct 2025withdrawn: Error in the setup causing installation issuesDeprecated

[deprecations] Remove generate-related deprecations up to v4.56 by @gante in #40729

New model additions

Qwen3 Next

The Qwen3-Next series represents the Qwen team's next-generation foundation models, optimized for extreme context length and large-scale parameter efficiency. The series introduces a suite of architectural innovations designed to maximize performance while minimizing computational cost:

Hybrid Attention: Replaces standard attention with the combination of Gated DeltaNet and Gated Attention, enabling efficient context modeling.
High-Sparsity MoE: Achieves an extreme low activation ratio as 1:50 in MoE layers — drastically reducing FLOPs per token while preserving model capacity.
Multi-Token Prediction(MTP): Boosts pretraining model performance, and accelerates inference.
Other Optimizations: Includes techniques such as zero-centered and weight-decayed layernorm, Gated Attention, and other stabilizing enhancements for robust training.

Built on this architecture, they trained and open-sourced Qwen3-Next-80B-A3B — 80B total parameters, only 3B active — achieving extreme sparsity and efficiency.

Despite its ultra-efficiency, it outperforms Qwen3-32B on downstream tasks — while requiring less than 1/10 of the training cost. Moreover, it delivers over 10x higher inference throughput than Qwen3-32B when handling contexts longer than 32K tokens.

For more details, please visit their blog Qwen3-Next (blog post).

Adding Support for Qwen3-Next by @bozheng-hit in #40771

Vault Gemma

VaultGemma is a text-only decoder model derived from Gemma 2, notably it drops the norms after the Attention and MLP blocks, and uses full attention for all layers instead of alternating between full attention and local sliding attention. VaultGemma is available as a pretrained model with 1B parameters that uses a 1024 token sequence length.

VaultGemma was trained from scratch with sequence-level differential privacy (DP). Its training data includes the same mixture as the Gemma 2 models, consisting of a number of documents of varying lengths. Additionally, it is trained using DP stochastic gradient descent (DP-SGD) and provides a (ε ≤ 2.0, δ ≤ 1.1e-10)-sequence-level DP guarantee, where a sequence consists of 1024 consecutive tokens extracted from heterogeneous data sources. Specifically, the privacy unit of the guarantee is for the sequences after sampling and packing of the mixture.

add: differential privacy research model by @RyanMullins in #40851

Qwen3 VL

Qwen3-VL is a multimodal vision-language model series, encompassing both dense and MoE variants, as well as Instruct and Thinking versions.

Building upon its predecessors, Qwen3-VL delivers significant improvements in visual understanding while maintaining strong pure text capabilities. Key architectural advancements include: enhanced MRope with interleaved layout for better spatial-temporal modeling, DeepStack integration to effectively leverage multi-level features from the Vision Transformer (ViT), and improved video understanding through text-based time alignment—evolving from T-RoPE to text timestamp alignment for more precise temporal grounding.

These innovations collectively enable Qwen3-VL to achieve superior performance in complex multimodal tasks.

Adding Support for Qwen3-VL Series by @JJJYmmm in #40795

Longcat Flash

The LongCatFlash model was proposed in LongCat-Flash Technical Report by the Meituan LongCat Team. LongCat-Flash is a 560B parameter Mixture-of-Experts (MoE) model that activates 18.6B-31.3B parameters dynamically (average ~27B). The model features a shortcut-connected architecture enabling high inference speed (>100 tokens/second) and advanced reasoning capabilities.

The abstract from the paper is the following:

We present LongCat-Flash, a 560 billion parameter Mixture-of-Experts (MoE) language model featuring a dynamic computation mechanism that activates 18.6B-31.3B parameters based on context (average ~27B). The model incorporates a shortcut-connected architecture enabling high inference speed (>100 tokens/second) and demonstrates strong performance across multiple benchmarks including 89.71% accuracy on MMLU and exceptional agentic tool use capabilities.

Tips:

LongCat-Flash uses a unique shortcut-connected MoE architecture that enables faster inference compared to traditional MoE models
The model supports up to 128k context length for long-form tasks
Dynamic parameter activation makes it computationally efficient while maintaining high performance
Best suited for applications requiring strong reasoning, coding, and tool-calling capabilities
The MoE architecture includes zero experts (nn.Identity modules) which act as skip connections, allowing tokens to bypass expert computation when appropriate

Add LongCat-Flash by @molbap in #40730

Flex Olmo

FlexOlmo is a new class of language models (LMs) that supports (1) distributed training without data sharing, where different model parameters are independently trained on closed datasets, and (2) data-flexible inference, where these parameters along with their associated data can be flexibly included or excluded from model inferences with no further training. FlexOlmo employs a mixture-of-experts (MoE) architecture where each expert is trained independently on closed datasets and later integrated through a new domain-informed routing without any joint training. FlexOlmo is trained on FlexMix, a corpus we curate comprising publicly available datasets alongside seven domain-specific sets, representing realistic approximations of closed sets.

You can find all the original FlexOlmo checkpoints under the FlexOlmo collection.

Add FlexOlmo model by @2015aroras in #40921

LFM2 VL

LFM2-VL first series of vision-language foundation models developed by Liquid AI. These multimodal models are designed for low-latency and device-aware deployment. LFM2-VL extends the LFM2 family of open-weight Liquid Foundation Models (LFMs) into the vision-language space, supporting both text and image inputs with variable resolutions.

Architecture

LFM2-VL consists of three main components: a language model backbone, a vision encoder, and a multimodal projector. LFM2-VL builds upon the LFM2 backbone, inheriting from either LFM2-1.2B (for LFM2-VL-1.6B) or LFM2-350M (for LFM2-VL-450M). For the vision tower, LFM2-VL uses SigLIP2 NaFlex encoders to convert input images into token sequences. Two variants are implemented:

Shape-optimized (400M) for more fine-grained vision capabilities for LFM2-VL-1.6B
Base (86M) for fast image processing for LFM2-VL-450M

The encoder processes images at their native resolution up to 512×512 pixels, efficiently handling smaller images without upscaling and supporting non-standard aspect ratios without distortion. Larger images are split into non-overlapping square patches of 512×512 each, preserving detail. In LFM2-VL-1.6B, the model also receives a thumbnail (a small, downscaled version of the original image capturing the overall scene) to enhance global context understanding and alignment. Special tokens mark each patch’s position and indicate the thumbnail’s start. The multimodal connector is a 2-layer MLP connector with pixel unshuffle to reduce image token count.

Add new model LFM2-VL by @zucchini-nlp in #40624

BLT

The BLT model was proposed in Byte Latent Transformer: Patches Scale Better Than Tokens by Artidoro Pagnoni, Ram Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li1, Chunting Zhou, Lili Yu, Jason Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman†, Srinivasan Iyer. BLT is a byte-level LLM that achieves tokenization-level performance through entropy-based dynamic patching.

The abstract from the paper is the following:

We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency and robustness. BLT encodes bytes into dynamically sized patches, which serve as the primary units of computation. Patches are segmented based on the entropy of the next byte, allocating more compute and model capacity where increased data complexity demands it. We present the first flop controlled scaling study of byte-level models up to 8B parameters and 4T training bytes. Our results demonstrate the feasibility of scaling models trained on raw bytes without a fixed vocabulary. Both training and inference efficiency improve due to dynamically selecting long patches when data is predictable, along with qualitative improvements on reasoning and long tail generalization. Overall, for fixed inference costs, BLT shows significantly better scaling than tokenization-based models, by simultaneously growing both patch and model size.

Usage Tips:

Dual Model Architecture: BLT consists of two separate trained models:
- Patcher (Entropy Model): A smaller transformer model that predicts byte-level entropy to determine patch boundaries and segment input.
- Main Transformer Model: The primary model that processes the patches through a Local Encoder, Global Transformer, and Local Decoder.
Dynamic Patching: The model uses entropy-based dynamic patching where:
- High-entropy regions (complex data) get shorter patches with more computational attention
- Low-entropy regions (predictable data) get longer patches for efficiency
- This allows the model to allocate compute resources where they're most needed
Local Encoder: Processes byte sequences with cross-attention to patch embeddings
Global Transformer: Processes patch-level representations with full attention across patches
Local Decoder: Generates output with cross-attention back to the original byte sequence
Byte-Level Tokenizer: Unlike traditional tokenizers that use learned vocabularies, BLT's tokenizer simply converts text to UTF-8 bytes and maps each byte to a token ID. There is no need for a vocabulary.

blt wip by @itazap in #38579

Qwen3 Omni MoE

The Qwen2.5-Omni model is a unified multiple modalities model proposed in Qwen2.5-Omni Technical Report from Qwen team, Alibaba Group.

Notes

Use [Qwen2_5OmniForConditionalGeneration] to generate audio and text output. To generate only one output type, use [Qwen2_5OmniThinkerForConditionalGeneration] for text-only and [Qwen2_5OmniTalkersForConditionalGeneration] for audio-only outputs.
Audio generation with [Qwen2_5OmniForConditionalGeneration] supports only single batch size at the moment.
In case out out-of-memory errors hwen working with video input, decrease processor.max_pixels. By default the maximum is set to a very arge value and high resolution visuals will not be resized, unless resolution exceeds processor.max_pixels.
The processor has its own [~ProcessorMixin.apply_chat_template] method to convert chat messages to model inputs.

Adding support for Qwen3Omni by @BakerBunker in #41025

Parakeet

Parakeet models, introduced by NVIDIA NeMo, are models that combine a Fast Conformer encoder with connectionist temporal classification (CTC), recurrent neural network transducer (RNNT) or token and duration transducer (TDT) decoder for automatic speech recognition.

Model Architecture

Fast Conformer Encoder: A linearly scalable Conformer architecture that processes mel-spectrogram features and reduces sequence length through subsampling. This is more efficient version of the Conformer Encoder found in FastSpeech2Conformer (see [ParakeetEncoder] for the encoder implementation and details).
ParakeetForCTC: a Fast Conformer Encoder + a CTC decoder
- CTC Decoder: Simple but effective decoder consisting of:
  - 1D convolution projection from encoder hidden size to vocabulary size (for optimal NeMo compatibility).
  - CTC loss computation for training.
  - Greedy CTC decoding for inference.

Add Parakeet by @nithinraok in #39062

EdgeTAM

The EdgeTAM model was proposed in EdgeTAM: On-Device Track Anything Model Chong Zhou, Chenchen Zhu, Yunyang Xiong, Saksham Suri, Fanyi Xiao, Lemeng Wu, Raghuraman Krishnamoorthi, Bo Dai, Chen Change Loy, Vikas Chandra, Bilge Soran.

EdgeTAM is an efficient adaptation of SAM 2 that introduces a 2D Spatial Perceiver architecture to optimize memory attention mechanisms for real-time video segmentation on mobile devices.

Add EdgeTAM by @yonigozlan in #39800

OLMO3

More details to come soon :eyes:

Add Olmo3 model by @2015aroras in #40778

Continuous batching

We are introducing Continuous Batching (CB) in this release, we consider it a stable feature. The main use case for CB is batched generation, which makes it very efficient in the context of GRPO training or evaluation. Thanks to CB, researchers or model developers are now free to use transformers in these contexts without having to spin up an additional inference engine.

CB currently supports both full attention and sliding window attention: this means that the vast majority of models are supported, like llama, gemma3, gpt-oss.

CB is also integrated with transformers serve, which means that you can deploy transformers as an OpenAI-compatible HTTP server. Here is a small snippet on how to use it:

import datasets
import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", dtype=torch.bfloat16, _attn_implementation="sdpa_paged", device_map="auto"
)
model.generation_config.max_new_tokens = 32
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", padding_side="left")
dataset = datasets.load_dataset("openai/gsm8k", "socratic", split="test")
tokenized_datasets = dataset.map(lambda x: tokenizer(x["question"]), batched=True)
simple_batch_inputs = [item["input_ids"] for item in tokenized_datasets]

batch_outputs = model.generate_batch(inputs=simple_batch_inputs)
for request in batch_outputs:
    print(tokenizer.decode(batch_outputs[request].generated_tokens))
"""
 Let's break down the problem step by step:

1. **Total eggs laid per day**:  
   Janet’s ducks lay **16 eggs per day**
 Let's break down the problem step by step:

1. **Blue fiber**: The robe takes **2 bolts** of blue fiber.
2. **White fiber
 To determine Josh's profit from flipping the house, let's go step by step.

---

### Step 1: Initial cost of the house
Josh buys the
 To find the total distance James runs in a week, we can break down the problem step by step:

1. **Sprints per session**: James runs 
 To determine how many cups of feed Wendi needs to give her chickens in the final meal of the day, let's go step by step.
"""

Breaking changes

🚨 Remove Group Beam Search decoding strategy by @manueldeprada in #40495
🚨 Remove Constrained Beam Search decoding strategy by @manueldeprada in #40518
🚨 Allow check_model_inputs in core VLMs by @zucchini-nlp in #40342
🔴 Update Glm4V to use config values by @zucchini-nlp in #40712
🚨 Fix Inconsistant input_feature length and attention_mask length in WhisperFeatureExtractor by @BakerBunker in #39221
⚠️ 🔴 Add ministral model by @manueldeprada in #40247
🔴 Move variable output controls to _prepare_generation_config by @manueldeprada in #40715
🔴Make center_crop fast equivalent to slow by @yonigozlan in #40856

Bugfixes and improvements

Fix collated reports upload filename by @ivarflakstad in #40556
pin pytest-rerunfailures<16.0 by @ydshieh in #40561
remove the redundant non maintained jieba and use rjieba instead by @divyanshsinghvi in #40383
Set test_all_params_have_gradient=False for DeepseekV2ModelTest by @ydshieh in #40566
processor tests - use dummy videos by @zucchini-nlp in #40537
[qwen-vl] fix position ids by @zucchini-nlp in #40490
Fix test_eager_matches_sdpa_inference not run for CLIP by @ydshieh in #40581
Fix CircleCI step passes in the case of pytest worker crash at test collection time by @ydshieh in #40552
Allow remi-or to run-slow by @ydshieh in #40590
Fix llava image processor by @zucchini-nlp in #40588
Update get_*_features methods + update doc snippets by @qubvel in #40555
Fix custom generate relative imports by @manueldeprada in #40480
Support batch size > 1 image-text inference by @hiyouga in #36682
Fix typos by @cyyever in #40585
Skip TvpImageProcessingTest::test_slow_fast_equivalence by @ydshieh in #40593
Fix inexistent imports by @cyyever in #40580
Add Copilot instructions by @Rocketknight1 in #40432
Fix siglip flaky test_eager_matches_sdpa_inference by @ydshieh in #40584
Fix for missing default values in encoder decoder by @remi-or in #40517
Fix quite a lot of FA tests by @Cyrilvallez in #40548
[Tests] Fixup duplicated mrope logic by @vasqu in #40592
Reduce more test data fetch by @ydshieh in #40595
Pin torchcodec to 0.5 in AMD docker by @remi-or in #40598
Multiple fixes to FA tests in AMD by @remi-or in #40498
Disable cache for TokenizerTesterMixin temporarily by @ydshieh in #40611
fix: continuous batching in transformers serve by @McPatate in #40479
Fix processor chat template by @zucchini-nlp in #40613
Avoid too many request caused by AutoModelTest::test_dynamic_saving_from_local_repo by @ydshieh in #40614
Fix flaky JambaModelTest.test_load_balancing_loss by @ydshieh in #40617
Add collated reports job to Nvidia CI by @ahadnagy in #40470
Remove unnecessary pillow version check by @cyyever in #40604
Fix invalid typing by @cyyever in #40612
Enable more ruff UP rules by @cyyever in #40579
Support TF32 flag for MUSA backend by @fmo-mt in #33187
Remove random flag by @Cyrilvallez in #40629
🌐 [i18n-KO] Translated deepseek_v3.md to Korean by @ssum21 in #39649
Fix too many requests in TestMistralCommonTokenizer by @ydshieh in #40623
fix: gas for gemma fixed by @yevvonlim in #40591
[auto-model] propagate kwargs by @zucchini-nlp in #40491
[CP] Add attention_mask to the buffer when the mask is causal by @kashif in #40619
Fix: PIL image load in Processing utils apply_chat_template by @abdokaseb in #40622
Skip test_prompt_lookup_decoding_matches_greedy_search for voxtral by @ydshieh in #40643
add DeepseekV3ForTokenClassification by @bzantium in #40641
fix MetaCLIP 2 wrong link & wrong model names in the docstrings by @voidism in #40565
Remove TF/Flax examples by @Rocketknight1 in #40654
Mark LongformerModelTest::test_attention_outputs as flaky by @ydshieh in #40655
fix pipeline dtype by @jiqing-feng in #40638
feat(serving): add healthcheck by @McPatate in #40653
Fix Metaclip modular conversion by @Rocketknight1 in #40660
Avoid attention_mask copy in qwen2.5 by @cyyever in #40658
Allow custom args in custom_generate Callables and unify generation args structure by @manueldeprada in #40586
Update check_determinism inside test_determinism by @ydshieh in #40661
Skip test_fast_is_faster_than_slow for Owlv2ImageProcessingTest by @ydshieh in #40663
Fix warning for output_attentions=True by @qubvel in #40597
Skip test_prompt_lookup_decoding_matches_greedy_search for qwen2_audio by @ydshieh in #40664
Remove overwritten GitModelTest::test_beam_search_generate by @ydshieh in #40666
refactor: use tolist instead of list comprehension calling .item() by @McPatate in #40646
Benchmarking V2: framework impl by @ahadnagy in #40486
Even more test data cached by @ydshieh in #40636
Skip more fast v.s slow image processor tests by @ydshieh in #40675
Avoid night torch CI not run because of irrelevant docker image failing to build by @ydshieh in #40677
Mark Aimv2ModelTest::test_eager_matches_sdpa_inference_04_fp16_pad_right_sdpa_kernels as flaky by @ydshieh in #40683
CircleCI docker images cleanup / update / fix by @ydshieh in #40681
Add sequence classification support for small Gemma 3 text models by @abdokaseb in #40562
Add codebook_dim attribute to DacVectorQuantize for DacResidualVectorQuantize.from_latents() by @flavioialongo in #40665
fix broken offline mode when loading tokenizer from hub by @winglian in #40669
Load a tiny video to make CI faster by @zucchini-nlp in #40684
Final test data cache - inside CI docker images by @ydshieh in #40689
add: embedding model by @RyanMullins in #40694
feat: support request cancellation by @McPatate in #40599
Fixing bug in Voxtral when merging text and audio embeddings by @rcogill in #40671
Change docker image to preview for the MI355 CI by @ahadnagy in #40693
Fix backward compatibility with accelerate in Trainer by @qgallouedec in #40668
Fix self.dropout_p is not defined for SamAttention/Sam2Attention by @yonigozlan in #40667
[Glm4.5V] fix vLLM support by @zucchini-nlp in #40696
Fix broken Llama4 accuracy in MoE part by @nvpohanh in #40609
Avoid T5GemmaModelTest::test_eager_matches_sdpa_inference being flaky by @ydshieh in #40702
Align assisted generate for unified signature in decoding methods by @manueldeprada in #40657
Fetch one missing test data by @ydshieh in #40703
Add Fast Image Processor for ImageGPT by @agamjots05 in #39592
Fetch more test data with hf_hub_download by @ydshieh in #40710
feat(serve): add healthcheck test by @McPatate in #40697
Fix parent classes of ProcessingKwargs by @cyyever in #40676
[tests] fix blip2 edge case by @gante in #40699
[moduar] Add missing self in post-process methods by @framonmar7 in #40711
[onnx] use logical or for grounding dino mask by @lmarshall12 in #40625
Fix parent classes of AllKwargsForChatTemplate by @cyyever in #40685
Fix arguments by @cyyever in #40605
[serve] re-enable tests by @gante in #40717
[tests] remove overwrites of removed test by @gante in #40720
Add Optional typing by @cyyever in #40686
[Gemma Embedding] Fix SWA by @vasqu in #40700
Keypoint matching docs by @merveenoyan in #40541
Skip VitMatteImageProcessingTest::test_fast_is_faster_than_slow by @ydshieh in #40713
refactor(serve): move request_id to headers by @McPatate in #40722
[Continous Batching] fix do_Sample=True in continuous batching by @kashif in #40692
Fix order of mask functions when using and/or_mask_function by @Cyrilvallez in #40753
Fix np array typing by @cyyever in #40741
Set accepts_loss_kwargs to False for ConvNext(|V2)ForImageClassification by @clinty in #40746
Add BF16 support check for MUSA backend by @fmo-mt in #40576
remove gemmas eager training warning by @August-murr in #40744
remove FSDP prefix when using save_pretrained with FSDP2 by @winglian in #40207
feat: err when unsupported attn impl is set w/ --continuous_batching by @McPatate in #40618
docs: add continuous batching to serving by @McPatate in #40758
Remove unnecessary tildes from documentation by @st81 in #40748
Fix more typos by @cyyever in #40627
Fix inconsistency in SeamlessM4T and SeamlessM4Tv2 docs by @clinty in #39364
Fix continue_final_message in apply_chat_template to prevent substring matching issues by @abdokaseb in #40732
🌐 [i18n-KO] Translated 'xclip.md' to Korean by @ssum21 in #39594
Fix Bark failing tests by @ebezzam in #39478
Add EfficientLoFTRImageProcessorFast for GPU-accelerated image processing by @LawJarp-A in #40215
Fix: swanlab public.cloud.experiment_url api error by @Zeyi-Lin in #40763
[generate] PromptLookupCandidateGenerator won't generate forbidden tokens by @gante in #40726
Support sliding window in CB by @remi-or in #40688
[deprecations] Remove generate-related deprecations up to v4.56 by @gante in #40729
rm src/transformers/convert_pytorch_checkpoint_to_tf2.py by @gante in #40718
[tests] update test_past_key_values_format and delete overwrites by @gante in #40701
[RoPE] run RoPE tests when the model uses RoPE by @gante in #40630
Fix crash when executing MambaCache sample code by @torotoki in #40557
[pipeline] ASR pipeline kwargs are forwared to generate by @gante in #40375
[docs] CPU install by @stevhliu in #40631
Adding Support for Qwen3-Next by @bozheng-hit in #40771
Fix gpt-oss router_indices in EP by @jiqing-feng in #40545
Remove reference of video_load_backend and video_fps for processor by @cyyever in #40719
[processors] Unbloating simple processors by @zucchini-nlp in #40377
Enable ruff on benchmark and scripts by @cyyever in #40634
Fix doc for PerceptionLMForConditionalGeneration forward. by @shuminghu in #40733
Fix typos in tests and util by @cyyever in #40780
Fix invalid PipelineParallel member by @cyyever in #40789
Use functools.cached_property by @cyyever in #40607
Read config pattern for Qwen3Next by @Cyrilvallez in #40792
Fix dotted model names by @August-murr in #40745
Fix the issue that csm model cannot work with pipeline mode. by @yuanwu2017 in #39349
Move num_items_in_batch to correct device before accelerator.gather by @ssharpe42 in #40773
Remove use_ipex option from Trainer by @cyyever in #40784
fix_image_processing_fast_for_glm4v by @lambertwjh in #40483
[Docs] Add missing class documentation for optimizer_schedules by @jijihuny in #31870, #23010)
Fix DeepSpeed mixed precision precedence over Accelerate defaults by @notkisk in #39856
feature: Add robust token counting with padding exclusion by @PrathmeshAdsod in #40416
Fix edge case for tokenize by @wangzhen0518 in #36277)
Fix config dtype parsing for Emu3 edge case by @Isotr0py in #40766
Align torch implementation of Gated DeltaNet in Qwen3-Next with fla library. by @bozheng-hit in #40807
Fix typos in src by @cyyever in #40782
add general hub test for Fast Image Processors in test_image_processing_utils by @namgyu-youn in #40086
Push generation config along with checkpoints by @qgallouedec in #40804
[Jetmoe] Fix RoPE by @vasqu in #40819
🌐 [i18n-KO] Translated clipseg.md to Korean by @HyunZ118 in #39903
Improve torch_dtype checks by @cyyever in #40808
Add VideoProcessors to auto-backend requirements by @Cyrilvallez in #40843
Adds Causal Conv 1D kernel for mamba models by @MekkCyber in #40765
Update no split modules in T5Gemma model by @npuichigo in #40810
Replace image classification loss functions to self.loss_function by @qubvel in #40764
Fix the misalignment between the l2norm in GDN of Qwen3-Next and the implementation in the FLA library. by @bozheng-hit in #40842
Fixes for continuous batching by @remi-or in #40828
[tests] re-enable aria fast tests by @gante in #40846
[SAM2] Fix inconsistent results with original implementation with input boxes by @yonigozlan in #40800
[Sam2Video] Fix video inference with batched boxes and add test by @yonigozlan in #40797
add: differential privacy research model by @RyanMullins in #40851
[test] Fix test_eager_matches_sdpa incorrectly skipped by @eustlb in #40852
[tests] move generative tests away from test_modeling_common.py by @gante in #40854
[generate] Always use decoder config to init cache by @gante in #40772
Use checkpoint in auto_class_docstring by @cyyever in #40844
Fix TrainingArguments.parallelism_config NameError with accelerate<1.10.1 by @albertvillanova in #40818
Redirect MI355 CI results to dummy dataset by @ahadnagy in #40862
[Bug fix #40813] Fix base_model_tp_plan of Starcoder2 model. by @greg-kwasniewski1 in #40814
[docstrings / type hints] Update outdated annotations for past_key_values by @gante in #40803
fix florence kwargs by @SunMarc in #40826
fix: XIELU act parameters not being casted to correct dtype by @NanoCode012 in #40812
Update model tags and integration references in bug report by @ArthurZucker in #40881
[Qwen3 Next] Use numerically stable rsqrt by @thalahors in #40848
Adding Support for Qwen3-VL Series by @JJJYmmm in #40795
[VaultGemma] Update expectations in integration tests by @vasqu in #40855
Fix modular consistency by @Cyrilvallez in #40883
Clarify passing is_causal in sdpa_attention_paged_forward by @cyyever in #40838
Use torch.expm1 and torch.log1p for better numerical results by @cyyever in #40860
Add Fast PromptDepthAnything Processor by @SamuelBarryCS in #40602
Fix deta loading & dataclass by @Cyrilvallez in #40878
Remove dict branch of attention_mask in sdpa_attention_paged_forward by @cyyever in #40882
🌐 [i18n-KO] Translated smolvlm.md to Korean by @HyunZ118 in #40414
🌐 [i18n-KO] Translated imageprocessor.md to Korean by @HyunZ118 in #39557
[generate] remove docs of a feature that no longer exists by @gante in #40895
Make debugging failing tests (check and update expect output values) easier 🔥 by @ydshieh in #40727
Fixing the call to kernelize by @MekkCyber in #40628
Fix getter regression by @molbap in #40824
Fix flaky Gemma3nAudioFeatureExtractionTest::test_dither by @ydshieh in #40902
[cache] Merge static sliding and static chunked layer by @Cyrilvallez in #40893
Harmonize CacheLayer names by @Cyrilvallez in #40892
[cache] Only use scalars in get_mask_sizes by @Cyrilvallez in #40907
Set seed for Glm4vIntegrationTest by @ydshieh in #40905
Add Olmo3 model by @2015aroras in #40778
remove dummy EncodingFast by @cyyever in #40864
Improve module name handling for local custom code by @XuehaiPan in #40809
Remove runner_map by @ydshieh in #40880
disable test_fast_is_faster_than_slow by @ydshieh in #40909
[gemma3] Gemma3ForConditionalGeneration compatible with assisted generation by @gante in #40791
[generate] misc fixes by @gante in #40906
Fix dtype in Paligemma by @zucchini-nlp in #40912
[Docs] Adding documentation of MXFP4 Quantization by @ariG23498 in #40885
Processor load with multi-processing by @zucchini-nlp in #40786
[Llama4] Remove image_sizes arg and deprecate vision_feature_layer by @yaswanth19 in #40832
Fix #40067: Add dedicated UMT5 support to GGUF loader (config, tokenizer, test) by @akshay-babbar in #40218
[torchao safetensors] renaming get_state_dict function by @liangel-02 in #40774
Adding activation kernels by @MekkCyber in #40890
Minor fix for #40727 by @ydshieh in #40929
Add support for Florence-2 training by @ducviet00 in #40914
Add LongCat-Flash by @molbap in #40730
[DOC] Add missing dates in model cards by @yonigozlan in #40922
[models] remove unused import torch.utils.checkpoint by @gante in #40934
Intel CPU dockerfile by @jiqing-feng in #40806
docs(i18n): Correct the descriptive text in the README_zh-hans.md by @lilin-1 in #40941
Fix trainer tests by @SunMarc in #40823
Fix Glm4vMoeIntegrationTest by @ydshieh in #40930
Raise error instead of warning when using meta device in from_pretrained by @Cyrilvallez in #40942
Consistent naming for images kwargs by @zucchini-nlp in #40834
Remove nested import logic for torchvision by @yonigozlan in #40940
Fix Glm4vModelTest::test_eager_matches_fa2_generate by @ydshieh in #40947
Update expected values for some test_speculative_generation by @ydshieh in #40949
Standardize audio embedding function name for audio multimodal models by @jackzhxng in #40919
Add FlexOlmo model by @2015aroras in #40921
Don't list dropout in eager_paged_attention_forward by @cyyever in #40924

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@hiyouga
- Support batch size > 1 image-text inference (#36682)
@cyyever
- Fix typos (#40585)
- Fix inexistent imports (#40580)
- Remove unnecessary pillow version check (#40604)
- Fix invalid typing (#40612)
- Enable more ruff UP rules (#40579)
- Avoid attention_mask copy in qwen2.5 (#40658)
- Fix parent classes of ProcessingKwargs (#40676)
- Fix parent classes of AllKwargsForChatTemplate (#40685)
- Fix arguments (#40605)
- Add Optional typing (#40686)
- Fix np array typing (#40741)
- Fix more typos (#40627)
- Remove reference of video_load_backend and video_fps for processor (#40719)
- Enable ruff on benchmark and scripts (#40634)
- Fix typos in tests and util (#40780)
- Fix invalid PipelineParallel member (#40789)
- Use functools.cached_property (#40607)
- Remove use_ipex option from Trainer (#40784)
- Fix typos in src (#40782)
- Improve torch_dtype checks (#40808)
- Use checkpoint in auto_class_docstring (#40844)
- Clarify passing is_causal in sdpa_attention_paged_forward (#40838)
- Use torch.expm1 and torch.log1p for better numerical results (#40860)
- Remove dict branch of attention_mask in sdpa_attention_paged_forward (#40882)
- remove dummy EncodingFast (#40864)
- Don't list dropout in eager_paged_attention_forward (#40924)
- Benchmarking V2: framework impl (#40486)
- Change docker image to preview for the MI355 CI (#40693)
- Redirect MI355 CI results to dummy dataset (#40862)
@voidism
- fix MetaCLIP 2 wrong link & wrong model names in the docstrings (#40565)
@RyanMullins
- add: embedding model (#40694)
- add: differential privacy research model (#40851)
@LawJarp-A
- Add EfficientLoFTRImageProcessorFast for GPU-accelerated image processing (#40215)
@bozheng-hit
- Adding Support for Qwen3-Next (#40771)
- Align torch implementation of Gated DeltaNet in Qwen3-Next with fla library. (#40807)
- Fix the misalignment between the l2norm in GDN of Qwen3-Next and the implementation in the FLA library. (#40842)
@wangzhen0518
- Fix edge case for tokenize (#36277) (#36555)
@HyunZ118
- 🌐 [i18n-KO] Translated clipseg.md to Korean (#39903)
- 🌐 [i18n-KO] Translated smolvlm.md to Korean (#40414)
- 🌐 [i18n-KO] Translated imageprocessor.md to Korean (#39557)
@JJJYmmm
- Adding Support for Qwen3-VL Series (#40795)
@SamuelBarryCS
- Add Fast PromptDepthAnything Processor (#40602)
@2015aroras
- Add Olmo3 model (#40778)
- Add FlexOlmo model (#40921)

Open source →

4.56.029 Aug 2025Breaking

The following commits are breaking changes in workflows that were either buggy or not working as expected.

New model additions

Dino v3

DINOv3 is a family of versatile vision foundation models that outperforms the specialized state of the art across a broad range of settings, without fine-tuning. DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks, significantly surpassing previous self- and weakly-supervised foundation models.

You can find all the original DINOv3 checkpoints under the DINOv3 collection.

Add Dino v3 by @qubvel in #40167

X-Codec

he X-Codec model was proposed in Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model by Zhen Ye, Peiwen Sun, Jiahe Lei, Hongzhan Lin, Xu Tan, Zheqi Dai, Qiuqiang Kong, Jianyi Chen, Jiahao Pan, Qifeng Liu, Yike Guo, Wei Xue

The X-Codec model is a neural audio codec that integrates semantic information from self-supervised models (e.g., HuBERT) alongside traditional acoustic information. This enables :

Music continuation : Better modeling of musical semantics yields more coherent continuations.
Text-to-Sound Synthesis : X-Codec captures semantic alignment between text prompts and generated audio.
Semantic aware audio tokenization: X-Codec is used as an audio tokenizer in the YuE lyrics to song generation model.

Add X-Codec model by @Manalelaidouni in #38248

Ovis 2

The Ovis2 is an updated version of the Ovis model developed by the AIDC-AI team at Alibaba International Digital Commerce Group.

Ovis2 is the latest advancement in multi-modal large language models (MLLMs), succeeding Ovis1.6. It retains the architectural design of the Ovis series, which focuses on aligning visual and textual embeddings, and introduces major improvements in data curation and training methods.

Add Ovis2 model and processor implementation by @thisisiron in #37088

MetaCLIP 2

MetaCLIP 2 is a replication of the original CLIP model trained on 300+ languages. It achieves state-of-the-art (SOTA) results on multilingual benchmarks (e.g., XM3600, CVQA, Babel‑ImageNet), surpassing previous SOTA such as mSigLIP and SigLIP‑2. The authors show that English and non-English worlds can mutually benefit and elevate each other.

Add MetaCLIP 2 by @NielsRogge in #39826

Florence 2

Florence-2 is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of vision and vision-language tasks. Florence-2 can interpret simple text prompts to perform tasks like captioning, object detection, and segmentation. It leverages the FLD-5B dataset, containing 5.4 billion annotations across 126 million images, to master multi-task learning. The model's sequence-to-sequence architecture enables it to excel in both zero-shot and fine-tuned settings, proving to be a competitive vision foundation model.

Add support for Florence-2 by @ducviet00 in #38188

SAM 2

SAM2 (Segment Anything Model 2) was proposed in Segment Anything in Images and Videos by Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, Christoph Feichtenhofer.

The model can be used to predict segmentation masks of any object of interest given an input image or video, and input points or bounding boxes.

Add Segment Anything 2 (SAM2) by @SangbumChoi in #32317

Kosmos 2.5

The Kosmos-2.5 model was proposed in KOSMOS-2.5: A Multimodal Literate Model by Microsoft.

The abstract from the paper is the following:

We present Kosmos-2.5, a multimodal literate model for machine reading of text-intensive images. Pre-trained on large-scale text-intensive images, Kosmos-2.5 excels in two distinct yet cooperative transcription tasks: (1) generating spatially-aware text blocks, where each block of text is assigned its spatial coordinates within the image, and (2) producing structured text output that captures styles and structures into the markdown format. This unified multimodal literate capability is achieved through a shared Transformer architecture, task-specific prompts, and flexible text representations. We evaluate Kosmos-2.5 on end-to-end document-level text recognition and image-to-markdown text generation. Furthermore, the model can be readily adapted for any text-intensive image understanding task with different prompts through supervised fine-tuning, making it a general-purpose tool for real-world applications involving text-rich images. This work also paves the way for the future scaling of multimodal large language models.

Add Kosmos-2.5 by @tic-top in #31711

HunYuan

More information at release 🤗

HunYuan opensource by @yjc9696 in #39606

Seed OSS

More information at release 🤗

Adding ByteDance Seed Seed-OSS by @Fazziekey in #40272

GLM-4.5V

More information at release 🤗

GLM-4.5V Model Support by @zRzRzRzRzRzRzR in #39805

Cache

Beyond a large refactor of the caching system in Transformers, making it much more practical and general, models using sliding window attention/chunk attention do not waste memory anymore when caching past states. It was allowed most notable by:

New DynamicSlidingWindowLayer & associated Cache by @Cyrilvallez in #40039

See the following improvements on memory usage for Mistral (using only sliding layers) and GPT-OSS (1 out of 2 layers is sliding) respectively: <img width="569" height="431" alt="image" src="https://github.com/user-attachments/assets/7f1688f4-b077-4840-a62c-bfa6131fe806" /> <img width="574" height="431" alt="image" src="https://github.com/user-attachments/assets/bb4a284f-961e-413d-b7e1-783bb5d8fb39" />

Beyond memory usage, it will also improve generation/forward speed by a large margin for large contexts, as only necessary states are passed to the attention computation, which is very sensitive to the sequence length.

Quantization

MXFP4

Since the GPT-OSS release which introduced the MXPF4 quantization type, several improvements have been made to the support, which should now stabilize.

Fix MXFP4 quantizer validation to allow CPU inference with dequantize option by @returnL in #39953
Enable gpt-oss mxfp4 on older hardware (sm75+) by @matthewdouglas in #39940
Fix typo and improve GPU kernel check error message in MXFP4 quantization by @akintunero in #40349)
Default to dequantize if cpu in device_map for mxfp4 by @MekkCyber in #39993
Fix GPT-OSS swiglu_limit not passed in for MXFP4 by @danielhanchen in #40197
[Mxfp4] Add a way to save with a quantization method by @ArthurZucker in #40176

New standard

Now that we deprecated tensorflow and jax, we felt that torch_dtype was not only misaligned with torch, but was redundant and hard to remember. For this reason, we switched to a much more standard dtype argument!

⚠️⚠️ Use dtype instead of torch_dtype everywhere! by @Cyrilvallez in #39782

torch_dtype will still be a valid usage for as long as needed to ensure a smooth transition, but new code should use dtype, and we encourage you to update older code as well!

Breaking changes

The following commits are breaking changes in workflows that were either buggy or not working as expected.

Saner hub-defaults for hybrid cache implementation

On models where the hub checkpoint specifies cache_implementation="hybrid" (static sliding window hybrid cache), UNSETS this value. This will make the model use the dynamic sliding window layers by default.

This default meant that there were widespread super slow 1st generate calls on models with hybrid caches, which should nol onger be the case.

🚨🚨 [generate] ignore cache_implementation="hybrid" hub defaults by @gante in #40135

Sine positional embeddings for MaskFormer & LRU cache

Cache the computation of sine positional embeddings for MaskFormer; results in a 6% performance improvement.

🚨 Use lru_cache for sine pos embeddings MaskFormer by @yonigozlan in #40007

Explicit cache initialization

Adds explicit cache initialization to prepare for the deprecation of the from_legacy_cache utility.

🚨 Always return Cache objects in modelings (to align with generate) by @manueldeprada in #39765

Default compilation with `fullgraph=False`

Having fullgraph set to True during compilation ended up being very restrictive, especially with the arrival of widely-used MoEs.

🚨🚨 Switch default compilation to fullgraph=False by @Cyrilvallez in #40137

Remove decoding strategies

The DoLa decoding strategy has been moved to the following remote-code repository a few versions ago: https://huggingface.co/transformers-community/dola

The Contrastive Search decoding strategy has been moved to the following remote-code repository a few versions ago: https://huggingface.co/transformers-community/contrastive-search

Both have now been removed from the library as a result.

🚨 Remove DoLa decoding strategy by @manueldeprada in #40082
🚨 Remove Contrastive Search decoding strategy by @manueldeprada in #40428

Fix sliding window in flash attention

Flash attention has used sliding window sizes which were off by one. This affected generations that had initially bigger contexts than the sliding window size.

:rotating_light: [Flash Attention] Fix sliding window size by @vasqu in #40163

Minimum Torch version is now 2.2

Torch 2.1 support has been unreliable for some time, so we've now made it official and bumped our minimum version to 2.2.

byebye torch 2.1 by @Rocketknight1 in #40317

Bugfixes and improvements

[CI] post-GptOss fixes for green CI by @gante in #39929
Avoid utils/check_bad_commit.py failing due to rate limit (requesting api.github.com) by @ydshieh in #39918
Fix CI: Tests failing on CPU due to torch.device('cpu').index being None by @manueldeprada in #39933
circleci: pin torch 2.7.1 until torchcodec is updated by @ydshieh in #39951
[docs] ko toc fix by @gante in #39927
docs: fix typo in 'quantization-aware training' by @luckyvickyricky in #39904
Fix grammatical error in MoE variable name: expert_hitted → expert_hit, hitted_experts → hit_experts by @Mihonarium in #39959
fix typo by @Tialo in #39936
[image processor] fix glm4v by @KeyKy in #39964
remove triton_kernels dep with kernels instead by @SunMarc in #39926
Fix fix_and_overwrite mode of utils/check_docstring.py by @manueldeprada in #39369
[bugfix] fix flash_attention_2 unavailable error on Ascend NPU by @FightingZhen in #39844
chore: update Deformable_Detr model card by @arpon-kapuria in #39902
Modular fix: remove the model name in find_file_type by @yonigozlan in #39897
Gemma3 fixes by @remi-or in #39960
[superglue] Fixed the way batch mask was applied to the scores before match assignment computation by @sbucaille in #39968
Support input_embeds in torch exportable decoders by @jackzhxng in #39836
Various test fixes for AMD by @remi-or in #39978
[Idefics] fix device mismatch by @zucchini-nlp in #39981
Fix gemma3n feature extractor's incorrect squeeze by @Isotr0py in #39919
[typing] Fix return typehint for decoder and inv_freq annotation by @qubvel in #39610
Fix consistency by @Cyrilvallez in #39995
Update expected output values after #39885 (part 1) by @ydshieh in #39990
Fix int4 quantized model cannot work with cpu by @yuanwu2017 in #39724
Fix missing video inputs for PerceptionLM. by @shuminghu in #39971
fix: remove CHAT_TEMPLATE import in tests for deepseek-vl by @geetu040 in #40003
Fix HGNetV2 Model Card and Image Classification Pipeline Usage Tips by @ducviet00 in #39965
Fix default values of getenv by @cyyever in #39867
FA2 can continue generation from cache by @zucchini-nlp in #39843
unpin torch<2.8 on circleci by @ydshieh in #40012
docs: fix duplication in 'en/optimizers.md' by @luckyvickyricky in #40014
Raising error when quantizing a quantized model by @MekkCyber in #39998
Update expected output values after #39885 (part 2) by @ydshieh in #40015
pin torchcodec==0.5.0 for now with torch 2.7.1 on daily CI by @ydshieh in #40013
Fix broken image inference for Fuyu model by @Isotr0py in #39915
Higgs modules_to_not_convert standardization by @MekkCyber in #39989
Fix an annoying flaky test by @zucchini-nlp in #40000
Harmonize past_key_value to past_key_valueS everywhere by @Cyrilvallez in #39956
Fix missing None default values for Gemma3n model in get_placeholder_mask by @Znerual in #39991)
[core] Refactor the Cache logic to make it simpler and more general by @Cyrilvallez in #39797
Tie weights recursively on all submodels by @Cyrilvallez in #39996
Bnb failling tests by @MekkCyber in #40026
fix notification_service.py about time_spent by @ydshieh in #40037
Revert "fix notification_service.py about time_spent" by @ydshieh in #40044
Update HuBERT model card according to template by @reedrya in #39742
unpin torchcodec==0.5.0 and use torch 2.8 on daily CI by @ydshieh in #40072
fix: resolve triton version check compatibility on windows by @Tsumugii24 in #39986
[qwen-vl] fix beam search with videos by @zucchini-nlp in #39726
[gemma3] update conversion key mapping by @zucchini-nlp in #39778
fix: move super().init after vision_config init in Mistral3Config by @starcatmeow in #40063
Remove deprecated cache-related objects by @Cyrilvallez in #40035
guard on model.eval when using torch.compile + FSDP2 by @winglian in #37413
Fix repo consistency by @zucchini-nlp in #40077
added Textnet fast image processor by @rahzaazhar in #39884
Fix time_spent in notification_service.py. by @ydshieh in #40081
chore: standardize DeBERTa model card by @Shoumik-Gandre in #37409
[GPT Big Code] Fix attention scaling by @vasqu in #40041
feat: extract rev in attn_implementation kernels via @ by @drbh in #40009
Update notification service MI325 by @ivarflakstad in #40078
Fix PerceptionLM image preprocessing for non-tiled image input. by @shuminghu in #40006
Revert FA2 kwargs construction by @zucchini-nlp in #40029
[fix] batch inference for llava_onevision by @cyr0930 in #40021
[docs] Zero Shot Object Detection Task by @ariG23498 in #40096
Update Glm4V processor and add tests by @zucchini-nlp in #39988
Add glm4.5&&glm4.5V doc by @lambertwjh in #40095
Causal loss for ForConditionalGeneration by @qgallouedec in #39973
Audio encodings now match conv2d weight dtype in Gemma3nAudioSSCPConvBlock by @Malav-P in #39743
New DynamicSlidingWindowLayer & associated Cache by @Cyrilvallez in #40039
Enable SIM rules by @cyyever in #39806
feat: add is_fast to ImageProcessor by @MilkClouds in #39603
Re-apply make style by @Cyrilvallez in #40106
Replace logger.warning with logger.warning_once in GradientCheckpointingLayer by @qgallouedec in #40091
Fix regression in mllama vision encoder by @Isotr0py in #40083
Switch the order of args in StaticCache (for BC and future logic) by @Cyrilvallez in #40100
Fix Qwen3 MoE GGUF architecture mismatch by @ctcanbol in #39976
Fix error on importing unavailable torch.distributed by @m-gallus in #40038
[Flash Attention] Fix flash attention integration by @vasqu in #40002
[trainer] ensure special tokens in model configs are aligned with tokenizer at train time by @gante in #38441
Fix Causality Handling in Flash Attention to Support Bidirectional Attention by @lucaswychan in #39707
[docs] Add reference to HF-maintained custom_generate collections by @gante in #39894
Add model card for MobileViT by @Shivamjan in #40033
remove sequence parallel in llama4 by @3outeille in #40084
🌐 [i18n-KO] Translated tiny_agents.md to Korean by @AhnJoonSung in #39913
[bugfix] Fix tensor device in Idefics2, Idefics3, and SmolVLM by @qgallouedec in #39975
changed xLSTMRMSNorm to RMSNorm by @nikitazuevblago in #40113
Fix QuantoQuantizedCache import issues by @manueldeprada in #40109
[serve] allow array content inputs for LLMs by @gante in #39829
decoding_method argument in generate by @manueldeprada in #40085
Collated reports by @ivarflakstad in #40080
DOCS: Add missing space in SECURITY.md by @shivaheidari in #40087
[trainer] handle case where EOS token is None in generation_config by @gante in #40127
Fix hidden torchvision>=0.15 dependency issue by @yonigozlan in #39928
🌐 [i18n-KO] Translated main_classes/processors.md to Korean by @TaskerJang in #39519
🌐 [i18n-KO] Translated jamba.md to Korean by @skwh54 in #39890
🌐 [i18n-KO] Translated main_classes/optimizer_schedules.md to Korean by @luckyvickyricky in #39713
🌐 [i18n-KO] Translated gpt2.md to Korean by @taemincode in #39808
🌐 [i18n-KO] Translated optimizers.md to Korean by @chelsseeey in #40011
🌐 [i18n-KO] Translated grounding-dino.md to Korean by @TaskerJang in #39861
🌐 [i18n-KO] Translated pipelines.md to Korean by @xhaktm00 in #39577
gpt oss is important by @ArthurZucker in #40139
Fix Janus by @Cyrilvallez in #40140
[docs] Fix ko toctree by @stevhliu in #40138
Remove an old badly designed test by @Cyrilvallez in #40142
updated visualBERT modelcard by @Anil-Red in #40057
🌐 [i18n-KO] Translated gemma3.md to Korean by @seopp in #39865
Fix quantized cache with only cache_implementation in generate by @Cyrilvallez in #40144
Add pytest marker: torch_compile_test and torch_export_test by @ydshieh in #39950
Update Dockerfiles to install packages inside a virtual environment by @Sai-Suraj-27 in #39098
Create self-scheduled-amd-mi355-caller.yml by @glegendre01 in #40134
[Cohere2Vision] remove unused arg by @zucchini-nlp in #40103
[efficientloftr] fix bugs and follow original cross attn implementation strictly by @sbucaille in #40141
Fix CI: Use correct import in SAM for torchvision InterpolationMode by @manueldeprada in #40160
[Continous Batching] set head_dim when config.head_dim is None by @kashif in #40159
Replace self.tokenizer by self.processing_class by @qgallouedec in #40119
[FA2] Fix it finally - revert fa kwargs preparation by @Cyrilvallez in #40161
[bugfix] fix flash-attention2 unavailable error for Ascend NPU by @FightingZhen in #40151
build: Add fast image processor tvp by @adutchengineer in #39529
Add GptOssForSequenceClassification for GPT-OSS models by @zyfedward in #40043
Standardize BARTpho model card: badges, new examples, fixed broken im… by @eshwanthkartitr in #40051
Add dates to the model docs by @MHRDYN7 in #39320
Pin torch to 2.7.1 on CircleCI for now by @ydshieh in #40174
Update dynamic attnt setter for multimodals by @zucchini-nlp in #39908
[MINOR:TYPO] Update base.py by @cakiki in #40169
make model doc device agnostic by @yao-matrix in #40143
fix to avoid modifying a view in place by @3outeille in #40162
Fix fsdp for generic-task models by @Cyrilvallez in #40191
Add repr to EncoderDecoderCache by @Cyrilvallez in #40195
Fix typos by @cyyever in #40175
Remove _prepare_flash_attention_from_position_ids by @cyyever in #40069
Avoid CUDA stream sync by @cyyever in #40060
Fix various Pylint warnings by @cyyever in #40107
Update: add type hints to check_tokenizers.py by @ajeet214 in #40094
Benchmarking improvements by @ahadnagy in #39768
docs: Update LayoutLM model card according to new standardized format by @Jin-HoMLee in #40129
Revert "Pin torch to 2.7.1 on CircleCI for now" + Final fix for too long with no output by @ydshieh in #40201
Use correct model_input_names for PixtralImageProcessor by @rohitrango in #40226
fix error vocab_size at Qwen2_5_VLForConditionalGeneration loss_function by @killight98 in #40130
[SAM 2] Change checkpoints in docs and tests by @yonigozlan in #40213
Fix more typos by @cyyever in #40212
Fix ESM token_dropout crash when using inputs_embeds instead of input_ids by @notkisk in #40181
AMD scheduled CI ref env file by @ivarflakstad in #40243
Fix more pylint warnings by @cyyever in #40204
remove transpose_for_scores call in ESM-2 by @pstjohn in #40210
Add chat_template (jinja2) as an extra dependency by @tboerstad in #40128
[typing] fix type annotation error in DepthPro model image processor by @MengAiDev in #40238
[serve] guard imports by @gante in #39825
[CI] Fix repo consistency by @vasqu in #40249
Fixes for EncoderDecoderCache by @remi-or in #40008
fix: Catch correct ConnectionError for additional_chat_templates by @akug in #39874
Model card for NLLB by @sahil-kabir in #40074
Correct typo and update notes in docs Readme by @PavloFesenko in #40234
Fix benchmark workflow by @ahadnagy in #40254
docs: Update OLMo model card by @rafakatri in #40233
Skip broken tests by @zucchini-nlp in #40157
Remove MI300 CI by @ivarflakstad in #40270
set inputs_embeds to None while generate to avoid audio encoder forward in generation process by @BakerBunker in #40248
[detection] fix attention mask for RT-DETR-based models by @materight in #40269
Fix slow static cache export tests by @jackzhxng in #40261
Fix setting attention for multimodal models by @zucchini-nlp in #39984
[detection] fix correct k_proj weight and bias slicing in D-FINE by @notkisk in #40257
Skipping pytree registration in case fsdp is enabled by @romitjain in #40075
Update image_processing_perception_lm_fast.py to allow for proper override of vision_input_type by @tyleryzhu in #40252
fix which routing method by @ArthurZucker in #40283
Fix chat CLI GPU loading and request_id validation issues by @robin-ede in #40230)
docs(layoutlm): add missing id=usage to <hfoptions> tag in LayoutLM model card by @Jin-HoMLee in #40273
Standardize RAG model card by @aayush226 in #40222
docs: Update TrOCR model card to new format by @AceHunterr in #40240
Update model card for gpt neox japanese by @ahnjj in #39862
SmolVLM and InternVL: Ensure pixel values are converted to the correct dtype for fp16/bf16 by @qgallouedec in #40121
Standardize BertGeneration model card by @nemitha2005 in #40250
Adjust ROCm test output expectations by @ahadnagy in #40279
SmolVLM test fixes by @ahadnagy in #40275
make model docs device agnostic (2) by @yao-matrix in #40256
[3/3] make docs device agnostic, all en docs for existing models done by @yao-matrix in #40298
Allow to be able to run torch.compile tests with fullgraph=True by @ydshieh in #40164
[FA] Fix dtype in varlen with position ids by @vasqu in #40295
[docs] delete more TF/Flax docs by @gante in #40289
Clean up X-Codec. by @ebezzam in #40271
Remove OTel SDK dependencies by @anuraaga in #40305
Fix GOT-OCR2 and Cohere2Vision image processor patches caculation by @Isotr0py in #40312
[fix] Pass adamw optimizer parameters to StableAdamW by @emapco in #40184
chore: fix typo in find_executable_batch_size to match new 0.9 ratio by @MilkClouds in #40206
:rotating_light: [Flash Attention] Fix sliding window size by @vasqu in #40163
Remove unnecessary contiguous calls for modern torch by @Rocketknight1 in #40315
Qwen2.5-Omni test fixes by @ahadnagy in #40307
Add back _tp_plan attribute by @rishub-tamirisa in #39944
byebye torch 2.1 by @Rocketknight1 in #40317
No more natten by @ydshieh in #40287
[GPT OSS] Refactor the tests as it was not properly checking the outputs by @ArthurZucker in #40288
Update CI with nightly torch workflow file by @ydshieh in #40306
Fix: Apply get_placeholder_mask in Ovis2 by @thisisiron in #40280
Update notification service amd_daily_ci_workflows definition by @ivarflakstad in #40314
One cache class to rule them all by @Cyrilvallez in #40276
Fix chunked attention mask with left-padding by @Cyrilvallez in #40324
[docs] remove flax references from /en/model_doc by @gante in #40311
Fix qwen-omni processor text only mode by @yuekaizhang in #40336
Change Qwen2RMSNorm to RMSNorm from PyTorch by @cyyever in #40066
Add DeepseekV3ForSequenceClassification for Deepseek V3 models by @abdokaseb in #40200
Fix deprecation warning version by @Cyrilvallez in #40343
Add missing arguments to class constructors by @cyyever in #40068
[docs] remove TF references from /en/model_doc by @gante in #40344
Fix: Only call Trainer.align_special_tokens if model has "config" attribute by @tomaarsen in #40322
add type hints by @wirthual in #40319
Fix an infinite loop bug in recursive search of relative imports by @eladsegal in #40326
Fix links in Glm4vMoe configuration classes to point to the correct H… by @vvvdwbvvv in #40310
T5 test and target device fixes by @ahadnagy in #40313
Update test_spm_converter_bytefallback_warning by @ydshieh in #40284
(small) fix conditional for input_ids and input_embeds in marian by @cyntqliu in #40045
Fix attention vizualizer by @molbap in #40285
[ModernBert] Prevent the attention mask from being None in ModernBertForSequenceClassification by @ashmikuz in #35991
Clean up XCodec and other codecs by @ebezzam in #40348
[serve] add cors warnings by @gante in #40112
[detection] use consistent dtype for Conditional and DAB DETR positional embeddings by @agkphysics in #40300
Remove more PyTorch 2.2 compatible code by @cyyever in #40337
[FA] Fix some model tests by @vasqu in #40350
Qwen2.5-VL test fixes for ROCm by @ahadnagy in #40308
[generate] handle support for cache classes when num enc layers != num dec layers by @gante in #40277
[4/N]more docs to device agnostic by @yao-matrix in #40355
DOCS: Clarification on the use of label_names as an argument to TrainingArguments by @huzaifa-jawad367 in #40353
Fix idefics3 vision embeddings indices dtype by @Isotr0py in #40360
wav2vec2 fixes by @remi-or in #40341
Change multimodal data links to HF hub by @zucchini-nlp in #40309
[pipelines] add support to skip_special_tokens in the main text generation pipelines by @gante in #40356
⚠️⚠️ Use dtype instead of torch_dtype everywhere! by @Cyrilvallez in #39782
[processor] move commonalities to mixin by @zucchini-nlp in #40339
[configuration] allow to overwrite kwargs from subconfigs by @zucchini-nlp in #40241
fix(example): align parameter names with the latest function definition for gdino by @developer0hye in #40369
Add GptOssForTokenClassification for GPT-OSS models by @abdokaseb in #40190
Bug Fix: Dynamically set return_lse flag in FlexAttention by @amd-lalithnc in #40352
Chat Template Doc Fixes by @Rocketknight1 in #40173
Rework the Cache documentation by @Cyrilvallez in #40373
Update README_zh-hans.md by @TardC in #40380
HF papers in doc by @qgallouedec in #40381
Run FA2 tests in CI by @ydshieh in #40397
Reactivate a lot of tests skipped for no reason anymore by @Cyrilvallez in #40378
:broom: :broom: :broom: Get set decoder cleanup by @molbap in #39509
fix to accept cumulative_seqlens from TransformersKwargs in FA by @Kurt232 in #40194
[docs] flax/jax purge by @gante in #40372
Fix typo: 'casual' -> 'causal' in code and documentation by @akintunero in #40371)
Fix CI (hunyuan moe does not support fullgraph) by @Cyrilvallez in #40423
Fix typo: 'seperator' to 'separator' in variable names by @Prawal-Sharma in #40389
Fix UnboundLocalError in WER metric computation by @prxshetty in #40402
Gpt oss optim by @jiqing-feng in #40304
Fix processing tests by @zucchini-nlp in #40379
Fix label smoothing incompatibility with multi-label classification by @avchauzov in #40296
Fix modular for modernbert-decoder by @Cyrilvallez in #40431
Update collated reports working directory and --path by @ivarflakstad in #40433
Add tokenizer_kwargs argument to the text generation pipeline by @Joshua-Chin in #40364
[docs] remove last references to transformers TF classes/methods by @gante in #40429
Remove working-dir from collated reports job by @ivarflakstad in #40435
🌐 [i18n-KO] Translated models.md to Korean by @Judy-Choi in #39518
Gemma3 text fixes: Add expectations for MI325 by @ahadnagy in #40384
Fix collated reports model directory traversal by @ivarflakstad in #40437
Fix https://github.com/huggingface/transformers/issues/40292 by @id01 in #40439
Fix collated reports uploading by @ivarflakstad in #40440
InternVL MI325 test expectations by @ahadnagy in #40387
Fix collated reports model name entry by @ivarflakstad in #40441
Fix non FA2 tests after FA2 installed in CI docker image by @ydshieh in #40430
Refactor ViT-like models by @qubvel in #39816
[Trainer] accelerate contextparallel support in trainer by @kashif in #40205
fix qwen25-vl grad acc by @iMountTai in #40333
[video processors] decode only sampled videos -> less RAM and faster processing by @zucchini-nlp in #39600
rename get_cuda_warm_up_factor to get_accelerator_warm_up_factor by @yao-matrix in #40363
Make cache_config not mandatory by @remi-or in #40316
Continuous batching refactor by @remi-or in #40426
flash_paged: s_aux may not exist by @pcuenca in #40434
Fix extra template loading by @Rocketknight1 in #40455
deci gguf support by @ved1beta in #38669
[fast_image_processor] fix image normalization for resize by @audioXD in #40436
[RoPE] explicit factor > implicit factor in YaRN by @gante in #40320
[pipeline] Add Keypoint Matching pipeline by @sbucaille in #39970
Update SegFormer model card by @GSNCodes in #40417
Not to shock AMD team by the cancelled workflow run notification ❤️ 💖 by @ydshieh in #40467
Fix nightly torch CI by @ydshieh in #40469
CI when PR merged to main by @ydshieh in #40451
Validate GptOssConfig rope config after it's fully initialized by @zifeitong in #40474
[modular] Use multi-processing + fix model import issue by @Cyrilvallez in #40481
[modular] Remove ambiguity in all calls to parent class methods + fix dependency graph by @Cyrilvallez in #40456
[ESM] support attention API by @zucchini-nlp in #40370
[EfficientLoFTR] dynamic image size support by @sbucaille in #40329
Fix qwen2_moe tests by @ydshieh in #40494
[Whisper] Add rocm expected results to certain tests by @ivarflakstad in #40482
Collated reports: no need to upload artifact by @ivarflakstad in #40502
Fix the CI workflow of merge to main by @ydshieh in #40503
docs(pixtral): Update Pixtral model card to new format by @BryanBradfo in #40442
[modular] Classes can now be defined and referenced in arbitrary order (without bringing unwanted dependencies) by @Cyrilvallez in #40507
Include machine type in collated reports filename by @ivarflakstad in #40514

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@remi-or
- Gemma3 fixes (#39960)
- Various test fixes for AMD (#39978)
- Fixes for EncoderDecoderCache (#40008)
- wav2vec2 fixes (#40341)
- Make cache_config not mandatory (#40316)
- Continuous batching refactor (#40426)
@sbucaille
- [superglue] Fixed the way batch mask was applied to the scores before match assignment computation (#39968)
- [efficientloftr] fix bugs and follow original cross attn implementation strictly (#40141)
- [pipeline] Add Keypoint Matching pipeline (#39970)
- [EfficientLoFTR] dynamic image size support (#40329)
@ducviet00
- Fix HGNetV2 Model Card and Image Classification Pipeline Usage Tips (#39965)
- Add support for Florence-2 (#38188)
@cyyever
- Fix default values of getenv (#39867)
- Enable SIM rules (#39806)
- Fix typos (#40175)
- Remove _prepare_flash_attention_from_position_ids (#40069)
- Avoid CUDA stream sync (#40060)
- Fix various Pylint warnings (#40107)
- Fix more typos (#40212)
- Fix more pylint warnings (#40204)
- Change Qwen2RMSNorm to RMSNorm from PyTorch (#40066)
- Add missing arguments to class constructors (#40068)
- Remove more PyTorch 2.2 compatible code (#40337)
@zRzRzRzRzRzRzR
- GLM-4.5V Model Support (#39805)
@SangbumChoi
- Add Segment Anything 2 (SAM2) (#32317)
@adutchengineer
- build: Add fast image processor tvp (#39529)
@MHRDYN7
- Add dates to the model docs (#39320)
@yao-matrix
- make model doc device agnostic (#40143)
- make model docs device agnostic (2) (#40256)
- [3/3] make docs device agnostic, all en docs for existing models done (#40298)
- [4/N]more docs to device agnostic (#40355)
- rename get_cuda_warm_up_factor to get_accelerator_warm_up_factor (#40363)
@Manalelaidouni
- Add X-Codec model (#38248)
@thisisiron
- Add Ovis2 model and processor implementation (#37088)
- Fix: Apply get_placeholder_mask in Ovis2 (#40280)
@tic-top
- Add Kosmos-2.5 (#31711)
@yjc9696
- HunYuan opensource (#39606)
@Fazziekey
- Addiing ByteDance Seed Seed-OSS (#40272)

Open source →

4.54.025 Jul 2025Breaking

Fix tests due to breaking change in accelerate by @SunMarc in #39451

Important news!

In order to become the source of truth, we recognize that we need to address two common and long-heard critiques about transformers:

transformers is bloated
transformers is slow

Our team has focused on improving both aspects, and we are now ready to announce this. The modeling files for the standard Llama models are down to 500 LOC and should be much more readable, keeping just the core of the modeling and hiding the "powerful transformers features." <img width="1583" height="974" alt="image" src="https://github.com/user-attachments/assets/f1075598-d63e-4184-b3af-c0d4b31cdde5" />

The MoEs are getting some kernel magic, enabling the use of the efficient megablocks kernels, setting a good precedent to allow the community to leverage any of the most powerful kernels developed for quantization as well! It should also be much more convenient to use with any attention implementation you want. This opens the door to some optimizations such as leveraging flash-attention on Metal (MPS Torch backend). <img width="2050" height="752" alt="image" src="https://github.com/user-attachments/assets/23ebfb20-7626-46a5-b264-76ffb8b8c811" />

This is but the tip of the iceberg: with the work on kernels that we're heavily pushing forward, expect speed-ups on several backends in the coming months!!

This release also includes the first steps to enabling efficient distributed training natively in transformers. Loading a 100B model takes ~3 seconds on our cluster — we hope this will be the norm for everyone! We are working on distributed checkpointing as well, and want to make sure our API can be easily used for any type of parallelism.

We want the community to benefit from all of the advances, and as always, include all hardware and platforms! We believe the kernels library will give the tools to optimize everything, making a big difference for the industry!

New models

Ernie 4.5

The Ernie 4.5 model was released in the Ernie 4.5 Model Family release by baidu. This family of models contains multiple different architectures and model sizes. This model in specific targets the base text model without mixture of experts (moe) with 0.3B parameters in total. It uses the standard Llama at its core.

Other models from the family can be found at Ernie 4.5 MoE.

[Ernie 4.5] Add ernie text models by @vasqu in #39228

Voxtral

Voxtral is an upgrade of Ministral 3B and Mistral Small 3B, extending its language capabilities with audio input support. It is designed to handle tasks such as speech transcription, translation, and audio understanding.

You can read more in Mistral's realease blog post.

The model is available in two checkpoints:

3B: mistralai/Voxtral-Mini-3B-2507
24B: mistralai/Voxtral-Small-24B-2507

Key Features

Voxtral builds on Ministral-3B by adding audio processing capabilities:

Transcription mode: Includes a dedicated mode for speech transcription. By default, Voxtral detects the spoken language and transcribes it accordingly.
Long-form context: With a 32k token context window, Voxtral can process up to 30 minutes of audio for transcription or 40 minutes for broader audio understanding.
Integrated Q&A and summarization: Supports querying audio directly and producing structured summaries without relying on separate ASR and language models.
Multilingual support: Automatically detects language and performs well across several widely spoken languages, including English, Spanish, French, Portuguese, Hindi, German, Dutch, and Italian.
Function calling via voice: Can trigger functions or workflows directly from spoken input based on detected user intent.
Text capabilities: Maintains the strong text processing performance of its Ministral-3B foundation.

Add voxtral by @eustlb in #39429

LFM2

LFM2 represents a new generation of Liquid Foundation Models developed by Liquid AI, specifically designed for edge AI and on-device deployment.

The models are available in three sizes (350M, 700M, and 1.2B parameters) and are engineered to run efficiently on CPU, GPU, and NPU hardware, making them particularly well-suited for applications requiring low latency, offline operation, and privacy.

LFM2 by @paulpak58 in #39340

DeepSeek v2

The DeepSeek-V2 model was proposed in DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model by DeepSeek-AI Team.

The model uses Multi-head Latent Attention (MLA) and DeepSeekMoE architectures for efficient inference and cost-effective training. It employs an auxiliary-loss-free strategy for load balancing and multi-token prediction training objective. The model can be used for various language tasks after being pre-trained on 14.8 trillion tokens and going through Supervised Fine-Tuning and Reinforcement Learning stages.

Add DeepSeek V2 Model into Transformers by @VladOS95-cyber in #36400

ModernBERT Decoder models

ModernBERT Decoder is the same architecture as ModernBERT but trained from scratch with a causal language modeling (CLM) objective. This allows for using the same architecture for comparing encoders and decoders. This is the decoder architecture implementation of ModernBERT, designed for autoregressive text generation tasks.

Like the encoder version, ModernBERT Decoder incorporates modern architectural improvements such as rotary positional embeddings to support sequences of up to 8192 tokens, unpadding to avoid wasting compute on padding tokens, GeGLU layers, and alternating attention patterns. However, it uses causal (unidirectional) attention to enable autoregressive generation.

Add ModernBERT Decoder Models - ModernBERT, but trained with CLM! by @orionw in #38967

EoMT

The Encoder-only Mask Transformer (EoMT) model was introduced in the CVPR 2025 Highlight Paper Your ViT is Secretly an Image Segmentation Model by Tommie Kerssies, Niccolò Cavagnero, Alexander Hermans, Narges Norouzi, Giuseppe Averta, Bastian Leibe, Gijs Dubbelman, and Daan de Geus. EoMT reveals Vision Transformers can perform image segmentation efficiently without task-specific components.

✨ Add EoMT Model || 🚨 Fix Mask2Former loss calculation by @yaswanth19 in #37610

Doge

Doge is a series of small language models based on the Doge architecture, aiming to combine the advantages of state-space and self-attention algorithms, calculate dynamic masks from cached value states using the zero-order hold method, and solve the problem of existing mainstream language models getting lost in context. It uses the wsd_scheduler scheduler to pre-train on the smollm-corpus, and can continue training on new datasets or add sparse activation feedforward networks from stable stage checkpoints.

<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/refs%2Fpr%2F426/transformers/model_doc/doge_architecture.png" alt="drawing" width="600"

Add Doge model by @LoserCheems in #35891

AIM v2

The AIMv2 model was proposed in Multimodal Autoregressive Pre-training of Large Vision Encoders by Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, Alexander T Toshev, Marcin Eichner, Moin Nabi, Yinfei Yang, Joshua M. Susskind, Alaaeldin El-Nouby.

The abstract from the paper is the following:

We introduce a novel method for pre-training of large-scale vision encoders. Building on recent advancements in autoregressive pre-training of vision models, we extend this framework to a multimodal setting, i.e., images and text. In this paper, we present AIMV2, a family of generalist vision encoders characterized by a straightforward pre-training process, scalability, and remarkable performance across a range of downstream tasks. This is achieved by pairing the vision encoder with a multimodal decoder that autoregressively generates raw image patches and text tokens. Our encoders excel not only in multimodal evaluations but also in vision benchmarks such as localization, grounding, and classification. Notably, our AIMV2-3B encoder achieves 89.5% accuracy on ImageNet-1k with a frozen trunk. Furthermore, AIMV2 consistently outperforms state-of-the-art contrastive models (e.g., CLIP, SigLIP) in multimodal image understanding across diverse settings.

Add Aimv2 model by @yaswanth19 in #36625

PerceptionLM

The PerceptionLM model was proposed in PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding by Jang Hyun Cho et al. It's a fully open, reproducible model for transparent research in image and video understanding. PLM consists of a vision encoder with a small scale (<8B parameters) LLM decoder.

PerceptionLM by @shuminghu in #37878

Efficient LoFTR

The EfficientLoFTR model was proposed in Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed by Yifan Wang, Xingyi He, Sida Peng, Dongli Tan and Xiaowei Zhou.

This model consists of matching two images together by finding pixel correspondences. It can be used to estimate the pose between them. This model is useful for tasks such as image matching, homography estimation, etc.

Add EfficientLoFTR model by @sbucaille in #36355

EVOLLA

Evolla is an advanced 80-billion-parameter protein-language generative model designed to decode the molecular language of proteins. It integrates information from protein sequences, structures, and user queries to generate precise and contextually nuanced insights into protein function. Trained on an unprecedented AI-generated dataset of 546 million protein question-answer pairs and 150 billion word tokens, Evolla significantly advances research in proteomics and functional genomics, providing expert-level insights and shedding light on the molecular logic encoded in proteins.

Add evolla rebase main by @zhoubay in #36232

DeepSeek VL

Deepseek-VL was introduced by the DeepSeek AI team. It is a vision-language model (VLM) designed to process both text and images for generating contextually relevant responses. The model leverages LLaMA as its text encoder, while SigLip is used for encoding images.

Add support for DeepseekAI's DeepseekVL by @geetu040 in #36248

xLSTM

The xLSTM model was proposed in xLSTM: Extended Long Short-Term Memory by Maximilian Beck*, Korbinian Pöppel*, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter and Sepp Hochreiter. xLSTM updates the original LSTM architecture to be competitive with Transformer models by introducing exponential gating, matrix memory expansion, and parallelizable training and ingestion.

The 7B model variant was trained by the xLSTM team Maximilian Beck, Korbinian Pöppel, Phillip Lippe, Richard Kurle, Patrick Blies, Sebastian Böck and Sepp Hochreiter at NXAI.

Add xlstm model by @Cyrilvallez in #39665

EXAONE 4.0

EXAONE 4.0 model is the language model, which integrates a Non-reasoning mode and Reasoning mode to achieve both the excellent usability of EXAONE 3.5 and the advanced reasoning abilities of EXAONE Deep. To pave the way for the agentic AI era, EXAONE 4.0 incorporates essential features such as agentic tool use, and its multilingual capabilities are extended to support Spanish in addition to English and Korean.

The EXAONE 4.0 model series consists of two sizes: a mid-size 32B model optimized for high performance, and a small-size 1.2B model designed for on-device applications.

Add EXAONE 4.0 model by @lgai-exaone in #39129

Parallelisation

We've added Expert Parallel support for Llama4, next release will include it for all model! You can just set a distributed_config with enable_expert_parallel=True. This is enabling efficient training of sparse Mixture-of-Experts (MoE) models across multiple devices. This allows each expert in the MoE layer to run in parallel (instead of previous TP which requires more communication), significantly improving scalability and memory efficiency.

Add ep by @ArthurZucker in #39501

Quantization

FP Quant

FP-Quant is a quantization method optimized for Blackwell-generation Nvidia GPUs, supporting efficient post-training quantization (PTQ) and quantization-aware training (QAT) of LLMs using MXFP4 and NVFP4 formats.

Currently, only PTQ with MXFP4 is available. You can quantize models on-the-fly using transformers:

from transformers import AutoModelForCausalLM, FPQuantConfig

model = AutoModelForCausalLM.from_pretrained(
    "qwen/Qwen3-8B",
    quantization_config=FPQuantConfig(),
    device_map="cuda",
    torch_dtype=torch.bfloat16,
)

FP-Quant requires a Blackwell GPU and runs via the QuTLASS library. No Blackwell GPU? Use FPQuantConfig(pseudoquant=True) to emulate quantization (no QuTLASS needed).

The following results show the inference speedup of QuTLASS MXFP4 over PyTorch BF16 in Transformers. MXFP4 gives consistent speedups across all batch sizes, reaching up to 4× faster at larger scales.

FP-Quant support by @BlackSamorez in #38696

Kernels

The kernels project aims to become the single trusted source for high-performance kernels in the Transformers ecosystem. We're working toward centralizing all kernels on the Hub, so updates, bug fixes, and improvements can happen in one place—no more scattered repos and no compilation headaches!

You can already try it out today by setting use_kernels=True in from_pretrained. Any contributor can build their kernel, register it and use it right away—no extra setup, more on this here

Even better: want to use Flash Attention 3? No need to deal with tricky compilation and missing symbols issues! Just drop in:

model.set_attn_implementation("kernels-community/flash-attn3")

This automatically fetches the right build for your setup (e.g. CUDA and PyTorch versions).

We’re also teaming up with amazing kernel devs from Unsloth, Liger, vLLM, and more to bring their work directly to the Hub—making it easier than ever to access amazing performance with a single line of code.

Kernels flash attn by @ArthurZucker in #39474

Transformers Serve

https://github.com/user-attachments/assets/9928f62b-543c-4b8a-b81b-4a6e262c229e

Over the past few months, we have been putting more and more functionality in the transformers chat utility, which offers a CLI-based app to chat with chat models. We've chosen to push this further by splitting the backend of transformers chat in a new, separate utility called transformers serve.

This is ideal for experimentation purposes, or to run models locally for personal and private use. It does not aim to compete with dedicated inference engines such as vLLM or SGLang.

Models of diverse modalities supported by transformers may be served with the transformers serve CLI. It spawns a local server that offers compatibility with the OpenAI SDK, which is the de-facto standard for LLM conversations and other related tasks. This way, you can use the server from many third party applications, or test it using the transformers chat CLI (docs).

The server supports the following REST APIs:

/v1/chat/completions
/v1/responses
/v1/audio/transcriptions
/v1/models

Relevant commits:

Split transformers chat and transformers serve by @LysandreJik in #38443
[serve] Cursor support, move docs into separate page, add more examples by @gante in #39133
Fix continuous batching in transformers serve by @LysandreJik in #39149
[server] add tests and fix passing a custom generation_config by @gante in #39230
[serve] Model name or path should be required by @LysandreJik in #39178
Random serve fixes by @pcuenca in #39176
[tests] tag serve tests as slow by @gante in #39343
Responses API in transformers serve by @LysandreJik in #39155
[serve] Add speech to text (/v1/audio/transcriptions) by @gante in #39434
Transformers serve VLM by @LysandreJik in #39454

Refactors

Significant refactors have been underway in transformers, aiming to reduce the complexity of the code. A metric we follow to see how the refactors impact our code is to follow the number of lines in a given model; we try to reduce it as much as possible, while keeping everything related to the forward pass and model definition in that file.

See the evolution here:

Some notable refactors:

KV caching

KV caches are now defined per layer, enabling new hybrid caches that mix different attention types. CacheProcessors also encapsulate cache quantization and offloading, making them easy to customize.

[cache refactor] Move all the caching logic to a per-layer approach by @manueldeprada in #39106

Handling specific attributes like `output_attentions` or `output_hidden_states`

Such attributes require very specific handling within the forward call, while they're not important to understand how the model works. We remove that code but keep the functionality by providing a better utility to handle it.

Refactor the way we handle outputs for new llamas and new models by @ArthurZucker in #39120

Setting the attention implementation

We refactor the way to explicitly set the attention implementation so that it has a method dedicated to it.

[refactor] set attention implementation by @zucchini-nlp in #38974

Breaking changes

[Whisper] 🚨 Fix pipeline word timestamp: timestamp token is end of token time !!! by @eustlb in #36632
🚨 Don't use cache in non-generative models by @zucchini-nlp in #38751
🚨🚨🚨 [eomt] make EoMT compatible with pipeline by @yaswanth19 in #39122
🚨🚨 Fix and simplify attention implementation dispatch and subconfigs handling by @Cyrilvallez in #39423
🚨🚨🚨 [Trainer] Enable average_tokens_across_devices by default in TrainingArguments by @Krish0909 in #39395
🔴 Fix EnCodec internals and integration tests by @ebezzam in #39431

Bugfixes and improvements

Add StableAdamW Optimizer by @SunMarc in #39446
[Flex Attn] Fix torch 2.5.1 incompatibilities by @vasqu in #37406
fix test_compare_unprocessed_logit_scores by @ydshieh in #39053
fix t5gemma tests by @ydshieh in #39052
Update SuperPoint model card by @sbucaille in #38896
fix layoutlmv3 tests by @ydshieh in #39050
[docs] Model contribution by @stevhliu in #38995
Update PEGASUS-X model card by @dross20 in #38971
[docs] @auto_docstring by @stevhliu in #39011
[docs] Tensor parallelism by @stevhliu in #38241
[Whisper] fix shape mismatch in tests by @eustlb in #39074
Cleanup Attention class for Siglip and dependent models by @yaswanth19 in #39040
fix Gemma3nProcessorTest by @ydshieh in #39068
Fix initialization of OneFormer by @bvantuan in #38901
Uninstallling Flash attention from quantization docker by @MekkCyber in #39078
fix a bunch of XPU UT failures on stock PyTorch 2.7 and 2.8 by @yao-matrix in #39069
Pipeline: fix unnecessary warnings by @eustlb in #35753
fix mistral3 tests by @ydshieh in #38989
fixed typo for docstring in prepare_inputs method by @JINO-ROHIT in #39071
TST PEFT integration tests with pipeline generate by @BenjaminBossan in #39086
add fast image processor nougat by @NahieliV in #37661
Add Fast Image Processor for mobileViT by @MinJu-Ha in #37143
guard torch distributed check by @tvukovic-amd in #39057
fix dots1 tests by @ydshieh in #39088
Add Fast Image Processor for Chameleon by @farrosalferro in #37140
Fix: unprotected import of tp plugin by @S1ro1 in #39083
TST Fix PEFT integration test bitsandbytes config by @BenjaminBossan in #39082
[fix] Add FastSpeech2ConformerWithHifiGan by @stevhliu in #38207
Sandeepyadav1478/2025 06 19 deberta v2 model card update by @sandeepyadav1478 in #38895
Fixes the failing test test_is_split_into_words in test_pipelines_token_classification.py by @st81 in #39079
skip some test_sdpa_can_dispatch_on_flash by @ydshieh in #39092
fix UT failures on XPU w/ stock PyTorch 2.7 & 2.8 by @yao-matrix in #39116
Fix some bug for finetune and batch infer For GLM-4.1V by @zRzRzRzRzRzRzR in #39090
docs: Gemma 3n audio encoder by @RyanMullins in #39087
All CI jobs with A10 by @ydshieh in #39119
Licenses by @LysandreJik in #39127
Fix chat by @gante in #39128
Enable XPU doc by @jiqing-feng in #38929
docs: correct two typos in awesome-transformers.md by @VladimirGutuev in #39102
switch default xpu tp backend to pytorch built-in XCCL from pytorch 2.8 by @yao-matrix in #39024
Update BigBirdPegasus model card by @dross20 in #39104
[Whisper] update token timestamps tests by @eustlb in #39126
Fix key mapping for VLMs by @bvantuan in #39029
Several fixes for Gemma3n by @Cyrilvallez in #39135
fix caching_allocator_warmup with tie weights by @jiqing-feng in #39070
feat: support indivisible shards for TP model loading and TPlizing. by @kmehant in #37220
[qwen2-vl] fix FA2 inference by @zucchini-nlp in #39121
[typing] LlamaAttention return typehint by @ArkVex in #38998
[VLMs] support passing embeds along with pixels by @zucchini-nlp in #38467
[superglue] fix wrong concatenation which made batching results wrong by @sbucaille in #38850
Fix missing fsdp & trainer jobs in daily CI by @ydshieh in #39153
Fix: Ensure wandb logs config in offline mode by @DavidS2106 in #38992
Change @lru_cache() to @lru_cache to match styles from #38883. by @rasmi in #39093
fix: remove undefined variable by @ybkurt in #39146
update bnb ground truth by @jiqing-feng in #39117
Suggest jobs to use in run-slow by @ydshieh in #39100
Update expected values (after switching to A10) by @ydshieh in #39157
fix llama tests by @ydshieh in #39161
Add activation sparsity reference in gemma3n doc by @ChongYou in #39160
fix default value of config to match checkpionts in LLaVa-OV models by @ved1beta in #39163
[smolvlm] fix video inference by @zucchini-nlp in #39147
Fix multimodal processor get duplicate arguments when receive kwargs for initialization by @Isotr0py in #39125
Blip2 fixes by @remi-or in #39080
Fix missing initializations for models created in 2024 by @bvantuan in #38987
Reduce Glm4v model test size significantly by @Cyrilvallez in #39173
[docs] ViTPose by @stevhliu in #38630
[generate] document non-canonical beam search default behavior by @gante in #39000
Update expected values (after switching to A10) - part 2 by @ydshieh in #39165
Update expected values (after switching to A10) - part 3 by @ydshieh in #39179
Test fixes for Aria (and some Expectation for llava_next_video) by @remi-or in #39131
[glm4v] fix video inference by @zucchini-nlp in #39174
when delaying optimizer creation only prepare the model by @winglian in #39152
Decouple device_map='auto' and tp_plan='auto' by @SunMarc in #38942
Fix many HPU failures in the CI by @IlyasMoutawwakil in #39066
[Dia] Change ckpt path in docs by @vasqu in #39181
Update expected values (after switching to A10) - part 4 by @ydshieh in #39189
[typing] better return typehints for from_pretrained by @qubvel in #39184
Update expected values (after switching to A10) - part 5 by @ydshieh in #39205
Update expected values (after switching to A10) - part 6 by @ydshieh in #39207
Add packed tensor format support for flex/sdpa/eager through the mask! by @Cyrilvallez in #39194
Update expected values (after switching to A10) - part 7 by @ydshieh in #39218
Update expected values (after switching to A10) - part 8 - Final by @ydshieh in #39220
[video processors] Support float fps for precise frame sampling by @zrohyun in #39134
Expectations re-order and corrected FA3 skip by @remi-or in #39195
[vjepa2] replace einsum with unsqueeze by @xenova in #39234
Fix missing fast tokenizer/image_processor in whisper/qwen2.5-omni processor by @Isotr0py in #39244
[modular] Follow global indexing and attribute setting, and their dependencies by @Cyrilvallez in #39180
fix typo in Gemma3n notes by @davanstrien in #39196
Don't send new comment if the previous one is less than 30 minutes (unless the content is changed) by @ydshieh in #39170
fix bug using FSDP V1 will lead to model device not properly set by @kaixuanliu in #39177
Make _compute_dynamic_ntk_parameters exportable by @xadupre in #39171
[modular] Simplify logic and docstring handling by @Cyrilvallez in #39185
[bugfix] fix flash attention 2 unavailable error on Ascend NPU by @FightingZhen in #39166
fix fastspeech2_conformer tests by @ydshieh in #39229
RotaryEmbeddings change is not None -> isinstance(..., dict) by @qubvel in #39145
Fix patch helper by @Cyrilvallez in #39216
enable xpu on kv-cache and hqq doc by @jiqing-feng in #39246
adjust input and output texts for test_modeling_recurrent_gemma.py by @kaixuanliu in #39190
Update tiny-agents example by @Wauplin in #39245
Add Korean translation for glossary.md by @JoosunH in #38804
Clarify per_device_train_batch_size scaling in TrainingArguments by @Shohail-Ismail in #38…
Add segmentation_maps support to MobileNetV2ImageProcessor by @simonreise in #37312
Simplify Mixtral and its modular children by @Cyrilvallez in #39252
fix some flaky tests in tests/generation/test_utils.py by @ydshieh in #39254
Update LED model card by @dross20 in #39233
Glm 4 doc by @zRzRzRzRzRzRzR in #39247
fix xpu failures on PT 2.7 and 2.8 w/o IPEX and enable hqq cases on XPU by @yao-matrix in #39187
Fix license text, duplicate assignment, and typo in constant names by @gudwls215 in #39250
Skip test_eager_matches sdpa generate and update an integration test for blip-like models by @ydshieh in #39248
remove broken block by @molbap in #39255
fix(generation): stop beam search per-instance when heuristic satisfied by @guang-yng in #38778
fix recompiles due to instance key, and deepcopy issues by @ArthurZucker in #39270
Fix errors when use verl to train GLM4.1v model by @kaln27 in #39199
[CI] fix docs by @gante in #39273
[pagged-attention] fix off-by-1 error in pagged attention generation by @kashif in #39258
[smollm3] add tokenizer mapping for smollm3 by @gante in #39271
Refactor PretrainedConfig.__init__ method to make it more explicit by @qubvel in #39158
fix flaky test_generate_compile_model_forward by @ydshieh in #39276
[lightglue] add support for remote code DISK keypoint detector by @sbucaille in #39253
Add torchcodec in docstrings/tests for datasets 4.0 by @lhoestq in #39156
Update T5gemma by @bzhangGo in #39210
[Tests] Update model_id in AIMv2 Tests by @yaswanth19 in #39281
Fix SDPA attention precision issue in Qwen2.5-VL by @JJJYmmm in #37363
[flash attn 3] bring back flags by @zucchini-nlp in #39294
fix aria tests by @ydshieh in #39277
skip test_torchscript_* for now until the majority of the community ask for it by @ydshieh in #39307
[modular] Allow method with the same name in case of @property decorator by @Cyrilvallez in #39308
[sliding window] revert and deprecate by @zucchini-nlp in #39301
🌐 [i18n-KO] Translated quark.md to Korean by @maximizemaxwell in #39268
Fix consistency and a few docstrings warnings by @Cyrilvallez in #39314
add stevhliu to the list in self-comment-ci.yml by @ydshieh in #39315
Updated the Model docs - for the MARIAN model by @emanrissha in #39138
skip files in src/ for doctest (for now) by @ydshieh in #39316
docs: update LLaVA-NeXT model card by @Bpriya42 in #38894
Fix typo: langauge -> language by @tomaarsen in #39317
Granite speech speedups by @avihu111 in #39197
Fix max_length_q and max_length_k types to flash_attn_varlen_func by @HollowMan6 in #37206
enable static cache on TP model by @jiqing-feng in #39164
Fix broken SAM after #39120 by @yonigozlan in #39289
Delete deprecated stuff by @zucchini-nlp in #38838
fix Glm4v batch videos forward by @Kuangdd01 in #39172
fix phi3 tests by @ydshieh in #39312
Handle DAC conversion when using weight_norm with newer PyTorch versions by @edwko in #36393
[modeling][lfm2] LFM2: Remove deprecated seen_tokens by @paulpak58 in #39342
[Core] [Offloading] Enable saving offloaded models with multiple shared tensor groups by @kylesayrs in #39263
Add a default value for position_ids in masking_utils by @Cyrilvallez in #39310
[modular] speedup check_modular_conversion with multiprocessing by @qubvel in #37456
Updated Switch Transformers model card with standardized format (Issue #36979) by @giuseppeCoccia in #39305
Fix link for testpypi by @Cyrilvallez in #39360
update cb TP by @ArthurZucker in #39361
fix failing test_sdpa_can_dispatch_on_flash by @ydshieh in #39259
Verbose error in fix mode for utils/check_docstrings.py by @manueldeprada in #38915
Remove device check in HQQ quantizer by @learning-chip in #39299
Add mistral common support by @juliendenize in #38906
Update Readme to Run Multiple Choice Script from Example Directory by @eromomon in #39323
Updated CamemBERT model card to new standardized format by @MShaheerMalik77 in #39227
fix gpt2 usage doc by @Xiang-cd in #39351
Update Model Card for Encoder Decoder Model by @ParagEkbote in #39272
update docker file to use latest timm (for perception_lm) by @ydshieh in #39380
Fix overriding Fast Image/Video Processors instance attributes affect other instances by @yonigozlan in #39363
[shieldgemma] fix checkpoint loading by @zucchini-nlp in #39348
[BLIP] remove cache from Qformer by @zucchini-nlp in #39335
[Qwen2.5-VL] Fix torch.finfo() TypeError for integer attention_mask_tensor by @dsnsabari in #39333
Deprecate AutoModelForVision2Seq by @zucchini-nlp in #38900
Fix Lfm2 and common tests by @Cyrilvallez in #39398
[examples] fix do_reduce_labels argument for run_semantic_segmentation_no_trainer by @eromomon in #39322
Totally rewrite how pipelines load preprocessors by @Rocketknight1 in #38947
Use np.pad instead of np.lib.pad. by @rasmi in #39346
[Docs] Fix typo in CustomTrainer compute_loss method and adjust loss reduction logic by @MilkClouds in #39391
Update phi4_multimodal.md by @tanuj-rai in #38830
[siglip] fix pooling comment by @sameerajashyam in #39378
Fix typo in /v1/models output payload by @alvarobartt in #39414
support loading qwen3 gguf by @44670 in #38645
Ignore extra position embeddings weights for ESM by @Rocketknight1 in #39063
set document_question_answering pipeline _load_tokenizer to True by @jiqing-feng in #39411
Fix invalid property by @cyyever in #39384
refactor: remove set_tracer_provider and set_meter_provider calls by @McPatate in #39422
Fix bugs from pipeline preprocessor overhaul by @Rocketknight1 in #39425
Fix bugs in pytorch example run_clm when streaming is enabled by @HRezaei in #39286
Remove deprecated audio utils functions by @jiangwangyi in #39330
Remove residual quantization attribute from dequantized models by @DWarez in #39373
handle training summary when creating modelcard but offline mode is set by @winglian in #37095
[vlm] fix loading of retrieval VLMs by @zucchini-nlp in #39242
docs: update SuperGlue docs by @sbucaille in #39406
docs: update LightGlue docs by @sbucaille in #39407
CI workflow for performed test regressions by @ahadnagy in #39198
[autodocstring] add video and audio inputs by @zucchini-nlp in #39420
[Core] [Offloading] Fix saving offloaded submodules by @kylesayrs in #39280
Remove double soft-max in load-balancing loss. Fixes #39055 . by @rudolfwilliam in #39056
Fixed a bug calculating cross entropy loss in JetMoeForCausalLM by @Phoenix-Shen in #37830
[chat template] add a testcase for kwargs by @zucchini-nlp in #39415
Fix L270 - hasattr("moe_args") returning False error by @wjdghks950 in #38715
Defaults to adamw_torch_fused for Pytorch>=2.8 by @cyyever in #37358
Change log level from warning to info for scheduled request logging in ContinuousBatchProcessor by @qgallouedec in #39372
Add cosine_with_min_lr_schedule_with_warmup_lr_rate scheduler in Trainer by @richardodliu in #31870
Fix missing definition of diff_file_url in notification service by @ahadnagy in #39445
add test scanner by @molbap in #39419
Remove runtime conditions for type checking by @cyyever in #37340
docs: add missing numpy import to minimal example by @IliasAarab in #39444
[cache] make all classes cache compatible finally by @zucchini-nlp in #38635
Fix typo in generation configuration for Janus model weight conversion by @thisisiron in #39432
Better typing for model.config by @qubvel in #39132
[Bugfix] [Quantization] Remove unused init arg by @kylesayrs in #39324
Fix processor tests by @zucchini-nlp in #39450
Remove something that should have never been there by @ArthurZucker in #38254
make the loss context manager easier to extend by @winglian in #39321
Fixes #39204: add fallback if get_base_model missing by @sebastianvlad1 in #39226
[CI] Fix partially red CI by @vasqu in #39448
Updated Megatron conversion script for gpt2 checkpoints by @LckyLke in #38969
Fix indentation bug in SmolVLM image processor causing KeyError by @Krish0909 in #39452
fix cached file error when repo type is dataset by @hiyouga in #36909
Improve grammar and clarity in perf_hardware.md by @ridima11 in #39428
create ijepa modelcard (ref : PR #36979 ). by @dhruvmalik007 in #39354
Corrections to PR #38642 and enhancements to Wav2Vec2Processor call and pad docstrings by @renet10 in #38822
fix(pipelines): QA pipeline returns fewer than top_k results in batch mode by @yushi2006 in #39193
fix max_length calculating using cu_seq_lens by @KKZ20 in #39341
Fix tests due to breaking change in accelerate by @SunMarc in #39451
Use newer typing notation by @cyyever in #38934
fix a comment typo in utils.py by @klimarissa17 in #39459
Update GemmaIntegrationTest::test_model_2b_bf16_dola by @ydshieh in #39362
Fix convert_and_export_with_cache failures for GPU models by @Stonepia in #38976
Enable some ruff checks for performance and readability by @cyyever in #39383
fix: ImageTextToTextPipeline handles user-defined generation_config by @peteryschneider in #39374
Update integration_utils.py by @zhaiji0727 in #39469
Add unified logits_to_keep support to LLMClass by @hellopahe in #39472
Fix typing order by @Tavish9 in #39467
[dependencies] temporary pyarrow pin by @gante in #39496
Slack CI bot: set default result for non-existing artifacts by @ahadnagy in #39499
[dependencies] Update datasets pin by @gante in #39500
[chat template] return assistant mask in processors by @zucchini-nlp in #38545
[gemma3] Fix do_convert_rgb in image processors. by @MohitIntel in #39438
Fix BatchEncoding.to() for nested elements by @eginhard in #38985
Add fast image processor SAM by @yonigozlan in #39385
Improve @auto_docstring doc and rename args_doc.py to auto_docstring.py by @yonigozlan in #39439
Update SAM/SAM HQ attention implementation + fix Cuda sync issues by @yonigozlan in #39386
Fix placeholders replacement logic in auto_docstring by @yonigozlan in #39433
[gemma3] support sequence classification task by @zucchini-nlp in #39465
[qwen2 vl] fix packing with all attentions by @zucchini-nlp in #39447
GLM-4 Update by @zRzRzRzRzRzRzR in #39393
Fix bad tensor shape in failing Hubert test. by @ebezzam in #39502
Fix the check in flex test by @Cyrilvallez in #39548
Rename _supports_flash_attn_2 in examples and tests by @zucchini-nlp in #39471
Fix Qwen Omni integration test by @Cyrilvallez in #39553
Fix pylint warnings by @cyyever in #39477
Raise TypeError instead of ValueError for invalid types by @Sai-Suraj-27 in #38660
Fix missing initializations for models created in 2023 by @bvantuan in #39239
use the enable_gqa param in torch.nn.functional.scaled_dot_product_at… by @sywangyi in #39412
Fix Docstring of BarkProcessor by @st81 in #39546
Refactor MambaCache to modeling_mamba.py by @manueldeprada in #38086
fix ndim check of device_mesh for TP by @winglian in #39538
[Fast image processor] refactor fast image processor glm4v by @yonigozlan in #39490
🌐 [i18n-KO] Translated perf_infer_gpu_multi.md to Korean by @luckyvickyricky in #39441
Refactor embedding input/output getter/setter by @molbap in #39339
[Fast image processors] Improve handling of image-like inputs other than images (segmentation_maps) by @yonigozlan in #39489
[CI] Fix post merge ernie 4.5 by @vasqu in #39561
Update modernbertdecoder docs by @orionw in #39453
Update OLMoE model card by @nlhmnlhmnlhm in #39344
[gemma3] fix bidirectional image mask by @zucchini-nlp in #39396
Bump AMD container for 2.7.1 PyTorch by @ahadnagy in #39458
Fixes needed for n-d parallelism and TP by @winglian in #39562
[timm_wrapper] add support for gradient checkpointing by @Yozer in #39287
Add AMD test expectations to DETR model by @ahadnagy in #39539
[docs] update attention implementation and cache docs by @zucchini-nlp in #39547
[docs] Create page on inference servers with transformers backend by @zucchini-nlp in #39550
Add AMD expectations to Mistral3 tests by @ahadnagy in #39481
Add AMD GPU expectations for LLaVA tests by @ahadnagy in #39486
General weight initialization scheme by @Cyrilvallez in #39579
[cache refactor] Move all the caching logic to a per-layer approach by @manueldeprada in #39106
Update docs/source/ko/_toctree.yml by @jungnerd in #39516
updated mistral3 model card by @cassiasamp in #39531
[Paged-Attention] Handle continuous batching for repetition penalty by @kashif in #39457
Torchdec RuntimeError catch by @SunMarc in #39580
Fix link in "Inference server backends" doc by @hmellor in #39589
[WIP] Add OneformerFastImageProcessor by @Player256 in #38343
🎯 Trackio integration by @qgallouedec in #38814
Mask2former & Maskformer Fast Image Processor by @SangbumChoi in #35685
Fix DynamicCache and simplify Cache classes a bit by @Cyrilvallez in #39590
Generic task-specific base classes by @Cyrilvallez in #39584
[Trackio] Allow single-gpu training and monitor power by @qgallouedec in #39595
Rename supports_static_cache to can_compile_fullgraph by @zucchini-nlp in #39505
FP-Quant support by @BlackSamorez in #38696
fix moe routing_weights by @llbdyiu66 in #39581
[idefics3] fix for vLLM by @zucchini-nlp in #39470
enable triton backend on awq xpu by @jiqing-feng in #39443
Allow device_mesh have multiple dim by @S1ro1 in #38949
Fix typos and grammar issues in documentation and code by @cluster2600 in #39598
Fix important models CI by @molbap in #39576
Move openai import by @ebezzam in #39613
Fix DAC integration tests and checkpoint conversion. by @ebezzam in #39313
Feature/standardize opt model card by @JoestarGagan in #39568
standardized YOLOS model card according to template in #36979 by @EthanV431 in #39528
[Docs] Translate audio_classification.md from English to Spanish by @weezymatt in #39513
Update recent processors for vLLM backend by @zucchini-nlp in #39583
[efficientloftr] fix model_id in tests by @sbucaille in #39621
[timm] new timm pin by @gante in #39640
[Voxtral] values for A10 runners by @eustlb in #39605
revert behavior of _prepare_from_posids by @winglian in #39622
Add owlv2 fast processor by @lmarshall12 in #39041
[attention] fix test for packed padfree masking by @zucchini-nlp in #39582
Fix: explicit not none check for tensors in flash attention by @jeffrey-dot-li in #39639
revert change to cu_seqlen_k and max_k when preparing from position_ids by @winglian in #39653
Make pytorch examples UV-compatible by @lhoestq in #39635
[docs] fix ko cache docs by @gante in #39644
make fixup by @gante in #39661
fix(voxtral): correct typo in apply_transcription_request by @rev2607 in #39572
Rename huggingface_cli to hf by @LysandreJik in #39630
🚨[Fast Image Processor] Force Fast Image Processor for Qwen2_VL/2_5_VL + Refactor by @yonigozlan in #39591
Fix ModernBERT Decoder model by @qubvel in #39671
[CI] revert device in test_export_static_cache by @gante in #39662
[Ernie 4.5] Post merge adaptations by @vasqu in #39664
Delete bad rebasing functions by @Cyrilvallez in #39672
Fixes the BC by @ArthurZucker in #39636
fix kyutai tests by @ydshieh in #39416
update expected outputs for whisper after #38778 by @ydshieh in #39304
Add missing flag for CacheLayer by @Cyrilvallez in #39678
Fix auto_docstring crashing when dependencies are missing by @yonigozlan in #39564
fix: HWIO to OIHW by @RyanMullins in #39200
Use auto_docstring for perception_lm fast image processor by @yonigozlan in #39679
bad_words_ids no longer slow on mps by @DWarez in #39556
Support typing.Literal as type of tool parameters or return value by @grf53 in #39633
fix break for ckpt without _tp_plan by @MoyanZitto in #39658
Fix tied weight test by @Cyrilvallez in #39680
Add padding-free to Granite hybrid moe models by @garrett361 in #39677

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@sbucaille
- Update SuperPoint model card (#38896)
- [superglue] fix wrong concatenation which made batching results wrong (#38850)
- [lightglue] add support for remote code DISK keypoint detector (#39253)
- docs: update SuperGlue docs (#39406)
- docs: update LightGlue docs (#39407)
- Add EfficientLoFTR model (#36355)
@yaswanth19
- Cleanup Attention class for Siglip and dependent models (#39040)
- ✨ Add EoMT Model || 🚨 Fix Mask2Former loss calculation (#37610)
- 🚨🚨🚨 [eomt] make EoMT compatible with pipeline (#39122)
- Add Aimv2 model (#36625)
- [Tests] Update model_id in AIMv2 Tests (#39281)
@bvantuan
- Fix initialization of OneFormer (#38901)
- Fix key mapping for VLMs (#39029)
- Fix missing initializations for models created in 2024 (#38987)
- Fix missing initializations for models created in 2023 (#39239)
@NahieliV
- add fast image processor nougat (#37661)
@MinJu-Ha
- Add Fast Image Processor for mobileViT (#37143)
@zRzRzRzRzRzRzR
- Fix some bug for finetune and batch infer For GLM-4.1V (#39090)
- Glm 4 doc (#39247)
- GLM-4 Update (#39393)
@simonreise
- Add segmentation_maps support to MobileNetV2ImageProcessor (#37312)
@LoserCheems
- Add Doge model (#35891)
@VladOS95-cyber
- Add DeepSeek V2 Model into Transformers (#36400)
@paulpak58
- LFM2 (#39340)
- [modeling][lfm2] LFM2: Remove deprecated seen_tokens (#39342)
@shuminghu
- PerceptionLM (#37878)
@juliendenize
- Add mistral common support (#38906)
@orionw
- Add ModernBERT Decoder Models - ModernBERT, but trained with CLM! (#38967)
- Update modernbertdecoder docs (#39453)
@cyyever
- Fix invalid property (#39384)
- Defaults to adamw_torch_fused for Pytorch>=2.8 (#37358)
- Remove runtime conditions for type checking (#37340)
- Use newer typing notation (#38934)
- Enable some ruff checks for performance and readability (#39383)
- Fix pylint warnings (#39477)
@jungnerd
- Update docs/source/ko/_toctree.yml (#39516)
@Player256
- [WIP] Add OneformerFastImageProcessor (#38343)
@SangbumChoi
- Mask2former & Maskformer Fast Image Processor (#35685)
@BlackSamorez
- FP-Quant support (#38696)

Open source →

4.53.026 Jun 2025Breaking

Several minimal breaking changes aiming to bring clearer defaults while greatly simplifying the library have been merged.

Release v4.53.0

Gemma3n

Gemma 3n models are designed for efficient execution on low-resource devices. They are capable of multimodal input, handling text, image, video, and audio input, and generating text outputs, with open weights for pre-trained and instruction-tuned variants. These models were trained with data in over 140 spoken languages.

Gemma 3n models use selective parameter activation technology to reduce resource requirements. This technique allows the models to operate at an effective size of 2B and 4B parameters, which is lower than the total number of parameters they contain. For more information on Gemma 3n's efficient parameter management technology, see the Gemma 3n page.

from transformers import pipeline
import torch

pipe = pipeline(
    "image-text-to-text",
    torch_dtype=torch.bfloat16,
    model="google/gemma-3n-e4b",
    device="cuda",
)
output = pipe(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg",
    text="<image_soft_token> in this image, there is"
)

print(output)

Dia

Dia is an opensource text-to-speech (TTS) model (1.6B parameters) developed by Nari Labs. It can generate highly realistic dialogue from transcript including nonverbal communications such as laughter and coughing. Furthermore, emotion and tone control is also possible via audio conditioning (voice cloning).

Model Architecture: Dia is an encoder-decoder transformer based on the original transformer architecture. However, some more modern features such as rotational positional embeddings (RoPE) are also included. For its text portion (encoder), a byte tokenizer is utilized while for the audio portion (decoder), a pretrained codec model DAC is used - DAC encodes speech into discrete codebook tokens and decodes them back into audio.

Add Dia model by @buttercrab in #38405

Kyutai Speech-to-Text

Kyutai STT is a speech-to-text model architecture based on the Mimi codec, which encodes audio into discrete tokens in a streaming fashion, and a Moshi-like autoregressive decoder. Kyutai’s lab has released two model checkpoints:

kyutai/stt-1b-en_fr: a 1B-parameter model capable of transcribing both English and French
kyutai/stt-2.6b-en: a 2.6B-parameter model focused solely on English, optimized for maximum transcription accuracy

Add kyutai stt by @eustlb in #38909

Read more about the model in the documentation

V-JEPA 2

V-JEPA 2 is a self-supervised approach to training video encoders developed by FAIR, Meta. Using internet-scale video data, V-JEPA 2 attains state-of-the-art performance on motion understanding and human action anticipation tasks. V-JEPA 2-AC is a latent action-conditioned world model post-trained from V-JEPA 2 (using a small amount of robot trajectory interaction data) that solves robot manipulation tasks without environment-specific data collection or task-specific training or calibration.

Add V-JEPA 2 by @qubvel in #38746

Read more about the model in the documentation.

Arcee

Arcee is a decoder-only transformer model based on the Llama architecture with a key modification: it uses ReLU² (ReLU-squared) activation in the MLP blocks instead of SiLU, following recent research showing improved training efficiency with squared activations. This architecture is designed for efficient training and inference while maintaining the proven stability of the Llama design.

The Arcee model is architecturally similar to Llama but uses x * relu(x) in MLP layers for improved gradient flow and is optimized for efficiency in both training and inference scenarios.

Add Arcee model support by @Crystalcareai in #38621

Read more about the model in the documentation.

ColQwen2

ColQwen2 is a variant of the ColPali model designed to retrieve documents by analyzing their visual features. Unlike traditional systems that rely heavily on text extraction and OCR, ColQwen2 treats each page as an image. It uses the Qwen2-VL backbone to capture not only text, but also the layout, tables, charts, and other visual elements to create detailed multi-vector embeddings that can be used for retrieval by computing pairwise late interaction similarity scores. This offers a more comprehensive understanding of documents and enables more efficient and accurate retrieval.

Add ColQwen2 to 🤗 transformers by @tonywu71 in #35778

Read more about the model in the documentation.

MiniMax

MiniMax is a powerful language model with 456 billion total parameters, of which 45.9 billion are activated per token. To better unlock the long context capabilities of the model, MiniMax adopts a hybrid architecture that combines Lightning Attention, Softmax Attention and Mixture-of-Experts (MoE). Leveraging advanced parallel strategies and innovative compute-communication overlap methods—such as Linear Attention Sequence Parallelism Plus (LASP+), varlen ring attention, Expert Tensor Parallel (ETP), etc., MiniMax's training context length is extended to 1 million tokens, and it can handle a context of up to 4 million tokens during the inference. On various academic benchmarks, MiniMax also demonstrates the performance of a top-tier model.

The architecture of MiniMax is briefly described as follows:

Total Parameters: 456B
Activated Parameters per Token: 45.9B
Number Layers: 80
Hybrid Attention: a softmax attention is positioned after every 7 lightning attention.
- Number of attention heads: 64
- Attention head dimension: 128
Mixture of Experts:
- Number of experts: 32
- Expert hidden dimension: 9216
- Top-2 routing strategy
Positional Encoding: Rotary Position Embedding (RoPE) applied to half of the attention head dimension with a base frequency of 10,000,000
Hidden Size: 6144
Vocab Size: 200,064

For more details refer to the release blog post.

Add support for MiniMax's MiniMax-Text-01 by @geetu040 in #35831

Read more about the model in the documentation.

Encoder-Decoder Gemma

T5Gemma (aka encoder-decoder Gemma) was proposed in a research paper by Google. It is a family of encoder-decoder large langauge models, developed by adapting pretrained decoder-only models into encoder-decoder. T5Gemma includes pretrained and instruction-tuned variants. The architecture is based on transformer encoder-decoder design following T5, with improvements from Gemma 2: GQA, RoPE, GeGLU activation, RMSNorm, and interleaved local/global attention.

T5Gemma has two groups of model sizes: 1) Gemma 2 sizes (2B-2B, 9B-2B, and 9B-9B), which are based on the offical Gemma 2 models (2B and 9B); and 2) T5 sizes (Small, Base, Large, and XL), where are pretrained under the Gemma 2 framework following T5 configuration. In addition, we also provide a model at ML size (medium large, ~2B in total), which is in-between T5 Large and T5 XL.

The pretrained varaints are trained with two objectives: prefix language modeling with knowledge distillation (PrefixLM) and UL2, separately. We release both variants for each model size. The instruction-turned varaints was post-trained with supervised fine-tuning and reinforcement learning.

Encoder-Decoder Gemma by @bzhangGo in #38332

Read more about the model in the documentation.

GLM-4.1V

The GLM-4.1V model architecture is added to transformers; no models have yet been released with that architecture. Stay tuned for the GLM team upcoming releases!

GLM-4.1V Model support by @zRzRzRzRzRzRzR in #38431

Read more about the model in the documentation.

Falcon H1

The FalconH1 model was developed by the TII Pretraining team. A comprehensive research paper covering the architecture, pretraining dynamics, experimental results, and conclusions is forthcoming. You can read more about this series in this website.

[MODEL] Add Falcon H1 by @younesbelkada in #38249

Read more about the model in the documentation.

LightGlue

The LightGlue model was proposed in LightGlue: Local Feature Matching at Light Speed by Philipp Lindenberger, Paul-Edouard Sarlin and Marc Pollefeys.

Similar to SuperGlue, this model consists of matching two sets of local features extracted from two images, its goal is to be faster than SuperGlue. Paired with the SuperPoint model, it can be used to match two images and estimate the pose between them. This model is useful for tasks such as image matching, homography estimation, etc.

The abstract from the paper is the following:

We introduce LightGlue, a deep neural network that learns to match local features across images. We revisit multiple design decisions of SuperGlue, the state of the art in sparse matching, and derive simple but effective improvements. Cumulatively, they make LightGlue more efficient - in terms of both memory and computation, more accurate, and much easier to train. One key property is that LightGlue is adaptive to the difficulty of the problem: the inference is much faster on image pairs that are intuitively easy to match, for example because of a larger visual overlap or limited appearance change. This opens up exciting prospects for deploying deep matchers in latency-sensitive applications like 3D reconstruction. The code and trained models are publicly available at this https URL

Add LightGlue model by @sbucaille in #31718

Read more about the model in the documentation.

dots.llm1

The abstract from the report is the following:

Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this report, we present dots.llm1, a large-scale MoE model that activates 14B parameters out of a total of 142B parameters, delivering performance on par with state-of-the-art models while reducing training and inference costs. Leveraging our meticulously crafted and efficient data processing pipeline, dots.llm1 achieves performance comparable to Qwen2.5-72B after pretraining on high-quality corpus and post-training to fully unlock its capabilities. Notably, no synthetic data is used during pretraining. To foster further research, we open-source intermediate training checkpoints spanning the entire training process, providing valuable insights into the learning dynamics of large language models.

[Model] add dots1 by @redmoe-moutain in #38143

Read more about the model in the documentation.

SmolLM3

SmolLM3 is a fully open, compact language model designed for efficient deployment while maintaining strong performance. It uses a Transformer decoder architecture with Grouped Query Attention (GQA) to reduce the kv cache, and no RoPE, enabling improved performance on long-context tasks. It is trained using a multi-stage training approach on high-quality public datasets across web, code, and math domains. The model is multilingual and supports very large context lengths. The instruct variant is optimized for reasoning and tool use.

Add SmolLM3 by @anton-l in #38755

Read more about the model in the documentation.

Performance optimizations

Kernels

In previous versions, installing the kernels library would automatically activate the custom kernels added to transformers, because the @use_kernel_forward_from_the_hub decorator directly swapped out the model’s forward method. This implicit behavior caused several issues for users — including problems with torch.compile, non-determinism, and inconsistent outputs.

To address this, we've introduced a new opt-in mechanism called kernelize. You can now enable kernel usage explicitly by passing use_kernels=True to from_pretrained. The use_kernel_forward_from_the_hub decorator now simply stores the kernel name that the user wants to use — and kernelize handles the rest under the hood.

Example

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    use_kernels=True
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")

input = "Hello"
input_ids = tokenizer(input, return_tensors="pt").to(model.device).input_ids
output = model.generate(input_ids, max_new_tokens=100)

print(tokenizer.decode(output[0], skip_special_tokens=True))

More kernels will be added over time — this will be a collaborative, community-driven effort to make transformers lighter and faster 🤗

Add kernelize to transformers by @MekkCyber in #38205

Flash Attention 3

Support for Flash Attention 3 is added across the most popular models.

Support for Flash Attention 3 by @EduardDurech in #38972

Notable repository maintenance & refactors

Several efforts refactoring the repository are happening in parallel. The direction is to greatly simplify the library, removing unnecessary codepaths. Whilst the efforts are spread across the library, they're particularly visible in each individual models; where non-modeling-specific code will be simplified and eventually removed.

We take the assumption that model-agnostic utilities shouldn't be in the modeling code. Things like the output of attentions, hidden states, router logits, are important for end-users but don't need to be explicitely displayed in the modeling code.

Apply GradientCheckpointingLayer to the whole repo by @qubvel in #38913
No more Tuple, List, Dict by @Rocketknight1 in #38797
Deprecate TF + JAX by @Rocketknight1 in #38758

Breaking changes

Several minimal breaking changes aiming to bring clearer defaults while greatly simplifying the library have been merged.

🔴 Update default dtype for pipelines to auto by @Vaibhavs10 in #38882
🚨🚨 Fix initialization of Mask2Former by @Cyrilvallez in #38864
:rotating_light: :rotating_light: Inherited CausalLM Tests by @Rocketknight1 in #37590
🚨Early-error🚨 config will error out if output_attentions=True and the attn implementation is wrong by @ArthurZucker in #38288
🔴 [VLM] modeling updates by @zucchini-nlp in #38317
:rotating_light: :rotating_light: Fix custom code saving by @Rocketknight1 in #37716
🚨🚨[core] Completely rewrite the masking logic for all attentions by @Cyrilvallez in #37866
🔴🔴🔴 [Attention] Refactor Attention Interface for Bart-based Models by @vasqu in #38108
🔴[Attention] Attention refactor for Whisper-based models by @vasqu in #38235
Add CB by @ArthurZucker in #38085

Bugfixes and improvements

CI reporting improvements by @ydshieh in #38230
Revert parallelism temporarily by @LysandreJik in #38240
tp plan should not be NONE by @ArthurZucker in #38255
[Falcon H1] Fix Typo in Integration Test by @dhiaEddineRhaiem in #38256
[compile] re-enable for Qwen-VL models by @zucchini-nlp in #38127
fix multi-image case for llava-onevision by @cyr0930 in #38084
Add tearDown method to Quark to solve OOM issues by @MekkCyber in #38234
Clearer error on import failure by @LysandreJik in #38257
[whisper] small changes for faster tests by @gante in #38236
Simplify DTensor Check for modeling_utils.py by @amd-xiaoyu12 in #38245
Improve typing in TrainingArgument by @cyyever in #36944
Fix: missing else branch to handle "--load_best_model_at_end" in training_args.py by @danielyxyang in #38217
assign the correct torchao data layout for xpu by @jiqing-feng in #37781
Remove Japanese sequence_classification doc and update references by @ritsumei-aoi in #38246
Protect ParallelInterface by @ArthurZucker in #38262
Update Model Card for Mamba by @ParagEkbote in #37863
docs(swin): Update Swin model card to standard format by @BryanBradfo in #37628
add XPU info print in print_env by @yao-matrix in #38282
[whisper] move processor test into processor test file 🧹 by @gante in #38266
[Whisper] handle deprecation of forced_decoder_ids by @gante in #38232
add liger-kernel to docker file by @ydshieh in #38292
Fix tp error when torch distributed is already initialized by @SunMarc in #38294
More typing in src/transformers/training_args.py by @cyyever in #38106
refine transformers env output by @yao-matrix in #38274
Update CI Docker base image for AMD tests by @ahadnagy in #38261
Fix HybridChunedCache & Llama4 by @Cyrilvallez in #38299
Oups typo for HybridChunkedCache by @Cyrilvallez in #38303
[Tests] Cleanup Janus Testcase by @yaswanth19 in #38311
[emu3] fix conversion script by @zucchini-nlp in #38297
Fix run_slow by @cyyever in #38314
Fix typo: change 'env' to 'environment' in .circleci/config.yml by @AbdessamadEnabih in #38273
Adds use_repr to model_addition_debugger_context by @RyanMullins in #37984
[tf/flax] handle forced_decoder_ids deletion by @gante in #38316
[Whisper + beam search] fix usage of beam_indices by @gante in #38259
Expose AutoModelForTimeSeriesPrediction for import by @jinan-zhou in #38307
[custom_generate] don't forward custom_generate and trust_remote_code by @gante in #38304
add vasqu to self-comment-ci.yml by @ydshieh in #38324
Fix some tests (especially compile with fullgraph=True on Python<3.11) by @Cyrilvallez in #38319
[performance_optim] reduce frequency of declaring attention_mask in Ascend NPU flash attention by @FightingZhen in #38278
refactor can_save_slow_tokenizer by @itazap in #37722
[FlexAttention] Reenable flex for encoder-decoder and make the test more robust by @vasqu in #38321
Enhance Model Loading By Providing Parallelism, Uses Optional Env Flag by @inf3rnus in #36835
Use Gradient Checkpointing Layer in Jamba & Blip Related Models by @alex-jw-brooks in #38310
Never fallback to eager implicitly by @Cyrilvallez in #38327
Remove duplicate docstring: resample by @qqii in #38305
Update BioGPT model card by @Aguedoom in #38214
docs(swinv2): Update SwinV2 model card to new standard format by @BryanBradfo in #37942
[docs]: update roformer.md model card by @KsuParkhamchuk in #37946
new failure CI reports for all jobs by @ydshieh in #38298
Hot fix for AMD CI workflow by @ydshieh in #38349
Uninstall kernels for AMD docker images by @ydshieh in #38354
[VLMs] add helpers for get/set embedding by @zucchini-nlp in #38144
switch to device agnostic device calling for test cases by @yao-matrix in #38247
[OPT] Fix attention scaling by @vasqu in #38290
Fix all import errors based on older torch versions by @Cyrilvallez in #38370
Fix incorrect batching audio index calculation for Phi-4-Multimodal by @Isotr0py in #38103
Protect get_default_device for torch<2.3 by @Cyrilvallez in #38376
[Falcon H1] Fix slow path forward pass by @dhiaEddineRhaiem in #38320
Improved cache docs by @manueldeprada in #38060
for now disable compile by @ArthurZucker in #38383
Use one utils/notification_service.py by @ydshieh in #38379
Better check in initialize_weights by @Cyrilvallez in #38382
fix typos by @DeVikingMark in #38336
fix typo: tokenizer -> tokenize by @foldl in #38357
Stop TF weight rename reDOS by @Rocketknight1 in #38325
[cli] cli usable without torch by @gante in #38386
update gemma tests by @ydshieh in #38384
Stop autoconverting custom code checkpoints by @Rocketknight1 in #37751
Add AMD MI300 CI caller leveraging self-hosted runner scale set workflow in hf-workflows by @jitesh-gupta in #38132
Fix image token mask in Gemma3 by @Cyrilvallez in #38295
[transformers x vLLM] standardize processors by @zucchini-nlp in #37915
[paligemma] fix processor with suffix by @zucchini-nlp in #38365
[video utils] group and reorder by number of frames by @zucchini-nlp in #38374
[aya vision] fix processor for vLLM by @zucchini-nlp in #38371
guard size mismatch check to only quantized models by @SunMarc in #38397
[chat] improvements for thinking models and reduce default verbosity by @gante in #38322
Fix convert to original state dict for VLMs by @hiyouga in #38385
[chat] use the checkpoint's generation_config.json as base parameterization by @gante in #38330
Fix Qwen2.5-VL Video Processor by @yeliudev in #38366
[CSM] infer codec model with no_grad + audio eos label by @eustlb in #38215
Add report_repo_id to mi300 workflow by @ivarflakstad in #38401
[CSM] update model id by @eustlb in #38211
[cleanup] delete deprecated kwargs in qwen2_audio 🧹 by @gante in #38404
[tests] remove overload for deleted test (test_offloaded_cache_implementation) by @gante in #37896
[mllama] Allow pixel_values with inputs_embeds by @dxoigmn in #38334
Update Model Card for Mamba-2 by @ParagEkbote in #37951
Updated Zoedepth model card by @miniMaddy in #37898
Updated BigBird Model card as per #36979. by @RogerSinghChugh in #37959
Updated BERTweet model card. by @RogerSinghChugh in #37981
New bart model card by @RogerSinghChugh in #37858
Update granite.md by @Tanuj-rai in #37791
Falcon-H1 - Fix auto_docstring and add can_return_tuple decorator by @yonigozlan in #38260
Updated model card for OLMo2 by @andyvu923 in #38394
Add mi300 to amd daily ci workflows definition by @ivarflakstad in #38415
Change slack channel for mi250 CI by @ivarflakstad in #38410
Fix an error in verify_tp_plan for keys without '.' by @liwii in #38420
[qwen-vl] Look for vocab size in text config by @zucchini-nlp in #38372
Update CsmForConditionalGenerationIntegrationTest by @ydshieh in #38424
enable large_gpu and torchao cases on XPU by @yao-matrix in #38355
Disable mi210 scheduled CI by @ivarflakstad in #38411
Update error when using additional and/or masks by @Cyrilvallez in #38429
Fix CircleCI not triggered when PR is opened from a branch of huggingface/transformers by @ydshieh in #38413
make Llama4TextMoe forward more readable by @JJJYmmm in #37529
[core] support tensor-valued _extra_state values in from_pretrained by @pstjohn in #38155
Fix typo in tokenization_utils_base.py docstring by @cwngan in #38418
Fix convert weights for InternVL by @yonigozlan in #38233
Trigger doc-builder job after style bot by @ydshieh in #38398
Remove redundant test_sdpa_equivalence test by @Rocketknight1 in #38436
Fix MoE gradient test by @Rocketknight1 in #38438
Fix from_args_and_dict ProcessorMixin by @yonigozlan in #38296
Fix handling of slow/fast image processors in image_processing_auto.py by @yonigozlan in #38161
Updated the Model docs - for the ALIGN model by @1himan in #38072
Updated the model card for ViTMAE by @mreraser in #38302
Model card for mobilenet v1 and v2 by @yuanjua in #37948
Merge type hints from microsoft/python-type-stubs (post dropping support for Python 3.8) by @Avasam in #38335
Fix GLM4 checkpoints by @ydshieh in #38412
feat: add cache retention for requests by @McPatate in #38446
[Tests] Clean up test cases for few models by @yaswanth19 in #38315
Fix TypeError in save_pretrained error handling (fixes #38422) by @rahulrshetty45 in #38449
Cleanup BatchFeature and BatchEncoding by @lgeiger in #38459
Fix Gemma3IntegrationTest by @ydshieh in #38471
[Qwen2.5-Omni] Fix dtype of cos,sin when used with flash attention by @HarryHsing in #38453
fix: handle no scheduler passed by user by @McPatate in #38407
make it go brrrr by @ArthurZucker in #38409
Fix convert_internvl_weights_to_hf.py to support local paths by @xvyv99 in #38264
Fix incorrect bbox_embed initialization when decoder_bbox_embed_share=False in GroundingDINO by @islemyakoubi in #38238
[Tests] Reduced model size for albert-test model by @saqlain2204 in #38480
Align TP check by @SunMarc in #38328
protect dtensor import by @SunMarc in #38496
[docs] add xpu environment variable for gpu selection by @faaany in #38194
Remove deprecated use_flash_attention_2 parameter by @cyyever in #37131
Fix setting FLASH_ATTENTION_DETERMINISTIC after importing by @HollowMan6 in #37185
[seamless_m4t] Skip some tests when speech is not available by @remi-or in #38430
Update Loss Functions to Accept Tensor num_items_in_batch by @NEREUScode in #38029
[generate] add soft deprecations on custom generation methods by @gante in #38406
[generate] move SinkCache to a custom_generate repo by @gante in #38399
remove unhandled parameter by @itazap in #38145
Fix amp deprecation issue by @SunMarc in #38100
[flax/mistral] support sliding_window: null in config by @yiding in #37402
Num parameters in model.safetensors.index.json by @LysandreJik in #38531
Remove type annotation in Siglip Attention Module by @yaswanth19 in #38503
Fix Gemma2IntegrationTest by @ydshieh in #38492
Fix blip2 tests by @ydshieh in #38510
[tests] expand flex-attn test for vision models by @zucchini-nlp in #38434
Don't use default attn if pre-set in sub-config by @zucchini-nlp in #38526
update emu3 test by @jiqing-feng in #38543
Update docker image to use av by @ydshieh in #38548
[bugfix] [WIP] fix apply_rotary_emb error on Ascend NPU by @FightingZhen in #38491
[TP] Change command in tests to python3 by @S1ro1 in #38555
Explicitly setting encoding in tokenization_utils_base.py by @Muqi1029 in #38553
Fix utils/notification_service.py by @ydshieh in #38556
Name change AOPermod -> ModuleFqn by @drisspg in #38456
Fix hqq issue by @SunMarc in #38551
[docs] Format fix by @stevhliu in #38414
[janus] Fix failing tests on mi3XX by @remi-or in #38426
Fix chameleon tests by @ydshieh in #38565
update utils/notification_service.py for AMD vs Nvidia by @ydshieh in #38563
Fix deepseekv3 by @ydshieh in #38562
[FlexAttn] Fix models with unique characteristics by @vasqu in #38433
fix(attention_visualizer): add default value for image_seq_length by @IceGiraffe in #38577
allow custom head_dim for qwen2_moe by @bzantium in #37188
Docs: fix code formatting in torchao docs by @Manalelaidouni in #38504
feat: add repository field to benchmarks table by @McPatate in #38582
[Dinov2] Enable device_map="auto" support by @aryanchauhan31 in #38487
tests/roformer: fix couple roformer tests on gpus by @dvrogozh in #38570
New gpt neo model card by @RogerSinghChugh in #38505
Updated deprecated typing imports with equivalents for Python 3.9+ by @Sai-Suraj-27 in #38546
added fast image processor for ZoeDepth and expanded tests accordingly by @henrikm11 in #38515
[qwen-omni] fix sliding window by @zucchini-nlp in #38525
Remove custom pytest and pluggy by @ydshieh in #38589
pin pandas by @ydshieh in #38605
Allow mlm_probability to be set to None when mlm=False in DataCollatorForLanguageModeling by @KameniAlexNea in #38522)
Avoid overwrite existing local implementation when loading remote custom model by @Isotr0py in #38474
fix spelling errors by @davidjsonn in #38608
Remove isort from dependencies by @Sai-Suraj-27 in #38616
Fix return_dict=False giving errors in a few VLM models by @ydshieh in #38519
docs: fix dark mode logo display. by @johncaged in #38586
Fix typo in LLaVa documentation by @mynameismon in #38618
[Nit] Add Note on SigOpt being in Public Archive Mode by @ParagEkbote in #38610
Updated Aria model card by @1himan in #38472
Fix MiniMax (docs and integration tests checkpoint) by @geetu040 in #38575
enable more test cases on xpu by @yao-matrix in #38572
Improve test_initialization by @ydshieh in #38607
Use torch 2.7.1 on CircleCI jobs by @ydshieh in #37856
[generation] bring back tests on vision models by @zucchini-nlp in #38603
update ColQwen2ModelIntegrationTest by @ydshieh in #38583
Improve test_initialization for SwiftFormer by @ydshieh in #38636
fix: support grad clipping for TP through replicating non-sharded modules by @kmehant in #36132
Don't run AriaForConditionalGenerationModelTest on CircleCI by @ydshieh in #38615
fix total batch size calculation in trainer by @inkcherry in #38286
fix torch_dtype on awq by @jiqing-feng in #38463
Better CI by @ydshieh in #38552
remove ipex_optimize_model usage by @yao-matrix in #38632
Skip torchscript tests for 2 models by @ydshieh in #38643
Fix InternVL integration test by @ydshieh in #38612
Use torch 2.7.1 on daily CI by @ydshieh in #38620
Fix qwen2-audio chat template audio placeholder insertion by @Isotr0py in #38640
Fixed modeling_auto.py MODEL_FOR_MASK_GENERATION_MAPPING_NAMES variable by @sbucaille in #38664
fix: "check out" as verb by @DePasqualeOrg in #38678
Fix attention mask expansion when converting to executorch by @pweglik in #38637
Fix some models import by @nicelulu in #38694
Fix retrieve function signature and remove faiss requirement by @Fiona-Waters in #38624
Fix TypeError: 'NoneType' object is not iterable for esm by @dbleyl in #38667)
Docs: update bitsandbytes torch.compile compatibility by @matthewdouglas in #38651
Drop as_target_processor from the call and pad methods by @marcndo in #38642
Created model card for XLM model by @AshAnand34 in #38595
Update XLM-RoBERTa model documentation with enhanced usage examples and improved layout by @AshAnand34 in #38596
Created model card for xlm-roberta-xl by @AshAnand34 in #38597
Fix aya_vision test by @ydshieh in #38674
Standardize ByT5 model card format by @yanamis in #38699
Fix smart resize by @rdonggroq in #38706
Update some tests for torch 2.7.1 by @ydshieh in #38701
Logging message for is_bitsandbytes_available() by @ved1beta in #38528
Fix llava tests by @ydshieh in #38722
Use OSError by @cyyever in #38712
[add-new-model-like] Robust search & proper outer '),' in tokenizer mapping by @alexzms in #38703
Fix typo in Language Modeling example scripts and update TPU type by @framoncg in #38652
Add AGENTS.md by @Rocketknight1 in #38734
New canine model card by @RogerSinghChugh in #38631
Fixed a multiple-devices issue in SmolVLM model by @remi-or in #38736
[llava] fix integration tests with Siglip by @zucchini-nlp in #38732
fix: Add method to get image features in PaliGemmaForConditionalGeneration by @YushunXiang in #38730
from 1.11.0, torchao.prototype.low_bit_optim is promoted to torchao.optim by @yao-matrix in #38689
fix: bf16 with TPU is allowed in configuration by @yevvonlim in #38670
[DeepSeek-V3] implement when q_lora_rank is None by @bzantium in #38743
Revert "Trigger doc-builder job after style bot" by @ydshieh in #38735
Add z-loss to Bamba for v2 by @daviswer in #37842
Better typing for num_items_in_batch by @SunMarc in #38728
Prepare for TF+Jax deprecation by @Rocketknight1 in #38760
Remove IPEX requirement for bitsandbytes on CPU by @matthewdouglas in #38594
Update repo consistency check by @Rocketknight1 in #38763
fix(qwen3_moe): pass kwargs to self_attn by @llllvvuu in #38691
Update pegasus model card by @dross20 in #38675
Make style bot trigger CI after push by @ydshieh in #38754
chore(pixtral): emit block attention mask when using flash attention by @starcatmeow in #38741
Update altCLIP model card by @EmileAydar in #38306
Add Qwen2 MoE model card by @rileyafox in #38649
[masking utils] check None instead of try/except by @zucchini-nlp in #38561
[Hotfix] Fix style bot by @ydshieh in #38779
Fix masking utils by @Cyrilvallez in #38783
[video processors] support frame sampling within processors by @zucchini-nlp in #38105
Skip some export tests on torch 2.7 by @ydshieh in #38677
Reduce verbosity for average_tokens_across_devices=True and world size = 1 by @qgallouedec in #38785
Update PULL_REQUEST_TEMPLATE.md by @qgallouedec in #38770
[docs] Add int4wo + 2:4 sparsity example to TorchAO README by @jcaip in #38592
Fix qwen_2_5 omni by @ydshieh in #38658
Fix llava_onevision tests by @ydshieh in #38791
Reword README in light of model definitions by @LysandreJik in #38762
Fix Typos in Comments: "quantitation" → "quantization", "averege" → "average" by @leopardracer in #38766
Initialize flash attn flag by @farnasirim in #38768
Fix mllama by @ydshieh in #38704
build: :pushpin: Remove upper bound on PyTorch by @KyleMylonakisProtopia in #38789
Remove all traces of low_cpu_mem_usage by @Cyrilvallez in #38792
[Docs] New DiT model card by @yushi2006 in #38721
Add missing div in Pegasus model card by @dross20 in #38773
Updated moonshine modelcard by @SohamPrabhu in #38711
refactor create_token_type_ids_from_sequences by @itazap in #37681
[docs] update cache docs with new info by @zucchini-nlp in #38775
Fix erroneous docstring for the ordering of SWA layers by @norpadon in #38794
Fix configs and doc for the Qwens by @Cyrilvallez in #38808
Unbreak optimum-executorch by @guangy10 in #38646
Disable custom MRA kernels for ROCm by @ahadnagy in #38738
Use HF papers by @qgallouedec in #38184
Simplify and update trl examples by @qgallouedec in #38772
Better pipeline type hints ✨ by @qubvel in #38049
Fix llava_next tests by @ydshieh in #38813
Expectation fixes and added AMD expectations by @remi-or in #38729
Use wandb.run.url instead of wandb.run.get_url() (deprecated) by @qgallouedec in #38817
Refactor DBRX tests to use CausalLMModelTest base classes by @Rocketknight1 in #38475
change fsdp_strategy to fsdp in TrainingArguments in accelerate doc by @PT-10 in #38807
Fix a minor security issue by @ydshieh in #38815
Fix trainer.py not showing signature columns by @nenesekai in #38465
Add V-JEPA for video classification model by @qubvel in #38788
fixed docstring in modular_qwen2_5_vl.py by @lawrencefeng17 in #38798
[docs] Update docs moved to the course by @stevhliu in #38800
[docs] updated roberta model card by @allmight05 in #38777
Updated Albert model Card by @souvikchand in #37753
[internvl] fix video inference by @zucchini-nlp in #38811
Fix redundant code in Janus by @yaswanth19 in #38826
bugfix: propage weight key_mapping to peft to fix 3.52 VLM renaming by @ManuelFay in #38627
Fix peft integration by @Cyrilvallez in #38841
Fix broken notebooks link in Italian training docs by @VolodymyrBg in #38834
Fix broken tag in Longformer model card by @dross20 in #38828
[BugFix] QA pipeline edge case: align_to_words=True in QuestionAnsweringPipeline can lead to duplicate answers by @yushi2006 in #38761
GraniteMoeHybrid: Allow for only shared expert case. by @shawntan in #38801
Updated aya_vision.md by @1himan in #38749
Remove merge conflict artifacts in Albert model doc by @druvdub in #38849
[video processor] fix BC when no video config if found by @zucchini-nlp in #38840
Fix incorrect width ratio calculation in Llama4 image processor by @Jingxiang-Zhang in #38842
Allow customization of sdpa in executorch.py by @kimishpatel in #38827
Fix qwen2_5_vl tests by @ydshieh in #38845
Improve auxiliary_in_channels default behavior in UperNet by @simonreise in #37540
Fix qwen3 tests by @ydshieh in #38862
Update CvT documentation with improved usage examples and additional … by @sezan92 in #38731
Update roc bert docs by @SohamPrabhu in #38835
Post-PR fixes! by @Rocketknight1 in #38868
enable misc test cases on XPU by @yao-matrix in #38852
Fix phi4_multimodal tests by @ydshieh in #38816
Fix qwen3_moe tests by @ydshieh in #38865
Fix HQQ model param device transfer issue by @HighCWu in #38466
Fixed markdown for BertTokenizer's '[CLS]' token. by @eu90h in #38506
null deepspeed_plugin in args for wandb callback fake trainer by @winglian in #38867
More PYUP fixes by @cyyever in #38883
Fix loop var naming by @Rocketknight1 in #38885
[bugfix] fix ATTN_MASK_NPU device mismatch error on multi-device NPU … by @qykong in #38876
log: Add logging when using split_batches and per_device_train_batch_size by @KeshavSingh29 in #38633
Docs: Add custom fine-tuning tutorial to TrOCR model page by @Ashutosh-4485 in #38847
36978 | Fast image processor for DPT model by @samrae7 in #37481
[video processor] fix slow tests by @zucchini-nlp in #38881
Update bamba model card by @druvdub in #38853
Add support for specifying revisions when pushing to Hub via internal Trainer call by @IsaacBreen in #36852
Use raise from e in hub.py utility by @Wauplin in #37241
[phi-4] use mel filters from audio utils by @eustlb in #36966
Fix fsmt tests by @ydshieh in #38904
Fix unnecessary super calls by @cyyever in #38897
align xpu's autocast behavior w/ cuda by using device agnostic torch APIs by @yao-matrix in #38284
Fix FalconMambaIntegrationTests by @ydshieh in #38566
Skip sdpa tests if submodule does not support sdpa by @ivarflakstad in #38907
Fix ReDOS in tokenizer digit substitution by @Rocketknight1 in #38844
feat: Add granite architectures to auto tokenizer name mappings by @gabe-l-hart in #38802
Allow make-fixup on main branch, albeit slowly by @Rocketknight1 in #38892
feat: add flexible Liger Kernel configuration to TrainingArguments by @hamza-hcompany in #38911
Remove deprecated classes in modeling_utils.py by @Cyrilvallez in #38919
Skip some tests for now by @ydshieh in #38931
Modernbert fixes by @remi-or in #38912
add pytorch-xpu Dockerfile by @yao-matrix in #38875
Remove ALL_LAYERNORM_LAYERS by @Cyrilvallez in #38922
[static cache] fix device map per layer in VLMs by @zucchini-nlp in #38488
Add kwargs for timm.create_model in TimmWrapper by @qubvel in #38860
Pin PyTorch extras for AMD containers by @ahadnagy in #38941
Correctly raise error for awq quantization by @Cyrilvallez in #38945
Fix more flaky test_initialization by @ydshieh in #38932
Switch to use A10 progressively by @ydshieh in #38936
Fix custom generate from local directory by @manueldeprada in #38916
Update blip model card by @devkade in #38513
Gaudi3 CI by @IlyasMoutawwakil in #38790
Fix DTensor import compatibility for PyTorch < 2.5 by @Benoqtr in #38836
Fix(informer): Correct tensor shape for input_size=1 by @Flink-ddd in #38856
[modular] CLI allows positional arguments, and more defaults names for the optional arg by @Cyrilvallez in #38979
Remove dead protected imports by @Cyrilvallez in #38980
Break tie in Expectations and gemma3 fixes by @remi-or in #38943
Add Idefics2/3 and SmolVLM Fast image processors + improvements for fast image processors by @yonigozlan in #38157
fix: add bool operator to tokenizer to avoid bloated asserts by @kallewoof in #38899
Add support for auto_docstring with model outputs by @yonigozlan in #38242
fix mistral and mistral3 tests by @ydshieh in #38978
[Feature] Support is_split_into_words in the TokenClassificationPipeline. by @yushi2006 in #38818
Fix rag by @ydshieh in #38585
[docs] Typos - Single GPU efficient training features by @casinca in #38964
[qwen] refactor attentions for vision/audio by @zucchini-nlp in #38930
Removing extra space in large command for speech-pretraining example by @dggaytan in #38705
[Attention] Small fix on output attentions by @vasqu in #38948
Fixes for Arcee model by @Cyrilvallez in #39001
Added scikit-learn to the example image-classification requirements.txt by @mylonjones in #37506
Update attention_visualizer.py by @Tanuj-rai in #37860
Skip non-selected experts for qwen3_moe by @seven-mile in #38133
Fix undeterministic order in modular dependencies by @Cyrilvallez in #39005
Granite speech - minor fixes to support training with the HF trainer by @avihu111 in #38833
Fix bugs in DynamicCache by @tugsbayasgalan in #37880
Update self-comment-ci.yml user list by @ivarflakstad in #39014
Skip sdpa dispatch on flash test due to unsupported head dims by @ivarflakstad in #39010
[HPU][Critical Issue Fix] ThreadPool instead of Pool for parallel pre-processing by @dsmertin in #39002
Add Hugging Face authentication procedure for IDEs (PyCharm, VS Code,… by @marcndo in #38954
[LightGlue] Fixed attribute usage from descriptor_dim to keypoint_detector_descriptor_dim by @sbucaille in #39021
Add zero dim tensor check when using flash_attention by @ranzhejiang in #38280
Fix graph break in torch.compile when using FA2 with attention_mask=None and batch size > 1 by @efsotr in #37332
[AutoModelForMaskGeneration] Remove duplicate code by @NielsRogge in #38622
[video processor] support torchcodec and decrease cuda memory usage by @zucchini-nlp in #38880
Drop unnecessary tokens in GPT2Model generation by @null-pointer-access in #39016
Fix the seamless_m4t cannot work on Gaudi by @yuanwu2017 in #38363
fix: astronomical loss with ModernBERT when using gradient checkpointing by @umarbutler in #38982)
fix gemma3 grad acc by @SunMarc in #37208
Remove script datasets in tests by @lhoestq in #38940
Fix grammatical error in models documentation by @marcndo in #39019
refactor: remove custom BarkLayerNorm by @eginhard in #39003
[Kyutai-STT] correct model type + model id by @eustlb in #39035
Two ReDOS fixes by @Rocketknight1 in #39013
[tests] remove TF tests (uses of require_tf) by @gante in #38944
Granite speech speedup + model saving bugfix by @avihu111 in #39028
Fix Bad Outputs in Fast Path for GraniteMoeHybrid by @alex-jw-brooks in #39033

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@ydshieh
- CI reporting improvements (#38230)
- add liger-kernel to docker file (#38292)
- add vasqu to self-comment-ci.yml (#38324)
- new failure CI reports for all jobs (#38298)
- Hot fix for AMD CI workflow (#38349)
- Uninstall kernels for AMD docker images (#38354)
- Use one utils/notification_service.py (#38379)
- update gemma tests (#38384)
- Update CsmForConditionalGenerationIntegrationTest (#38424)
- Fix CircleCI not triggered when PR is opened from a branch of huggingface/transformers (#38413)
- Trigger doc-builder job after style bot (#38398)
- Fix GLM4 checkpoints (#38412)
- Fix Gemma3IntegrationTest (#38471)
- Fix Gemma2IntegrationTest (#38492)
- Fix blip2 tests (#38510)
- Update docker image to use av (#38548)
- Fix utils/notification_service.py (#38556)
- Fix chameleon tests (#38565)
- update utils/notification_service.py for AMD vs Nvidia (#38563)
- Fix deepseekv3 (#38562)
- Remove custom pytest and pluggy (#38589)
- pin pandas (#38605)
- Fix return_dict=False giving errors in a few VLM models (#38519)
- Improve test_initialization (#38607)
- Use torch 2.7.1 on CircleCI jobs (#37856)
- update ColQwen2ModelIntegrationTest (#38583)
- Improve test_initialization for SwiftFormer (#38636)
- Don't run AriaForConditionalGenerationModelTest on CircleCI (#38615)
- Better CI (#38552)
- Skip torchscript tests for 2 models (#38643)
- Fix InternVL integration test (#38612)
- Use torch 2.7.1 on daily CI (#38620)
- Fix aya_vision test (#38674)
- Update some tests for torch 2.7.1 (#38701)
- Fix llava tests (#38722)
- Revert "Trigger doc-builder job after style bot" (#38735)
- Make style bot trigger CI after push (#38754)
- [Hotfix] Fix style bot (#38779)
- Skip some export tests on torch 2.7 (#38677)
- Fix qwen_2_5 omni (#38658)
- Fix llava_onevision tests (#38791)
- Fix mllama (#38704)
- Fix llava_next tests (#38813)
- Fix a minor security issue (#38815)
- Fix qwen2_5_vl tests (#38845)
- Fix qwen3 tests (#38862)
- Fix phi4_multimodal tests (#38816)
- Fix qwen3_moe tests (#38865)
- Fix fsmt tests (#38904)
- Fix FalconMambaIntegrationTests (#38566)
- Skip some tests for now (#38931)
- Fix more flaky test_initialization (#38932)
- Switch to use A10 progressively (#38936)
- fix mistral and mistral3 tests (#38978)
- Fix rag (#38585)
@ArthurZucker
- tp plan should not be NONE (#38255)
- Protect ParallelInterface (#38262)
- Add CB (#38085)
- 🚨Early-error🚨 config will error out if output_attentions=True and the attn implementation is wrong (#38288)
- for now disable compile (#38383)
- make it go brrrr (#38409)
@younesbelkada
- [MODEL] Add Falcon H1 (#38249)
@cyr0930
- fix multi-image case for llava-onevision (#38084)
@cyyever
- Improve typing in TrainingArgument (#36944)
- More typing in src/transformers/training_args.py (#38106)
- Fix run_slow (#38314)
- Remove deprecated use_flash_attention_2 parameter (#37131)
- Use OSError (#38712)
- More PYUP fixes (#38883)
- Fix unnecessary super calls (#38897)
@ritsumei-aoi
- Remove Japanese sequence_classification doc and update references (#38246)
@yao-matrix
- add XPU info print in print_env (#38282)
- refine transformers env output (#38274)
- switch to device agnostic device calling for test cases (#38247)
- enable large_gpu and torchao cases on XPU (#38355)
- enable more test cases on xpu (#38572)
- remove ipex_optimize_model usage (#38632)
- from 1.11.0, torchao.prototype.low_bit_optim is promoted to torchao.optim (#38689)
- enable misc test cases on XPU (#38852)
- align xpu's autocast behavior w/ cuda by using device agnostic torch APIs (#38284)
- add pytorch-xpu Dockerfile (#38875)
@vasqu
- 🔴🔴🔴 [Attention] Refactor Attention Interface for Bart-based Models (#38108)
- [FlexAttention] Reenable flex for encoder-decoder and make the test more robust (#38321)
- [OPT] Fix attention scaling (#38290)
- 🔴[Attention] Attention refactor for Whisper-based models (#38235)
- [FlexAttn] Fix models with unique characteristics (#38433)
- [Attention] Small fix on output attentions (#38948)
@itazap
- refactor can_save_slow_tokenizer (#37722)
- remove unhandled parameter (#38145)
- refactor create_token_type_ids_from_sequences (#37681)
@eustlb
- [CSM] infer codec model with no_grad + audio eos label (#38215)
- [CSM] update model id (#38211)
- [phi-4] use mel filters from audio utils (#36966)
- Add kyutai stt (#38909)
- [Kyutai-STT] correct model type + model id (#39035)
@RogerSinghChugh
- Updated BigBird Model card as per #36979. (#37959)
- Updated BERTweet model card. (#37981)
- New bart model card (#37858)
- New gpt neo model card (#38505)
- New canine model card (#38631)
@1himan
- Updated the Model docs - for the ALIGN model (#38072)
- Updated Aria model card (#38472)
- Updated aya_vision.md (#38749)
@Avasam
- Merge type hints from microsoft/python-type-stubs (post dropping support for Python 3.8) (#38335)
@remi-or
- [seamless_m4t] Skip some tests when speech is not available (#38430)
- [janus] Fix failing tests on mi3XX (#38426)
- Fixed a multiple-devices issue in SmolVLM model (#38736)
- Expectation fixes and added AMD expectations (#38729)
- Modernbert fixes (#38912)
- Break tie in Expectations and gemma3 fixes (#38943)
@tonywu71
- Add ColQwen2 to 🤗 transformers (#35778)
@geetu040
- Add support for MiniMax's MiniMax-Text-01 (#35831)
- Fix MiniMax (docs and integration tests checkpoint) (#38575)
@sbucaille
- Fixed modeling_auto.py MODEL_FOR_MASK_GENERATION_MAPPING_NAMES variable (#38664)
- Add LightGlue model (#31718)
- [LightGlue] Fixed attribute usage from descriptor_dim to keypoint_detector_descriptor_dim (#39021)
@samrae7
- 36978 | Fast image processor for DPT model (#37481)
@Crystalcareai
- Add Arcee model support (#38621)
@zRzRzRzRzRzRzR
- GLM-4.1V Model support (#38431)
@bzhangGo
- Encoder-Decoder Gemma (#38332)
@redmoe-moutain
- [Model] add dots1 (#38143)
@EduardDurech
- Support for Flash Attention 3 (#38972)

Open source →

4.52.120 May 2025Security

[chat template] fix security vulnerability by @zucchini-nlp in #37523

New models

Qwen2.5-Omni

The Qwen2.5-Omni model is a unified multiple modalities model proposed in Qwen2.5-Omni Technical Report from Qwen team, Alibaba Group.

The abstract from the technical report is the following:

We present Qwen2.5-Omni, an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner. To enable the streaming of multimodal information inputs, both audio and visual encoders utilize a block-wise processing approach. This strategy effectively decouples the handling of long sequences of multimodal data, assigning the perceptual responsibilities to the multimodal encoder and entrusting the modeling of extended sequences to a large language model.

Such a division of labor enhances the fusion of different modalities via the shared attention mechanism. To synchronize the timestamps of video inputs with audio, we organized the audio and video sequentially in an interleaved manner and propose a novel position embedding approach, named TMRoPE (Time-aligned Multimodal RoPE). To concurrently generate text and speech while avoiding interference between the two modalities, we propose Thinker-Talker architecture.

In this framework, Thinker functions as a large language model tasked with text generation, while Talker is a dual-track autoregressive model that directly utilizes the hidden representations from the Thinker to produce audio tokens as output. Both the Thinker and Talker models are designed to be trained and inferred in an end-to-end manner. For decoding audio tokens in a streaming manner, we introduce a sliding-window DiT that restricts the receptive field, aiming to reduce the initial package delay. Qwen2.5-Omni outperforms the similarly sized Qwen2-VL and Qwen2-Audio in both image and audio capabilities. Furthermore, Qwen2.5-Omni achieves state-of-the-art performance on multimodal benchmarks like Omni-Bench.

Notably, Qwen2.5-Omni is the first open-source model to achieve a level of performance in end-to-end speech instruction following that is comparable to its capabilities with text inputs, as evidenced by benchmarks such as MMLU and GSM8K. As for speech generation, Qwen2.5-Omni’s streaming Talker outperform most existing streaming and non-streaming alternatives in robustness and naturalness.

SAM-HQ

SAM-HQ (High-Quality Segment Anything Model) was proposed in Segment Anything in High Quality by Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu.

The model is an enhancement to the original SAM model that produces significantly higher quality segmentation masks while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability.

SAM-HQ introduces several key improvements over the original SAM model:

High-Quality Output Token: A learnable token injected into SAM's mask decoder for higher quality mask prediction
Global-local Feature Fusion: Combines features from different stages of the model for improved mask details
Training Data: Uses a carefully curated dataset of 44K high-quality masks instead of SA-1B
Efficiency: Adds only 0.5% additional parameters while significantly improving mask quality
Zero-shot Capability: Maintains SAM's strong zero-shot performance while improving accuracy

The abstract from the paper is the following:

The recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced dataset of 44k masks, which takes only 4 hours on 8 GPUs.

Tips:

SAM-HQ produces higher quality masks than the original SAM model, particularly for objects with intricate structures and fine details
The model predicts binary masks with more accurate boundaries and better handling of thin structures
Like SAM, the model performs better with input 2D points and/or input bounding boxes
You can prompt multiple points for the same image and predict a single high-quality mask
The model maintains SAM's zero-shot generalization capabilities
SAM-HQ only adds ~0.5% additional parameters compared to SAM
Fine-tuning the model is not supported yet

GraniteMoeHybrid

The GraniteMoeHybrid model builds on top of GraniteMoeSharedModel and Bamba. Its decoding layers consist of state space layers or MoE attention layers with shared experts. By default, the attention layers do not use positional encoding.

D-FINE

The D-FINE model was proposed in D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement by Yansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, Feng Wu

The abstract from the paper is the following:

We introduce D-FINE, a powerful real-time object detector that achieves outstanding localization precision by redefining the bounding box regression task in DETR models. D-FINE comprises two key components: Fine-grained Distribution Refinement (FDR) and Global Optimal Localization Self-Distillation (GO-LSD). FDR transforms the regression process from predicting fixed coordinates to iteratively refining probability distributions, providing a fine-grained intermediate representation that significantly enhances localization accuracy. GO-LSD is a bidirectional optimization strategy that transfers localization knowledge from refined distributions to shallower layers through self-distillation, while also simplifying the residual prediction tasks for deeper layers. Additionally, D-FINE incorporates lightweight optimizations in computationally intensive modules and operations, achieving a better balance between speed and accuracy. Specifically, D-FINE-L / X achieves 54.0% / 55.8% AP on the COCO dataset at 124 / 78 FPS on an NVIDIA T4 GPU. When pretrained on Objects365, D-FINE-L / X attains 57.1% / 59.3% AP, surpassing all existing real-time detectors. Furthermore, our method significantly enhances the performance of a wide range of DETR models by up to 5.3% AP with negligible extra parameters and training costs. Our code and pretrained models: this https URL.

CSM

The Conversational Speech Model (CSM) is the first open-source contextual text-to-speech model released by Sesame. It is designed to generate natural-sounding speech with or without conversational context. This context typically consists of multi-turn dialogue between speakers, represented as sequences of text and corresponding spoken audio.

Model Architecture: CSM is composed of two LLaMA-style auto-regressive transformer decoders: a backbone decoder that predicts the first codebook token and a depth decoder that generates the remaining tokens. It uses the pretrained codec model Mimi, introduced by Kyutai, to encode speech into discrete codebook tokens and decode them back into audio.

The original csm-1b checkpoint is available under the Sesame organization on Hugging Face.

BitNet

Trained on a corpus of 4 trillion tokens, this model demonstrates that native 1-bit LLMs can achieve performance comparable to leading open-weight, full-precision models of similar size, while offering substantial advantages in computational efficiency (memory, energy, latency).

LlamaGuard

Llama Guard 4 is a new multimodal model designed to detect inappropriate content in images and text, whether used as input or generated as output by the model. It’s a dense 12B model pruned from Llama 4 Scout model, and it can run on a single GPU (24 GBs of VRAM). It can evaluate both text-only and image+text inputs, making it suitable for filtering both inputs and outputs of large language models. This enables flexible moderation pipelines where prompts are analyzed before reaching the model, and generated responses are reviewed afterwards for safety. It can also understand multiple languages.

TimesFM

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model proposed in A decoder-only foundation model for time-series forecasting by Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. It is a decoder only model that uses non-overlapping patches of time-series data as input and outputs some output patch length prediction in an autoregressive fashion.

The abstract from the paper is the following:

Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset. Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus, and can work well across different forecasting history lengths, prediction lengths and temporal granularities.

MLCD

The MLCD models were released by the DeepGlint-AI team in unicom, which focuses on building foundational visual models for large multimodal language models using large-scale datasets such as LAION400M and COYO700M, and employs sample-to-cluster contrastive learning to optimize performance. MLCD models are primarily used for multimodal visual large language models, such as LLaVA.

Janus

The Janus Model was originally proposed in Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation by DeepSeek AI team and later refined in Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. Janus is a vision-language model that can generate both image and text output, it can also take both images and text as input.

[!NOTE] The model doesn't generate both images and text in an interleaved format. The user has to pass a parameter indicating whether to generate text or image.

The abstract from the original paper is the following:

In this paper, we introduce Janus, an autoregressive framework that unifies multimodal understanding and generation. Prior research often relies on a single visual encoder for both tasks, such as Chameleon. However, due to the differing levels of information granularity required by multimodal understanding and generation, this approach can lead to suboptimal performance, particularly in multimodal understanding. To address this issue, we decouple visual encoding into separate pathways, while still leveraging a single, unified transformer architecture for processing. The decoupling not only alleviates the conflict between the visual encoder's roles in understanding and generation, but also enhances the framework's flexibility. For instance, both the multimodal understanding and generation components can independently select their most suitable encoding methods. Experiments show that Janus surpasses previous unified model and matches or exceeds the performance of task-specific models. The simplicity, high flexibility, and effectiveness of Janus make it a strong candidate for next-generation unified multimodal models.

The abstract from the aforementioned Janus-Pro paper, released afterwards, is the following:

In this work, we introduce Janus-Pro, an advanced version of the previous work Janus. Specifically, Janus-Pro incorporates (1) an optimized training strate (2) expanded training data, and (3) scaling to larger model size. With these improvements, Janus-Pro achieves significant advancements in both multimodal understanding and text-to-image instruction-following capabilities, while also enhancing the stability of text-to-image generation. We hope this work will inspire further exploration in the field. Code and models are publicly available.

InternVL

The InternVL3 family of Visual Language Models was introduced in InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models.

The abstract from the paper is the following:

We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs.

<small> Overview of InternVL3 models architecture, which is the same as InternVL2.5. Taken from the <a href="https://huggingface.co/OpenGVLab/InternVL3-1B">original checkpoint.</a> </small>

<small> Comparison of InternVL3 performance on OpenCompass against other SOTA VLLMs. Taken from the <a href="https://huggingface.co/OpenGVLab/InternVL3-1B">original checkpoint.</a> </small>

Kernel integration

We integrate some kernels in the transformers library via the kernels package: https://github.com/huggingface/kernels We start with some kernels in the Llama model, and we iterate to identify the best performance optimizations

Llama Kernel integration by @MekkCyber in #37092
[kernels] use original forward at compile time by @gante in #37604

TP support

In the previous release, we've added TP support in order to run distributed inference. However, this is not supported for all quantization methods. We are progressively adding support to it. Right now, only compressed-tensors, fp8 and fp8-fbgemm support it.

Attention Quantization with FBGemm & TP by @MekkCyber in #37384
Restrict & Explain tp_plan for FBgemm by @MekkCyber in #37404

Quantization

AutoRound

From the AutoRound contributors:

AutoRound is an advanced quantization algorithm that delivers strong accuracy, even at 2-bit precision. It leverages sign gradient descent to fine-tune both rounding values and min-max clipping thresholds in just 200 steps ... More details here: https://github.com/intel/auto-round

Add AutoRound quantization support by @wenhuach21 in #37393

Quantization Documentation

We have added two new sections to better understand and get started with quantization:

Add "selecting a quantization method" doc by @DerekLiu35 in #37159
Update quantization docs by @DerekLiu35 in #37439

GGUF

We've added GGUF support to gemma3 family models.

Add GGUF support to Gemma3 Text backbone by @Isotr0py in #37424
Support loading Gemma3 QAT GGUF models by @Isotr0py in #37649

Fast image processors

Most Vision Models and VLMs in Transformers can now benefit from fast image processors. By utilizing torch/torchvision functional transforms, these processors offer a substantial speedup when processing images compared to PiL/numpy functions, and support processing on both CPU and CUDA.

See the list of updated models: https://github.com/huggingface/transformers/issues/36978
Learn more about fast image processors: Fast Image Processors

Add Fast Image Processor for Perceiver by @rootonchair in #37176
Add Fast Image Processor for Flava by @rootonchair in #37135
Add Fast Image Processor for LayoutLMv2 by @rootonchair in #37203
Add Fast Image Processor for LayoutLMv3 by @rootonchair in #37201
Add Fast Image Processor for Donut by @rootonchair in #37081
Add Fast LeViT Processor by @keetrap in #37154
Add Fast Mobilenet-V2 Processor by @keetrap in #37113
Add Fast owlvit Processor by @keetrap in #37164
Add ImageProcessorFast to BiT processor by @Yann-CV in #37180
Add Fast Yolos Processor by @keetrap in #37292
Add Fast Chinese-CLIP Processor by @keetrap in #37012
Add Fast Conditional-DETR Processor by @keetrap in #37071
Fix broken add-fast-image-processor CLI by @yonigozlan in #37499
Bridgetower fast image processor by @rootonchair in #37373
Add Fast Grounding-Dino Processor by @keetrap in #37108
Add Fast PVT Processor by @keetrap in #37204
Add Fast Image Processor for PoolFormer by @rootonchair in #37182
Add Fast Image Processor for MobileNetV1 by @dmdaksh in #37111
Fast image processor for VitMatte added and bug in slow version fixed by @henrikm11 in #37616
[Fast Processor] BEiT by @ariG23498 in #37005
Add Swin2SR ImageProcessorFast by @thisisiron in #37169
Add Fast Image Processor for vilt by @devxaitist in #37304

AutoDocstring

The new @auto_docstring decorator makes it easier to add proper documentation when contributing a model without bloating the modeling code:

[AutoDocstring] Based on inspect parsing of the signature by @ArthurZucker and @yonigozlan in https://github.com/huggingface/transformers/pull/33771
More info on how to use @auto_docstring: AutoDocstring

Custom `generate`

We now support custom generate methods to be loaded from model.generate. The custom generate methods can be stored on the Hub, enabling quick distribution of experiments regarding new caches, decoding methods, heuristics, ...

from transformers import AutoModelForCausalLM, AutoTokenizer

# `generate` with `custom_generate` -> `generate` uses custom code
# note: calling the custom method prints "✨ using a custom generation method ✨"
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", device_map="auto")

inputs = tokenizer(["The quick brown"], return_tensors="pt").to(model.device)
gen_out = model.generate(**inputs, custom_generate="transformers-community/custom_generate_example", trust_remote_code=True)
print(tokenizer.batch_decode(gen_out, skip_special_tokens=True))

You can find the docs here, and all custom generation methods by searching for the custom_generate tag.

[generate] Run custom generation code from the Hub by @gante in #36405

Chat CLI

The transformers-cli command is updated to be simpler and cleaner, specifically for its chat variant.

The following is now possible and recommended:

transformers chat Qwen/Qwen2.5-3B-Instruct

Additionally, almost any generate flag can now be passed as a positional argument, present and future, as opposed to being limited to a set of hardcoded flags, for example:

transformers chat Qwen/Qwen2.5-0.5B-Instruct do_sample=False max_new_tokens=10

Transformers cli clean command by @LysandreJik in #37657
[chat] clean code and add base help by @gante in #37892
[chat] generate parameterization powered by GenerationConfig and UX-related changes by @gante in #38047

Breaking changes

🚨 rm already deprecated pad_to_max_length arg by @itazap in #37617
🚨🚨🚨 Fix forward of Dinov2ForImageClassification for models with registers by @psandovalsegura in #37836
🔴 [VLM] Add base model without head by @zucchini-nlp in #37033
🔴 Video processors as a separate class by @zucchini-nlp in #35206
🚨🚨 Allow saving and loading multiple "raw" chat template files by @Rocketknight1 in #36588
🔴 Update CLIP vision attention to new attention interface by @molbap in #37498
🚨🚨 Setup -> setupclass conversion by @Rocketknight1 in #37282

Deprecations

The agents folder is finally removed from transformers in favour of using smolagents.

[agents] remove agents 🧹 by @gante in #37368

We are moving away from torch 2.0 as it has been released more than two years ago.

byebye torch 2.0 by @ydshieh in #37277

General bugfixes and improvements

fix flex attn when optional args aren't passed by @winglian in #37327
fix llama4 training by @hiyouga in #37319
Fix deepspeed with quantization by @Cyrilvallez in #37324
Fix init empty weights without accelerate by @Cyrilvallez in #37337
Use Python 3.9 syntax in examples by @cyyever in #37279
Fix torchao usage by @jiqing-feng in #37034
enable 2 llama UT cases on xpu by @yao-matrix in #37126
Avoid build crashes when torch.version.xpu doesn't exist and fix Llama4 processor tests by @Rocketknight1 in #37346
fix derived berts _init_weights by @Cyrilvallez in #37341
Update translation template by @stevhliu in #37294
Remove HQQ from caching allocator warmup by @Cyrilvallez in #37347
updated model card for Mistral by @NahieliV in #37156
Update model-card for DINOv2 by @shubham0204 in #37104
Update falcon mamba card by @ricalanis in #37253
Update Model card for GPT2 by @ash-01xor in #37101
Improvements in Gemma2 model card by @devesh-2002 in #37076
Update Model Card for Jamba by @ParagEkbote in #37152
Add bnb to the list of supported quantization methods for LLama4 by @MekkCyber in #37348
Updated Model-card for donut by @Logeswaran7 in #37290
Remove unnecessary attr assignment by @tugsbayasgalan in #36837
more fixes for post-training llama4 by @winglian in #37329
Fixing flex attention for torch=2.6.0 by @SalmanMohammadi in #37285
Multiple llama4 fixe by @ArthurZucker in #37353
Expose blip2qformer by @alex-jw-brooks in #37254
convert float for yarn related arguments in rope_scaling by @bzantium in #37139
Use Python 3.9 syntax in tests by @cyyever in #37343
A bit of cleaning 🧹🧹 by @Cyrilvallez in #37215
fix deepspeed job by @ydshieh in #37284
Set vision config to None for Gemma 1B conversion by @RyanMullins in #37366
[llama 4] dynamic rope decorator by @gante in #37365
Skip non-selected experts for mixtral and qwen2_moe by @Coco58323 in #32429
[core] remove GenerationMixin inheritance by default in PreTrainedModel by @gante in #37173
prune LM Head for USD by @jmamou in #36695
fix(qwen): fix shape error when using tp by @KimmiShi in #36947
Preserve requires_grad in pre quantized model by @jerryzh168 in #37354
Update composition flag usage by @zucchini-nlp in #36263
fix: llama4 conversion script no_rope_layers by @jmkuebler in #37359
update deepspeed docker by @SunMarc in #37371
Fix warning message for PEFT models in text-generation pipeline #36783 by @falconlee236 in #36887
Apply torchfix to replace deprecated functions: _pytree._register_pytree_node and torch.cpu.amp.autocast by @bzhong-solink in #37372
Fix some failing AWQ tests by @DerekLiu35 in #37383
the fix that did not get in by @ArthurZucker in #37370
handle torch version edge cases by @winglian in #37399
Add warning when failed to acquire other user's lock at model download by @manueldeprada in #37395
Handle torch ver in flexattn by @Kh4L in #37400
Fix Llama4 offset by @Cyrilvallez in #37414
Offloaded hybrid cache for Llama4 by @Cyrilvallez in #37401
mark llama4 as not supported with fa2 by @winglian in #37416
update kernels to 0.4.3 by @ArthurZucker in #37419
Send trainer/fsdp/deepspeed CI job reports to a single channel by @ydshieh in #37411
from_pretrained should handle xpu case by @sywangyi in #37382
Allow rocm systems to run these tests by @ivarflakstad in #37278
use rms_norm_eps for the L2Norm for Llama4 by @ArthurZucker in #37418
[chat-template] Unify tests and clean up 🧼 by @zucchini-nlp in #37275
Fix new failure reports not including anything other than tests/models/ by @ydshieh in #37415
Quark Quantization gated repo by @MekkCyber in #37412
Add image classifier donut & update loss calculation for all swins by @eljandoubi in #37224
Correctly drop tokens in SwitchTransformer by @mario-aws in #37123
Fix require_read_token by @MekkCyber in #37422
fix: use mtime by default in Trainer._rotate_checkpoints with automatic fallback by @Jerry-Terrasse in #37260
(Part 2) feat: allow for tp_size attr for tplizing the model by @kmehant in #37054
Adding to self_comment_ci.yml by @MekkCyber in #37426
[Feat] Support npu in modeling models by @duanjunwen in #37369
Remove old code for PyTorch, Accelerator and tokenizers by @cyyever in #37234
enhance require_deterministic_for_xpu by @yao-matrix in #37437
Fixes: Corrects file path for CUDA kernels by @DonggeunYu in #37438
Simplify soft dependencies and update the dummy-creation process by @LysandreJik in #36827
Update-kernel-pin by @ArthurZucker in #37448
Add moe kernels by @ArthurZucker in #37376
Fix the test fetcher by @LysandreJik in #37452
Remove triton mlp kernel, not compiling for some models by @MekkCyber in #37449
[processor] clean up mulitmodal tests by @zucchini-nlp in #37362
[Regression] Fix Quark quantized model loading after refactorization by @BowenBao in #37407
prevent creating a view/leaf param for low rank optimizers w FSDP by @winglian in #37379
Disable kernels for quantization by @MekkCyber in #37446
Add weights_only=True to torch.load by @cyyever in #37062
Add XPU case to is_torch_bf16_gpu_available by @cyyever in #37132
nit: typing use Llama4TextConfig instead of Llama4Config by @kmehant in #37430
Delete hubconf.py by @Rocketknight1 in #37455
Fix typing issues with SigLip2 by @EricWiener in #37356
fix: (llama4) fix no_split_modules to be picked up for fsdpv1 and v2 sharding by @kmehant in #37462
make test_snowman_image_captioning pass on XPU, by sharing same atol w/ ROCM by @yao-matrix in #37480
Remove fsspec dependency which isn't directly used by transformers by @cyyever in #37318
Fix tests failed with gated repos. by @ydshieh in #37484
[ci] fix doc builder by @zucchini-nlp in #37489
Fixed broken links by @cypherpepe in #37466
Detect and fix most _init_weights() issues - make it work for composite models by @Cyrilvallez in #37070
[bug] deprecated deta load_cuda_kernel, MultiScaleDeformableAttention by @chagmgang in #37443
Fix mask handling for flex attention in llama/gemma2/mistral/qwen2 by @flukeskywalker in #37381
Fix wrong argparse type in modular checker script by @seven-mile in #37472
Fixing gated repo issues by @MekkCyber in #37463
[qwen-omni] fix processor by @zucchini-nlp in #37493
Remove deprecation warning for num_logits_to_keep by @Cyrilvallez in #37149
Don't auto-assign reviewers when the author is in HF by @Rocketknight1 in #37500
Detect and use device context manager or global device in from_pretrained by @Cyrilvallez in #37216
Change default value of attn_temperature_tuning by @gmlwns2000 in #37501
Llama4: remove redundant transpose of router_logits by @pbelevich in #37468
fix: Restore explicit error surfacing for unexpected hub exceptions by @manueldeprada in #37525
Fix missing return type for MLCD docs by @qubvel in #37527
fix and enhance pipeline_webserver.md by @yao-matrix in #36992
VDR task guide by @merveenoyan in #37485
Update VITS model card by @princepride in #37335
Refactor ColPali model documentation by @Soum-Soum in #37309
enable 5 cases on XPU by @yao-matrix in #37507
enable several cases on XPU by @yao-matrix in #37516
enable test_offloaded_cache_implementation on XPU by @yao-matrix in #37514
Fix BitsAndBytesConfig JSON serialization in TrainingArguments by @astefanutti in #37520
enable 3 mpt test cases on XPU by @yao-matrix in #37546
enable 6 rt_detr_v2 cases on xpu by @yao-matrix in #37548
More appropriate cuda warmup in resource-constrained hardware by @Cyrilvallez in #37550
Fixes hqq by following a new path for bias parameter in pre_quantized models by @MekkCyber in #37530
convert scale and zero to cuda when using HQQ backend by @phymhan in #37425
Keep Quark loading through meta device by @BowenBao in #37538
Refactor torchao docs by @MekkCyber in #37490
add FlashAttentionKwargs and seq_idx to flat collator by @garrett361 in #36456
docs(typo): Update ISSUES.md, fix a small typo by @<NOT FOUND> in #37542
Fix device issue for tapas (with as_tensor) by @ydshieh in #37551
Make Ignored Columns ValueError More Informative by @wbuchanan in #33299
Fix TimesFm doc issue by @Cyrilvallez in #37552
Run test_can_load_with_global_device_set using a subprocess by @ydshieh in #37553
Fix pixel attention mask padding in smolvlm by @ManuelFay in #37497
[vlm] adjust max length for special tokens by @zucchini-nlp in #37342
Add EfficientNet Image PreProcessor by @zshn25 in #37055
Fix Mamba2 Grouped SSD Support in the torch_forward Path by @cyang49 in #37533
All models can be initialized on meta device by @Cyrilvallez in #37563
[chat template] fix security vulnerability by @zucchini-nlp in #37523
[qwen-vl] Standardize config by @zucchini-nlp in #37268
[TimesFM] use the main revison instead of revision for integration test by @kashif in #37558
Fix qwen2audio wanr -> warn by @alex-jw-brooks in #37559
Small fix on context manager detection by @Cyrilvallez in #37562
[phi4] update conversion by @zucchini-nlp in #37579
docs: fix typo by @tonyksong in #37567
Ensure positive warm-up size by @Cyrilvallez in #37581
Update Phi4 converter by @Cyrilvallez in #37594
Fix Quark quantization config by @MekkCyber in #37578
Gaudi: Add the bf16 support for hpu by @yuanwu2017 in #37568
Fix some GPU OOM after #37553 by @ydshieh in #37591
remove _run_third_party_device_tests by @jiqing-feng in #37445
[Bugfix] Fix flash-attention func param mismatch and softmax_scale default value mistake on Ascend NPU by @FightingZhen in #37575
Flag SpeechT5 flaky test by @molbap in #37587
enable 6 gemma2 cases on XPU by @yao-matrix in #37564
enable 6 modeling cases on XPU by @yao-matrix in #37571
[Gemma3] compile ✨ by @gante in #37447
Model debugger upgrades by @molbap in #37391
[VLMs] use only xxx_token_id for multimodal tokens by @zucchini-nlp in #37573
fix 2 encoder_decoder issues on XPU by @yao-matrix in #37572
fix issue that some example with no trainer use accelerator.end_train… by @we1559 in #37435
Deprecate modeling_utils.py classes by @qubvel in #37298
Fixing the example in generation strategy doc by @jeasinema in #37598
chore: update model card for SigLIP by @saswatmeher in #37585
Fix InternVL attention when using qk_norm (38B and 78B) by @yonigozlan in #37620
Remove torchvision requirement from AutoImageProcessor by @LysandreJik in #37457
Allow Exclusion of Input IDs from RepetitionPenaltyLogitsProcessor by @alex-jw-brooks in #37625
fix link in kv_cache.md by @manueldeprada in #37652
Update longformer.md by @JihadHammoud02 in #37622
Refactor phi doc by @JihadHammoud02 in #37583
Fix Qwen2.5-Omni get_chunked_index chunking functionality by @imkero in #37631
[fix] make legacy bnb code work by @cyr0930 in #37331
[fix gemma] Set default value for output_attentions parameter in Gemma2 and Gemma… by @chenin-wang in #37633
Restructure torchao quantization examples by @jerryzh168 in #37592
Add test to ensure unknown exceptions reraising in utils/hub.py::cached_files() by @manueldeprada in #37651
[test] update test_past_key_values_format by @gante in #37614
[tests] Stricter generate + compilation test -- no recompilations allowed by @gante in #37629
Fix ValueError when eval_do_concat_batches=False with examples by @jeffhataws in #37621
Fixes #37219 : RecurrentGemma crashes for inputs longer than sliding window length by @manueldeprada in #37613
Introduce GradientCheckpointingLayer by @qubvel in #37223
[qwen-omni] fix training by @zucchini-nlp in #37517
Fix duplicated weights in fp8 quantization by @Cyrilvallez in #37667
Correct warm-up with fp8 by @Cyrilvallez in #37670
Fixing quantization tests by @MekkCyber in #37650
Fix autoround docs by @SunMarc in #37675
Fix no_split_modules for Llama4 pretrained models by @astefanutti in #37673
Refactor bitsandbytes doc by @MekkCyber in #37668
enable mllama cases on xpu by @yao-matrix in #37644
enable 6 granite cases on xpu by @yao-matrix in #37569
[cleanup] remove old scripts in /scripts 🧹 🧹 by @gante in #37676
[docs] only build en docs in push CI by @gante in #37677
typo update in the parameter name by @LunaticMaestro in #37655
[Docs] Move models to appropriate section by @NielsRogge in #37338
Add counters for dataset classes by @jiangyukunok in #37636
enable blip2 and emu3 cases on XPU by @yao-matrix in #37662
🌐 [i18n-KO] Translated siglip.md to Korean by @devxaitist in #37145
Updated model card for mbart and mbart50 by @Vishesh-Mistry in #37619
fix: remove classmethod from Qwen2_5OmniConfig.get_text_config by @shahruk10 in #37690
enable cpu offloading for Bark on xpu by @yao-matrix in #37599
Pin torch == 2.6 on PR CI docker images for now by @ydshieh in #37695
[cleanup] remove /model_cards 🧹 🧹 by @gante in #37685
Add maintainers for ROCm/Intel XPU/Ascend NPU by @Rocketknight1 in #37678
[CI] add back sacrebleu (and document why) by @gante in #37700
TransfoXL is deprecated, don't keep it in tested examples! by @Rocketknight1 in #37707
[internvl] fix chat template by @zucchini-nlp in #37656
Qwen 2.5 Omni: apply video defaults by @pcuenca in #37660
[tests, qwen2_5_omni] fix flaky tests by @gante in #37721
Process inputs directly in apply_chat_template in image-text-to-text pipeline by @yonigozlan in #35616
enable 4 test_trainer cases on XPU by @yao-matrix in #37645
Fix Aria tests by @jiqing-feng in #37444
Fix inference bugs in Qwen2.5 Omni by @BakerBunker in #37701
Fix torchao doc examples by @MekkCyber in #37697
[tests] fix test_nemotron_8b_generation_sdpa by @faaany in #37665
Make sure torch_is_available before using torch.distributed by @MekkCyber in #37693
[VLMs] fix flash-attention tests by @zucchini-nlp in #37603
fix: learning_rate logged as tensor causing save issue with deepspeed by @NanoCode012 in #37704
Fix embeds_to_talker device in Qwen2.5-Omni by @BakerBunker in #37739
Correctly raise errors when downloading tokenizer files by @Cyrilvallez in #37740
[performance_optim] define flash attention mask on NPU device directly by @FightingZhen in #37698
Skip all AriaForConditionalGenerationIntegrationTest on T4 by @ydshieh in #37746
Update MllamaForConditionalGenerationIntegrationTest by @ydshieh in #37750
Expand quantized data type support for tensor parallelism by @amd-xiaoyu12 in #37719
[cache] fix HybridCache init when device is passed by @gante in #37718
GPT2Model StaticCache support by @poedator in #35761
[generate] skip compilation on cpu offload by @gante in #37709
updated hidden_features for FlaxDinov2SwiGLUFFN in Dinov2 by @premmurugan229 in #37747
Fix qwen2_5 get_rope_index tensor device locations by @rphmeier in #37597
[generate] fix default autocompile case on gpu by @gante in #37756
Fix wrong input shapes in doc-string of models by @kkew3 in #37729
Refine parameter type annotations by @flashJd in #37666
Fix tied weight loading with TP and loading sub state_dicts by @Cyrilvallez in #37758
Fix load of rng state for resuming training from checkpoint by @winglian in #37162
Fix typos in comments by @co63oc in #37694
[deps] pin max torch version by @gante in #37760
Guard DeepSpeed imports by @lewtun in #37755
Fix auto-round hfoption by @MekkCyber in #37759
Update model card for Gemma by @afafelwafi in #37674
🌐 [i18n-KO] Translated roberta.md to Korean by @garongkim in #37069
[causal mask] fix preparation with multi-gpu by @zucchini-nlp in #37612
unpin pytest<8 by @ydshieh in #37768
Align gpt2 mask preparation to #37612 by @Cyrilvallez in #37787
Fix typos in strings and comments by @co63oc in #37784
Fix tensor parallel with non-floating dtypes by @Cyrilvallez in #37790
Force torch>=2.6 with torch.load to avoid vulnerability issue by @Cyrilvallez in #37785
fix mpt test of different outputs from cuda by @jiqing-feng in #37691
[i18n-KO] Translated keypoint_detection.md to Korean by @rlaalsrl0922 in #36649
chore: update SigLIP2 model card by @saswatmeher in #37624
fix performance issue in convert_ids_to_tokens by @martin-harmonic in #37773
Fix error message in hub.py by @srai9 in #37796
Gemma3 is Torch Exportable by @guangy10 in #37728
Fix the fsdp config cannot work issue. by @yuanwu2017 in #37549
Define warmup allocator for torchao quantization by @MekkCyber in #37764
Fix typos in strings and comments by @co63oc in #37799
[doc] fix the code examples in qwen doc by @jiangyukunok in #37803
Fix: Correct tensor shape comment in Mamba modeling by @ShadyPi in #37801
[RT-DETR] Improve docs by @NielsRogge in #37814
FIX: Faulty PEFT tests by @BenjaminBossan in #37757
Add Optional to remaining types by @cyyever in #37808
Fix error of HPU TP by @yuanwu2017 in #37782
change XLA deprecated api by @SunMarc in #37741
[config] revert #37603 by @zucchini-nlp in #37821
[modular] Fix the prefix-based renaming if the old and new model share a common name suffix by @Cyrilvallez in #37829
[tests] fix flaky pattern in test_generate_continue_from_past_key_values by @gante in #37724
[tests] reorganize cache tests and clean memory between tests by @gante in #37684
Revert change that breaks on Torch 2.1 by @Rocketknight1 in #37531
Fix check of unecessary packages (issue #37626) by @HichTala in #37825
Fix cache get item return type hints by @ChengLyu in #37847
Fix Bitnet tokenizer in pipeline by @MekkCyber in #37861
docs: Details for ambigious channel dimension assignment by @yaner-here in #37600
Processor chat template: pass custom kwargs by @pcuenca in #37852
Add Intel Gaudi doc by @regisss in #37855
🌐 [i18n-KO] Translated electra.md to Korean by @Kim-Ju-won in #36763
Update modeling_llama4.py by @monk1337 in #37841
Skip is_flaky tests in the CI by @Rocketknight1 in #37723
Allow override inputs to export recipe by @guangy10 in #37508
enable internvl UTs on XPU by @yao-matrix in #37779
Llama Guard updates by @pcuenca in #37872
update Clean_up_tokenization_spaces typos. by @zhanluxianshen in #37865
fix error for _register_pytree_node in torch2.1.0 and fix bf16 assertion in xpu and npu by @jiaqiw09 in #37839
make sure lr is not a tensor by @winglian in #37881
Fix qwen2-vl-docs. by @zhanluxianshen in #37879
uniformize kwargs for VisionTextDualEncoder by @tibor-reiss in #34563
Fix: reassign in qwen3 moe model by @linkedlist771 in #37848
update comment in image_processing_base.py to reference image_process… by @arjunaskykok in #37864
Support FlaxPreTrainedModel to load model checkpoint from local subfolder safetensors by @Melody-coder923 in #37732
[tests] Test all cache implementations by @gante in #37873
[tests] reset logs in torch.compile test by @gante in #37894
Fix Qwen3 tp plan with FP8 by @MekkCyber in #37871
Enhance documentation to explain chat-based few-shot prompting by @MostHumble in #37828
Support AOPerModuleConfig and include_embedding by @jerryzh168 in #37802
fixed gemma3 collection path pointing to llama 2 collection. by @dmgcsilva in #37899
Fix typos in strings and comments by @co63oc in #37910
Improve performance of load_state_dict by @woct0rdho in #37902
🌐 [i18n-KO] Translated gpu_selection.md to Korean by @nsbg in #36757
Add usage example for DINOv2 by @baldassarreFe in #37398
Aligning modling code for GPT2 to work with vLLM (fallback) by @ariG23498 in #36934
Break weight tying when quantizing input embedding by @jerryzh168 in #37905
[docs] logits docstring by @gante in #37929
[D-FINE] Update names by @NielsRogge in #37957
More fault tolerant notification service by @ivarflakstad in #37924
[core] reuse unused reserved cuda memory when loading models by @gante in #37920
Use T4 single GPU runner with more CPU RAM by @ydshieh in #37961
[generate] Fix vocab_size access for multimodal models by @kurzdev in #37937
Fix incorrect type annotation in get_auxiliary_logits by @Tanuj-rai in #37955
[Ready to Merge][HFQuantizer] Squelch pydantic warnings by @kylesayrs in #37726
Add GraniteMoeHybrid support for 4.0 by @Ssukriti in #37658
add xpu memory check by @faaany in #37969
[tests] Smaller model in slow cache tests by @gante in #37922
[llava] one pixel is missing from padding when length is odd by @cyr0930 in #37819
add job links to new model failure report by @ydshieh in #37973
fix docs serving typos. by @zhanluxianshen in #37936
Small typo lines 47 and 199 perf_infer_gpu_one.md by @nlhmnlhmnlhm in #37938
Fix typos by @omahs in #37978
[speech2text] fix init of sinusoidal embeddings by @gante in #37931
Fix typo by @lkm2835 in #37964
enable xpu in test_trainer by @yao-matrix in #37774
fix FSDP + torch.compile bug when saving pretrained model by @Joaquinecc in #37725
Enable granite speech 3.3 tests by @alex-jw-brooks in #37560
Fix donut backtracking by @Rocketknight1 in #37788
Fix Qwen models export with torch 2.7 by @guangy10 in #37985
[offload] respect max_memory argument when factoring in unused reserved memory by @gante in #37982
make aya vision 5 integration tests pass on xpu by @yao-matrix in #37990
[chat template] separate jinja logic from tokenizers by @zucchini-nlp in #37602
remove duplicate code by @kaixuanliu in #37991
Add a check to import_utils.py to allow for use of faiss_gpu installation by @Fiona-Waters in #37997
[CSM] tiny fix on generation by @eustlb in #38001
Fix pad image transform for batched inputs by @sebasv in #37544
Add ALL_ATTENTION_FUNCTIONS compatibility for Pixtral model by @uminaty in #37960
Enable RUF013 to enforce optional typing by @cyyever in #37266
Fix Optional typing by @qubvel in #38018
Print commit SHA on slack message for new model notification. by @ydshieh in #38019
[CI] remove duplicated message on GH comment to run slow tests by @gante in #37970
[caches] Raise exception on offloaded static caches + multi device by @gante in #37974
Skip test_push_to_hub_with_saves_each_epoch for now by @ydshieh in #38022
Fix incorrect installation instructions (for issue #37476) by @Zephyr271828 in #37640
Fix wording in torchscript.md by @Madghostek in #38004
[VLMs] support attention backends by @zucchini-nlp in #37576
make test_speculative_decoding_non_distil device-agnostic by @faaany in #38010
enable mamba2 integration cases on xpu by @yao-matrix in #38006
update bnb tests by @jiqing-feng in #38011
[AutoDocstring] Based on inspect parsing of the signature by @ArthurZucker and @yonigozlan in #33771
fix document masking for chunked attention by @winglian in #37429
make mistral3 pass on xpu by @yao-matrix in #37882
enable utils test cases on XPU by @yao-matrix in #38005
[Temporary] Log some information in some pytest/pluggy internal places by @ydshieh in #37996
Trigger CircleCI via GitHub Actions when ready for review by @ydshieh in #37885
Disable Trigger CircleCI via GitHub Actions when ready for review` by @ydshieh in #38038
Do not erase a cache_position passed explicitly to generate(), if there is one by @FremyCompany in #37986
Support for version spec in requires & arbitrary mismatching depths across folders by @LysandreJik in #37854
Re-Enable Trigger CircleCI via GitHub Actions when "ready for review" by @ydshieh in #37885)
Fix reduce-labels in BEIT Fast Image Processor by @simonreise in #38042
Fix cache update! by @Cyrilvallez in #38046
Fix linalg.norm for CovnNextV2 by @qubvel in #38015
enable generation fsdp/utils cases on XPU by @yao-matrix in #38009
fix(conversion): Fix size mismatch error during TF->PT model loading by @arjunaskykok in #38014
[VLM] fix loading issues by @zucchini-nlp in #38051
Fix OneFormer integration test by @qubvel in #38016
Add AMD expectation to test_gpt2_sample by @ivarflakstad in #38079
docs: fix md style by @imba-tjd in #38057
Fix mt5 test on AMD devices by @ivarflakstad in #38081
chore(qwen2): display warning log only when sliding window attention … by @edwardzjl in #36316
fix the inconsist docstring in apply_chat_template by @lenijwp in #38069
Fix tot update in trainer by @efsotr in #37923
update seed_worker to set seed based on worker_id and rank by @gathierry in #37980
uninstall kernels from docker images by @ydshieh in #38083
Refactor image processor phi4 by @yonigozlan in #36976
update require_read_token by @ydshieh in #38093
add timeout for downloading the librispeech_asr dataset by @faaany in #38073
fix: Propagate lr_scheduler_kwargs options to create LR Scheduler when LayerWiseDummyOptimizer is used by @BlackNoodle in #34559
Disable report callbacks for certain training tests by @ivarflakstad in #38088
[smolvlm] skip the test by @zucchini-nlp in #38099
Fix bug in prefill_chunk_size that ignores disable_compile flag by @xmarva in #38067
Fix past_key_values type hint in model output types by @ChengLyu in #37953
[bug] fix llava processor to calculate unpadding size correctly by @cyr0930 in #37988
fix check_bad commit.py gives wrong results by @ydshieh in #38107
Fix InternVL interpolate_pos_encoding and add to video_processing_auto by @yonigozlan in #38092
[CSM] update test for t4 runners by @eustlb in #38110
Add style bot by @SunMarc in #38102
Fix description and formatting errors in code docs by @bilibili12433014 in #38074
enable finegrained_fp8 and granite_speech cases on XPU by @yao-matrix in #38036
[video processor] fix tests by @zucchini-nlp in #38104
Fix temporal padding in Qwen2VLImageProcessor when the number of frames is not divisible by temporal_patch_size by @ritwickchaudhry in #38076
Fix auto batch size finder test by @ivarflakstad in #38125
Add config validation and style tweaks by @Kirire in #37589
Update trainer.md by @guspuffygit in #38113
[docs] add uv installation instructions for source builds by @arjunaskykok in #37968
Add manueldeprada to run_slow whitelist by @manueldeprada in #38126
enable d_fine finetuning properly by @SangbumChoi in #37962
Fix incorrect attention mask truncate in WhisperFlashAttention2 by @OliBomby in #36477
[Qwen3] Qwen3 MoE add tp plan for expert mlps by @hgt312 in #38135
enable csm integration cases on xpu, all passed by @yao-matrix in #38140
Remove head mask in generative models by @zucchini-nlp in #35786
Hotfix: Flash Attention 2 support in Pixtral by @uminaty in #38146
enable trainer test cases on xpu by @yao-matrix in #38138
disable deepspeed when setting up fake trainer by @winglian in #38101
Omit creation of positional IDs within ESM if applicable by @simonlevine in #38089
[FIX] Save speed metrics to logs by @pavelgein in #38136
enable autoround cases on XPU by @yao-matrix in #38167
Include output embedding as well with include_embedding flag by @jerryzh168 in #37935
Fix Qwen2.5 Omni SinusoidsPositionEmbedding precision by @BakerBunker in #38151
Add optional RMSNorm support to BitNet quantization (config + layers) by @Codys12 in #38087
[VLMs] add helpers to get multimodal encodings by @zucchini-nlp in #37743
Bart: new cache format by @zucchini-nlp in #35314
clean autoawq cases on xpu by @yao-matrix in #38163
Disable Trigger CircleCI by ready for review by @ydshieh in #38171
Disable convert to draft workflow by @ydshieh in #38177
remove some commands from fetch_tests CircleCI job by @ydshieh in #38176
Feat: add warnings for unused keys and rules in tensor parallel by @S1ro1 in #37893
[ESM] Add flash-attention-2 backend for ESM-2 by @pstjohn in #38023
Add args support for fast image processors by @yonigozlan in #37018
Fix import torchao.prototype.low_bit_optim since torchao v0.11 by @baptxste in #38174
fix bug in distributed loss test by @techkang in #38166
[tests] remove test_sdpa_equivalence (redundant) by @gante in #37911
Add Granite Speech Support by @alex-jw-brooks in #36801
Add glm4 by @ArthurZucker in #37388
Add Qwen2.5-Omni by @BakerBunker in #36752
Add MLCD model by @tanhuajie in #36182
Add TimesFM Time Series Forecasting Model by @jinan-zhou in #34082
Add Janus model by @yaswanth19 in #36053
Add InternVL (2.5 MPO) by @yonigozlan in #35968
Add Bitnet model by @MekkCyber in #37742
Samhq model addition by @sushmanthreddy in #35147
Add D-FINE Model into Transformers by @VladOS95-cyber in #36261
Add CSM model by @eustlb in #36719

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@cyyever
- Use Python 3.9 syntax in examples (#37279)
- Use Python 3.9 syntax in tests (#37343)
- Remove old code for PyTorch, Accelerator and tokenizers (#37234)
- Add weights_only=True to torch.load (#37062)
- Add XPU case to is_torch_bf16_gpu_available (#37132)
- Remove fsspec dependency which isn't directly used by transformers (#37318)
- Add Optional to remaining types (#37808)
- Enable RUF013 to enforce optional typing (#37266)
@yao-matrix
- enable 2 llama UT cases on xpu (#37126)
- enhance require_deterministic_for_xpu (#37437)
- make test_snowman_image_captioning pass on XPU, by sharing same atol w/ ROCM (#37480)
- fix and enhance pipeline_webserver.md (#36992)
- enable 5 cases on XPU (#37507)
- enable several cases on XPU (#37516)
- enable test_offloaded_cache_implementation on XPU (#37514)
- enable 3 mpt test cases on XPU (#37546)
- enable 6 rt_detr_v2 cases on xpu (#37548)
- enable 6 gemma2 cases on XPU (#37564)
- enable 6 modeling cases on XPU (#37571)
- fix 2 encoder_decoder issues on XPU (#37572)
- enable mllama cases on xpu (#37644)
- enable 6 granite cases on xpu (#37569)
- enable blip2 and emu3 cases on XPU (#37662)
- enable cpu offloading for Bark on xpu (#37599)
- enable 4 test_trainer cases on XPU (#37645)
- enable internvl UTs on XPU (#37779)
- enable xpu in test_trainer (#37774)
- make aya vision 5 integration tests pass on xpu (#37990)
- enable mamba2 integration cases on xpu (#38006)
- make mistral3 pass on xpu (#37882)
- enable utils test cases on XPU (#38005)
- enable generation fsdp/utils cases on XPU (#38009)
- enable finegrained_fp8 and granite_speech cases on XPU (#38036)
- enable csm integration cases on xpu, all passed (#38140)
- enable trainer test cases on xpu (#38138)
- enable autoround cases on XPU (#38167)
- clean autoawq cases on xpu (#38163)
@alex-jw-brooks
- Expose blip2qformer (#37254)
- Add Granite Speech Support (#36801)
- Fix qwen2audio wanr -> warn (#37559)
- Allow Exclusion of Input IDs from RepetitionPenaltyLogitsProcessor (#37625)
- Enable granite speech 3.3 tests (#37560)
@BakerBunker
- Add Qwen2.5-Omni (#36752)
- Fix inference bugs in Qwen2.5 Omni (#37701)
- Fix embeds_to_talker device in Qwen2.5-Omni (#37739)
- Fix Qwen2.5 Omni SinusoidsPositionEmbedding precision (#38151)
@rootonchair
- Add Fast Image Processor for Perceiver (#37176)
- Add Fast Image Processor for Flava (#37135)
- Add Fast Image Processor for LayoutLMv2 (#37203)
- Add Fast Image Processor for LayoutLMv3 (#37201)
- Add Fast Image Processor for Donut (#37081)
- Bridgetower fast image processor (#37373)
- Add Fast Image Processor for PoolFormer (#37182)
@flukeskywalker
- Fix mask handling for flex attention in llama/gemma2/mistral/qwen2 (#37381)
@keetrap
- Add Fast LeViT Processor (#37154)
- Add Fast Mobilenet-V2 Processor (#37113)
- Add Fast owlvit Processor (#37164)
- Add Fast Yolos Processor (#37292)
- Add Fast Chinese-CLIP Processor (#37012)
- Add Fast Conditional-DETR Processor (#37071)
- Add Fast Grounding-Dino Processor (#37108)
- Add Fast PVT Processor (#37204)
@tanhuajie
- Add MLCD model (#36182)
@jinan-zhou
- Add TimesFM Time Series Forecasting Model (#34082)
@yaswanth19
- Add Janus model (#36053)
@saswatmeher
- chore: update model card for SigLIP (#37585)
- chore: update SigLIP2 model card (#37624)
@cyr0930
- [fix] make legacy bnb code work (#37331)
- [llava] one pixel is missing from padding when length is odd (#37819)
- [bug] fix llava processor to calculate unpadding size correctly (#37988)
@wenhuach21
- Add AutoRound quantization support (#37393)
@devxaitist
- 🌐 [i18n-KO] Translated siglip.md to Korean (#37145)
- Add Fast Image Processor for vilt (#37304)
@co63oc
- Fix typos in comments (#37694)
- Fix typos in strings and comments (#37784)
- Fix typos in strings and comments (#37799)
- Fix typos in strings and comments (#37910)
@guangy10
- Gemma3 is Torch Exportable (#37728)
- Allow override inputs to export recipe (#37508)
- Fix Qwen models export with torch 2.7 (#37985)
@sushmanthreddy
- Samhq model addition (#35147)
@VladOS95-cyber
- Add D-FINE Model into Transformers (#36261)
@Ssukriti
- Add GraniteMoeHybrid support for 4.0 (#37658)

Open source →

4.50.021 Mar 2025Security

Security fix for benchmark.yml by @ydshieh in #36402

Release v4.50.0

New Model Additions

Model-based releases

Starting with version v4.49.0, we have been doing model-based releases, additionally to our traditional, software-based monthly releases. These model-based releases provide a tag from which models may be installed.

Contrarily to our software-releases; these are not pushed to pypi and are kept on our GitHub. Each release has a tag attributed to it, such as:

v4.49.0-Gemma-3
v4.49.0-AyaVision

⚠️ As bugs are identified and fixed on each model, the release tags are updated so that installing from that tag always gives the best experience possible with that model.

Each new model release will always be based on the current state of the main branch at the time of its creation. This ensures that new models start with the latest features and fixes available.

For example, if two models—Gemma-3 and AyaVision—are released from main, and then a fix for gemma3 is merged, it will look something like this:

              o---- v4.49.0-Gemma-3 (includes AyaVision, plus main fixes)
            /                  \  
---o--o--o--o--o-- (fix for gemma3) --o--o--o main
       \          
        o---- v4.49.0-AyaVision

We strive to merge model specific fixes on their respective branches as fast as possible!

Gemma 3

Gemma 3 is heavily referenced in the following model-based release and we recommend reading these if you want all the information relative to that model.

The Gemma 3 model was proposed by Google. It is a vision-language model composed by a SigLIP vision encoder and a Gemma 2 language decoder linked by a multimodal linear projection.

It cuts an image into a fixed number of tokens same way as Siglip if the image does not exceed certain aspect ratio. For images that exceed the given aspect ratio, it crops the image into multiple smaller pacthes and concatenates them with the base image embedding.

One particularity is that the model uses bidirectional attention on all the image tokens. Also, the model interleaves sliding window local attention with full causal attention in the language backbone, where each sixth layer is a full causal attention layer.

Gemma3 by @RyanMullins in #36658

Shield Gemma2

ShieldGemma 2 is built on Gemma 3, is a 4 billion (4B) parameter model that checks the safety of both synthetic and natural images against key categories to help you build robust datasets and models. With this addition to the Gemma family of models, researchers and developers can now easily minimize the risk of harmful content in their models across key areas of harm as defined below:

No Sexually Explicit content: The image shall not contain content that depicts explicit or graphic sexual acts (e.g., pornography, erotic nudity, depictions of rape or sexual assault).
No Dangerous Content: The image shall not contain content that facilitates or encourages activities that could cause real-world harm (e.g., building firearms and explosive devices, promotion of terrorism, instructions for suicide).
No Violence/Gore content: The image shall not contain content that depicts shocking, sensational, or gratuitous violence (e.g., excessive blood and gore, gratuitous violence against animals, extreme injury or moment of death).

We recommend using ShieldGemma 2 as an input filter to vision language models, or as an output filter of image generation systems. To train a robust image safety model, we curated training datasets of natural and synthetic images and instruction-tuned Gemma 3 to demonstrate strong performance.

Shieldgemma2 #36678 by @RyanMullins

Aya Vision

AyaVision is heavily referenced in the following model-based release and we recommend reading these if you want all the information relative to that model.

The Aya Vision 8B and 32B models is a state-of-the-art multilingual multimodal models developed by Cohere For AI. They build on the Aya Expanse recipe to handle both visual and textual information without compromising on the strong multilingual textual performance of the original model.

Aya Vision 8B combines the Siglip2-so400-384-14 vision encoder with the Cohere CommandR-7B language model further post-trained with the Aya Expanse recipe, creating a powerful vision-language model capable of understanding images and generating text across 23 languages. Whereas, Aya Vision 32B uses Aya Expanse 32B as the language model.

Key features of Aya Vision include:

Multimodal capabilities in 23 languages
Strong text-only multilingual capabilities inherited from CommandR-7B post-trained with the Aya Expanse recipe and Aya Expanse 32B
High-quality visual understanding using the Siglip2-so400-384-14 vision encoder
Seamless integration of visual and textual information in 23 languages.

Add aya by @ArthurZucker in #36521

Mistral 3.1

Mistral 3.1 is heavily referenced in the following model-based release and we recommend reading these if you want all the information relative to that model.

Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.

It is ideal for:

Fast-response conversational agents.
Low-latency function calling.
Subject matter experts via fine-tuning.
Local inference for hobbyists and organizations handling sensitive data.
Programming and math reasoning.
Long document understanding.
Visual understanding.

Add Mistral3 by @Cyrilvallez in #36790

Smol VLM 2

SmolVLM-2 is heavily referenced in the following model-based release and we recommend reading these if you want all the information relative to that model.

SmolVLM2 is an adaptation of the Idefics3 model with two main differences:

It uses SmolLM2 for the text model.
It supports multi-image and video inputs

SmolVLM2 by @orrzohar in #36126

SigLIP-2

SigLIP-2 is heavily referenced in the following model-based release and we recommend reading these if you want all the information relative to that model.

The SigLIP2 model was proposed in SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features by Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner and Xiaohua Zhai.

The model comes in two variants

FixRes - model works with fixed resolution images (backward compatible with SigLIP v1)
NaFlex - model works with variable image aspect ratios and resolutions (SigLIP2 in transformers)

Add SigLIP 2 by @qubvel in #36323

Prompt Depth Anything

PromptDepthAnything is a high-resolution, accurate metric depth estimation model that leverages prompting, inspired by its success in vision-language (VLMs) and large language models (LLMs). Using iPhone LiDAR as a prompt, the model generates precise depth maps at up to 4K resolution, unlocking the potential of depth foundation models.

Add Prompt Depth Anything Model by @haotongl in #35401

New tool: attention visualization

We add a new tool to transformers to visualize the attention layout of a given model. It only requires a model ID as input, and will load the relevant tokenizer/model and display what the attention mask looks like. Some examples:


from transformers.utils.attention_visualizer import AttentionMaskVisualizer
visualizer = AttentionMaskVisualizer("meta-llama/Llama-3.2-3B-Instruct")
visualizer("A normal attention mask")

visualizer = AttentionMaskVisualizer("mistralai/Mistral-Small-24B-Instruct-2501")
visualizer("A normal attention mask with a long text to see how it is displayed, and if it is displayed correctly")

visualizer = AttentionMaskVisualizer("google/paligemma2-3b-mix-224")
visualizer("<img> You are an assistant.", suffix = "What is on the image?")

visualizer = AttentionMaskVisualizer("google/gemma-2b")
visualizer("You are an assistant. Make sure you print me") # we should have slidiing on non sliding side by side

visualizer = AttentionMaskVisualizer("google/gemma-3-27b-it")
visualizer("<img>You are an assistant. Make sure you print me") # we should have slidiing on non sliding side by side

Add attention visualization tool by @ArthurZucker in #36630

Deprecating transformers.agents in favor of smolagents

We are deprecating transformers.agents in favour of the smolagents library. Read more about smolagents here.

Deprecate transformers.agents by @aymeric-roucher in #36415

Quantization

We support adding custom quantization method by using the @register_quantization_config and @register_quantizer decorator:

@register_quantization_config("custom")
class CustomConfig(QuantizationConfigMixin):
   pass

@register_quantizer("custom")
class CustomQuantizer(HfQuantizer):
   pass

quantized_model = AutoModelForCausalLM.from_pretrained(
    "facebook/opt-350m", quantization_config=CustomConfig(), torch_dtype="auto"
)

Added Support for Custom Quantization by @keetrap in #35915
Add Example for Custom quantization by @MekkCyber in #36286

AMD is developing its in-house quantizer named Quark released under MIT license, which supports a broad range of quantization pre-processing, algorithms, dtypes and target hardware. You can now load a model quantized by quark library:

# pip install amd-quark

model_id = "EmbeddedLLM/Llama-3.1-8B-Instruct-w_fp8_per_channel_sym"
model = AutoModelForCausalLM.from_pretrained(model_id)
model = model.to("cuda")

Support loading Quark quantized models in Transformers by @fxmarty-amd and @BowenBao in #36372

Torchao is augmented with autoquant support, CPU-quantization, as well as new AOBaseConfig object instances for more advanced configuration.

Add autoquant support for torchao quantizer by @jerryzh168 in #35503
enable torchao quantization on CPU by @jiqing-feng in #36146
Add option for ao base configs by @drisspg in #36526

Tensor Parallelism implementation changes

At loading time, the parallelization is now applied module-by-module, so that no memory overhead is required compared to what the final weight distribution will be!

TP initialization module-by-module by @Cyrilvallez in #35996

Generation

This release includes two speed upgrades to generate:

Assisted generation now works with ANY model as an assistant, even with do_sample=True;

from transformers import pipeline
import torch

prompt = "Alice and Bob"
checkpoint = "google/gemma-2-9b"
assistant_checkpoint = "double7/vicuna-68m"

pipe = pipeline(
    "text-generation",
    model=checkpoint,
    assistant_model=assistant_checkpoint,
    do_sample=True
)
pipe_output = pipe(prompt, max_new_tokens=50, do_sample=True)
print(pipe_output[0]["generated_text"])

Beam search was vectorized, and should be significantly faster with a large num_beams. The speedup is more visible on smaller models, where model.forward doesn't dominate the total run time.

Universal Speculative Decoding CandidateGenerator by @keyboardAnt, @jmamou, and @gauravjain14 in #35029
[generate] ✨ vectorized beam search ✨ by @gante in #35802

Documentation

A significant redesign of our documentation has wrapped-up. The goal was to greatly simplify the transformers documentation, making it much more easy to navigate. Let us know what you think!

[docs] Redesign by @stevhliu in #31757

Notable repo maintenance

The research examples folder that was hosted in transformers is no more. We have moved it out of transformers and in the following repo: github.com/huggingface/transformers-research-projects/

Remove research projects by @Rocketknight1 in #36645

We have updated our flex attention support so as to have it be on-par with our Flash Attention 2 support.

Proper_flex by @ArthurZucker in #36643

More models support flex attention now thanks to @qubvel

Refactor Attention implementation for ViT-based models by @qubvel in #36545

First integration of hub kernels for deformable detr!

Use deformable_detr kernel from the Hub (#36853) by @danieldk

Bugfixes and improvements

[tests] fix EsmModelIntegrationTest::test_inference_bitsandbytes by @faaany in #36225
Fix LlavaForConditionalGenerationModelTest::test_config after #36077 by @ydshieh in #36230
AMD DeepSpeed image additional HIP dependencies by @ivarflakstad in #36195
[generate] remove cache v4.47 deprecations by @gante in #36212
Add missing atol to torch.testing.assert_close where rtol is specified by @ivarflakstad in #36234
[tests] remove tf/flax tests in /generation by @gante in #36235
[generate] Fix encoder decoder models attention mask by @eustlb in #36018
Add compressed tensor in quant dockerfile by @SunMarc in #36239
[tests] remove test_export_to_onnx by @gante in #36241
Au revoir flaky test_fast_is_faster_than_slow by @ydshieh in #36240
Fix TorchAoConfig not JSON serializable by @andrewor14 in #36206
Remove flakiness in VLMs by @zucchini-nlp in #36242
feat: add support for tensor parallel training workflow with accelerate by @kmehant in #34194
Fix XGLM loss computation (PyTorch and TensorFlow) by @damianoamatruda in #35878
GitModelIntegrationTest - flatten the expected slice tensor by @ivarflakstad in #36260
Added Support for Custom Quantization by @keetrap in #35915
Qwen2VL fix cos,sin dtypes to float when used with deepspeed by @ArdalanM in #36188
Uniformize LlavaNextVideoProcessor kwargs by @yonigozlan in #35613
Add support for post-processing kwargs in image-text-to-text pipeline by @yonigozlan in #35374
Add dithering to the Speech2TextFeatureExtractor API. by @KarelVesely84 in #34638
[tests] remove pt_tf equivalence tests by @gante in #36253
TP initialization module-by-module by @Cyrilvallez in #35996
[tests] deflake dither test by @gante in #36284
[tests] remove flax-pt equivalence and cross tests by @gante in #36283
[tests] make test_from_pretrained_low_cpu_mem_usage_equal less flaky by @gante in #36255
Add Example for Custom quantization by @MekkCyber in #36286
docs: Update README_zh-hans.md by @hyjbrave in #36269
Fix callback handler reference by @SunMarc in #36250
Make cache traceable by @IlyasMoutawwakil in #35873
Fix broken CI on release branch due to missing conversion files by @ydshieh in #36275
Ignore conversion files in test fetcher by @ydshieh in #36251
SmolVLM2 by @orrzohar in #36126
Fix typo in Pixtral example by @12v in #36302
fix: prevent second save in the end of training if last step was saved already by @NosimusAI in #36219
[smolvlm] make CI green by @gante in #36306
Fix default attention mask of generate in MoshiForConditionalGeneration by @cyan-channel-io in #36171
VLMs: even more clean-up by @zucchini-nlp in #36249
Add SigLIP 2 by @qubvel in #36323
[CI] Check test if the GenerationTesterMixin inheritance is correct 🐛 🔫 by @gante in #36180
[tests] make quanto tests device-agnostic by @faaany in #36328
Uses Collection in transformers.image_transforms.normalize by @CalOmnie in #36301
Fix exploitable regexes in Nougat and GPTSan/GPTJNeoXJapanese by @Rocketknight1 in #36121
[tests] enable bnb tests on xpu by @faaany in #36233
Improve model loading for compressed tensor models by @rahul-tuli in #36152
Change slack channel for mi250 CI to amd-hf-ci by @ivarflakstad in #36346
Add autoquant support for torchao quantizer by @jerryzh168 in #35503
Update amd pytorch index to match base image by @ivarflakstad in #36347
fix(type): padding_side type should be Optional[str] by @shenxiangzhuang in #36326
[Modeling] Reduce runtime when loading missing keys by @kylesayrs in #36312
notify new model merged to main by @ydshieh in #36375
Update modeling_llava_onevision.py by @yinsong1986 in #36391
Load models much faster on accelerator devices!! by @Cyrilvallez in #36380
[modular] Do not track imports in functions by @Cyrilvallez in #36279
Fix is_causal fail with compile by @Cyrilvallez in #36374
enable torchao quantization on CPU by @jiqing-feng in #36146
Update _get_eval_sampler to reflect Trainer.tokenizer is deprecation self.tokenizer -> self.processing_class by @yukiman76 in #36315
Fix doc formatting in forward passes & modular by @Cyrilvallez in #36243
Added handling for length <2 of suppress_tokens for whisper by @andreystarenky in #36336
addressing the issue #34611 to make FlaxDinov2 compatible with any batch size by @MHRDYN7 in #35138
tests: revert change of torch_require_multi_gpu to be device agnostic by @dvrogozh in #35721
[tests] enable autoawq tests on XPU by @faaany in #36327
fix audio classification pipeline fp16 test on cuda by @jiqing-feng in #36359
chore: fix function argument descriptions by @threewebcode in #36392
Fix pytorch integration tests for SAM by @qubvel in #36397
[CLI] add import guards by @gante in #36376
Fix convert_to_rgb for SAM ImageProcessor by @MSt-10 in #36369
Security fix for benchmark.yml by @ydshieh in #36402
Fixed VitDet for non-squre Images by @cjfghk5697 in #35969
Add retry hf hub decorator by @muellerzr in #35213
Deprecate transformers.agents by @aymeric-roucher in #36415
Fixing the docs corresponding to the breaking change in torch 2.6. by @Narsil in #36420
add recommendations for NPU using flash_attn by @zheliuyu in #36383
fix: prevent model access error during Optuna hyperparameter tuning by @emapco in #36395
Universal Speculative Decoding CandidateGenerator by @keyboardAnt in #35029
Fix compressed tensors config by @MekkCyber in #36421
Update form pretrained to make TP a first class citizen by @ArthurZucker in #36335
Fix Expected output for compressed-tensors tests by @MekkCyber in #36425
restrict cache allocator to non quantized model by @SunMarc in #36428
Change PR to draft when it is (re)opened by @ydshieh in #36417
Fix permission by @ydshieh in #36443
Fix another permission by @ydshieh in #36444
Add contents: write by @ydshieh in #36445
[save_pretrained ] Skip collecting duplicated weight by @wejoncy in #36409
[generate] torch.distributed-compatible DynamicCache by @gante in #36373
Lazy import libraries in src/transformers/image_utils.py by @hmellor in #36435
Fix hub_retry by @ydshieh in #36449
[GroundingDino] Fix grounding dino loss 🚨 by @EduardoPach in #31828
Fix loading models with mismatched sizes by @qubvel in #36463
[docs] fix bug in deepspeed config by @faaany in #36081
Add Got-OCR 2 Fast image processor and refactor slow one by @yonigozlan in #36185
Fix couples of issues from #36335 by @SunMarc in #36453
Fix _load_state_dict_into_meta_model with device_map=None by @hlky in #36488
Fix loading zero3 weights by @muellerzr in #36455
Check TRUST_REMOTE_CODE for RealmRetriever for security by @ydshieh in #36511
Fix kwargs UserWarning in SamImageProcessor by @MSt-10 in #36479
fix torch_dtype, contiguous, and load_state_dict regression by @SunMarc in #36512
Fix some typos in docs by @co63oc in #36502
chore: fix message descriptions in arguments and comments by @threewebcode in #36504
Fix pipeline+peft interaction by @Rocketknight1 in #36480
Fix edge case for continue_final_message by @Rocketknight1 in #36404
[Style] fix E721 warnings by @kashif in #36474
Remove unused code by @Rocketknight1 in #36459
[docs] Redesign by @stevhliu in #31757
Add aya by @ArthurZucker in #36521
chore: Fix typos in docs and examples by @co63oc in #36524
Fix bamba tests amd by @ivarflakstad in #36535
Fix links in quantization doc by @MekkCyber in #36528
chore: enhance messages in docstrings by @threewebcode in #36525
guard torch version for uint16 by @SunMarc in #36520
Fix typos in tests by @co63oc in #36547
Fix typos . by @zhanluxianshen in #36551
chore: enhance message descriptions in parameters,comments,logs and docstrings by @threewebcode in #36554
Delete redundancy if case in model_utils by @zhanluxianshen in #36559
Modular Conversion --fix_and_overwrite on Windows by @hlky in #36583
Integrate SwanLab for offline/online experiment tracking and local visualization by @ShaohonChen in #36433
[bark] fix loading of generation config by @gante in #36587
[XGLM] tag tests as slow by @gante in #36592
fix: argument by @ariG23498 in #36558
Mention UltraScale Playbook 🌌 in docs by @NouamaneTazi in #36589
avoid errors when the size of input_ids passed to PrefixConstrainedLogitsProcessor is zero by @HiDolen in #36489
Export base streamer. by @AndreasAbdi in #36500
Github action for auto-assigning reviewers by @Rocketknight1 in #35846
Update chat_extras.md with content correction by @krishkkk in #36599
Update "who to tag" / "who can review" by @gante in #36394
Fixed datatype related issues in DataCollatorForLanguageModeling by @capemox in #36457
Fix check for XPU. PyTorch >= 2.6 no longer needs ipex. by @tripzero in #36593
[HybridCache] disable automatic compilation by @gante in #36620
Fix auto-assign reviewers by @Rocketknight1 in #36631
chore: fix typos in language models by @threewebcode in #36586
[docs] Serving LLMs by @stevhliu in #36522
Refactor some core stuff by @ArthurZucker in #36539
Fix bugs in mllama image processing by @tjohnson31415 in #36156
Proper_flex by @ArthurZucker in #36643
Fix AriaForConditionalGeneration flex attn test by @ivarflakstad in #36604
Remove remote code warning by @Rocketknight1 in #36285
Stop warnings from unnecessary torch.tensor() overuse by @Rocketknight1 in #36538
[docs] Update docs dependency by @stevhliu in #36635
Remove research projects by @Rocketknight1 in #36645
Fix gguf docs by @SunMarc in #36601
fix typos in the docs directory by @threewebcode in #36639
Gemma3 by @RyanMullins in #36658
HPU support by @IlyasMoutawwakil in #36424
fix block mask typing by @ArthurZucker in #36661
[CI] gemma 3 make fix-copies by @gante in #36664
Fix bnb regression due to empty state dict by @SunMarc in #36663
[core] Large/full refactor of from_pretrained by @Cyrilvallez in #36033
Don't accidentally mutate the base_model_tp_plan by @Rocketknight1 in #36677
Fix Failing GPTQ tests by @MekkCyber in #36666
Remove hardcoded slow image processor class in processors supporting fast ones by @yonigozlan in #36266
[quants] refactor logic for modules_to_not_convert by @SunMarc in #36672
Remove differences between init and preprocess kwargs for fast image processors by @yonigozlan in #36186
Refactor siglip2 fast image processor by @yonigozlan in #36406
Fix rescale normalize inconsistencies in fast image processors by @yonigozlan in #36388
[Cache] Don't initialize the cache on meta device by @gante in #36543
Update config.torch_dtype correctly by @SunMarc in #36679
Fix slicing for 0-dim param by @SunMarc in #36580
Changing the test model in Quanto kv cache by @MekkCyber in #36670
fix wandb hp search unable to resume from sweep_id by @bd793fcb in #35883
Upgrading torch version and cuda version in quantization docker by @MekkCyber in #36264
Change Qwen2_VL image processors to have init and call accept the same kwargs by @yonigozlan in #36207
fix type annotation for ALL_ATTENTION_FUNCTIONS by @WineChord in #36690
Fix dtype for params without tp_plan by @Cyrilvallez in #36681
chore: fix typos in utils module by @threewebcode in #36668
[CI] Automatic rerun of certain test failures by @gante in #36694
Add loading speed test by @Cyrilvallez in #36671
fix: fsdp sharded state dict wont work for save_only_model knob by @kmehant in #36627
Handling an exception related to HQQ quantization in modeling by @MekkCyber in #36702
Add GGUF support to T5-Encoder by @Isotr0py in #36700
Final CI cleanup by @Rocketknight1 in #36703
Add support for fast image processors in add-new-model-like CLI by @yonigozlan in #36313
Gemma3 processor typo by @Kuangdd01 in #36710
Make the flaky list a little more general by @Rocketknight1 in #36704
Cleanup the regex used for doc preprocessing by @Rocketknight1 in #36648
[model loading] don't gc.collect() if only 1 shard is used by @gante in #36721
Fix/best model checkpoint fix by @seanswyi in #35885
Try working around the processor registration bugs by @Rocketknight1 in #36184
[tests] Parameterized test_eager_matches_sdpa_inference by @gante in #36650
🌐 [i18n-KO] Translated codegen.md to Korean by @maximizemaxwell in #36698
Fix post_init() code duplication by @Cyrilvallez in #36727
Fix grad accum arbitrary value by @IlyasMoutawwakil in #36691
[Generation, Gemma 3] When passing a custom generation_config, overwrite default values with the model's base generation_config by @gante in #36684
🚨🚨🚨 Fix sdpa in SAM and refactor relative position embeddings by @geetu040 in #36422
enable/disable compile for quants methods by @SunMarc in #36519
fix can_generate by @jiqing-feng in #36570
Allow ray datasets to be used with trainer by @FredrikNoren in #36699
fix xpu tests by @jiqing-feng in #36656
Fix test isolation for clear_import_cache utility by @sambhavnoobcoder in #36345
Fix TrainingArguments.torch_empty_cache_steps post_init check by @pkuderov in #36734
[MINOR:TYPO] Update hubert.md by @cakiki in #36733
[CI] remove redundant checks in test_eager_matches_sdpa_inference by @gante in #36740
[docs] Update README by @stevhliu in #36265
doc: Clarify is_decoder usage in PretrainedConfig documentation by @d-kleine in #36724
fix typos in the tests directory by @threewebcode in #36717
chore: fix typos in tests directory by @threewebcode in #36785
Fixing typo in gemma3 image_processor_fast and adding a small test by @Zebz13 in #36776
Fix gemma3_text tokenizer in mapping by @LysandreJik in #36793
Add Mistral3 by @Cyrilvallez in #36790
fix hqq due to recent modeling changes by @SunMarc in #36771
Update SHA for tj-actions/changed-files by @ydshieh in #36795
Loading optimizations by @Cyrilvallez in #36742
Fix Mistral3 tests by @yonigozlan in #36797
Fix casting dtype for qunatization by @SunMarc in #36799
Fix chameleon's TypeError because inputs_embeds may None by @YenFuLin in #36673
Support custom dosctrings in modular by @yonigozlan in #36726
[generate] ✨ vectorized beam search ✨ by @gante in #35802
Expectations test utils by @ivarflakstad in #36569
fix "Cannot copy out of meta tensor; no data!" issue for BartForConditionalGeneration model by @yao-matrix in #36572
Remove dist": "loadfile" for pytest in CircleCI jobs by @ydshieh in #36811
Fix Device map for bitsandbytes tests by @MekkCyber in #36800
[Generation] remove leftover code from end-to-end compilation by @gante in #36685
Add attention visualization tool by @ArthurZucker in #36630
Add option for ao base configs by @drisspg in #36526
enable OffloadedCache on XPU from PyTorch 2.7 by @yao-matrix in #36654
[gemma 3] multimodal checkpoints + AutoModelForCausalLM by @gante in #36741
One more fix for reviewer assignment by @Rocketknight1 in #36829
Support tracable dynamicKVcache by @tugsbayasgalan in #36311
Add Space to Bitsandbytes doc by @MekkCyber in #36834
quick fix fast_image_processor register error by @JJJYmmm in #36716
Update configuration_qwen2.py by @michaelfeil in #36735
Just import torch AdamW instead by @Rocketknight1 in #36177
Move the warning to the documentation for DataCollatorWithFlattening by @qgallouedec in #36707
Fix swanlab global step by @Zeyi-Lin in #36728
Disable inductor config setter by default by @HDCharles in #36608
[ForCausalLMLoss] allow users to pass shifted labels by @stas00 in #36607
fix tiktoken convert to pass AddedToken to Tokenizer by @itazap in #36566
Saving Trainer.collator.tokenizer in when Trainer.processing_class is None by @innerNULL in #36552
Pass num_items_in_batch directly to loss computation by @eljandoubi in #36753
Fix fp16 ONNX export for RT-DETR and RT-DETRv2 by @qubvel in #36460
Update deprecated Jax calls by @rasmi in #35919
[qwen2 audio] remove redundant code and update docs by @gante in #36282
Pass state dict by @phos-phophy in #35234
[modular] Sort modular skips by @gante in #36304
[generate] clarify docstrings: when to inherit GenerationMixin by @gante in #36605
Update min safetensors bis by @SunMarc in #36823
Fix import for torch 2.0, 2.1 - guard typehint for "device_mesh" by @qubvel in #36768
Gemma 3: Adding explicit GenerationConfig and refactoring conversion … by @RyanMullins in #36833
Fix: remove the redundant snippet of _whole_word_mask by @HuangBugWei in #36759
Shieldgemma2 by @RyanMullins in #36678
Fix ONNX export for sequence classification head by @echarlaix in #36332
Fix hqq skipped modules and dynamic quant by @mobicham in #36821
Use pyupgrade --py39-plus to improve code by @cyyever in #36843
Support loading Quark quantized models in Transformers by @fxmarty-amd in #36372
DeepSpeed tensor parallel+ZeRO by @inkcherry in #36825
Refactor Attention implementation for ViT-based models by @qubvel in #36545
Add Prompt Depth Anything Model by @haotongl in #35401
Add model visual debugger by @molbap in #36798
[torchao] revert to get_apply_tensor_subclass by @SunMarc in #36849
Gemma3: fix test by @zucchini-nlp in #36820
[CI] fix update metadata job by @gante in #36850
Add support for seed in DataCollatorForLanguageModeling by @capemox in #36497
Refactor Aya Vision with modular by @yonigozlan in #36688
Mllama: raise better error by @zucchini-nlp in #35934
[CI] doc builder without custom image by @gante in #36862
FIX FSDP plugin update for QLoRA by @BenjaminBossan in #36720
Remove call to .item in get_batch_samples by @regisss in #36861
chore: fix typos in the tests directory by @threewebcode in #36813
Make ViTPooler configurable by @sebbaur in #36517
Revert "Update deprecated Jax calls by @ArthurZucker in #35919)"
[generate] model defaults being inherited only happens for newer models by @gante in #36881
:red_circle: :red_circle: :red_circle: supersede paligemma forward to shift pos id indexing by @molbap in #36859
Gemma 3 tests expect greedy decoding by @molbap in #36882
Use deformable_detr kernel from the Hub by @danieldk in #36853
Minor Gemma 3 fixes by @molbap in #36884
Fix: dtype cannot be str by @zucchini-nlp in #36262

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@IlyasMoutawwakil
- Make cache traceable (#35873)
- HPU support (#36424)
- Fix grad accum arbitrary value (#36691)
@orrzohar
- SmolVLM2 (#36126)
@threewebcode
- chore: fix function argument descriptions (#36392)
- chore: fix message descriptions in arguments and comments (#36504)
- chore: enhance messages in docstrings (#36525)
- chore: enhance message descriptions in parameters,comments,logs and docstrings (#36554)
- chore: fix typos in language models (#36586)
- fix typos in the docs directory (#36639)
- chore: fix typos in utils module (#36668)
- fix typos in the tests directory (#36717)
- chore: fix typos in tests directory (#36785)
- chore: fix typos in the tests directory (#36813)
@aymeric-roucher
- Deprecate transformers.agents (#36415)
@keyboardAnt
- Universal Speculative Decoding CandidateGenerator (#35029)
@EduardoPach
- [GroundingDino] Fix grounding dino loss 🚨 (#31828)
@co63oc
- Fix some typos in docs (#36502)
- chore: Fix typos in docs and examples (#36524)
- Fix typos in tests (#36547)
@RyanMullins
- Gemma3 (#36658)
- Gemma 3: Adding explicit GenerationConfig and refactoring conversion … (#36833)
- Shieldgemma2 (#36678)
@cyyever
- Use pyupgrade --py39-plus to improve code (#36843)
@haotongl
- Add Prompt Depth Anything Model (#35401)
@danieldk
- Use deformable_detr kernel from the Hub (#36853)

Open source →

4.49.017 Feb 2025Security

Security fix for self-comment-ci.yml by @ydshieh in #35548

New models

Helium

Helium-1 preview is a lightweight language model with 2B parameters, targeting edge and mobile devices. It supports the following languages: English, French, German, Italian, Portuguese, Spanish.

Add-helium by @ArthurZucker in #35669

Qwen2.5-VL

The Qwen2.5-VL model is an update to Qwen2-VL from Qwen team, Alibaba Group.

The abstract from this update is the following:

Qwen2.5-VL marks a major step forward from Qwen2-VL, built upon the latest Qwen2.5 LLM. We’ve accelerated training and testing through the strategic implementation of window attention within the ViT. The ViT architecture itself has been refined with SwiGLU and RMSNorm, aligning it more closely with the LLM’s structure. A key innovation is the expansion of native dynamic resolution to encompass the temporal dimension, in addition to spatial aspects. Furthermore, we’ve upgraded MRoPE, incorporating absolute time alignment on the time axis to allow the model to effectively capture temporal dynamics, regardless of frame rate, leading to superior video understanding.

add qwen2.5vl by @ShuaiBai623 in #35569

SuperGlue

The SuperGlue model was proposed in SuperGlue: Learning Feature Matching with Graph Neural Networks by Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz and Andrew Rabinovich.

This model consists of matching two sets of interest points detected in an image. Paired with the SuperPoint model, it can be used to match two images and estimate the pose between them. This model is useful for tasks such as image matching, homography estimation, etc.

Add SuperGlue model by @sbucaille in #29886

Granite Vision Support

The Granite Vision model is a variant of LLaVA-NeXT, leveraging a Granite language model alongside a SigLIP visual encoder. It utilizes multiple concatenated vision hidden states as its image features, similar to VipLlava. It also uses a larger set of image grid pinpoints than the original LlaVa-NeXT models to support additional aspect ratios.

Granite Vision Support by @alex-jw-brooks in #35579

Zamba2

Zamba2 is a large language model (LLM) trained by Zyphra, and made available under an Apache 2.0 license.

Zamba2-1.2B, Zamba2-2.7B and Zamba2-7B are hybrid models combining state-space models (Specifically Mamba) and transformer, and were trained using next-token prediction. Zamba2 uses shared transformer layers after every 6 mamba blocks. It uses the Mistral v0.1 tokenizer. We came to this architecture after a series of ablations at small scales. Zamba2-1.2B, Zamba2-2.7B and Zamba2-7B were pre-trained on 2T and 3T tokens, respectively.

Add Zamba2 by @pglorio in #34517

GOT-OCR 2.0

GOT-OCR2 works on a wide range of tasks, including plain document OCR, scene text OCR, formatted document OCR, and even OCR for tables, charts, mathematical formulas, geometric shapes, molecular formulas and sheet music. While this implementation of the model will only output plain text, the outputs can be further processed to render the desired format, with packages like pdftex, mathpix, matplotlib, tikz, verovio or pyecharts. The model can also be used for interactive OCR, where the user can specify the region to be recognized by providing the coordinates or the color of the region’s bounding box.

Add GOT-OCR 2.0 to Transformers by @yonigozlan in #34721

DAB-DETR

DAB-DETR is an enhanced variant of Conditional DETR. It utilizes dynamically updated anchor boxes to provide both a reference query point (x, y) and a reference anchor size (w, h), improving cross-attention computation. This new approach achieves 45.7% AP when trained for 50 epochs with a single ResNet-50 model as the backbone.

Add DAB-DETR for object detection by @conditionedstimulus in #30803

Depth PRO

DepthPro is a foundation model for zero-shot metric monocular depth estimation, designed to generate high-resolution depth maps with remarkable sharpness and fine-grained details. It employs a multi-scale Vision Transformer (ViT)-based architecture, where images are downsampled, divided into patches, and processed using a shared Dinov2 encoder. The extracted patch-level features are merged, upsampled, and refined using a DPT-like fusion stage, enabling precise depth estimation.

Add Apple's Depth-Pro for depth estimation by @geetu040 in #34583

RT-DETRv2

An improved Real-Time DEtection TRansformer (RT-DETR). RT-DETRv2 refines RT-DETR by introducing selective multi-scale feature extraction, a discrete sampling operator for broader deployment compatibility. These improvements yield a 0.3 to 1.4 increase in mAP metrics on the COCO dataset, all while maintaining the same parameter count and frames-per-second (FPS) performance.

Adding RTDETRv2 by @jadechoghari in #34773

Transformers-CLI

Transformers' CLI welcomes a new command: chat. This command starts a conversation with the model of your choosing directly in your terminal.

This feature exists in TRL and has been migrated to transformers for easier usage.

[Chat] Add Chat from TRL 🐈 by @gante in #35714

Processor Standardization

An ongoing work is to standardize the image processors so that their API is equivalent. Additionally, the processors are given a fast variant so that they are never blockers in the image processing pipelines.

In this release, several processors have been standardized and have seen their fast version be contributed.

OwlViT/Owlv2 post processing standardization by @qubvel in #34929
OmDet Turbo processor standardization by @qubvel in #34937
Grounding DINO Processor standardization by @qubvel in #34853
Refactoring of ImageProcessorFast by @yonigozlan in #35069
add Qwen2-VL image processor fast by @yonigozlan in #35733
Remove Multi-threaded image conversion for fast image processors by @yonigozlan in #36105

Breaking changes

DPT segmentation maps

DPT image processors did not support segmentation_maps, instead only requiring images. This has been fixed. This adds an argument to the preprocess method, therefore users using arguments as positional arguments with that method may see changed behavior. We recommend using keyword arguments for such methods so as to not be bothered by the addition of new features.

🔴 🔴 🔴 Added segmentation maps support for DPT image processor by @simonreise in #34345

Image classification pipeline and single vs multi-label

The problem_type in the config.json file was read incorrectly by the pipeline, which mapped single-label to multi-label losses, and vice-versa. This has been fixed.

🚨🚨🚨 image-classification pipeline single-label and multi-label prob type squashing fns (sigmoid vs softmax) are backwards by @rwightman in #35848

Fixing the LayerNorm beta/gamma renames

The description of the pull request is the easiest way to understand the problem, why it exists, and how it is solved; please read the description below:

🚨🚨🚨 An attempt to fix #29554. Include 'LayerNorm.' in gamma/beta rename scope, optimize string search. by @rwightman in #35615

VLM cleanup

The ignore_index property of the llava configuration has been removed as it was not serving a purpose.

🔴 VLM: compile compatibility by @zucchini-nlp in #35724

Quantization

Quantization has received several improvements and fixes, including the contribution of FP8 quantization and the HIGGS quantization interface.

Additionally, we're replacing the AutoGPTQ implementaiton with GPTQModel from ModelCloud (see repository here)).

GPTQModel originated as major refractor of AutoGPTQ but is now a full-stand-in replacement with cleaner api, up-to-date model support, faster inference, higher quality quants.

Enable gptqmodel by @ jiqing-feng in #35012
Split and clean up GGUF quantization tests by @Isotr0py in #35502
Display warning for unknown quants config instead of an error by @SunMarc in #35963
Adding FP8 Quantization to transformers by @MekkCyber in #36026
New HIGGS quantization interfaces, JIT kernel compilation support. by @BlackSamorez in #36148

Generate

[generate] revert change in Aria: the maximum cache length must match max_length by @gante in #36120
🧹 remove generate-related objects and methods scheduled for removal in v4.48 by @gante in #35677
[generate] can instantiate GenerationConfig(cache_implementation="static") by @gante in #35679
[generate] return Cache object even if passed in a legacy format by @gante in #35673
[generate] update docstring of SequenceBiasLogitsProcessor by @gante in #35699
Test: generate with torch.compile(model.forward) as a fast test by @gante in #34544
[generate] move max time tests by @gante in #35962
[generate] shape checks in tests compatible with fixed-length caches (+ some minor fixes) by @gante in #35993

Pipelines

Pipelines have received several bug fixes and improvements which are detailed below.

Stop mutating input dicts in audio classification pipeline by @Rocketknight1 in #35754
fix document qa bf16 pipeline by @jiqing-feng in #35456
fix low-precision audio classification pipeline by @jiqing-feng in #35435
[pipeline] missing import regarding assisted generation by @gante in #35752
Output dicts support in text generation pipeline by @jonasrohw in #35092
Fix Audio Classification Pipeline top_k Documentation Mismatch and Bug #35736 by @sambhavnoobcoder in #35771

Bugfixes and improvements

Fix flaky test_custom_4d_attention_mask by @ydshieh in #35606
Use inherit tempdir makers for tests + fix failing DS tests by @muellerzr in #35600
Added error when sequence length is bigger than max_position_embeddings by @Taha1506 in #32156
Let EarlyStoppingCallback not require load_best_model_at_end by @muellerzr in #35101
Fix flaky test_beam_search_low_memory by @ydshieh in #35611
Skip MobileNetV1ModelTest::test_batching_equivalence for now by @ydshieh in #35614
Update codeowners with individual model owners by @Rocketknight1 in #35595
Fix device in rope module when using dynamic updates by @Cyrilvallez in #35608
Fix whisper compile by @jiqing-feng in #35413
Removed some duplicated code by @Sai-Suraj-27 in #35637
[Phi] bias should be True by @ArthurZucker in #35650
Enable different torch dtype in sub models by @zucchini-nlp in #34873
[Compile] Only test compiling model forward pass by @ArthurZucker in #35658
[tests] make cuda-only tests device-agnostic by @faaany in #35607
[i18n-ar] Translated file : docs/source/ar/tasks/token_classification.md into Arabic by @AhmedAlmaghz in #35193
Fix zero_shot_image_classification documentation guide link in SigLIP by @aretrace in #35671
Fix : adding einops lib in the CI docker for some bitsandbytes tests by @MekkCyber in #35652
Update torchao.md: use auto-compilation by @martin0258 in #35490
Fix : HQQ config when hqq not available by @MekkCyber in #35655
Fix expected output for ggml test by @MekkCyber in #35686
Fix : add require_read_token for gemma2 gated model by @MekkCyber in #35687
Enhanced Installation Section in README.md by @egojoseph in #35094
Enhance DataCollatorForLanguageModeling with Configurable Token Replacement Probabilities by @mahdibaghbanzadeh in #35251
Clean-up composite configs by @zucchini-nlp in #34603
Add future import for Py < 3.10 by @Rocketknight1 in #35666
Enable gptqmodel by @jiqing-feng in #35012
Fix : Nemotron Processor in GGUF conversion by @MekkCyber in #35708
Fix typo in /docs/source/ja/model_doc/decision_transformer.md URL by @hiroaki222 in #35705
Replace deprecated batch_size with max_batch_size when using HybridCache by @mtreinik in #35498
Fix: Falcon tie_word_embeddings in GGUF by @MekkCyber in #35715
Fix condition when GA loss bug fix is not performed by @techkang in #35651
Fix the bug that Trainer cannot correctly call torch_jit_model_eval by @Wanguy in #35722
[generation] fix type hint by @gante in #35725
Add proper jinja2 error by @Rocketknight1 in #35533
Optimize ForCausalLMLoss by removing unnecessary contiguous() call to reduce memory overhead by @efsotr in #35646
Modular: support for importing functions from any file by @Cyrilvallez in #35692
Remove batch size argument warning when unjustified by @quintenroets in #35519
[cache] add a test to confirm we can use cache at train time by @gante in #35709
Remove pt_to_tf by @gante in #35672
Added resource class configuration option for check_circleci_user job by @Sai-Suraj-27 in #32866
Fix some tests by @Cyrilvallez in #35682
Unable to use MimiModel with DeepSpeed ZeRO-3 by @anferico in #34735
check is added for the report_to variable in TrainingArguments by @alpertunga-bile in #35403
Added liger_kernel compatibility with PeftModel by @ambroser53 in #35680
Restore is_torch_greater_or_equal_than for backward compatibility by @tlrmchlsmth in #35734
Revert "Unable to use MimiModel with DeepSpeed ZeRO-3" by @eustlb in #35755
ci: fix xpu skip condition for test_model_parallel_beam_search by @dvrogozh in #35742
Use AMD CI workflow defined in hf-workflows by @ivarflakstad in #35058
Fix CI for VLMs by @zucchini-nlp in #35690
Security fix for self-comment-ci.yml by @ydshieh in #35548
[ViTPose] Convert more checkpoints by @NielsRogge in #35638
fix register_buffer in MimiEuclideanCodebook by @anferico in #35759
remove code owners as it was generating too much noise BUT by @ArthurZucker in #35784
Skip Falcon 7B GGML Test by @MekkCyber in #35783
[fix] cannot import name 'Pop2PianoFeatureExtractor' from 'transformers' by @faaany in #35604
transformers.image_transforms.normalize wrong types by @CalOmnie in #35773
Patch moonshine by @eustlb in #35731
Don't import torch.distributed when it's not available by @booxter in #35777
Fix vits low-precision dtype by @jiqing-feng in #35418
Tool calling: support more types by @aymeric-roucher in #35776
Fixes, improvements to timm import behaviour by @rwightman in #35800
modular_model_converter bugfix on assignments by @nikosanto13 in #35642
Deterministic sorting in modular converter when adding new functions by @Cyrilvallez in #35795
Fix "test_chat_template_dict" in video LLMs by @zucchini-nlp in #35660
Update AMD Docker image by @ivarflakstad in #35804
Add LlavaImageProcessor by @NielsRogge in #33191
Byebye test_batching_equivalence's flakiness by @ydshieh in #35729
[Doc] Adding blog post to model doc for TimmWrapper by @ariG23498 in #35744
add a new flax example for Bert model inference by @louie-tsai in #34794
Support adamw_torch_8bit by @fzyzcjy in #34993
Auto-add timm tag to timm-wrapper models. by @pcuenca in #35794
Fix : BLOOM tie_word_embeddings in GGUF by @MekkCyber in #35812
Fixed typo in autoawq version number in an error message for IPEX backend requirements. by @InfroLab in #35815
Remove deprecated get_cached_models by @Wauplin in #35809
Optimized set_initialized_submodules. by @LagPixelLOL in #35493
[i18n-ar] Translated file: docs/source/ar/tasks/masked_language_modeling.md into Arabic by @AhmedAlmaghz in #35198
move fastspeech to audio models by @eustlb in #35788
Improve modular documentation by @Cyrilvallez in #35737
[Mimi] update test expected values for t4 runners by @eustlb in #35696
Remove old benchmark code by @gante in #35730
Remove pyav pin to allow python 3.11 to be used by @CalOmnie in #35823
Another security patch for self-comment-ci.yml by @ydshieh in #35816
Init cache on meta device by @zucchini-nlp in #35164
Hotfix: missing working-directory in self-comment-ci.yml by @ydshieh in #35833
[gpt2] fix generation tests by @gante in #35822
Fix : Nemotron tokenizer for GGUF format by @MekkCyber in #35836
Fix head_dim in config extracted from Gemma2 GGUF model by @Isotr0py in #35818
[chat] docs fix by @gante in #35840
Fix compatibility issues when using auto_gptq with these older versions by @LRL-ModelCloud in #35830
Add PyTorch version check for FA backend on AMD GPUs by @mht-sharma in #35813
Fix NoneType type as it requires py>=3.10 by @SunMarc in #35843
[ tests] remove some flash attention class tests by @ArthurZucker in #35817
[Backend support] Allow num_logits_to_keep as Tensor + add flag by @Cyrilvallez in #35757
Fix GA loss for Deepspeed by @timjeffrey10 in #35808
Fix uploading processors/tokenizers to WandB on train end by @jack89roberts in #35701
Fix more CI tests by @ArthurZucker in #35661
[DOC] Fix contamination and missing paragraph in translation by @Yosshi999 in #35851
Fix typo by @SilverSoldier in #35854
fix apply_chat_template() padding choice by @baoyf4244 in #35828
Fix test_pipelines_video_classification that was always failing by @CalOmnie in #35842
Fix Llava-NeXT / Llava-NeXT Video / Llava-OneVision's token unpadding mismatch by @sheryc in #35779
use torch.testing.assertclose instead to get more details about error in cis by @ArthurZucker in #35659
add xpu device check in device_placement by @faaany in #35865
Add Rocketknight1 to self-comment-ci.yml by @ydshieh in #35881
[doctest] Fixes by @stevhliu in #35863
Fix fast image processor warnings in object detection examples by @sugendran in #35892
Update deepspeed amd image by @ivarflakstad in #35906
Fix typing in audio_utils.chroma_filter_bank by @CalOmnie in #35888
[docs] uv install by @stevhliu in #35821
Fix the config class comparison for remote code models by @Rocketknight1 in #35592
Close Zamba2Config code block by @Rocketknight1 in #35914
[docs] Fix Zamba2 by @stevhliu in #35916
Remove _supports_static_cache = True for some model classes by @ydshieh in #34975
Use rocm6.2 for AMD images by @ivarflakstad in #35930
Add default TP plan for all models with backend support by @Cyrilvallez in #35870
Fix: loading DBRX back from saved path by @zucchini-nlp in #35728
Fix mask slicing for models with HybridCache by @Cyrilvallez in #35681
Qwen-2-5-VL: fix CI by @zucchini-nlp in #35935
Fix TP initialization by @Cyrilvallez in #35860
fix(FA): QKV not being casted to target_dtype for FA with dpo lora by @NanoCode012 in #35834
Remove INC notebook reference in documentation by @echarlaix in #35936
use torch constraints to check if covariance is positive definite during mean resizing. by @abuelnasr0 in #35693
fix test_generated_length_assisted_generation by @keyboardAnt in #34935
Update unwrap_and_save_reload_schedule to use weights_only=False by @ydshieh in #35952
Update squad_convert_example_to_features to work with numpy v2 by @ydshieh in #35955
Fix flaky test_assisted_decoding_matches_greedy_search by @ydshieh in #35951
Trainer Refactor: Part 1 by @muellerzr in #35567
update docker file transformers-pytorch-deepspeed-latest-gpu by @ydshieh in #35940
[tests] further fix Tester object has no attribute '_testMethodName' by @faaany in #35781
Update README.md by @BlessedTatonka in #35958
fix iterator overflow when gradient accumulation is 1 by @winglian in #35960
Fix is_causal being a tensor by @IlyasMoutawwakil in #35791
[bart] minor test fixes by @gante in #35965
Pixtral: vectorize patch embeddings and enable tests by @zucchini-nlp in #35122
Whisper: fix static cache CI by @zucchini-nlp in #35852
Less flaky for TimmBackboneModelTest::test_batching_equivalence by @ydshieh in #35971
Support batching for UsefulSensors Moonshine by @njeffrie in #35922
not to use A100 for benchmark.yml by @ydshieh in #35974
Handle empty change indices in SAM's mask to rle conversion by @MSt-10 in #35665
Add support for nested images to LLava and VipLLava by @yonigozlan in #35558
[Moonshine] compute head_dim_padding at init by @eustlb in #35984
[Moshi] disable automatic compilation if the model can't compile by @gante in #35992
use torch 2.6 for daily CI by @ydshieh in #35985
Update-tp test by @ArthurZucker in #35844
Add mean_resizing for every VLMs' resizing_token_embeddings() by @YenFuLin in #35717
Update Granite Vision Model Path / Tests by @alex-jw-brooks in #35998
Qwen2-VL: fix rope delta calculation by @zucchini-nlp in #36013
Fix custom kernel for DeformableDetr, RT-Detr, GroindingDINO, OmDet-Turbo in Pytorch 2.6.0 by @qubvel in #35979
apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True by @mrsndmn in #35582
layernorm_decay_fix by @Ryoo72 in #35927
Update Mistral converter by @Cyrilvallez in #35967
Refactor (and fix) gpt_neox by @Cyrilvallez in #35610
Fix device mismatch error in Whisper model during feature extraction by @thedebugger in #35866
Fix RMSNormGated in Zamba2 by @pglorio in #35943
Commont bot CI for other jobs (generation / quantization) by @ydshieh in #35341
Hotfix for self-comment-ci.yml by @ydshieh in #36030
feat(ci): ignore trufflehog unverified results by @McPatate in #36031
CircleCI with python 3.9 by @ydshieh in #36027
Update tests regarding attention types after #35235 by @ydshieh in #36024
Fix Gemma2 synced multi-GPU generation by @ManukyanD in #35232
Fix synced multi-GPU generation with LLMs and VLMs by @ManukyanD in #35893
Add XPU type for work-around -inf mask causing sdpa NaN issue in modeling files by @Liangliang-Ma in #35647
add support for empty list as input to create_model_card by @ROZBEH in #36042
DeepSpeed github repo move sync by @stas00 in #36021
[docs] no hard coding cuda as bnb has multi-backend support by @faaany in #35867
[docs] fix bugs in the bitsandbytes documentation by @faaany in #35868
[docs] no hard-coding cuda by @faaany in #36043
Fix how we compute the final non-padding token for ForSequenceClassification models by @Rocketknight1 in #35911
Add Qwen2VLImageProcessorFast into Qwen2VLProcessor by @yeliudev in #35987
Iterative generation using Input embeds and past_key_values by @yaswanth19 in #35890
Fix usage of unpad_input function by @pavelgein in #35925
Fix repo consistency by @ydshieh in #36063
Update test_flash_attn_2_can_dispatch_composite_models by @ydshieh in #36050
Paligemma: fix generation with Gemma2 by @zucchini-nlp in #36044
Save checkpoint to temporary directory to handle partial saves during failures by @SilverSoldier in #35580
Nail in edge case of torch dtype being overriden permantly in the case of an error by @muellerzr in #35845
Fix words typos in ggml test. by @zhanluxianshen in #36060
Fix model kwargs by @muellerzr in #35875
Fix StopStringCriteria to handle tokens above len(tokenizer) by @Rocketknight1 in #35797
[docs] fix outdated example code in trainer.md by @faaany in #36066
Adding RT-DETRv2 for object detection by @jadechoghari in #34773
Fix bug in apply_rotary_pos_emb_flashatt: in Qwen2-5-VL by @DeepWaved in #36065
Move audio top_k tests to the right file and add slow decorator by @Rocketknight1 in #36072
Fix OS err by @muellerzr in #36094
[docs] fix model checkpoint name by @faaany in #36075
[docs] fix typo by @faaany in #36080
[docs] fix not-working example code in perf_infer_gpu_one.md by @faaany in #36087
fix MllamaVisionAttention typehint by @kylesayrs in #35975
Processors: allow tuples of images when checking by @zucchini-nlp in #36084
Chat template: update for processor by @zucchini-nlp in #35953
Paligemma: revert #36084 by @zucchini-nlp in #36113
Support constant lr with cooldown by @LoserCheems in #35453
Enable pytest live log and show warning logs on GitHub Actions CI runs by @ydshieh in #35912
Refactor OPT model by @jiqing-feng in #36101
Revert checkpoint tmp dir by @SunMarc in #36112
[Bugfix] fix file name of docstring in utils/check_table.py by @kkscilife in #36108
fix bnb warning by @SunMarc in #36116
AutoformerForPrediction test add atol by @ivarflakstad in #36017
Fix nighlty CIs: missing atols by @ArthurZucker in #35903
Add common test for torch.export and fix some vision models by @qubvel in #35124
fix: typos in documentation files by @maximevtush in #36122
update awesome-transformers.md. by @zhanluxianshen in #36115
Fix max size deprecated warning by @HichTala in #34998
Fix CI issues by @molbap in #35662
update tiktoken integ to use converted by @ArthurZucker in #36135
Make output_dir Optional in TrainingArguments #27866 by @sambhavnoobcoder in #35735
[docs] minor doc fix by @faaany in #36127
[docs] update awq doc by @faaany in #36079
Add pipeline parallel plan to PretrainedConfig and PreTrainedModel by @hmellor in #36091
add RAdamScheduleFree optimizer by @nhamanasu in #35313
added warning to Trainer when label_names is not specified for PeftModel by @MilkClouds in #32085
Whisper: remove redundant assisted generation tests by @gante in #34814
Add utility for Reload Transformers imports cache for development workflow #35508 by @sambhavnoobcoder in #35858
VLM: enable skipped tests by @zucchini-nlp in #35746
[commands] remove deprecated/inoperational commands by @gante in #35718
Fix Gradient Checkpointing for Deberta & Deberta-V2 using PEFT / Adapters by @lenglaender in #35898
🚨 Remove cache migration script by @Wauplin in #35810
multi-gpu: fix tensor device placements for various models by @dvrogozh in #35763
Optim: APOLLO optimizer integration by @zhuhanqing in #36062
Fix multi gpu loss sync condition, add doc and test by @techkang in #35743
adding option to save/reload scaler by @hsilva664 in #34932
Update doc re list of models supporting TP by @kwen2501 in #35864
Add more rigerous non-slow grad accum tests by @muellerzr in #35668
Fix test fetcher by @ydshieh in #36129
skip test_initialization for VitPoseBackboneModelTest for now by @ydshieh in #36154
Add git LFS to AMD docker image by @ivarflakstad in #36016
Mllama fsdp by @blbadger in #36000
Fix PaliGemma Pad Token Masking During Training #35855 by @sambhavnoobcoder in #35859
Add reminder config to issue template and print DS version in env by @Ben-Schneider-code in #35156
Fix Gemma2 dtype issue when storing weights in float16 precision by @Nerogar in #35398
Replace deprecated update_repo_visibility by @Wauplin in #35970
Fix tests for vision models by @qubvel in #35654
qwen2.5vl: fix bugs when using flash2+bf16 or num_return_sequences>1 by @gewenbin0992 in #36083
docs: fix return type annotation of get_default_model_revision by @MarcoGorelli in #35982
Fix PretrainedTokenizerFast check => Fix PretrainedTokenizerFast Save by @CL-ModelCloud in #35835
Move DataCollatorForMultipleChoice from the docs to the package by @bauwenst in #34763
Helium documentation fixes by @LysandreJik in #36170
Remove loading custom kernel for RT-DETRv2 by @qubvel in #36098
[Modular] skip modular checks based on diff by @gante in #36130
Fix red CI by @ArthurZucker in #36174
Fix : fix doc fp8 by @MekkCyber in #36173
Efficient Inference Kernel for SpQR by @elvircrn in #34976
fix training issues by @ArthurZucker in #36158
add disable compile option by @ArthurZucker in #36161
CI: avoid human error, automatically infer generative models by @gante in #33212
Use tqdm auto by @SmartManoj in #35726
Optimize Qwen2VL vision model by precomputing cos/sin embeds before ViT blocks by @li-plus in #35837
Make check_repository_consistency run faster by MP by @ydshieh in #36175
Fix the key name for _load_rng_state under torch.cuda by @wizyoung in #36138
Follow up to SpQR integration by @MekkCyber in #36176
Fix a mistake in #36175 by @ydshieh in #36179
Fix make_batched_videos and add tests by @yonigozlan in #36143
Uniformize OwlViT and Owlv2 processors by @yonigozlan in #35700
Add support for partial rotary embeddings in Phi3 model by @garg-amit in #35947
CI: fix test-save-trainer by @zucchini-nlp in #36191
Chat template docs by @zucchini-nlp in #36163
Add ImageProcessorFast to Qwen2.5-VL processor by @Isotr0py in #36164
Prepare processors for VideoLLMs by @zucchini-nlp in #36149
Add require_read_token to fp8 tests by @MekkCyber in #36189
Revert qwen2 breaking changes related to attention refactor by @ArthurZucker in #36162
Guard against unset resolved_archive_file by @dmlap in #35628
[Bugfix] Fix reloading of pixtral/llava configs by @kylesayrs in #36077

Significant community contributions

The following contributors have made significant changes to the library over the last release:

@jiqing-feng
- Fix whisper compile (#35413)
- Enable gptqmodel (#35012)
- fix document qa bf16 pipeline (#35456)
- Fix vits low-precision dtype (#35418)
- fix low-precision audio classification pipeline (#35435)
- Refactor OPT model (#36101)
@AhmedAlmaghz
- [i18n-ar] Translated file : docs/source/ar/tasks/token_classification.md into Arabic (#35193)
- [i18n-ar] Translated file: docs/source/ar/tasks/masked_language_modeling.md into Arabic (#35198)
@sbucaille
- Add SuperGlue model (#29886)
@Isotr0py
- Fix head_dim in config extracted from Gemma2 GGUF model (#35818)
- Split and clean up GGUF quantization tests (#35502)
- Add ImageProcessorFast to Qwen2.5-VL processor (#36164)
@ShuaiBai623
- add qwen2.5vl (#35569)
@alex-jw-brooks
- Granite Vision Support (#35579)
- Update Granite Vision Model Path / Tests (#35998)
@pglorio
- Add Zamba2 (#34517)
- Fix RMSNormGated in Zamba2 (#35943)
@conditionedstimulus
- Add DAB-DETR for object detection (#30803)
@jadechoghari
- Adding RT-DETRv2 for object detection (#34773)
@geetu040
- Add Apple's Depth-Pro for depth estimation (#34583)
@zhuhanqing
- Optim: APOLLO optimizer integration (#36062)
@bauwenst
- Move DataCollatorForMultipleChoice from the docs to the package (#34763)
@elvircrn
- Efficient Inference Kernel for SpQR (#34976)

Open source →

transformers

Release timeline

Releases

Transformers v5 release notes

Highlights

Significant API changes

Dynamic weight loading

Tokenization

Backend Architecture Changes: moving away from the slow/fast tokenizer separation

Defining a tokenizers outside of the existing backends

API Changes

1. Direct tokenizer initialization with vocab and merges

2. Simplified decoding API

3. Unified encoding API

4. apply_chat_template returns BatchEncoding

5. Removed legacy configuration file saving:

6. Model-Specific Changes

Testing Changes

7. Deprecated / Modified Features

Performance

MoE Performance

Core performance

Library-wide changes with lesser impact

Default dtype update

Shard size

use_auth_token

Attention-related features

Updates to supported torch APIs

Quantization changes

Configuration

Processing

Tokenization

Processing classes

Modeling

Generate

Trainer

New Features

Breaking Changes

Removing arguments without deprecation cycle in TrainingArguments due to low usage

Removing deprecated arguments in TrainingArguments

Removing deprecated arguments in Trainer

Removed features for Trainer

New defaults for Trainer

Pipeline

PushToHubMixin

CLI

Removal of the run method

Environment variables

Requirements update

New model additions in v5

CWM

SAM3

LFM2 MoE

VideoLlama 3

AudioFlamingo 3

Nanochat

FastVLM

PaddleOCR-VL

SAM: Perception Encoder Audiovisual

Jais2

Pixio

Ernie 4.5 VL MoE

GLM-ASR

GLM 4.7 Flash

GLM Image

LWDetr

LightOnOCR

Bugfixes and improvements

Release candidate v5.0.0rc3

New models:

What's Changed

New Contributors

What's Changed

MoEs and performances:

Tokenization:

Core

New models

Quantization

Breaking changes

New Contributors

4. `apply_chat_template` returns `BatchEncoding`

Default `dtype` update

`use_auth_token`

Removing arguments without deprecation cycle in `TrainingArguments` due to low usage

Removing deprecated arguments in `TrainingArguments`

Removing deprecated arguments in `Trainer`

Removed features for `Trainer`

New defaults for `Trainer`

Removal of the `run` method

4. `apply_chat_template` returns `BatchEncoding`

`use_auth_token`

Removing arguments without deprecation cycle in `TrainingArguments` due to low usage

Removing deprecated arguments in `TrainingArguments`

Removing deprecated arguments in `Trainer`

Removed features for `Trainer`

New defaults for `Trainer`

Removal of the `run` method