NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI
PyPI, #2808 most downloaded on PyPI
Train transformer language models with reinforcement learning.
Latest stable
1.18.0
Last release 9 days ago
13 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
7 years old
109 releases · first in 2020
Drag across the ruler to compare, or tap a release to open it.
💥 What this changes for you. If you used the experimental AsyncGRPOTrainer in v1.17, the breaking changes all landed before the promotion, so they don…
Warning
The submodules of trl.experimental.async_grpo are gone. from trl.experimental.async_grpo.async_grpo_config import AsyncGRPOConfig, and imports from async_rollout_worker, packing, vllm_client or weight_transfer in that package, no longer work. Import the trainer from trl, the rollout loop from trl.trainer.async_grpo_trainer, and the vLLM client from trl.generation.async_vllm_client (renamed AsyncVLLMClient). from trl.experimental.async_grpo import AsyncGRPOConfig, AsyncGRPOTrainer still works, with a FutureWarning, until v2.0.0.
AsyncGRPOTrainer is now a stable trainerAsyncGRPOTrainer runs GRPO with generation decoupled from training. A rollout worker in a background process streams completions from a vLLM server, scores them and queues them, while the training loop pulls from the queue and updates the model. Generation and gradient updates overlap instead of alternating, and the trainer sends its new weights to the server over NCCL every weight_sync_steps steps.
In v1.17 it lived in trl.experimental.async_grpo: seven files, three ways to batch and bound generation, two constructor arguments for swapping out internals, and metric names that didn't match GRPO's. This cycle cleaned that up in about 40 PRs, tracked in #7841: config renames, one timeout instead of three, one batching mode, generation held to max_staleness, GRPO's metric names and init checks, a public rollout loop, and the two-file layout every stable trainer has. Then #8288 moved it to the top-level package.
# Before
from trl.experimental.async_grpo import AsyncGRPOConfig, AsyncGRPOTrainer
# Now
from trl import AsyncGRPOConfig, AsyncGRPOTrainerThe old path still works and emits a FutureWarning (removal in v2.0.0). The rollout loop extension point lives next to the trainer, from trl.trainer.async_grpo_trainer import AsyncRolloutLoop, Rollout, and isn't exported from trl. The OpenEnv / Harbor harness stays experimental, as HarnessAsyncGRPOTrainer in trl.experimental.async_grpo.openenv_harness. Logged metric names can still change between releases.
Quick start, from the docs. Install with pip install trl[vllm,kernels,math_verify], then start the vLLM server on GPU 0:
CUDA_VISIBLE_DEVICES=0 VLLM_SERVER_DEV_MODE=1 vllm serve Qwen/Qwen3.5-2B \
--weight-transfer-config '{"backend": "nccl"}' \
--logprobs-mode processed_logprobs \
--api-server-count 8 \
--max-model-len 4096Then train on GPU 1 with CUDA_VISIBLE_DEVICES=1 python train_async_grpo.py:
# train_async_grpo.py
from datasets import load_dataset
from trl import AsyncGRPOTrainer
from trl.rewards import accuracy_reward
if __name__ == "__main__": # the rollout worker is a spawned process that re-imports this script
dataset = load_dataset("trl-lib/DeepMath-103K", split="train")
trainer = AsyncGRPOTrainer(
model="Qwen/Qwen3.5-2B",
reward_funcs=accuracy_reward,
train_dataset=dataset,
)
trainer.train()This exact example runs end to end. It's a sanity check, not a speedup comparison:
| Model / data / reward | Qwen/Qwen3.5-2B, trl-lib/DeepMath-103K, accuracy_reward |
| Hardware | 2x B200: vLLM on one GPU, the trainer on the other |
| Config | default AsyncGRPOConfig: fp32 weights with bf16 autocast, 8 generations per prompt, up to 2048 completion tokens, token_budget read from --max-model-len 4096 |
| Step time | ~7 s/step, after ~5 min of warmup |
| Throughput | ~7k trained tokens/s |
| Trainer vs vLLM log-prob gap | 0.007 to 0.02 |
What the trainer covers, all documented on the trainer page:
tools, environment_factory and multi-turn rollouts.score_centering=True). score_centering=False gives GRPO's clipped objective with the same masked ratio.frac_zero_std_groups.top_p or min_p, set top_k > 0 and the trainer normalizes its log-probs over the candidate set vLLM sampled from.--enable-lora and each weight sync sends only the adapter (about 160 MB at r=32) instead of merged weights.AsyncRolloutLoop, override filter_prompt, generate_one, generate_one_turn, compute_advantages, setup or close, and set it as loop_cls on a trainer subclass. The trainer's constructor doesn't change.💥 What this changes for you. If you used the experimental AsyncGRPOTrainer in v1.17, the breaking changes all landed before the promotion, so they don't have to happen after it. AsyncDistillationTrainer (still experimental) got the same constructor removals, config renames, timeout merge, batching and staleness changes and metric conventions in the same PRs.
Constructor arguments:
weight_transfer= is removed. Weight sync is built from the config: NCCL through vLLM's weight-transfer API, or adapter sync for LoRA. The server is a plain vllm serve.rollout_worker= is removed. To change how rollouts are produced, subclass AsyncRolloutLoop and set loop_cls (see above).HarnessAsyncGRPOTrainer replaces building a HarnessRolloutWorker yourself and passing it as rollout_worker=.Config:
| v1.17 (experimental) | v1.18 |
|---|---|
max_inflight_tasks (-1 = automatic) |
max_concurrent_rollouts, a fixed ceiling, 512 by default |
heartbeat_stale_after_s |
rollout_worker_timeout (same meaning, same 300 s default) |
request_timeout, weight_sync_timeout |
removed: vllm_server_timeout (default 1800 s, was 240 s) now covers server readiness, every request and every weight-sync call |
queue_maxsize |
removed: generation is held to max_staleness, counted in tokens |
fork_threshold_tokens |
removed: a multi-turn rollout always forks on re-tokenization drift, so no generated token loses its training signal |
token_budget <= 0 (fixed-count batching) |
removed: the token-budget batcher is the only mode. Use None (read max_model_len from the server) or a positive int |
repetition_penalty |
removed: the sampler's log-probs include it and the trainer's don't, which biases the importance ratio |
remove_unused_columns=True (ignored) |
defaults to False, and True raises: every dataset column goes to the reward functions |
Behavior to check after upgrading:
vllm_server_timeout (1800 s).max_concurrent_rollouts no longer derives from per_device_train_batch_size. With OpenEnv / Harbor, the environment server's MAX_CONCURRENT_ENVS must be at least max_concurrent_rollouts + 1: raise it, or lower max_concurrent_rollouts. The Harbor example's --max-inflight flag is now --max-concurrent-rollouts.rollout/backpressure_s used to measure time waiting on a full queue. It now measures time waiting on the staleness limit.Metrics: where AsyncGRPO and GRPO log the same quantity, AsyncGRPO now mostly uses the GRPO name. The async-only metrics keep their namespaces (rollout/, sample/, batch/, perf/, tools/, env/), with one meaning per suffix: _count is a count over the logging window, _s a duration, _per_s a rate, _frac a fraction. Dashboards and scripts that read the old names need the new ones.
| v1.17 (experimental) | v1.18 |
|---|---|
rewards/<func> |
rewards/<func>/mean, plus rewards/<func>/group_std |
reward_std |
reward_group_std (within-group, unlike GRPO's batch-wide reward_std) |
ratio |
sampling/importance_sampling_ratio/{mean,min,max} |
kl (k3 estimate against the sampler) |
removed. sampling/sampling_logp_difference/{mean,max} is GRPO's measure of the same gap, and kl stays free for a reference-model KL |
frac_reward_zero_std |
frac_zero_std_groups (GRPO and RLOO keep frac_reward_zero_std) |
reward_kept |
reward_trained |
rollout/inflight, rollout/inflight_cap |
rollout/concurrency, rollout/concurrency_cap |
rollout/kv_cache_usage |
rollout/vllm_kv_cache_usage_frac |
sample/rollout_queue_size |
rollout/queue_size |
rollout/generated_tok_s |
rollout/generated_tokens_per_s |
perf/forwarded_tok_s_fwd_bwd, perf/forwarded_tok_s_wall_clock, perf/trained_tok_s_wall_clock |
perf/fwd_bwd_forwarded_tokens_per_s, perf/wall_clock_forwarded_tokens_per_s, perf/wall_clock_trained_tokens_per_s |
perf/mfu_fwd_bwd, perf/mfu_wall_clock |
perf/fwd_bwd_mfu, perf/wall_clock_mfu |
sample/dropped_stale_total, batch/dropped_oversize_total, rollout/vllm_retry_total, tools/<name>/call_total, tools/<name>/failure_total, tools/unknown_name_total |
the same names ending in _count |
rollout/vllm_preemptions_total |
rollout/vllm_preemption_count |
rollout/realign_frac |
removed, with fork_threshold_tokens |
New: num_tokens, completions/{mean,min,max}_length, completions/{mean,min,max}_terminated_length, completions/clipped_ratio, reward_zero_std_groups (the mean reward of the dropped groups) and completions/duplicate_frac. Fixed: tools/call_frequency and tools/failure_frequency now count every rollout, as in GRPO, and rewards/<func>/mean counts once per scored rollout.
The PRs, by @AmineDiro, @qgallouedec and @lewtun unless noted:
weight_transfer protocol (#8065) and AsyncDistillation's rollout_worker argument (#8066), split the rollout loop into generate, reward and advantage stages (#8073), make it public as loop_cls (#8096), drop rollout_worker (#8107), and replace HarnessRolloutWorker with HarnessAsyncGRPOTrainer (#8105), all by @AmineDiro.max_inflight_tasks and heartbeat_stale_after_s and merge the vLLM timeouts (#8063), always fork on re-tokenization drift (#8159), keep only the token-budget batcher (#8168), hold generation to max_staleness in tokens (#8170, #8243), all by @AmineDiro. Drop repetition_penalty (#8139) and default remove_unused_columns to False (#8281) by @qgallouedec._count and _per_s suffixes (#8186) and namespaces (#8188) by @AmineDiro. Terminated-length metrics (#8247) and per-rollout reward means (#8250) by @qgallouedec. Saturation and duplicate-rollout metrics (#7901) and the zero-std and trained reward renames (#8309) by @lewtun.accelerator_config code (#8191), inline the rollout worker and packing into the trainer and move the vLLM client and weight transfer to trl.generation (#8274), promote to the stable API (#8288) and announce it in the README (#8292), by @AmineDiro. Rewrite the docs on the layout of the stable trainer pages (#8131), tag the model with trl and async-grpo (#8209), and run the train test against a transformers-backed vLLM server (#8137), by @qgallouedec.GRPO server mode now renormalizes over vLLM's candidate set. With top_p, min_p or structured outputs, vLLM's log-probs are normalized over the tokens left after filtering. Colocate mode already handled this. In server mode the trainer normalized over the whole vocabulary, so GRPO warned that the importance sampling ratio was biased, or refused structured outputs. A vLLM server (>= 0.28) started with --return-sampling-mask now returns the candidate set of each sampled token, VLLMClient.generate(return_sampling_mask=True) reads it, and GRPO turns it on when the server reports the flag. With Qwen2.5-0.5B-Instruct, top_k=10 and a vLLM 0.32 server, the ratio stays between 0.85 and 1.27 for structured outputs and top_p=0.9, where it was 1e-13 to 1e-31 for structured outputs without the candidate set. trl vllm-serve doesn't add the flag by default, since the server then rejects every request with top_k=0 or temperature=0. By @albertvillanova in #8235
UDPO loss for noisy preferences. DPOTrainer gets loss_type="udpo", the normalized Unbiased DPO loss from Unbiased Alignment for Large Language Models with Noisy Preferences. It corrects for preference labels flipped with probability ε, read from label_smoothing in [0.0, 0.5), and unlike "robust" it stays unbiased when ε is overestimated.
from trl import DPOConfig
training_args = DPOConfig(loss_type="udpo", label_smoothing=0.4)Privileged-context distillation in GOLD. The teacher sees per-example extra context while the student keeps its original prompt, and both score the same completion tokens. Turn it on with use_privileged_context=True, and set privileged_context_column and teacher_prompt_template as needed. It supports text, off-policy (lmbda=0) standard JSD distillation only. See the GOLD docs. By @eshwanthkartitr in #6437
MXFP4 and FP8 checkpoints load dequantized. With kernels installed, any trainer given openai/gpt-oss-20b by name refused to train: transformers rejects full fine-tuning of a quantized model, and PEFT finds nothing to adapt in the packed MXFP4 experts. Trainers now load MXFP4 and FP8 checkpoints dequantized. A quantization_config in model_init_kwargs still wins. FP8 needs transformers>=5.0.0, DeepSeek-V3/R1 still fail to dequantize upstream (huggingface/transformers#49873), and before transformers 5.23 MXFP4 experts come back bfloat16 whatever the dtype. By @qgallouedec in #8070
import trl no longer scans every installed package. It built a package-to-distribution map with importlib.metadata.packages_distributions(), which stats every file of every installed distribution without a top_level.txt (torch 2.14 ships none). The map is gone. Measured on a network filesystem, 8 alternating runs:
| before | after | |
|---|---|---|
import trl |
13.1 s | 0.08 s |
from trl import SFTTrainer |
40.7 s | 27.4 s |
The second row gains less because transformers still makes its own call (huggingface/transformers#49591). By @qgallouedec in #7947
More chat templates for tool calling and assistant-only loss. add_response_schema (so GRPOTrainer with tools) and assistant_only_loss=True in SFT now work on many more checkpoints. All by @albertvillanova:
get_training_chat_template gains require_prefix_preservation, and SFT uses TRL's stored training template even when the chat template has {% generation %} markers of its own (#8184)trl <cmd> launches with torchrun. Arguments the script doesn't know go to torchrun (--nproc_per_node, --nnodes, ...), and DeepSpeed and FSDP are set through the training arguments (--deepspeed zero3.json, --fsdp), like for any Trainer. accelerate launch keeps working: until v2.0.0, trl still uses it, with a FutureWarning, for arguments only accelerate knows (--num_processes, --accelerate_config) and when an accelerate config default file exists. examples/accelerate_configs/ is replaced by examples/deepspeed_configs/*.json and examples/fsdp_configs/fsdp2.json. See the CLI docs. By @qgallouedec in #7180
padding_free=True works with max_length in SFT. SFT raised whenever padding_free=True met a max_length (including the default 1024) without packing. Truncation now happens during dataset preparation for padding-free too, so the guard only blocked a valid config. By @qgallouedec in #8077
trl/other, by @qgallouedec in #7926activation_checkpointing_* methods on transformers 5.23, so GRPO, RLOO, DPO, KTO and Distillation stop emitting a FutureWarning, by @albertvillanova in #8194enable_input_require_grads() before transformers 5.0, which handles it, by @qgallouedec in #8252 and @albertvillanova in #8266flash-attn4 version 1 on Blackwell, which trains Qwen3.8-27B on B200 / B300, by @younesbelkada in #7987vLLM support follows the 5-month window: the minimum goes from 0.22.0 to 0.23.0, and 0.33.x is now supported, so the range is vllm>=0.23.0,<0.34.0. vLLM 0.33.0 requires torch 2.14.0, so pip install -U "trl[vllm]" pulls it.
Several of these change what a run trains on. The first one needs action on models you already saved.
MoE models saved by SFTTrainer v1.7.0 to v1.17.1 compute router logits at every forward, inference included. SFT wrote output_router_logits=True and router_aux_loss_coef into the model config and saved them, so fine-tuned MoE models kept returning their router logits, and an overridden coef carried over to the next fine-tune. Like GRPO, RLOO, DPO and KTO, SFT now leaves the config alone. It keeps the config the model was loaded with, though, so a model saved by one of these versions keeps the flag even after another fine-tune. To reset it, set it back to False and save again:
model = AutoModelForCausalLM.from_pretrained("path/to/model")
model.config.output_router_logits = False # model.config.text_config.output_router_logits for a VLM
model.save_pretrained("path/to/model")By @qgallouedec in #8103
CISPO gradients were 5x too small by default. CISPO clips the importance weight itself, but an unset epsilon_high fell back to epsilon=0.2, so every on-policy weight was clipped to 0.2. With loss_type="cispo", an unset epsilon_high now defaults to 5.0, the value the docstring already recommended. On the repro, grad_norm goes from 0.0539 to 0.2693 and cispo_clip_ratio from 0.375 to 0. By @qgallouedec in #7917
Assistant-only loss trained tokens the generation prompt already ends with. Several training templates put the turn opener or the pre-filled thinking block inside {% generation %}, so SFT trained tokens the model never generates. The rendered text doesn't change, only the mask: Qwen3 (#8072), GLM-4-MoE (#8122), DeepSeek-R1/V3 and R1-Distill (#8060), Nemotron 3 (#8217), gpt-oss (#8219), Muse Glimmer (#8221), and the Qwen3.6 and Qwen3.8 revisions shipped since 2026-11-07, which bring their own markers (#8199, #8187). The DeepSeek-R1/V3 training template also dropped the reasoning of assistant turns from the target (#8061). All by @albertvillanova.
AsyncGRPO and AsyncDistillation reused prefix-cache KV computed by older weights. Each weight sync pauses vLLM without clearing its prefix cache, so a prefix shared by many prompts (system prompt, few-shot examples, tool schemas) kept the KV of the weights that first served it for the whole run. Rollout requests are now salted by model version. On a Qwen3.5-35B-A3B LoRA run with a ~2.4k-token shared prefix, mean train reward over steps 11-20 went from 0.726 to 0.781. By @AmineDiro in #8300
GKD divided its loss by the length of answers it threw away. On-policy and seq_kd steps counted num_items_in_batch on the dataset labels, then replaced them with generated completions. On the repro, num_items_in_batch goes from 172 to 16, the number of tokens the loss sums over. By @qgallouedec in #7925
One rejected token vetoed a whole VESPO sequence. With loss_type="vespo" and vllm_importance_sampling_mode="token_mask", a token outside the ratio bounds got a ratio of 0, whose clamped log collapsed the sequence weight to ~0, while the token itself kept its gradient. Rejected tokens are now dropped from both the sequence weight and the loss. By @yusufozdil in #8108
GRPO configurations that trained nothing without an error now raise:
repetition_penalty with the vLLM importance sampling correction: vLLM applies the penalty to the log-probs it returns, so the ratio dropped to ~1e-28 and the gradient to 0. By @qgallouedec in #7915top_k > 0: the ratio fell to about 1e-17. By @albertvillanova in #8145Crashes and hangs:
Rollout worker child has failed. By @sergiopaniego in #8038model_init_kwargs={"distributed_config": DistributedConfig(...)}) failed at init on Qwen3.5/3.6 hybrids, and its merged LoRA weight sync crashed on DTensor base weights. By @AmineDiro in #8294 and #8295AttributeError, by @qgallouedec in #8290num_experts (Granite MoE, granite-4.0-h, PhiMoE), by @qgallouedec in #7919beta != 0) and RLOO with a pretrained adapter that has modules_to_save failed at the first vLLM weight sync, by @qgallouedec in #8079DPOTrainer with use_weighting=True and the "sft" loss raised a RuntimeError at the first step, by @Sergasgr in #7651<|vision_start|> and crash the forward, by @albertvillanova in #8166content key, which transformers 5.23 produces, by @albertvillanova in #8228The rest:
<|observation|> stop token as content, and the legacy schema (transformers < 5.13) returned tool calls as plain text, by @albertvillanova in #8030 and Note truncated.
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Keep a parenthesis inside an LFM2.5 tool-call argument from closing the call by @albertvillanova in #7942
Full Changelog: v1.17.0...v1.17.1
🔥 Mistral Large 4: fine-tune a 1T-parameter model on one node
TRL now supports mistralai/Mistral-Large-4-1T-A52B: 1T parameters, 52B active. The new LoRA example fine-tunes it with SFTTrainer on a single 8×B300 node, on top of its FP8 weights: the experts are split 8 ways with token dispatch, everything else is sharded across the 8 GPUs at load time by transformers' DistributedConfig, and LoRA trains attention and the dense and shared-expert MLPs.
Measured: 32.7k tokens per step in ~8 s, 142 GiB per GPU, with packing and assistant-only loss on reasoning conversations.
| mistral-large4-trl_v7_hq.mp4 |
training_args = SFTConfig(
model_init_kwargs={
"dtype": "auto", # keep the FP8 weights
"distributed_config": DistributedConfig(fsdp_size=8, ep_size=8),
},
packing=True,
assistant_only_loss=True,
...
)
trainer = SFTTrainer(model="mistralai/Mistral-Large-4-1T-A52B", args=training_args, peft_config=LoraConfig(...), ...)assistant_only_loss=True works out of the box: TRL ships a training chat template that matches the released checkpoint's, including its reasoning_effort levels (low, medium, high). Needs transformers>=5.22.0, peft>=0.21.0 and accelerate>=1.15.0.
vLLM normalizes its log-probabilities over the tokens left after top_k, top_p and min_p filtering, while the trainer normalized over the whole vocabulary. Every sampled token's log-ratio was off by the log of the probability mass the filter kept, so the vLLM importance sampling ratio was biased towards 0 (around 1e-4 instead of 1 with top_p=0.9). The trainers now normalize over the candidate set each token was sampled from, following Keep Sampling Mask from DeepSeek-V3.2.
top_k with min_p in AsyncGRPO so the importance sampling ratio stays unbiased by @albertvillanova in #7937💥 What this changes for you. vLLM bounds the candidate set it returns with top_k, so with vLLM and the importance sampling correction on (the default), GRPOConfig now raises when top_p < 1.0 or min_p > 0.0 is set without top_k > 0. Set top_k (e.g. top_k=512), or vllm_importance_sampling_correction=False.
AsyncGRPO's old log-probs are the vLLM sampler's, so its ratio mixes staleness with the numerical mismatch between the server and the trainer. With an FP8 sampler and a bf16 trainer, the clipped objective collapses: reward goes from 0.60 to 0.34 (FP8 weights) or 0.10 (FP8 weights + KV cache) by step 300, against 0.71 with a bf16 sampler.
Score centering with masked importance sampling is now the default (score_centering=True, importance_sampling_min=0.5, importance_sampling_max=5.0). With it, the FP8 sampler trains exactly like the bf16 one (0.72). It costs 15–20% of generation throughput.
💥 What this changes for you. AsyncGRPO runs now train with a different objective by default, and tokens whose ratio to the sampler falls outside [0.5, 5.0] are dropped. Set score_centering=False to keep the clipped objective.
by @AmineDiro in #7760
max_inflight_tasks, so requests stop queueing and being preempted on the server. No new config: everything is read from the server. By @AmineDiro in #7610TreeRow.pack goes from 250 s to 31 s on rank 0 over a 15-step run, and from 1040 ms to 66 ms on a 28k-token multi-turn row. By @AmineDiro in #7798Tree packing (v1.16) now supports the Gated DeltaNet hybrids: Qwen3.5, Qwen3.6, Qwen3-Next and OLMo-Hybrid. Since v1.16 the trainer refused them at init; before it, sequence packing accepted them but was silently wrong on them. The causal convolution and the delta-rule recurrence now follow the tree instead of the row, through FLA's chunked kernels.
by @AmineDiro in #7686
The chunked log-probability and KL/JSD heads get the two precision fixes from A Fix for Softmax and Cross-Entropy: confident targets keep their small gradients, and matching teachers produce zero gradients. Entropy-preserving reinforcement learning (Appendix B.4) reports the same high-confidence policy-gradient issue.
GOLDTrainer gets off-policy cross-tokenizer distillation based on X-Token: set lmbda=0.0 and xtoken_loss_type="p_kl" or "h_kl" with a precomputed projection matrix.
TRL's trainers now run on AMD Instinct GPUs through PyTorch's ROCm build, with a guide and an AMD CI job that runs the test suite on every PR.
transformers>=5.20.1. By @qgallouedec in #7629AsyncGRPOTrainer by @kashif in #6676, and port instantiated models, MoE aux loss and the injected-worker LoRA name to AsyncDistillationTrainer by @kashif in #7984top_k with top_p < 1.0 or min_p > 0.0 in GRPO with vLLM importance sampling (see above) by @albertvillanova in #7866torch>=2.6.0 and drop the pre-2.6 FSDP2 and accelerator fallbacks by @albertvillanova in #7642accelerate>=1.11.0 and drop the FSDP2 ignored-modules fallback by @albertvillanova in #7797peft>=0.17.0 and drop the target_parameters guards by @albertvillanova in #7640mm_token_type_ids to the model in Online DPO by @albertvillanova in #7859Failure modes made explicit instead of failing silently or deep in the stack, all by @albertvillanova:
PeftModel passed as ref_model in DPO and KTO (#7808)accuracy_reward by @albertvillanova in #7875GRPOConfig docstring (#7751) and in the GRPO vLLM tip (#7800) by @sergiopaniegomax_length by @qgallouedec in #7913tree_gated_delta_rule.py by @albertvillanova in #7938AMD, all by @albertvillanova: relax the confident-target tolerances for AMD (#7981, #8005)
Minimum versions, all by @albertvillanova: run tests_min_versions on Python 3.11 and pin every dependency at its floor (#7741), including bitsandbytes (#7746), liger-kernel (#7846) and deepspeed (#7855); document it (#7745) and when to add, drop and change the default Python version (#7743)
Docker images, all by @albertvillanova: install Python with uv (#7738), use Python 3.13 (#7739), install kernels through the TRL kernels extra (#7815), remove redundant installs (#7813), skip the uv cache in the release image (#7817), require vLLM >= 0.31.0 (#7820) and explain the floor (#7862), bump uv and upgrade ca-certificates (#7880)
Real objects, not mocks, all by @albertvillanova: real generation outputs in the GOLD and GKD tests (#7902), real config files in the TrlParser tests (#7903), tiny models instead of full-size checkpoints in the GOLD fixtures (#7904), with a new tiny SmolLM3 (#7905, #7906), a real trainer in the IW-OPD tests (#7976), real DistillationLogits in the SDFT tests (#7977), and real objects in the OpenEnv tests (#7985, #7986, #7989, #7990)
MoE LoRA coverage, all by @albertvillanova: test that LoRA on an MoE model trains its routed experts (#7636) and its dense and shared expert layers (#7768), and the error and warning for PEFT versions that skip expert layers (#7860)
Other, all by @albertvillanova: run the test workflows on push to main only (#7848), use the latest Python patch release in pip workflows (#7881), test the math-verify rewards in CI (#7928), check the LFM2.5-2.6B chat template against the Hub (#7907), skip the async GRPO tests on transformers < 5.2.0 (#7750), skip the Liger fixture and tests without an accelerator (#7868), import trl first in the tests so peft 0.17 imports with transformers v5 (#7873), cap the KV cache in the continuous batching tests (#7825), fix a port race in the OpenReward fixture (#7886), pass enable_thinking only to templates that use it (#7898), UTF-8 in tests (#7764), plus GOLD SmolVLM test handling against transformers 5.20.1 (#7728, #7791, #7810, #7929)
Full Changelog: v1.16.0...v1.17.0
🛠️ TRL now trains on AWS Trainium
The fused LM head from v1.15 is what made Neuron tractable; the remaining gaps are closed here. There is a guide to go with it.
Neuron recompiles once per input shape, so an unpadded run stalls about a minute on every new sequence length. pad_to_multiple_of now defaults to "auto": 128 on Neuron, None elsewhere, so nothing changes on GPU. Trainium2, Qwen3-0.6B SFT, 20 steps: 130 s with the default against 260 s without.
pad_to_multiple_of to "auto" by @qgallouedec in #7724trl env and telemetry by @qgallouedec in #7672AsyncGRPO now packs every micro-batch row as a prefix tree instead of a flat sequence. Multi-turn and multi-generation rollouts repeat a lot of prefix: on a shared prompt, those tokens used to be forwarded once per row. Tree packing forwards them once.
rollout 0: [ prompt ] [ completion A ] prompt ──┬── completion A ── later turn
rollout 1: [ prompt ] [ completion B ] => └── completion B
rollout 2: [ prompt ] [ completion A ] [ … ]
Chains without a fork collapse into segments, and each segment's queries attend to the K/V gathered along its root-to-segment path in a single FA3 varlen call. No mask tensor: every layer except attention runs once per unique token, and only K/V is duplicated along shared prefixes.
Qwen3-1.7B, 32K tokens per rollout, forward + backward:
| layout | packing ratio | sequence packing | tree packing | peak GiB |
|---|---|---|---|---|
| shared prompt | 1.00 | 8677 ms | 8737 ms (0.99×) | 35.2 → 35.2 |
| shared prompt | 2.29 | 8266 ms | 4341 ms (1.90×) | 35.2 → 18.0 |
| shared prompt | 3.37 | 8179 ms | 2755 ms (2.97×) | 35.2 → 13.6 |
| nested fork | 1.60 | 8557 ms | 6217 ms (1.38×) | 35.2 → 23.8 |
| nested fork | 3.56 | 8172 ms | 2635 ms (3.10×) | 35.2 → 13.5 |
There is no packing option to choose: with nothing shared a tree row is exactly the old sequence packing, at a ~1% cost.
by @AmineDiro in #7328
Support for meta-models/Muse-Spark-Open-816B-A42B: the tiny-model script, coverage in the VLM tests (SFT, DPO, KTO, chat-template utils, fused LM head), and a LoRA SFT example on 8 nodes of 8×H100 with DistributedConfig(fsdp_size=64, ep_size=8), token dispatch and packing. Measured at 14.2 s/step, 131k tokens/step, 40.2 GB per GPU.
Needs transformers>=5.20.0; the adapter save additionally needs transformers#49500 with peft 0.21.2.
by @qgallouedec in #7536
parse_response now receives the tools, so tool-call arguments come back typed from the tool's own schema instead of whatever the raw parse produced.
by @qgallouedec in #7650
num_train_epochs by @lewtun in #7628pad_to_multiple_of by @qgallouedec in #7671None for unscored env.reward in OpenReward by @albertvillanova in #7541nn.DataParallel in RewardTrainer by @albertvillanova in #7571accelerate>=1.10.1 and drop the ParallelismConfig shim by @albertvillanova in #7643torch.compile results in the Triton loss kernels on torch < 2.11. torch_compile=True trained with loss=0.0 and grad_norm=nan at every step. Inductor passes scalar kernel args as constexpr before torch 2.11, the .to(tl.float32) casts raised on those, and Inductor then assumed every input was mutated and miscompiled. By @albertvillanova in #7688max_length (#7703) and for processors with 5D pixel values (#7704), both by @albertvillanovatorch.compile(fullgraph=True) of selective_log_softmax_and_entropy on torch 2.6 by @albertvillanova in #7719torch.compile by @qgallouedec in #7600is_triton_available once in the fused LM head kernels so Dynamo does not warn by @qgallouedec in #7731A sweep making failure modes explicit instead of crashing deep in the stack, all by @albertvillanova:
use_kernels with transformers 5.2.0 and 5.3.0 (#7577), and if kernels is installed with transformers 5.3.0 in the tiny NemotronH scripts (#7575)Real objects, not mocks. The bulk of this cycle, applying the repo's testing rule across the suite, all by @albertvillanova: run the GRPO entropy mask tests on a real trainer instead of a Mock (#7566), the teacher logprob tests on a real trainer instead of a MagicMock (#7579), build real GOLD trainers in the init (#7582), input preparation (#7614), on-policy generation (#7613, #7615), output decoding (#7633), ULD helper (#7634) and no-backend tokenizer (#7635) tests, check GOLD dataset preparation through the trainer (#7632), and build real async trainers in the rollout state checkpoint tests (#7583)
Coverage, all by @albertvillanova: test SFT (#7552) and DPO/KTO/Reward/GRPO/RLOO (#7601) on a model split across devices, test fp16 training across six trainers (#7572)
Experimental suite on main, all by @albertvillanova: run the experimental tests on pushes to main (#7675), nightly (#7681), on demand (#7678) and when their Makefile target or dependencies change (#7680), and report failures to Slack (#7676)
Other, by @albertvillanova unless otherwise noted: use Python 3.13 by default in CI by @qgallouedec in #7648, remove the slow test infrastructure (#7604), rename the distributed smoke tests to distributed tests (#7631), skip the kernel tests when Triton is not installed (#7563), guard the aiohttp import in the experimental async rollout workers (#7578), drop use_cpu and bf16 settings the CPU fixture already covers (#7612), re-record the sft_fa2 invariant reference after the fused LM head by @qgallouedec (#7591), plus warning filters in #7586, #7588, #7590, #7645, #7727
Full Changelog: v1.15.0...v1.16.0
use_liger_kernel=True is deprecated in these four trainers and will be removed in v2.0.0. Liger's layer kernels still apply, but fused_linear_cross_en…
SFT, DPO, KTO, GRPO, RLOO and Distillation now score tokens with a fused LM head: a Triton kernel projects the hidden states through the LM head in tiles and reduces to per-token log-probs and entropy directly, so the [batch, seq, vocab] logits tensor is never built. It is on by default, there is nothing to turn on.
Max trainable sequence length (gemma-3-1b, 262k vocabulary, 79 GiB of GPU memory, bf16, batch size 1, gradient checkpointing, sdpa):
| Trainer | v1.14.2 | v1.15.0 | |
|---|---|---|---|
| DPO | 10,240 | 59,392 | 5.80× |
| KTO (batch size 2) | 9,216 | 63,488 | 6.89× |
| GRPO (scoring only) | 28,672 | 114,688 | 4.00× |
| RLOO (scoring only) | 23,552 | 100,352 | 4.26× |
SFT (loss_type="nll") |
20,480 | 107,520 | 5.25× |
SFT (default chunked_nll) |
107,520 | 107,520 | 1.00× |
At 8,192 tokens, peak memory drops 52% to 82% and steps are 2.3% to 10.9% faster:
| Trainer | Peak GiB v1.14.2 → v1.15.0 | Tokens/s |
|---|---|---|
| DPO | 48.97 → 12.00 (-75.5%) | 1.035× |
| KTO | 64.91 → 11.98 (-81.5%) | 1.045× |
| GRPO | 24.15 → 10.30 (-57.4%) | 1.023× |
| RLOO | 29.28 → 14.02 (-52.1%) | 1.029× |
SFT (nll) |
30.33 → 8.87 (-70.7%) | 1.029× |
The gain is largest for small models with large vocabularies. SFT's previous default (chunked_nll) already avoided full logits, so it is unchanged. Scoring needs Triton on a GPU (Linux with CUDA, ROCm or XPU).
💥 What this changes for you. The full-logits scoring path, the use_liger_kernel chunked path and _forward_redirection are gone from GRPO, RLOO, DPO and KTO:
lm_head now raises. Use modules_to_save=["lm_head"] instead.use_liger_kernel=True is deprecated in these four trainers and will be removed in v2.0.0. Liger's layer kernels still apply, but fused_linear_cross_entropy is forced to False (it would replace the fused head) and setting it explicitly raises. Use model_init_kwargs={"use_kernels": True}.compute_metrics and compute_loss(..., return_outputs=True) still receive the full logits, from an extra forward pass taken only when they are used. Both warn that this is deprecated and will be removed in v2.0.0.Nothing else the trainers logged is lost: the kernel gained log_sum_sq_probs (WPO weighting), mean_logits (logits/chosen, logits/rejected) and is_top1 (DPO's mean_token_accuracy) as opt-in outputs, each checked against the full logits in tests.
1× B300 with the PyTorch allocator capped at 79 GiB (about an H100 80GB's usable memory), google/gemma-3-1b-pt built from its config with random bf16 weights, lr 1e-6, AdamW, synthetic token ids at exact length. Max length: one fresh process per attempt, 2 training steps, doubling from 8k then bisection to 1024 tokens, arms interleaved round-robin in the same job. Throughput: 12 steps, first 2 discarded, median of 3 A/B/A/B repeats in the same job. Noise floor (A vs A) is 0.1% to 0.8%, below every gap reported. GRPO/RLOO replace generation with fixed random completions, so only scoring is measured.
Not measured: H100 hardware itself, flash-attention (sdpa only), real generation, multi-GPU.
patch_fused_lm_head to add_fused_lm_head by @qgallouedec in #7415logits_scaling and lm_head_multiplier in the fused LM head by @YaseenBashaT in #7439With gradient checkpointing on, the attention output is now saved during the forward instead of being recomputed in the backward, recovering most of the checkpointing slowdown at long context for one extra hidden-state-sized tensor per layer. Same eager SAC approach torchtitan uses under FSDP2, no torch.compile needed.
SFTConfig(
gradient_checkpointing=True,
gradient_checkpointing_kwargs={"selective": True},
)PEFT + DeepSpeed ZeRO-3 is rejected: that combination needs reentrant checkpointing, which SAC cannot use.
assistant_only_loss=True now works on vision datasets. SFTTrainer used to refuse it because processors did not return usable assistant masks once image placeholders were expanded; transformers#48793 fixed that, and it ships in transformers 5.18.
SFTConfig(assistant_only_loss=True) # now fine with an image datasetThe vision collator tokenizes conversational examples through the processor's apply_chat_template and requests the assistant masks to mask non-assistant tokens in the labels. Older transformers still raises, now with the required version in the message.
by @albertvillanova in #7377
Prompts and completions are logged as raw conversation lists instead of batch_decode'd flat strings, so the table renders turn by turn. Much more readable for multi-turn and tool-calling setups, and for VLMs.
The rendered system prompt (including tool definitions injected by the chat template) is no longer visible, since the original messages are shown rather than the fully-templated string.
by @qgallouedec in #5309
AsyncDistillationTrainer by @kashif in #7476loss_is_scaled_for_ga by @qgallouedec in #7508base_url in VLLMClient by @qgallouedec in #7462processor_kwargs in the VLM SFT collator by @qgallouedec in #7547trainer.end() at the end of the scripts and examples by @qgallouedec in #7384nn.DataParallel (and drop the multi-GPU slow test job) by @qgallouedec in #7407use_liger_kernel is deprecated in DPOTrainer, KTOTrainer, GRPOTrainer and RLOOTrainer and will be removed in v2.0.0. These trainers now use the fused LM head, so only Liger's layer kernels apply; use model_init_kwargs={"use_kernels": True} instead. Docs updated by @qgallouedec in #7535prepare_multimodal_messages_vllm by @behroozazarkhalili in #6946SFTTrainer not appending EOS when dataset_text_field is not "text" by @JoeyTan21 in #7446precompute_ref_log_probs under FSDP with no reference model by @qgallouedec in #7507requests and urllib3 imports in the vLLM client by @qgallouedec in #7545, and in the experimental async vLLM clients by @albertvillanova in #7561fix(cpo, orpo): truncate responses independently to prevent empty completions by @RohanMali2003 in #6588[MFU] Use device peak flops instead of hardcoded H100's by @dwarez in #7197model_init_kwargs in CLI scripts by @DaoyuanLi2816 in #7394resume_from_checkpoint in CLI scripts by @DaoyuanLi2816 in #7391trl vllm-serve by @albertvillanova in #7379FIX: test_offloading_with_peft_models on L40, register model params in activation offloading tests by @AmineDiro in #7380gate_proj LoRA target from the Nemotron 3 SFT example by @behroozazarkhalili in #6866in_proj in the Nemotron 3 LoRA example by @behroozazarkhalili in #6870sft_qwen3_8b_1m_context.py with a version check by @kumarrah2002 in #7537expandable_segments allocator config for memory tuning by @akshansh47 in #5794SimpleNamespace, object.__new__ and mocks in tests in the agent instructions by @qgallouedec in #7459Chat template coverage, so drift stops recurring, all by @albertvillanova: document every stored chat template and check it in CI (#7502), add a script checking that reference Hub repos still ship a stored chat template (#7523), check it weekly (#7521), and keep the earlier LFM2 revision under test (#7527)
Tiny-model config alignment, continuing the sweep, all by @albertvillanova: DeepSeek-V3 (#7398) and V3-0528 (#7399), GPT-OSS (#7424), Qwen2-VL (#7425) and Qwen2.5-VL (#7426) mrope sections, Nemotron-3-Nano (#7443), Nemotron-3.5-Lightning (#7444), Nemotron-3-Super (#7447), Nemotron-3-Ultra (#7448), LFM2 (#7400), LFM2.5 (#7532)
Test quality, following the "real objects, not mocks" rule, all by @albertvillanova: build real trainers instead of object.__new__ in the GRPO and SDFT tests (#7531), build real configs instead of SimpleNamespace in the GOLD and GKD tests (#7564), build the async rollout loops through __init__ (#7555), run the async GRPO rollout loop tests on the real tokenizer and response parser (#7556), align the DFT loss tests on a real model output (#7481)
Suite hygiene, all by @albertvillanova unless noted: move the vLLM training (#7488), continuous batching (#7487) and SFT activation offloading (#7477) tests to the regular suite, unmark the vLLM client/server tests as slow (#7490), run them on a single xdist worker (#7489), replace the SFT slow tests with an fp16 test (#7550), remove the always-skipped Gemma 3n tests (#7485), drop the no-op slow and low_priority markers (#7486), stop piping the vLLM test server output (#7414), run the VLM test server on the last visible accelerator (#7412), test the vLLM client against the documented vllm serve command (#7378), inject MODEL_REVISIONS into the Auto loaders (#7528), mirror the pad token onto the model config in the callback tests (#7383), remove the DataParallel gather warning filter (#7554), tie the mamba-ssm autocast warning filter removal to transformers 5.6 (#7431), remove the nightly cache cleanup workflow (#7395), scope GITHUB_TOKEN permissions per job by @hf-security-analysis (#7381), bump the doc-builder workflow pin by @paulinebm (#7542), Dependabot group bump (#7411)
Hotfixes: xfail the GRPO continuous batching test under DataParallel (
Note truncated.
Patch release fixing two cases of silently wrong training and three crashes.
Patch release fixing two cases of silently wrong training and three crashes.
Worth calling out: #7505 affects GRPO, RLOO and Distillation on the default generation path (use_vllm=False). Models that end a turn with an id only their generation config declares (Gemma 3/4, Phi-3.5, ...) did not stop at the end of the turn, and everything generated after it was trained on, up to max_completion_length. The only visible sign was completions/clipped_ratio close to 1. With vLLM, generation stopped correctly but the metrics were still wrong, and mask_truncated_completions=True dropped every finished completion. Runs on models whose tokenizer eos is their only end-of-turn id are unchanged.
precompute_ref_log_probs under FSDP with no reference model by @qgallouedec in #7507Full Changelog: v1.14.1...v1.14.2
Fix the server-mode crash at the first weight sync on vLLM 0.20 to 0.25 by @albertvillanova in #7410
Full Changelog: v1.14.0...v1.14.1
trl.losses is gone: DPO, KTO and GRPO stream their own log-probs
trl.losses is gone: DPO, KTO and GRPO stream their own log-probsWarning
from trl.losses import FusedLinearDPOLoss (or FusedLinearKTOLoss, FusedLinearGRPOLoss, FusedLinearJSDLoss) no longer works. The module introduced in v1.13 has been removed.
v1.13 vendored Liger's chunked_loss into trl.losses as a holding action, not a destination (#7063). The problem it was holding: those classes reimplement each trainer's loss math, so TRL carried two copies of every formula, and copies drift. That drift is where the bugs were. GRPO's grad_norm differed from the default path, KTO ignored the reference model under PEFT, KTO class weights and DPO label_smoothing were silently dropped. Worst of all, under plain DDP the fused path ran the loss on the unwrapped model, so DistributedDataParallel's reducer was never armed and gradients were never all-reduced: every rank silently kept its own.
The fix is to stop reimplementing. All DPO, KTO and GRPO need from the fused path is per-token log-probs of the selected tokens, and TRL already had _ChunkedLogProbFunction for exactly that: it streams the vocabulary with an online logsumexp and recomputes in the backward pass. Each trainer now runs backbone -> _ChunkedLogProbFunction -> its own existing loss code, unchanged. One implementation of each loss, and the memory win is kept, because full logits are still never materialized.
Thirteen restrictions existed only because the loss had been rewritten in a form that could not express those options. Most are gone: DPO with use_liger_kernel=True now accepts mixed loss types, f-divergences and precomputed reference log-probs, and GRPO gains entropy and off-policy masking. Still refused: use_weighting, compute_metrics, a PEFT adapter on lm_head, prompt-learning PEFT, and the MoE auxiliary loss.
Single H100, Qwen3-0.6B, bs=4, seq 512:
| config | median step | peak |
|---|---|---|
| v1.13 fused, inline ref | 0.2243 s | 7.05 GB |
| v1.14 chunked, inline ref | 0.2457 s (+9.5%) | 7.05 GB |
v1.14 chunked, precompute_ref_log_probs=True |
0.1971 s (-12.1%) | 4.83 GB (-31%) |
The third row is the point: that configuration did not exist before, because the fused path rejected precomputed reference log-probs outright.
use_liger_kernel=True still works and still enables Liger's model kernels through transformers. In DPO, GRPO and KTO it now selects TRL's chunked log-probability path instead of Liger's fused loss.
The loss head is the only hot path TRL owns: transformers already kernelizes the model internals, and nothing on the Hub covers what happens after the decoder. On one H100 (bf16, V=151936, H=4096, 8192 tokens), selective_log_softmax plus entropy_from_logits cost 12.8 ms and 2.32 GiB over [8, 1024, 151936]. A fused Triton kernel does the same in 0.89 ms, roughly 14x, and lands closer to the fp32 reference than the bf16 path it replaces. GRPO runs this two to four times per step.
It now lives in-tree at trl.kernels and is on by default on CUDA, ROCm and XPU, with the PyTorch implementation as fallback. It was briefly loaded from the Hub; that was reversed because this kernel is pure Triton and compiles at runtime, so the Hub bought nothing while costing a version gate unrelated to whether the kernel works, a download that fails offline, trust_remote_code=True for anyone with kernels installed, a silent except Exception: pass wrapped around all of it, and 13 of 13 tests skipped in CI.
GRPOWithReplayBufferTrainer, BCOTrainer, PRMTrainer, XPOTrainer, NashMDTrainer and the GSPO-token trainer are gone from trl.experimental.
What changed is not the guarantee, which was always that trl.experimental has none, but the fact that the decision is now written down. Removals used to be argued one pull request at a time, with every thread relitigating the policy before it got to the feature. #7182 settled what the judgment weighs, and #7254 put it in the docs: usage, external issues and pull requests that come from people actually running the feature, whether a stable trainer already covers it, maintenance cost, downstream consumers, owner, and age. Deliberately inputs to a judgment rather than a test: no threshold, no notice cycle, and the numbers stay in the removal pull request where you can check them. A paper index entry outlives the code it described.
The numbers are what make the case. BCOTrainer was 1,600 lines carrying its own reference-model handling, DeepSpeed preparation, tokenization and eval loop, for 11 genuine trainings in two months; it was also the only reason TRL depended on scikit-learn and joblib at all, and both are now gone from pyproject.toml. GRPOWithReplayBufferTrainer had hand-copied _generate_and_score_completions from GRPO and drifted to 385 of 616 lines different, for 8 trainings.
Also breaking, and covered above: trl.losses was removed along the way.
compute_loss redirected through the distributed wrapper only for ZeRO-3 and FSDP, so on plain DDP the loss ran on the unwrapped model, DistributedDataParallel.forward() never ran, its reducer was never armed, and every rank silently kept its own gradients. By @qgallouedec in #7247_pack_wrapped sized the output from the sliced view but read from the raw child buffer, so it re-read from the start of the buffer instead of from the slice, and datasets.map(batched=True) passes sliced tables. By @qgallouedec in #6670log_metric on rank 0 alone hung on NCCL, and log_extra crashed inside gather_object with a 1EB allocation. By @behroozazarkhalili in #7382Trainer actually reads, so it no longer realigns them at train time and logs a change the user did not make (part of #7093): eos in the trainers (#7127), written to the text config for composite models (#7315), the reward model pad token (#7318), and the requested pad token applied to vision datasets (#7316, #7352)trl.experimental by @albertvillanova in #7254, the rule the six removals above were made underTiny models, a sweep bringing every tiny test model in line with its reference, all by @albertvillanova:
tie_word_embeddings across the board (#7130)use_mrope (#7332), use_mambapy (#7333), rope_scaling (#7334), cache_implementation and hidden_act (#7335), activation_dropout and prefix (#7336), attention_bias (#7337, #7338)head_dim reduced for Gemma (#7216) and Gemma2 (#7227), ffn_dim passed for OPT (#7232), the Phi3 leftover replaced by the 3 and 3.5 variants (#7239), three with no test consumer retired (#7186)Tests run only when they can be affected, all by @albertvillanova:
main (#7172) and then reverted (#7201), because GitHub buckets cancelled with failure and main looked broken.Release automation. Publishing now happens on a tag push, and refuses unless VERSION in the tagged commit matches the tag (#7165). Release notes come out sectioned, from a label-to-section map (#7166) with documentation and maintenance labels applied automatically from the changed paths (#7205, #7210). Plus the dev version bump (#7160).
Test reliability, all by @albertvillanova: the rerun filter matches the out-of-memory message rather than the exception type (#7292) and needs pytest-rerunfailures>=16.7 to see wrapped exceptions (#7325); ACCELERATE_MIXED_PRECISION is restored between tests (#7321); the Llava assistant-mask tests are un-xfailed now that transformers fixed them (#7373); non-identity adapters in the use_adapter tests (#7275); MODEL_REVISIONS injected into the PEFT loaders (#7288); the reward model pad token covered (#7350).
Workflow hardening. Token permissions narrowed in the test workflows (#7287), the workflows hardened (#7300), and the upstream issue behind the trufflehog digest pin linked (#7214).
Other. Dependabot bumps (#7192, #7299), the latest-release tests moved from T4 to L40S (#7184) with the Slack action checkout workaround dropped (#7183), the kernel publish job run under a personal namespace (#7343), the token sync comments reworded (#7167) and all three generation-config eos shapes covered (#7168), LM-head gradient buffers and GEMMs skipped when its parameters are frozen (#7076), and the end-of-turn loss mask warning made actionable (#7230).
The supported vLLM window moves from >=0.19.1,<=0.28.0 to >=0.20.0,<=0.30.0.
Note truncated.
One column per quarter.