NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning
Latest stable
0.15.0
Last release 7 days ago
10 Nov 2026
Ships fairly regularly
a new release about every 6 weeks
Nearly every release is documented
notes for 31 of 31 stable releases
Nothing withdrawn
no release was ever pulled
4 years old
37 releases · first in 2022
Drag across the ruler to compare, or tap a release to open it.
[CI] Deprecate Windows GPU CCI by @osalpekar in https://github.com/pytorch/rl/pull/1387
This release provides many new features and bug fixes.
TorchRL now publishes Apple Silicon compatible wheels. We drop coverage of python 3.7 in favour of 3.11.
Most algorithms have been cleaned and designed to reach (at least) SOTA results.
Compatibility with MARL settings has been drastically improved, and we provide a good amount of MARL examples within the library:
A prototype RLHF training script is also proposed (#1597)
A whole new category of offline RL algorithms have been integrated: Decision transformers.
One of the major new features of the library is the introduction of the terminated / truncated / done distinction at no cost within the library. All third-party and primary environments are now compatible with this, as well as losses and data collection primitives (collector etc). This feature is also compatible with complex data structures, such as those found in MARL training pipelines.
All losses are now compatible with tensordict-free inputs, for a more generic deployment.
Atari games can now benefit from a EndOfLifeTransform that allows to use the end-of-life as a done state in the loss (#1605)
We provide a KL transform to add a KL factor to the reward in RLHF settings.
Action masking is made possible through the ActionMask transform (#1421)
VC1 is also integrated for better image embedding.
min/maximum -> low/high by @vmoens in https://github.com/pytorch/rl/pull/1500We provide GRU alongside LSTM for POMDP training.
MARL model coverage is now richer of a MultiAgentMLP and MultiAgentCNN! Other improvments for MARL include coverage for nested keys in most places of the library (losses, data collection, environments...)/
env,vec_env and collectors by @matteobettini in https://github.com/pytorch/rl/pull/1462OrnsteinUhlenbeckProcess by @matteobettini in https://github.com/pytorch/rl/pull/1305exist_ok on logging folder by @matteobettini in https://github.com/pytorch/rl/pull/1561We now cover SMAC-v2, PettingZoo, IsaacGymEnvs (prototype) and RoboHive. The D4RL dataset can now be used without the eponym library, which permit training with more recent or older versions of gym.
We provide several speed improvements, in particular for data collection.
_split_and_pad_sequence by @Blonck in https://github.com/pytorch/rl/pull/1147torch.no_grad() for rendering in multiagent PPO tutorial by @matteobettini in https://github.com/pytorch/rl/pull/1511LazyStackedCompositeSpec and introducing consolidate_spec by @matteobettini in https://github.com/pytorch/rl/pull/1392step_mdp() by @matteobettini in https://github.com/pytorch/rl/pull/1334RoundRobinWriter.extend(data) by @xmaples in https://github.com/pytorch/rl/pull/1295rand_action() in BaseEnv by @matteobettini in https://github.com/pytorch/rl/pull/1267RewardSum transform for multiple reward keys by @matteobettini in https://github.com/pytorch/rl/pull/1544step_mdp nested keys by @matteobettini in https://github.com/pytorch/rl/pull/1339tanh_loc in the documentation of TruncatedNormal. by @skandermoalla in https://github.com/pytorch/rl/pull/1205wait instead of is_set to get results in ParallelEnv by @vmoens in https://github.com/pytorch/rl/pull/1562A great THANKS to our contributors, in particular (but not in any particular order) @skandermoalla, @matteobettini, @BY571 and @albertbou92 for their tremendous dedication.
Full Changelog: https://github.com/pytorch/rl/compare/v0.1.1...v0.2.0
Your coding agent can read these notes before it upgrades. Set up the MCP server →
[Refactor] Deprecate interaction_mode by @vmoens in https://github.com/pytorch/rl/pull/1067
_DataCollector -> DataCollectorBase by @vmoens in https://github.com/pytorch/rl/pull/985time_dimand docfix by @matteobettini in https://github.com/pytorch/rl/pull/1014make_composite_from_td by @vmoens in https://github.com/pytorch/rl/pull/1042Full Changelog: https://github.com/pytorch/rl/compare/v0.1.0...v0.1.1
First official beta release of the library!
First official beta release of the library!
torch.distributed collectors by @vmoens in https://github.com/pytorch/rl/pull/934Full Changelog: https://github.com/pytorch/rl/compare/v0.0.5...v0.1.0
[BugFix] Deprecate tensordict.set check skips in transforms by @vmoens in https://github.com/pytorch/rl/pull/951
We change the env.step API, see https://github.com/pytorch/rl/pull/941 for more info.
to_one_hot and to_categorical to take a tensor as parameter by @riiswa in https://github.com/pytorch/rl/pull/816state_dict by @fedebotu in https://github.com/pytorch/rl/pull/944Full Changelog: https://github.com/pytorch/rl/compare/v0.0.4...v0.0.5
[Quality] Deprecate .ipynb tutos by @vmoens in https://github.com/pytorch/rl/pull/710
__repr__ tests by @sladebot in https://github.com/pytorch/rl/pull/435_inplace_update cls parameter passing in __new__ by @nicolas-dufour in https://github.com/pytorch/rl/pull/464device_safe() with device by @ordinskiy in https://github.com/pytorch/rl/pull/485TensorDictPrimer transform by @nicolas-dufour in https://github.com/pytorch/rl/pull/456erase() method for torchrl.timeit by @nicolas-dufour in https://github.com/pytorch/rl/pull/480passing_devices to None by @himjohntang in https://github.com/pytorch/rl/pull/477passing_devices to None" by @vmoens in https://github.com/pytorch/rl/pull/494passing_devices to None by @vmoens in https://github.com/pytorch/rl/pull/495step_tensordict in step_mdp by @romainjln in https://github.com/pytorch/rl/pull/512device argument for modules.models by @yushiyangk in https://github.com/pytorch/rl/pull/524"_" out_keys in tensordictmodel by @jlesuffleur in https://github.com/pytorch/rl/pull/589ProbabilisticTensorDictModule keys by @vmoens in https://github.com/pytorch/rl/pull/603dim argument in TensorDict.squeeze by @jgonik in https://github.com/pytorch/rl/pull/608copy_ changes the index if the dest and source memmap tensors share the same file location by @vmoens in https://github.com/pytorch/rl/pull/631TransformedEnv by @alexanderlobov in https://github.com/pytorch/rl/pull/630step_mdp method in data collector by @adityagandhamal in https://github.com/pytorch/rl/pull/637implement_for decorator by @ordinskiy in https://github.com/pytorch/rl/pull/618deepcopy in transforms.py by @romainjln in https://github.com/pytorch/rl/pull/625standard_normal for RewardScaling by @adityagandhamal in https://github.com/pytorch/rl/pull/682tensordict.nn.TensorDictModule by @tcbegley in https://github.com/pytorch/rl/pull/700speed branch merge on tensordict by @vmoens in https://github.com/pytorch/rl/pull/755common.py by @matteobettini in https://github.com/pytorch/rl/pull/769_set_seed abstract by @matteobettini in https://github.com/pytorch/rl/pull/770ParallelEnv handling of done flag by @matteobettini in https://github.com/pytorch/rl/pull/788HabitatEnv entry in docs by @matteobettini in https://github.com/pytorch/rl/pull/798torchrl.record by @vmoens in https://github.com/pytorch/rl/pull/854Full Changelog: https://github.com/pytorch/rl/compare/v0.0.2a...v0.0.4b
[Quality] Deprecate .ipynb tutos by @vmoens in https://github.com/pytorch/rl/pull/710
__repr__ tests by @sladebot in https://github.com/pytorch/rl/pull/435_inplace_update cls parameter passing in __new__ by @nicolas-dufour in https://github.com/pytorch/rl/pull/464device_safe() with device by @ordinskiy in https://github.com/pytorch/rl/pull/485TensorDictPrimer transform by @nicolas-dufour in https://github.com/pytorch/rl/pull/456erase() method for torchrl.timeit by @nicolas-dufour in https://github.com/pytorch/rl/pull/480passing_devices to None by @himjohntang in https://github.com/pytorch/rl/pull/477passing_devices to None" by @vmoens in https://github.com/pytorch/rl/pull/494passing_devices to None by @vmoens in https://github.com/pytorch/rl/pull/495step_tensordict in step_mdp by @romainjln in https://github.com/pytorch/rl/pull/512device argument for modules.models by @yushiyangk in https://github.com/pytorch/rl/pull/524"_" out_keys in tensordictmodel by @jlesuffleur in https://github.com/pytorch/rl/pull/589ProbabilisticTensorDictModule keys by @vmoens in https://github.com/pytorch/rl/pull/603dim argument in TensorDict.squeeze by @jgonik in https://github.com/pytorch/rl/pull/608copy_ changes the index if the dest and source memmap tensors share the same file location by @vmoens in https://github.com/pytorch/rl/pull/631TransformedEnv by @alexanderlobov in https://github.com/pytorch/rl/pull/630step_mdp method in data collector by @adityagandhamal in https://github.com/pytorch/rl/pull/637implement_for decorator by @ordinskiy in https://github.com/pytorch/rl/pull/618deepcopy in transforms.py by @romainjln in https://github.com/pytorch/rl/pull/625standard_normal for RewardScaling by @adityagandhamal in https://github.com/pytorch/rl/pull/682tensordict.nn.TensorDictModule by @tcbegley in https://github.com/pytorch/rl/pull/700speed branch merge on tensordict by @vmoens in https://github.com/pytorch/rl/pull/755common.py by @matteobettini in https://github.com/pytorch/rl/pull/769_set_seed abstract by @matteobettini in https://github.com/pytorch/rl/pull/770ParallelEnv handling of done flag by @matteobettini in https://github.com/pytorch/rl/pull/788HabitatEnv entry in docs by @matteobettini in https://github.com/pytorch/rl/pull/798torchrl.record by @vmoens in https://github.com/pytorch/rl/pull/854Full Changelog: https://github.com/pytorch/rl/compare/v0.0.2a...v0.0.4b
[Quality] Deprecate .ipynb tutos by @vmoens in https://github.com/pytorch/rl/pull/710
__repr__ tests by @sladebot in https://github.com/pytorch/rl/pull/435_inplace_update cls parameter passing in __new__ by @nicolas-dufour in https://github.com/pytorch/rl/pull/464device_safe() with device by @ordinskiy in https://github.com/pytorch/rl/pull/485TensorDictPrimer transform by @nicolas-dufour in https://github.com/pytorch/rl/pull/456erase() method for torchrl.timeit by @nicolas-dufour in https://github.com/pytorch/rl/pull/480passing_devices to None by @himjohntang in https://github.com/pytorch/rl/pull/477passing_devices to None" by @vmoens in https://github.com/pytorch/rl/pull/494passing_devices to None by @vmoens in https://github.com/pytorch/rl/pull/495step_tensordict in step_mdp by @romainjln in https://github.com/pytorch/rl/pull/512device argument for modules.models by @yushiyangk in https://github.com/pytorch/rl/pull/524"_" out_keys in tensordictmodel by @jlesuffleur in https://github.com/pytorch/rl/pull/589ProbabilisticTensorDictModule keys by @vmoens in https://github.com/pytorch/rl/pull/603dim argument in TensorDict.squeeze by @jgonik in https://github.com/pytorch/rl/pull/608copy_ changes the index if the dest and source memmap tensors share the same file location by @vmoens in https://github.com/pytorch/rl/pull/631TransformedEnv by @alexanderlobov in https://github.com/pytorch/rl/pull/630step_mdp method in data collector by @adityagandhamal in https://github.com/pytorch/rl/pull/637implement_for decorator by @ordinskiy in https://github.com/pytorch/rl/pull/618deepcopy in transforms.py by @romainjln in https://github.com/pytorch/rl/pull/625standard_normal for RewardScaling by @adityagandhamal in https://github.com/pytorch/rl/pull/682tensordict.nn.TensorDictModule by @tcbegley in https://github.com/pytorch/rl/pull/700speed branch merge on tensordict by @vmoens in https://github.com/pytorch/rl/pull/755common.py by @matteobettini in https://github.com/pytorch/rl/pull/769_set_seed abstract by @matteobettini in https://github.com/pytorch/rl/pull/770ParallelEnv handling of done flag by @matteobettini in https://github.com/pytorch/rl/pull/788HabitatEnv entry in docs by @matteobettini in https://github.com/pytorch/rl/pull/798torchrl.record by @vmoens in https://github.com/pytorch/rl/pull/854Full Changelog: https://github.com/pytorch/rl/compare/v0.0.2a...0.0.4a
The main changes introduced by this release are:
The main changes introduced by this release are:
permute(...).to_tensordict() by @vmoens in https://github.com/pytorch/rl/pull/260torch.stack with destination by @khmigor in https://github.com/pytorch/rl/pull/245BinarizeReward by @srikanthmg85 in https://github.com/pytorch/rl/pull/302Compose.insert and TransformedEnv.insert_transform by @rmartimov in https://github.com/pytorch/rl/pull/304TensorDict.view to remove unnecessary ViewedTensorDict object creation by @bamaxw in https://github.com/pytorch/rl/pull/319__setitem__ for SubTensorDict by @vmoens in https://github.com/pytorch/rl/pull/369ProbabilisticTensorDictModule dist key mapping support by @nicolas-dufour in https://github.com/pytorch/rl/pull/376share_memory_() by @fdabek1 in https://github.com/pytorch/rl/pull/412biased_softplus semantic to allow for minimum scale setting by @nicolas-dufour in https://github.com/pytorch/rl/pull/428set*() methods for TensorDictBase class by @zeenolife in https://github.com/pytorch/rl/pull/438next_ to the out_key by @ggimler3 in https://github.com/pytorch/rl/pull/449device_safe in replay buffer code by @vmoens in https://github.com/pytorch/rl/pull/454batch_lock attribute in EnvBase by @nicolas-dufour in https://github.com/pytorch/rl/pull/399__repr__ tests by @sladebot in https://github.com/pytorch/rl/pull/435_inplace_update cls parameter passing in __new__ by @nicolas-dufour in https://github.com/pytorch/rl/pull/464device_safe() with device by @ordinskiy in https://github.com/pytorch/rl/pull/485TensorDictPrimer transform by @nicolas-dufour in https://github.com/pytorch/rl/pull/456erase() method for torchrl.timeit by @nicolas-dufour in https://github.com/pytorch/rl/pull/480passing_devices to None by @himjohntang in https://github.com/pytorch/rl/pull/477passing_devices to None" by @vmoens in https://github.com/pytorch/rl/pull/494passing_devices to None by @vmoens in https://github.com/pytorch/rl/pull/495step_tensordict in step_mdp by @romainjln in https://github.com/pytorch/rl/pull/512device argument for modules.models by @yushiyangk in https://github.com/pytorch/rl/pull/524"_" out_keys in tensordictmodel by @jlesuffleur in https://github.com/pytorch/rl/pull/589ProbabilisticTensorDictModule keys by @vmoens in https://github.com/pytorch/rl/pull/603dim argument in TensorDict.squeeze by @jgonik in https://github.com/pytorch/rl/pull/608copy_ changes the index if the dest and source memmap tensors share the same file location by @vmoens in https://github.com/pytorch/rl/pull/631TransformedEnv by @alexanderlobov in https://github.com/pytorch/rl/pull/630step_mdp method in data collector by @adityagandhamal in https://github.com/pytorch/rl/pull/637implement_for decorator by @ordinskiy in https://github.com/pytorch/rl/pull/618deepcopy in transforms.py by @romainjln in https://github.com/pytorch/rl/pull/625standard_normal for RewardScaling by @adityagandhamal in https://github.com/pytorch/rl/pull/682Full Changelog: https://github.com/pytorch/rl/compare/v0.0.1...0.0.3
[BugFix] Fixed compose which ignored inv_transforms of child by @nicolas-dufour in https://github.com/facebookresearch/rl/pull/328
__setitem__ for SubTensorDict by @vmoens in https://github.com/facebookresearch/rl/pull/369ProbabilisticTensorDictModule dist key mapping support by @nicolas-dufour in https://github.com/facebookresearch/rl/pull/376share_memory_() by @fdabek1 in https://github.com/facebookresearch/rl/pull/412biased_softplus semantic to allow for minimum scale setting by @nicolas-dufour in https://github.com/facebookresearch/rl/pull/428set*() methods for TensorDictBase class by @zeenolife in https://github.com/facebookresearch/rl/pull/438next_ to the out_key by @ggimler3 in https://github.com/facebookresearch/rl/pull/449device_safe in replay buffer code by @vmoens in https://github.com/facebookresearch/rl/pull/454batch_lock attribute in EnvBase by @nicolas-dufour in https://github.com/facebookresearch/rl/pull/399Full Changelog: https://github.com/facebookresearch/rl/compare/v0.0.1c...v0.0.2a
Adding additional checks to TensorDict.view to remove unnecessary ViewedTensorDict object creation by @bamaxw in https://github.com/facebookresearch/r
TensorDict.view to remove unnecessary ViewedTensorDict object creation by @bamaxw in https://github.com/facebookresearch/rl/pull/319Full Changelog: https://github.com/facebookresearch/rl/compare/v0.0.1b...v0.0.1c
[Feature] Nested tensordicts by @vmoens in https://github.com/facebookresearch/rl/pull/256
Supports nested tensordicts:
Padding for tensordicts:
Speed improvements:
Logging capabilities:
Doc
permute(...).to_tensordict() by @vmoens in https://github.com/facebookresearch/rl/pull/260torch.stack with destination by @khmigor in https://github.com/facebookresearch/rl/pull/245BinarizeReward by @srikanthmg85 in https://github.com/facebookresearch/rl/pull/302Compose.insert and TransformedEnv.insert_transform by @rmartimov in https://github.com/facebookresearch/rl/pull/304Full Changelog: https://github.com/facebookresearch/rl/compare/v0.0.1...v0.0.1b
Nothing published for this version
One column per quarter.