NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1901 most downloaded on PyPI
The Deep Learning framework to train, deploy, and ship AI products Lightning fast.
Last release 8 days ago
10 Sep 2026
Ships unpredictably
gaps range from 8 days to 4 months
Nearly every release is documented
notes for 57 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
4 years old
172 releases · first in 2022
One column per quarter.
Nothing published for this version
Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.5.0...2.5.0.post0
Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.5.0...2.5.0.post0
Updated checkpointing documentation to mark resume_from_checkpoint as deprecated
Lightning AI :zap: is excited to announce the release of Lightning 2.5.
Lightning 2.5 comes with improvements on several fronts, with zero API changes. Our users love it stable, we keep it stable :smile:.
Talking about love :heart:, the lightning, pytorch-lightning and lightning-fabric packages are collectively getting more than 10M downloads per month :open_mouth:, for a total of over 180M downloads :exploding_head: since the early days . It's incredible to see PyTorch Lightning getting such a strong adoption across the industry and the sciences.
Release 2.5 embraces PyTorch 2.5, and it marks some of its more recent directions as officially supported, namely tensor subclass-based APIs like Distributed Tensors and TorchAO, in combination with torch.compile.
Here's a couple of examples:
<details><summary>Distributed FP8 transformer with PyTorch Lightning</summary>
Full example here
import lightning as L
import torch
import torch.nn as nn
import torch.nn.functional as F
from lightning.pytorch.demos import Transformer, WikiText2
from lightning.pytorch.strategies import ModelParallelStrategy
from torch.distributed._composable.fsdp.fully_shard import fully_shard
from torch.utils.data import DataLoader
from torchao.float8 import Float8LinearConfig, convert_to_float8_training
class LanguageModel(L.LightningModule):
def __init__(self, vocab_size):
super().__init__()
self.vocab_size = vocab_size
self.model = None
def configure_model(self):
if self.model is not None:
return
with torch.device("meta"):
model = Transformer(
vocab_size=self.vocab_size,
nlayers=16,
nhid=4096,
ninp=1024,
nhead=32,
)
float8_config = Float8LinearConfig(
# pip install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/ triton-nightly # noqa
pad_inner_dim=True,
)
def module_filter_fn(mod: torch.nn.Module, fqn: str):
# we skip the decoder because it typically vocabulary size
# is not divisible by 16 as required by float8
return fqn != "decoder"
convert_to_float8_training(model, config=float8_config, module_filter_fn=module_filter_fn)
for module in model.modules():
if isinstance(module, (nn.TransformerEncoderLayer, nn.TransformerDecoderLayer)):
fully_shard(module, mesh=self.device_mesh)
fully_shard(model, mesh=self.device_mesh)
self.model = torch.compile(model)
def training_step(self, batch):
input, target = batch
output = self.model(input, target)
loss = F.nll_loss(output, target.view(-1))
self.log("train_loss", loss, prog_bar=True)
return loss
def configure_optimizers(self):
return torch.optim.Adam(self.parameters(), lr=1e-4)
def train():
L.seed_everything(42)
dataset = WikiText2()
train_dataloader = DataLoader(dataset, num_workers=8, batch_size=1)
model = LanguageModel(vocab_size=dataset.vocab_size)
mp_strategy = ModelParallelStrategy(
data_parallel_size=4,
tensor_parallel_size=1,
)
trainer = L.Trainer(strategy=mp_strategy, max_steps=100, precision="bf16-true", accumulate_grad_batches=8)
trainer.fit(model, train_dataloader)
trainer.print(torch.cuda.memory_summary())
if __name__ == "__main__":
torch.set_float32_matmul_precision("high")
train()
</details>
<details><summary>Distributed FP8 transformer with Fabric</summary>
Full example here
import lightning as L
import torch
import torch.nn as nn
import torch.nn.functional as F
from lightning.fabric.strategies import ModelParallelStrategy
from lightning.pytorch.demos import Transformer, WikiText2
from torch.distributed._composable.fsdp.fully_shard import fully_shard
from torch.distributed.device_mesh import DeviceMesh
from torch.utils.data import DataLoader
from torchao.float8 import Float8LinearConfig, convert_to_float8_training
from tqdm import tqdm
def configure_model(model: nn.Module, device_mesh: DeviceMesh) -> nn.Module:
float8_config = Float8LinearConfig(
# pip install -U --index-url <https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/> triton-nightly # noqa
pad_inner_dim=True,
)
def module_filter_fn(mod: torch.nn.Module, fqn: str):
# we skip the decoder because it typically vocabulary size
# is not divisible by 16 as required by float8
return fqn != "decoder"
convert_to_float8_training(model, config=float8_config, module_filter_fn=module_filter_fn)
for module in model.modules():
if isinstance(module, (torch.nn.TransformerEncoderLayer, torch.nn.TransformerDecoderLayer)):
fully_shard(module, mesh=device_mesh)
fully_shard(model, mesh=device_mesh)
return torch.compile(model)
def train():
L.seed_everything(42)
batch_size = 8
micro_batch_size = 1
max_steps = 100
dataset = WikiText2()
dataloader = DataLoader(dataset, num_workers=8, batch_size=micro_batch_size)
with torch.device("meta"):
model = Transformer(
vocab_size=dataset.vocab_size,
nlayers=16,
nhid=4096,
ninp=1024,
nhead=32,
)
strategy = ModelParallelStrategy(data_parallel_size=4, tensor_parallel_size=1, parallelize_fn=configure_model)
fabric = L.Fabric(precision="bf16-true", strategy=strategy)
fabric.launch()
model = fabric.setup(model)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
optimizer = fabric.setup_optimizers(optimizer)
dataloader = fabric.setup_dataloaders(dataloader)
iterable = tqdm(enumerate(dataloader), total=len(dataloader)) if fabric.is_global_zero else enumerate(dataloader)
steps = 0
for i, batch in iterable:
input, target = batch
is_accumulating = i % (batch_size // micro_batch_size) != 0
with fabric.no_backward_sync(model, enabled=is_accumulating):
output = model(input, target)
loss = F.nll_loss(output, target.view(-1))
fabric.backward(loss)
if not is_accumulating:
fabric.clip_gradients(model, optimizer, max_norm=1.0)
optimizer.step()
optimizer.zero_grad()
steps += 1
if fabric.is_global_zero:
iterable.set_postfix_str(f"train_loss={loss.item():.2f}")
if steps == max_steps:
break
fabric.print(torch.cuda.memory_summary())
if __name__ == "__main__":
torch.set_float32_matmul_precision("high")
train()
</details>
As these examples show, it's now easier than ever to take your PyTorch Lightning module and run it with FSDP2 and/or tensor parallelism in FP8 precision, using the ModelParallelStrategy we introduced in 2.4.
Also note the use of distributed tensor APIs, TorchAO APIs, and torch.compile directly in the configure_model hook (or in the parallelize function in Fabric's ModelParallelStrategy), as opposed to the LightningModule as a whole. The advantage with this approach is that you can just copy-paste the parallelize functions that come with native PyTorch models directly in configure_model and get the same effect, no head-scratching involved :nerd_face:.
Talking about head scratching, we also made a pass at the PyTorch Lightning internals and hardened the parts where we keep track of progress counters during training, validation, testing, as well as learning rate scheduling, in relation to resuming from checkpoints. We now made sure there are no (to the best of our knowledge) edge cases where stopping and resuming from checkpoints can change the sequence of loops or other internal states. Fault tolerance for the win :partying_face:!
Alright! Feel free to take a look at the full changelog below.
And of course: the best way to use PyTorch Lightning and Fabric is through Lightning Studio :zap:. Access GPUs, train models, deploy and more with zero setup. Focus on data and models - not infrastructure.
<a name="changelog"></a>
<a name="changelog-pytorch"></a>
<details open><summary>Added</summary>
step parameter to TensorBoardLogger.log_hyperparams to visualize changes during training (#20176)str method to datamodule (#20301)Trainer.save_checkpoint (#20405)</details>
<details open><summary>Changed</summary>
resume_from_checkpoint as deprecated (#20477)np.random.SeedSequence() in pl_worker_init_function() to robustly seed NumPy-dependent dataloader workers (#20369)2.5 (#20351)_ (#20221)BytesIO as path in .to_onnx() (#20172)</details>
<details open><summary>Removed</summary>
List[int] as input type for Trainer when accelerator="cpu" (#20399)</details>
<details open><summary>Fixed</summary>
convert_module in FSDP to avoid using more memory than necessary during initialization (#20323)configure_optimizers when running with ReduceLROnPlateau (#20471)configure_optimizers example (#20420)_class_path parameter (#20221)</details>
<a name="changelog-fabric"></a>
<details open><summary>Added</summary>
step parameter to TensorBoardLogger.log_hyperparams to visualize changes during training (#20176)ddp_find_unused_parameters_true alias in Fabric's DDPStrategy (#20125)</details>
<details open><summary>Changed</summary>
np.random.SeedSequence() in pl_worker_init_function() to robustly seed NumPy-dependent dataloader workers (#20369)2.5 (#20351)</details>
<details open><summary>Removed</summary>
</details>
<details open><summary>Fixed</summary>
convert_module in FSDP to avoid using more memory than necessary during initialization (#20323)</details>
</br>
Full commit list: 2.4.0 -> 2.5.0
<a name="contributors"></a>
We thank all folks who submitted issues, features, fixes and doc changes. It's the only way we can collectively make Lightning :zap: better for everyone, nice job!
In particular, we would like to thank the authors of the pull-requests above, in no particular order:
@ringohoffman @MrWhatZitToYaa @jedyang97 @chualanagit @lantiga @AlessandroW @kazuar @t-vi @01AbhiSingh @WangYue0000 @amorehead @EricCousineau-TRI @mauvilsa @Borda @pete-mcelroy @ali-alshaar7 @GdoongMathew @farhadrgh @tshu-w @LukasSalchow @awindmann @dadwadw233 @qingquansong
Thank you :heart: and we hope you'll keep them coming!
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Avoid LightningCLI saving hyperparameters with class_path and init_args since this would be a breaking change
Lightning AI :zap: is excited to announce the release of Lightning 2.4. This is mainly a compatibility upgrade for PyTorch 2.4 and Python 3.12, with a sprinkle of a few features and bug fixes.
Did you know? The Lightning philosophy extends beyond a boilerplate-free deep learning framework: We've been hard at work bringing you Lightning Studio. Code together, prototype, train, deploy, host AI web apps. All from your browser, with zero setup.
<a name="changelog"></a>
<a name="changelog-pytorch"></a>
<details open><summary>Added</summary>
dump_stats flag to AdvancedProfiler (#19703)verbose to the seed_everything() function (#20108)TQDMProgressBar now provides an option to retain prior training epoch bars (#19578)ModelSummary table (#20159)</details>
<details open><summary>Changed</summary>
.fit(), .evaluate(), .test() or .predict() now terminates all processes launched by the Trainer and exits the program (#19976)seed_everything(..., workers=True) (#20055)</details>
<details open><summary>Removed</summary>
</details>
<details open><summary>Fixed</summary>
class_path and init_args since this would be a breaking change (#20068)seed_everything() (#20108)_LoggerConnector's _ResultMetric to move all registered keys to the device of the logged value if needed (#19814)_optimizer_to_device logic for special 'step' key in optimizer state causing performance regression (#20019)ModelSummary when model has distributed parameters (DTensor) (#20163)</details>
<a name="changelog-fabric"></a>
<details open><summary>Added</summary>
verbose to the seed_everything() function (#20108)</details>
<details open><summary>Changed</summary>
seed_everything(..., workers=True) (#20055)</details>
<details open><summary>Removed</summary>
</details>
<details open><summary>Fixed</summary>
_lazy_load() function (#20121)_optimizer_to_device logic for special 'step' key in optimizer state causing performance regression (#20019)</details>
</br>
Full commit list: 2.3.0 -> 2.4.0
<a name="contributors"></a>
We thank all our contributors who submitted pull requests for features, bug fixes and documentation updates.
Chuck Norris can solve NP-hard problems in polynomial time. In fact, any problem is easy when Chuck Norris solves it.
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
This release removes the code from the main lightning package that was reported in CVE-2024-5980.
This release removes the code from the main lightning package that was reported in CVE-2024-5980.
Includes a minor bugfix that avoids a conflict with the entrypoint command with another package #20041.
Includes a minor bugfix that avoids a conflict with the entrypoint command with another package #20041.
Includes minor bugfixes and stability improvements.
Includes minor bugfixes and stability improvements.
Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.3.0...2.3.1
Lightning AI is excited to announce the release of Lightning 2.3 :zap:
Lightning AI is excited to announce the release of Lightning 2.3 :zap:
Did you know? The Lightning philosophy extends beyond a boilerplate-free deep learning framework: We've been hard at work bringing you Lightning Studio. Code together, prototype, train, deploy, host AI web apps. All from your browser, with zero setup.
This release introduces experimental support for Tensor Parallelism and 2D Parallelism, PyTorch 2.3 support, and several bugfixes and stability improvements.
<a name="highlights"></a>
<a name="highlights-tensor-parallel"></a>
Tensor parallelism (TP) is a technique that splits up the computation of selected layers across GPUs to save memory and speed up distributed models. To enable TP as well as other forms of parallelism, we introduce a ModelParallelStrategy for both Lightning Trainer and Fabric. Under the hood, TP is enabled through new experimental PyTorch APIs like DTensor and torch.distributed.tensor.parallel.
Enabling TP in a model with PyTorch Lightning requires you to implement the LightningModule.configure_model() method where you convert selected layers of a model to paralellized layers. This is an advanced feature, because it requires a deep understanding of the model architecture. Open the tutorial Studio to learn the basics of Tensor Parallelism.
<a target="_blank" href="https://lightning.ai/lightning-ai/studios/tensor-parallelism-supercharging-large-model-training-with-pytorch-lightning"> <img src="https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/app-2/studio-badge.svg" alt="Open In Studio"/> </a>
import lightning as L
from lightning.pytorch.strategies import ModelParallelStrategy
from torch.distributed.tensor.parallel import ColwiseParallel, RowwiseParallel
from torch.distributed.tensor.parallel import parallelize_module
# 1. Implement the `configure_model()` method in LightningModule
class LitModel(L.LightningModule):
def __init__(self):
super().__init__()
self.model = FeedForward(8192, 8192)
def configure_model(self):
# Lightning will set up a `self.device_mesh` for you
tp_mesh = self.device_mesh["tensor_parallel"]
# Use PyTorch's distributed tensor APIs to parallelize the model
plan = {
"w1": ColwiseParallel(),
"w2": RowwiseParallel(),
"w3": ColwiseParallel(),
}
parallelize_module(self.model, tp_mesh, plan)
def training_step(self, batch):
...
# 2. Create the strategy
strategy = ModelParallelStrategy()
# 3. Configure devices and set the strategy in Trainer
trainer = L.Trainer(accelerator="cuda", devices=2, strategy=strategy)
trainer.fit(...)
<details><summary>Full training example (requires at least 2 GPUs).</summary>
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.distributed.tensor.parallel import ColwiseParallel, RowwiseParallel
from torch.distributed.tensor.parallel import parallelize_module
import lightning as L
from lightning.pytorch.demos.boring_classes import RandomDataset
from lightning.pytorch.strategies import ModelParallelStrategy
class FeedForward(nn.Module):
def __init__(self, dim, hidden_dim):
super().__init__()
self.w1 = nn.Linear(dim, hidden_dim, bias=False)
self.w2 = nn.Linear(hidden_dim, dim, bias=False)
self.w3 = nn.Linear(dim, hidden_dim, bias=False)
def forward(self, x):
return self.w2(F.silu(self.w1(x)) * self.w3(x))
class LitModel(L.LightningModule):
def __init__(self):
super().__init__()
self.model = FeedForward(8192, 8192)
def configure_model(self):
if self.device_mesh is None:
return
# Lightning will set up a `self.device_mesh` for you
tp_mesh = self.device_mesh["tensor_parallel"]
# Use PyTorch's distributed tensor APIs to parallelize the model
plan = {
"w1": ColwiseParallel(),
"w2": RowwiseParallel(),
"w3": ColwiseParallel(),
}
parallelize_module(self.model, tp_mesh, plan)
def training_step(self, batch):
output = self.model(batch)
loss = output.sum()
return loss
def configure_optimizers(self):
return torch.optim.AdamW(self.model.parameters(), lr=3e-3)
def train_dataloader(self):
# Trainer configures the sampler automatically for you such that
# all batches in a tensor-parallel group are identical
dataset = RandomDataset(8192, 64)
return torch.utils.data.DataLoader(dataset, batch_size=8, num_workers=2)
strategy = ModelParallelStrategy()
trainer = L.Trainer(
accelerator="cuda",
devices=2,
strategy=strategy,
max_epochs=1,
)
model = LitModel()
trainer.fit(model)
trainer.print(f"Peak memory usage: {torch.cuda.max_memory_allocated() / 1e9:.02f} GB")
</details>
</br>
Applying TP in a model with Fabric requires you to implement a special function where you convert selected layers of a model to paralellized layers. This is an advanced feature, because it requires a deep understanding of the model architecture. Open the tutorial Studio to learn the basics of Tensor Parallelism.
<a target="_blank" href="https://lightning.ai/lightning-ai/studios/tensor-parallelism-supercharging-large-model-training-with-lightning-fabric"> <img src="https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/app-2/studio-badge.svg" alt="Open In Studio"/> </a>
import lightning as L
from lightning.fabric.strategies import ModelParallelStrategy
from torch.distributed.tensor.parallel import ColwiseParallel, RowwiseParallel
from torch.distributed.tensor.parallel import parallelize_module
# 1. Implement the parallelization function for your model
def parallelize_feedforward(model, device_mesh):
# Lightning will set up a device mesh for you
tp_mesh = device_mesh["tensor_parallel"]
# Use PyTorch's distributed tensor APIs to parallelize the model
plan = {
"w1": ColwiseParallel(),
"w2": RowwiseParallel(),
"w3": ColwiseParallel(),
}
parallelize_module(model, tp_mesh, plan)
return model
# 2. Pass the parallelization function to the strategy
strategy = ModelParallelStrategy(parallelize_fn=parallelize_feedforward)
# 3. Configure devices and set the strategy in Fabric
fabric = L.Fabric(accelerator="cuda", devices=2, strategy=strategy)
fabric.launch()
<details><summary>Full training example (requires at least 2 GPUs).</summary>
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.distributed.tensor.parallel import ColwiseParallel, RowwiseParallel
from torch.distributed.tensor.parallel import parallelize_module
import lightning as L
from lightning.pytorch.demos.boring_classes import RandomDataset
from lightning.fabric.strategies import ModelParallelStrategy
class FeedForward(nn.Module):
def __init__(self, dim, hidden_dim):
super().__init__()
self.w1 = nn.Linear(dim, hidden_dim, bias=False)
self.w2 = nn.Linear(hidden_dim, dim, bias=False)
self.w3 = nn.Linear(dim, hidden_dim, bias=False)
def forward(self, x):
return self.w2(F.silu(self.w1(x)) * self.w3(x))
def parallelize_feedforward(model, device_mesh):
# Lightning will set up a device mesh for you
tp_mesh = device_mesh["tensor_parallel"]
# Use PyTorch's distributed tensor APIs to parallelize the model
plan = {
"w1": ColwiseParallel(),
"w2": RowwiseParallel(),
"w3": ColwiseParallel(),
}
parallelize_module(model, tp_mesh, plan)
return model
strategy = ModelParallelStrategy(parallelize_fn=parallelize_feedforward)
fabric = L.Fabric(accelerator="cuda", devices=2, strategy=strategy)
fabric.launch()
# Initialize the model
model = FeedForward(8192, 8192)
model = fabric.setup(model)
# Define the optimizer
optimizer = torch.optim.AdamW(model.parameters(), lr=3e-3)
optimizer = fabric.setup_optimizers(optimizer)
# Define dataset/dataloader
dataset = RandomDataset(8192, 64)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=8)
dataloader = fabric.setup_dataloaders(dataloader)
# Simplified training loop
for i, batch in enumerate(dataloader):
output = model(batch)
loss = output.sum()
fabric.backward(loss)
optimizer.step()
optimizer.zero_grad()
fabric.print(f"Iteration {i} complete")
fabric.print(f"Peak memory usage: {torch.cuda.max_memory_allocated() / 1e9:.02f} GB")
</details>
</br>
<a name="highlights-2d-parallel"></a>
Tensor Parallelism by itself can be very effective for efficient inference of very large models. For training, TP is typically combined with other forms of parallelism, such as FSDP, to increase throughput and scalability on large clusters with 100s of GPUs. The new ModelParallelStrategy in this release supports the combination of TP + FSDP, which is referred to as 2D parallelism.
For an introduction to this feature, please also refer to the tutorial Studios (PyTorch Lightning, Lightning Fabric). At the moment, the PyTorch team is reimplementing FSDP under the name FSDP2 with the aim to make it compose well with other parallelisms such as TP. Therefore, for the experimental 2D parallelism support, you'll need to switch to using FSDP2 with the new ModelParallelStrategy. Please refer to our docs (PyTorch Lightning, Lightning Fabric) and stay tuned for future releases as these APIs mature.
<a name="highlights-model-summary"></a>
The model summary table that gets displayed when you run Trainer.fit() now contains a new column "Mode" that shows the training mode each layer is in (#19468).
| Name | Type | Params | Mode
-----------------------------------------------------------------
0 | model | Sam | 93.7 M | train
1 | model.image_encoder | ImageEncoderViT | 89.7 M | eval
2 | model.prompt_encoder | PromptEncoder | 6.2 K | train
3 | model.mask_decoder | MaskDecoder | 4.1 M | train
-----------------------------------------------------------------
93.7 M Trainable params
0 Non-trainable params
93.7 M Total params
374.942 Total estimated model params size (MB)
A module in PyTorch is always either in train (default) or eval mode.
This improvement should give users more visibility into the state of their model and help debug issues, for example when you need to make sure certain layers of the model are frozen.
<a name="highlights-forward-methods"></a>
Until now, Lightning Fabric warned the user in case the forward pass of the model or a subset of its modules was conducted through methods other than the dedicated forward method of the PyTorch module. The reason for this is that PyTorch needs to run special hooks in case of DDP/FSDP and other strategies to function properly, and not running through the real forward method would skip these hooks and lead to correctness issues.
In Lightning Fabric 2.3, we added a feature to explicitly mark alternative forward methods so that Fabric can add the necessary rerouting behind the scenes:
import lightning as L
fabric = L.Fabric(devices=2, strategy="ddp")
fabric.launch()
model = MyModel()
model = fabric.setup(model)
# OK: Calling the model directly
output = model(input)
# ERROR: Calling another method that calls forward indirectly
prediction = model.generate(input)
# New: Mark special forward methods explicitly before using them
model.mark_forward_method(model.generate)
# OK: Now can use `model.generate()` in DDP/FSDP without issues
prediction = model.generate(input)
Find the full example and more details in our docs.
<a name="bc-changes"></a>
The 2.0 series of Lightning releases guarantees core API stability: No name changes, argument renaming, hook removals etc. on core interfaces (Trainer, LightningModule, etc.) unless a feature is specifically marked experimental. Here we list a few behavioral changes made in places where the change was justified if it significantly improves the user experience, improves performance, or fixes the correctness of a feature. These changes will likely not impact most users.
It is no longer allowed to skip training_step() by returning None in distributed training (#19918). The following usage was previously possible but would result in unpredictable hangs and timeouts in distributed training:
def training_step(self, batch):
loss = ...
if loss.isnan():
# No longer allowed in multi-GPU!
# Raises error in Lightning >= 2.3
return None
return loss
We decided to raise an error if the user attempts to return None when running in a multi-GPU setting.
prepare_data() hook in LightningModule and LightningDataModule is now subject to a barrier without timeout to avoid long-running tasks to be interrupted (#19448). Similarly, also in Fabric the Fabric.rank_zero_first context manager now uses an infinite barrier (#19448).<a name="changelog"></a>
<a name="changelog-pytorch"></a>
<details><summary>Added</summary>
ModelSummary and RichModelSummary callbacks now display the training mode of each layer in the column "Mode" (#19468)load_from_checkpoint support for LightningCLI when using dependency injection (#18105)on_exception hook to LightningDataModule (#19601)ModelParallelStrategy to support 2D parallelism (#19878, #19888)torch.distributed.destroy_process_group in atexit handler if process group needs destruction (#19931)FSDPStrategy(device_mesh=...) argument (#19504)</details>
<details><summary>Changed</summary>
prepare_data() hook in LightningModule and LightningDataModule is now subject to a barrier without timeout to avoid long-running tasks to be interrupted (#19448)drop_last for prediction (#19678)training_step() by returning None in distributed training (#19918)</details>
<details><summary>Removed</summary>
Trainer(strategy="bagua")) (#19445)</details>
<details><summary>Fixed</summary>
WandbLogger.log_hyperparameters() raising an error if hyperparameters are not JSON serializable (#19769)ModelCheckpoint(save_last=...) argument (#19808)epoch_loop.restarting to avoid full validation run after LearningRateFinder (#19818)</details>
<a name="changelog-fabric"></a>
<details><summary>Added</summary>
fabric consolidate in the new CLI (#19560)_FabricModule.mark_forward_method() (#19690)ModelParallelStrategy to support 2D parallelism (#19846, #19852, #19870, #19872)torch.distributed.destroy_process_group in atexit handler if process group needs destruction (#19931)FSDPStrategy(device_mesh=...) argument (#19504)</details>
<details><summary>Changed</summary>
lightning run model to fabric run (#19442, #19527)Fabric.rank_zero_first context manager now uses a barrier without timeout to avoid long-running tasks to be interrupted (#19448)fabric.backward() when it is needed by the strategy or precision selection (#19447, #19493)_BackwardSyncControl can now control what to do when gradient accumulation is disabled (#19577)</details>
<details><summary>Removed</summary>
</details>
<details><summary>Fixed</summary>
</details>
</br>
Full commit list: 2.2.0 -> 2.3.0
<a name="contributors"></a>
We thank all our contributors who submitted pull requests for features, bug fixes and documentation updates.
Chuck Norris is a big fan and daily user of Lightning Studio.
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →