App

Fixed

• Fixed HTTPClient retry for flow/work queue (#19837)

PyTorch

No Changes.

Fabric

No Changes.

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.2.3...2.2.4

PyTorch

Fixed

• Fixed WandbLogger.log_hyperparameters() raising an error if hyperparameters are not JSON serializable (#19769)

Fabric

No Changes.

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.2.2...2.2.3

PyTorch

Fixed

• Fixed an issue causing a TypeError when using torch.compile as a decorator (#19627)
• Fixed a KeyError when saving a FSDP sharded checkpoint and setting save_weights_only=True (#19524)

Fabric

Fixed

• Fixed an issue causing a TypeError when using torch.compile as a decorator (#19627)
• Fixed issue where some model methods couldn't be monkeypatched after being Fabric wrapped (#19705)
• Fixed an issue causing weights to be reset in Fabric.setup() when using FSDP (#19755)

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.2.1...2.2.2

Contributors

@ankitgola005 @awaelchli @Borda @carmocca @dmitsf @dvoytan-spark @fnhirwa

PyTorch

Fixed

• Fixed an issue with CSVLogger trying to append to file from a previous run when the version is set manually (#19446)
• Fixed the divisibility check for Trainer.accumulate_grad_batches and Trainer.log_every_n_steps in ThroughputMonitor (#19470)
• Fixed support for Remote Stop and Remote Abort with NeptuneLogger (#19130)
• Fixed infinite recursion error in precision plugin graveyard (#19542)

Fabric

Fixed

• Fixed an issue with CSVLogger trying to append to file from a previous run when the version is set manually (#19446)

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.2.0post...2.2.1

Contributors

@Raalsky @awaelchli @carmocca @Borda

If we forgot someone due to not matching commit email with GitHub account, let us know :]

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.2.0...2.2.0.post0

Lightning AI is excited to announce the release of Lightning 2.2 :zap:

Did you know? The Lightning philosophy extends beyond a boilerplate-free deep learning framework: We've been hard at work bringing you Lightning Studio. Code together, prototype, train, deploy, host AI web apps. All from your browser, with zero setup.

While our previous release was packed with many big new features, this time around we're rolling out mainly improvements based on feedback from the community. And of course, as the name implies, this release fully supports the latest PyTorch 2.2 :tada:

• Highlights
  • Monitoring Throughput
  • Improved Handling of Evaluation Mode
  • Converting FSDP Checkpoints
  • Improvements to Compiling DDP/FSDP in Fabric
  • Saving and Loading DataLoader State
  • Non-strict Checkpoint Loading in Trainer
• Notable Changes
• Full Changelog
  • PyTorch Lightning
  • Lightning Fabric
• Contributors

<a name="highlights"></a>

Highlights

<a name="highlights-throughput"></a>

Monitoring Throughput

Lightning now has built-in utilities to measure throughput metrics such as batches/sec, samples/sec and Model FLOP Utilization (MFU) (#18848).

Trainer:

For the Trainer, this comes in form of a ThroughputMonitor callback. In order to track samples/sec, you need to provide a function to tell the monitor how to extract the batch dimension from your input. Furthermore, if you want to track MFU, you can provide a sample forward pass and the ThroughputMonitor will automatically estimate the utilization based on the hardware you are running on:

import lightning as L
from lightning.pytorch.callbacks import ThroughputMonitor
from lightning.fabric.utilities.throughput import measure_flops


class MyModel(LightningModule):
    def setup(self, stage):
        with torch.device("meta"):
            model = MyModel()

        def sample_forward():
            batch = torch.randn(..., device="meta")
            return model(batch)

        self.flops_per_batch = measure_flops(model, sample_forward, loss_fn=torch.Tensor.sum)


throughput = ThroughputMonitor(
    batch_size_fn=lambda batch: batch.size(0),
    # optional, if your samples have a length (like number of tokens)
    sample_fn=lambda batch: batch.size(1)
)
trainer = L.Trainer(log_every_n_steps=10, callbacks=throughput, logger=...)
model = MyModel()
trainer.fit(model)

The results get automatically sent to the logger if one is configured on the Trainer.

Fabric:

For Fabric, the ThroughputMonitor is a simple utility object on which you call .update() and compute_and_log() during the training loop:

import lightning as L
from lightning.fabric.utilities import ThroughputMonitor


fabric = L.Fabric(logger=...)
throughput = ThroughputMonitor(fabric)

t0 = time()
for batch_idx, batch in enumerate(train_dataloader):
    do_work()
    torch.cuda.synchronize()  # required or else time() won't be correct
    throughput.update(
        time=(time() - t0), 
        batches=batch_idx, 
        samples=(batch_idx * batch_size)
    )
    if batch_idx % 10 == 0:
        throughput.compute_and_log(step=batch_idx)

Check out our TinyLlama LLM pretraining script for a full example using Fabric's ThroughputMonitor.

The troughput utilities can report:

• batches per second (per process and across process)
• samples per second (per process and across process)
• items per second (e.g. tokens) (per process and across process)
• flops per second (per process and across process)
• model flops utilization (MFU) (per process)
• total time, total samples, total batches, and total items (per process)

<a name="highlights-eval"></a>

Improved Handling of Evaluation Mode

When you train a model and have validation enabled, the Trainer automatically calls .eval() when transitioning to the validation loop, and .train() when validation ends. Until now, this had the unfortunate side effect that any submodules in your LightningModule that were in evaluation mode get reset to train mode. In Lightning 2.2, the Trainer now captures the mode of every submodule before switching to validation, and restores the mode the modules were in when validation ends (#18951, #18951, #18951). This improvement will help users avoid silent correctness bugs and removes boilerplate code for managing frozen layers.

import lightning as L


class LitModel(L.LightningModule):
    def __init__(self):
        super().__init__()
        self.trainable_module = ...
        
        # This will now stay in eval mode
        self.frozen_module = ...
        self.frozen_module.eval()
        
    def training_step(self, batch):
        # Previously, modules were all in train mode
        # Now: Modules are in mode they were set up with
        assert self.trainable_module.training
        assert not self.frozen_module.training
        ...
        
    def validation_step(self, batch):
        # All modules are in eval mode
        ...
    
    
model = LitModel()
trainer = L.Trainer()
trainer.fit(model)

If you have overridden any of the LightningModule.on_{validation,test,predict}_model_{eval,train} hooks, they will still get called and execute your custom logic, but they are no longer required if you added them to preserve the eval mode of frozen modules.

[!IMPORTANT] In some libraries, for example HuggingFace, models are created in evaluation mode by default (e.g. HFModel.from_pretrained(...)). Starting from 2.2, you will have to set .train() on these models if you intend to train them.

<a name="highlights-consolidate-fsdp"></a>

Converting FSDP Checkpoints

In the previous release, we introduced distributed checkpointing with FSDP to speed up saving and loading checkpoints for big models. These checkpoints are in a special format saved in a folder with shards from each GPU in a separate file. While these checkpoints can be loaded back with Lightning Trainer or Fabric very easily, they aren't easy to load or process externally. In Lightning 2.2, we introduced a CLI utility that lets you consolidate the checkpoint folder to a single file that can be loaded in raw PyTorch with torch.load() for example (#19213).

Given you saved a distributed checkpoint, you can then convert it like so:

# For Trainer checkpoints:
python -m lightning.pytorch.utilities.consolidate_checkpoint path/to/my/checkpoint


# For Fabric checkpoints:
python -m lightning.fabric.utilities.consolidate_checkpoint path/to/my/checkpoint

Read more about distributed checkpointing in our documentation: Trainer, Fabric.

<a name="highlights-compile"></a>

Improvements to Compiling DDP/FSDP in Fabric

PyTorch 2.0+ introduced torch.compile, a powerful tool to speed up your models without changing the code. We now added a comprehensive guide how to use torch.compile correctly with tips and tricks to help you troubleshoot common issues. On top of that, Fabric.setup() will now reapply torch.compile on top of DDP/FSDP if you are enabling these strategies (#19280).

import lightning as L

# Select a distributed strategy (DDP, FSDP, ...)
fabric = L.Fabric(strategy="ddp", devices=8)

# Compile your model before `.setup()`
model = torch.compile(model)

# Now automatically handles compiling also over DDP/FSDP
model = fabric.setup(model)

# You can opt-out if it is causing trouble
model = fabric.setup(model, _reapply_compile=False)

You might see fewer graph breaks, but there won't be any significant speed-ups with this. We introduced this mainly to make Fabric ready for future improvements from PyTorch to optimizing distributed operations.

<a name="highlights-dataloader-state"></a>

Saving and Loading DataLoader State

If you use a dataloader/iterable that implements the .state_dict() and .load_state_dict() interface, the Trainer will now automatically save and load their state in the checkpoint (#19361).

import lightning as L


class MyDataLoader:
    """A dataloader that implements the 'stateful' interface."""
    
    def state_dict(self):
        # Return a dictionary with state
        return {"batches_fetched": ...}
    
    def load_state_dict(self, state_dict):
        # Load the state from the checkpoint
        self.batches_fetched = state_dict["batches_fetched"]


model = ...
dataloader = MyDataLoader()
trainer = L.Trainer()

# Saves checkpoints that include the dataloader state
trainer.fit(model, dataloader)

# When you resume training, the dataloader can now load its state
trainer.fit(model, dataloader, ckpt_path="path/to/my/checkpoint")

Note that the standard PyTorch DataLoader does not support this stateful interface. This feature only works on loaders that implement these two methods. A dataloader that supports full fault-tolerance will be included in our upcoming release of Lightning Data - a library to optimize data preprocessing and streaming in the cloud. Stay tuned!

<a name="highlights-non-strict"></a>

Non-strict Checkpoint Loading in Trainer

A feature that has been requested for a long time by the community is non-strict checkpoint loading. By default, a checkpoint in PyTorch is loaded with strict=True to ensure all keys in the saved checkpoint match what's in the model's state dict. However, in some use cases it might make sense to exclude certain weights from being included in the checkpoint. When resuming training, the user would then be required to set strict=False, which wasn't configurable until now.

You can now set the attribute strict_loading=False on your LightningModule if you want to allow loading partial checkpoints (#19404).

import lightning as L

class LitModel(L.LightningModule):
    def __init__(self):
        super().__init__()
        
        # This model only trains the decoder, we don't save the encoder
        self.encoder = from_pretrained(...).requires_grad_(False)
        self.decoder = Decoder()
        
        # Set to False because we only care about the decoder
        self.strict_loading = False
    
    def state_dict(self):
        # Don't save the encoder, it is not being trained
        return {k: v for k, v in super().state_dict().items() if "encoder" not in k}

...

trainer = L.Trainer()
model = LitModel()

# Will load weights with `.load_state_dict(strict=model.strict_loading)`
trainer.fit(model, ckpt_path="path/to/checkpoint")

Full documentation here.

<a name="bc-changes"></a>

Notable Changes

The 2.0 series of Lightning releases guarantees core API stability: No name changes, argument renaming, hook removals etc. on core interfaces (Trainer, LightningModule, etc.) unless a feature is specifically marked experimental. Here we list a few behavioral changes made in places where the change was justified if it significantly improves the user experience, improves performance, or fixes the correctness of a feature. These changes will likely not impact most users.

<a name="highlights-save-last"></a>

ModelCheckpoint's save-last Feature

In Lightning 2.1, we made the ModelCheckpoint(..., save_last=True) feature save a symbolic link to the last saved checkpoint instead of rewriting the checkpoint (#18748). This time saver is especially useful for large models who take a while to save. However, many users were confused by the new behavior and wanted it turned off, saving a copy instead of a symbolic link like before. In Lightning 2.2, we are reverting this decision and make the linking opt-in (#19191):

from lightning.pytorch.callbacks import ModelCheckpoint

# In 2.1 saves a symbolic link "last.ckpt" to the last checkpoint saved
# In 2.2 saves "last.ckpt" as a copy of the last checkpoint saved
checkpoint = ModelCheckpoint("./my_checkpoints", save_last=True)

# You can opt-in to save a symlink (if possible)
checkpoint = ModelCheckpoint("./my_checkpoints", save_last="link")

<a name="highlights-seed-everything"></a>

Removed Problematic Default Seeding

The seed_everything(x) utility function is useful to set the seed for several libraries like PyTorch, NumPy and Python in a single line of code. However, until now you were allowed to omit passing a seeding value, in which case the function picked a seed value randomly. In certain cases, for example when processes are launched externally (e.g., SLURM, torchelastic etc.), this default behavior is dangerous because each process will independently choose a random seed. This can affect sampling, randomized validation splits, and other behaviors that rely on each process having the same seed. In 2.2, we removed this default behavior and default to a seed value 0 (#18846):

from lightning.pytorch.utilities import seed_everything

# Set the random seed for PyTorch, NumPy, Python etc.
seed_everything(42)

# Not setting a value now defaults to 0
seed_everything()

In the unlikely event that you relied on the previous behavior, you now have to choose the seed randomly yourself:

seed_everything(random.randint(0, 1000000))

Miscellaneous Changes

• Dropped support for PyTorch 1.12 (#19300)
• The columns in the metrics.csv file produced by CSVLogger are now sorted alphabetically (#19159)
• Added support for meta-device initialization and materialization of 4-bit Bitsandbytes layers (#19150)
• Added TransformerEnginePrecision(fallback_compute_dtype=) to control the dtype of operations that don't support fp8 (#19082)
• We renamed the TransformerEnginePrecision(dtype=) argument to weights_dtype and made it required (#19082)
• The LightningModule.load_from_checkpoint() function now calls .configure_model() on the model if it is overridden, to ensure all layers can be loaded from the checkpoint (#19036)

<a name="changelog"></a>

CHANGELOG

<a name="changelog-pytorch"></a>

PyTorch Lightning

<details><summary>Added</summary>

• Added lightning.pytorch.callbacks.ThroughputMonitor to track throughput and log it (#18848)
• The Trainer now restores the training mode set through .train() or .eval() on a submodule-level when switching from validation to training (#18951)
• Added support for meta-device initialization and materialization of 4-bit Bitsandbytes layers (#19150)
• Added TransformerEnginePrecision(fallback_compute_dtype=) to control the dtype of operations that don't support fp8 (#19082)
• Added the option ModelCheckpoint(save_last='link') to create a symbolic link for the 'last.ckpt' file (#19191)
• Added a utility function and CLI to consolidate FSDP sharded checkpoints into a single file (#19213)
• The TQDM progress bar now respects the env variable TQDM_MINITERS for setting the refresh rate (#19381)
• Added support for saving and loading stateful training DataLoaders (#19361)
• Added shortcut name strategy='deepspeed_stage_1_offload' to the strategy registry (#19075)
• Added support for non-strict state-dict loading in Trainer via the new LightningModule.strict_loading = True | False attribute (#19404)

</details>

<details><summary>Changed</summary>

• seed_everything() without passing in a seed no longer randomly selects a seed, and now defaults to 0 (#18846)
• The LightningModule.on_{validation,test,predict}_model_{eval,train} now only get called if they are overridden by the user (#18951)
• The Trainer.fit() loop no longer calls LightningModule.train() at the start; it now preserves the user's configuration of frozen layers (#18951)
• The LightningModule.load_from_checkpoint() function now calls .configure_model() on the model if it is overridden, to ensure all layers can be loaded from the checkpoint (#19036)
• Restored usage of step parameter when logging metrics with NeptuneLogger (#19126)
• Changed the TransformerEnginePrecision(dtype=) argument to weights_dtype and made it required (#19082)
• The columns in the metrics.csv file produced by CSVLogger are now sorted alphabetically (#19159)
• Reverted back to creating a checkpoint copy when ModelCheckpoint(save_last=True) instead of creating a symbolic link (#19191)

</details>

<details><summary>Deprecated</summary>

• Deprecated all precision plugin classes under lightning.pytorch.plugins with the suffix Plugin in the name (#18840)

</details>

<details><summary>Removed</summary>

• Removed support for PyTorch 1.12 (#19300)

</details>

<details><summary>Fixed</summary>

• Fixed issue where the precision="transformer-engine" argument would not replace layers by default (#19082)
• Fixed issue where layers created in LightningModule.setup or LightningModule.configure_model wouldn't get converted when using the Bitsandbytes or TransformerEngine plugins (#19061)
• Fixed the input validation logic in FSDPStrategy to accept a device_mesh (#19392)

</details>

<a name="changelog-fabric"></a>

Lightning Fabric

<details><summary>Added</summary>

• Added lightning.fabric.utilities.ThroughputMonitor and lightning.fabric.utilities.Throughput to track throughput and log it (#18848)
• Added lightning.fabric.utilities.AttributeDict for convenient dict-attribute access to represent state in script (#18943)
• Added support for meta-device initialization and materialization of 4-bit Bitsandbytes layers (#19150)
• Added TransformerEnginePrecision(fallback_compute_dtype=) to control the dtype of operations that don't support fp8 (#19082)
• Added support for clipping gradients by value with FSDP (#19236)
• Added a utility function and CLI to consolidate FSDP sharded checkpoints into a single file (#19213)
• Added support for re-compiling the model inside Fabric.setup() over the FSDP/DDP wrappers (#19280)

</details>

<details><summary>Changed</summary>

• seed_everything() without passing in a seed no longer randomly selects a seed, and now defaults to 0 (#18846)
• Changed the TransformerEnginePrecision(dtype=) argument to weights_dtype and made it required (#19082)
• The columns in the metrics.csv file produced by CSVLogger are now sorted alphabetically (#19159)

</details>

<details><summary>Removed</summary>

• Removed support for PyTorch 1.12 (#19300)

</details>

<details><summary>Fixed</summary>

• Fixed parsing of v100s GPUs in get_available_flops (#18952)
• Fixed issue where the precision="transformer-engine" argument would not replace layers by default (#19082)
• Fixed the input validation logic in FSDPStrategy to accept a device_mesh (#19392)

</details>

</br>

Full commit list: 2.1.0 -> 2.2.0

<a name="contributors"></a>

Contributors

Everyone who contributed between 2.1 and 2.2, in no particular order:

Veteran

@nik777 @Raalsky @wouterzwerink @AleksanderWWW @awaelchli @nohalon @ioangatop @Borda @ethanwharris @BoringDonut @mauvilsa @parambharat @tchaton @ryan597 @adamjstewart @rasbt @carmocca

New

@hiaoxui @VictorPrins @jaswon @AMHermansen @JalinWang @MF-FOOM @unacanal @Jamim @harishb00 @asingh9530 @dipta007 @daturkel @jerrymannil @mjbommar @shenmishajing @paganpasta @lauritsf @andyland @mathematicalmichael

Did you know?

Chuck Norris is a big fan and daily user of PyTorch Lightning.

This is a preview release for Lightning 2.2.0.

Fabric

Fixed

• Fixed an issue preventing Fabric to run on CPU when the system's CUDA driver is outdated or broken (#19234)
• Fixed typo in kwarg in SpikeDetection (#19282)

---

PyTorch

Fixed

• Fixed Trainer not expanding the default_root_dir if it has the ~ (home) prefix (#19179)
• Fixed warning for Dataloader if num_workers=1 and CPU count is 1 (#19224)
• Fixed WandbLogger.watch() method annotation to accept None for the log parameter (#19237)
• Fixed an issue preventing the Trainer to run on CPU when the system's CUDA driver is outdated or broken (#19234)
• Fixed an issue with the ModelCheckpoint callback not saving relative symlinks with ModelCheckpoint(save_last="link") (#19303)
• Fixed issue where the _restricted_classmethod_impl would incorrectly raise a TypeError on inspection rather than on call (#19332)
• Fixed exporting __version__ in __init__ (#19221)

---

Full Changelog: https://github.com/Lightning-AI/pytorch-lightning/compare/2.1.3...2.1.4

Contributors

@andyland @asingh9530 @awaelchli @Borda @daturkel @dipta007 @lauritsf @mjbommar @shenmishajing @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Changed

• Lightning App: Use the batch get endpoint (#19180)
• Drop starsessions from App's requirements (#18470)
• Optimize loading time for chunks to be there (#19109)

---

Data

Added

• Add fault tolerance StreamingDataset (#19052)
• Add numpy support for the StreamingDataset (#19050)
• Add fault tolerance for the StreamingDataset (#19049)
• Add direct s3 support to the StreamingDataset (#19044)
• Add disk usage check before downloading files (#19041)

Changed

• Cleanup chunks right away if the dataset doesn't fit within the cache in StreamingDataset (#19168)
• StreamingDataset improve deletion strategy (#19118)
• Improve StreamingDataset Speed (#19114)
• Remove time in the Data Processor progress bar (#19108)
• Optimize loading time for chunks to be there (#19109)
• Resolve path for StreamingDataset (#19094)
• Make input dir in DataProcessor required (#18910)
• Remove the LightningDataset relying on un-maintained torchdata (#19019)

Fixed

• Resolve checkpointing for the Streaming Dataset (#19123)
• Resolve Item Loader bugs (#19017)

---

Fabric

Fixed

• Avoid moving the model to device if move_to_device=False is passed (#19152)
• Fixed broadcast at initialization in MPIEnvironment (#19074)

---

PyTorch

Changed

• LightningCLI no longer allows setting a normal class instance as default. A lazy_instance can be used instead (#18822)

Fixed

• Fixed checks for local file protocol due to fsspec changes in 2023.10.0 (#19023)
• Fixed automatic detection of 'last.ckpt' files to respect the extension when filtering (#17072)
• Fixed an issue where setting CHECKPOINT_JOIN_CHAR or CHECKPOINT_EQUALS_CHAR would only work on the ModelCheckpoint class but not on an instance (#19054)
• Fixed ModelCheckpoint not expanding the dirpath if it has the ~ (home) prefix (#19058)
• Fixed handling checkpoint dirpath suffix in NeptuneLogger (#18863)
• Fixed an edge case where ModelCheckpoint would alternate between versioned and unversioned filename (#19064)
• Fixed broadcast at initialization in MPIEnvironment (#19074)
• Fixed the tensor conversion in self.log to respect the default dtype (#19046)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/2.1.2...2.1.3

Contributors

@AleksanderWWW, @awaelchli, @borda, @carmocca, @dependabot[bot], @mauvilsa, @MF-FOOM, @tchaton, @yassersouri

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Changed

• Forced plugin server to use localhost (#18976)
• Enabled bundling additional files into app source (#18980)
• Limited rate of requests to http queue (#18981)

---

Fabric

Fixed

• Fixed precision default from environment (#18928)

---

PyTorch

Fixed

• Fixed an issue causing permission errors on Windows when attempting to create a symlink for the "last" checkpoint (#18942)
• Fixed an issue where Metric instances from torchmetrics wouldn't get moved to the device when using FSDP (#18954)
• Fixed an issue preventing the user to Trainer.save_checkpoint() an FSDP model when Trainer.test/validate/predict() ran after Trainer.fit() (#18992)

---

Contributors

@awaelchli, @carmocca, @ethanwharris, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

Full Changelog: https://github.com/Lightning-AI/lightning/compare/2.1.1...2.1.2

App

Added

• add flow fail() (#18883)

Fixed

• Fix failing lightning cli entry point (#18821)

---

Fabric

Changed

• Calling a method other than forward that invokes submodules is now an error when the model is wrapped (e.g., with DDP) (#18819)

Fixed

• Fixed false-positive warnings about method calls on the Fabric-wrapped module (#18819)
• Refined the FSDP saving logic and error messaging when the path exists (#18884)
• Fixed layer conversion under Fabric.init_module() context manager when using the BitsandbytesPrecision plugin (#18914)

---

PyTorch

Fixed

• Fixed an issue when replacing an existing last.ckpt file with a symlink (#18793)
• Fixed an issue when BatchSizeFinder steps_per_trial parameter ends up defining how many validation batches to run during the entire training (#18394)
• Fixed an issue saving the last.ckpt file when using ModelCheckpoint on a remote filesystem, and no logger is used (#18867)
• Refined the FSDP saving logic and error messaging when the path exists (#18884)
• Fixed an issue parsing the version from folders that don't include a version number in TensorBoardLogger and CSVLogger (#18897)

---

Contributors

@awaelchli, @borda, @BoringDonut, @carmocca, @hiaoxui, @ioangatop, @nohalon, @rasbt, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

Full Changelog: https://github.com/Lightning-AI/lightning/compare/2.1.0...2.1.1

Lightning AI is excited to announce the release of Lightning 2.1 :zap: It's the culmination of work from 79 contributors who have worked on features, bug-fixes, and documentation for a total of over 750+ commits since v2.0.

The theme of 2.1 is "bigger, better, faster": Bigger because training large multi-billion parameter models has gotten even more efficient thanks to FSDP, efficient initialization and sharded checkpointing improvements, better because it's easier than ever to scale models without making substantial code changes or installing third-party packages and faster because it leverages the latest hardware features to speed up training in low-bit precision thanks to new precision plugins like bitsandbytes and transformer engine. And of course, as the name implies, this release fully leverages the latest features in PyTorch 2.1 :tada:

• Highlights
  • Improvements To Large-Scale Training With FSDP
  • True Half-Precision
  • Bitsandbytes Quantization
  • Transformer Engine
  • Lightning on TPU Goes Brrr
  • Granular Control Over Checkpoints in Fabric
• Backward Incompatible Changes
  • PyTorch Lightning
  • Lightning Fabric
• Full Changelog
  • PyTorch Lightning
  • Lightning Fabric
  • Lightning App
• Contributors

<a name="highlights"></a>

Highlights

<a name="highlights-fsdp"></a>

Improvements To Large-Scale Training With FSDP

The FSDP strategy for training large billion-parameter models gets substantial improvements and new features in Lightning 2.1, both in Trainer and Fabric (in case you didn't know, Fabric is the latest addition to the Lightning family of tools to scale models without the boilerplate code). FSDP is now more user-friendly to configure, has memory management and speed improvements, and we have a brand new end-to-end user guide with best practices (Trainer, Fabric).

Efficient Saving and Loading of Large Checkpoints

When training large billion-parameter models with FSDP, saving and resuming training, or even just loading model parameters for finetuning can be challenging, as users are are often plagued by out-of-memory errors and speed bottlenecks.

In 2.1, we made several improvements. Starting with saving checkpoints, we added support for distributed/sharded checkpoints, enabled through the setting state_dict_type in the strategy (#18364, #18358):

Trainer:

import lightning as L
from lightning.pytorch.strategies import FSDPStrategy

# Default used by the strategy
strategy = FSDPStrategy(state_dict_type="full")

# Enable saving distributed checkpoints
strategy = FSDPStrategy(state_dict_type="sharded")

trainer = L.Trainer(strategy=strategy, ...)

Fabric:

import lightning as L
from lightning.fabric.strategies import FSDPStrategy

# Saving distributed checkpoints is the default
strategy = FSDPStrategy(state_dict_type="sharded")

# Save consolidated (single file) checkpoints
strategy = FSDPStrategy(state_dict_type="full")

fabric = L.Fabric(strategy=strategy, ...)

Distributed checkpoints are the fastest and most memory efficient way to save the state of very large models. The distributed checkpoint format also makes it efficient to load these checkpoints back for resuming training in parallel, and it reduces the impact on CPU memory usage significantly. Furthermore, we've also introduced lazy-loading for non-distributed checkpoints (#18150, #18379), which greatly reduces the impact on CPU memory usage when loading a consolidated (single-file) checkpoint (e.g. for finetuning). Learn more about these features in our FSDP guides (Trainer, Fabric).

Fast and Memory-Optimized Initialization

A major challenge that users face when working with large models such as LLMs is dealing with the extreme memory requirements. Even something as simple as instantiating a model becomes non-trivial if the model is so large it won't fit in a single GPU or even a single machine. In Lightning 2.1, we are introducing empty-weights initialization through the Fabric.init_module() (#17462, #17627) and Trainer.init_module()/LightningModule.configure_model() (#18004, #18004, #18385) methods:

Trainer:

import lightning as L

class MyModel(L.LightningModule):
    def __init__(self):
        super().__init__()
        # Delay initialization of model to `configure_model()`

    def configure_model(self):
        # Model initialized in correct precision and weights on meta-device
        self.model = ...

    ...

trainer = L.Trainer(strategy="fsdp", ...)
trainer.fit(model)

Fabric:

import lightning as L

fabric = L.Fabric(strategy="fsdp", ...)

# Model initialized in correct precision and weights on meta-device
with fabric.init_module(empty_init=True):
    model = ...
    

# You can also initialize buffers and tensors directly on device and dtype
with fabric.init_tensor():
    model.mask.create()
    model.kv_cache.create()
    x = torch.randn(4, 128)

# Materialization and sharding of model happens inside here
model = fabric.setup(model)

Read more about this new feature and its other benefits in our docs (Trainer, Fabric).

User-Friendly Configuration

We made it super easy to configure the sharding- and activation-checkpointing policy when you want to auto-wrap particular layers of your model for advanced control (#18045, #18084).

  import lightning as L
  from lightning.pytorch.strategies import FSDPStrategy
- from torch.distributed.fsdp.wrap import ModuleWrapPolicy

- strategy = FSDPStrategy(auto_wrap_policy=ModuleWrapPolicy({MyTransformerBlock}))
+ strategy = FSDPStrategy(auto_wrap_policy={MyTransformerBlock})
  trainer = L.Trainer(strategy=strategy, ...)

Furthermore, the sharding strategy can now be conveniently set with a string value (#18087):

  import lightning as L
  from lightning.pytorch.strategies import FSDPStrategy
- from torch.distributed.fsdp.fully_sharded_data_parallel import ShardingStrategy

- strategy = FSDPStrategy(sharding_strategy=ShardingStrategy.SHARD_GRAD_OP)
+ strategy = FSDPStrategy(sharding_strategy="SHARD_GRAD_OP")
  trainer = L.Trainer(strategy=strategy, ...)

You no longer need to remember the long PyTorch imports! Fabric also supports all these improvements shown above.

<a name="highlights-half-precision"></a>

True Half-Precision

Lightning now supports true half-precision for training and inference with all built-in strategies (#18193, #18217, #18213, #18219). With this setting, the memory required to store the model weights is only half of what is normally needed when running with float32. In addition, you get the same speed benefits as mixed precision training (precision="16-mixed") has:

import lightning as L

# default
trainer = L.Trainer(precision="32-true")

# train with model weights in `torch.float16`
trainer = L.Trainer(precision="16-true")

# train with model weights in `torch.bfloat16`
# (if hardware supports it)
trainer = L.Trainer(precision="bf16-true")

The same settings are also available in Fabric! We recommend to try bfloat16 training (precision="bf16-true") as it is often more numerically stable than regular 16-bit precision (precision="16-true").

<a name="highlights-bitsandbytes"></a>

Bitsandbytes Quantization

With the new Bitsandbytes precision plugin #18655, you can now quantize your model for significant memory savings during training, finetuning, or inference with a selection of several state-of-the-art quantization algorithms (int8, fp4, nf4 and more). For the first time, Trainer and Fabric make bitsandbytes easy to use for general models.

Trainer:

import lightning as L
from lightning.pytorch.plugins import BitsandbytesPrecisionPlugin

# this will pick out the compute dtype automatically, by default `bfloat16`
precision = BitsandbytesPrecisionPlugin("nf4-dq")
trainer = L.Trainer(plugins=precision)

Fabric:

import lightning as L
from lightning.fabric.plugins import BitsandbytesPrecision

# this will pick out the compute dtype automatically, by default `bfloat16`
precision = BitsandbytesPrecision("nf4-dq")
trainer = L.Fabric(plugins=precision)

Learn more!

<a name="highlights-transformer-engine"></a>

Transformer Engine

The Transformer Engine by NVIDIA is a library for accelerating transformer layers on the new Hopper (H100) generation of GPUs. With the integration in Lightning Trainer and Fabric (#17597, #18459), you have easy access to the 8-bit mixed precision for significant speed ups:

Trainer:

import lightning as L

# Select 8-bit mixed precision via TransformerEngine, with model weights in float16
trainer = L.Trainer(precision="transformer-engine-float16")

Fabric:

import lightning as L

# Select 8-bit mixed precision via TransformerEngine, with model weights in float16
fabric = L.Fabric(precision="transformer-engine-float16")

More configuration options are available through the respective plugins in Trainer and Fabric.

<a name="highlights-tpu"></a>

Lightning on TPU Goes Brrr

Lightning 2.1 runs on the latest generation of TPU hardware on Google Cloud! TPU-v4 and TPU-v5 (#17227) are now fully supported both in Fabric and Trainer and run using the new PjRT runtime by default (#17352). PjRT is the runtime used by Jax and has shown an average improvement of 35% on benchmarks.

Trainer:

import lightning as L

trainer = L.Trainer(accelerator="tpu", devices=8)
model = MyModel()
trainer.fit(model)  # uses PjRT if available

Fabric:

import lightning as L


def train(fabric):
    ...

fabric = L.Fabric(accelerator="tpu")
fabric.launch(train)  # uses PjRT if available

And what's even more exciting, you can now scale massive multi-billion parameter models on TPUs using FSDP (#17421).

import lightning as L
from lightning.fabric.strategies import XLAFSDPStrategy

strategy = XLAFSDPStrategy(
    # Most arguments from the PyTorch native FSDP strategy are also available here!
    auto_wrap_policy={Block},
    activation_checkpointing_policy={Block},
    state_dict_type="full",
    sequential_save=True,
)
    
fabric = L.Fabric(devices=8, strategy=strategy)
fabric.launch(finetune)

You can find a full end-to-end finetuning example script in our Lit-GPT repository. The new XLA-FSDP strategy is experimental and currently only available in Fabric. Support in the Trainer will follow in the future.

<a name="highlights-fabric-checkpoints"></a>

Granular Control Over Checkpoints in Fabric

Several improvements for checkpoint saving and loading have landed in Fabric, enabling more fine-grained control over what is saved/loaded while reducing boilerplate code:

1. There is a new Fabric.load_raw() method with which you can load model- or optimizer state-dicts saved externally by a non-Fabric application (e.g., raw PyTorch) (#18049)

   import lightning as L
   
   fabric = L.Fabric()
   model = MyModel()
   
   # A model weights file saved by your friend who doesn't use Fabric
   fabric.load_raw("path/to/model.pt", model)
   
   # Equivalent to this:
   # model.load_state_dict(torch.load("path/to/model.pt"))
   

2. A new parameter Fabric.load(..., strict=True|False) to disable strict loading (#17645)

   import lightning as L
   
   fabric = L.Fabric()
   model = MyModel()
   state = {"model": model}
   
   # strict loading is the default
   fabric.load("path/to/checkpoint.ckpt", state, strict=True)
   
   # disable strict loading
   fabric.load("path/to/checkpoint.ckpt", state, strict=False)
   

3. A new parameter Fabric.save(..., filter=...) that enables you to exclude certain parameters of your model without writing boilerplate code for it (#17845)

   import lightning as L
   
   fabric = L.Fabric()
   model, optimizer = ...
   
   state = {"model": model, "optimizer": optimizer, "foo": 123}
   
   # save only the weights that match a pattern
   filter = {"model": lambda k, v: "weight" in k}
   fabric.save("path/to/checkpoint.ckpt", state, filter=filter)
   

You can read more about the new options in our checkpoint guide.

<a name="bc-changes"></a>

Backward Incompatible Changes

The release of PyTorch Lightning 2.0 was a big step into a new chapter: It brought a more polished API and removed a lot of legacy code and outdated as well as experimental features, at the cost of a long list of breaking changes resulting in more work needed than usual to upgrade from 1.9 to 2.0. Moving forward, we promised to maintain full backward compatibility of our public core APIs to guarantee a smooth upgrade experience for everyone, and with 2.1 we are happy to deliver on this promise. A few exceptions were made in places where the change was justified if it significantly improves the user experience, improves performance, or fixes the correctness of a feature. These changes will likely not impact most users.

<a name="bc-changes-pytorch"></a>

PyTorch Lightning

TPU/XLA Changes

When selecting device indices via devices=[i], the Trainer now selects the i-th TPU core (0-based, previously it was 1-based) (#17227)

Before:

# Selects the first TPU core (1-based index)
trainer = Trainer(accelerator="tpu", devices=[1])

Now:

# Selects the second TPU core (0-based index)
trainer = Trainer(accelerator="tpu", devices=[1])

Multi-GPU in Jupyter Notebooks

Due to lack of reliability, Trainer now only runs on one GPU instead of all GPUs in a Jupyter notebook if devices="auto" (default) (#18291)

Before:

import lightning as L

# In Jupyter notebooks, this would select all available GPUs (DDP)
trainer = L.Trainer(accelerator="cuda", devices="auto")

Now:

# In Jupyter notebooks, this now selects only one GPU (the first)
trainer = L.Trainer(accelerator="cuda", devices="auto")

# You can still explicitly select multiple
trainer = L.Trainer(accelerator="cuda", devices=8)

Device Access in Setup Hook

• During LightningModule.setup(), the self.device now returns the device the module will be placed on instead of cpu (#18021)

Before:

def setup(self, stage):
    # CPU regardless of the accelerator used
    print(self.device)

Now:

def setup(self, stage):
    # CPU/CUDA/MPS/XLA depending on accelerator
    print(self.device)

Miscellaneous Changes

• self.loged tensors are now kept in the original device to reduce unnecessary host-to-device synchronizations (#17334)
• The FSDPStrategy now loads checkpoints after the configure_model/configure_sharded_model hook (#18358)
• The FSDPStrategy.load_optimizer_state_dict and FSDPStrategy.load_model_state_dict are a no-op now (#18358)
• Removed experimental support for torchdistx due to a lack of project maintenance (#17995)
• Dropped support for PyTorch 1.11 (#18691)

<a name="bc-changes-fabric"></a>

Lightning Fabric

We thank the community for the amazing feedback we got for Fabric so far - keep it coming. The list of breaking changes is short and won't affect the vast majority of users.

Sharding Context Manager in Fabric.run()

We removed automatic sharding support with Fabric.run or using fabric.launch(fn). This only impacts FSDP and DeepSpeed strategy users who use this way of launching. Please note that Fabric.run is a legacy construct from the LightningLite days, and is not recommended today. Please instantiate your large FSDP or DeepSpeed model under the newly added fabric.init_module context manager (#17832).

Before:

import lightning as L

def train(fabric):
    # FSDP's `enable_wrap` context or `deepspeed.zero.Init()`
    # were applied automaticaly here
    model = LargeModel()
    ...
        
fabric = L.Fabric()
fabric.launch(train)

Now:

def train(fabric):
    # Use `init_module` explicitly to apply these context managers
    with fabric.init_module():
        model = LargeModel()
    ...

Multi-GPU in Jupyter Notebooks

Due to lack of reliability, Fabric now only runs on one GPU instead of all GPUs in a Jupyter notebook if devices="auto" (default) (#18291)

Before:

import lightning as L

# In Jupyter notebooks, this would select all available GPUs (DDP)
fabric = L.Fabric(accelerator="cuda", devices="auto")

Now:

# In Jupyter notebooks, this now selects only one GPU (the first)
fabric = L.Fabric(accelerator="cuda", devices="auto")

# You can still explicitly select multiple
fabric = L.Fabric(accelerator="cuda", devices=8)

<a name="changelog"></a>

CHANGELOG

<a name="changelog-pytorch"></a>

PyTorch Lightning

<details><summary>Added</summary>

• Added metrics_format attribute to RichProgressBarTheme class (#18373)
• Added CHECKPOINT_EQUALS_CHAR attribute to ModelCheckpoint class (#17999)
• Added **summarize_kwargs to ModelSummary and RichModelSummary callbacks (#16788)
• Added support for the max_size_cycle|max_size|min_size iteration modes during evaluation (#17163)
• Added support for the TPU-v4 architecture (#17227)
• Added support for XLA's new PJRT runtime (#17352)
• Check for invalid TPU device inputs (#17227)
• Added XLAStrategy(sync_module_states=bool) to control whether to broadcast the parameters to all devices (#17522)
• Added support for multiple optimizer parameter groups when using the FSDP strategy (#17309)
• Enabled saving the full model state dict when using the FSDPStrategy (#16558)
• Update LightningDataModule.from_datasets to support arbitrary iterables (#17402)
• Run the DDP wrapper in a CUDA stream (#17334)
• Added SaveConfigCallback.save_config to ease use cases such as saving the config to a logger (#17475)
• Enabled optional file versioning of model checkpoints (#17320)
• Added the process group timeout argument FSDPStrategy(timeout=...) for the FSDP strategy (#17274)
• Added FSDPStrategy(activation_checkpointing_policy=...) to customize the layer policy for automatic activation checkpointing (requires torch>=2.1) (#18045)
• Added CLI option --map-to-cpu to the checkpoint upgrade script to enable converting GPU checkpoints on a CPU-only machine (#17527)
• Added non-layer param count to the model summary (#17005)
• Updated LearningRateMonitor to log monitored values to trainer.callback_metrics (#17626)
• Added log_weight_decay argument to LearningRateMonitor callback (#18439)
• Added Trainer.print() to print on local rank zero only (#17980)
• Added Trainer.init_module() context manager to instantiate large models efficiently directly on device, dtype (#18004)
  • Creates the model parameters in the desired dtype (torch.float32, torch.float64) depending on the 'true' precision choice in Trainer(precision='32-true'|'64-true')
• Added the LightningModule.configure_model() hook to instantiate large models efficiently directly on device, dtype, and with sharding support (#18004)
  • Handles initialization for FSDP models before wrapping and the Zero stage 3 initialization for DeepSpeed before sharding
• Added support for meta-device initialization with Trainer.init_module(empty_init=True) in FSDP (#18385)
• Added lightning.pytorch.plugins.PrecisionPlugin.module_init_context() and lightning.pytorch.strategies.Strategy.tensor_init_context() context managers to control model and tensor instantiation (#18004)
• Automatically call xla_model.mark_step() before saving checkpoints with XLA (#17882)
• Added a callback for spike-detection (#18014)
• Added the ability to set the torch.distributed.fsdp.ShardingStrategy via string in FSDPStrategy (#18087)
• Improved error messages when attempting to load a DeepSpeed checkpoint at an invalid path (#17795)
• Allowed accessing rank information in the main process before processes are launched when using the XLAStrategy (#18194)
• Added support for true half-precision training via Trainer(precision="16-true"|"bf16-true") (#18193, #18217, #18213, #18219)
• Added automatic process cleanup to avoid zombie child processes and stalls when exceptions are raised (#18218)
• Added validation of user input for devices and num_nodes when running with SLURM or TorchElastic (#18292)
• Added support for saving checkpoints with either full state-dict or sharded state dict via FSDPStrategy(state_dict_type="full"|"sharded") (#18364)
• Added support for loading sharded/distributed checkpoints in FSDP (#18358)
• Made the text delimiter in the rich progress bar configurable (#18372)
• Improved the error messaging and instructions when handling custom batch samplers in distributed settings (#18402)
• Added support for mixed 8-bit precision as Trainer(precision="transformer-engine") using Nvidia's Transformer Engine (#18459)
• Added support for linear layer quantization with Trainer(plugins=BitsandbytesPrecision()) using bitsandbytes (#18655)
• Added support for passing the process group to the FSDPStrategy (#18583)
• Enabled the default process group configuration for FSDP's hybrid sharding (#18583)
• Added lightning.pytorch.utilities.suggested_max_num_workers to assist with setting a good value in distributed settings (#18591)
• Improved the num_workers warning to give a more accurate upper limit on the num_workers suggestion (#18591)
• Added lightning.pytorch.utilities.is_shared_filesystem utility function to automatically check whether the filesystem is shared between machines (#18586)
• Added support for returning an object of type Mapping from LightningModule.training_step() (#18657)
• Added the hook LightningModule.on_validation_model_zero_grad() to allow overriding the behavior of zeroing the gradients before entering the validation loop (#18710)

</details>

<details><summary>Changed</summary>

• Changed default metric formatting from round(..., 3) to ".3f" format string in MetricsTextColumn class (#18483)
• Removed the limitation to call self.trainer.model.parameters() in LightningModule.configure_optimizers() (#17309)
• Trainer(accelerator="tpu", devices=[i])" now selects the i-th TPU core (0-based, previously it was 1-based) (#17227)
• Allow using iterable-style datasets with TPUs (#17331)
• Increased the minimum XLA requirement to 1.13 (#17368)
• self.loged tensors are now kept in the original device to reduce unnecessary host-to-device synchronizations (#17334)
• Made the run initialization in WandbLogger lazy to avoid creating artifacts when the CLI is used (#17573)
• Simplified redirection of *_step methods in strategies by removing the _LightningModuleWrapperBase wrapper module (#17531)
• Support kwargs input for LayerSummary (#17709)
• Dropped support for wandb versions older than 0.12.0 in WandbLogger (#17876)
• During LightningModule.setup(), the self.device now returns the device the module will be placed on instead of cpu (#18021)
• Increased the minimum supported wandb version for WandbLogger from 0.12.0 to 0.12.10 (#18171)
• The input tensors now get cast to the right precision type before transfer to the device (#18264)
• Improved the formatting of emitted warnings (#18288)
• Broadcast and reduction of tensors with XLA-based strategies now preserve the input's device (#18275)
• The FSDPStrategy now loads checkpoints after the configure_model/configure_sharded_model hook (#18358)
• The FSDPStrategy.load_optimizer_state_dict and FSDPStrategy.load_model_state_dict are a no-op now (#18358)
• The Trainer.num_val_batches, Trainer.num_test_batches and Trainer.num_sanity_val_batches now return a list of sizes per dataloader instead of a single integer (#18441)
• The *_step(dataloader_iter) flavor now no longer takes the batch_idx in the signature (#18390)
• Calling next(dataloader_iter) now returns a triplet (batch, batch_idx, dataloader_idx) (#18390)
• Calling next(combined_loader) now returns a triplet (batch, batch_idx, dataloader_idx) (#18390)
• Due to lack of reliability, Trainer now only runs on one GPU instead of all GPUs in a Jupyter notebook if devices="auto" (default) (#18291)
• Made the batch_idx argument optional in validation_step, test_step and predict_step to maintain consistency with training_step (#18512)
• The TQDMProgressBar now consistently shows it/s for the speed even when the iteration time becomes larger than one second (#18593)
• The LightningDataModule.load_from_checkpoint and LightningModule.load_from_checkpoint methods now raise an error if they are called on an instance instead of the class (#18432)
• Enabled launching via torchrun in a SLURM environment; the TorchElasticEnvironment now gets chosen over the SLURMEnvironment if both are detected (#18618)
• If not set by the user, Lightning will set OMP_NUM_THREADS to num_cpus / num_processes when launching subprocesses (e.g. when DDP is used) to avoid system overload for CPU-intensive tasks (#18677)
• The ModelCheckpoint no longer deletes files under the save-top-k mechanism when resuming from a folder that is not the same as the current checkpoint folder (#18750)
• The ModelCheckpoint no longer deletes the file that was passed to Trainer.fit(ckpt_path=...) (#18750)
• Calling trainer.fit() twice now raises an error with strategies that spawn subprocesses through multiprocessing (ddp_spawn, xla) (#18776)
• The ModelCheckpoint now saves a symbolic link if save_last=True and save_top_k != 0 (#18748)

</details>

<details><summary>Deprecated</summary>

• Deprecated the SingleTPUStrategy (strategy="single_tpu") in favor of SingleDeviceXLAStrategy (strategy="single_xla") (#17383)
• Deprecated the TPUAccelerator in favor of XLAAccelerator (#17383)
• Deprecated the TPUPrecisionPlugin in favor of XLAPrecisionPlugin (#17383)
• Deprecated the TPUBf16PrecisionPlugin in favor of XLABf16PrecisionPlugin (#17383)
• Deprecated the Strategy.post_training_step method (#17531)
• Deprecated the LightningModule.configure_sharded_model hook in favor of LightningModule.configure_model (#18004)
• Deprecated the LightningDoublePrecisionModule wrapper in favor of calling Trainer.precision_plugin.convert_input() (#18209)

</details>

<details><summary>Removed</summary>

• Removed the XLAStrategy.is_distributed property. It is always True (#17381)
• Removed the SingleTPUStrategy.is_distributed property. It is always False (#17381)
• Removed experimental support for torchdistx due to a lack of project maintenance (#17995)
• Removed support for PyTorch 1.11 (#18691)

</details>

<details><summary>Fixed</summary>

• Fixed an issue with reusing the same model across multiple trainer stages when using the DeepSpeedStrategy (#17531)
• Fixed the saving and loading of FSDP optimizer states (#17819)
• Fixed FSDP re-applying activation checkpointing when the user had manually applied it already (#18006)
• Fixed issue where unexpected exceptions would leave the default torch dtype modified when using true precision settings (#18500)
• Fixed issue where not including the batch_idx argument in the training_step would disable gradient accumulation (#18619)
• Fixed the replacement of callbacks returned in LightningModule.configure_callbacks when the callback was a subclass of an existing Trainer callback (#18508)
• Fixed Trainer.log_dir not returning the correct directory for the CSVLogger (#18548)
• Fixed redundant input-type casting in FSDP precision (#18630)
• Fixed numerical issues when reducing values in low precision with self.log (#18686)
• Fixed an issue that would cause the gradients to be erased if validation happened in the middle of a gradient accumulation phase (#18710)
• Fixed redundant file writes in CSVLogger (#18567)
• Fixed an issue that could lead to checkpoint files being deleted accidentally when resuming training (#18750)

</details>

<a name="changelog-fabric"></a>

Lightning Fabric

<details><summary>Added</summary>

• Added support for the TPU-v4 architecture (#17227)
• Added support for XLA's new PJRT runtime (#17352)
• Added support for Fully Sharded Data Parallel (FSDP) training with XLA (#18126, #18424, #18430)
• Check for invalid TPU device inputs (#17227)
• Added XLAStrategy(sync_module_states=bool) to control whether to broadcast the parameters to all devices (#17522)
• Added support for joint setup of model and optimizer with FSDP (#17305)
• Added support for handling multiple parameter groups in optimizers set up with FSDP (#17305)
• Added support for saving and loading sharded model and optimizer state with FSDPStrategy (#17323)
• Added a warning when calling methods on _FabricModule that bypass the strategy-specific wrappers (#17424)
• Added Fabric.init_tensor() context manager to instantiate tensors efficiently directly on device and dtype (#17488)
• Added Fabric.init_module() context manager to instantiate large models efficiently directly on device, dtype, and with sharding support (#17462)
  • Creates the model parameters in the desired dtype (torch.float32, torch.float64, torch.float16, or torch.bfloat16) depending on the 'true' precision choice in Fabric(precision='32-true'|'64-true'|'16-true'|'bf16-true')
  • Handles initialization for FSDP models before wrapping and the Zero stage 3 initialization for DeepSpeed before sharding
• Added support for empty weight initialization with Fabric.init_module(empty_init=True) for checkpoint loading (#17627)
• Added support for meta-device initialization with Fabric.init_module(empty_init=True) in FSDP (#18122)
• Added lightning.fabric.plugins.Precision.module_init_context() and lightning.fabric.strategies.Strategy.module_init_context() context managers to control model and tensor instantiation (#17462)
• lightning.fabric.strategies.Strategy.tensor_init_context() context manager to instantiate tensors efficiently directly on device and dtype (#17607)
• Run the DDP wrapper in a CUDA stream (#17334)
• Added support for true half-precision as Fabric(precision="16-true"|"bf16-true") (#17287)
• Added support for mixed 8-bit precision as Fabric(precision="transformer-engine") using Nvidia's Transformer Engine (#17597)
• Added support for linear layer quantization with Fabric(plugins=BitsandbytesPrecision()) using bitsandbytes (#18655)
• Added error messaging for missed .launch() when it is required (#17570)
• Added support for saving checkpoints with either full state-dict or sharded state dict via FSDPStrategy(state_dict_type="full"|"sharded") (#17526)
• Added support for loading a full-state checkpoint file into a sharded model (#17623)
• Added support for calling hooks on a LightningModule via Fabric.call (#17874)
• Added the parameter Fabric.load(..., strict=True|False) to enable non-strict loading of partial checkpoint state (#17645)
• Added the parameter Fabric.save(..., filter=...) to enable saving a partial checkpoint state (#17845)
• Added support for loading optimizer states from a full-state checkpoint file (#17747)
• Automatically call xla_model.mark_step() before saving checkpoints with XLA (#17882)
• Automatically call xla_model.mark_step() after optimizer.step() with XLA (#17883)
• Added support for all half-precision modes in FSDP precision plugin (#17807)
• Added FSDPStrategy(activation_checkpointing_policy=...) to customize the layer policy for automatic activation checkpointing (requires torch>=2.1) (#18045)
• Added a callback for spike-detection (#18014)
• Added the ability to set the torch.distributed.fsdp.ShardingStrategy via string in FSDPStrategy (#18087)
• Improved error messages when attempting to load a DeepSpeed checkpoint at an invalid path (#17795)
• Added Fabric.load_raw() for loading raw PyTorch state dict checkpoints for model or optimizer objects (#18049)
• Allowed accessing rank information in the main process before processes are launched when using the XLAStrategy (#18194)
• Added automatic process cleanup to avoid zombie child processes and stalls when exceptions are raised (#18218)
• Added validation of user input for devices and num_nodes when running with SLURM or TorchElastic (#18292)
• Improved the error messaging and instructions when handling custom batch samplers in distributed settings (#18402)
• Added support for saving and loading stateful objects other than modules and optimizers (#18513)
• Enabled the default process group configuration for FSDP's hybrid sharding (#18583)
• Added lightning.fabric.utilities.suggested_max_num_workers to assist with setting a good value in distributed settings (#18591)
• Added lightning.fabric.utilities.is_shared_filesystem utility function to automatically check whether the filesystem is shared between machines (#18586)
• Removed support for PyTorch 1.11 (#18691)
• Added support for passing the argument .load_state_dict(..., assign=True|False) on Fabric-wrapped modules in PyTorch 2.1 or newer (#18690)

</details>

<details><summary>Changed</summary>

• Allow using iterable-style datasets with TPUs (#17331)
• Increased the minimum XLA requirement to 1.13 (#17368)
• Fabric argument validation now only raises an error if conflicting settings are set through the CLI (#17679)
• DataLoader re-instantiation is now only performed when a distributed sampler is required (#18191)
• Improved the formatting of emitted warnings (#18288)
• Broadcast and reduction of tensors with XLA-based strategies now preserve the input's device (#18275)
• Due to lack of reliability, Fabric now only runs on one GPU instead of all GPUs in a Jupyter notebook if devices="auto" (default) (#18291)
• Enabled launching via torchrun in a SLURM environment; the TorchElasticEnvironment now gets chosen over the SLURMEnvironment if both are detected (#18618)
• If not set by the user, Lightning will set OMP_NUM_THREADS to num_cpus / num_processes when launching subprocesses (e.g. when DDP is used) to avoid system overload for CPU-intensive tasks (#18677)

</details>

<details><summary>Deprecated</summary>

• Deprecated the DDPStrategy.is_distributed property. This strategy is distributed by definition (#17381)
• Deprecated the SingleTPUStrategy (strategy="single_tpu") in favor of SingleDeviceXLAStrategy (strategy="single_xla") (#17383)
• Deprecated the TPUAccelerator in favor of XLAAccelerator (#17383)
• Deprecated the TPUPrecision in favor of XLAPrecision (#17383)
• Deprecated the TPUBf16Precision in favor of XLABf16Precision (#17383)

</details>

<details><summary>Removed</summary>

• Removed automatic sharding support with Fabric.run or using fabric.launch(fn). This only impacts FSDP and DeepSpeed strategy users. Please instantiate your module under the newly added fabric.init_module context manager (#17832)
• Removed the unsupported checkpoint_io argument from the FSDPStrategy (#18192)

</details>

<details><summary>Fixed</summary>

• Fixed issue where running on TPUs would select the wrong device index (#17227)
• Removed the need to call .launch() when using the DP-strategy (strategy="dp") (#17931)
• Fixed FSDP re-applying activation checkpointing when the user had manually applied it already (#18006)
• Fixed FSDP re-wrapping the module root when the user had manually wrapped the model (#18054)
• Fixed issue where unexpected exceptions would leave the default torch dtype modified when using true precision settings (#18500)
• Fixed redundant input-type casting in FSDP precision (#18630)
• Fixed an issue with find_usable_cuda_devices(0) incorrectly returning a list of devices (#18722)
• Fixed redundant file writes in CSVLogger (#18567)

</details>

<a name="changelog-app"></a>

Lightning App

<details><summary>Added</summary>

• Allow customizing gradio components with lightning colors (#17054)

</details>

<details><summary>Changed</summary>

• Changed LocalSourceCodeDir cache_location to not use home in some certain cases (#17491)

</details>

<details><summary>Removed</summary>

• Remove cluster commands from the CLI (#18151)

</details>

Full commit list: https://github.com/Lightning-AI/lightning/compare/2.0.0...2.1.0

<a name="contributors"></a>

Contributors

Veteran

@adamjstewart @akreuzer @ethanwharris @dmitsf @lantiga @nicolai86 @pl-ghost @carmocca @awaelchli @justusschock @edenlightning @belerico @lightningforever @nisheethlahoti @tchaton @yurijmikhalevich @mauvilsa @rlizzo @rusmux @yhl48 @Liyang90 @jerome-habana @JustinGoheen @Borda @speediedan @SkafteNicki @dcfidalgo

New

@saryazdi @parambharat @kshitij12345 @woqidaideshi @colehawkins @md-121 @gkroiz @idc9 @BoringDonut @OmerShubi @ishandutta0098 @ryan597 @leng-yue @alicanb @One-sixth @santurini @SpirinEgor @KogaiIrina @shanmugamr1992 @janeyx99 @asmith26 @dingusagar @AleksanderWWW @strawberrypie @solyaH @kaczmarj @voidful @water-vapor @bkiat1123 @rhiga2 @baskrahmer @felipewhitaker @mukhery @Quasar-Kim @robieta @one-matrix @jere357 @schmidt-ai @schuhschuh @anio @rjarun8 @callumhay @minhlong94 @klieret @giorgioskij @shihaoyin @JonathanRayner @NripeshN @marcimarc1 @bilelomrani1 @NikolasWolke @0x404 @quintenroets @Borodin @amorehead @SebastianGer @ioangatop @Tribhuvan0 @f0k @sameertantry @kwsp @nik777 @matsumotosan

Did you know?

When Chuck Norris trains a neural network, it not only learns, but it also gains the ability to defend itself from adversarial attacks by roundhouse kicking them into submission.

App

Fixed

• Replace LightningClient with import from lightning_cloud (#18544)

---

Fabric

Fixed

• Fixed an issue causing the _FabricOptimizer.state to remain outdated after loading with load_state_dict (#18488)

---

PyTorch

Fixed

• Fixed an issue that wouldn't prevent the user to set the log_model parameter in WandbLogger via the LightningCLI (#18458)
• Fixed the display of v_num in the progress bar when running with Trainer(fast_dev_run=True) (#18491)
• Fixed UnboundLocalError when running with python -O (#18496)
• Fixed visual glitch with the TQDM progress bar leaving the validation bar incomplete before switching back to the training display (#18503)
• Fixed false positive warning about logging interval when running with Trainer(fast_dev_run=True) (#18550)

---

Contributors

@awaelchli, @borda, @justusschock, @SebastianGer

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Changed

• Change top folder (#18212)
• Remove _handle_is_headless calls in app run loop (#18362)

Fixed

• refactor path to root preventing circular import (#18357)

---

Fabric

Changed

• On XLA, avoid setting the global rank before processes have been launched as this will initialize the PJRT computation client in the main process (#16966)

Fixed

• Fixed model parameters getting shared between processes when running with strategy="ddp_spawn" and accelerator="cpu"; this has a necessary memory impact, as parameters are replicated for each process now (#18238)
• Removed false positive warning when using fabric.no_backward_sync with XLA strategies (#17761)
• Fixed issue where Fabric would not initialize the global rank, world size, and rank-zero-only rank after initialization and before launch (#16966)
• Fixed FSDP full-precision param_dtype training (16-mixed, bf16-mixed and 32-true configurations) to avoid FSDP assertion errors with PyTorch < 2.0 (#18278)

---

PyTorch

Changed

• On XLA, avoid setting the global rank before processes have been launched as this will initialize the PJRT computation client in the main process (#16966)
• Fix inefficiency in rich progress bar (#18369)

Fixed

• Fixed FSDP full-precision param_dtype training (16-mixed and bf16-mixed configurations) to avoid FSDP assertion errors with PyTorch < 2.0 (#18278)
• Fixed an issue that prevented the use of custom logger classes without an experiment property defined (#18093)
• Fixed setting the tracking uri in MLFlowLogger for logging artifacts to the MLFlow server (#18395)
• Fixed redundant iter() call to dataloader when checking dataloading configuration (#18415)
• Fixed model parameters getting shared between processes when running with strategy="ddp_spawn" and accelerator="cpu"; this has a necessary memory impact, as parameters are replicated for each process now (#18238)
• Properly manage fetcher.done with dataloader_iter (#18376)

---

Contributors

@awaelchli, @Borda, @carmocca, @quintenroets, @rlizzo, @speediedan, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Changed

• Removed the top-level import lightning.pdb; import lightning.app.pdb instead (#18177)
• Client retries forever (#18065)

Fixed

• Fixed an issue that would prevent the user to set the multiprocessing start method after importing lightning (#18177)

---

Fabric

Changed

• Disabled the auto-detection of the Kubeflow environment (#18137)

Fixed

• Fixed issue where DDP subprocesses that used Hydra would set hydra's working directory to current directory (#18145)
• Fixed an issue that would prevent the user to set the multiprocessing start method after importing lightning (#18177)
• Fixed an issue with Fabric.all_reduce() not performing an inplace operation for all backends consistently (#18235)

---

PyTorch

Added

• Added LightningOptimizer.refresh() to update the __dict__ in case the optimizer it wraps has changed its internal state (#18280)

Changed

• Disabled the auto-detection of the Kubeflow environment (#18137))

Fixed

• Fixed a Missing folder exception when using a Google Storage URL as a default_root_dir (#18088)
• Fixed an issue that would prevent the user to set the multiprocessing start method after importing lightning (#18177)
• Fixed the gradient unscaling logic if the training step skipped backward (by returning None) (#18267)
• Ensure that the closure running inside the optimizer step has gradients enabled, even if the optimizer step has it disabled (#18268)
• Fixed an issue that could cause the LightningOptimizer wrapper returned by LightningModule.optimizers() have different internal state than the optimizer it wraps (#18280)

---

Contributors

@0x404, @awaelchli, @bilelomrani1, @borda, @ethanwharris, @nisheethlahoti

If we forgot someone due to not matching commit email with GitHub account, let us know :]

2.0.6

App

• Fixed handling a None request in the file orchestration queue (#18111)

---

Fabric

• Fixed TensorBoardLogger.log_graph not unwrapping the _FabricModule (#17844)

---

PyTorch

• LightningCLI not saving correctly seed_everything when run=True and seed_everything=True (#18056)
• Fixed validation of non-PyTorch LR schedulers in manual optimization mode (#18092)
• Fixed an attribute error for _FaultTolerantMode when loading an old checkpoint that pickled the enum (#18094)

---

Contributors

@awaelchli, @lantiga, @mauvilsa, @shihaoyin

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• plugin: store source app (#17892)
• added colocation identifier (#16796)
• Added exponential backoff to HTTPQueue put (#18013)
• Content for plugins (#17243)

Changed

• Save a reference to created tasks, to avoid tasks disappearing (#17946)

---

Fabric

Added

• Added validation against misconfigured device selection when using the DeepSpeed strategy (#17952)

Changed

• Avoid info message when loading 0 entry point callbacks (#17990)

Fixed

• Fixed the emission of a false-positive warning when calling a method on the Fabric-wrapped module that accepts no arguments (#17875)
• Fixed check for FSDP's flat parameters in all parameter groups (#17914)
• Fixed automatic step tracking in Fabric's CSVLogger (#17942)
• Fixed an issue causing the torch.set_float32_matmul_precision info message to show multiple times (#17960)
• Fixed loading model state when Fabric.load() is called after Fabric.setup() (#17997)

---

PyTorch

Fixed

• Fixed delayed creation of experiment metadata and checkpoint/log dir name when using WandbLogger (#17818)
• Fixed incorrect parsing of arguments when augmenting exception messages in DDP (#17948)
• Fixed an issue causing the torch.set_float32_matmul_precision info message to show multiple times (#17960)
• Added missing map_location argument for the LightningDataModule.load_from_checkpoint function (#17950)
• Fix support for neptune-client (#17939)

---

Contributors

@anio, @awaelchli, @borda, @ethanwharris, @lantiga, @nicolai86, @rjarun8, @schmidt-ai, @schuhschuh, @wouterzwerink, @yurijmikhalevich

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Fixed

• bumped several dependencies to address security vulnerabilities.

---

Fabric

Fixed

• Fixed validation of parameters of plugins.precision.MixedPrecision (#17687)
• Fixed an issue with HPU imports leading to performance degradation (#17788)

---

PyTorch

Changed

• Changes to the NeptuneLogger (#16761):
  • It now supports neptune-client 0.16.16 and neptune >=1.0, and we have replaced the log() method with append() and extend().
  • It now accepts a namespace Handler as an alternative to Run for the run argument. This means that you can call it NeptuneLogger(run=run["some/namespace"]) to log everything to the some/namespace/ location of the run.

Fixed

• Fixed validation of parameters of plugins.precision.MixedPrecisionPlugin (#17687)
• Fixed deriving default map location in LightningModule.load_from_checkpoint when there is an extra state (#17812)

---

Contributors

@akreuzer, @awaelchli, @borda, @jerome-habana, @kshitij12345

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• Added the property LightningWork.public_ip that exposes the public IP of the LightningWork instance (#17742)
• Add missing python-multipart dependency (#17244)

Changed

• Made type hints public (#17100)

Fixed

• Fixed LightningWork.internal_ip that was mistakenly exposing the public IP instead; now exposes the private/internal IP address (#17742)
• Fixed resolution of the latest version in CLI (#17351)
• Fixed property raised instead of returned (#17595)
• Fixed get project (#17617, #17666)

---

Fabric

Added

• Added support for Callback registration through entry points (#17756)

Changed

• Made type hints public (#17100)
• Support compiling a module after it was set up by Fabric (#17529)

Fixed

• Fixed computing the next version folder in CSVLogger (#17139)
• Fixed inconsistent settings for FSDP Precision (#17670)

---

PyTorch

Changed

• Made type hints public (#17100)

Fixed

• CombinedLoader only starts DataLoader workers when necessary when operating in sequential mode (#17639)
• Fixed a potential bug with uploading model checkpoints to Neptune.ai by uploading files from stream (#17430)
• Fixed signature inspection of decorated hooks (#17507)
• The WandbLogger no longer flattens dictionaries in the hyperparameters logged to the dashboard (#17574)
• Fixed computing the next version folder in CSVLogger (#17139)
• Fixed a formatting issue when the filename in ModelCheckpoint contained metrics that were substrings of each other (#17610)
• Fixed WandbLogger ignoring the WANDB_PROJECT environment variable (#16222)
• Fixed inconsistent settings for FSDP Precision (#17670)
• Fixed an edge case causing overlapping samples in DDP when no global seed is set (#17713)
• Fallback to module available check for mlflow (#17467)
• Fixed LR finder max val batches (#17636)
• Fixed multithreading checkpoint loading (#17678)

---

Contributors

@adamjstewart, @AleksanderWWW, @awaelchli, @baskrahmer, @bkiat1123, @borda, @carmocca, @ethanwharris, @leng-yue, @lightningforever, @manangoel99, @mukhery, @Quasar-Kim, @water-vapor, @yurijmikhalevich

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Fixed

• Resolved Lightning App with remote storage (#17426)
• Fixed AppState, streamlit example (#17452)

---

Fabric

Changed

• Enable precision autocast for LightningModule step methods in Fabric (#17439)

Fixed

• Fixed an issue with LightningModule.*_step methods bypassing the DDP/FSDP wrapper (#17424)
• Fixed device handling in Fabric.setup() when the model has no parameters (#17441)

---

PyTorch

Fixed

• Fixed Model.load_from_checkpoint("checkpoint.ckpt", map_location=map_location) would always return model on CPU (#17308)
• Fixed Sync module states during non-fit (#17370)
• Fixed an issue that caused num_nodes not to be set correctly for FSDPStrategy (#17438)

---

Contributors

@awaelchli, @borda, @carmocca, @ethanwharris, @ryan597, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Fixed

• Fix frontend hosts when running with multi-process in the cloud (#17324)

---

Fabric

No changes.

---

PyTorch

Fixed

• Make the is_picklable function more robust (#17270)

---

Contributors

@eng-yue @ethanwharris @Borda @awaelchli @carmocca

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

No changes

---

Fabric

Changed

• Generalized Optimizer validation to accommodate both FSDP 1.x and 2.x (#16733)

---

PyTorch

Changed

• Pickling the LightningModule no longer pickles the Trainer (#17133)
• Generalized Optimizer validation to accommodate both FSDP 1.x and 2.x (#16733)
• Disable torch.inference_mode with torch.compile in PyTorch 2.0 (#17215)

Fixed

• Fixed issue where pickling the module instance would fail with a DataLoader error (#17130)
• Fixed WandbLogger not showing "best" aliases for model checkpoints when ModelCheckpoint(save_top_k>0) is used (#17121)
• Fixed the availability check for rich that prevented Lightning to be imported in Google Colab (#17156)
• Fixed parsing the precision config for inference in DeepSpeedStrategy (#16973)
• Fixed issue where torch.compile would fail when logging to WandB (#17216)

---

Contributors

@Borda @williamFalcon @lightningforever @adamjstewart @carmocca @tshu-w @saryazdi @parambharat @awaelchli @colehawkins @woqidaideshi @md-121 @yhl48 @gkroiz @idc9 @speediedan

If we forgot someone due to not matching commit email with GitHub account, let us know :]

Lightning AI is excited to announce the release of Lightning 2.0 :zap:

• Highlights
• Backward Incompatible Changes
  • PyTorch
  • Fabric
• Full Changelog
  • PyTorch
  • Fabric
  • App
• Contributors

Over the last couple of years PyTorch Lightning has become the preferred deep learning framework for researchers and ML developers around the world, with close to 50 million downloads and 18k OSS projects, from top universities to leading labs.

With the help of over 800 contributors, we have added many features and functionalities to make it the most complete research toolkit possible, but some of these changes also introduced issues:

• API changes to the trainer
• Trainer code became harder to follow
• Many integrations made Lightning appear bloated
• The trainer became harder to customize / takes away what I instead need to tweak / have control over.

To make the research experience better, we are introducing 2.0:

• No API changes - We commit to backward compatibility in the 2.0 series
• Simplified abstraction layers, removed legacy functionality, integrations out of the main repo. This improves the project's readability and debugging experience.
• Introducing Fabric. Scale any PyTorch model with just a few lines of code. Read-on!

<a name="highlights"></a>

Highlights

PyTorch 2.0 and torch.compile

Lightning 2.0 is best friends with PyTorch 2.0. You can torch.compile your LightningModules now!

import torch
import lightning as L

model = LitModel()
# This will compile forward and {training,validation,test,predict}_step 
compiled_model = torch.compile(model)

trainer = L.Trainer()
trainer.fit(compiled_model)

PyTorch reports that on average, "models runs 43% faster in training on an NVIDIA A100 GPU. At Float32 precision, it runs 21% faster on average and at AMP Precision it runs 51% faster on average" (source). If you want to learn more about torch.compile and how such speedups can be achieved, read the official PyTorch 2.0 blog post.

Automatic accelerator selection (#16847)

The Trainer now chooses accelerator="auto", strategy="auto", devices="auto" as defaults. This automatically detects the best hardware on your system (TPUs, GPUs, Apple Silicon, etc.) and chooses as many devices as are available.

import lightning as L

# Selects accelerator, devices and strategy automatically!
trainer = L.Trainer()

# Same as:
trainer = L.Trainer(accelerator="auto", strategy="auto", devices="auto")

For example, on a 8-GPU server, this will implicitly select Trainer(accelerator="cuda", strategy="ddp", devices=8).

Support for arbitrary iterables (#16726)

Previously, the Trainer supported DataLoader-like iterables. However, with this release, users can now work with any iterable that implements the Python iterable definition. This includes custom data structures, such as user-defined classes and generators, as well as built-in Python objects.

To use this new feature, return any iterable (or collection of iterables) from the dataloader hooks.

def train_dataloader(self):
    return DataLoader(...)
    return list(range(1000))
    
    # pass loaders as a dict. This will create batches like this:
    # {'a': batch_from_loader_a, 'b': batch_from_loader_b}
    return {"a": DataLoader(...), "b": DataLoader(...)}
    
    # pass loaders as list. This will create batches like this:
    # [batch_from_dl_1, batch_from_dl_2]
    return [DataLoader(...), DataLoader(...)]
    
    # arbitrary nesting
    # {'a': [batch_from_dl_1, batch_from_dl_2], 'b': [batch_from_dl_3, batch_from_dl_4]}
    return {"a": [dl1, dl2], "b": [dl3, dl4]}

Read our data section for more information.

Redesigned multi-dataloader support (#16743, #16784, #16939)

Lightning automatically collates the batches from multiple iterables based on a "mode". This is done with our newly revamped CombinedLoader class.

from lightning.pytorch.utilities import CombinedLoader

iterables = {"a": DataLoader(), "b": DataLoader()}
# Lightning uses this under the hood, but this way you can change the "mode"
combined_loader = CombinedLoader(iterables, mode="min_size")

model = ...
trainer = Trainer()
trainer.fit(model, combined_loader)

The following modes are supported:

• min_size: stops after the shortest iterable (the one with the lowest number of items) is done.
• max_size_cycle: stops after the longest iterable (the one with most items) is done, while cycling through the rest of the iterables.
• max_size: stops after the longest iterable (the one with most items) is done, while returning None for the exhausted iterables.
• sequential: completely consumes ecah iterable sequentially, and returns a triplet (data, idx, iterable_idx)

If you have a need for a different "mode", feel free to open a feature request! Adding new modes is now very simplified. These improvements also allowed us to simplify the trainer's loops by abstracting this logic inside the CombinedLoader.

Barebones Trainer mode (#16854)

A new Trainer argument Trainer(barebones=...) was added (default is False) to disable all features that may impact the raw speed of the training loop. This allows users to quickly and fairily compare the runtime of a Lightning script with a raw PyTorch script.

This is how you enable it:

import lightning as L

# Default: False
trainer = L.Trainer(barebones=True)

A message informs about the changed settings:

You are running in `Trainer(barebones=True)` mode. All features that may impact raw speed have been disabled to facilitate analyzing the Trainer overhead. Specifically, the following features are deactivated:
 - Checkpointing: `Trainer(enable_checkpointing=True)`
 - Progress bar: `Trainer(enable_progress_bar=True)`
 - Model summary: `Trainer(enable_model_summary=True)`
 - Logging: `Trainer(logger=True)`, `Trainer(log_every_n_steps>0)`, `LightningModule.log(...)`, `LightningModule.log_dict(...)`
 - Sanity checking: `Trainer(num_sanity_val_steps>0)`
 - Development run: `Trainer(fast_dev_run=True)`
 - Anomaly detection: `Trainer(detect_anomaly=True)`
 - Profiling: `Trainer(profiler=...)`

Tip: This feature is also very useful for unit testing!

Better progress bar (#16695)

Based on feedback from users, we decided to separate the training progress bar from the validation bar. This greatly improves the time estimates (since validation is usually faster) and resolves confusion around the total batches being processed in an epoch.

This is how the bar looked in versions before 2.0:

Epoch 3:  21%|██        | 28/128 [00:36<01:32, 23.12it/s, loss=0.163]
Validation DataLoader 0:  38%|███      | 12/32 [00:12<00:20,  1.01s/it]

Note how the total batches (128) is the sum of the training batches (32) and the three validation runs (3 x 32). And this is how the progress bar looks like now:

Epoch 3:  50%|█████     | 16/32 [00:36<01:32, 23.12it/s]
Validation DataLoader 0:  38%|███      | 12/32 [00:12<00:20,  1.01s/it]

Note how the batch counts are now separate. The training progress bar pauses until validation is completed.

<a name="highlights-fabric"></a>

Lightning Fabric

Lightning 2.0 is the official release for Lightning Fabric :tada:

<img src="https://pl-public-data.s3.amazonaws.com/assets_lightning/fabric/PyTorch-to-Fabric-Spectrum-2.svg" alt="Fabric spans across a large spectrum - from raw PyTorch all the way to high-level PyTorch Lightning" style="margin-top: 50px; margin-bottom: 50px" />

Fabric is the fast and lightweight way to scale PyTorch models without boilerplate code.

• Easily switch from running on CPU to GPU (Apple Silicon, CUDA, ...), TPU, multi-GPU or even multi-node training
• State-of-the-art distributed training strategies (DDP, FSDP, DeepSpeed) and mixed precision out of the box
• Handles all the boilerplate device logic for you
• Brings useful tools to help you build a trainer (callbacks, logging, checkpoints, ...)
• Designed with multi-billion parameter models in mind

:open_book: Go to Fabric documentation :open_book:

  import torch
  import torch.nn as nn
  from torch.utils.data import DataLoader, Dataset

+ from lightning.fabric import Fabric

  class PyTorchModel(nn.Module):
      ...

  class PyTorchDataset(Dataset):
      ...

+ fabric = Fabric(accelerator="cuda", devices=8, strategy="ddp")
+ fabric.launch()

- device = "cuda" if torch.cuda.is_available() else "cpu"
  model = PyTorchModel(...)
  optimizer = torch.optim.SGD(model.parameters())
+ model, optimizer = fabric.setup(model, optimizer)
  dataloader = DataLoader(PyTorchDataset(...), ...)
+ dataloader = fabric.setup_dataloaders(dataloader)
  model.train()

  for epoch in range(num_epochs):
      for batch in dataloader:
          input, target = batch
-         input, target = input.to(device), target.to(device)
          optimizer.zero_grad()
          output = model(input)
          loss = loss_fn(output, target)
-         loss.backward()
+         fabric.backward(loss)
          optimizer.step()
          lr_scheduler.step()

<a name="bc-changes"></a>

Backward Incompatible Changes

This section outlines notable changes that are not backward compatible with previous versions. The full list of changes and removals can be found in the Full Changelog below.

Since 2.0 is a major release, we took the opportunity to take our APIs to the next level and make considerable changes to reduce the backwards incompatible changes in the future. To alleviate this, we will commit to continue supporting the 1.9.x line of releases by doing bug-fix releases with any important fixes that are necessary.

<a name="bc-changes-pytorch"></a>

PyTorch

The *_epoch_end hooks were removed (#16520)

Since the very beginning of Lightning, the LightningModule offered the convenient *_epoch_end end hooks in which users could reduce metrics collected across the entire epoch to log them. For this to work, Lightning had to store all the outputs returned from the *_step methods internally to be able to send them to the *_epoch_end hook.

This "silent" accumulation of memory lead to many users scratching their head when they found that after training for 20 hours the epoch crashed randomly with an out-of-memory error. This also meant that avoiding this behaviour required code changes, as simply overriding this hook would force this behaviour, regardless of whether you used the outputs. This was exacerbated by users not knowing the difference between the on_train_epoch_end (does not store outputs) and training_epoch_end (does store outputs) hooks.

Based on this feedback, we decided to remove this mechanism completely. Lightning 2.0 favors simplicity and speed over convenience features.

Before:

import lightning as L

class LitModel(L.LightningModule):
    
    def training_step(self, batch, batch_idx):
        ...
        return {"loss": loss, "banana": banana}
    
    # `outputs` is a list of all bananas returned in the epoch
    def training_epoch_end(self, outputs):
        avg_banana = torch.cat(out["banana"] for out in outputs).mean()  

Now:

import lightning as L

class LitModel(L.LightningModule):
    def __init__(self):
        super().__init__()
        # 1. Create a list to hold the outputs of `*_step`
        self.bananas = []
    
    def training_step(self, batch, batch_idx):
        ...
        # 2. Add the outputs to the list
        # You should be aware of the implications on memory usage
        self.bananas.append(banana)
        return loss
    
    # 3. Rename the hook to `on_*_epoch_end`
    def on_train_epoch_end(self):
        # 4. Do something with all outputs
        avg_banana = torch.cat(self.bananas).mean()
        # Don't forget to clear the memory for the next epoch!
        self.bananas.clear()

The new way makes the user to manage their own list of outputs, and with it also the responsibility of managing the memory correctly. If an OOM happens, the user can identify the issue in their own code rather than having to guess what the Trainer does.

The prediction epoch end hook is a special case of this change where the outputs argument has been removed from on_predict_epoch_end(trainer, pl_module) but can still be accessed via the attribute trainer.predict_loop.outputs

You can find a migration guide for this change in this PR's description.

Working with multiple optimizers (#16539)

Lightning 2.0 removed the special optimizer loop that would manage multiple optimizers in the automatic mode in favor of an overall easier to understand and debuggable training loop code. Training with multiple optimizers is now restricted to the "manual optimization mode":

Before:

import lightning as L

class LitModel(L.LightningModule):
    
    def configure_optimizers(self):
        ...
        return optimizer1, optimizer2

    def training_step(self, batch, batch_idx, optimizer_idx)
        if optimizer_idx == 0:
            ...
            return loss0
        if optimizer_idx == 1:
            ...
            return loss1

Now:

import lightning as L

class LitModel(L. LightningModule):
    
    def __init__(self):
        super().__init__()
        # 1. Switch to manual optimization mode
        self.automatic_optimization = False
    
    def configure_optimizers(self):
        ...
        # 2. Return multiple optimizers, same as before
        return optimizer1, optimizer2

    # 3. Remove the `optimizer_idx` argument from `training_step`
    def training_step(self, batch, batch_idx)
        
        # 4. Grab all optimizers you want to work with
        optimizer1, optimizer2 = self.optimizers()
        ...
        
        # 5. Perform backward manually, step optimzers, etc.
        self.manual_backward(loss0)
        optimizer1.step()
        optimizer1.zero_grad()
        
        # 6. In manual optimization, you don't need to return anything
        return None

You may also find the utility methods self.toggle_optimizer() and self.untoggle_optimizer() useful if you need to restrict parameters that require gradients to a specific optimizer. For a complete example, see our simple GAN implementation.

Truncated backpropagation through time (TBPTT) (#16172)

Similar to the multi-optimizer loop mentioned above, truncated backpropagation through time (TBPTT) was a loop that added a lot of complexity to the Trainer. Over time, TBPTT has fallen out of fashion and today the demand from users is so low that we decided to drop special support in the framework in favor of simplifying the Trainer. TBPTT can still be done in manual optimization.

Before:

import lightning as L

class LitModel(L.LightningModule):

    def __init__(self):
        super().__init__()
        self.truncated_bptt_steps = 10
        self.my_rnn = ...
        
    def training_step(self, batch, batch_idx, hiddens):
        ...
        loss, hiddens = self.my_rnn(..., hiddens)
        ...
        return loss, hiddens

Now:

import lightning as L

class LitModel(L.LightningModule):

    def __init__(self):
        super().__init__()
        
         # 1. Switch to manual optimization
        self.automatic_optimization = False
        
        self.truncated_bptt_steps = 10
        self.my_rnn = ...
        
    # 2. Remove the `hiddens` argument
    def training_step(self, batch, batch_idx):
        
        # 3. Split the batch in chunks along the time dimension
        split_batches = split_batch(batch, self.truncated_bptt_steps)
        
        hiddens = ...  # 3. Choose the initial hidden state
        for split_batch in range(split_batches):
            # 4. Perform the optimization in a loop
            loss, hiddens = self.my_rnn(split_batch, hiddens)
            self.backward(loss)
            optimizer.step()
            optimizer.zero_grad()
            
            # 5. "Truncate"
            hiddens = hiddens.detach()
        
        # 6. Remove the return of `hiddens`
        # Returning loss in manual optimization is not needed
        return None

Working with multiple dataloaders (#16800, #16753)

To simplify the Trainer interface and with the goal of simpler iterable support inside the Trainer, we removed theTrainer(multiple_trainloader_mode=...) argument. The mode is now agnostic to the trainer stage ("train" previously) and it's easier to debug and understand for the user as the logic is all encapsulated in the CombinedLoader

Before:

import lightning as L

class LitModel(L.LightningModule):
    def train_dataloader(self):
        return [DataLoader(), DataLoader()]

model = LitModel()
trainer = Trainer(multiple_trainloader_mode="min_size")
trainer.fit(model)

Now:

import lightning as L
from lightning.pytorch.utilities import CombinedLoader

class LitModel(L.LightningModule):
    def train_dataloader(self):
        iterables = [DataLoader(), DataLoader()]
        return CombinedLoader(iterables, mode="min_size")

model = LitModel()
trainer = Trainer()
trainer.fit(model)

Related to this, we cleaned up which hooks need the dataloader_idx as an input argument. Now it's only required if you use multiple dataloaders. Don't worry, the Trainer will automatically check if it's required for you and tell you about it.

Accessing dataloaders (#16726, #16800)

In the case that you require access to the DataLoader or Dataset objects, iterables for each step can be accessed via the trainer properties Trainer.train_dataloader, Trainer.val_dataloaders, Trainer.test_dataloaders, and Trainer.predict_dataloaders.

These properties will match exactly what was returned in your *_dataloader hooks or passed to the Trainer, meaning that if you returned a dictionary of dataloaders, these will return a dictionary of dataloaders. This wasn't the case in before 2.0:

Before:

# Passing 1 dataloader per stage
train_dataset = trainer.train_dataloader.loaders.dataset
val_dataset = trainer.val_dataloaders[0].dataset

# Passing 2 dataloaders per stage
train_dataset = trainer.train_dataloader.loaders[0].dataset
val_dataset = trainer.val_dataloaders[0].dataset

Now:

# Passing 1 dataloader per stage
train_dataset = trainer.train_dataloader.dataset
val_dataset = trainer.val_dataloaders.dataset

# Passing 2 dataloaders per stage
train_dataset = trainer.train_dataloader[0].dataset
val_dataset = trainer.val_dataloaders[0].dataset

The Tuner and Trainer broke up (#16462)

The Tuner and Trainer are no longer together. The two Trainer arguments Trainer(auto_lr_find=..., auto_scale_batch_size=...) and the Trainer.tune() method were removed to make the Trainer leaner and easier to work with.

Before:

import lightning as L

# Tune learning rate
trainer = L.Trainer(auto_lr_find=True)
trainer.tune(model)

# Tune batch size
trainer = L.Trainer(auto_scale_batch_size=True)
trainer.tune(model)

# Fit using tuned settings
trainer.fit(model)

Now:

import lightning as L

# 1. Create the Trainer
trainer = L.Trainer()

# 2. Create the Tuner
tuner = L.pytorch.tuner.Tuner(trainer)

# 3. Tune learning rate
tuner.lr_find(...)

# 4. Tune batch size
tuner.scale_batch_size(...)

# Fit using tuned settings
trainer.fit(model)

You can find more documentation about the tuner here.

Standardized device selection and automation

In Lightning 1.6.0, we simplified the Trainer's signature by collapsing the four accelerator-specific device arguments into a single one called Trainer(devices=...). In 2.0, we are now dropping support for the old Trainer(gpus=..., tpu_cores=..., ipus=..., num_processes=...).

Before:

import lightning as L

# Multiple CPU processes
trainer = L.Trainer(num_processes=2)

# GPU selection
trainer = L.Trainer(gpus=4)

# TPU core selection
trainer = L.Trainer(tpu_cores=[1])

# Graphcore IPU devices
trainer = L.Trainer(ipus=1)

Now:

import lightning as L

# Select devices and accelerator separately
trainer = L.Trainer(accelerator="cpu", devices=2)
trainer = L.Trainer(accelerator="gpu", devices=4)
trainer = L.Trainer(accelerator="tpu", devices=[1])
trainer = L.Trainer(accelerator="ipu", devices=1)

# Or let Lightning detect the accelerator automatically
trainer = L.Trainer(accelerator="auto", devices=2)

# `accelerator="auto"` is the default
trainer = L.Trainer(devices=2)

In addition, the Trainer(auto_select_gpus=...) also got removed (#16184). This was a problematic feature that was not well documented, often misundertood, and lead to DDP stalls due to race conditions in the device selection. It is recommended to use the devices="auto" instead or, if the exact previous behavior is desired, use the utility function.

Before:

import lightning as L

trainer = L.Trainer(auto_select_gpus=True, devices=2)

Now:

import lightning as L

# Recommended
trainer = L.Trainer(devices="auto")

# Alternatively, use the utility function (with care!)
from lightning.pytorch.accelerators import find_usable_cuda_devices

trainer = L.Trainer(devices=find_usable_cuda_devices(2), strategy="ddp_spawn")

The slow and clunky data-parallel strategy (#16748)

PyTorch and Lightning have discouraged the use of the old-style DataParallel (DP) training for more than two years now. This method of multi-GPU training is slow and has many limitations that impact users, even more so in Lightning. Since DP has fallen out of fashion, and other strategies (DDP, DeepSpeed, etc.) have emerged without the same limitations, Lightning is now dropping DP completely.

Before:

import lightning as L

# Enables DP, but with many limitations
trainer = L.Trainer(strategy="dp", devices=8)

Now:

import lightning as L

# DDP is recommended for multi-GPU training
trainer = L.Trainer(strategy="ddp", devices=8)

# If devices > 1, it selects ddp for you
trainer = L.Trainer(devices=8)

# This is the same
trainer = L.Trainer(strategy="auto", devices=8)

Note that the DDP strategy now gets selected automatically when multiple devices are selected. In Jupyter notebooks (Google Colab, Kaggle, etc.), Lightning will automatically select a fork-based DDP strategy (strategy="ddp_notebook").

By extension, the LightningModule no longer has the training_step_end(), validation_step_end() and test_step_end() hooks because they were only used for reducing the outputs in DP. If you have other code that needs to run at the end of the step, you can migrate it to the corresponding *_batch_end hook for example.

Loop customization has a new face

Loop customization was our attempt to make the Trainer more customizable, hackable, and extensible. The modularity it brought had many pros, but it turned out that many users didn't need customization and favored readability and simplicity instead. Hence, the concept of loop customization was completely removed from 2.0.

For users who like build custom training loops, there is now a new paradigm with Lightning Fabric and "Build your own Trainer" (BYOT). Check out the Fabric documentation and the super hackable, 500-lines trainer template.

Mixed precision overhaul (#16783)

Based on feedback, we decided to make the names for precision backends in Trainer(precision=...) clearer and less ambiguous. For example, the previous notation Trainer(precision=16) (which is still allowed to be used) suggested to some users that all of the weights and tensors would be stored in a float16 format, which is not true. To solve this misunderstanding, we now distinguish these modes with "true" and "mixed" suffixes in the names:

Recommended value	Short form
"64-true"	"64", 64
"32-true"	"32", 32
"16-mixed"	"16", 16
"bf16-mixed"	"bf16"

All documentation and examples are now recommending the new, less ambiguous names.

Apex mixed precision gets replaced with AMP (#16149)

In addition to reworking the precision settings, we removed the NVIDIA/Apex integration which got deprecated in 1.9.0. Apex itself has deprecated the mixed precision module and recommends the native torch.amp module in PyTorch. For Lightning, this means you should switch to Trainer(precision="16-mixed").

Before:

import lightning as L

# This required installing nvidia/apex
trainer = L.Trainer(amp_backend="apex", amp_level="O2")

Now:

import lightning as L

# Rely on PyTorch's native mixed precision
trainer = L.Trainer(precision="16-mixed")

Native FSDP replaces Fairscale FSDP (#16400)

With the recent annoucement that FSDP becomes production ready in PyTorch 2.0, we are dropping the support for the experimental Fairscale version of FSDP and go all in on the native implementation instead.

Before:

import lightning as L

# Short-hand names (fairscale)
trainer = L.Trainer(strategy="ddp_sharded" | "ddp_fully_sharded" | "fsdp")

# Shorthand names (native PyTorch)
trainer = L.Trainer(strategy="fsdp_native" | "fsdp_native_full_shard_offload")

# Or using the strategy instance
from lightning.pytorch.strategies import DDPShardedStrategy, DDPFullyShardedStrategy

trainer = L.Trainer(strategy=DDPShardedStrategy(...))
trainer = L.Trainer(strategy=DDPFullyShardedStrategy(...))

Now:

import lightning as L

# Shorthand names (native PyTorch)
trainer = L.Trainer(strategy="fsdp" | "fsdp_cpu_offload")

# Or using the strategy instance
from lightning.pytorch.strategies import FSDPStrategy

trainer = L.Trainer(strategy=FSDPStrategy(...))

Resuming from checkpoints (#16167)

Resuming from a checkpoint is no longer done by specifying the filename in the Trainer constructor. The Trainer(resume_from_checkpoint=...) argument was renamed and moved to the individual Trainer methods.

Before:

import lightning as L

trainer = L.Trainer(resume_from_checkpoint="/path/to/checkpoint.ckpt")
trainer.fit(model)
trainer.test(model)
...

Now:

import lightning as L

# 1. Remove `resume_from_checkpoint` from the Trainer
trainer = L.Trainer()

# 2. Add the path to any of the trainer methods you want to run
trainer.fit(model, ckpt_path="/path/to/checkpoint.ckpt")
trainer.test(model, ckpt_path="/path/to/checkpoint.ckpt")
...

We also added support for setting the checkpoint path statefully:

Now:

import lightning as L

trainer = L.Trainer()

# set the checkpoint path with a setter
trainer.ckpt_path = "/path/to/checkpoint.ckpt"
trainer.fit(model)
# remember to clear it before continuing
trainer.ckpt_path = None
trainer.test()
...

Logging the loss to the progress bar (#16192)

In previous versions of Lightning, the Trainer used to automatically compute a running mean of the training loss and log it to the progress bar. We are removing this feature to favor speed over automation. The loss will no longer appear in the progress bar unless the user explicitly adds it.

Before:

Epoch 8:  53%|█████    | 17/32 [5.13/s, v_num=2, loss=0.5643]

Now:

Epoch 8:  53%|█████    | 17/32 [5.13/s, v_num=2]

def training_step(self, batch, batch_idx):
    ...

    # Add this if you need the loss to be displayed in the progress bar
    self.log("loss", loss, prog_bar=True, on_step=True)
    return loss

The brittle argument parsing utilities (#16708)

In previous versions of Lightning, the Trainer.add_argparse_args and Trainer.from_argparse_args utility functions helped the user construct a commandline interface using Python's argparse library. Over time, new functionality was added to the Trainer that allowed most of the frequently used arguments to accept many different types. This made the arg-parsing utilities very brittle and limited in its scope. Ultimately, we decided to remove it in 2.0 in favor of more robust solutions like LightningCLI or third-party commandline tools (Click, Hydra, etc.).

Before:

import lightning as L
import argparse


parser = argparse.ArgumentParser()
parser = Trainer.add_argparse_args(parser)
args = parser.parse_args()
trainer = L.Trainer.from_argparse_args(args)
...

Now:

Example using the LightningCLI:

from lightning.pytorch.cli import LightningCLI

LightningCLI(MyLitModel, MyLitDataModule)

Run it with python train.py fit --trainer.max_epochs=2 for example. Alternatively, you can add the argparse arguments you want manually:

import lightning as L
import argparse

parser = argparse.ArgumentParser()

# 1. Add the arguments you need
parser.add_argument("--accelerator", type=str, default="cuda")
args = parser.parse_args()

# 2. Pass them into the Trainer
trainer = L.Trainer(**vars(args))
...

Gradient norm tracking (#16745)

Until now, automatic tracking of the gradient norm was possible through the Trainer(track_grad_norm=...) argument. This functionality has now moved to a utility function and can be easily added in two lines of code to the LightningModule.

Before:

import lightning as L

trainer = L.Trainer(track_grad_norm=2)

# Optionally customize logging in LightningModule
def log_grad_norm(self, grad_norm_dict):
    self.log(...)

Now:

from lightning.pytorch.utilities import grad_norm

# 1. Override LightningModule hook
def on_before_optimizer_step(self, optimizer):
    # 2. Inspect the (unscaled) gradients here
    self.log_dict(grad_norm(self, norm_type=2))

This enables users to customize how the gradient norm is computed and logged, without needing to wrangle with the Trainer or override the log_grad_norm hook.

Speeding up DDP with find_unused_parameters (#16611)

When your model doesn't use all parameters in the forward-backward pass, PyTorch's DDP wrapper will freak out and inform you about it with an error. For this reason, and for the longest time, Lightning has set find_unused_parameters=True so that this error can be avoided. However, depending on the model, this can have significant performance impact (slowing down your training). With Lightning 2.0, we switch this flag back to find_unused_parameters=False (default) and favor speed over convenience.

Before:

import lightning as L

# Previously, you had to manually override this for speed
trainer = L.Trainer(strategy="ddp_find_unused_parameters_false")

Now:

import lightning as L

# Now, you get the best speed by default for all ddp variants
trainer = L.Trainer(strategy="ddp")
trainer = L.Trainer(strategy="ddp_spawn")

# We now have these if you need them
trainer = L.Trainer(strategy="ddp_find_unused_parameters_true")

However, it can still happen that users run into problems with unused parametes in their model. Lightning now overrides PyTorch's default error message with a custom one to help users resolve the problem.

Sampler replacement in distributed strategies (#16829)

We renamed the Trainer(replace_sampler_ddp=...) argument to Trainer(use_distributed_sampler=...) to communicate that the sampler gets created not only for the DDP strategies, but all distributed strategies that need it. Its function is still the same as before, and most users don't need to change its default value.

Before:

import lightning as L

# Hmm, I wonder, does this only apply to DDP?
trainer = L.Trainer(replace_sampler_ddp=True)

Now:

import lightning as L

# Aha! Lightning uses a distributed sampler by default, got it!
trainer = L.Trainer(use_distributed_sampler=True)

PyTorch 1.10 and Python 3.7 no longer supported (#16492, #16579)

Whenever there is a new PyTorch version, or a new Python version, it is time to say good bye to the oldest one we support. With the introduction of PyTorch 2.0, we are dropping support for PyTorch 1.10 to continue our support window of the four latest versions: 1.11, 1.12, 1.13 and 2.0. Similarly, with Lightning 2.0 we support the latest three versions of Python: 3.8, 3.9, and 3.10 (3.11 is coming soon).

If you are interested which range of PyTorch or Python versions a particular Lightning version supports, see our compatibility matrix.

Removed experimental fault-tolerance support (#16516, #16533)

To simplify reading and debugging the codebase, we removed the experimental support for fault-tolerance which was under the PL_FAULT_TOLERANT_TRAINING= environment flag. We are looking at ways to re-implement this. If you are interested in this feature, don't hesistate to reach out to us or create a feature request.

Some of the features it included were now ported to stable APIs. One feature is the new callback to save a checkpoint on exception:

Now:

import lightning as L
from lightning.pytorch.callbacks import OnExceptionCheckpoint

on_exception = OnExceptionCheckpoint(".")
trainer = L.Trainer(callbacks=on_exception)

Another feature is automatic SIGTERM handling:

Now:

import lightning as L

trainer = L.Trainer()

if trainer.received_sigterm:
    ...

Removed support for self.log()ing a dictionary (#16389)

Our logging mechanism previously supported log("key", {"something": 123}) (not using log_dict). However, this added significant complexity to the implementation with little benefit, as these keys could not be monitored by our Callbacks and most logger implementations do not support this notation. If you were using this feature with a compatible logger, you can still publish data directly to the Logger using self.logger.log_metrics().

Removed trainer.reset_*_dataloader() methods (#16726)

These methods were not intended for public use as they could leave the Trainer in an undefined state. As a result, we have removed them. To achieve the same functionality, you can use the Trainer(reload_dataloaders_every_n_epochs=...) argument.

Removed the Trainer(move_metrics_to_cpu=True) argument (#16358)

This flag was designed to reduce device memory allocation at the end of an epoch. However, due to design issues, it could leave non-CPU runs in a non-functional state. Since the memory savings were minimal compared to other components and users can still manually control their metrics, we decided to remove this flag.

Separate the Gradient Accumulation Scheduler from Trainer (#16729)

We removed support for passing a scheduling dictionary to Trainer(accumulate_grad_batches=...). The same functionality can still be achieved by simply passing the callback in. This simplifies the Trainer and the overall validation logic.

Before:

trainer = Trainer(accumulate_grad_batches={"1": 5, "10": 3})

Now:

from lightning.pytorch.callbacks import GradientAccumulationScheduler
trainer = Trainer(callbacks=GradientAccumulationScheduler({"1": 5, "10": 3}))

<a name="bc-changes-fabric"></a>

Fabric

LightningLite is dead, long live Lightning Fabric!

Over time, LightningLite has evolved from a simple onboarding tool to a powerful toolbox enabling users to build performant and hackable trainers. In 1.9.0, we renamed it to Lightning Fabric and gave users early access to its new features. In 2.0, we are dropping LightningLite from the package completely.

Before:

# There were two different ways of importing Lite in <= 1.9.0
from pytorch_lighting.lite import LightningLite
from lightning.lite import LightningLite

# You had to subclass LightningLite and implement `run()`
class MyTrainer(LightningLite):
    
    def run(self):
        ...
        self.backward(loss)
        ...

lite = LightningLite(...)
lite.run()

Now:

# 1. Import Fabric directly from the lightning package
import lightning as L

# 2. Instantiate Fabric directly, without subclassing
fabric = L.Fabric(...)

# 3. Use it in your training loop
fabric.backward(loss)
...

Learn more about Fabric and what it can do in the new docs!

<a name="changelog"></a>

CHANGELOG

<a name="changelog-pytorch"></a>

PyTorch

<details><summary>Added</summary>

• Added migration logic to warn about checkpoints with apex AMP state (#16161)
• Added the Trainer.ckpt_path = ... setter to statefully set the checkpoint path to load. This can act as a replacement for the removed Trainer(resume_from_checkpoint=...) flag (#16187)
• Added an argument include_cuda in pytorch_lightning.utilities.seed.isolate_rng to disable managing torch.cuda's rng (#16423)
• Added Tuner.lr_find(attr_name=...) to specify custom learning rate attribute names (#16462)
• Added an OnExceptionCheckpoint callback to save a checkpoint on exception (#16512)
• Added support for running the MLFlowLogger with the mlflow-skinny package (16513)
• Added a Trainer.received_sigterm property to check whether a SIGTERM signal was received (#16501)
• Added support for cascading a SIGTERM signal to launched processes after the launching process (rank 0) receives it (#16525)
• Added a kill method to launchers to kill all launched processes (#16525)
• Added suffix option to DDP strategy names to enable find_unused_parameters=True, for example strategy="ddp_find_unused_parameters_true" (#16611)
• Added a new method Strategy.on_exception to the strategy base interface (#16646)
• Added support for predict_step(dataloader_iter, batch_index) (#16726)
• Added support for arbitrary iterables as dataloaders (#16726)
• Added "sequential" mode support to CombinedLoader to consume multiple iterables in sequence (#16743, #16784)
• Added "max_size" mode support to CombinedLoader to consume multiple iterables entirely without cycling (#16939
• Added a Trainer(barebones=True) argument where all features that may impact raw speed are disabled (#16854)
• Added support for writing logs remote file systems on CSVLoggers. (#16880)
• Added DDPStrategy(start_method=...) argument, defaulting to 'popen' (#16809)
• Added checks for whether the iterables used by the loops are valid (#17007)

</details>

<details><summary>Changed</summary>

• The Trainer's signal handlers are now registered for trainer.{validate,test,predict} (#17017)
• Renamed ProgressBarBase to ProgressBar (#17058)
• The Trainer now chooses accelerator="auto", strategy="auto", devices="auto" as defaults (#16847)
• "Native" suffix removal (#16490)

• strategy="fsdp_native" is now strategy="fsdp"
• strategy="fsdp_native_full_shard_offload" is now strategy="fsdp_cpu_offload"
• pytorch_lightning.strategies.fully_sharded_native.DDPFullyShardedNativeStrategy is now pytorch_lightning.strategies.fsdp.FSDPStrategy
• pytorch_lightning.plugins.precision.fsdp_native_native_amp.FullyShardedNativeNativeMixedPrecisionPlugin is now pytorch_lightning.plugins.precision.fsdp.FSDPMixedPrecisionPlugin
• pytorch_lightning.plugins.precision.native_amp is now pytorch_lightning.plugins.precision.amp
• NativeSyncBatchNorm is now TorchSyncBatchNorm

• Changed the default of LearningRateFinder(update_attr=...) and Tuner.lr_find(update_attr=...) to True (#16462)
• Renamed the pl.utilities.exceptions.GracefulExitException to SIGTERMException (#16501)
• The Callback.on_train_epoch_end hook now runs after the LightningModule.on_train_epoch_end hook for instances of EarlyStopping and Checkpoint callbacks (#16567)
• The LightningModule.{un}toggle_optimizer methods no longer accept a optimizer_idx argument to select the relevant optimizer. Instead, the optimizer object can be passed in directly (#16560)
• Manual optimization is now required for working with multiple optimizers (#16539)
• DDP's find_unused_parameters now defaults to False (#16611)
• The strategy selected by accelerator="hpu" now defaults to find_unused_parameters=False (#16611)
• The main progress bar displayed during training no longer includes the combined progress for validation (#16695)
• Renamed TQDMProgressBar.main_progress_bar to TQDMProgressBar.train_progress_bar (#16695)
• Marked the progress tracking classes as protected (#17009)
• Marked the lightning.pytorch.trainer.configuration_validator.verify_loop_configurations function as protected (#17009)
• Marked the lightning.pytorch.utiltiies.distirbuted.register_ddp_comm_hook function as protected (#17009)
• Marked lightning.pytorch.utilities.supporters.CombinedDataset as protected (#16714)
• Marked the {Accelerator,Signal,Callback,Checkpoint,Data,Logger}Connector classes as protected (#17008)
• Marked the lightning.pytorch.trainer.connectors.signal_connector.HandlersCompose class as protected (#17008)
• Disabled strict loading in multiprocessing launcher ("ddp_spawn", etc.) when loading weights back into the main process (#16365)
• Renamed CombinedLoader.loaders to CombinedLoader.iterables (#16743)
• Renamed Trainer(replace_sampler_ddp=...) to Trainer(use_distributed_sampler=...) (#16829)
• Moved the CombinedLoader class from lightning.pytorch.trainer.supporters to lightning.pytorch.combined_loader (#16819)
• The top-level loops now own the data sources and combined dataloaders (#16726)
• The trainer.*_dataloader properties now return what the user returned in their LightningModule.*_dataloader() hook (#16726, #16800)
• The dataloader_idx argument is now optional for the on_{validation,test,predict}_batch_{start,end} hooks. Remove it or default it to 0 if you don't use multiple dataloaders (#16753)
• Renamed TPUSpawnStrategy to XLAStrategy (#16781)
• Renamed strategy='tpu_spawn' to strategy='xla' and strategy='tpu_spawn_debug' to strategy='xla_debug' (#16781)
• Changed arguments for precision settings (from [64|32|16|bf16] to ["64-true"|"32-true"|"16-mixed"|"bf16-mixed"]) (#16783)
• When using multiple devices, the strategy now defaults to "ddp" instead of "ddp_spawn" when none is set (#16780)
• The selection Trainer(strategy="ddp_spawn", ...) no longer falls back to "ddp" when a cluster environment gets detected (#16780)
• Predict's custom BatchSampler that tracks the batch indices no longer consumes the entire batch sampler at the beginning (#16826)
• Gradient norm tracking with track_grad_norm no longer rounds the norms to 4 digits, but instead logs them at full resolution (#16877)
• Merged the DDPSpawnStrategy into DDPStrategy (#16809)
• The NeptuneLogger now requires neptune>=1.0.0 (#16888)
• Changed minimum supported version of rich from 10.14.0 to 12.13.0 (#16798)
• Removed the lightning.pytorch.overrides.torch_distributed.broadcast_object_list function (#17011)
• The ServableModule is now an abstract interface (#17000)
• The psutil package is now required for CPU monitoring (#17010)
• The Trainer no longer accepts positional arguments to (#17022)

</details>

<details><summary>Removed</summary>

• Removed support for PyTorch 1.10 (#16492)
• Removed support for Python 3.7 (#16579)
• Removed the pytorch_lightning.lite module in favor of lightning_fabric (#15953)
• nvidia/apex removal (#16149)
  • Removed pytorch_lightning.plugins.NativeMixedPrecisionPlugin in favor of pytorch_lightning.plugins.MixedPrecisionPlugin
  • Removed the LightningModule.optimizer_step(using_native_amp=...) argument
  • Removed the Trainer(amp_backend=...) argument
  • Removed the Trainer.amp_backend property
  • Removed the Trainer(amp_level=...) argument
  • Removed the pytorch_lightning.plugins.ApexMixedPrecisionPlugin class
  • Removed the pytorch_lightning.utilities.enums.AMPType enum
  • Removed the DeepSpeedPrecisionPlugin(amp_type=..., amp_level=...) arguments
• Removed Trainer(strategy='horovod') support (#16150)
• FairScale removal (in favor of PyTorch's FSDP implementation) (#16400)
  • Removed the pytorch_lightning.overrides.fairscale.LightningShardedDataParallel class
  • Removed the pytorch_lightning.plugins.precision.fully_sharded_native_amp.FullyShardedNativeMixedPrecisionPlugin class
  • Removed the pytorch_lightning.plugins.precision.sharded_native_amp.ShardedNativeMixedPrecisionPlugin class
  • Removed the pytorch_lightning.strategies.fully_sharded.DDPFullyShardedStrategy (fsdp) class
  • Removed the pytorch_lightning.strategies.sharded.DDPShardedStrategy (ddp_sharded) class
  • Removed the pytorch_lightning.strategies.sharded_spawn.DDPSpawnShardedStrategy (ddp_sharded_spawn) class
• Removed legacy device arguments in Trainer (#16171)
  • Removed the Trainer(gpus=...) argument
  • Removed the Trainer(tpu_cores=...) argument
  • Removed the Trainer(ipus=...) argument
  • Removed the Trainer(num_processes=...) argument
• Removed the deprecated pytorch_lightning.utilities.AllGatherGrad class (#16360)
• Removed the deprecated resume_from_checkpoint Trainer argument (#16167)
• Removed the deprecated pytorch_lightning.profiler module (#16359)
• Removed deadlock detection / process reconciliation (PL_RECONCILE_PROCESS=1) (#16204)
• Removed the {training,validation,test}_epoch_end hooks which would retain step outputs in memory. Alternative implementations are suggested by implementing their on_*_epoch_end hooks instead (#16520)
• Removed the outputs argument from the on_predict_epoch_end hook. You can access them via trainer.predict_loop.predictions (#16655)
• Removed support for the experimental PL_FAULT_TOLERANT_TRAINING environment flag (#16516, #16533)
• Removed the deprecated LightningCLI arguments (#16380)
  • save_config_filename
  • save_config_overwrite
  • save_config_multifile
  • description
  • env_prefix
  • env_parse
• Removed the deprecated pl.strategies.utils.on_colab_kaggle function (#16437)
• Removed the deprecated code in:
  • pl.core.mixins (#16424)
  • pl.utilities.distributed (#16390)
  • pl.utilities.apply_func (#16413)
  • pl.utilities.xla_device (#16404)
  • pl.utilities.data (#16440)
  • pl.utilities.device_parser (#16412)
  • pl.utilities.optimizer (#16439)
  • pl.utilities.seed (#16422)
  • pl.utilities.cloud_io (#16438)
• Removed the deprecated Accelerator.setup_environment method (#16436)
• Mark the forward_module argument as required (#16386)
  • Removed the deprecated pl_module argument from the distributed module wrappers
  • Removed the deprecated pytorch_lightning.overrides.base.unwrap_lightning_module function
  • Removed the pytorch_lightning.overrides.distributed.LightningDistributedModule class
  • Removed the deprecated pytorch_lightning.overrides.fairscale.unwrap_lightning_module_sharded function
  • Removed the pytorch_lightning.overrides.fairscale.LightningDistributedModule class
• Removed the deprecated automatic GPU selection (#16184)
  • Removed the Trainer(auto_select_gpus=...) argument
  • Removed the pytorch_lightning.tuner.auto_gpu_select.{pick_single_gpu,pick_multiple_gpus} functions
• Removed support for loop customization
  • Removed Loop.replace() (#16361)
  • Removed Loop.connect() (#16384)
  • Removed the trainer.{fit,validate,test,predict}_loop properties (#16384)
  • Removed the default Loop.run() implementation (#16384)
  • The loop classes are now marked as protected (#16445)
  • The fetching classes are now marked as protected (#16664)
• The lightning.pytorch.overrides.distributed.IndexBatchSamplerWrapper class is now marked as protected (#16826)
• Removed the DataLoaderLoop, EvaluationEpochLoop, and PredictionEpochLoop classes (#16726)
• Removed trainer.reset_*_dataloader() methods in favor of Loop.setup_data() for the top-level loops (#16726)
• Removed special support for truncated backpropagation through time (TBPTT) (#16172)
  • Removed the LightningModule.truncated_bptt_steps attribute
  • Removed the LightningModule.tbptt_split_batch hook
  • The LightningModule.training_step no longer accepts a hiddens argument
  • Removed the pytorch_lightning.loops.batch.TrainingBatchLoop
  • Removed the FitLoop.split_idx property
  • Removed the LoggerConnector.on_train_split_start method
• Removed the experimental PL_INTER_BATCH_PARALLELISM environment flag (#16355)
• Removed the Trainer(move_metrics_to_cpu=True) argument (#16358)
• Removed the LightningModule.precision attribute (#16203)
• Removed the automatic addition of a moving average of the training_step loss in the progress bar. Use self.log("loss", ..., prog_bar=True) instead. (#16192)
• Removed support for passing a dictionary value to self.log() (#16389)
• Removed Trainer.model setter (#16462)
• Removed the argument Trainer(multiple_trainloader_mode=...). You can use CombinedLoader(..., mode=...) directly now (#16800)
• Removed the unused lightning.pytorch.utilities.finite_checks.print_nan_gradients function (#16682)
• Removed the unused lightning.pytorch.utilities.finite_checks.detect_nan_parameters function (#16682)
• Removed the unused lightning.pytorch.utilities.parsing.flatten_dict function (#16744)
• Removed the unused lightning.pytorch.utilities.metrics.metrics_to_scalars function (#16681)
• Removed the unused lightning.pytorch.utilities.supporters.{SharedCycleIteratorState,CombinedLoaderIterator} classes (#16714)
• Tuner removal
  • Removed the deprecated trainer.tuning property (#16379)
  • Removed the deprecated TrainerFn.TUNING and RunningStage.TUNING enums (#16379)
  • Removed Trainer.tune() in favor of Tuner(trainer).{lr_find,scale_batch_size} (#16462)
  • Removed Trainer(auto_scale_batch_size=...) in favor of Tuner(trainer).scale_batch_size() (#16462)
  • Removed Trainer(auto_lr_find=...) in favor of Tuner(trainer).lr_find() (#16462)
• Removed the on_tpu argument from LightningModule.optimizer_step hook (#16537)
• Removed the using_lbfgs argument from LightningModule.optimizer_step hook (#16538)
• Removed the Trainer.data_parallel property. Use isinstance(trainer.strategy, ParallelStrategy) instead (#16703)
• Removed the Trainer.prediction_writer_callbacks property (#16759)
• Removed support for multiple optimizers in automatic optimization mode (#16539)
  • Removed opt_idx argument from BaseFinetuning.finetune_function callback method
  • Removed opt_idx argument from Callback.on_before_optimizer_step callback method
  • Removed optimizer_idx as an optional argument in LightningModule.training_step
  • Removed optimizer_idx argument from LightningModule.on_before_optimizer_step
  • Removed optimizer_idx argument from LightningModule.configure_gradient_clipping
  • Removed optimizer_idx argument from LightningModule.optimizer_step
  • Removed optimizer_idx argument from LightningModule.optimizer_zero_grad
  • Removed optimizer_idx argument from LightningModule.lr_scheduler_step
  • Removed support for declaring optimizer frequencies in the dictionary returned from LightningModule.configure_optimizers
  • Removed arguments optimizer and optimizer_idx from LightningModule.backward
  • Removed optimizer_idx argument from PrecisionPlugin.optimizer_step and all of its overrides in subclasses
  • Removed optimizer_idx argument from PrecisionPlugin.{optimizer_step,backward} and all of its overrides in subclasses
  • Removed optimizer_idx argument from Strategy.{optimizer_step,backward} and all of its overrides in subclasses
  • Removed Trainer.optimizer_frequencies attribute
• Removed Strategy.dispatch (#16618)
• Removed PrecisionPlugin.dispatch (#16618)
• Removed legacy argparse utilities (#16708)
  • Removed LightningDataModule methods: add_argparse_args(), from_argparse_args(), parse_argparser(), get_init_arguments_and_types()
  • Removed class methods from Trainer: default_attributes(), from_argparse_args(), parse_argparser(), match_env_arguments(), add_argparse_args()
  • Removed functions from lightning.pytorch.utilities.argparse: from_argparse_args(), parse_argparser(), parse_env_variables(), get_init_arguments_and_types(), add_argparse_args()
  • Removed functions from lightning.pytorch.utilities.parsing: import str_to_bool(), str_to_bool_or_int(), str_to_bool_or_str()
• Removed support for passing a scheduling dictionary to Trainer(accumulate_grad_batches=...) (#16729)
• Removed support for DataParallel (strategy='dp') and the LightningParallelModule-Wrapper, (#16748)
• Removed the unused lightning.pytorch.utilities.supporters.{SharedCycleIteratorState,CombinedLoaderIterator} classes (#16714)
• Removed ProgressBarBase.{train_batch_idx,val_batch_idx,test_batch_idx,predict_batch_idx} properties (#16760)
• Removed the fit_loop.{min,max}_steps setters (#16803)
• Removed the Trainer(track_grad_norm=...) argument (#16745)
• Removed the LightningModule.log_grad_norm() hook method (#16745)
• Removed the QuantizationAwareTraining callback (#16750)
• Removed the ColossalAIStrategy and ColossalAIPrecisionPlugin in favor of the new lightning-colossalai package (#16757, #16778)
• Removed the training_step_end, validation_step_end, and test_step_end hooks from the LightningModule in favor of the *_batch_end hooks (#16791)
• Removed the lightning.pytorch.strategies.DDPSpawnStrategy in favor of DDPStrategy(start_method='spawn') (merged both classes) (#16809)
• Removed registration of ShardedTensor state dict hooks in LightningModule.__init__ with torch>=2.1 (#16892)
• Removed the lightning.pytorch.core.saving.ModelIO class interface (#16999)
• Removed the unused lightning.pytorch.utilities.memory.get_model_size_mb function (#17001)

</details>

<details><summary>Fixed</summary>

• Fixed an issue where DistributedSampler.set_epoch wasn't getting called during trainer.predict (#16785, #16826)
• Fixed an issue with comparing torch versions when using a version of torch built from source (#17030)
• Improved the error message for installing tensorboard or tensorboardx (#17053)

</details>

<a name="changelog-fabric"></a>

Fabric

<details><summary>Added</summary>

• Added Fabric.all_reduce (#16459)
• Added support for saving and loading DeepSpeed checkpoints through Fabric.save/load() (#16452)
• Added support for automatically calling set_epoch on the dataloader.batch_sampler.sampler (#16841)
• Added support for writing logs to remote file systems with the CSVLogger (#16880)
• Added support for frozen dataclasses in the optimizer state (#16656)
• Added lightning.fabric.is_wrapped to check whether a module, optimizer, or dataloader was already wrapped by Fabric (#16953)

</details>

<details><summary>Changed</summary>

• Fabric now chooses accelerator="auto", strategy="auto", devices="auto" as defaults (#16842)
• Checkpoint saving and loading redesign (#16434)
  • Changed the method signatrue of Fabric.save and Fabric.load
  • Changed the method signature of Strategy.save_checkpoint and Fabric.load_checkpoint
  • Fabric.save accepts a state that can contain model and optimizer references
  • Fabric.load can now load state in-place onto models and optimizers
  • Fabric.load returns a dictionary of objects that weren't loaded into the state
  • Strategy.save_checkpoint and Fabric.load_checkpoint are now responsible for accessing the state of the model and optimizers
• DataParallelStrategy.get_module_state_dict() and DDPStrategy.get_module_state_dict() now correctly extracts the state dict without keys prefixed with 'module' (#16487)
• "Native" suffix removal (#16490)
  • strategy="fsdp_full_shard_offload" is now strategy="fsdp_cpu_offload"
  • lightning.fabric.plugins.precision.native_amp is now lightning.fabric.plugins.precision.amp
• Enabled all shorthand strategy names that can be supported in the CLI (#16485)
• Renamed strategy='tpu_spawn' to strategy='xla' and strategy='tpu_spawn_debug' to strategy='xla_debug' (#16781)
• Changed arguments for precision settings (from [64|32|16|bf16] to ["64-true"|"32-true"|"16-mixed"|"bf16-mixed"]) (#16767)
• The selection Fabric(strategy="ddp_spawn", ...) no longer falls back to "ddp" when a cluster environment gets detected (#16780)
• Renamed setup_dataloaders(replace_sampler=...) to setup_dataloaders(use_distributed_sampler=...) (#16829)

</details>

<details><summary>Removed</summary>

• Removed support for PyTorch 1.10 (#16492)
• Removed support for Python 3.7 (#16579)

</details>

<details><summary>Fixed</summary>

• Fixed issue where the wrapped dataloader iter() would be called twice (#16841)
• Improved the error message for installing tensorboard or tensorboardx (#17053)

</details>

<a name="changelog-app"></a>

App

<details><summary>Added</summary>

• Added --zip option to the lightning cp command to copy content from the Cloud Platform Filesystem as a zipfile

</details>

<details><summary>Changed</summary>

• Changed minimum supported version of rich from 10.14.0 to 12.13.0 (#16798)

</details>

<details><summary>Removed</summary>

• Removed support for Python 3.7 (#16579)

</details>

Full commit list: https://github.com/Lightning-AI/lightning/compare/1.9.0...2.0.0

<a name="contributors"></a>

Contributors

Veteran

@aniketmaurya @Atharva-Phatak @awaelchli @Borda @carmocca @dmitsf @edenlightning @hhsecond @janpawlowskiof @justusschock @krshrimali @leoleoasd @martenlienen @narJH27 @rusmux @SauravMaheshkar @shenoynikhil @tshu-w @wouterzwerink

New

@coreyjadams @dconathan @RuRo @sergeevii123 @BrianPulfer @akkefa @belerico @tupini07 @jihoonkim2100 @eamonn-zh @lightningforever @mauvilsa @muhammadanas0716 @AdityaKane2001 @dtuit @themattinthehatt @janpawlowskiof @rusmux @Erotemic @janEbert

If we forgot somebody or you have a suggestion, find us on Discord :zap:

Did you know?

Chuck Norris jokes aren't actually jokes, they are the truth. Chuck Norris actually can do all the things people say he can.

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.9.0...2.0.0rc0

App

Changed

• Added healthz endpoint to plugin server (#16882)
• System customization syncing for jobs run (#16932)

---

Fabric

Changed

• Let TorchCollective works on the torch.distributed WORLD process group by default (#16995)

Fixed

• fixed for all _cuda_clearCublasWorkspaces on teardown (#16907)
• Improved the error message for installing tensorboard or tensorboardx (#17053)

---

PyTorch

Changed

• Changed to the NeptuneLogger (#16761):
  • It now supports neptune-client 0.16.16 and neptune >=1.0, and we have replaced the log() method with append() and extend().
  • It now accepts a namespace Handler as an alternative to Run for the run argument. This means that you can call it like NeptuneLogger(run=run["some/namespace"]) to log everything to the some/namespace/ location of the run.
• Allow sys.argv and args in LightningCLI (#16808)
• Moveed HPU broadcast override to the HPU strategy file (#17011)

Depercated

• Removed registration of ShardedTensor state dict hooks in LightningModule.__init__ with torch>=2.1 (#16892)
• Removed the lightning.pytorch.core.saving.ModelIO class interface (#16974)

Fixed

• Fixed num_nodes not being set for DDPFullyShardedNativeStrategy (#17160)
• Fixed parsing the precision config for inference in DeepSpeedStrategy (#16973)
• Fixed the availability check for rich that prevented Lightning to be imported in Google Colab (#17156)
• Fixed for all _cuda_clearCublasWorkspaces on teardown (#16907)
• The psutil package is now required for CPU monitoring (#17010)
• Improved the error message for installing tensorboard or tensorboardx (#17053)

---

Contributors

@awaelchli, @belerico, @carmocca, @colehawkins, @dmitsf, @Erotemic, @ethanwharris, @kshitij12345, @borda

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Removed

• Removed implicit ui testing with testing.run_app_in_cloud in favor of headless login and app selection (#16741)

---

Fabric

Added

• Added Fabric(strategy="auto") support (#16916)

Fixed

• Fixed edge cases in parsing device ids using NVML (#16795)
• Fixed DDP spawn hang on TPU Pods (#16844)
• Fixed an error when passing find_usable_cuda_devices(num_devices=-1) (#16866)

---

PyTorch

Added

• Added Fabric(strategy="auto") support. It will choose DDP over DDP-spawn, contrary to strategy=None (default) (#16916)

Fixed

• Fixed DDP spawn hang on TPU Pods (#16844)
• Fixed edge cases in parsing device ids using NVML (#16795)
• Fixed backwards compatibility for lightning.pytorch.utilities.parsing.get_init_args (#16851)

---

Contributors

@ethanwharris, @carmocca, @awaelchli, @justusschock , @dtuit, @Liyang90

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Fixed

• Fixed lightning open command and improved redirects (#16794)

---

Fabric

Fixed

• Fixed an issue causing a wrong environment plugin to be selected when accelerator=tpu and devices > 1 (#16806)
• Fixed parsing of defaults for --accelerator and --precision in Fabric CLI when accelerator and precision are set to non-default values in the code (#16818)

---

PyTorch

Fixed

• Fixed an issue causing a wrong environment plugin to be selected when accelerator=tpu and devices > 1 (#16806)

---

Contributors

@ethanwharris, @carmocca, @awaelchli, @borda, @tchaton, @yurijmikhalevich

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• Added Storage Commands (#16740)
  • rm: Delete files from your Cloud Platform Filesystem
• Added lightning connect data to register data connection to private s3 buckets (#16738)

---

Fabric

Fixed

• Fixed an attribute error and improved input validation for invalid strategy types being passed to Fabric (#16693)

---

PyTorch

Changed

• Disabled strict loading in multiprocessing launcher ("ddp_spawn", etc.) when loading weights back into the main process (#16365)

Fixed

• Fixed an attribute error and improved input validation for invalid strategy types being passed to Trainer (#16693)
• Fixed early stopping triggering extra validation runs after reaching min_epochs or min_steps (#16719)

---

Contributors

@akihironitta, @awaelchli, @borda, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• Added lightning open command (#16482)
• Added experimental support for interruptable GPU in the cloud (#16399)
• Added FileSystem abstraction to simply manipulate files (#16581)
• Added Storage Commands (#16606)
  • ls: List files from your Cloud Platform Filesystem
  • cd: Change the current directory within your Cloud Platform filesystem (terminal session based)
  • pwd: Return the current folder in your Cloud Platform Filesystem
  • cp: Copy files between your Cloud Platform Filesystem and local filesystem
• Prevent to cd into non-existent folders (#16645)
• Enabled cp (upload) at project level (#16631)
• Enabled ls and cp (download) at project level (#16622)
• Added lightning connect data to register data connection to s3 buckets (#16670)
• Added support for running with multiprocessing in the cloud (#16624)
• Initial plugin server (#16523)
• Connect and Disconnect node (#16700)

Changed

• Changed the default LightningClient(retry=False) to retry=True (#16382)
• Add support for async predict method in PythonServer and remove torch context (#16453)
• Renamed lightning.app.components.LiteMultiNode to lightning.app.components.FabricMultiNode (#16505)
• Changed the command lightning connect to lightning connect app for consistency (#16670)
• Refactor cloud dispatch and update to new API (#16456)
• Updated app URLs to the latest format (#16568)

Fixed

• Fixed a deadlock causing apps not to exit properly when running locally (#16623)
• Fixed the Drive root_folder not parsed properly (#16454)
• Fixed malformed path when downloading files using lightning cp (#16626)
• Fixed app name in URL (#16575)

---

Fabric

Fixed

• Fixed error handling for accelerator="mps" and ddp strategy pairing (#16455)
• Fixed strict availability check for torch_xla requirement (#16476)
• Fixed an issue where PL would wrap DataLoaders with XLA's MpDeviceLoader more than once (#16571)
• Fixed the batch_sampler reference for DataLoaders wrapped with XLA's MpDeviceLoader (#16571)
• Fixed an import error when torch.distributed is not available (#16658)

---

Pytorch

Fixed

• Fixed an unintended limitation for calling save_hyperparameters on mixin classes that don't subclass LightningModule/LightningDataModule (#16369)
• Fixed an issue with MLFlowLogger logging the wrong keys with .log_hyperparams() (#16418)
• Fixed logging more than 100 parameters with MLFlowLogger and long values are truncated (#16451)
• Fixed strict availability check for torch_xla requirement (#16476)
• Fixed an issue where PL would wrap DataLoaders with XLA's MpDeviceLoader more than once (#16571)
• Fixed the batch_sampler reference for DataLoaders wrapped with XLA's MpDeviceLoader (#16571)
• Fixed an import error when torch.distributed is not available (#16658)

---

Contributors

@akihironitta, @awaelchli, @borda, @BrianPulfer, @ethanwharris, @hhsecond, @justusschock, @Liyang90, @RuRo, @senarvi, @shenoynikhil, @tchaton

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• Added a possibility to set up basic authentication for Lightning apps (#16105)

Changed

• The LoadBalancer now uses internal ip + port instead of URL exposed (#16119)
• Added support for logging in different trainer stages with DeviceStatsMonitor (#16002)
• Changed lightning_app.components.serve.gradio to lightning_app.components.serve.gradio_server (#16201)
• Made cluster creation/deletion async by default (#16185)

Fixed

• Fixed not being able to run multiple lightning apps locally due to port collision (#15819)
• Avoid relpath bug on Windows (#16164)
• Avoid using the deprecated LooseVersion (#16162)
• Porting fixes to autoscaler component (#16249)
• Fixed a bug where lightning login with env variables would not correctly save the credentials (#16339)

---

Fabric

Added

• Added Fabric.launch() to programmatically launch processes (e.g. in Jupyter notebook) (#14992)
• Added the option to launch Fabric scripts from the CLI, without the need to wrap the code into the run method (#14992)
• Added Fabric.setup_module() and Fabric.setup_optimizers() to support strategies that need to set up the model before an optimizer can be created (#15185)
• Added support for Fully Sharded Data Parallel (FSDP) training in Lightning Lite (#14967)
• Added lightning_fabric.accelerators.find_usable_cuda_devices utility function (#16147)
• Added basic support for LightningModules (#16048)
• Added support for managing callbacks via Fabric(callbacks=...) and emitting events through Fabric.call() (#16074)
• Added Logger support (#16121)
  • Added Fabric(loggers=...) to support different Logger frameworks in Fabric
  • Added Fabric.log for logging scalars using multiple loggers
  • Added Fabric.log_dict for logging a dictionary of multiple metrics at once
  • Added Fabric.loggers and Fabric.logger attributes to access the individual logger instances
  • Added support for calling self.log and self.log_dict in a LightningModule when using Fabric
  • Added access to self.logger and self.loggers in a LightningModule when using Fabric
• Added lightning_fabric.loggers.TensorBoardLogger (#16121)
• Added lightning_fabric.loggers.CSVLogger (#16346)
• Added support for a consistent .zero_grad(set_to_none=...) on the wrapped optimizer regardless of which strategy is used (#16275)

Changed

• Renamed the class LightningLite to Fabric (#15932, #15938)
• The Fabric.run() method is no longer abstract (#14992)
• The XLAStrategy now inherits from ParallelStrategy instead of DDPSpawnStrategy (#15838)
• Merged the implementation of DDPSpawnStrategy into DDPStrategy and removed DDPSpawnStrategy (#14952)
• The dataloader wrapper returned from .setup_dataloaders() now calls .set_epoch() on the distributed sampler if one is used (#16101)
• Renamed Strategy.reduce to Strategy.all_reduce in all strategies (#16370)
• When using multiple devices, the strategy now defaults to "ddp" instead of "ddp_spawn" when none is set (#16388)

Removed

• Removed support for FairScale's sharded training (strategy='ddp_sharded'|'ddp_sharded_spawn'). Use Fully-Sharded Data Parallel instead (strategy='fsdp') (#16329)

Fixed

• Restored sampling parity between PyTorch and Fabric dataloaders when using the DistributedSampler (#16101)
• Fixes an issue where the error message wouldn't tell the user the real value that was passed through the CLI (#16334)

---

PyTorch

Added

• Added support for native logging of MetricCollection with enabled compute groups (#15580)
• Added support for custom artifact names in pl.loggers.WandbLogger (#16173)
• Added support for DDP with LRFinder (#15304)
• Added utilities to migrate checkpoints from one Lightning version to another (#15237)
• Added support to upgrade all checkpoints in a folder using the pl.utilities.upgrade_checkpoint script (#15333)
• Add an axes argument ax to the .lr_find().plot() to enable writing to a user-defined axes in a matplotlib figure (#15652)
• Added log_model parameter to MLFlowLogger (#9187)
• Added a check to validate that wrapped FSDP models are used while initializing optimizers (#15301)
• Added a warning when self.log(..., logger=True) is called without a configured logger (#15814)
• Added support for colossalai 0.1.11 (#15888)
• Added LightningCLI support for optimizer and learning schedulers via callable type dependency injection (#15869)
• Added support for activation checkpointing for the DDPFullyShardedNativeStrategy strategy (#15826)
• Added the option to set DDPFullyShardedNativeStrategy(cpu_offload=True|False) via bool instead of needing to pass a configuration object (#15832)
• Added info message for Ampere CUDA GPU users to enable tf32 matmul precision (#16037)
• Added support for returning optimizer-like classes in LightningModule.configure_optimizers (#16189)

Changed

• Switch from tensorboard to tensorboardx in TensorBoardLogger (#15728)
• From now on, Lightning Trainer and LightningModule.load_from_checkpoint automatically upgrade the loaded checkpoint if it was produced in an old version of Lightning (#15237)
• Trainer.{validate,test,predict}(ckpt_path=...) no longer restores the Trainer.global_step and trainer.current_epoch value from the checkpoints - From now on, only Trainer.fit will restore this value (#15532)
• The ModelCheckpoint.save_on_train_epoch_end attribute is now computed dynamically every epoch, accounting for changes to the validation dataloaders (#15300)
• The Trainer now raises an error if it is given multiple stateful callbacks of the same time with colliding state keys (#15634)
• MLFlowLogger now logs hyperparameters and metrics in batched API calls (#15915)
• Overriding the on_train_batch_{start,end} hooks in conjunction with taking a dataloader_iter in the training_step no longer errors out and instead shows a warning (#16062)
• Move tensorboardX to extra dependencies. Use the CSVLogger by default (#16349)
• Drop PyTorch 1.9 support (#15347)

Deprecated

• Deprecated description, env_prefix and env_parse parameters in LightningCLI.__init__ in favour of giving them through parser_kwargs (#15651)
• Deprecated pytorch_lightning.profiler in favor of pytorch_lightning.profilers (#16059)
• Deprecated Trainer(auto_select_gpus=...) in favor of pytorch_lightning.accelerators.find_usable_cuda_devices (#16147)
• Deprecated pytorch_lightning.tuner.auto_gpu_select.{pick_single_gpu,pick_multiple_gpus} in favor of pytorch_lightning.accelerators.find_usable_cuda_devices (#16147)
• nvidia/apex deprecation (#16039)
  • Deprecated pytorch_lightning.plugins.NativeMixedPrecisionPlugin in favor of pytorch_lightning.plugins.MixedPrecisionPlugin
  • Deprecated the LightningModule.optimizer_step(using_native_amp=...) argument
  • Deprecated the Trainer(amp_backend=...) argument
  • Deprecated the Trainer.amp_backend property
  • Deprecated the Trainer(amp_level=...) argument
  • Deprecated the pytorch_lightning.plugins.ApexMixedPrecisionPlugin class
  • Deprecates the pytorch_lightning.utilities.enums.AMPType enum
  • Deprecates the DeepSpeedPrecisionPlugin(amp_type=..., amp_level=...) arguments
• horovod deprecation (#16141)
  • Deprecated Trainer(strategy="horovod")
  • Deprecated the HorovodStrategy class
• Deprecated pytorch_lightning.lite.LightningLite in favor of lightning.fabric.Fabric (#16314)
• FairScale deprecation (in favor of PyTorch's FSDP implementation) (#16353)
  • Deprecated the pytorch_lightning.overrides.fairscale.LightningShardedDataParallel class
  • Deprecated the pytorch_lightning.plugins.precision.fully_sharded_native_amp.FullyShardedNativeMixedPrecisionPlugin class
  • Deprecated the pytorch_lightning.plugins.precision.sharded_native_amp.ShardedNativeMixedPrecisionPlugin class
  • Deprecated the pytorch_lightning.strategies.fully_sharded.DDPFullyShardedStrategy class
  • Deprecated the pytorch_lightning.strategies.sharded.DDPShardedStrategy class
  • Deprecated the pytorch_lightning.strategies.sharded_spawn.DDPSpawnShardedStrategy class

Removed

• Removed deprecated pytorch_lightning.utilities.memory.get_gpu_memory_map in favor of pytorch_lightning.accelerators.cuda.get_nvidia_gpu_stats (#15617)
• Temporarily removed support for Hydra multi-run (#15737)
• Removed deprecated pytorch_lightning.profiler.base.AbstractProfiler in favor of pytorch_lightning.profilers.profiler.Profiler (#15637)
• Removed deprecated pytorch_lightning.profiler.base.BaseProfiler in favor of pytorch_lightning.profilers.profiler.Profiler (#15637)
• Removed deprecated code in pytorch_lightning.utilities.meta (#16038)
• Removed the deprecated LightningDeepSpeedModule (#16041)
• Removed the deprecated pytorch_lightning.accelerators.GPUAccelerator in favor of pytorch_lightning.accelerators.CUDAAccelerator (#16050)
• Removed the deprecated pytorch_lightning.profiler.* classes in favor of pytorch_lightning.profilers (#16059)
• Removed the deprecated pytorch_lightning.utilities.cli module in favor of pytorch_lightning.cli (#16116)
• Removed the deprecated pytorch_lightning.loggers.base module in favor of pytorch_lightning.loggers.logger (#16120)
• Removed the deprecated pytorch_lightning.loops.base module in favor of pytorch_lightning.loops.loop (#16142)
• Removed the deprecated pytorch_lightning.core.lightning module in favor of pytorch_lightning.core.module (#16318)
• Removed the deprecated pytorch_lightning.callbacks.base module in favor of pytorch_lightning.callbacks.callback (#16319)
• Removed the deprecated Trainer.reset_train_val_dataloaders() in favor of Trainer.reset_{train,val}_dataloader (#16131)
• Removed support for LightningCLI(seed_everything_default=None) (#16131)
• Removed support in LightningLite for FairScale's sharded training (strategy='ddp_sharded'|'ddp_sharded_spawn'). Use Fully-Sharded Data Parallel instead (strategy='fsdp') (#16329)

Fixed

• Enhanced reduce_boolean_decision to accommodate any-analogous semantics expected by the EarlyStopping callback (#15253)
• Fixed the incorrect optimizer step synchronization when running across multiple TPU devices (#16020)
• Fixed a type error when dividing the chunk size in the ColossalAI strategy (#16212)
• Fixed bug where the interval key of the scheduler would be ignored during manual optimization, making the LearningRateMonitor callback fail to log the learning rate (#16308)
• Fixed an issue with MLFlowLogger not finalizing correctly when status code 'finished' was passed (#16340)

---

Contributors

@1SAA, @akihironitta, @AlessioQuercia, @awaelchli, @bipinKrishnan, @Borda, @carmocca, @dmitsf, @erhoo82, @ethanwharris, @Forbu, @hhsecond, @justusschock, @lantiga, @lightningforever, @Liyang90, @manangoel99, @mauvilsa, @nicolai86, @nohalon, @rohitgr7, @schmidt-jake, @speediedan, @yMayanand

If we forgot someone due to not matching commit email with GitHub account, let us know :]

App

Added

• Added partial support for fastapi Request annotation in configure_api handlers (#16047)
• Added a nicer UI with URL and examples for the autoscaler component (#16063)
• Enabled users to have more control over scaling out/in intervals (#16093)
• Added more datatypes to the serving component (#16018)
• Added work.delete method to delete the work (#16103)
• Added display_name property to LightningWork for the cloud (#16095)
• Added ColdStartProxy to the AutoScaler (#16094)
• Added status endpoint, enable ready (#16075)
• Implemented ready for components (#16129)

Changed

• The default start_method for creating Work processes locally on macOS is now 'spawn' (previously 'fork') (#16089)
• The utility lightning.app.utilities.cloud.is_running_in_cloud now returns True during the loading of the app locally when running with --cloud (#16045)
• Updated Multinode Warning (#16091)
• Updated app testing (#16000)
• Changed overwrite to True (#16009)
• Simplified messaging in cloud dispatch (#16160)
• Added annotations endpoint (#16159)

Fixed

• Fixed PythonServer messaging "Your app has started" (#15989)
• Fixed auto-batching to enable batching for requests coming even after the batch interval but is in the queue (#16110)
• Fixed a bug where AutoScaler would fail with min_replica=0 (#16092
• Fixed a non-thread safe deepcopy in the scheduler (#16114)
• Fixed HTTP Queue sleeping for 1 sec by default if no delta was found (#16114)
• Fixed the endpoint info tab not showing up in the AutoScaler UI (#16128)
• Fixed an issue where an exception would be raised in the logs when using a recent version of streamlit (#16139)
• Fixed e2e tests (#16146)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.5.post0...1.8.6

App

• Fixed install/upgrade - removing single quote (#16079)
• Fixed bug where components that are re-instantiated several times failed to initialize if they were modifying self.lightningignore (#16080)
• Fixed a bug where apps that had previously been deleted could not be run again from the CLI (#16082)

Pytorch

• Add function to remove checkpoint to allow override for extended classes (#16067)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.5...1.8.5.post0

App

Added

• Added Lightning{Flow,Work}.lightningignores attributes to programmatically ignore files before uploading to the cloud (#15818)
• Added a progress bar while connecting to an app through the CLI (#16035)
• Support running on multiple clusters (#16016)
• Added guards to cluster deletion from cli (#16053)
• Added creation of the default .lightningignore that ignores venv (#16056)

Changed

• Cleanup cluster waiting (#16054)

Fixed

• Fixed DDPStrategy import in app framework (#16029)
• Fixed AutoScaler raising an exception when non-default cloud compute is specified (#15991)
• Fixed and improvements of login flow (#16052)
• Fixed the debugger detection mechanism for the lightning App in VSCode (#16068)

Pytorch

• some minor cleaning

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.4.post0...1.8.5

App

• Fixed MultiNode Component to use separate cloud computes (#15965)
• Fixed Registration for CloudComputes of Works in L.app.structures (#15964)
• Fixed a bug where auto-upgrading to the latest lightning via the CLI could get stuck in a loop (#15984)

Pytorch

• Fixed the XLAProfiler not recording anything due to mismatching of action names (#15885)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.4...1.8.4.post0

App

Added

• Add code_dir argument to tracer run (#15771)
• Added the CLI command lightning run model to launch a LightningLite accelerated script (#15506)
• Added the CLI command lightning delete app to delete a lightning app on the cloud (#15783)
• Added a CloudMultiProcessBackend which enables running a child App from within the Flow in the cloud (#15800)
• Utility for pickling work object safely even from a child process (#15836)
• Added AutoScaler component (#15769)
• Added the property ready of the LightningFlow to inform when the Open App should be visible (#15921)
• Added private work attributed _start_method to customize how to start the works (#15923)
• Added a configure_layout method to the LightningWork which can be used to control how the work is handled in the layout of a parent flow (#15926)
• Added the ability to run a Lightning App or Component directly from the Gallery using lightning run app organization/name (#15941)
• Added automatic conversion of list and dict of works and flows to structures (#15961)

Changed

• The MultiNode components now warn the user when running with num_nodes > 1 locally (#15806)
• Cluster creation and deletion now waits by default [#15458
• Running an app without a UI locally no longer opens the browser (#15875)
• Show a message when BuildConfig(requirements=[...]) is passed but a requirements.txt file is already present in the Work (#15799)
• Show a message when BuildConfig(dockerfile="...") is passed but a Dockerfile file is already present in the Work (#15799)
• Dropped name column from cluster list (#15721)
• Apps without UIs no longer activate the "Open App" button when running in the cloud (#15875)
• Wait for full file to be transferred in Path / Payload (#15934)

Removed

• Removed the SingleProcessRuntime (#15933)

Fixed

• Fixed SSH CLI command listing stopped components (#15810)
• Fixed bug when launching apps on multiple clusters (#15484)
• Fixed Sigterm Handler causing thread lock which caused KeyboardInterrupt to hang (#15881)
• Fixed MPS error for multinode component (defaults to cpu on mps devices now as distributed operations are not supported by pytorch on mps) (#15748)
• Fixed the work not stopped when successful when passed directly to the LightningApp (#15801)
• Fixed the PyTorch Inference locally on GPU (#15813)
• Fixed the enable_spawn method of the WorkRunExecutor (#15812)
• Fixed require/import decorator (#15849)
• Fixed a bug where using L.app.structures would cause multiple apps to be opened and fail with an error in the cloud (#15911)
• Fixed PythonServer generating noise on M1 (#15949)
• Fixed multiprocessing breakpoint (#15950)
• Fixed detection of a Lightning App running in debug mode (#15951)
• Fixed ImportError on Multinode if package not present (#15963)

Lite

• Fixed shuffle=False having no effect when using DDP/DistributedSampler (#15931)

Pytorch

Changed

• Direct support for compiled models (#15922)

Fixed

• Fixed issue with unsupported torch.inference_mode() on hpu backends (#15918)
• Fixed LRScheduler import for PyTorch 2.0 (#15940)
• Fixed fit_loop.restarting to be False for lr finder (#15620)
• Fixed torch.jit.script-ing a LightningModule causing an unintended error message about deprecated use_amp property (#15947)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.3...1.8.4

App

Changed

• Fixed the PyTorch Inference locally on GPU (#15813)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.3...1.8.3

App

Changed

• Deduplicate top-level lighting CLI command groups (#15761)
  • lightning add ssh-key CLI command has been transitioned to lightning create ssh-key
  • lightning remove ssh-key CLI command has been transitioned to lightning delete ssh-key
• Set Torch inference mode for prediction (#15719)
• Improved LightningTrainerScript start-up time (#15751)
• Disable XSRF protection in StreamlitFrontend to support upload in localhost (#15684)

Fixed

• Fixed debugging with VSCode IDE (#15747)
• Fixed setting property to the LightningFlow (#15750)

Lite

Changed

• Temporarily removed support for Hydra multi-run (#15737)

Pytorch

Changed

• Temporarily removed support for Hydra multi-run (#15737)
• Switch from tensorboard to tensorboardx in TensorBoardLogger (#15728)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.2...1.8.3

App

Added

• Added title and description to ServeGradio (#15639)
• Added a friendly error message when attempting to run the default cloud compute with a custom base image configured (#14929)

Changed

• Improved support for running apps when dependencies aren't installed (#15711)
• Changed the root directory of the app (which gets uploaded) to be the folder containing the app file, rather than any parent folder containing a .lightning file (#15654)
• Enabled MultiNode Components to support state broadcasting (#15607)
• Prevent artefactual "running from outside your current environment" error (#15647)
• Rename failed -> error in tables (#15608)

Fixed

• Fixed race condition to over-write the frontend with app infos (#15398)
• Fixed bi-directional queues sending delta with Drive Component name changes (#15642)
• Fixed CloudRuntime works collection with structures and accelerated multi node startup time (#15650)
• Fixed catimage import (#15712)
• Parse all lines in app file looking for shebangs to run commands (#15714)

Lite

Fixed

• Fixed the automatic fallback from LightningLite(strategy="ddp_spawn", ...) to LightningLite(strategy="ddp", ...) when on an LSF cluster (#15103)

Pytorch

Fixed

• Make sure save_dir can be empty str (#15638](https://github.com/PyTorchLightning/pytorch-lightning/issues/15638))
• Fixed the automatic fallback from Trainer(strategy="ddp_spawn", ...) to Trainer(strategy="ddp", ...) when on an LSF cluster (#15103](https://github.com/PyTorchLightning/pytorch-lightning/issues/15103))

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.1...1.8.2

App

Added

• Added the start method to the work (#15523)
• Added a MultiNode Component to run with distributed computation with any frameworks (#15524)
• Expose RunWorkExecutor to the work and provides default ones for the MultiNode Component (#15561)
• Added a start_with_flow flag to the LightningWork which can be disabled to prevent the work from starting at the same time as the flow (#15591)
• Added support for running Lightning App with VSCode IDE debugger (#15590)
• Added bi-directional delta updates between the flow and the works (#15582)
• Added --setup flag to lightning run app CLI command allowing for dependency installation via app comments (#15577)
• Auto-upgrade / detect environment mis-match from the CLI (#15434)
• Added Serve component (#15609)

Changed

• Changed the flow.flows to be recursive wont to align the behavior with the flow.works (#15466)
• The params argument in TracerPythonScript.run no longer prepends -- automatically to parameters (#15518)
• Only check versions / env when not in the cloud (#15504)
• Periodically sync database to the drive (#15441)
• Slightly safer multi node (#15538)
• Reuse existing commands when running connect more than once (#15471)

Fixed

• Fixed writing app name and id in connect.txt file for the command CLI (#15443)
• Fixed missing root flow among the flows of the app (#15531)
• Fixed bug with Multi Node Component and add some examples (#15557)
• Fixed a bug where payload would take a very long time locally (#15557)
• Fixed an issue with the lightning CLI taking a long time to error out when the cloud is not reachable (#15412)

Lite

Fixed

• Fix an issue with the SLURM srun detection causing permission errors (#15485)
• Fixed the import of lightning_lite causing a warning 'Redirects are currently not supported in Windows or MacOs' (#15610)

PyTorch

Fixed

• Fixed TensorBoardLogger not validating the input array type when logging the model graph (#15323)
• Fixed an attribute error in ColossalAIStrategy at import time when torch.distributed is not available (#15535)
• Fixed an issue when calling fs.listdir with file URI instead of path in CheckpointConnector (#15413)
• Fixed an issue with the BaseFinetuning callback not setting the track_running_stats attribute for batch normaliztion layers (#15063)
• Fixed an issue with WandbLogger(log_model=True|'all) raising an error and not being able to serialize tensors in the metadata (#15544)
• Fixed the gradient unscaling logic when using Trainer(precision=16) and fused optimizers such as Adam(..., fused=True) (#15544)
• Fixed model state transfer in multiprocessing launcher when running multi-node (#15567)
• Fixed manual optimization raising AttributeError with Bagua Strategy (#12534)
• Fixed the import of pytorch_lightning causing a warning 'Redirects are currently not supported in Windows or MacOs' (#15610)

---

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.0...1.8.1

What's Changed

• Implement freeze batchnorm with freezing track running stats by @PososikTeam in https://github.com/Lightning-AI/lightning/pull/15063
• Pkg: fix parsing versions by @Borda in https://github.com/Lightning-AI/lightning/pull/15401
• Remove pytest as a requirement to run app by @manskx in https://github.com/Lightning-AI/lightning/pull/15449

New Contributors

• @PososikTeam made their first contribution in https://github.com/Lightning-AI/lightning/pull/15063

Full Changelog: https://github.com/Lightning-AI/lightning/compare/1.8.0...1.8.0.post1

The core team is excited to announce the release of Lightning 1.8 :zap:

• Highlights
• Backward Incompatible Changes
• Deprecations
• Full Changelog
• Contributors

Lightning v1.8 is the culmination of work from 52 contributors who have worked on features, bug-fixes, and documentation for a total of over 550+ commits since v1.7.

<a name="highlights"></a>

Highlights

Colossal-AI

Colossal-AI focuses on improving efficiency when training large-scale AI models with billions of parameters. With the new Colossal-AI strategy in Lightning 1.8, you can train existing models like GPT-3 with up to half as many GPUs as usually needed. You can also train models up to twice as big with the same number of GPUs, saving you significant cost. Here is how you use it:

# Select the strategy with good defaults
trainer = Trainer(strategy="colossalai")

# or tune parameters to your liking
from lightning.pytorch.strategies import ColossalAIStrategy

trainer = Trainer(strategy=ColossalAIStrategy(placement_policy="cpu", ...))

You can find Colossal-AI's benchmarks with Lightning on GPT-2 here.

Under the hood, Colossal-AI implements different parallelism algorithms that are especially interesting for the development of SOTA transformer models:

• Data Parallelism
• Pipeline Parallelism
• 1D, 2D, 2.5D, 3D Tensor Parallelism
• Sequence Parallelism
• Zero Redundancy Optimization

Learn how to install and use Colossal-AI effectively with Lightning here.

NOTE: This strategy is marked as experimental. Stay tuned for more updates in the future.

Secrets for Lightning Apps

Introducing encrypted secrets (#14612), a feature requested by Lightning App users :tada:!

Encrypted secrets allow you to securely pass private data to your apps, like API keys, access tokens, database passwords, or other credentials, without exposing them in your code.

1. Add a secret to your Lightning account in lightning.ai (read more here)

2. Add an environment variable to your app to read the secret:

   # somewhere in your Flow or Work:
   GitHubComponent(api_token=os.environ["API_TOKEN"])
   

3. Pass the secret to your app run with the following command:

   lightning run app app.py --cloud --secret API_TOKEN=github_api_token
   

These secrets are encrypted and stored in the Lightning database. Nothing except your app can access the value.

NOTE: This is an experimental feature.

CLI Commands for Lightning Apps

Introducing CLI commands for apps (#13602)! As a Lightning App builder, if you want to easily create a CLI interface for users to interract with your app, then this is for you.

Here is an example where users can dynamically create notebooks from the CLI. All you need to do is implement the configure_commands hook on the LightningFlow:

import lightning as L
from commands.notebook.run import RunNotebook


class Flow(L.LightningFlow):
    ...

    def configure_commands(self):
        # Return a list of dictionaries with commands:
        return [{"run notebook": RunNotebook(method=self.run_notebook)}]


app = L.LightningApp(Flow())

Once the app is running with lightning run app app.py, you can connect to the app with the following command:

lightning connect {app name} -y

and run the command that was configured:

lightning run notebook --name=my_notebook_name

<a style="visibility:hidden">For a full tutorial and running example, visit our docs. TODO: add to docs</a> NOTE: This is an experimental feature.

Auto-wrapping for FSDP Strategy

In Lightning v1.7, we introduced an integration for PyTorch FSDP in the form of our FSDP strategy, which allows you to train huge models with billions of parameters sharded across hundreds of GPUs and machines.

# Native FSDP implementation
trainer = Trainer(strategy="fsdp_native")

We are continuing to improve the support for this feature by adding automatic wrapping of layers for use cases where the model fits into CPU memory, but not into GPU memory (#14383).

Here are some examples:

Case 1: Model is so large that it does not fit into CPU memory. Construct your layers in the configure_sharded_model hook and wrap the large ones you want to shard across GPUs:

class MassiveModel(LightningModule):
    ...
    
    # Create model here and wrap the large layers for sharding
    def configure_sharded_model(self):
        for i, layer in enumerate(self.block):
            self.block[i] = wrap(layer)
        ...

Case 2: Model fits into CPU memory, but not into GPU memory. In Lightning v1.8, you no longer need to do anything special here, as we can automatically wrap the layers for you using FSDP's policy:

model = MassiveModel()
trainer = Trainer(
    accelerator="gpu", 
    devices=8, 
    strategy="fsdp_native",  # or strategy="fsdp" for fairscale
    precision=16
)

# Automatically wraps the layers here:
trainer.fit(model)

Case 3: Model fits into GPU memory. No action required, use any strategy you want.

Note: if you want to manually wrap layers for more control, you can still do that!

Read more about FSDP and how layer wrapping works in our docs.

New Tuner Callbacks

In this release, we focused on Tuner improvements and introduced two new callbacks that can help you customize the batch size finder and learning rate finder as per your use case.

Batch Size Finder (#11089)

1. You can customize the BatchSizeFinder callback to run at different epochs. This feature is useful while fine-tuning models since you can't always use the same batch size after unfreezing the backbone.

   from lightning.pytorch.callbacks import BatchSizeFinder
   
   
   class FineTuneBatchSizeFinder(BatchSizeFinder):
       def __init__(self, milestones, *args, **kwargs):
           super().__init__(*args, **kwargs)
           self.milestones = milestones
   
       def on_fit_start(self, *args, **kwargs):
           return
   
       def on_train_epoch_start(self, trainer, pl_module):
           if trainer.current_epoch in self.milestones or trainer.current_epoch == 0:
               self.scale_batch_size(trainer, pl_module)
   
   
   trainer = Trainer(callbacks=[FineTuneBatchSizeFinder(milestones=(5, 10))])
   trainer.fit(...)
   

2. Run batch size finder for validate/test/predict.

   from lightning.pytorch.callbacks import BatchSizeFinder
   
   
   class EvalBatchSizeFinder(BatchSizeFinder):
       def __init__(self, *args, **kwargs):
           super().__init__(*args, **kwargs)
   
       def on_fit_start(self, *args, **kwargs):
           return
   
       def on_test_start(self, trainer, pl_module):
           self.scale_batch_size(trainer, pl_module)
   
   
   trainer = Trainer(callbacks=[EvalBatchSizeFinder()])
   trainer.test(...)
   

Learning Rate Finder (#13802)

You can now use the LearningRateFinder callback to run at different intervals. This feature is useful when fine-tuning models, for example.

from lightning.pytorch.callbacks import LearningRateFinder


class FineTuneLearningRateFinder(LearningRateFinder):
    def __init__(self, milestones, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.milestones = milestones

    def on_fit_start(self, *args, **kwargs):
        return

    def on_train_epoch_start(self, trainer, pl_module):
        if trainer.current_epoch in self.milestones or trainer.current_epoch == 0:
            self.lr_find(trainer, pl_module)

trainer = Trainer(callbacks=[FineTuneLearningRateFinder(milestones=(5, 10))])
trainer.fit(...)

LightningCLI Improvements

Even though the LightningCLI class is designed to help in the implementation of command line tools, there are instances when it might be more desirable to run directly from Python. In Lightning 1.8, you can now do this (#14596):

from lightning.pytorch.cli import LightningCLI

def cli_main(args):
    cli = LightningCLI(MyModel, ..., args=args)
    ...

Anywhere in your program, you can now call the CLI directly:

cli_main(["--trainer.max_epochs=100", "--model.encoder_layers=24"])

Learn about all features of the LightningCLI!

Improvements to the SLURM Support

Multi-node training on a SLURM cluster has been supported since the inception of Lightning Trainer, and has seen several improvements over time thanks to many community contributions. And we just keep going! In this release, we've added two quality of life improvements:

• The preemption/termination signal is now configurable (#14626):

  # the default signal is SIGUSR1
  trainer = Trainer(plugins=[SLURMEnvironment(requeue_signal=signal.SIGUSR1)])
  
  # customize it for your cluster
  trainer = Trainer(plugins=[SLURMEnvironment(requeue_signal=signal.SIGHUP)])
  

• Automatic requeuing of jobs now also works for array jobs (#15040)! Array jobs are a convenient way to group/launch several scripts at once. When the SLURM scheduler interrupts your jobs, Lightning will save a checkpoint, resubmit a new job, and, once the scheduler allocates resources, the Trainer will resume from where it left off.

Read more about our SLURM integration here.

<a name="bc-changes"></a>

Backward Incompatible Changes

This section outlines notable changes that are not backward compatible with previous versions. The full list of changes and removals can be found in the CHANGELOG below.

Callback hooks for loading and saving checkpoints

The signature and behavior of the on_load_checkpoint and on_save_checkpoint callback hooks have changed (#14835):

Before:

def on_save_checkpoint(self, trainer, pl_module, checkpoint):
    ...
    # previously, we were able to return state here
    return state

def on_load_checkpoint(self, trainer, pl_module, callback_state):
    # previously, only the state for this callback was passed in as argument
    ...

Now:

def on_save_checkpoint(self, trainer, pl_module, checkpoint):
    ...
    # returning a value here is no longer supported
    # you can modify the checkpoint dict directly
    return None


def state_dict(self):
    ...
    # Now, return state from this new method
    return state


def on_load_checkpoint(self, trainer, pl_module, checkpoint):
    # previously, only the state for this callback was passed in as argument
    ...
    
    
def load_state_dict(self, state):
    # Now, the state for this callback gets passed to this new method
    ...

DataModule hooks for loading and saving checkpoints

The on_save_checkpoint and on_load_checkpoint hooks on the LightningDataModule have been removed in favor of the state_dict and load_state_dict methods:

-def on_save_checkpoint(self, checkpoint):
-    checkpoint["banana"] = self.banana
+def state_dict(self):
+    return dict(banana=self.banana)


-def on_load_checkpoint(self, checkpoint):
-    self.banana = checkpoint["banana"]
+def load_state_dict(self, state):
+    self.banana = state["banana"]

Callback hooks

We removed some Callback hooks that were ambiguous to use Removed deprecated callback hooks (#14834):

Old name	New name
on_batch_start	on_train_batch_start
on_batch_end	on_train_batch_end
on_epoch_start	on_train_epoch_start
on_epoch_start	on_validation_epoch_start
on_epoch_start	on_test_epoch_start
on_pretrain_routine_start	on_fit_start

Trainer Device Attributes

We cleaned up the properties related to device indices (#14829).

The attributes Trainer.{devices,gpus,num_gpus,ipus,tpu_cores,num_processes,root_gpu,data_parallel_device_ids} have been removed in favor of accelerator-agnostic attributes:

trainer = Trainer(...)

# access the number of devices the trainer uses on this machine ...
print(trainer.num_devices)

# ... or the device IDs
print(trainer.device_ids)

Setting the torch-distributed backend

In previous versions of Lightning, switching between the "gloo" and "nccl" backends for multi-GPU, multi-node training was possible through setting an environment variable like so:

PL_TORCH_DISTRIBUTED_BACKEND="gloo" python train.py

But not all strategies support changing the backend in this way. From now on, the backend has to be set in the code (#14693):

trainer = Trainer(strategy=DDPStrategy(process_group_backend="gloo"))

The default remains "nccl", and you should choose "gloo" only for debugging purposes.

Logging with multiple loggers

Logging with multiple loggers can be super useful (and super easy with Lightning). For example, you could be using one logger to record sensitive image logs to a hosted MLFlow server within your organization, and at the same time log loss curves online to WandB.

trainer = Trainer(
    loggers=[WandbLogger(...), MLFlowLogger(...)]
)

Here are two major changes that apply when using multiple loggers in 1.8:

• Checkpoints and profiler reports no longer go to a strange folder with a long, hard to remember name (#14325). From now on, these arifacts will land in the version folder of the first logger in the list.

• The loggers used to be wrapped by a LoggerCollection object, so that when you accessed trainer.logger you could log to all of them simultaneously. However, this "magic" caused confusion and errors among users and we decided to simplify this (#14283):

  # now returns the first logger in the list
  print(trainer.logger)
  
  # access all loggers in a list with plural
  loggers = trainer.loggers
  
  for logger in loggers:
      logger.do_something()
  

<a name="deprecations"></a>

Deprecations

Why is Lightning deprecating APIs in every release?

Many users have this question, and it is a fair one! Deprecations are a normal part of API evolution in all software. We continually improve Lightning, which means we make APIs like class names, methods, hooks and arguments clear, easy to remember, and general enough to adopt more functionality in the future. Sometimes we have to let old things go to build new and better products.

Learn more about our deprecation window here.

So far, we have followed the pattern of removing deprecated functionality and APIs after two minor versions of deprecation. From Lightning 1.8 onward, we will additionaly convert warnings to error messages after the deprecation phase ends. This way, we can greatly improve the upgrade experience with helpful messages for users who skip more than two minor Lightning versions. The exception to this rule are experimental features, which are marked as such in our documentation.

Here is a summary of major deprecations introduced in 1.8:

API	Removal version	Alternative
Argument Trainer(amp_level=...)	1.10	Trainer(plugins=[ApexMixedPrecisionPlugin(amp_level=...)])
Function unwrap_lightning_module	1.10	Strategy.lightning_module
Function unwrap_lightning_module_sharded	1.10	Strategy.lightning_module
Import pl.core.mixins.DeviceDtypeModuleMixin	1.10	No longer supported
Argument LightningCLI(save_config_filename=...)	1.10	LightningCLI(save_config_kwargs=dict(config_filename=...))
Argument LightningCLI(save_config_overwrite=...)	1.10	LightningCLI(save_config_kwargs=dict(overwrite=...))
Argument LightningCLI(save_config_multifile=...)	1.10	LightningCLI(save_config_kwargs=dict(multifile=...))
Enum TrainerFn.TUNING	1.10	No longer supported
Enum RunningStage.TUNING	1.10	No longer supported
Attribute Trainer.tuning	1.10	No longer supported

<a name="changelog"></a>

CHANGELOG

Lightning App

<details><summary>Added</summary>

• Added load_state_dict and state_dict hooks for LightningFlow components (#14100)
• Added a --secret option to CLI to allow binding secrets to app environment variables when running in the cloud (#14612)
• Added support for running the works without cloud compute in the default container (#14819)
• Added an HTTPQueue as an optional replacement for the default redis queue (#14978
• Added support for configuring flow cloud compute (#14831)
• Added support for adding descriptions to commands either through a docstring or the DESCRIPTION attribute (#15193
• Added a try / catch mechanism around request processing to avoid killing the flow (#15187
• Added an Database Component (#14995
• Added authentication to HTTP queue (#15202)
• Added support to pass a LightningWork to the LightningApp (#15215
• Added support getting CLI help for connected apps even if the app isn't running (#15196
• Added support for adding requirements to commands and installing them when missing when running an app command (#15198
• Added Lightning CLI Connection to be terminal session instead of global (#15241
• Added support for managing SSH-keys via CLI (#15291)
• Add a JustPyFrontend to ease UI creation with https://github.com/justpy-org/justpy (#15002)
• Added a layout endpoint to the Rest API and enable to disable pulling or pushing to the state (#15367
• Added support for functions for configure_api and configure_commands to be executed in the Rest API process (#15098
• Added support to start lightning app on cloud without needing to install dependencies locally (#15019

</details>

<details><summary>Changed</summary>

• Improved the show logs command to be standalone and re-usable (#15343
• Removed the --instance-types option when creating clusters (#15314)

</details>

<details><summary>Fixed</summary>

• Fixed an issue when using the CLI without arguments (#14877)
• Fixed a bug where the upload files endpoint would raise an error when running locally (#14924)
• Fixed BYOC cluster region selector -> hiding it from help since only us-east-1 has been tested and is recommended ([#15277]https://github.com/Lightning-AI/lightning/pull/15277)
• Fixed a bug when launching an app on multiple clusters (#15226)
• Fixed a bug with a default CloudCompute for Lightning flows (#15371)

</details>

Lightning Trainer

<details><summary>Added</summary>

• Added support for requeueing slurm array jobs (#15040)
• Added native AMP support for ddp_fork (and associated alias strategies) with CUDA GPUs (#14983)
• Added BatchSizeFinder callback (#11089)
• Added LearningRateFinder callback (#13802)
• Tuner now supports a new method argument which will determine when to run the BatchSizeFinder: one of fit, validate, test or predict (#11089)
• Added prefix to log message in seed_everything with rank info (#14031)
• Added support for auto wrapping for DDPFullyShardedNativeStrategy (#14252)
• Added support for passing extra init-parameters to the LightningDataModule.from_datasets (#14185)
• Added support for saving sharded optimizer state dict outside of DDPShardedStrategy (#14208)
• Added support for auto wrapping for DDPFullyShardedStrategy (#14383)
• Integrate the lightning_utilities package ( #14475, #14537, #14556, #14558, #14575, #14620)
• Added args parameter to LightningCLI to ease running from within Python (#14596)
• Added WandbLogger.download_artifact and WandbLogger.use_artifact for managing artifacts with Weights and Biases (#14551)
• Added an option to configure the signal SLURM sends when a job is preempted or requeued (#14626)
• Added a warning when the model passed to LightningLite.setup() does not have all parameters on the same device (#14822)
• The CometLogger now flags the Comet Experiments as being created from Lightning for analytics purposes (#14906)
• Introduce ckpt_path="hpc" keyword for checkpoint loading (#14911)
• Added a more descriptive error message when attempting to fork processes with pre-initialized CUDA context (#14709)
• Added support for custom parameters in subclasses of SaveConfigCallback (#14998)
• Added inference_mode flag to Trainer to let users enable/disable inference mode during evaluation (#15034)
• Added LightningLite.no_backward_sync for control over efficient gradient accumulation with distributed strategies (#14966)
• Added a sanity check that scripts are executed with the srun command in SLURM and that environment variables are not conflicting (#15011)
• Added an error message when attempting to launch processes with python -i and an interactive-incompatible strategy (#15293)

</details>

<details><summary>Changed</summary>

• The Trainer.{fit,validate,test,predict,tune} methods now raise a useful error message if the input is not a LightningModule (#13892)
• Raised a MisconfigurationException if batch transfer hooks are overriden with IPUAccelerator (#13961)
• Replaced the unwrapping logic in strategies with direct access to unwrapped LightningModule (#13738)
• Enabled on_before_batch_transfer for DPStrategy and IPUAccelerator (#14023)
• When resuming training with Apex enabled, the Trainer will now raise an error (#14341)
• Included torch.cuda rng state to the aggregate _collect_rng_states() and _set_rng_states() (#14384)
• Changed trainer.should_stop to not stop in between an epoch and run until min_steps/min_epochs only (#13890)
• The pyDeprecate dependency is no longer installed (#14472)
• When using multiple loggers, by default checkpoints and profiler output now get saved to the log dir of the first logger in the list (#14325)
• In Lightning Lite, state-dict access to the module wrapper now gets passed through to the original module reference (#14629)
• Removed fall-back to LightningEnvironment when number of SLURM tasks does not correspond to number of processes in Trainer (#14300)
• Aligned DDP and DDPSpawn strategies in setting up the environment (#11073)
• Integrated the Lite Precision plugins into the PL Precision plugins - the base class in PL now extends the lightning_lite.precision.Precision base class (#14798)
  • The PrecisionPlugin.backward signature changed: The closure_loss argument was renamed to tensor
  • The PrecisionPlugin.{pre_,post_}backward signature changed: The closure_loss argument was renamed to tensor and moved as the first argument
  • The PrecisionPlugin.optimizer_step signature changed: The model, optimizer_idx and closure arguments need to be passed as keyword arguments now
• Trainer queries the CUDA devices through NVML if available to avoid initializing CUDA before forking, which eliminates the need for the PL_DISABLE_FORK environment variable introduced in v1.7.4 (#14631)
• The MLFlowLogger.finalize() now sets the status to FAILED when an exception occurred in Trainer, and sets the status to FINISHED on successful completion (#12292)
• It is no longer needed to call model.double() when using precision=64 in Lightning Lite (#14827)
• HPC checkpoints are now loaded automatically only in slurm environment when no specific value for ckpt_path has been set (#14911)
• The Callback.on_load_checkpoint now gets the full checkpoint dictionary and the callback_state argument was renamed checkpoint (#14835)
• Moved the warning about saving nn.Module in save_hyperparameters() to before the deepcopy (#15132)
• To avoid issues with forking processes, from PyTorch 1.13 and higher, Lightning will directly use the PyTorch NVML-based check for torch.cuda.device_count and from PyTorch 1.14 and higher, Lightning will configure PyTorch to use a NVML-based check for torch.cuda.is_available. (#15110, #15133)
• The NeptuneLogger now uses neptune.init_run instead of the deprecated neptune.init to initialize a run (#15393)

</details>

<details><summary>Deprecated</summary>

• Deprecated LightningDeepSpeedModule (#14000)
• Deprecated amp_level from Trainer in favour of passing it explictly via precision plugin (#13898)
• Deprecated the calls to pytorch_lightning.utiltiies.meta functions in favor of built-in https://github.com/pytorch/torchdistx support (#13868)
• Deprecated the unwrap_lightning_module and unwrap_lightning_module_sharded utility functions in favor of accessing the unwrapped LightningModule on the strategy directly (#13738)
• Deprecated the pl_module argument in LightningParallelModule, LightningDistributedModule, LightningShardedDataParallel, LightningBaguaModule and LightningDeepSpeedModule wrapper classes (#13738)
• Deprecated the on_colab_kaggle function (#14247)
• Deprecated the internal pl.core.mixins.DeviceDtypeModuleMixin class (#14511, #14548)
• Deprecated all functions in pytorch_lightning.utilities.xla_device (#14514, #14550)
  • Deprecated the internal inner_f function
  • Deprecated the internal pl_multi_process function
  • Deprecated the internal XLADeviceUtils.xla_available staticmethod
  • Deprecated the XLADeviceUtils.tpu_device_exists staticmethod in favor of pytorch_lightning.accelerators.TPUAccelerator.is_available()
• Deprecated pytorch_lightning.utilities.distributed.tpu_distributed in favor of lightning_lite.accelerators.tpu.tpu_distributed (#14550)
• Deprecated all functions in pytorch_lightning.utilities.cloud_io in favor of lightning_lite.utilities.cloud_io (#14515)
• Deprecated the functions in pytorch_lightning.utilities.apply_func in favor of lightning_utilities.core.apply_func (#14516, #14537)
• Deprecated all functions in pytorch_lightning.utilities.device_parser (#14492, #14753)
  • Deprecated the pytorch_lightning.utilities.device_parser.determine_root_gpu_device in favor of lightning_lite.utilities.device_parser.determine_root_gpu_device
  • Deprecated the pytorch_lightning.utilities.device_parser.parse_gpu_ids in favor of lightning_lite.utilities.device_parser.parse_gpu_ids
  • Deprecated the pytorch_lightning.utilities.device_parser.is_cuda_available in favor of lightning_lite.accelerators.cuda.is_cuda_available
  • Deprecated the pytorch_lightning.utilities.device_parser.num_cuda_devices in favor of lightning_lite.accelerators.cuda.num_cuda_devices
  • Deprecated the pytorch_lightning.utilities.device_parser.parse_cpu_cores in favor of lightning_lite.accelerators.cpu.parse_cpu_cores
  • Deprecated the pytorch_lightning.utilities.device_parser.parse_tpu_cores in favor of lightning_lite.accelerators.tpu.parse_tpu_cores
  • Deprecated the pytorch_lightning.utilities.device_parser.parse_hpus in favor of pytorch_lightning.accelerators.hpu.parse_hpus
• Deprecated duplicate SaveConfigCallback parameters in LightningCLI.__init__: save_config_kwargs, save_config_overwrite and save_config_multifile. New save_config_kwargs parameter should be used instead (#14998)
• Deprecated TrainerFn.TUNING, RunningStage.TUNING and trainer.tuning property (#15100)
• Deprecated custom pl.utilities.distributed.AllGatherGrad implementation in favor of PyTorch's (#15364)

</details>

<details><summary>Removed</summary>

• Removed the deprecated Trainer.training_type_plugin property in favor of Trainer.strategy (#14011)
• Removed all deprecated training type plugins (#14011)
• Removed the deprecated DDP2Strategy (#14026)
• Removed the deprecated DistributedType and DeviceType enum classes (#14045)
• Removed deprecated support for passing the rank_zero_warn warning category positionally (#14470)
• Removed the legacy and unused Trainer.get_deprecated_arg_names() (#14415)
• Removed the deprecated on_train_batch_end(outputs) format when multiple optimizers are used and TBPTT is enabled (#14373)
• Removed the deprecated training_epoch_end(outputs) format when multiple optimizers are used and TBPTT is enabled (#14373)
• Removed the experimental pytorch_lightning.utiltiies.meta functions in favor of built-in https://github.com/pytorch/torchdistx support (#13868)
• Removed the deprecated LoggerCollection; Trainer.logger and LightningModule.logger now returns the first logger when more than one gets passed to the Trainer (#14283)
• Removed the deprecated the trainer.lr_schedulers (#14408)
• Removed the deprecated LightningModule.{on_hpc_load,on_hpc_save} hooks in favor of the general purpose hooks LightningModule.{on_load_checkpoint,on_save_checkpoint} (#14315)
• Removed deprecated support for old torchtext versions (#14375)
• Removed deprecated support for the old neptune-client API in the NeptuneLogger (#14727)
• Removed the deprecated weights_save_path Trainer argumnent and Trainer.weights_save_path property (#14424)
• Removed the deprecated (#14471)
  • pytorch_lightning.utilities.distributed.rank_zero_only in favor of pytorch_lightning.utilities.rank_zero.rank_zero_only
  • pytorch_lightning.utilities.distributed.rank_zero_debug in favor of pytorch_lightning.utilities.rank_zero.rank_zero_debug
  • pytorch_lightning.utilities.distributed.rank_zero_info in favor of pytorch_lightning.utilities.rank_zero.rank_zero_info
  • pytorch_lightning.utilities.warnings.rank_zero_warn in favor of pytorch_lightning.utilities.rank_zero.rank_zero_warn
  • pytorch_lightning.utilities.warnings.rank_zero_deprecation in favor of pytorch_lightning.utilities.rank_zero.rank_zero_deprecation
  • pytorch_lightning.utilities.warnings.LightningDeprecationWarning in favor of pytorch_lightning.utilities.rank_zero.LightningDeprecationWarning
• Removed deprecated Trainer.num_processes attribute in favour of Trainer.num_devices (#14423)
• Removed the deprecated Trainer.data_parallel_device_ids hook in favour of Trainer.device_ids (#14422)
• Removed the deprecated class TrainerCallbackHookMixin (#14401)
• Removed the deprecated BaseProfiler and AbstractProfiler classes (#14404)
• Removed the deprecated way to set the distributed backend via the environment variable PL_TORCH_DISTRIBUTED_BACKEND, in favor of setting the process_group_backend in the strategy constructor (#14693)
• Removed deprecated callback hooks (#14834)
  • Callback.on_configure_sharded_model in favor of Callback.setup
  • Callback.on_before_accelerator_backend_setup in favor of Callback.setup
  • Callback.on_batch_start in favor of Callback.on_train_batch_start
  • Callback.on_batch_end in favor of Callback.on_train_batch_end
  • Callback.on_epoch_start in favor of Callback.on_{train,validation,test}_epoch_start
  • Callback.on_epoch_end in favor of Callback.on_{train,validation,test}_epoch_end
  • Callback.on_pretrain_routine_{start,end} in favor of Callback.on_fit_start
• Removed the deprecated device attributes Trainer.{devices,gpus,num_gpus,ipus,tpu_cores} in favor of the accelerator-agnostic Trainer.num_devices (#14829)
• Removed the deprecated LightningIPUModule (#14830)
• Removed the deprecated Logger.agg_and_log_metrics hook in favour of Logger.log_metrics and the agg_key_funcs and agg_default_func arguments. (#14840)
• Removed the deprecated precision plugin checkpoint hooks PrecisionPlugin.on_load_checkpoint and PrecisionPlugin.on_save_checkpoint (#14833)
• Removed the deprecated Trainer.root_gpu attribute in favor of Trainer.strategy.root_device (#14829)
• Removed the deprecated Trainer.use_amp and LightningModule.use_amp attributes (#14832)
• Removed the deprecated callback hooks Callback.on_init_start and Callback.on_init_end (#14867)
• Removed the deprecated Trainer.run_stage in favor of Trainer.{fit,validate,test,predict} (#14870)
• Removed the deprecated SimpleProfiler.profile_iterable and AdvancedProfiler.profile_iterable attributes (#14864)
• Removed the deprecated Trainer.verbose_evaluate (#14884)
• Removed the deprecated Trainer.should_rank_save_checkpoint (#14885)
• Removed the deprecated TrainerOptimizersMixin (#14887)
• Removed the deprecated Trainer.lightning_optimizers (#14889)
• Removed the deprecated TrainerDataLoadingMixin (#14888)
• Removed the deprecated Trainer.call_hook in favor of Trainer._call_callback_hooks, Trainer._call_lightning_module_hook, Trainer._call_ttp_hook, and Trainer._call_accelerator_hook (#14869)
• Removed the deprecated Trainer.{validated,tested,predicted}_ckpt_path (#14897)
• Removed the deprecated device_stats_monitor_prefix_metric_keys (#14890)
• Removed the deprecated LightningDataModule.on_save/load_checkpoint hooks (#14909)
• Removed support for returning a value in Callback.on_save_checkpoint in favor of implementing Callback.state_dict (#14835)

</details>

<details><summary>Fixed</summary>

• Fixed an issue with LightningLite.setup() not setting the .device attribute correctly on the returned wrapper (#14822)
• Fixed an attribute error when running the tuner together with the StochasticWeightAveraging callback (#14836)
• Fixed MissingFieldException in offline mode for the NeptuneLogger() (#14919)
• Fixed wandb save_dir is overridden by None dir when using CLI (#14878)
• Fixed a missing call to LightningDataModule.load_state_dict hook while restoring checkpoint using LightningDataModule.load_from_checkpoint (#14883)
• Fixed torchscript error with containers of LightningModules (#14904)
• Fixed reloading of the last checkpoint on run restart (#14907)
• SaveConfigCallback instances should only save the config once to allow having the overwrite=False safeguard when using LightningCLI(..., run=False) (#14927)
• Fixed an issue with terminating the trainer profiler when a StopIteration exception is raised while using an IterableDataset (#14940)
• Do not update on-plateau schedulers when reloading from an end-of-epoch checkpoint (#14702)
• Fixed Trainer support for PyTorch built without distributed support (#14971)
• Fixed batch normalization statistics calculation in StochasticWeightAveraging callback (#14866)
• Avoided initializing optimizers during deepspeed inference (#14944)
• Fixed LightningCLI parse_env and description in subcommands (#15138)
• Fixed an exception that would occur when creating a multiprocessing.Pool after importing Lightning (#15292)
• Fixed a pickling error when using RichProgressBar together with checkpointing (#15319)
• Fixed the RichProgressBar crashing when used with distributed strategies (#15376)
• Fixed an issue with RichProgressBar not resetting the internal state for the sanity check progress (#15377)
• Fixed an issue with DataLoader re-instantiation when the attribute is an array and the default value of the corresponding argument changed (#15409)

</details>

Full commit list: https://github.com/PyTorchLightning/pytorch-lightning/compare/1.7.0...1.8.0

<a name="contributors"></a>

Contributors

Veteran

@akihironitta @ananthsub @AndresAlgaba @ar90n @Atharva-Phatak @awaelchli @BongYang @Borda @carmocca @dependabot @donlapark @ethanwharris @Felonious-Spellfire @hhsecond @jerome-habana @JustinGoheen @justusschock @kaushikb11 @krishnakalyan3 @krshrimali @luca-medeiros @manangoel99 @manskx @mauvilsa @MrShevan @nicolai86 @nmiculinic @otaj @Queuecumber @rlizzo @rohitgr7 @rschireman @SeanNaren @speediedan @tchaton @tshu-w

New

@Birch-san @clementpoiret @HalestormAI @thongonary @alecmerdler @adam-lightning @yurijmikhalevich @lijm1358 @robert-s-lee @panos-is @kacperlukawski @alro923 @dmitsf @Anner-deJong @cschell @nishantb06 @Callidior @j0rd1smit @MarcSkovMadsen @KralaBenjamin @robertomest @daniel347x @pierocor @datumbox @nohalon @pritamsoni-hsr @nandwalritik @gilfree @ritsuki1227 @christopher-nguyen-re @JulesGM @jgbos @dconathan @jsr-p @NeoKish @Blaizzy @suyash-811 @alexkuzmik @ziyadsheeba @geoffrey-g-delhomme @amrutha1098 @AlessioQuercia @ver217 @Helias @zxvix @1SAA @fabiofumarola @luca3rd @kimpty @PaulLerner @rbracco @wouterzwerink

If we forgot somebody or you have a suggestion, find support here :zap:

Did you know?

Chuck Norris can write functions of infinite recursion ... and have them return.