PyG 2.3.1 includes a variety of bugfixes.

Bug Fixes

• Fixed cugraph GNN layer support for pylibcugraphops==23.04 (#7023)
• Removed DeprecationWarning of TypedStorage usage in DataLoader (#7034)
• Fixed a bug in FastHGTConv that computed values via parameters used to compute the keys (#7050)
• Fixed numpy incompatiblity when reading files in Planetoid datasets (#7141)
• Fixed utils.subgraph on unordered inputs (#7187)
• Fixed support for Data.num_edges for native torch.sparse.Tensor adjacency matrices (#7104)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.3.0...2.3.1

We are thrilled to announce the release of PyG 2.3 🎉

• Highlights
• Breaking Changes
• Deprecations
• Features
• Bugfixes
• Full Changelog

PyG 2.3 is the culmination of work from 59 contributors who have worked on features and bug-fixes for a total of over 470 commits since torch-geometric==2.2.0.

Highlights

PyTorch 2.0 Support

PyG 2.3 is fully compatible with the next generation release of PyTorch, bringing many new innovations and features such as torch.compile() and Python 3.11 support to PyG out-of-the-box. In particular, many PyG models and functions are speeded up significantly using torch.compile() in torch >= 2.0.0.

We have prepared a full tutorial and a set of examples to get you going with torch.compile() immediately:

import torch_geometric
from torch_geometric.nn import GraphSAGE

model = GraphSAGE(in_channels, hidden_channels, num_layers, out_channels)
model = model.to(device)

model = torch_geometric.compile(model)

Overall, we observed runtime improvements of nearly up to 300%:

Model	Mode	Forward	Backward	Total	Speedup
GCN	Eager	2.6396s	2.1697s	4.8093s
GCN	Compiled	1.1082s	0.5896s	1.6978s	2.83x
GraphSAGE	Eager	1.6023s	1.6428s	3.2451s
GraphSAGE	Compiled	0.7033s	0.7465s	1.4498s	2.24x
GIN	Eager	1.6701s	1.6990s	3.3690s
GIN	Compiled	0.7320s	0.7407s	1.4727s	2.29x

Please note that torch.compile() within PyG is in beta mode and under active development. For example, currently torch.compile(model, dynamic=True) does not yet work seamlessly, but fixes are on its way. We are very eager to improve its support across the whole PyG code base, so do not hesitate to reach out if you notice anything unexpected.

Infrastructure Changes

With the recent upstreams of torch-scatter and torch-sparse to native PyTorch, we are happy to announce that any installation of the extension packages torch-scatter, torch-sparse, torch-cluster and torch-spline-conv is now fully optional.

All it takes to install PyG is now encapsulated into a single command

pip install torch-geometric

and finally resolves a lot of previous installation issues.

Extension packages are still picked up for the following use-cases (if installed):

• pyg-lib: Heterogeneous GNN operators and graph sampling routines like NeighborLoader
• torch-scatter: Accelerated "min" and "max" reductions
• torch-sparse: SparseTensor support
• torch-cluster: Graph clustering routines like knn or radius
• torch-spline-conv: SplineConv support

We recommend to start with a minimal installation, and only install additional dependencies once you actually get notified about them being missing during PyG usage.

Native PyTorch Sparse Tensor Support

With the recent additions of torch.sparse_csr_tensor and torch.sparse_csc_tensor classes and accelerated sparse matrix multiplication routines to PyTorch, we finally enable MessagePassing on pure PyTorch sparse tensors as well. In particular, you can now use torch.sparse_csr_tensor and torch.sparse_csc_tensor as a drop-in replacement for torch_sparse.SparseTensor:

from torch_geometric.nn import GCN
import torch_geometric.transforms as T
from torch_geometric.datasets import Planetoid

transform = T.ToSparseTensor(layout=torch.sparse_csr)
dataset = Planetoid("Planetoid", name="Cora", transform=transform)

model = GCN(in_channels, hidden_channels, num_layers=2)
model = model(data.x, data.adj_t)

Nearly all of the native PyG layers have been tested to work seamlessly with native PyTorch sparse tensors (#5906, #5944, #6003, #6033, #6514, #6532, #6748, #6847, #6868, #6874, #6897, #6930, #6932, #6936, #6937, #6939, #6947, #6950, #6951, #6957).

Explainability Framework

In PyG 2.2 we introduced the torch_geometric.explain package that provides a flexible interface to generate and visualize GNN explanations using various algorithms. We are happy to add the following key improvements on this front:

1. Support for explaining heterogeneous GNNs via HeteroExplanation
2. New visualization tools to visualize_feature_importance and to visualize_graph explanations
3. May new datasets and metrics to evaluate explanation algorithms
4. Several new explanation algorithms such as CaptumExplainer, PGExplainer, AttentionExplainer, PGMExplainer, and GraphMaskExplainer
5. Support for node-level, link-level and graph-level explanations

Using the new explainer interface is as simple as:

explainer = Explainer(
    model=model,
    algorithm=CaptumExplainer('IntegratedGradients'),
    explanation_type='model',
    model_config=dict(
        mode='multiclass_classification',
        task_level='node',
        return_type='log_probs',
    ),
    node_mask_type='attributes',
    edge_mask_type='object',
)

explanation = explainer(data.x, data.edge_index)

Read more about torch_geometric.explain in our newly added tutorial and example scripts. We also added a blog post that describes the new interface and functionality in depth.

Accelerations

Together with Intel and NVIDIA, we are excited about new PyG accelerations:

• [Experimental] Support for native cugraph-based GNN layers for commonly used layers such as CuGraphSAGEConv, CuGraphGATConv, and CuGraphRGCNConv (#6278, #6388, #6412):

  RGCN with neighbor sampling:

  Dataset	CuGraphRGCNConv (ms)	FastRGCNConv (ms)	RGCNConv (ms)
  AIFB	7,2	13,4	70
  BGS	7,1	8,8	146,9
  MUTAG	8,3	21,8	47,6
  AM	17,5	51	330,1

  Full-batch GAT:

  Dataset	CuGraphGATConv (ms)	GATConv (ms)
  Cora	7	8,7
  Citeseer	7	9
  Pubmed	8,2	11,4

  GraphSAGE with neighbor sampling:

  Dataset	CuGraphSAGEConv (ms)	SAGEConv (ms)
  ogbn-products	2591,8	3040,3

• A fast alternative to HGTConv via FastHGTConv that utilizes pyg-lib integration for improved runtimes. Overall, FastHGTConv achieves a speed-up of approximately 300% compared to the original implementation (#6178).

• A fast implementation of HeteroDictLinear that utilizes pyg-lib integration for improved runtimes (#6178).

• GNN inference and training optimizations on CPU within native PyTorch 2.0. Optimizations include:

  • scatter_reduce: performance hotspot in message passing when edge_index is stored in Coordinate format (COO).
  • gather: backward of scatter_reduce, specially tuned for the GNN compute when the index is an expanded tensor.
  • torch.sparse.mm with reduce flag: performance hotspot in message passing when the edge_index is stored in Compressed Sparse Row (CSR). Supported reduce flags are "sum", "mean", "amax" and "amin". On OGB benchmarks, a 1.12x - 4.07x performance speedup is measured (PyTorch 1.13.1 vs PyTorch 2.0) for single node inference and training.

• Introduction of index_sort via pyg-lib>=0.2.0, which implements a (way) faster alternative to sorting one-dimensional indices compared to torch.sort (#6554). Overall, this achieves speed-ups in dataset loading times by up to 600%.

• Introduction of AffinityMixin to accelerate PyG workflows on CPU. CPU affinity can be enabled via the AffinityMixin.enable_cpu_affinity() method for num_workers > 0 data loading use-cases, and will guarantee that a separate core is assigned to each worker at initialization. Over all benchmarked model/dataset samples, the average training time is decreased by up to 1.85x. We added an in-depth tutorial on how to speed-up your PyG workflows on CPU.

Additional Highlights

Documentation Revamp

The documentation has undergone a revision of design and structure, making it faster to load and easier to navigate. Take a look at its new design here.

Community Sprint: Improving Code Coverage

We had our third community sprint in the last two weeks of January. The goal was to improve code coverage by writing more thorough tests. Thanks to the efforts of many contributors, the total code coverage went from ~85% to ~92% (#6528, #6523, #6538, #6555, #6558, #6568, #6573, #6578, #6597, #6600, #6618, #6619, #6621, #6623, #6637, #6638, #6640, #6645, #6648, #6647, #6653, #6657, #6662, #6664, #6667, #6668, #6669, #6670, #6671, #6673, #6675, #6676, #6677, #6678, #6681, #6683, #6703, #6720, #6735, #6736, #6763, #6781, #6797, #6799, #6824, #6858)

Breaking Changes

• Temporal sampling in NeighborLoader will now also sample nodes with an equal timestamp to the seed time. Changed from sampling only nodes with a smaller timestamp (requires pyg-lib>=0.2.0) (#6517)
• Changed the interface and implementation of GraphMultisetTransformer such that GNN execution is no longer performed inside its module (#634))
• Unify Explanation.node_mask and Explanation.node_feat_mask into a single attribute in Explainer (#6267)
• Moved ExplainerConfig arguments to the Explainer class (#6176)
• Moved PyTorch Lightning data modules to torch_geometric.data.lightning (#6140)
• Removed target_index argument in the Explainer interface (#6270)
• Removed the Aggregation.set_validate_args option (#6175)
• Removed the usage of __dunder__ names in MessagePassing (#6999)

Deprecations

• The usage of datasets.BAShapes is now deprecated. Use the BAGraph graph generator to generate Barabasi-Albert graphs instead (#6072)

Features

Layers, Models and Examples

• Added the DenseGATConv layer (https://github.com/pyg-team/pytorch_geometric/pull/6928)
• Added the DistMult KGE model (https://github.com/pyg-team/pytorch_geometric/pull/6958)
• Added the ComplEx KGE model (#6898)
• Added the TransE KGE model (#6314)
• Added HeteroLayerNorm and HeteroBatchNorm layers (#6838)
• Added a TemporalEncoding module (#6785)
• Added the SimpleConv to perform non-trainable propagation (#6718)
• Added torch.jit examples for example/film.py and example/gcn.py (#6602)
• Added the AntiSymmetricConv layer (#6577)
• Added a PyGModelHubMixin for Huggingface model hub integration (#5930, #6591)
• Added the PGMExplainer (#6149, #6588, #6589)
• Added internal ToHeteroLinear and ToHeteroMessagePassing modules to accelerate to_hetero functionality (#5992, #6456)
• Added the GraphMaskExplainer (#6284)
• Added the GRBCDAttack and PRBCDAttack adversarial attack models (#5972)
• Added the CaptumExplainer (#6383, #6387, #6433, #6487)
• Added the GNNFF model (#5866)
• Added MLPAggregation, SetTransformerAggregation, GRUAggregation, and DeepSetsAggregation as adaptive readout functions (#6301, #6336, #6338) (https://github.com/pyg-team/pytorch_geometric/pull/6331), #6332)
• Added the GPSConv Graph Transformer layer (#6326, #6327)
• Added the PGExplainer (#6204)
• Added the AttentionExplainer (#6279)
• Added the PointGNNConv layer (#6194)
• Added the RandLA-Net architecture as classification and segmentation examples (#5117)

Datasets

• Added the AirfRANS dataset (#6287)
• Added the HeterophilousGraphDataset suite (#6846)
• Added support for the revised version of the MD17 dataset (#6734)
• Added the BAMultiShapesDataset (#6541)
• Added the Taobao dataset and a corresponding example (#6144)
• Added the Freebase FB15k_237 dataset (#3204)
• Added the BA2MotifDataset explainer dataset (#6257)
• Added the CycleMotif motif generator to generate n-node cycle shaped motifs (#6256)
• Added the InfectionDataset to evaluate explanations (#6222)
• Added a CustomMotif motif generator (#6179)
• Added the ERGraph graph generator to generate Ergos-Renyi (ER) graphs (#6073)
• Added a general ExplainerDataset to evaluate explanation methods (#6104)

Loaders

• Enabled NeighborLoader to return the number of sampled nodes and edges per hop, and added corresponding trim_to_layer functionality for more efficient NeighborLoader use-cases (#6661, #6834)
• Added a ZipLoader to execute multiple NodeLoader or LinkLoader instances (#6829)
• Added a seed_time attribute to temporal NodeLoader outputs in case input_time is given (#6196)

Transformations

• Added the Pad transformation (#5940, #6697, #6731, #6758)
• Added a RemoveDuplicatedEdges transformation (#6709)

General Improvements

• Added a memory-efficient utils.one_hot implementation (https://github.com/pyg-team/pytorch_geometric/pull/7005)
• Optimized utils.softmax implementation (#6113, #6155, #6805)
• Optimized topk implementation for graph pooling on large graphs (#6123)
• Added common utils.select and utils.narrow functionality to support filtering of both tensors and lists (#6162)
• Support normalization customization in get_mesh_laplacian (#6790)
• Added CPU-optimized spmm functionality via CSR format (#6699, #6759)
• Added TorchScript support to the RECT_L model (#6727)
• Added TorchScript support to the Node2Vec model (#6726)
• Added utils.to_edge_index to convert sparse tensors to edge indices and edge attributes (#6728)
• Added TorchScript support to the LINKX model (#6712)
• Added dropout option to GraphMultisetTransformer (#6484)
• Added option to customize loader arguments for evaluation in LightningNodeData and LightningLinkData (#6450, #6456)
• Added option to customize num_neighbors in NeighborSampler after instantiation (#6446)
• Added support to define a custom HeteroData mini-batch class in remote backends (#6377)
• Allow the usage of ChebConv within GNNExplainer (https://github.com/pyg-team/pytorch_geometric/pull/6778)
• Added Dataset.to_datapipe functionality for converting PyG datasets into a PyTorch DataPipe(#6141)
• Added to_nested_tensor and from_nested_tensor functionality (#6329, #6330, [#6331]
• Added networkit conversion utilities (#6321)
• Added Data.update() and HeteroData.update functionality (#6313)
• Added HeteroData.set_value_dict functionality (https://github.com/pyg-team/pytorch_geometric/pull/6961, https://github.com/pyg-team/pytorch_geometric/pull/6974)
• Added the (un)faithfulness explainability metric (#6090)
• Added the fidelity explainability metric (#6116, #6510)
• Added characterization_score and fidelity_curve_auc explainer metrics (#6188)
• Added subgraph visualization of GNN explanations (#6235, #6271)
• Added a weighted negative sampling option in LinkNeighborLoader (#6264)
• Added a get_embeddings function (#6201)
• Added Explanation.visualize_feature_importance to support node feature importance visualizations (#6094)
• Added heterogeneous graph support to explainers via HeteroExplanation (#6091, #6218)
• Added a summary method for PyG/PyTorch models (#5859, #6161)
• Added an input_time option to LightningNodeData and transform_sampler_output to NodeLoader and LinkLoader (#6187)
• Added Data.edge_subgraph and HeteroData.edge_subgraph functionalities (#6193)

Bug Fixes

• Fixed a bug in Data.subgraph() and HeteroData.subgraph() for bipartite graphs (#6613, #6654)
• Fixed a bug in PNAConv and DegreeScalerAggregation to correctly incorporate degree statistics of isolated nodes (#6609)
• Fixed a bug in which Data.to_heterogeneous filtered attributes in the wrong dimension (#6522)
• Fixed a bug in to_hetero when using an uninitialized submodule without implementing reset_parameters (#6863)
• Fixed a bug in get_mesh_laplacian (#6790)
• Fixed a bug in which masks were not properly masked in GNNExplainer on link prediction tasks (#6787)
• Fixed the ImbalancedSampler when operating on a sliced InMemoryDataset (#6374)
• Fixed the approximate PPR variant in transforms.GDC to not crash on graphs with isolated nodes (#6242)
• Fixed the filtering of node features in transforms.RemoveIsolatedNodes (#6308)
• Fixed a bug in DimeNet that causes an output dimension mismatch (#6305)
• Fixed Data.to_heterogeneous when used with an empty edge_index (#6304)
• Fixed a bug in the output order in HeteroLinear for un-sorted type vectors (#6198)

Full Changelog

<details> <summary><b>Added</b></summary>

• Added a memory-efficient utils.one_hot implementation (#7005)
• Added HeteroDictLinear and an optimized FastHGTConv module (#6178, #6998)
• Added the DenseGATConv module (#6928)
• Added trim_to_layer utility function for more efficient NeighborLoader use-cases (#6661)
• Added the DistMult KGE model (#6958)
• Added HeteroData.set_value_dict functionality (#6961, #6974)
• Added PyTorch >= 2.0 support (#6934, #7000)
• Added PyTorch Lightning >= 2.0 support (#6929)
• Added the ComplEx KGE model (#6898)
• Added option to write benchmark results to csv (#6888)
• Added HeteroLayerNorm and HeteroBatchNorm layers (#6838)
• Added the HeterophilousGraphDataset suite (#6846)
• Added support for sparse tensor in full batch mode inference benchmark (#6843)
• Enabled NeighborLoader to return number of sampled nodes and edges per hop (#6834)
• Added ZipLoader to execute multiple NodeLoader or LinkLoader instances (#6829)
• Added common utils.select and utils.narrow functionality to support filtering of both tensors and lists (#6162)
• Support normalization customization in get_mesh_laplacian (#6790)
• Added the TemporalEncoding module (#6785)
• Added CPU-optimized spmm_reduce functionality via CSR format (#6699, #6759)
• Added support for the revised version of the MD17 dataset (#6734)
• Added TorchScript support to the RECT_L model (#6727)
• Added TorchScript support to the Node2Vec model (#6726)
• Added utils.to_edge_index to convert sparse tensors to edge indices and edge attributes (#6728)
• Fixed expected data format in PolBlogs dataset (#6714)
• Added SimpleConv to perform non-trainable propagation (#6718)
• Added a RemoveDuplicatedEdges transform (#6709)
• Added TorchScript support to the LINKX model (#6712)
• Added torch.jit examples for example/film.py and example/gcn.py(#6602)
• Added Pad transform (#5940, #6697, #6731, #6758)
• Added full batch mode to the inference benchmark (#6631)
• Added cat aggregation type to the HeteroConv class so that features can be concatenated during grouping (#6634)
• Added torch.compile support and benchmark study (#6610, #6952, #6953, #6980, #6983, #6984, #6985, #6986, #6989, #7002)
• Added the AntiSymmetricConv layer (#6577)
• Added a mixin for Huggingface model hub integration (#5930, #6591)
• Added support for accelerated GNN layers in nn.conv.cugraph via cugraph-ops (#6278, #6388, #6412)
• Added accelerated index_sort function from pyg-lib for faster sorting (#6554)
• Fix incorrect device in EquilibriumAggregration (#6560)
• Added bipartite graph support in dense_to_sparse() (#6546)
• Add CPU affinity support for more data loaders (#6534, #6922)
• Added the BAMultiShapesDataset (#6541)
• Added the interfaces of a graph pooling framework (#6540)
• Added automatic n_id and e_id attributes to mini-batches produced by NodeLoader and LinkLoader (#6524)
• Added PGMExplainer to torch_geometric.contrib (#6149, #6588, #6589)
• Added a NumNeighbors helper class for specifying the number of neighbors when sampling (#6501, #6505, #6690)
• Added caching to is_node_attr() and is_edge_attr() calls (#6492)
• Added ToHeteroLinear and ToHeteroMessagePassing modules to accelerate to_hetero functionality (#5992, #6456)
• Added GraphMaskExplainer (#6284)
• Added the GRBCD and PRBCD adversarial attack models (#5972)
• Added dropout option to SetTransformer and GraphMultisetTransformer (#6484)
• Added option to customize loader arguments for evaluation in LightningNodeData and LightningLinkData (#6450, #6456)
• Added option to customize num_neighbors in NeighborSampler after instantiation (#6446)
• Added the Taobao dataset and a corresponding example for it (#6144)
• Added pyproject.toml (#6431)
• Added the torch_geometric.contrib sub-package (#6422)
• Warn on using latest documentation (#6418)
• Added basic pyright type checker support (#6415)
• Added a new external resource for link prediction (#6396)
• Added CaptumExplainer (#6383, #6387, #6433, #6487, #6966)
• Added support for custom HeteroData mini-batch class in remote backends (#6377)
• Added the GNNFF model (#5866)
• Added MLPAggregation, SetTransformerAggregation, GRUAggregation, and DeepSetsAggregation as adaptive readout functions (#6301, #6336, #6338)
• Added Dataset.to_datapipe for converting PyG datasets into a torchdata DataPipe(#6141)
• Added to_nested_tensor and from_nested_tensor functionality (#6329, #6330, #6331, #6332)
• Added the GPSConv Graph Transformer layer and example (#6326, #6327)
• Added networkit conversion utilities (#6321)
• Added global dataset attribute access via dataset.{attr_name} (#6319)
• Added the TransE KGE model and example (#6314)
• Added the Freebase FB15k_237 dataset (#3204)
• Added Data.update() and HeteroData.update() functionality (#6313)
• Added PGExplainer (#6204)
• Added the AirfRANS dataset (#6287)
• Added AttentionExplainer (#6279)
• Added (un)faithfulness explainability metric (#6090)
• Added fidelity explainability metric (#6116, #6510)
• Added subgraph visualization of GNN explanations (#6235, #6271)
• Added weighted negative sampling option in LinkNeighborLoader (#6264)
• Added the BA2MotifDataset explainer dataset (#6257)
• Added CycleMotif motif generator to generate n-node cycle shaped motifs (#6256)
• Added the InfectionDataset to evaluate explanations (#6222)
• Added characterization_score and fidelity_curve_auc explainer metrics (#6188)
• Added get_message_passing_embeddings (#6201)
• Added the PointGNNConv layer (#6194)
• Added GridGraph graph generator to generate grid graphs (#6220
• Added explainability metrics for when ground truth is available (#6137)
• Added visualize_feature_importance to support node feature visualizations (#6094)
• Added heterogeneous graph support to Explanation framework (#6091, #6218)
• Added a CustomMotif motif generator (#6179)
• Added ERGraph graph generator to generate Ergos-Renyi (ER) graphs (#6073)
• Added BAGraph graph generator to generate Barabasi-Albert graphs - the usage of datasets.BAShapes is now deprecated (#6072
• Added explainability benchmark dataset framework (#6104)
• Added seed_time attribute to temporal NodeLoader outputs in case input_time is given (#6196)
• Added Data.edge_subgraph and HeteroData.edge_subgraph functionalities (#6193)
• Added input_time option to LightningNodeData and transform_sampler_output to NodeLoader and LinkLoader (#6187)
• Added summary for PyG/PyTorch models (#5859, #6161)
• Started adding torch.sparse support to PyG (#5906, #5944, #6003, #6033, #6514, #6532, #6748, #6847, #6868, #6874, #6897, #6930, #6932, #6936, #6937, #6939, #6947, #6950, #6951, #6957)
• Add inputs_channels back in training benchmark (#6154)
• Added support for dropping nodes in utils.to_dense_batch in case max_num_nodes is smaller than the number of nodes (#6124)
• Added the RandLA-Net architecture as an example (#5117)

</details>

<details> <summary><b>Changed</b></summary>

• Drop internal usage of __dunder__ names (#6999)
• Changed the interface of sort_edge_index, coalesce and to_undirected to only return single edge_index information in case the edge_attr argument is not specified (#6875, #6879, #6893)
• Fixed a bug in to_hetero when using an uninitialized submodule without implementing reset_parameters (#6863)
• Fixed a bug in get_mesh_laplacian (#6790)
• Fixed a bug in which masks were not properly masked in GNNExplainer on link prediction tasks (#6787)
• Allow the usage of ChebConv within GNNExplainer (#6778)
• Allow setting the EdgeStorage.num_edges property (#6710)
• Fixed a bug in utils.bipartite_subgraph() and updated docs of HeteroData.subgraph() (#6654)
• Properly reset the data_list cache of an InMemoryDataset when accessing dataset.data (#6685)
• Fixed a bug in Data.subgraph() and HeteroData.subgraph() (#6613)
• Fixed a bug in PNAConv and DegreeScalerAggregation to correctly incorporate degree statistics of isolated nodes (#6609)
• Improved code coverage (#6523, #6538, #6555, #6558, #6568, #6573, #6578, #6597, #6600, #6618, #6619, #6621, #6623, #6637, #6638, #6640, #6645, #6648, #6647, #6653, #6657, #6662, #6664, #6667, #6668, #6669, #6670, #6671, #6673, #6675, #6676, #6677, #6678, #6681, #6683, #6703, #6720, #6735, #6736, #6763, #6781, #6797, #6799, #6824, #6858)
• Fixed a bug in which data.to_heterogeneous() filtered attributs in the wrong dimension (#6522)
• Breaking Change: Temporal sampling will now also sample nodes with an equal timestamp to the seed time (requires pyg-lib>0.1.0) (#6517)
• Changed DataLoader workers with affinity to start at cpu0 (#6512)
• Allow 1D input to global_*_pool functions (#6504)
• Add information about dynamic shapes in RGCNConv (#6482)
• Fixed the use of types removed in numpy 1.24.0 (#6495)
• Fixed keyword parameters in examples/mnist_voxel_grid.py (#6478)
• Unified LightningNodeData and LightningLinkData code paths (#6473)
• Allow indices with any integer type in RGCNConv (#6463)
• Re-structured the documentation (#6420, #6423, #6429, #6440, #6443, #6445, #6452, #6453, #6458, #6459, #6460, #6490, #6491, #6693, #6744)
• Fix the default arguments of DataParallel class (#6376)
• Fix ImbalancedSampler on sliced InMemoryDataset (#6374)
• Breaking Change: Changed the interface and implementation of GraphMultisetTransformer (#6343)
• Fixed the approximate PPR variant in transforms.GDC to not crash on graphs with isolated nodes (#6242)
• Added a warning when accesing InMemoryDataset.data (#6318)
• Drop SparseTensor dependency in GraphStore (#5517)
• Replace NeighborSampler with NeighborLoader in the distributed sampling example (#6204)
• Fixed the filtering of node features in transforms.RemoveIsolatedNodes (#6308)
• Fixed a bug in DimeNet that causes a output dimension mismatch (#6305)
• Fixed Data.to_heterogeneous() with empty edge_index (#6304)
• Unify Explanation.node_mask and Explanation.node_feat_mask (#6267)
• Moved thresholding config of the Explainer to Explanation (#6215)
• Fixed a bug in the output order in HeteroLinear for un-sorted type vectors (#6198)
• Breaking Change: Move ExplainerConfig arguments to the Explainer class (#6176)
• Refactored NeighborSampler to be input-type agnostic (#6173)
• Infer correct CUDA device ID in profileit decorator (#6164)
• Correctly use edge weights in GDC example (#6159)
• Breaking Change: Moved PyTorch Lightning data modules to torch_geometric.data.lightning (#6140)
• Make torch_sparse an optional dependency (#6132, #6134, #6138, #6139)
• Optimized utils.softmax implementation (#6113, #6155, #6805)
• Optimized topk implementation for large enough graphs (#6123)

</details>

<details> <summary><b>Removed</b></summary>

• torch-sparse is now an optional dependency (#6625, #6626, #6627, #6628, #6629, #6630)
• Removed most of the torch-scatter dependencies (#6394, #6395, #6399, #6400, #6615, #6617)
• Removed the deprecated classes GNNExplainer and Explainer from nn.models (#6382)
• Removed target_index argument in the Explainer interface (#6270)
• Removed Aggregation.set_validate_args option (#6175)

</details>

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.2.0...2.3.0

New Contributors

• @CharlesGaydon made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5117
• @mova made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6123
• @Humbertzhang made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6143
• @dongyukang1 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6159
• @edwag made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6183
• @toenshoff made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6198
• @shenoynikhil made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6073
• @andreasbinder made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6194
• @ZeynepP made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6090
• @karimsr4 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6279
• @FlorentExtrality made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6287
• @anton-bushuiev made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6306
• @marekdedic made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6242
• @davidbuterez made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6301
• @ken2403 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5866
• @bwroblew made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6376
• @manangoel99 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6396
• @binlee52 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6439
• @bartekxk made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6463
• @forest1040 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6478
• @sigeisler made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5972
• @steveazzolin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6541
• @JinL0 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6557
• @HelgeS made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6560
• @jaypmorgan made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6562
• @jreniecki made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6570
• @tingyu66 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6388
• @gravins made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6577
• @karuna-bhaila made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6597
• @DylanSand made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6634
• @soumik12345 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6641
• @sivonxay made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6648
• @kimfalk made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6651
• @dsciebu made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6710
• @bmarenco made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6715
• @RobDHess made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6734
• @cemunds made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6810
• @berkekisin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6838
• @varun-tandon made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6904

We are excited to announce the release of PyG 2.2 🎉🎉🎉

• Highlights
• Breaking Changes
• Deprecations
• Features
• Bugfixes
• Full Changelog

PyG 2.2 is the culmination of work from 78 contributors who have worked on features and bug-fixes for a total of over 320 commits since torch-geometric==2.1.0.

Highlights

pyg-lib Integration

We are proud to release and integrate pyg-lib==0.1.0 into PyG, the first stable version of our new low-level Graph Neural Network library to drive all CPU and GPU acceleration needs of PyG (#5330, #5347, #5384, #5388).

You can install pyg-lib as described in our README.md:

pip install pyg-lib -f https://data.pyg.org/whl/torch-${TORCH}+${CUDA}.html

import pyg_lib

Once pyg-lib is installed, it will get automatically picked up by PyG, e.g., to accelerate neighborhood sampling routines or to accelerate heterogeneous GNN execution:

• pyg-lib provides fast and optimized CPU routines to iteratively sample neighbors in homogeneous and heterogeneous graphs, and heavily improves upon the previously used neighborhood sampling techniques utilized in PyG.

• pyg-lib provides efficient GPU-based routines to parallelize workloads in heterogeneous graphs across different node types and edge types. We achieve this by leveraging type-dependent transformations via NVIDIA CUTLASS integration, which is flexible to implement most heterogeneous GNNs with, and efficient, even for sparse edge types or a large number of different node types.

GraphStore and FeatureStore Abstractions

PyG 2.2 includes numerous primitives to easily integrate with simple paradigms for scalable graph machine learning, enabling users to train GNNs on graphs far larger than the size of their machine's available memory. It does so by introducing simple, easy-to-use, and extensible abstractions of a FeatureStore and a GraphStore that plug directly into existing familiar PyG interfaces (see here for the accompanying tutorial).

feature_store = CustomFeatureStore()
feature_store['paper', 'x', None] = ...  # Add paper features
feature_store['author', 'x', None] = ...  # Add author features

graph_store = CustomGraphStore()
graph_store['edge', 'coo'] = ...  # Add edges in "COO" format

# `CustomGraphSampler` knows how to sample on `CustomGraphStore`:
graph_sampler = CustomGraphSampler(
    graph_store=graph_store,
    num_neighbors=[10, 20],
    ...
)

from torch_geometric.loader import NodeLoader
loader = NodeLoader(
    data=(feature_store, graph_store),
    node_sampler=graph_sampler,
    batch_size=20,
    input_nodes='paper',
)

for batch in loader:
    pass

Data loading and sampling routines are refactored and decomposed into torch_geometric.loader and torch_geometric.sampler modules, respectively (#5563, #5820, #5456, #5457, #5312, #5365, #5402, #5404, #5418).

Optimized and Fused Aggregations

PyG 2.2 further accelerates scatter aggregations based on CPU/GPU and with/without backward computation paths (requires torch>=1.12.0 and torch-scatter>=2.1.0) (#5232, #5241, #5353, #5386, #5399, #6051, #6052).

We also optimized the usage of nn.aggr.MultiAggregation by fusing the computation of multiple aggregations together (see here for more details) (#6036, #6040).

Here are some benchmarking results on PyTorch 1.12 (summed over 1000 runs):

Aggregators	Vanilla	Fusion
[sum, mean]	0.3325s	0.1996s
[sum, mean, min, max]	0.7139s	0.5037s
[sum, mean, var]	0.6849s	0.3871s
[sum, mean, var, std]	1.0955s	0.3973s

Lastly, we have incorporated "fused" GNN operators via the dgNN package, starting with a FusedGATConv implementation (#5140).

Community Sprint: Type Hints and TorchScript Support

We are running regular community sprints to get our community more involved in building PyG. Whether you are just beginning to use graph learning or have been leveraging GNNs in research or production, the community sprints welcome members of all levels with different types of projects.

We had our first community sprint on 10/12 to fully-incorporate type hints and TorchScript support over the entire code base. The goal was to improve usability and cleanliness of our codebase. We had 20 contributors participating, contributing to 120 type hints within 2 weeks, adding around 2400 lines of code (#5842, #5603, #5659, #5664, #5665, #5666, #5667, #5668, #5669, #5673, #5675, #5673, #5678, #5682, #5683, #5684, #5685, #5687, #5688, #5695, #5699, #5701, #5702, #5703, #5706, #5707, #5710, #5714, #5715, #5716, #5722, #5724, #5725, #5726, #5729, #5730, #5731, #5732, #5733, #5743, #5734, #5735, #5736, #5737, #5738, #5747, #5752, #5753, #5754, #5756, #5757, #5758, #5760, #5766, #5767, #5768, #5781, #5778, #5797, #5798, #5799, #5800, #5806, #5810, #5811, #5828, #5847, #5851, #5852).

Explainability

Our second community sprint began on 11/15 with the goal to improve the explainability capabilities of PyG. With this, we introduce the torch_geometric.explain module to provide a unified set of tools to explain the predictions of a PyG model or to explain the underlying phenomenon of a dataset.

Some of the features developed in the sprint are incorporated into this release:

• Added the torch_geometric.explain module (#5804, #6054, #6089)
• Moved and adapted the GNNExplainer module to torch_geometric.explain (#5967, #6065). See here and here for the accompanying examples.
• Extended GNNExplainer to support edge level explanations (#6056)
• Added explainability support for heterogeneous GNNs via to_captum_model and to_captum_input (#5886, #5934)

data = HeteroData(...)
model = HeteroGNN(...)

# Explain predictions on heterogenenous graphs for output node 10:
captum_model = to_captum_model(model, mask_type, output_idx, metadata)
inputs, additional_forward_args = to_captum_input(data.x_dict, data.edge_index_dict, mask_type)

ig = IntegratedGradients(captum_model)
ig_attr = ig.attribute(
    inputs=inputs,
    target=int(y[output_idx]),
    additional_forward_args=additional_forward_args,
    internal_batch_size=1,
)

Breaking Changes

• Renamed drop_unconnected_nodes to drop_unconnected_node_types and drop_orig_edges to drop_orig_edge_types in AddMetapaths (#5490)

Deprecations

• The usage of nn.models.GNNExplainer is now deprecated in favor of explain.GNNExplainer
• The usage of utils.dropout_adj is now deprecated in favor of utils.dropout_edge
• The usage of loader.RandomNodeSampler is now deprecated in favor of loader.RandomNodeLoader
• The usage of to_captum is now deprecated in favor of to_captum_model.

Features

Layers, Models and Examples

• Added a "Link Prediction on MovieLens" Colab notebook (#5823)
• Added a bipartite link-prediction example (#5834)
• Added the SSGConv layer (#5599)
• Added the WLConvContinuous layer for performing WL-refinement with continuous attributes (#5316)
• Added the PositionalEncoding module (#5381)
• Added a node classification example instrumented with Weights and Biases (#5192)

Data Loaders

• Added support for triplet sampling in LinkNeighborLoader (#6004)
• Added temporal_strategy = uniform/last option to NeighborLoader and LinkNeighborLoader (#5576)
• Added a disjoint option to NeighborLoader and LinkNeighborLoader (#5717, #5775)
• Added HeteroData support in RandomNodeLoader (#6007
• Added int32-based edge_index support in NeighborLoader (#5948)
• Added support for input_time in NeighborLoader (#5763)
• Added np.memmap support in NeighborLoader (#5696)
• Added CPU affinitization support to NeighborLoader (#6005)

Transformations

• Added a FeaturePropagation transform (#5387)
• Added IndexToMask and MaskToIndex transforms (#5375, #5455)
• Added shuffle_node, mask_feature and add_random_edge augmentations (#5548)
• Added dropout_node, dropout_edge and dropout_path augmentations (#5481, #5495, #5531)
• Added a AddRandomMetaPaths transform that adds edges based on random walks along a metapath (#5397)
• Added a utils.to_smiles function (#6038)
• Added HeteroData support for transforms.Constant (#5700)

Datasets

• Added the LRGBDataset to include 5 datasets from the Long Range Graph Benchmark (#5935)
• Added the HydroNet water cluster dataset (#5537, #5902, #5903)
• Added the DGraphFin dynamic graph dataset (#5504)
• Added the official splits to the MalNetTiny dataset (#5078)
• Added a print_summary method to torch_geometric.data.Dataset (#5438)

General Improvements

• Added training and inference benchmark scripts (#5774, #5830, #5878, #5293, #5341, #5242, #5258, #5881, #5254)
• Added the utils.assortativity function to compute the degree assortativity coefficient (#5587)
• Add support for filling labels with dummy values in HeteroData.to_homogeneous() (#5540)
• Added torch.onnx.export support (see here for an example) (#5877, #5997)
• Added option to make normalization coefficients trainable in PNAConv (#6039)
• Added a semi_grad option in VarAggregation and StdAggregation (#6042)
• Added a warning for invalid node and edge type names in HeteroData (#5990)
• Added lr_scheduler_solver and customized lr_scheduler classes (#5942)
• Added to_fixed_size graph transformer (#5939)
• Added support for symbolic tracing in the SchNet model (#5938)
• Added support for customizing the interaction graph in the SchNet model (#5919)
• Added SparseTensor support to SuperGATConv (#5888)
• Added TorchScript support for AttentiveFP (#5868)
• Added a return_semantic_attention_weights argument HANConv (#5787)
• Added temperature value customization in dense_mincut_pool (#5908)
• Added support for a tuple of in_channels in GENConv for bipartite message passing (#5627, #5641)
• Added Aggregation.set_validate_args option to skip validation of dim_size (#5290)
• Added BaseStorage.get() functionality (#5240)
• Added support for batches of size one in BatchNorm (#5530, #5614)
• The AttentionalAggregation module can now be applied to compute attention on a per-feature level (#5449)
• Added TorchScript support to ASAPooling (#5395)
• Updated the unsupervised GraphSAGE example to leverage LinkNeighborLoader (#5317)
• Added better out-of-bounds error message in MessagePassing (#5339)
• Added support to customize the activation function in PNAConv (#5262)

Bugfixes

• Fixed a bug in TUDataset, in which node features were wrongly constructed whenever node_attributes only hold a single feature (e.g., in PROTEINS) (#5441)
• Fixed a bug in the VirtualNode transform, in which node features were mistakenly treated as edge features (#5819)
• Fixed a bug when applying several scalers with PNAConv (#5514)
• Fixed setter and getter handling in BaseStorage (#5815)
• Fixed the auto_select_device routine in GraphGym for pytorch_lightning>=1.7 (#5677)
• Fixed RandomLinkSplit in case there aren't enough negative edges to sample (#5642)
• Fixed the in-place modification to mode_kwargs in MultiAggregation (#5601)
• Fixed the utils.to_dense_adj routine in case edge_index is empty (#5476)
• Fixed the PointTransformerConv to now correctly use sum aggregation (#5332)
• Fixed the output of InMemoryDataset.num_classes in case a transform modifies data.y (#5274)
• Fail gracefully on GLIBC errors within torch-spline-conv (#5276)

Full Changelog

<details> <summary><b>Added</b></summary>

• Extended GNNExplainer to support edge level explanations (#6056)
• Added CPU affinitization for NodeLoader (#6005)
• Added triplet sampling in LinkNeighborLoader (#6004)
• Added FusedAggregation of simple scatter reductions (#6036)
• Added a to_smiles function (#6038)
• Added option to make normalization coefficients trainable in PNAConv (#6039)
• Added semi_grad option in VarAggregation and StdAggregation (#6042)
• Allow for fused aggregations in MultiAggregation (#6036, #6040)
• Added HeteroData support for to_captum_model and added to_captum_input (#5934)
• Added HeteroData support in RandomNodeLoader (#6007)
• Added bipartite GraphSAGE example (#5834)
• Added LRGBDataset to include 5 datasets from the Long Range Graph Benchmark (#5935)
• Added a warning for invalid node and edge type names in HeteroData (#5990)
• Added PyTorch 1.13 support (#5975)
• Added int32 support in NeighborLoader (#5948)
• Add dgNN support and FusedGATConv implementation (#5140)
• Added lr_scheduler_solver and customized lr_scheduler classes (#5942)
• Add to_fixed_size graph transformer (#5939)
• Add support for symbolic tracing of SchNet model (#5938)
• Add support for customizable interaction graph in SchNet model (#5919)
• Started adding torch.sparse support to PyG (#5906, #5944, #6003)
• Added HydroNet water cluster dataset (#5537, #5902, #5903)
• Added explainability support for heterogeneous GNNs (#5886)
• Added SparseTensor support to SuperGATConv (#5888)
• Added TorchScript support for AttentiveFP (#5868)
• Added num_steps argument to training and inference benchmarks (#5898)
• Added torch.onnx.export support (#5877, #5997)
• Enable VTune ITT in inference and training benchmarks (#5830, #5878)
• Add training benchmark (#5774)
• Added a "Link Prediction on MovieLens" Colab notebook (#5823)
• Added custom sampler support in LightningDataModule (#5820)
• Added a return_semantic_attention_weights argument HANConv (#5787)
• Added disjoint argument to NeighborLoader and LinkNeighborLoader (#5775)
• Added support for input_time in NeighborLoader (#5763)
• Added disjoint mode for temporal LinkNeighborLoader (#5717)
• Added HeteroData support for transforms.Constant (#5700)
• Added np.memmap support in NeighborLoader (#5696)
• Added assortativity that computes degree assortativity coefficient (#5587)
• Added SSGConv layer (#5599)
• Added shuffle_node, mask_feature and add_random_edge augmentation methdos (#5548)
• Added dropout_path augmentation that drops edges from a graph based on random walks (#5531)
• Add support for filling labels with dummy values in HeteroData.to_homogeneous() (#5540)
• Added temporal_strategy option to neighbor_sample (#5576)
• Added torch_geometric.sampler package to docs (#5563)
• Added the DGraphFin dynamic graph dataset (#5504)
• Added dropout_edge augmentation that randomly drops edges from a graph - the usage of dropout_adj is now deprecated (#5495)
• Added dropout_node augmentation that randomly drops nodes from a graph (#5481)
• Added AddRandomMetaPaths that adds edges based on random walks along a metapath (#5397)
• Added WLConvContinuous for performing WL refinement with continuous attributes (#5316)
• Added print_summary method for the torch_geometric.data.Dataset interface (#5438)
• Added sampler support to LightningDataModule (#5456, #5457)
• Added official splits to MalNetTiny dataset (#5078)
• Added IndexToMask and MaskToIndex transforms (#5375, #5455)
• Added FeaturePropagation transform (#5387)
• Added PositionalEncoding (#5381)
• Consolidated sampler routines behind torch_geometric.sampler, enabling ease of extensibility in the future (#5312, #5365, #5402, #5404), #5418)
• Added pyg-lib neighbor sampling (#5384, #5388)
• Added pyg_lib.segment_matmul integration within HeteroLinear (#5330, #5347))
• Enabled bf16 support in benchmark scripts (#5293, #5341)
• Added Aggregation.set_validate_args option to skip validation of dim_size (#5290)
• Added SparseTensor support to inference and training benchmark suite (#5242, #5258, #5881)
• Added experimental mode in inference benchmarks (#5254)
• Added node classification example instrumented with Weights and Biases (W&B) logging and W&B Sweeps (#5192)
• Added experimental mode for utils.scatter (#5232, #5241, #5386)
• Added missing test labels in HGBDataset (#5233)
• Added BaseStorage.get() functionality (#5240)
• Added a test to confirm that to_hetero works with SparseTensor (#5222)
• Added torch_geometric.explain module with base functionality for explainability methods (#5804, #6054, #6089)

</details>

<details> <summary><b>Changed</b></summary>

• Moved and adapted GNNExplainer from torch_geometric.nn to torch_geometric.explain.algorithm (#5967, #6065)
• Optimized scatter implementations for CPU/GPU, both with and without backward computation (#6051, #6052)
• Support temperature value in dense_mincut_pool (#5908)
• Fixed a bug in which VirtualNode mistakenly treated node features as edge features (#5819)
• Fixed setter and getter handling in BaseStorage (#5815)
• Fixed path in hetero_conv_dblp.py example (#5686)
• Fix auto_select_device routine in GraphGym for PyTorch Lightning>=1.7 (#5677)
• Support in_channels with tuple in GENConv for bipartite message passing (#5627, #5641)
• Handle cases of not having enough possible negative edges in RandomLinkSplit (#5642)
• Fix RGCN+pyg-lib for LongTensor input (#5610)
• Improved type hint support (#5842, #5603, #5659, #5664, #5665, #5666, #5667, #5668, #5669, #5673, #5675, #5673, #5678, #5682, #5683, #5684, #5685, #5687, #5688, #5695, #5699, #5701, #5702, #5703, #5706, #5707, #5710, #5714, #5715, #5716, #5722, #5724, #5725, #5726, #5729, #5730, #5731, #5732, #5733, #5743, #5734, #5735, #5736, #5737, #5738, #5747, #5752, #5753, #5754, #5756, #5757, #5758, #5760, #5766, #5767, #5768), #5781, #5778, #5797, #5798, #5799, #5800, #5806, #5810, #5811, #5828, #5847, #5851, #5852)
• Avoid modifying mode_kwargs in MultiAggregation (#5601)
• Changed BatchNorm to allow for batches of size one during training (#5530, #5614)
• Integrated better temporal sampling support by requiring that local neighborhoods are sorted according to time (#5516, #5602)
• Fixed a bug when applying several scalers with PNAConv (#5514)
• Allow . in ParameterDict key names (#5494)
• Renamed drop_unconnected_nodes to drop_unconnected_node_types and drop_orig_edges to drop_orig_edge_types in AddMetapaths (#5490)
• Improved utils.scatter performance by explicitly choosing better implementation for add and mean reduction (#5399)
• Fix to_dense_adj with empty edge_index (#5476)
• The AttentionalAggregation module can now be applied to compute attentin on a per-feature level (#5449)
• Ensure equal lenghts of num_neighbors across edge types in NeighborLoader (#5444)
• Fixed a bug in TUDataset in which node features were wrongly constructed whenever node_attributes only hold a single feature (e.g., in PROTEINS) (#5441)
• Breaking change: removed num_neighbors as an attribute of loader (#5404)
• ASAPooling is now jittable (#5395)
• Updated unsupervised GraphSAGE example to leverage LinkNeighborLoader (#5317)
• Replace in-place operations with out-of-place ones to align with torch.scatter_reduce API (#5353)
• Breaking bugfix: PointTransformerConv now correctly uses sum aggregation (#5332)
• Improve out-of-bounds error message in MessagePassing (#5339)
• Allow file names of a Dataset to be specified as either property and method (#5338)
• Fixed separating a list of SparseTensor within InMemoryDataset (#5299)
• Improved name resolving of normalization layers (#5277)
• Fail gracefully on GLIBC errors within torch-spline-conv (#5276)
• Fixed Dataset.num_classes in case a transform modifies data.y (#5274)
• Allow customization of the activation function within PNAConv (#5262)
• Do not fill InMemoryDataset cache on dataset.num_features (#5264)
• Changed tests relying on dblp datasets to instead use synthetic data (#5250)
• Fixed a bug for the initialization of activation function examples in custom_graphgym (#5243)
• Allow any integer tensors when checking edge_index input to message passing (5281)

</details>

<details> <summary><b>Removed</b></summary>

• Removed scatter_reduce option from experimental mode (#5399)

</details>

Full commit list: https://github.com/pyg-team/pytorch_geometric/compare/2.1.0...2.2.0

We are excited to announce the release of PyG 2.1.0 🎉🎉🎉

• Highlights
• Breaking Changes
• Deprecations
• Features
• Bugfixes
• Full Changelog

PyG 2.1.0 is the culmination of work from over 60 contributors who have worked on features and bug-fixes for a total of over 320 commits since torch-geometric==2.0.4.

Highlights

Principled Aggregations

See here for the accompanying tutorial.

Aggregation functions play an important role in the message passing framework and the readout functions of Graph Neural Networks. Specifically, many works in the literature (Hamilton et al. (2017), Xu et al. (2018), Corso et al. (2020), Li et al. (2020), Tailor et al. (2021), Bartunov et al. (2022)) demonstrate that the choice of aggregation functions contributes significantly to the representational power and performance of the model.

To facilitate further experimentation and unify the concepts of aggregation within GNNs across both MessagePassing and global readouts, we have made the concept of Aggregation a first-class principle in PyG (#4379, #4522, #4687, #4721, #4731, #4762, #4749, #4779, #4863, #4864, #4865, #4866, #4872, #4927, #4934, #4935, #4957, #4973, #4973, #4986, #4995, #5000, #5021, #5034, #5036, #5039, #4522, #5033, #5085, #5097, #5099, #5104, #5113, #5130, #5098, #5191). As of now, PyG provides support for various aggregations — from simple ones (e.g., mean, max, sum), to advanced ones (e.g., median, var, std), learnable ones (e.g., SoftmaxAggregation, PowerMeanAggregation), and exotic ones (e.g., LSTMAggregation, SortAggregation, EquilibriumAggregation). Furthermore, multiple aggregations can be combined and stacked together:

from torch_geometric.nn import MessagePassing, SoftmaxAggregation

class MyConv(MessagePassing):
    def __init__(self, ...):
        # Combines a set of aggregations and concatenates their results.
        # The interface also supports automatic resolution.
        super().__init__(aggr=['mean', 'std', SoftmaxAggregation(learn=True)])

Link-level Neighbor Loader

We added a new LinkNeighborLoader class for training scalable GNNs that perform edge-level predictions on giant graphs (#4396, #4439, #4441, #4446, #4508, #4509, #4868). LinkNeighborLoader comes with automatic support for both homogeneous and heterogenous data, and supports link prediction via automatic negative sampling as well as edge-level classification and regression models:

from torch_geometric.loader import LinkNeighborLoader

loader = LinkNeighborLoader(
    data,
    num_neighbors=[30] * 2,  # Sample 30 neighbors for each node for 2 iterations
    batch_size=128,  # Use a batch size of 128 for sampling training links
    edge_label_index=data.edge_index,  # Use the entire graph for supervision
    negative_sampling_ratio=1.0,  # Sample negative edges
)

sampled_data = next(iter(loader))
print(sampled_data)
>>> Data(x=[1368, 1433], edge_index=[2, 3103], edge_label_index=[2, 256], edge_label=[256])

Neighborhood Sampling based on Temporal Constraints

Both NeighborLoader and LinkNeighborLoader now support temporal sampling via the time_attr argument (#4025, #4877, #4908, #5137, #5173). If set, temporal sampling will be used such that neighbors are guaranteed to fulfill temporal constraints, i.e. neighbors have an earlier timestamp than the center node:

from torch_geometric.loader import NeighborLoader

data['paper'].time = torch.arange(data['paper'].num_nodes)

loader = NeighborLoader(
    data,
    input_nodes='paper',
    time_attr='time',  # Only sample papers that appeared before the seed paper
    num_neighbors=[30] * 2,
    batch_size=128,
)

Note that this feature requires torch-sparse>=0.6.14.

Functional DataPipes

See here for the accompanying example.

PyG now fully supports data loading using the newly introduced concept of DataPipes in PyTorch for easily constructing flexible and performant data pipelines (#4302, #4345, #4349). PyG provides DataPipe support for batching multiple PyG data objects together and for applying any PyG transform:

datapipe = FileOpener(['SMILES_HIV.csv'])
datapipe = datapipe.parse_csv_as_dict()
datapipe = datapipe.parse_smiles(target_key='HIV_active')
datapipe = datapipe.in_memory_cache()  # Cache graph instances in-memory.
datapipe = datapipe.shuffle()
datapipe = datapipe.batch_graphs(batch_size=32)

datapipe = FileLister([root_dir], masks='*.off', recursive=True)
datapipe = datapipe.read_mesh()
datapipe = datapipe.in_memory_cache()  # Cache graph instances in-memory.
datapipe = datapipe.sample_points(1024)  # Use PyG transforms from here.
datapipe = datapipe.knn_graph(k=8)
datapipe = datapipe.shuffle()
datapipe = datapipe.batch_graphs(batch_size=32)

Breaking Changes

• The torch_geometric.utils.metric package has been removed. We now recommend to use the torchmetrics package instead.
• len(batch) of the data.Batch class will now return the number of graphs inside the batch, not the number of attributes (#4931)

Deprecations

• The usage of the torch_geometric.nn.glob package is now deprecated in favor of torch_geometric.nn.aggr
• The usage of RandomTranslate is now deprecated in favor of RandomJitter (#4828)

Features

Layers, Models and Examples

• Added the GroupAddRev module with support for reducing training GPU memory (#4671, #4701, #4715, #4730) [Example]
• Added the MaskLabel module for performing masked label propagation (#4197) **[Example]
• Added the DimeNetPlusPlus module (#4432, #4699, #4700, #4800)
• Added the MeanSubtractionNorm module (#5068)
• Added the DynamicBatchSampler for filling a mini-batch with a variable number of samples up to a maximum size (#4972)
• Added PyTorch Lightning support in GraphGym (#4511, #4516 #4531, #4689, #4843)
• Added an example of using PyG with PyTorch Ignite (#4487)
• Added an example for unsupervised heterogeneous graph learning (#3189)
• Added an example for unsupervised GraphSAGE on the PPI dataset (#4416)
• Added the EdgeCNN model (#4991)
• Added an example to load a trained PyG model in C++ (#4307)

Transformations

• Added the AddPositionalEncoding transforms with two implementations: AddLaplacianEigenvectorPE and AddRandomWalkPE (#4521)
• Added the Rooted transform with two implementations: RootedEgoNets and RootedRWSubgraph (#3926)
• Added support for computing weighted metapaths in AddMetapaths (#5049)

Datasets

• Added the Genius and Wiki datasets to the LINKXDataset (#4570, #4600)
• Added the AQSOL dataset (#4626)
• Added Geom-GCN splits to the Planetoid datasets (#4442)

General Improvements

• Added support for GATv2Conv in the GAT model (#4357)
• Added support for projecting features before propagation in SAGEConv (#4437)
• Added a MessagePassing.explain_message() method to customize making explanations on messages (#4278, #4448))
• Added the MLP.plain_last = False option (4652)
• Added support graph-level attributes in networkx conversion (#4343)
• Added Data.validate() and HeteroData.validate() functionality to validate the correctness of the data (#4885)
• Added TorchScript support to JumpingKnowledge module (#4805)
• Added predict() support to the LightningNodeData module (#4884)
• Added support for renaming node types via HeteroData.rename() (#4329)
• Added HeteroData.num_features functionality (#4504)
• Added HeteroData.subgraph, HeteroData.node_type_subgraph and HeteroData.edge_type_subgraph functionality (#4243)
• Added HeteroData support to the RemoveIsolatedNodes transform (#4479)
• Added support for graph-level outputs in to_hetero (#4582)
• Added HeteroData.is_undirected() support (#4604)
• Added HeteroData.node_items() and HeteroData.edge_items() functionality (#4644)
• Added HeteroData.subgraph() support (#4635)
• Added node-wise normalization mode in LayerNorm (#4944)
• Added utils.unbatch and utils.unbatch_edge_index functionality for splitting an edge_index tensor according to a batch vector (#4628, #4903)
• Added scalable inference mode in BasicGNN with layer-wise neighbor loading (#4977)
• Added fine grained options for setting bias and dropout per layer in the MLP model (#4981)
• Added support for BasicGNN models within to_hetero (#5091)
• Let ImbalancedSampler accept torch.Tensor as input (#5138)
• Allow edge_type == rev_edge_type argument in RandomLinkSplit (#4757)

Bugfixes

• Fixed a bug in RGATConv that produced device mismatches for "f-scaled" mode (#5187]
• Fixed a bug in GINEConv bug for non-Sequential neural network layers (#5154]
• Fixed a bug in HGTLoader which produced outputs with missing edge types, will require torch-sparse>=0.6.15 (#5067)
• Fixed a bug in load_state_dict for Linear with strict=False mode (5094)
• Fixed data.num_node_features computation for sparse matrices (5089)
• Fixed a bug in which GraphGym did not create new non-linearity functions but re-used existing ones (4978)
• Fixed BasicGNN for num_layers=1, which now respects a desired number of out_channels (#4943)
• Fixed a bug in data.subgraph for 0-dim tensors (#4932)
• Fixed a bug in InMemoryDataset inferring wrong length for lists of tensors (#4837)
• Fixed a bug in TUDataset where pre_filter was not applied whenever pre_transform was present (#4842)
• Fixed access of edge types in HeteroData via two node types when there exists multiple relations between them (#4782)
• Fixed a bug in HANConv in which destination node features rather than source node features were propagated (#4753)
• Fixed a ranking protocol bug in the RGCN link prediction example (#4688)
• Fixed the interplay between TUDataset and pre_transform transformations that modify node features (#4669)
• The bias argument in TAGConv is now correctly applied (#4597)
• Fixed filtering of attributes in samplers in case __cat_dim__ != 0 (#4629)
• Fixed SparseTensor support in NeighborLoader (#4320)
• Fixed average degree handling in PNAConv (#4312)
• Fixed a bug in from_networkx in case some attributes are PyTorch tensors (#4486)
• Fixed a missing clamp in the DimeNet model (#4506, #4562)
• Fixed the download link in DBP15K (#4428)
• Fixed an autograd bug in DimeNet when resetting parameters (#4424)
• Fixed bipartite message passing in case flow="target_to_source" (#4418)
• Fixed a bug in which num_nodes was not properly updated in the FixedPoints transform (#4394)
• Fixed a bug in which GATConv was not jittable (#4347)
• Fixed a bug in which nn.models.GAT did not produce out_channels many output channels (#4299)
• Fixed a bug in mini-batching with empty lists as attributes (#4293)
• Fixed a bug in which GCNConv could not be combined with to_hetero on heterogeneous graphs with one node type (#4279)

Full Changelog

<details> <summary><b>Added</b></summary>

• Added edge_label_time argument to LinkNeighborLoader (#5137, #5173)
• Let ImbalancedSampler accept torch.Tensor as input (#5138)
• Added flow argument to gcn_norm to correctly normalize the adjacency matrix in GCNConv (#5149)
• NeighborSampler supports graphs without edges (#5072)
• Added the MeanSubtractionNorm layer (#5068)
• Added pyg_lib.segment_matmul integration within RGCNConv (#5052, #5096)
• Support SparseTensor as edge label in LightGCN (#5046)
• Added support for BasicGNN models within to_hetero (#5091)
• Added support for computing weighted metapaths in AddMetapaths (#5049)
• Added inference benchmark suite (#4915)
• Added a dynamically sized batch sampler for filling a mini-batch with a variable number of samples up to a maximum size (#4972)
• Added fine grained options for setting bias and dropout per layer in the MLP model (#4981)
• Added EdgeCNN model (#4991)
• Added scalable inference mode in BasicGNN with layer-wise neighbor loading (#4977)
• Added inference benchmarks (#4892, #5107)
• Added PyTorch 1.12 support (#4975)
• Added unbatch_edge_index functionality for splitting an edge_index tensor according to a batch vector (#4903)
• Added node-wise normalization mode in LayerNorm (#4944)
• Added support for normalization_resolver (#4926, #4951, #4958, #4959)
• Added notebook tutorial for torch_geometric.nn.aggr package to documentation (#4927)
• Added support for follow_batch for lists or dictionaries of tensors (#4837)
• Added Data.validate() and HeteroData.validate() functionality (#4885)
• Added LinkNeighborLoader support to LightningDataModule (#4868)
• Added predict() support to the LightningNodeData module (#4884)
• Added time_attr argument to LinkNeighborLoader (#4877, #4908)
• Added a filter_per_worker argument to data loaders to allow filtering of data within sub-processes (#4873)
• Added a NeighborLoader benchmark script (#4815, #4862)
• Added support for FeatureStore and GraphStore in NeighborLoader (#4817, #4851, #4854, #4856, #4857, #4882, #4883, #4929, #4992, #4962, #4968, #5037, #5088)
• Added a normalize parameter to dense_diff_pool (#4847)
• Added size=None explanation to jittable MessagePassing modules in the documentation (#4850)
• Added documentation to the DataLoaderIterator class (#4838)
• Added GraphStore support to Data and HeteroData (#4816)
• Added FeatureStore support to Data and HeteroData (#4807, #4853)
• Added FeatureStore and GraphStore abstractions (#4534, #4568)
• Added support for dense aggregations in global_*_pool (#4827)
• Added Python version requirement (#4825)
• Added TorchScript support to JumpingKnowledge module (#4805)
• Added a max_sample argument to AddMetaPaths in order to tackle very dense metapath edges (#4750)
• Test HANConv with empty tensors (#4756, #4841)
• Added the bias vector to the GCN model definition in the "Create Message Passing Networks" tutorial (#4755)
• Added transforms.RootedSubgraph interface with two implementations: RootedEgoNets and RootedRWSubgraph (#3926)
• Added ptr vectors for follow_batch attributes within Batch.from_data_list (#4723)
• Added torch_geometric.nn.aggr package (#4687, #4721, #4731, #4762, #4749, #4779, #4863, #4864, #4865, #4866, #4872, #4934, #4935, #4957, #4973, #4973, #4986, #4995, #5000, #5034, #5036, #5039, #4522, #5033, #5085, #5097, #5099, #5104, #5113, #5130, #5098, #5191)
• Added the DimeNet++ model (#4432, #4699, #4700, #4800)
• Added an example of using PyG with PyTorch Ignite (#4487)
• Added GroupAddRev module with support for reducing training GPU memory (#4671, #4701, #4715, #4730)
• Added benchmarks via wandb (#4656, #4672, #4676)
• Added unbatch functionality (#4628)
• Confirm that to_hetero() works with custom functions, e.g., dropout_adj (4653)
• Added the MLP.plain_last=False option (4652)
• Added a check in HeteroConv and to_hetero() to ensure that MessagePassing.add_self_loops is disabled (4647)
• Added HeteroData.subgraph() support (#4635)
• Added the AQSOL dataset (#4626)
• Added HeteroData.node_items() and HeteroData.edge_items() functionality (#4644)
• Added PyTorch Lightning support in GraphGym (#4511, #4516 #4531, #4689, #4843)
• Added support for returning embeddings in MLP models (#4625)
• Added faster initialization of NeighborLoader in case edge indices are already sorted (via is_sorted=True) (#4620, #4702)
• Added AddPositionalEncoding transform (#4521)
• Added HeteroData.is_undirected() support (#4604)
• Added the Genius and Wiki datasets to nn.datasets.LINKXDataset (#4570, #4600)
• Added nn.aggr.EquilibrumAggregation implicit global layer (#4522)
• Added support for graph-level outputs in to_hetero (#4582)
• Added CHANGELOG.md (#4581)
• Added HeteroData support to the RemoveIsolatedNodes transform (#4479)
• Added HeteroData.num_features functionality (#4504)
• Added support for projecting features before propagation in SAGEConv (#4437)
• Added Geom-GCN splits to the Planetoid datasets (#4442)
• Added a LinkNeighborLoader for training scalable link predictions models #4396, #4439, #4441, #4446, #4508, #4509)
• Added an unsupervised GraphSAGE example on PPI (#4416)
• Added support for LSTM aggregation in SAGEConv (#4379)
• Added support for floating-point labels in RandomLinkSplit (#4311, #4383)
• Added support for torch.data DataPipes (#4302, #4345, #4349)
• Added support for the cosine argument in the KNNGraph/RadiusGraph transforms (#4344)
• Added support graph-level attributes in networkx conversion (#4343)
• Added support for renaming node types via HeteroData.rename (#4329)
• Added an example to load a trained PyG model in C++ (#4307)
• Added a MessagePassing.explain_message method to customize making explanations on messages (#4278, #4448))
• Added support for GATv2Conv in the nn.models.GAT model (#4357)
• Added HeteroData.subgraph functionality (#4243)
• Added the MaskLabel module and a corresponding masked label propagation example (#4197)
• Added temporal sampling support to NeighborLoader (#4025)
• Added an example for unsupervised heterogeneous graph learning based on "Deep Multiplex Graph Infomax" (#3189) </details>

<details> <summary><b>Changed</b></summary>

• Changed docstring for RandomLinkSplit (#5190)
• Switched to PyTorch scatter_reduce implementation - experimental feature (#5120)
• Fixed RGATConv device mismatches for f-scaled mode (#5187]
• Allow for multi-dimensional edge_labels in LinkNeighborLoader (#5186]
• Fixed GINEConv bug with non-sequential input (#5154]
• Improved error message (#5095)
• Fixed HGTLoader bug which produced outputs with missing edge types (#5067)
• Fixed dynamic inheritance issue in data batching (#5051)
• Fixed load_state_dict in Linear with strict=False mode (5094)
• Fixed typo in MaskLabel.ratio_mask (5093)
• Fixed data.num_node_features computation for sparse matrices (5089)
• Fixed torch.fx bug with torch.nn.aggr package (#5021))
• Fixed GenConv test (4993)
• Fixed packaging tests for Python 3.10 (4982)
• Changed act_dict (part of graphgym) to create individual instances instead of reusing the same ones everywhere (4978)
• Fixed issue where one-hot tensors were passed to F.one_hot (4970)
• Fixed bool arugments in argparse in benchmark/ (#4967)
• Fixed BasicGNN for num_layers=1, which now respects a desired number of out_channels (#4943)
• len(batch) will now return the number of graphs inside the batch, not the number of attributes (#4931)
• Fixed data.subgraph generation for 0-dim tensors (#4932)
• Removed unnecssary inclusion of self-loops when sampling negative edges (#4880)
• Fixed InMemoryDataset inferring wrong len for lists of tensors (#4837)
• Fixed Batch.separate when using it for lists of tensors (#4837)
• Correct docstring for SAGEConv (#4852)
• Fixed a bug in TUDataset where pre_filter was not applied whenever pre_transform was present (#4842)
• Renamed RandomTranslate to RandomJitter - the usage of RandomTranslate is now deprecated (#4828)
• Do not allow accessing edge types in HeteroData with two node types when there exists multiple relations between these types (#4782)
• Allow edge_type == rev_edge_type argument in RandomLinkSplit (#4757)
• Fixed a numerical instability in the GeneralConv and neighbor_sample tests (#4754)
• Fixed a bug in HANConv in which destination node features rather than source node features were propagated (#4753)
• Fixed versions of checkout and setup-python in CI (#4751)
• Fixed protobuf version (#4719)
• Fixed the ranking protocol bug in the RGCN link prediction example (#4688)
• Math support in Markdown (#4683)
• Allow for setter properties in Data (#4682, #4686)
• Allow for optional edge_weight in GCN2Conv (#4670)
• Fixed the interplay between TUDataset and pre_transform that modify node features (#4669)
• Make use of the pyg_sphinx_theme documentation template (#4664, #4667)
• Refactored reading molecular positions from sdf file for qm9 datasets (4654)
• Fixed MLP.jittable() bug in case return_emb=True (#4645, #4648)
• The generated node features of StochasticBlockModelDataset are now ordered with respect to their labels (#4617)
• Fixed typos in the documentation (#4616, #4824, #4895, #5161)
• The bias argument in TAGConv is now actually applied (#4597)
• Fixed subclass behaviour of process and download in Datsaet (#4586)
• Fixed filtering of attributes for loaders in case __cat_dim__ != 0 (#4629)
• Fixed SparseTensor support in NeighborLoader (#4320)
• Fixed average degree handling in PNAConv (#4312)
• Fixed a bug in from_networkx in case some attributes are PyTorch tensors (#4486)
• Added a missing clamp in DimeNet (#4506, #4562)
• Fixed the download link in DBP15K (#4428)
• Fixed an autograd bug in DimeNet when resetting parameters (#4424)
• Fixed bipartite message passing in case flow="target_to_source" (#4418)
• Fixed a bug in which num_nodes was not properly updated in the FixedPoints transform (#4394)
• PyTorch Lightning >= 1.6 support (#4377)
• Fixed a bug in which GATConv was not jittable (#4347)
• Fixed a bug in which the GraphGym config was not stored in each specific experiment directory (#4338)
• Fixed a bug in which nn.models.GAT did not produce out_channels-many output channels (#4299)
• Fixed mini-batching with empty lists as attributes (#4293)
• Fixed a bug in which GCNConv could not be combined with to_hetero on heterogeneous graphs with one node type (#4279) </details>

<details> <summary><b>Removed</b></summary>

• Remove internal metrics in favor of torchmetrics (#4287) </details>

Full commit list: https://github.com/pyg-team/pytorch_geometric/compare/2.0.4...2.1.0

PyG 2.0.4 🎉

A new minor PyG version release, bringing PyTorch 1.11 support to PyG. It further includes a variety of new features and bugfixes:

Features

• Added Quiver examples for multi-GU training using GraphSAGE (#4103), thanks to @eedalong and @luomai
• nn.model.to_captum: Full integration of explainability methods provided by the Captum library (#3990, #4076), thanks to @RBendias
• nn.conv.RGATConv: The relational graph attentional operator (#4031, #4110), thanks to @fork123aniket
• nn.pool.DMoNPooling: The spectral modularity pooling operator (#4166, #4242), thanks to @fork123aniket
• nn.*: Support for shape information in the documentation (#3739, #3889, #3893, #3946, #3981, #4009, #4120, #4158), thanks to @saiden89 and @arunppsg and @konstantinosKokos
• loader.TemporalDataLoader: A dataloader to load a TemporalData object in mini-batches (#3985, #3988), thanks to @otaviocx
• loader.ImbalancedSampler: A weighted random sampler that randomly samples elements according to class distribution (#4198)
• transforms.VirtualNode: A transform that adds a virtual node to a graph (#4163)
• transforms.LargestConnectedComponents: Selects the subgraph that corresponds to the largest connected components in the graph (#3949), thanks to @abojchevski
• utils.homophily: Support for class-insensitive edge homophily (#3977, #4152), thanks to @hash-ir and @jinjh0123
• utils.get_mesh_laplacian: Mesh Laplacian computation (#4187), thanks to @daniel-unyi-42

Datasets

• Added a dataset cheatsheet to the documentation that collects import graph statistics across a variety of datasets supported in PyG (#3807, #3817) (please consider helping us filling its remaining content)
• datasets.EllipticBitcoinDataset: A dataset of Bitcoin transactions (#3815), thanks to @shravankumar147

Minor Changes

• nn.models.MLP: MLPs can now either be initialized via a list of channels or by specifying hidden_channels and num_layers (#3957)
• nn.models.BasicGNN: Final Linear transformations are now always applied (except for jk=None) (#4042)
• nn.conv.MessagePassing: Message passing modules that make use of edge_updater are now jittable (#3765), thanks to @Padarn
• nn.conv.MessagePassing: (Official) support for min and mul aggregations (#4219)
• nn.LightGCN: Initialize embeddings via xavier_uniform for better model performance (#4083), thanks to @nishithshowri006
• nn.conv.ChebConv: Automatic eigenvalue approximation (#4106), thanks to @daniel-unyi-42
• nn.conv.APPNP: Added support for optional edge_weight, (690a01d), thanks to @YueeXiang
• nn.conv.GravNetConv: Support for torch.jit.script (#3885), thanks to @RobMcH
• nn.pool.global_*_pool: The batch vector is now optional (#4161)
• nn.to_hetero: Added a warning in case to_hetero is used on HeteroData metadata with unused destination node types (#3775)
• nn.to_hetero: Support for nested modules (ea135bf)
• nn.Sequential: Support for indexing (#3790)
• nn.Sequential: Support for OrderedDict as input (#4075)
• datasets.ZINC: Added an in-depth description of the task (#3832), thanks to @gasteigerjo
• datasets.FakeDataset: Support for different feature distributions across different labels (#4065), thanks to @arunppsg
• datasets.FakeDataset: Support for custom global attributes (#4074), thanks to @arunppsg
• transforms.NormalizeFeatures: Features will no longer be transformed in-place (ada5b9a)
• transforms.NormalizeFeatures: Support for negative feature values (6008e30)
• utils.is_undirected: Improved efficiency (#3789)
• utils.dropout_adj: Improved efficiency (#4059)
• utils.contains_isolated_nodes: Improved efficiency (970de13)
• utils.to_networkx: Support for to_undirected options (upper triangle vs. lower triangle) (#3901, #3948), thanks to @RemyLau
• graphgym: Support for custom metrics and loggers (#3494), thanks to @RemyLau
• graphgym.register: Register operations can now be used as class decorators (#3779, #3782)
• Documentation: Added a few exercises at the end of documentation tutorials (#3780), thanks to @PabloAMC
• Documentation: Added better installation instructions to CONTRIBUTUNG.md (#3803, #3991, #3995), thanks to @Cho-Geonwoo and @RBendias and @RodrigoVillatoro
• Refactor: Clean-up dependencies (#3908, #4133, #4172), thanks to @adelizer
• CI: Improved test runtimes (#4241)
• CI: Additional linting check via yamllint (#3886)
• CI: Additional linting check via isort (66b1780), thanks to @mananshah99
• torch.package: Model packaging via torch.package (#3997)

Bugfixes

• data.HeteroData: Fixed a bug in data.{attr_name}_dict in case data.{attr_name} does not exist (#3897)
• data.Data: Fixed data.is_edge_attr in case data.num_edges == 1 (#3880)
• data.Batch: Fixed a device mismatch bug in case a batch object was indexed that was created from GPU tensors (e6aa4c9, c549b3b)
• data.InMemoryDataset: Fixed a bug in which copy did not respect the underlying slice (d478dcb, #4223)
• nn.conv.MessagePassing: Fixed message passing with zero nodes/edges (#4222)
• nn.conv.MessagePassing: Fixed bipartite message passing with flow="target_to_source" (#3907)
• nn.conv.GeneralConv: Fixed an issue in case skip_linear=False and in_channels=out_channels (#3751), thanks to @danielegrattarola
• nn.to_hetero: Fixed model transformation in case node type names or edge type names contain whitespaces or dashes (#3882, b63a660)
• nn.dense.Linear: Fixed a bug in lazy initialization for PyTorch < 1.8.0 (973d17d, #4086)
• nn.norm.LayerNorm: Fixed a bug in the shape of weights and biases (#4030), thanks to @marshka
• nn.pool: Fixed torch.jit.script support for torch-cluster functions (#4047)
• datasets.TOSCA: Fixed a bug in which indices of faces started at 1 rather than 0 (8c282a0), thanks to @JRowbottomGit
• datasets.WikiCS: Fixed WikiCS to be undirected by default (#3796), thanks to @pmernyei
• Resolved inconsistency between utils.contains_isolated_nodes and data.has_isolated_nodes (#4138)
• graphgym: Fixed the loss function regarding multi-label classification (#4206), thanks to @RemyLau
• Documentation: Fixed typos, grammar and bugs (#3840, #3874, #3875, #4149), thanks to @itamblyn and @chrisyeh96 and @finquick