NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #2062 most downloaded on PyPI
NVIDIA cuDNN Frontend — Python and C++ Graph API with SOTA attention (SDPA / Flash Attention), MoE grouped GEMM fusions, and FP8/MXFP8 kernels for Hopper, Blackwell, and Rubin GPUs.
Latest stable
1.32.3
Last release 7 days ago
10 Nov 2026
Ships fairly regularly
a new release about every 3 weeks
Nearly every release is documented
notes for 40 of 41 stable releases
Nothing withdrawn
no release was ever pulled
3 years old
41 releases · first in 2024
Drag across the ruler to compare, or tap a release to open it.
cuDNN Frontend v1.21.0 is the recommended version for cuDNN 9.20.0 and later releases.
cuDNN Frontend v1.21.0 is the recommended version for cuDNN 9.20.0 and later releases.
Added new kernels for the GEMM fusions.
Grouped GEMM + GLU: Unified grouped GEMM GLU API supporting dense and discrete MoE weight layouts with optional bias. Grouped GEMM + dGLU: Unified grouped GEMM dGLU backward API supporting dense and discrete MoE weight layouts with optional bias. Discrete Grouped GEMM + SwiGLU: Per-expert-pointer SwiGLU grouped GEMM for MoE workloads without weight packing. Discrete Grouped GEMM + dSwiGLU: Per-expert-pointer dSwiGLU backward grouped GEMM for MoE workloads without weight packing. Uses dSwiGLU/dGeGLU backward epilogue. Grouped GEMM + dSwiglu: dSwiglu activation fused with Grouped GEMM Grouped GEMM + Quant: Grouped GEMM with output quantization for MoE FC2/dFC1 workloads
Your coding agent can read these notes before it upgrades. Set up the MCP server →
cuDNN Frontend v1.20.0 is the recommended version for cuDNN 9.20.0 and later releases.
cuDNN Frontend v1.20.0 is the recommended version for cuDNN 9.20.0 and later releases.
Allow GEMM + Amax, GEMM + SwiGLU, Grouped GEMM + SwiGLU, Grouped GEMM + dSwiglu, and NSA kernels to run on GB300.
Improve the reproducer tool to report and reproduce SDPA failures.
Pinning the pybind version to prevent failures with older versions.
Pinning the pybind version to prevent failures with older versions.
Restore support for cuda-12 toolkit that was accidentally dropped in 1.19.0 release.
cuDNN Frontend v1.19.0 is the recommended version for cuDNN 9.19.1 and later releases.
sm_version on the cuDNN graph.cuDNN Frontend v1.19.0 is the recommended version for cuDNN 9.19.1 and later releases.
cuDNN Frontend v1.19.0 is the recommended version for cuDNN 9.19.1 and later releases.
sm_version on the cuDNN graph.cuDNN Frontend v1.18.0 is the recommended version for cuDNN 9.18.1 and later releases.
cuDNN Frontend v1.18.0 is the recommended version for cuDNN 9.18.1 and later releases.
New open source kernel for Grouped Gemm and Swiglu fussion
New Features: Allows support for dynamic shapes for fprop. This will help reduce the graph building across different batch and sequence lengths.
Support Surface:
More samples:
moe_grouped_matmul. See cpp sample and documentation for API reference.cuDNN Frontend v1.17.0 is the recommended version for cuDNN 9.17.0 and later releases.
cuDNN Frontend v1.17.0 is the recommended version for cuDNN 9.17.0 and later releases.
Native Sparse Attention : The Native Sparse Attention (NSA) module implements Native Sparse attention as described in the Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention. Samples of usage for Blackwell architecture in test/python/fe_api/nsa
Gemm/Swiglu : Gemm_Swiglu now supports block-scaled FP8/FP4 datatypes. API changes:
BufferError in non-pytorch tensors.cuDNN Frontend v1.16.0 is the recommended version for cuDNN 9.15.0 and later releases.
cuDNN Frontend v1.16.0 is the recommended version for cuDNN 9.15.0 and later releases.
This release introduces open-source implementations of commonly requested fused kernels for select architectures (Blackwell). These experimental kernels may require additional dependencies such as CuteDSL. The initial release includes:
Additional dependencies can be installed optionally using pip install nvidia-cudnn-frontend[cutedsl]. Usage examples and detailed documentation are available in the test/python/fe_api directory.
Please submit issue reports for additional kernel requests or bug reports.
Block Mask Support: Starting with cuDNN 9.14.0, SDPA attributes now support block masks to exclude tiles that do not require computation. Refer to the sample implementation for usage details.
Bug Fix: Resolved an invalid memory access (IMA) issue in SDPA backward propagation (fixed in cuDNN backend version 9.15.1 and later) that occurred when s_kv is not a multiple of 128, padding mask is disabled, and operations are performed in CUDA graph replay mode.
BehaviorNote_t::CUDNN_BEHAVIOR_NOTE_CUBLASLT_DEPENDENCY as a behavior note. This enables filtering of engine configurations (execution plans) that use cuBLAS as a backend, available starting with cuDNN version 9.15.0.Block Scale Quantization: Added Python bindings for block scale quantize operations (#173). Refer to the sample implementation for usage details.
Dependency Optimization: PyTorch is no longer a required dependency for cuDNN Frontend (#177).
Tensor Alignment: Enhanced tensor descriptor API to accept alignment as an attribute (#153).
Plan Generation Control: Updated cudnnGetPlan API to accept an optional maximum plan count parameter, enabling users to limit the number of plans built and autotuned.
cudnn frontend v1.15 is the preferred cudnn frontend version for cuDNN version 9.13.1 and above.
cudnn frontend v1.15 is the preferred cudnn frontend version for cuDNN version 9.13.1 and above.
cudnn.Graph API that enables interoperability between torch.tensors and the cudnn frontend API. Sample code for performing a matmul with bias addition:B, M, N, K = 16, 128, 128, 512
a_gpu = torch.randn(B, M, K, device="cuda", dtype=torch.bfloat16)
b_gpu = torch.randn(B, K, N, device="cuda", dtype=torch.bfloat16)
d_gpu = torch.randn(1, M, N, device="cuda", dtype=torch.bfloat16)
with cudnn.Graph(
intermediate_data_type=cudnn.data_type.FLOAT,
compute_data_type=cudnn.data_type.FLOAT,
inputs=["mm::A", "mm::B", "bias::bias"],
outputs=["bias::OUT_0"],
) as graph:
AB = graph.matmul(
name="mm",
A=a_gpu,
B=b_gpu,
)
C = graph.bias(name="bias", input=AB, bias=d_gpu)
C.set_output(True)
c_gpu = graph(a_gpu, b_gpu, d_gpu, handle=handle)
All notebooks under samples/python have been updated to showcase the flexibility of this API.
Graph now includes a warmup method that triggers kernel loading by performing a fake graph capture. This improves the startup time for running the initial kernel in the actual run and prevents deadlocks when used with other modules (e.g., NCCL).set_score_max and set_score_sum_exp to allow the kernel to output max attention score and sum of exponents.s_q==1 and s_kv==1.)COMPLEX_FP32 and COMPLEX_FP64 datatypes. (Requires cuDNN v9.14.0 or later.)fe::HeurMode_t::A over fe::HeurMode_t::FALLBACK.swish function now accepts a swish_beta parameter.📢 cuDNN Frontend v1.14.1 — Release Notes
📢 cuDNN Frontend v1.14.1 — Release Notes
9.11 and fixed in 9.13) affecting certain large head-dimension combinations of d_qk and d_v.fp16_fwd_with_sink_token.cpp
fp16_bwd_with_sink_token.cpp✅ Recommended Action: Upgrade to cuDNN Frontend v1.14.1 for full compatibility with cuDNN 9.13.0+, improved SDPA support, additional normalization support, and deviceless graph compilation features.
Preferred version for: cuDNN 9.12.0 and above Minimum Python version: 3.9 (previously 3.8, now obsolete) Updated pip wheels: Available for Python 3.13
Preferred version for: cuDNN 9.12.0 and above
Minimum Python version: 3.9 (previously 3.8, now obsolete)
Updated pip wheels: Available for Python 3.13
9.11) affecting certain large head-dimension combinations of d_qk and d_v.LayerNorm with ReLU bitmask dump.test_deviceless_aot_compilation.py✅ Recommended Action: Upgrade to cuDNN Frontend v1.14,0 for full compatibility with cuDNN 9.12.0+, improved SDPA support, additional normalization support, and deviceless graph compilation features.
When generate_stats is true, the output will contain the stats tensor. When migrating from is_inference (which is now deprecated), note that generate_…
cudnn frontend v1.13 is the preferred cudnn frontend version for cudnn version 9.11.0 and above.
Introduces device descriptor, which allows for device-less compilation of cudnn graph on a target GPU. See newly added sample and documentation.
Introduced generate_stats as a replacement for is_inference, to improve clarity. When generate_stats is true, the output will contain the stats tensor. When migrating from is_inference (which is now deprecated), note that generate_stats has the opposite meaning, so pass it the negation of the bool that was passed to is_inference.
Improved support checks for left and right diagonal bands in conjunction with the diagonal alignment.
Improved error handling for large head dimension (d > 128) in sdpa bprop.
Published improved SDPA training benchmarks for fp8 and fp16/bf16 graph patterns.
Enable int4 Weight only Quantization for matmul. See example
Allow block scale dequantize (required for low precision matmul) to take 2-D scale factor.
Allow reductions to accept deterministic as a attribute.
Added pybinds for block scale dequantize.
cudnn frontend v1.12 is the preferred cudnn frontend version for cudnn version 9.9.0 and above.
cudnn frontend v1.12 is the preferred cudnn frontend version for cudnn version 9.9.0 and above.
cudnn_frontend v1.12 is the minimum cudnn frontend version required to work with cuda 13.0 and above
Update the dlpack version and cmake minimum required version to be 3.18
Allows compilation and loading of cudnn frontend with cudnn-jit packages.
Introduce Adaptive Layernorm (fprop and bprop) operation in cudnn.
std::array<std::shared_ptr<Tensor_attributes>, 3>
adalayernorm(std::shared_ptr<Tensor_attributes>& input,
std::shared_ptr<Tensor_attributes>& scale,
std::shared_ptr<Tensor_attributes>& bias,
AdaLayernorm_attributes attributes);
std::array<std::shared_ptr<Tensor_attributes>, 3> adalayernorm_backward(
std::shared_ptr<Tensor_attributes> dy,
std::shared_ptr<Tensor_attributes> x,
std::shared_ptr<Tensor_attributes> scale,
AdaLayernorm_backward_attributes options);
Please refer to samples for usage.
cudnn.jit and cudnn.graph for simpler graph creation in python. Refer the matmul sample for usage.Allows large embedded dimension (d > 128) for fprop across Ampere, Hopper, and Blackwell architectures for bf16/fp16.
Added better validation checks for sliding window attention for cudnn version 9.9.0 and below.
Sliding windown attention now supports cases when s_q > s_kv
sdpa_fp8 operation now pads correctly with negative infinity on masking operation rather than high negative value. This improves the numerical stability of the sdpa operation with fp8 data type.
Paged attention now supports page tables in a packed format
Fixed the dlopen of cudart.so to look for the binary with version name.
Correctly fail when SDPA bprop is called on Blackwell with embedded dimension (d) > 128.
cudnn frontend v1.11 is the preferred cudnn frontend version for cudnn version 9.8.0 and above. With cuDNN frontend v1.11, the minimum supported cudnn
cudnn frontend v1.11 is the preferred cudnn frontend version for cudnn version 9.8.0 and above. With cuDNN frontend v1.11, the minimum supported cudnn version is 9.0.0.
Note: The FE will continue to build and run with cudnn_v8, until explicitly marked as compilation failure.
score_mod=partial(
custom_mask,
mod_tensor=mod_tensor,
neg_inf=neg_inf_tensor,
seq_len_q=seq_len_q,
seq_len_kv=seq_len_kv,
)
std::shared_ptr<Tensor_attributes>
concatenate(std::vector<std::shared_ptr<Tensor_attributes>>, Concatenate_attributes);
pip wheels compatible with windows x86_64 architecture are now available on pypi.
sdpa paged attention API now supports Q tensor to be ragged when used with cudnn version 9.7.0 and above.
Users can now pass the CMake flag -DCMAKE_CXX_FLAGS="-DNV_CUDNN_FRONTEND_DISABLE_LOGGING" to disable logging in the cuDNN frontend.
Adds a new sample to showcase native cudagraph creation from cudnn for sdpa bprop operation. Fixed a bug when using the update_cuda_graph API to update cuda graph for sdpa bprop operation.
Updates the create_container_and_page_table example function to use the layout that's desired for the more performant kernel."
Fixes memory leak in the test harness for some legacy tests that use ragged tensors.
Fixes a bug introduced in the benchmarking script that prevented the sdpa cudnn operation from being executed. This was because the use_padding_mask attribute was made mandatory for the sdpa operation. This has been fixed as well.
Updates the paged attention sample to not cause illegal memory access when changing the dimensions of the tensors in the sample.
Updates the DgradDReluBNBwdWeight sample to perform the right operation for the dgrad + drelu fusion.
cudnn frontend v1.10 is the preferred cudnn frontend to be used for cudnn backend 9.7.0 and later as it adds to the Blackwell specific features.
cudnn frontend v1.10 is the preferred cudnn frontend to be used for cudnn backend 9.7.0 and later as it adds to the Blackwell specific features.
cudnn Frontend v1.10 introduces two new operators, block_scale_quantize and block_scale_dequantize to specify the scaling and de-scaling of low precision datatypes supported from Blackwell GPU onwards.
create_execution_plan(int64_t const engine_id, std::unordered_map<KnobType_t, int64_t> const &knobs) allows creation
of a custom execution plan with hardcoded engine and knobs. Added a
sample in samples/cpp/misc/custom_plan.cpp to showcase how to work
with different Engine and Knobs.
Users can now query behavior notes of a particular execution plan
using get_behavior_notes(std::vector<BehaviorNote_t> ¬es) const and
get_behavior_notes_for_plan_at_index(int64_t const index, std::vector<BehaviorNote_t> ¬es) const functions.
SDPA operations now accept both left window and right window size with respect to diagonal. See Attention.md for more details.
SDPA operations now accept a diagonal alignment for the Attention
score matrix to be used describe the above window. When s_q != s_kv,
and causal mask is on this can be used to specify if the diagonal is top
left or bottom right.
Bottom right causal masking can now be enabled on the sdpa_fp8 operation.
SDPA_attributes and SDPA_bprop_attributes now accepts a score_mod function through set_score_mod and set_score_mod_bprop API. The function accepts a c
SDPA_attributes and SDPA_bprop_attributes now accepts a score_mod function through set_score_mod and set_score_mod_bprop API. The function accepts a custom chain of pointwise operations which operate on the Attention Score Matrix. Some common functors like causal mask, sliding window mask, soft capping etc. have been added to the headers as reference. More examples of usage have been added in samples for fprop and bprop.
Added support for THD format and sliding window mask.
Added support for THD format and Bottom right causal mask.
Added support for bottom right causal masking with sliding window mask
Added a new parameter called set_max_total_seq_len_q/set_max_total_seq_len_kv on the sdpa bprop node. This will help reduce the workspace size required when running with THD format.
Allow creation of serialized json for dgrad, wgrad and resample operations.
Added more diagnostic message when the compiled version of cudnn does not match the run-time version of cudnn.
Fixed an issue where log messages unparseable data at the end of messages.
Fixed an issue where while building the python pip wheel would hang.
Fixed natively creating cuda graphs for SDPA with alibi masks.
SDPA forward operation now supports paged attention on cudnn 9.5.0 and later by setting the appropriate page table descriptors. SDPA_attributes now ac
SDPA forward operation now supports paged attention on cudnn 9.5.0 and later by setting the appropriate page table descriptors. SDPA_attributes now accepts set_paged_attention_k_table and set_paged_attention_v_table to input these descriptors. Please refer to samples for usage : cpp samples, python samples. See docs for more API details. Paged attention allows for more efficient memory usage by storing K/V caches in non-contiguous memory, and using page tables to reconstruct them. For more information, refer to the cudnn_graph Library, and the Paged Attention paper
cudnn graph now allows user to directly build native cuda_graph for given sub_graph (requires cudnn 9.5.0). There are two APIs:
populate_cuda_graph : add the cudnn nodes to the empty cuda_graph provided as input.update_cuda_graph : update the populated cuda graph with necessary data pointers.
See docs and backend documentation for more details.Kernel cache for dynamic shapes are now supported in python. Added a sample to showcase usage.
graph.deselect_engines(str: ) has now a python equivalent through pybind11.
graph.tensor(...) can now accept int64_t scalars directly. (Previously limited to int32_t, float and fp16 data types).
fp8 sdpa attention now allows dropout and padding mask. Requires cudnn 9.5.0 and above.
More enhancements to pointwise output stride inferencing (for broadcast operation). For non-unary operands, the broadcasted tensor can now be either at IN_0 or IN_1.
SDPA backward operation now allows d upto 256 for Hopper. Requires cudnn 9.5.0 and above.
Fixed an issue while querying cudnnGetLastErrorString() from the backend. The error_t object will now have more meaningful message.
Fixed build issues seen with clang-19 compiler.
Fixed an issue where it was assumed a graph with bias in sdpa_bprop will always have a dbias.
Kernel Cache support for dynamic graphs Added New APIs to enable kernel cache support for graphs with dynamic shapes. Please refer to documentation fo
Added examples Convolution fprop dynamic shape, CSBR Graph dynamic shape, Matmul dynamic shape and Bias + Matmul dynamic shape to showcase use of dynamic shapes and kernel cache.
error_t
get_plan_name(std::string &name) const;
error_t
get_plan_name_at_index(int64_t plan_index, std::string &name) const;
Note: This name can be used later if you want to deselect_plan_by_name, if run into any potential errors.
query_tensor_with_uid(int64_t const uid, Tensor_attributes &tensor) const;sdpa fp16 bprop node can now compute dbias when padding mask is enabled (requires cudnn 9.4.0 and above).
sdpa fp8 (forward and bprop) nodes now support optional bias, dropout and padding mask(requires cudnn 9.4.0 and above).
Matmul fp8 node can now accept M,N,K overrides.
Added new python notebooks for implementing BatchNorm and BatchNorm bprop using cuDNN.
Updated benchmark numbers with cudnn 9.4.0 for fp16 and fp8 datatypes.
Fixed compilation issues when NV_CUDNN_DISABLE_EXCEPTION is enabled.
Fixed a crash when the output dimension of dgrad node is not specified. This now returns an error message instead.
Fixed incorrect SDPA stats stride inferencing.
Fixed a bug in sdpa test when sliding window attention is enabled and query sequence length (s_q) is greater than key length (s_kv). This case is now not supported.
Fixed an issue where custom dropout mask was not correctly applied.
-fvisibility=hidden for the pip wheels generated to avoid symbol conflicts with other modules that use cudnn frontend.c * d * h * w > 2 **31) tensors.Graph Slice Operation: Introduced the graph.slice operation for slicing input tensors. Refer to docs/operations/Slice.md for detailed documentation an
graph.slice operation for slicing input tensors. Refer to docs/operations/Slice.md for detailed documentation and samples/cpp/misc/slice.cpp for a C++ sample. Pybinds for this operation have also been added.set_sm_count(int32_t type) graph property to support the SM Carveout feature introduced in Ampere and Hopper GPUs. Engines that do not support SM_COUNT will return NOT_SUPPORTED.set_convolution_mode attribute to convolution attributes in forward propagation (fprop), data gradient (dgrad), and weight gradient (wgrad). Previously, this was hardcoded to CUDNN_CROSS_CORRELATION in the 1.x API.sdpa_fp8_backward node.graph.execute() by optimizing sub-node tree traversal, collected UIDs, workspace modifications, and workspace size.graph.validate() by deferring graph expansion to a later stage (build_operation_graph).create_execution_plans if called without the preceding build_operation_graph.graph.build() calls.CMAKE_SOURCE_DIR with PROJECT_SOURCE_DIR in CMake files for better integration. See the relevant pull request for more details.samples/python folder for more information.[Enhancement] Allows stride value of 0 indicating repetition of tensor in those dimensions.
[Enhancement] Allows stride value of 0 indicating repetition of tensor in those dimensions.
[Bug fix] Fixed an issue, where cudnn-frontend (1.5.0) when built with cudnn version 9.1.1 and below, runs into issues when run with 9.2.0 and above.
v1.5.1
[Bug fix] Fixed an issue, where cudnn-frontend (1.5.0) when built with cudnn version 9.1.1 and below, runs into issues when run with 9.2.0 and above.
[New feature] With cudnn backend 9.2.0 and above, Graph::check_support can determine support check for runtime engines without invoking the nvrtc comp
[New feature] With cudnn backend 9.2.0 and above, Graph::check_support can determine support check for runtime engines without invoking the nvrtc compiler. This allows users to check the support surface of cudnn without invoking the nvrtc compilation.
[New feature] Python pip wheel now contains the necessary c++ development headers.
[New feature] Sliding window attention is now supported as an attribute to the sdpa forward and bprop node. Usage: sdpa_attributes.set_sliding_window_length(window_length)
[New feature] Bottom right aligned causal masking is now supported as an attribute to the sdpa forward and bprop node. Usage: sdpa_attributes.use_causal_mask_bottom_right(true)
[New feature] SDPA bprop attributes can choose deterministic algorithm using the use_deterministic_algorithm API.
[New feature] Allow users to filter candidate execution plans of graph by its shared memory usage in cudnn 9.2.0 and later.
[Bug fix] A runtime error if chosen execution plan candidate is incorrectly set in the backend has been fixed. This would happen when check_support does not correctly filter by the workspace size.
[Bug fix] selecting/deselecting by behavior and numerical notes has now been fixed and works as intended.
[Debugging] A new tool for easy reproduction of a failure using the json representation of the graph can be found here.
[Samples] Restructured the cpp samples into categories for easier navigation.
[Samples] Added a sample to showcase how different plans can be built in parallel in separate threads.
[Compilation enhancement] Added a new macro CUDNN_FRONTEND_SKIP_NLOHMANN_JSON as compilation flag to not have nlohman::json as compilation dependency. Users lose access to certain API functions like print, key, serialize, deserialzie that depend on the library.
[Enhancement] Serialization of resample operation is now supported.
[Enhancement] Bug template has been added for new github issues
[New] Added a benchmark folder which contains a sample docker file to compare cudnn implementation of sdpa with that of the pytorch implementation.
[New] Added a benchmark folder which contains a sample docker file to compare cudnn implementation of sdpa with that of the pytorch implementation.
[Enhancement] Once an engine is de-selected by name, it will not be built as part of check support.
[Enhancement] The cudnn backend search order for wheels is as follows: (a) It will dlopen libcudnn.so.MAJOR_VERSION in the site packages. (b) It will try to dlopen unversioned libcudnn.so. This way pypi cudnn package nvidia-cudnn-cu* gets priority over default search path.
[Enhancement] Allow embedding dimension up to 256 (currently limited to 128) in sdpa fprop operation.
[Bug fix] Update the scale and bias shapes in batch norm sample.
[New API] Added new operations sdpa_fp8_forward and sdpa_fp8_backward to perform scaled dot prodcut attention of fp8 tensors. See more details in the
[New API] Added new operations sdpa_fp8_forward and sdpa_fp8_backward to perform scaled dot prodcut attention of fp8 tensors. See more details in the docs/operations/Attention.md and cpp sample in samples/cpp/mha.cpp. Pybinds for the fp8 nodes are also added.
[New API] Added new operation for resample forward operation. Add a new sample samples/cpp/resample.cpp to show its usage.
[New API] Add a new API deselect_engines(std::vector<std::string> const &engine_names) which blocks certain engine configs from running.
[New API] Add new APIs select_numeric_notes and select_behavior_notes to allow user select engine configs which have the selected numeric and behavior notes respectively.
[Python API] Added a custom exception cudnnGraphNotSupportedException to the python API to distinguish between graphs that are actually not supported as compared to programming errors.
[Python API] Added a new backend_version_string which returns the backend version in canonical form (eg. 9.1.0) instead of a version number.
[Bug Fix] Fixed issues with compilation on clang19 and c++20 standard.
[Bug Fix] Updated the workspace computation for sdpa fprop node. Previously, workspace was calculated for alibi slopes irrespective of whether alibi mask was turned on or not.
[Bug Fix] Fixed deserialization of fused scalars.
One column per month.