NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI
PyPI, #3229 most downloaded on PyPI
Download compute kernels
Latest stable
0.18.2
Last release 13 days ago
09 Nov 2026
Ships fairly regularly
a new release about every 3 weeks
Most releases are documented
notes for 48 of 60 stable releases
Nothing withdrawn
no release was ever pulled
2 years old
63 releases · first in 2025
Drag across the ruler to compare, or tap a release to open it.
kernels: make sigstore optional again (backport) by @danieldk in https://github.com/huggingface/kernels/pull/979
Full Changelog: https://github.com/huggingface/kernels/compare/v0.18.1...v0.18.2
Accept older ROCm minor versions within the same major (#959) by @Abdennacer-Badaoui in https://github.com/huggingface/kernels/pull/961
Full Changelog: https://github.com/huggingface/kernels/compare/v0.18.0...v0.18.1
Your coding agent can read these notes before it upgrades. Set up the MCP server →
For instance, backend = "cuda" becomes language = "cpp-cuda". This is a breaking change: build.toml files with an older edition can be migrated with k…
Building kernels for TPUs (torch_tpu/Pallas) is now supported using the tpu backend:
[general]
# ...
backends = ["tpu"]
See the relu-tpu kernel for a full example.
First off, this release adds initial support for Torch 2.15, including support for CUDA 13.4 and ROCm 7.14.
As kernels gets increasingly used by larger projects like Transformers, we would like to increase the number of supported Torch versions. We will reveal our support policy soon, but as a first step, we have added back support for building kernels for Torch 2.10 to 2.12.
We have also extended Torch Stable ABI handling to build non-stable ABI versions for releases that precede the targeted ABI version. For instance, if the stable ABI supports Torch >= 2.13, 2.10-2.12 will be built as regular non-stable ABI builds.
kernels describe-api and kernels cache lsThe new kernels describe-api subcommand lists the functions and layers that a
kernel provides:
$ kernels describe-api sayakpaul/liger-kernels --trust-remote-code
Repository: sayakpaul/liger-kernels
Revision: v2
Variant: torch-cuda
Functions:
LigerForCausalLMLoss
liger_rotary_pos_emb
Layers:
LigerRMSNorm has_backward=unknown can_torch_compile=unknown
LigerSwiGLUMLP has_backward=unknown can_torch_compile=True
...
The information comes from a symbols.json that is now generated for every
kernel build.
kernels cache ls lists the kernels in your local cache, similar to hf cache ls.
$ hf cache ls
ID SIZE LAST_ACCESSED LAST_MODIFIED VERSION VARIANTS
----------------------------------- ---- -------------- -------------- ------- --------------------------------
kernel/kernels-community/activation 2.5M 45 minutes ago 45 minutes ago v1 torch28-cxx11-cu126-x86_64-linux
torch28-cxx11-cu128-x86_64-linux
torch28-cxx11-cu129-x86_64-linux
kernel/kernels-community/activation 2.6M 23 minutes ago 23 minutes ago v2 torch28-cxx11-cu126-x86_64-linux
torch28-cxx11-cu128-x86_64-linux
torch28-cxx11-cu129-x86_64-linux
kernels now verifies kernels while they are loaded:
sigstore dependency is installed.Evaluating kernel flakes got a lot faster through multiple changes:
pandoc).On the ReLU example kernel, these changes cut down evaluation time from 42.6s to 22.2s. However, more importantly, the improved sharing makes evaluation time grow linearly onlywith the number of backend versions and not the number of Torch/backend version combinations.
kernels versions is deprecated in favor of kernels variants. This subcommand lists the variants of a kernel and which of them are compatible with and preferred for your machine. It supports --only-compatible, --all-versions, --version, and --json.build.toml edition 6: the backend kernel option is replaced by language, which takes <language>-<backend> (cpp-cpu, cpp-cuda, cpp-metal, cpp-rocm, cpp-xpu, or rust-cpu). For instance, backend = "cuda" becomes language = "cpp-cuda". This is a breaking change: build.toml files with an older edition can be migrated with kernel-builder update-build.HFKernelsClientBuilder by @danieldk in https://github.com/huggingface/kernels/pull/822Oid throughout kernels by @danieldk in https://github.com/huggingface/kernels/pull/827egg_info.writers entry point module path by @jiqing-feng in https://github.com/huggingface/kernels/pull/829gfx11{02,03,50,51} ROCm archs by @danieldk in https://github.com/huggingface/kernels/pull/839-mtls-dialect on ROCM >= 7.14 by @danieldk in https://github.com/huggingface/kernels/pull/840cxx11 ABI tag from Linux builds by @danieldk in https://github.com/huggingface/kernels/pull/856kernels by @danieldk in https://github.com/huggingface/kernels/pull/842Func] Allow proper preservations/reference across copy and pickle by @vasqu in https://github.com/huggingface/kernels/pull/873__all__ by @sayakpaul in https://github.com/huggingface/kernels/pull/878Func] Fix more complex global/compile regions by @vasqu in https://github.com/huggingface/kernels/pull/903GPU_LANG to CMake by @danieldk in https://github.com/huggingface/kernels/pull/901-cxx11 tag from test that expected it by @danieldk in https://github.com/huggingface/kernels/pull/924packaging by @danieldk in https://github.com/huggingface/kernels/pull/938collect_machine_info by @danieldk in https://github.com/huggingface/kernels/pull/940DeviceRepos by @danieldk in https://github.com/huggingface/kernels/pull/941Full Changelog: https://github.com/huggingface/kernels/compare/v0.17.0...v0.18.0
Backport ROCm version fixes (#902, #959) by @Abdennacer-Badaoui in https://github.com/huggingface/kernels/pull/962
Full Changelog: https://github.com/huggingface/kernels/compare/v0.17.2...v0.17.3
Fix fetch build fallback (#848) by @SunMarc in https://github.com/huggingface/kernels/pull/868
Full Changelog: https://github.com/huggingface/kernels/compare/v0.17.1...v0.17.2
Fix egg_info.writers entry point module path (#829) by @jiqing-feng in https://github.com/huggingface/kernels/pull/829 and https://github.com/huggingf
egg_info.writers entry point module path (#829) by @jiqing-feng in https://github.com/huggingface/kernels/pull/829 and https://github.com/huggingface/kernels/pull/830Full Changelog: https://github.com/huggingface/kernels/compare/v0.17.0...v0.17.1
doc: add a migration section for deprecated kernel functions by @danieldk in https://github.com/huggingface/kernels/pull/679
get_kernel now raises an exception if the kernel does not support the architecture of the GPU in the machine. For instance, if a kernel is only compiled for Hopper (CUDA capability 9.0) is loaded on a Blackwell GPU with CUDA capability 12.0 an exception is raised. Previously, an incompatible kernel would load fine and then (loudly or quietly) fail at kernel launch time.
In some cases, an AOT kernel may support more architectures than the metadata declares, for instance when the kernel has a Triton fall-back path. For such kernels, the check can be disabled:
from kernels import get_kernel, has_kernel
has_kernel("kernels-community/flash-attn3", version=1, check_arch=False)
flash_attn3 = get_kernel("kernels-community/flash-attn3", version=1, check_arch=False)
Kernels can now depend on other kernels from the Hub. Dependencies are declared in the general section of build.toml. For example:
[general]
# ...
kernel-depends = [
{ repo-id = "kernels-community/activation", version = 1 },
{ repo-id = "kernels-community/einops", version = 1 },
]
The kernel dependencies can than be retrieved when the kernel is imported using the new get_kernel_dep function:
import kernels
activation = kernels.get_kernel_dep("kernels-community/activation")
einops = kernels.get_kernel_dep("kernels-community/einops")
Warning: using kernel dependencies breaks support for kernels<0.17. For this reason, we recommend you to only start using kernel dependencies when version 0.17.0 has been out for a while. The kernels minimum version metadata described below will be ported to kernels 0.16 and will help with this and future migrations.
kernels minimum versionTo introduce new features more gracefully in the future, using newer kernels features (such as kernel dependencies) will write the minimum required version to the kernel metadata. The kernels client will use this to verify that it is compatible with the kernel and, if not, suggest what version of kernels to install.
use_kernel_forward_from_hub now accepts an optional condition argument. This condition is applied to the layer when kernelize is called. This is useful when a layer supports multiple configurations, but kernelization is only supported for one configuration:
@use_kernel_forward_from_hub(
"SwiGLUMLP",
condition=lambda module: module.config.hidden_act == "silu",
)
class MyMLP(nn.Module):
...
kernel-builder now has experimental support for building TPU torch_tpu/Pallas kernels. backends = ["tpu"] produces a torch-tpu noarch variant.
The XPU/SYCL build supports Intel Crescent Island.
The Helion DSL is now supported through the new helion Python dependency (python-depends = ["helion"]). See the blog post Helion x 🤗 HF Kernels: Building and Shipping Out-of-the-box Performant Kernels for more information.
kernels only loads kernels from a curated set of trusted publishers. Until now, the only way to load a kernel from another publisher was to opt in with trust_remote_code=True, which disables the check entirely. You can now pass a list of repository IDs instead, so that only the given repositories are allowed:
get_kernel(
"some-other-org/my-kernel",
version=1,
trust_remote_code=["some-other-org/my-kernel"],
)
kernels infoThe new kernels info subcommand describes a kernel from a Hub repo ID or a local path, printing its name, version, license, upstream/source repositories, Python dependencies, and supported backends:
$ kernels info kernels-community/activation
Repository: kernels-community/activation
Revision: v1
Name: activation
Version: 1
License: Apache-2.0
Upstream: -
Source: -
Python dependencies: -
Backends: cuda, metal
Kernel metadata now records the provenance of a build in a provenance object. The provenance information contains the Git commit of the kernel and the builder and whether both were in a clean or dirty state.
kernels 0.15.1 added experimental support for the Torch stable ABI. The stable ABI was enabled by setting torch.stable-abi to the stable ABI version that the kernel should be compiled with. Unfortunately, the stable ABI is not complete enough yet to be supported on all backends. For this reason, we have decided to change the torch.stable-abi option to take a table with versions per-backend:
[torch.stable-abi]
cuda = "2.11"
rocm = "2.9"
Backends that are not listed are compiled without the stable ABI. This makes it possible to opt-in to the stable ABI per backend and use different stable ABI versions per backend.
This release adds support for Torch 2.13 and 2.14 and removes Torch 2.11 and 2.12.
stable-abi is now a table rather than a single Torch version. build.toml files can be migrated to edition 5 with kernel-builder update-build.check_arch=False.get_kernel instead of importing the kernel directly.kernels-data is not a separate Python package anymore, but is directly integrated into the kernels package.
cxx-flags for Torch and tvm-ffi sources by @danieldk in https://github.com/huggingface/kernels/pull/680HIP_ARCHITECTURES gets properly cleared by @danieldk in https://github.com/huggingface/kernels/pull/684kernel-depends option to general options in build.toml by @danieldk in https://github.com/huggingface/kernels/pull/749GitHash by GitStatus by @danieldk in https://github.com/huggingface/kernels/pull/765fetchFromHuggingFace: add by @danieldk in https://github.com/huggingface/kernels/pull/766build.toml parsing to kernels-data by @danieldk in https://github.com/huggingface/kernels/pull/768kernels-data by @danieldk in https://github.com/huggingface/kernels/pull/771{rocm,xpu}Packages: read version manifests from directory by @danieldk in https://github.com/huggingface/kernels/pull/797use_kernel_forward_from_hub by @danieldk in https://github.com/huggingface/kernels/pull/796user_agent in layers by @sayakpaul in https://github.com/huggingface/kernels/pull/805kernels-data Python binding into kernels by @danieldk in https://github.com/huggingface/kernels/pull/818kernels-data to kernels-common by @danieldk in https://github.com/huggingface/kernels/pull/821Full Changelog: https://github.com/huggingface/kernels/compare/v0.16.1...v0.17.0
This release adds a version check to require the user to upgrade when a kernel does not support this kernels version.
This release adds a version check to require the user to upgrade when a kernel does not support this kernels version.
Nothing published for this version
use_kernel_func_from_hub, FuncRepository, LocalFuncRepository, and LockedFuncRepository are now deprecated.
This release adds preliminary support for signing kernels. Kernel signing is currently in an experimental phase and the details may still change. For this reason, signatures are currently not yet validated when downloading a kernel. Kernel verification consists of two parts:
metadata.json. During verification, all files must be present and have the correct hash. Verification also fails when there are files that are not specified in the digest.metadata.json.sigstore. The signature is made using sigstore, which signs artifacts using ephemeral signing keys, reducing the impact of key theft. During verification, the signature is used to validate that metadata.json was not tampered with and that the signature was made from a trusted repository and workflow.Since the kernel is not verified yet during retrieval during the experimental stage, we provide the kernels verify-signature command-line utility that can be used to verify a kernel on the Hub. The kernel and kernel version to verify should be provided as arguments:
$ kernels verify-signature kernels-community/flash-attn4 0
✅ torch-cuda: kernel metadata is correctly signed
All kernels-community kernels are signed. If you want to experiment with kernel signing yourself, there are two changes you need to make:
nix flake update to get the latest version of kernel-builder. The latest version embeds the kernel digest in the metadata.cosign to sign the kernel. This cannot be done as part of the build itself, since signing using ephemeral kernels requires internet access and the kernel build sandbox does not provide internet access. You can use the kernels-community workflow as an example of how to set up metadata signing.kernel-builder now supports the cpu-kernels skill for writing, optimizing, and benchmarking C++ kernels using AVX2/AVX512. For example, to add the skill to Claude, use:
$ kernel-builder skills add --skill cpu-kernels --claude
use_kernel_func_from_hub, FuncRepository, LocalFuncRepository, and LockedFuncRepository are now deprecated.
To make a function extensible by a layer, you can now use the same decorator as for layers (use_kernel_forward_from_hub). This makes it clearer that the function is actually replaced by a layer. We have also added the use_kernelized_func decorator to attach such a function to the layer wherein it is used to make it discoverable by kernelize. Here is a full example:
# Make silu_and_mul replaceable with a kernel layer registered as `silu_and_mul`.
@use_kernel_forward_from_hub("silu_and_mul")
def silu_and_mul(x: torch.Tensor) -> torch.Tensor:
d = x.shape[-1] // 2
return F.silu(x[..., :d]) * x[..., d:]
# Attach the function to the layer where it is used to make it discoverable by `kernelize`.
@use_kernelized_func(silu_and_mul)
class FeedForward(nn.Module):
def __init__(self, in_features: int, out_features: int):
self.linear = nn.Linear(in_features, out_features)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return silu_and_mul(self.linear(x))
The FuncRepository, LocalFuncRepository, and LockedFuncRepository classes will not be replaced. They allowed using an arbitrary function from a kernel as a layer. However, this was easily misused and did not have a clean way of marking such a function as supporting torch.compile or backwards passes. Going forward, they should be made available as regular kernel layers that can be used with LayerRepository and its local/locked versions.
For more information, see the layer documentation.
Kernels support a small set of curated Python dependencies, such as einops, nvidia-cute-dsl, and apache-tvm-ffi. These dependencies are now also provided as extras of the kernels package, curated for CUDA and curated-xpu for XPU:
# CUDA
$ pip install 'kernels[curated]'
# XPU
$ pip install 'kernels[curated-xpu]'
This can be used to install all dependencies that a kernel might use.
The documentation now provides an overview of the kernel-builder architecture.
FuncRepository] Add ability of detecting flags by @vasqu in https://github.com/huggingface/kernels/pull/607hash subcommand and hook up in Nix by @danieldk in https://github.com/huggingface/kernels/pull/618--filter-unsigned by @danieldk in https://github.com/huggingface/kernels/pull/653OIDCSourceRepositoryURI by @danieldk in https://github.com/huggingface/kernels/pull/652pyext in torch-noarch by @danieldk in https://github.com/huggingface/kernels/pull/658kernel repo publishing rights by @sayakpaul in https://github.com/huggingface/kernels/pull/662ptxas version than CUDA version by @danieldk in https://github.com/huggingface/kernels/pull/671Full Changelog: https://github.com/huggingface/kernels/compare/v0.15.1...v0.16.0
This release adds support for can_torch_compile/can_backward to FuncRepository.
This release adds support for can_torch_compile/can_backward to FuncRepository.
As announced by deprecation warnings in previous releases, specifying the kernel version is now required when loading a kernel. E.g.
As announced by deprecation warnings in previous releases, specifying the kernel version is now required when loading a kernel. E.g.
# Not valid anymore!
activation = kernels.get_kernel("kernels-community/activation")
is now invalid, instead use:
activation = kernels.get_kernel("kernels-community/activation", version=1)
The Hub page for a kernel shows the latest available kernel version. kernels will also warn if the loaded kernel is not the latest version. Full specification of the version helps avoiding breaking existing code as a result of kernel API changes. When the API of a kernel changes, the kernel author must bump up the API version so that downstream code that hasn't been updated for the API change yet, can continue to use the previous version.
This release adds support for the Torch stable ABI. When a kernel uses the Torch stable API and sets the the ABI version in build.toml, the kernel will be built to be compatible with that Torch version and later. For instance, the targeted Torch version can be set to 2.10 by setting stable-abi in build.toml:
[torch]
stable-abi = "2.10"
Using the stable ABI has large benefits:
Functions like get_kernel that normally use rely on network access now work with HF_HUB_OFFLINE=1. The kernel will be loaded if it was downloaded before, otherwise an exception will be raised. Using HF_HUB_OFFLINE disables trusted publisher verification (since this requires internet access).
kernel-builder now offers a skill for writing Intel XPU kernels contributed by @danielfleischer. For instance, to add the XPU kernels skill for Claude, use:
$ kernel-builder skills add --claude --skill xpu-kernels
Up till this release, kernel-builder has always linked libstdc++ statically. However, this lead to issues for some kernels where both the statically linked instance and the dynamically linked instance would try to initialize the same global memory, leading to segfaults and other issues. We didn't encounter this behavior before because most kernels only use C++ code for simple wrapping of the actual compute functions. However, we have encountered some kernels using facilities like C++ std::regex, which triggers global locale initialization. To resolve these issues, we switched to dynamic linking of libstdc++.
To enable dynamic linking while still being fully compatile with manylinux_2_28, we rewrap the EL8 gcc toolchain that is used by manylinux_2_28 using Nix and expose it as a stdenv. This allows us to build kernels with this toolchain. For more technical details, see: https://huggingface.co/docs/kernels/builder/design-nix-builder#manylinux228-compatibility
We now have a page that describes how to set up a kernel development environment in your IDE. Currently Visual Studio Code is covered, but we plan to add additional IDEs and editors in the future.
--major option by @danieldk in https://github.com/huggingface/kernels/pull/550Device by @danieldk in https://github.com/huggingface/kernels/pull/571LocalLayerRepository.__str__ by @danieldk in https://github.com/huggingface/kernels/pull/597kernel-builder check-abi by @danieldk in https://github.com/huggingface/kernels/pull/596Full Changelog: https://github.com/huggingface/kernels/compare/v0.14.1...v0.15.1
This is a bugfix release to use the Hub API to check that a publisher is trusted.
This is a bugfix release to use the Hub API to check that a publisher is trusted.
deprecation for version and revision check. by @sayakpaul in https://github.com/huggingface/kernels/pull/450
Kernels are now a separate repository type on the Hub. This brings many usability improvements to kernels. For example, you can view all kernels that are hosted on the Hub on the kernel overview page:
https://huggingface.co/kernels
This page allows you to filter kernels by supported backends (CUDA, XPU, Metal, etc.) and specific accelerators such as NVIDIA H100 or Apple M5 Max. The page for a kernel will also show the supported accelerators, operating systems, architectures and Torch versions. For instance, the flash-attn2 page shows all supported hardware and architectures:
https://huggingface.co/kernels/kernels-community/flash-attn2
Starting with kernels 0.14, we only support the new kernel repository type. If you would like to upload kernels to the Hub, you can request support for kernel repositories for your user or organization under Settings -> Account.
Kernels have had support for optional metadata to state dependencies, etc. However, we have made including metadata mandatory. This allows users of kernels to query metadata such as the kernel's license and name. But it also made it possible to load kernels by their unique identifier that is also used in their Torch ops name. This makes it easier to debug kernels, since the dynamically loaded name corresponds to the operator name.
We have also added support for querying metadata of kernels that have been loaded:
>>> for kernel in kernels.get_loaded_kernels():
... metadata = kernel.metadata
... repo_info = kernel.repo_info
... print(metadata.id, repo_info.repo_id, metadata.backend.backend_type)
_relu_metal_c835f43 kernels-community/relu metal
To improve security and restrict loading of arbitrary code, kernels will by default only load kernels from trusted publishers. To load other kernels, use the trust_remote_code option:
get_kernel("some-other-org/my-kernel", version=1, trust_remote_code=True)
This release adds support for Torch 2.12, currently based on RC9. The main branch will be updated with the final release when it is available, but there are typically no ABI changes in (late) release candidates, so building kernels with RC9 should also work on the final release.
get_loaded_kernels() by @cbensimon in https://github.com/huggingface/kernels/pull/428result, build, or target dir by @danieldk in https://github.com/huggingface/kernels/pull/464Metadata by @danieldk in https://github.com/huggingface/kernels/pull/481build.toml when specified by @danieldk in https://github.com/huggingface/kernels/pull/485None by @danieldk in https://github.com/huggingface/kernels/pull/493make pin-actions target to pin all GitHub actions by @danieldk in https://github.com/huggingface/kernels/pull/498Metadata from kernels-data by @danieldk in https://github.com/huggingface/kernels/pull/499Full Changelog: https://github.com/huggingface/kernels/compare/v0.13.0...v0.14.0
* feat: add test and docs for get_loaded_kernels.
Fix failing tests (#497)
* feat: add test and docs for get_loaded_kernels.
* fix existing failing tests
* fix 2
* style
---------
Co-authored-by: Daniël de Kok <me@danieldk.eu>
Nothing published for this version
kernels 0.13.0 is a feature-packed release with among other things an improved CLI for building kernels (kernel-builder), Torch 2.11 support, and a te
kernels 0.13.0 is a feature-packed release with among other things an improved CLI for building kernels (kernel-builder), Torch 2.11 support, and a tech-preview of TVM FFI support.
kernel-builder CLI overhaulThe build2cmake command has been renamed to kernel-builder. This new tool can be used to develop, build, and upload kernels without directly using Nix.
These are the main subcommands for the new kernel-builder CLI:
kernel-builder init: scaffold a new kernel, including tests and benchmarks.kernel-builder build: build a kernel.kernel-builder build-and-copy: build a kernel and copy artifacts to the build directory.kernel-builder build-and-upload: build a kernel and upload it to the Hub.kernel-builder create-pyproject: create Python project such as pyproject.toml to develop kernels in IDEs and editors.kernel-builder devshell / kernel-builder testshell — drop into a development or test shell for a kernel.kernel-builder upload: upload a built kernel to the Hugging Face Hub.kernel-builder list-variants — list all supported build variants for a kernel.The build, devshell, and testshell subcommands accept a --variant flag to select a specific build variant. All subcommands accept a directory argument instead of requiring a specific working directory.
An installation script is also provided to help new users get a working kernel-builder environment set up quickly, including Nix, the binary cache, and the required trusted-user configuration. Go to the following page for information on how to get started:
https://huggingface.co/docs/kernels/main/en/builder/writing-kernels#quick-install
kernel-builder now supports Torch 2.11. Torch 2.9 support has been removed in accordance with our policy of supporting the two latest PyTorch versions.
kernels 0.13 adds support for TVM FFI kernels. TVM FFI aims to be a single ABI for multiple frameworks, such as Torch, JAX, NumPy, and CuPy. TVM FFI support is a tech preview. For instance, we might still make changes to the build.toml options for TVM FFI, change the kernel source layout, or change the provided helper functions.
The kernels examples directory provides ReLU and CUTLASS example kernels that use TVM FFI.
kernel-builder now supports card filling. If the kernel source repository contains a CARD.md template, building a kernel will fill the template with details about the kernel. When a kernel is uploaded (with kernel-builder upload or kernel-builder build-and-upload), the card will be uploaded as the README.md of the Hub repository. The default card template can be generated with kernels init.
kernels skillsWe added a new CLI command for installing an agent-compatible skill. Use kernels skills add to install the skills for AI coding assistants like Claude, Codex, and OpenCode. For now, only the cuda-kernels skill is supported. Skill files are downloaded from the huggingface/kernels directory in this repository. ROCm kernel skills are on the way.
Kernels can now be overridden locally without changing any get_kernel call sites. Set the LOCAL_KERNELS environment variable to a colon-separated list of org/repo=local_path pairs:
LOCAL_KERNELS=kernels-community/activation=/path/to/local/activation
This is useful for testing kernel changes locally before uploading them to the Hub.
This is useful when running some operations on CPU while the rest of the model runs on a GPU.
Large kernel uploads are now automatically split across multiple commits to stay within Hub limits, rather than failing or requiring manual intervention for kernels with many files.
metadata.json is correctly added to the output of Windows builds by @danieldk in https://github.com/huggingface/kernels/pull/242torchVersions argument of genKernelFlakeOutputs by @danieldk in https://github.com/huggingface/kernels/pull/246render_binding and render_extensions by @danieldk in https://github.com/huggingface/kernels/pull/248setup.py and move writing to common module by @danieldk in https://github.com/huggingface/kernels/pull/250common by @danieldk in https://github.com/huggingface/kernels/pull/249render_deps function by @danieldk in https://github.com/huggingface/kernels/pull/251cli by @danieldk in https://github.com/huggingface/kernels/pull/269get_kernel: support specifying the backend by @danieldk in https://github.com/huggingface/kernels/pull/268kernels skills add to the cli by @burtenshaw in https://github.com/huggingface/kernels/pull/278ci-test package by @danieldk in https://github.com/huggingface/kernels/pull/281repo_id in the card usage. by @sayakpaul in https://github.com/huggingface/kernels/pull/284python_dependencies.json path by @danieldk in https://github.com/huggingface/kernels/pull/310build2cmake -> kernel-builder, move nix bits to nix-builder by @danieldk in https://github.com/huggingface/kernels/pull/340kernel-builder generate -> kernel-builder create-pyproject by @danieldk in https://github.com/huggingface/kernels/pull/344pyproject::compat -> pyproject::common by @danieldk in https://github.com/huggingface/kernels/pull/366completions subcommand for generating shell completions by @danieldk in https://github.com/huggingface/kernels/pull/388main by @sayakpaul in https://github.com/huggingface/kernels/pull/389devshell/testshell by @danieldk in https://github.com/huggingface/kernels/pull/394metadata.json and build.toml datastructures to separate crate by @danieldk in https://github.com/huggingface/kernels/pull/395strict from huggingface_hub by @sayakpaul in https://github.com/huggingface/kernels/pull/393kernel-builder upload in the Nix builder by @danieldk in https://github.com/huggingface/kernels/pull/406torch attr by @danieldk in https://github.com/huggingface/kernels/pull/426system_variants by @danieldk in https://github.com/huggingface/kernels/pull/423init/generate-readme subcommands by @danieldk in https://github.com/huggingface/kernels/pull/438kernel-builder in place of nix by @danieldk in https://github.com/huggingface/kernels/pull/437Full Changelog: https://github.com/huggingface/kernels/compare/v0.12.3...v0.13.0
Backport kernels changes for FA4 support by @danieldk in https://github.com/huggingface/kernels/pull/383
kernels changes for FA4 support by @danieldk in https://github.com/huggingface/kernels/pull/383Full Changelog: https://github.com/huggingface/kernels/compare/v0.12.2...v0.12.3
This release add experimental Neuron + NKI support to kernels. build2cmake support is currently only available on the main branch.
This release add experimental Neuron + NKI support to kernels. build2cmake support is currently only available on the main branch.
Full Changelog: https://github.com/huggingface/kernels/compare/v0.12.1...v0.12.2
Set version to 0.12.1.dev0 by @danieldk in https://github.com/huggingface/kernels/pull/233
Full Changelog: https://github.com/huggingface/kernels/compare/v0.12.0...v0.12.1
For more information, refer to the guide to adopting kernel versions. Getting kernels without a version is deprecated in kernels 0.12 and will become…
kernel-builder has been merged into the kernels repository. This makes it easier for us to coordinate changes that affect both the kernels Python library and the builder. To switch to the new repo when building kernels, replace the following line in flake.nix
kernel-builder.url = "github:huggingface/kernel-builder";
by
kernel-builder.url = "github:huggingface/kernels";
As a result of the merge, the documentation of kernel-builder is now also available at: https://huggingface.co/docs/kernels/
Before kernels 0.12, kernels could be pulled from a repository without specifying a version. This led to the issue that kernels would typically pull from main. As a result, incompatible changes to the main branch would break downstream use of kernels. To avoid this in the future, we introduce kernel versions. In the future, each kernel will have a major version and when a kernel is uploaded to the Hub, it will be uploaded to the corresponding version branch. The kernel author will bump the kernel version when there are incompatible changes. In this way, kernels can evolve their APIs without breakage for existing users. Versioning can be enabled for a kernel by specifying the version in build.toml:
[general]
version = 1
This will add the kernel version to the kernel's metadata and the kernel upload command will upload builds to the v1 version branch.
Kernel users can pull from a version branch using the version argument. For example:
activation = get_kernel("kernels-community/activation", version=1)
For more information, refer to the guide to adopting kernel versions. Getting kernels without a version is deprecated in kernels 0.12 and will become an error in 0.14 (except for local kernels).
Support for PyTorch 2.10 has been added to the builder. Support for Torch 2.8 has been removed in accordance with our policy to support the two latest Torch versions.
kernels 0.12 adds the experimental kernels benchmark subcommand. This will run benchmarks for a given kernel, if available. The kernels benchmark command will be extended and documented in the upcoming releases.
kernel-builder into the kernels repo by @danieldk in https://github.com/huggingface/kernels/pull/215get_kernel call in kernels README by @danieldk in https://github.com/huggingface/kernels/pull/224kernels upload without a benchmark by @danieldk in https://github.com/huggingface/kernels/pull/230Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.7...v0.12.0
Set version to 0.11.7.dev0 by @danieldk in https://github.com/huggingface/kernels/pull/204
Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.6...v0.11.7
Set version to 0.11.6.dev0 by @danieldk in https://github.com/huggingface/kernels/pull/199
Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.5...v0.11.6
Set version to 0.11.5.dev0 by @danieldk in https://github.com/huggingface/kernels/pull/196
Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.4...v0.11.5
This release extends support for curated Python dependencies and synchronizes support with upcoming kernel-builder changes.
This release extends support for curated Python dependencies and synchronizes support with upcoming kernel-builder changes.
kernel-builder main branch by @danieldk in https://github.com/huggingface/kernels/pull/195Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.3...v0.11.4
Up until now, it was only possible to extend existing layers with kernel layers from the Hub. Starting with this release it's also possible to extend
Up until now, it was only possible to extend existing layers with kernel layers from the Hub. Starting with this release it's also possible to extend them with kernel functions from the Hub. For instance, a silu-and-mul layer
@use_kernel_forward_from_hub("SiluAndMul")
class SiluAndMul(nn.Module):
def forward(self, input: torch.Tensor) -> torch.Tensor:
d = input.shape[-1] // 2
return F.silu(input[..., :d]) * input[..., d:]
can now be extended with a silu_and_mul function from the Hub:
with use_kernel_mapping({
"SiluAndMul": {
"cuda": FuncRepository(
repo_id="kernels-community/activation",
func_name="silu_and_mul",
),
}
}):
kernelize(...)
We have added the FuncRepository, LocalFuncRepository, and LockedFuncRepository classes to load functions from regular, local, and locked repositories.
The counterpart to the previous enhancement is that functions can now also be made extensible using the new use_kernel_func_from_hub decorator:
@use_kernel_forward_from_hub("silu_and_mul")
def silu_and_mul(x: torch.Tensor) -> torch.Tensor:
d = x.shape[-1] // 2
return F.silu(x[..., :d]) * x[..., d:]
This will implicitly replace the function by a Torch nn.Module. Since Torch modules implement __call__, it can still be called as a function:
out = silu_and_mul(x)
However, when the function stored as part of a model/layer, it will also be kernelized:
class FeedForward(nn.Module):
def __init__(self, in_features: int, out_features: int):
self.linear = nn.Linear(in_features, out_features)
# Note: silu_and_mul is a Torch module.
self.silu_and_mul = silu_and_mul
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.silu_and_mul(self.linear(x))
Similar to layers, the function can be kernelized using both a Hub layer and a Hub function.
kernels.layer into several modules by @danieldk in https://github.com/huggingface/kernels/pull/187Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.2...v0.11.3
This change introduces support for loading noarch kernels. In the future, we will start emitting deprecation warnings for universal kernels (to eventu…
This version supports the new noarch build variant that replaces universal kernels. Noarch builds use the build variant format torch-<backend>. This solves two issues that the universal variant has:
This change introduces support for loading noarch kernels. In the future, we will start emitting deprecation warnings for universal kernels (to eventually remove support).
Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.1...v0.11.2
This version adds support for kernel Python dependencies. So far, we mostly considered kernels to be either pure PyTorch + Triton _or_ compiled CUDA/R
This version adds support for kernel Python dependencies. So far, we mostly considered kernels to be either pure PyTorch + Triton or compiled CUDA/ROCm/XPU with a small Torch wrapper. This assumption made kernels easy to deploy everywhere, since they do not have external dependencies. However, DSLs for writing kernels, such as the CUTLASS DSL, are becoming increasingly popular.
To accommodate such DSLs without bringing back the issues that dependencies have, we allow a small, curated set of dependencies. This kernels release implements the client side, validating dependencies and checking that they are installed. The next version of kernel-builder will add dependencies to builds.
build directoriesThus far, kernels were stored in build/<variant>/<module_name>. This version of kernels also supports kernels that are stored in build/<variant>. This solves the issue where are kernel cannot be loaded when module_name does not match the repository name (e.g. after a rename). The next version of kernel-builder will build such flattened kernels (with a compatibility layer for older versions of kernels).
to-wheel: support flat builds by @danieldk in https://github.com/huggingface/kernels/pull/185Full Changelog: https://github.com/huggingface/kernels/compare/v0.11.0...v0.11.1
get_kernel supports two types of new build variants
get_kernel supports two types of new build variants
cpu build variant, this will be used as a fall-back if the Torch build not support CUDA, Metal, ROCm, or XPU.Support for registering CPU layers will be added in a future release.
kernels upload improvementskernels upload supports uploading from a result or build directory in addition to uploading from the main kernel directory.
The --repo_id option of kernels upload has been renamed to --repo-id for consistency.
kernel -> kernels tag by @danieldk in https://github.com/huggingface/kernels/pull/173Full Changelog: https://github.com/huggingface/kernels/compare/v0.10.4...v0.11.0
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
start to build xpu kernels from torch 2.7 by @sywangyi in https://github.com/huggingface/kernels/pull/165
Full Changelog: https://github.com/huggingface/kernels/compare/v0.10.3...v0.10.4
This release adds a check subcommand to the kernels command. This subcommand can be used to check the ABI compatibility of a kernel on the Hub. For ex
kernels checkThis release adds a check subcommand to the kernels command. This subcommand can be used to check the ABI compatibility of a kernel on the Hub. For example:
$ kernels check kernels-community/activation
Checking variant: torch29-cxx11-cu130-x86_64-linux
Dynamic library activation/_activation_beeaae6.abi3.so:
🐍 Python ABI 3.9 compatible
🐧 manylinux_2_28 compatible
[...]
Checking variant: torch29-cxx11-cu126-x86_64-linux
Dynamic library activation/_activation_beeaae6.abi3.so:
🐍 Python ABI 3.9 compatible
🐧 manylinux_2_28 compatible
kernels upload now has an additional --branch option to upload a kernel to a branch.
Makefile to run formatting one-shot by @sayakpaul in https://github.com/huggingface/kernels/pull/157kernels check subcommand by @danieldk in https://github.com/huggingface/kernels/pull/158torch.compile by @sayakpaul in https://github.com/huggingface/kernels/pull/159Full Changelog: https://github.com/huggingface/kernels/compare/v0.10.2...v0.10.3
This release adds full support for Intel XPU devices, including kernel layers. XPU variants use the form: torch -cxx -xpu -x86_64-linux.
This release adds full support for Intel XPU devices, including kernel layers. XPU variants use the form: torch<torch-version>-cxx<C++-ABI>-xpu<OneAPI-version>-x86_64-linux.
kernel upload utilityUpload kernels to the Hub in a single command. For example, to upload the kernel in the current directory:
$ kernels upload . --repo_id="username/kernelname"
The repository will also be created (publicly) if it does not exist yet. For more information, see the documentation.
get_kernel/LayerRepository version argument by @danieldk in https://github.com/huggingface/kernels/pull/147to-wheel subcommand by @danieldk in https://github.com/huggingface/kernels/pull/149forward methods? by @danieldk in https://github.com/huggingface/kernels/pull/153kernels upload tests to be staging tests by @sayakpaul in https://github.com/huggingface/kernels/pull/152Full Changelog: https://github.com/huggingface/kernels/compare/v0.10.1...v0.10.2
XPU: look up kernel by framework version by @sywangyi in https://github.com/huggingface/kernels/pull/139
Full Changelog: https://github.com/huggingface/kernels/compare/v0.10.0...v0.10.1
Before this release, get_local_kernel would only work with the top-level kernel directory (that contains build). This function will now also work with
Before this release, get_local_kernel would only work with the top-level kernel directory (that contains build). This function will now also work with the top-level directory (mykernel), the build directory (mykernel/build), and the build variant directory (mykernel/build/torch28-cxx11-cu128-x86_64-linux).
The default for the mode argument of kernelize is removed.
Before this change, the default mode was Mode.TRAIN | Mode.COMPILE. This had the benefit that by default, kernelize would use kernels that support all use cases. However, it would skip e.g. inference-only kernels, which degrades performance when the user forgets to set mode when using kernels for inference.
cpu is not (yet) a supported device type by @danieldk in https://github.com/huggingface/kernels/pull/132mode argument of kernelize by @danieldk in https://github.com/huggingface/kernels/pull/136Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
This release adds the rocm device type. For instance to register a kernel that supports both CUDA and ROCm, you can use:
This release adds the rocm device type. For instance to register a kernel that supports both CUDA and ROCm, you can use:
kernel_layer_mapping = {
"SiluAndMul": {
"cuda": LayerRepository(
repo_id="kernels-community/activation",
layer_name="SiluAndMul",
),
"rocm": LayerRepository(
repo_id="kernels-community/activation",
layer_name="SiluAndMul",
)
}
}
register_kernel_mapping(kernel_layer_mapping)
For development and debugging it can often be useful to load kernel layers from a local directory. This is supported by the new LocalLayerRepository class. You can directly use the output of kernel-builder. For example:
kernel_layer_mapping = {
"SiluAndMul": {
"cuda": LocalLayerRepository(
repo_path="/home/daniel/kernels/activation",
package_name="activation",
layer_name="SiluAndMul",
)
}
}
register_kernel_mapping(kernel_layer_mapping)
LocalLayerRepository to load from a local repo by @danieldk in https://github.com/huggingface/kernels/pull/123Full Changelog: https://github.com/huggingface/kernels/compare/v0.8.1...v0.9.0
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
get_kernel adds a version argument, which you can use to fetch the latest version compatible with the given version specifier:
get_kernel adds a version argument, which you can use to fetch the latest version compatible with the given version specifier:
activation = kernels.get_kernel("kernels-community/activation", version=">=0.0.3,<0.1")
Version bounds are now also supported when registering layers:
kernel_layer_mapping = {
"SiluAndMul": {
"cuda": LayerRepository(
repo_id="kernels-community/activation",
version=">=0.0.3,<0.1",
layer_name="SiluAndMul",
)
}
}
Layers can now also use version locks from kernels.lock by using LockedLayerRepository:
kernel_layer_mapping = {
"SiluAndMul": {
"cuda": LockedLayerRepository(
repo_id="kernels-community/activation",
layer_name="SiluAndMul",
)
}
}
See the kernel locking documentation for more information.
get_kernel: allow Python-style version specifiers by @danieldk in https://github.com/huggingface/kernels/pull/111LayerRepository by @danieldk in https://github.com/huggingface/kernels/pull/113Full Changelog: https://github.com/huggingface/kernels/compare/v0.8.0...v0.8.1
Before this release, when using kernelize with a mode, it would only look up the exact match in the kernel mapping. Starting with this release, kernel
Before this release, when using kernelize with a mode, it would only look up the exact match in the kernel mapping. Starting with this release, kernelize will fall back to other compatible modes. For instance, when a model is kernelized as
model = kernelize(model, mode=Mode.INFERENCE)
kernelize will try the following kernel mappings (in-order):
Mode.INFERENCEMode.INFERENCE | Mode.TORCH_COMPILEMode.TRAININGMode.TRAINING | Mode.TORCH_COMPILEMode.FALLBACKsince all these modes are compatible with inference. See the kernel modes documentation for more information and a list per mode of the possible fallbacks.
It is now possible to register multiple CUDA kernels with different capabilities. This will allow you to provide e.g. different kernels for Ada, Hopper, and Blackwell GPUs. See the docs on Registering kernels for specific CUDA capabilities for more information.
Mode.DEFAULT has been renamed to Mode.FALLBACK for clarity.
Full Changelog: https://github.com/huggingface/kernels/compare/v0.7.0...v0.8.0
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
This version contains an API change to the kernelize function that makes it possible to use different kernels for inference/training/torch.compile. Th
This version contains an API change to the kernelize function that makes it possible to use different kernels for inference/training/torch.compile. This requires a small adjustment to how kernelize is called, see the kernelize documentation for more information. In short, to kernelize a model for inference, use:
model = MyModel(...)
model = kernelize(model, mode=Mode.INFERENCE)
For training:
model = MyModel(...)
model = kernelize(model, mode=Mode.TRAINING)
get_local_kernel function by @danieldk in https://github.com/huggingface/kernels/pull/102Full Changelog: https://github.com/huggingface/kernels/compare/v0.6.2...v0.7.0
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
CI: main triton-layer-norm has docs, branch is gone by @danieldk in https://github.com/huggingface/kernels/pull/99
Full Changelog: https://github.com/huggingface/kernels/compare/v0.6.1...v0.6.2
This release adds an experimental generate-readme subcommand to the kernels utility. This command generates a README containing the API docs for a ker
This release adds an experimental generate-readme subcommand to the kernels utility. This command generates a README containing the API docs for a kernel. Functions and layers can be annotated with Hugging Face doc-builder formatting.
Example kernel with API docs: https://huggingface.co/kernels-community/triton-layer-norm
The layer API has been redesigned to make it compatible with torch.compile. Prior to this change, the use_kernel_forward_from_hub decorator would repl
The layer API has been redesigned to make it compatible with torch.compile. Prior to this change, the use_kernel_forward_from_hub decorator would replace a layer's forward method by one that uses dynamic kernel dispatch. However, this type of data-dependent branching is not compatible with torch compile.
In the new API, use_kernel_forward_from hub only associates a layer with a kernel name. Then the replacement of forward methods is done by a new kernelize function:
from kernels import kernelize
model = ModelWithKernelLayers(...)
kernelize(model)
Y = model(X)
See the layers documentation for more information about the layer API.
kernels now has experimental support for loading Metal kernels. For example:
import torch
from kernels import get_kernel
relu = get_kernel("kernels-test/relu-metal")
x = torch.arange(-10, 10, dtype=dtype, device="mps")
y = metal_kernel.relu(x)
The kernels utility now has a to-wheel subcommand for converting Hub kernels to Python wheels for legacy deployment scenarios. For example, to make Python wheels for the activation kernel:
$ uv run kernels to-wheel kernels-community/activation 0.0.3
...
☸️ activation-0.0.3+torch25cu124cxx98-cp39-abi3-manylinux_2_28_x86_64.whl
☸️ activation-0.0.3+torch26cu124cxx98-cp39-abi3-manylinux_2_28_x86_64.whl
☸️ activation-0.0.3+torch27cu118cxx11-cp39-abi3-manylinux_2_28_x86_64.whl
Full Changelog: https://github.com/huggingface/kernels/compare/v0.5.0...v0.6.0
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
locking docs: fix command name (kernel -> kernels) by @danieldk in https://github.com/huggingface/kernels/pull/74
kernel -> kernels) by @danieldk in https://github.com/huggingface/kernels/pull/74torch.compile by @danieldk in https://github.com/huggingface/kernels/pull/79Full Changelog: https://github.com/huggingface/kernels/compare/v0.4.4...v0.5.0
Nothing published for this version
Backport: support noarch build variants (torch- ), see #190.
Backport: support noarch build variants (torch-<backend>), see #190.
Do not use kernels without backward when training by @danieldk in https://github.com/huggingface/kernels/pull/68
has_kernel function by @danieldk in https://github.com/huggingface/kernels/pull/69Full Changelog: https://github.com/huggingface/kernels/compare/v0.4.3...v0.4.4
Add more details about the ABI requirements by @danieldk in https://github.com/huggingface/kernels/pull/63
manylinux_2_28 by @danieldk in https://github.com/huggingface/kernels/pull/65DISABLE_KERNEL_MAPPING env var for completely disabling kernel mappings by @danieldk in https://github.com/huggingface/kernels/pull/70Full Changelog: https://github.com/huggingface/kernels/compare/v0.4.2...v0.4.3
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
One column per month.