NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #285 most downloaded on PyPI
ONNX Runtime is a runtime accelerator for Machine Learning models
Latest stable
1.32.0
Last release 9 days ago
13 Nov 2026
Ships fairly regularly
a new release about every 3 weeks
Nearly every release is documented
notes for 59 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
8 years old
65 releases · first in 2018
Drag across the ruler to compare, or tap a release to open it.
All ONNX Runtime Training packages have been deprecated. ORT 1.19.2 was the last release for which onnxruntime-training (PyPI), onnxruntime-training-c…
Release Manager: @apsonawane
Full release notes for ONNX Runtime generate() API v0.5.0 can be found here.
Full release notes for ONNX Runtime Extensions v0.13 can be found here.
Big thank you to the release manager @apsonawane, as well as @snnn, @jchen351, @sheetalarkadam, and everyone else who made this release possible!
Tianlei Wu, Yi Zhang, Yulong Wang, Scott McKay, Edward Chen, Adrian Lizarraga, Wanming Lin, Changming Sun, Dmitri Smirnov, Jian Chen, Jiajia Qin, Jing Fang, George Wu, Caroline Zhu, Hector Li, Ted Themistokleous, mindest, Yang Gu, jingyanwangms, liqun Fu, Adam Pocock, Patrice Vignola, Yueqing Zhang, Prathik Rao, Satya Kumar Jandhyala, Sumit Agarwal, Xu Xing, aciddelgado, duanshengliu, Guenther Schmuelling, Kyle, Ranjit Ranjan, Sheil Kumar, Ye Wang, kunal-vaishnavi, mingyueliuh, xhcao, zz002, 0xdr3dd, Adam Reeve, Arne H Juul, Atanas Dimitrov, Chen Feiyue, Chester Liu, Chi Lo, Erick Muñoz, Frank Dong, Jake Mathern, Julius Tischbein, Justin Chu, Xavier Dupré, Yifan Li, amarin16, anujj, chenduan-amd, saurabh, sfatimar, sheetalarkadam, wejoncy, Akshay Sonawane, AlbertGuan9527, Bin Miao, Christian Bourjau, Claude, Clément Péron, Emmanuel, Enrico Galli, Fangjun Kuang, Hann Wang, Indy Zhu, Jagadish Krishnamoorthy, Javier Martinez, Jeff Daily, Justin Beavers, Kevin Chen, Krishna Bindumadhavan, Lennart Hannink, Luis E. P., Mauricio A Rovira Galvez, Michael Tyler, PARK DongHa, Peishen Yan, PeixuanZuo, Po-Wei (Vincent), Pranav Sharma, Preetha Veeramalai, Sophie Schoenmeyer, Vishnudas Thaniel S, Xiang Zhang, Yi-Hong Lyu, Yufeng Li, goldsteinn, mcollinswisc, mguynn-intc, mingmingtasd, raoanag, shiyi, stsokolo, vraspar, wangshuai09
Full changelog: v1.19.2...v1.20.0
Your coding agent can read these notes before it upgrades. Set up the MCP server →
ORT 1.19.2 is a small patch release, fixing some broken workflows and introducing bug fixes.
@prathikr, @mszhanyi, @edgchen1, @tianleiwu, @wangyems, @aciddelgado, @mindest, @snnn, @baijumeswani, @MaanavD
Thanks to everyone who helped ship this release smoothly!
Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.19.0...v1.19.2
Remove calls to deprecated api’s
TensorRT
CUDA
CPU
QNN
OpenVINO
DirectML
Changming Sun, Baiju Meswani, Scott McKay, Edward Chen, Jian Chen, Wanming Lin, Tianlei Wu, Adrian Lizarraga, Chester Liu, Yi Zhang, Yulong Wang, Hector Li, kunal-vaishnavi, pengwa, aciddelgado, Yifan Li, Xu Xing, Yufeng Li, Patrice Vignola, Yueqing Zhang, Jing Fang, Chi Lo, Dmitri Smirnov, mingyueliuh, cloudhan, Yi-Hong Lyu, Ye Wang, Ted Themistokleous, Guenther Schmuelling, George Wu, mindest, liqun Fu, Preetha Veeramalai, Justin Chu, Xiang Zhang, zz002, vraspar, kailums, guyang3532, Satya Kumar Jandhyala, Rachel Guo, Prathik Rao, Maximilian Müller, Sophie Schoenmeyer, zhijiang, maggie1059, ivberg, glen-amd, aamajumder, Xavier Dupré, Vincent Wang, Suryaprakash Shanmugam, Sheil Kumar, Ranjit Ranjan, Peishen Yan, Frank Dong, Chen Feiyue, Caroline Zhu, Adam Louly, Ștefan Talpalaru, zkep, winskuo-quic, wejoncy, vividsnow, vivianw-amd, moyo1997, mcollinswisc, jingyanwangms, Yang Gu, Tom McDonald, Sunghoon, Shubham Bhokare, RuomeiMS, Qingnan Duan, PeixuanZuo, Pavan Goyal, Nikolai Svakhin, KnightYao, Jon Campbell, Johan MEJIA, Jake Mathern, Hans, Hann Wang, Enrico Galli, Dwayne Robinson, Clément Péron, Chip Kerchner, Chen Fu, Carson M, Adam Reeve, Adam Pocock.
Big thank you to everyone who contributed to this release!
Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.18.1...v1.19.0
ONNX Runtime Python packages now have numpy dependency >=1.21.6, <2.0. Support for numpy 2.0 will be added in a future release.
Announcements:
TensorRT EP:
Packages:
This patch release also features various other bug fixes, including a CUDA 12.5 build error fix.
Big thank you to @yf711 for driving this release as the release manager and to all our contributors!
@yf711 @jchen351 @mszhanyi @snnn @wangyems @jywu-msft @skottmckay @chilo-ms @moraxu @kevinch-nv @pengwa @wejoncy @pranavsharma @Craigacp @jslhcl @adrianlizarraga @inisis @jeffbloo @mo-ja @kunal-vaishnavi @sumitsays @neNasko1 @yufenglee @dhruvbird @wangshuai09 @xiaoyu-work @axinging @yuslepukhin @YUNQIUGUO @shubhambhokare1 @fs-eire @afantino951 @tboby @HectorSVC @baijumeswani
…iOS cocoapods are being deprecated. Please use the onnxruntime-android Android package, and onnxruntime-c/onnxruntime-objc cocoapods, which support ON…
--rv64, --riscv_toolchain_root, and --riscv_qemu_path.--use_binskim_compliant_compile_flags. Note: All our release binaries are built with this flag, but when building ONNX Runtime from source, this flag is default OFF.TensorRT
CUDA
QNN
OpenVINO
AUTO:GPU,CPU will only create GPU blob, not CPU blob.DirectML
Yi Zhang, Yulong Wang, Adrian Lizarraga, Changming Sun, Scott McKay, Tianlei Wu, Peng Wang, Hector Li, Edward Chen, Dmitri Smirnov, Patrice Vignola, Guenther Schmuelling, Ye Wang, Chi Lo, Wanming Lin, Xu Xing, Baiju Meswani, Peixuan Zuo, Vincent Wang, Markus Tavenrath, Lei Cao, Kunal Vaishnavi, Rachel Guo, Satya Kumar Jandhyala, Sheil Kumar, Yifan Li, Jiajia Qin, Maximilian Müller, Xavier Dupré, Yi-Hong Lyu, Yufeng Li, Alejandro Cid Delgado, Adam Louly, Prathik Rao, wejoncy, Zesong Wang, Adam Pocock, George Wu, Jian Chen, Justin Chu, Xiaoyu, guyang3532, Jingyan Wang, raoanag, Satya Jandhyala, Hariharan Seshadri, Jiajie Hu, Sumit Agarwal, Peter Mcaughan, Zhijiang Xu, Abhishek Jindal, Jake Mathern, Jeff Bloomfield, Jeff Daily, Linnea May, Phoebe Chen, Preetha Veeramalai, Shubham Bhokare, Wei-Sheng Chin, Yang Gu, Yueqing Zhang, Guangyun Han, inisis, ironman, Ivan Berg, Liqun Fu, Yu Luo, Rui Ren, Sahar Fatima, snadampal, wangshuai09, Zhenze Wang, Andrew Fantino, Andrew Grigorev, Ashwini Khade, Atanas Dimitrov, AtomicVar, Belem Zhang, Bowen Bao, Chen Fu, Dhruv Matani, Fangrui Song, Francesco, Frank Dong, Hans Chen, He Li, Heflin Stephen Raj, Jambay Kinley, Masayoshi Tsutsui, Matttttt, Nanashi, Phoebe Chen, Pranav Sharma, Segev Finer, Sophie Schoenmeyer, TP Boudreau, Ted Themistokleous, Thomas Boby, Xiang Zhang, Yongxin Wang, Zhang Lei, aamajumder, danyue, Duansheng Liu, enximi, fxmarty, kailums, maggie1059, mindest, mo-ja, moyo1997 Big thank you to everyone who contributed to this release!
Update copying API header files to make Linux logic consistent with Windows (#19736) - @mszhanyi
General:
Build System & Packages:
Core:
CUDA EP:
TensorRT EP:
Web:
Windows:
Kernel Optimizations:
Models:
This patch release also includes additional fixes by @spampana95 and @enximi. Big thank you to all our contributors!
This patch release includes the following updates:
This patch release includes the following updates:
Updated calls to deprecated TensorRT APIs (e.g., enqueue_v2 → enqueue_v3).
In the next release, we will totally drop support for Windows ARM32.
python -m pip install cerberus flatbuffers h5py numpy>=1.16.6 onnx packaging protobuf sympy setuptools>=41.4.0
pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT/pypi/simple/ onnxruntime-training
pip install torch-ort
python -m torch_ort.configure
Installation instructions can also be accessed here.Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: Changming Sun, Yulong Wang, Tianlei Wu, Yi Zhang, Jian Chen, Jiajia Qin, Adrian Lizarraga, Scott McKay, Wanming Lin, pengwa, Hector Li, Chi Lo, Dmitri Smirnov, Edward Chen, Xu Xing, satyajandhyala, Rachel Guo, PeixuanZuo, RandySheriffH, Xavier Dupré, Patrice Vignola, Baiju Meswani, Guenther Schmuelling, Jeff Bloomfield, Vincent Wang, cloudhan, zesongw, Arthur Islamov, Wei-Sheng Chin, Yifan Li, raoanag, Caroline Zhu, Sheil Kumar, Ashwini Khade, liqun Fu, xhcao, aciddelgado, kunal-vaishnavi, Aditya Goel, Hariharan Seshadri, Ye Wang, Adam Pocock, Chen Fu, Jambay Kinley, Kaz Nishimura, Maximilian Müller, Yang Gu, guyang3532, mindest, Abhishek Jindal, Justin Chu, Numfor Tiapo, Prathik Rao, Yufeng Li, cao lei, snadampal, sophies927, BoarQing, Bowen Bao, George Wu, Jiajie Hu, MistEO, Nat Kershaw (MSFT), Sumit Agarwal, Ted Themistokleous, ivberg, zhijiang, Christian Larson, Frank Dong, Jeff Daily, Nicolò Lucchesi, Pranav Sharma, Preetha Veeramalai, Cheng Tang, Xiang Zhang, junchao-loongson, petermcaughan, rui-ren, shaahji, simonjub, trajep, Adam Louly, Akshay Sonawane, Artem Shilkin, Atanas Dimitrov, AtanasDimitrovQC, BODAPATIMAHESH, Bart Verhagen, Ben Niu, Benedikt Hilmes Brian Lambert, David Justice, Deoksang Kim, Ella Charlaix, Emmanuel Ferdman, Faith Xu, Frank Baele, George Nash, hans00, computerscienceiscool, Jake Mathern, James Baker, Jiangzhuo, Kevin Chen, Lennart Hannink, Lukas Berbuer, Mike Guo, Milos Puzovic, Mustafa Ateş Uzun, Peishen Yan, Ran Gal, Ryan Hill, Steven Roussey, Suryaprakash Shanmugam, Vadym Stupakov, Yiming Hu, Yueqing Zhang, Yvonne Chen, Zhang Lei, Zhipeng Han, aimilefth, gunandrose4u, kailums, kushalpatil07, kyoshisuki, luoyu-intel, moyo1997, tbqh, weischan-quic, wejoncy, winskuo-quic, wirthual, yuwenzho
Stable Diffusion XL demo update by @tianleiwu in https://github.com/microsoft/onnxruntime/pull/18496
The patch release includes updates on:
The patch release includes updates on:
Fix type of weights and activations in the ONNX quantizer
Patch release for 1.16
Support for serialization of models >=2GB
session.disable_cpu_ep_fallbackThis ORT release is accompanied by updates to onnxruntime-extensions. Features include:
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: fs-eire, edgchen1, snnn, pengwa, mszhanyi, PeixuanZuo, tianleiwu, adrianlizarraga, baijumeswani, cloudhan, satyajandhyala, yuslepukhin, RandyShuai, RandySheriffH, skottmckay, Honry, dependabot[bot], HectorSVC, jchen351, chilo-ms, YUNQIUGUO, justinchuby, PatriceVignola, guschmue, yf711, Craigacp, smk2007, RyanUnderhill, jslhcl, wschin, kunal-vaishnavi, mindest, xadupre, fdwr, hariharans29, AdamLouly, wejoncy, chenfucn, pranavsharma, yufenglee, zhijxu-MS, jeffdaily, natke, jeffbloo, liqunfu, wangyems, er3x3, nums11, yihonglyu, sumitsays, zhanghuanrong, askhade, wenbingl, jingyanwangms, ashari4, gramalingam, georgen117, sfatimar, BowenBao, hanbitmyths, stevenlix, jywu-msft
This release fixed the following issues:
This release fixed the following issues:
Added Python 3.11 support (deprecate 3.7, support 3.8-3.11) in packages for Onnxruntime CPU, Onnxruntime-GPU, Onnxruntime-directml, and onnxruntime-tr…
Starting from the next release(ONNX Runtime 1.16.0), at operating system level we will drop the support for
At compiler level we will drop the support for
Also, we will remove the onnxruntime_DISABLE_ABSEIL build option since we will upgrade protobuf and the new protobuf version will need abseil.
Two new execution providers: JS EP and QNN EP.
Added support for built-in pre and post processing for NLP scenarios: classification, question-answering, text-prediction
Added support for built-in pre and post processing for Speech Recognition (Whisper)
Added support for built-in post processing for Object Detection (YOLO). Non-max suppression, draw bounding boxes
Additional CoreML and NNAPI kernels to support customer scenarios
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, fs-eire, edgchen1, wejoncy, mszhanyi, PeixuanZuo, pengwa, jchen351, cloudhan, tianleiwu, PatriceVignola, wangyems, adrianlizarraga, chenfucn, HectorSVC, baijumeswani, justinchuby, skottmckay, yuslepukhin, RandyShuai, RandySheriffH, natke, YUNQIUGUO, smk2007, jslhcl, chilo-ms, yufenglee, RyanUnderhill, hariharans29, zhanghuanrong, askhade, wschin, jywu-msft, mindest, zhijxu-MS, dependabot[bot], xadupre, liqunfu, nums11, gramalingam, Craigacp, fdwr, shalvamist, jstoecker, yihonglyu, sumitsays, stevenlix, iK1D, pranavsharma, georgen117, sfatimar, MaajidKhan, satyajandhyala, faxu, jcwchen, hanbitmyths, jeffbloo, souptc, ytaous kunal-vaishnavi
This patch addresses packaging issues and bug fixes on top of v1.14.0:
This patch addresses packaging issues and bug fixes on top of v1.14.0:
Only select packages were published for this patch release; others can be found in the attachments below:
Building ORT from source will require cmake version >=3.24 instead of >=3.18.
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, skottmckay, edgchen1, hariharans29, tianleiwu, yufenglee, guoyu-wang, yuslepukhin, fs-eire, pranavsharma, iK1D, baijumeswani, tracysh, thiagocrepaldi, askhade, RyanUnderhill, wangyems, fdwr, RandySheriffH, jywu-msft, zhanghuanrong, smk2007, pengwa, liqunfu, shahasad, mszhanyi, SherlockNoMad, xadupre, jignparm, HectorSVC, ytaous, weixingzhang, stevenlix, tiagoshibata, faxu, wschin, souptc, ashbhandare, RandyShuai, chilo-ms, PeixuanZuo, cloudhan, dependabot[bot], jeffbloo, chenfucn, linkerzhang, duli2012, codemzs, oliviajain, natke, YUNQIUGUO, Craigacp, sumitsays, orilevari, BowenBao, yangchen-MS, hanbitmyths, satyajandhyala, MaajidKhan, smkarlap, sfatimar, jchen351, georgen117, wejoncy, PatriceVignola, adrianlizarraga, justinchuby, zhangxiang1993, gineshidalgo99, tlh20, xzhu1900, jeffdaily, suryasidd, yihonglyu, liuziyue, chentaMS, jcwchen, ybrnathan, ajindal1, zhijxu-MS, gramalingam, WilBrady, garymm, kkaranasos, ashari4, martinb35, AdamLouly, zhangyaobit, vvchernov, jingyanwangms, wenbingl, daquexian, sreekanth-yalachigere, NonStatic2014, mayavijx, mindest, jstoecker, manashgoswami, Andrews548, baowenlei, kunal-vaishnavi
A protobuf security issue CVE-2022-1941 that impact users who load ONNX models from untrusted sources, for example, a deep learning inference service…
Add (bias) and Transpose of Q/K/V into one kernel for Attention and LongformerAttention.Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, baijumeswani#2baijumeswani, edgchen1, iK1D, skottmckay, cloudhan, tianleiwu, fs-eire, mszhanyi, WilBrady, hariharans29, chenfucn, fdwr, yuslepukhin, wejoncy, PeixuanZuo, pengwa, yufenglee, jchen351, justinchuby, dependabot[bot], RandySheriffH, sumitsays, wschin, wangyems, YUNQIUGUO, ytaous, pranavsharma, vvchernov, natke, Craigacp, RandyShuai, smk2007, zhangyaobit, jcwchen, yihonglyu, georgen117, chilo-ms, ashbhandare, faxu, jstoecker, gramalingam, garymm, jeffbloo, xadupre, jywu-msft, askhade, RyanUnderhill, thiagocrepaldi, mindest, jingyanwangms, wenbingl, ashari4, sfatimar, MaajidKhan, souptc, HectorSVC, weixingzhang, zhanghuanrong
This patch addresses packaging issues and bug fixes on top of v1.12.0.
This patch addresses packaging issues and bug fixes on top of v1.12.0.
See #12418 for full list of specific fixes included
…API as soon as possible. The legacy API will be deprecated in next release (ORT 1.13).
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, edgchen1, fdwr, skottmckay, iK1D, fs-eire, mszhanyi, WilBrady, justinchuby, tianleiwu, PeixuanZuo, garymm, yufenglee, adrianlizarraga, yuslepukhin, dependabot[bot], chilo-ms, vvchernov, oliviajain, ytaous, hariharans29, sumitsays, wangyems, pengwa, baijumeswani, smk2007, RandySheriffH, gramalingam, xadupre, yihonglyu, zhangyaobit, YUNQIUGUO, jcwchen, chenfucn, souptc, chandru-r, jstoecker, hanbitmyths, RyanUnderhill, georgen117, jywu-msft, mindest, sfatimar, HectorSVC, Craigacp, jeffdaily, zhijxu-MS, natke, stevenlix, jeffbloo, guoyu-wang, daquexian, faxu, jingyanwangms, adtsai, wschin, weixingzhang, wenbingl, MaajidKhan, ashbhandare, ajindal1, zhanghuanrong, tiagoshibata, askhade, liqunfu
This is a patch release on 1.11.0 with the following fixes:
This is a patch release on 1.11.0 with the following fixes:
All official packages are attached, and Python packages are additionally published to PyPi.
C# packages now tested with .NET 5. .NET Core 2.1 support is deprecated as it has reached end of life support on August 21, 2021. We will closely foll…
experimental/opencl branch for users to try. Provide feedback through Issues and Discussions in the repo.Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, edgchen1, skottmckay, yufenglee, wangyems, yuslepukhin, gwang-msft, iK1D, chilo-ms, fdwr, ytaous, RandySheriffH, hanbitmyths, chenfucn, yihonglyu, ajindal1, fs-eire, souptc, tianleiwu, YUNQIUGUO, hariharans29, oliviajain, xadupre, ashari4, RyanUnderhill, jywu-msft, weixingzhang, baijumeswani, georgen117, natke, Craigacp, jeffdaily, JingqiaoFu, zhanghuanrong, satyajandhyala, smk2007, ryanlai2, askhade, thiagocrepaldi, jingyanwangms, pengwa, scxiao, ashbhandare, BowenBao, SherlockNoMad, sumitsays, sfatimar, mosdav, harshithapv, liqunfu, tiagoshibata, gineshidalgo99, pranavsharma, jcwchen, nkreeger, xkszltl, faxu, suffiank, stevenlix, jeffbloo, feihugis
As noted in the deprecation notice in ORT 1.9, InferenceSession now requires the providers parameters to be set when enabling Execution Providers othe…
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: snnn, gineshidalgo99, fs-eire, gwang-msft, edgchen1, hariharans29, skottmckay, jeffdaily, baijumeswani, fdwr, smk2007, suffiank, souptc, RyanUnderhill, iK1D, yuslepukhin, chilo-ms, satyajandhyala, hanbitmyths, thiagocrepaldi, wschin, tianleiwu, pengwa, xadupre, zhanghuanrong, SherlockNoMad, wangyems, RandySheriffH, ashbhandare, tiagoshibata, yufenglee, mindest, sumitsays, MaajidKhan, gramalingam, tracysh, georgen117, jywu-msft, sfatimar, martinb35, nkreeger, ytaous, ashari4, stevenlix, chandru-r, jingyanwangms, mosdav, raviskolli, faxu, liqunfu, kit1980, weixingzhang, pranavsharma, jcwchen, chenfucn, BowenBao, jeffbloo
SessionOptionsAppendExecutionProvider_TensorRT API is deprecated; use SessionOptionsAppendExecutionProvider_TensorRT_V2
uname -m output of your target device.--_use_extensions option to run models with custom operators implemented in onnxruntime-extensionsCUDA EP
TensorRT EP
OpenVINO EP
DirectML EP
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: edgchen1, gwang-msft, tianleiwu, fs-eire, hariharans29, skottmckay, baijumeswani, RyanUnderhill, iK1D, souptc, nkreeger, liqunfu, pengwa, SherlockNoMad, wangyems, chilo-ms, thiagocrepaldi, KeDengMS, suffiank, oliviajain, chenfucn, satyajandhyala, yuslepukhin, pranavsharma, tracysh, yufenglee, hanbitmyths, ytaous, YUNQIUGUO, zhanghuanrong, stevenlix, jywu-msft, chandru-r, duli2012, smk2007, wschin, MaajidKhan, tiagoshibata, xadupre, RandySheriffH, ashbhandare, georgen117, Tixxx, harshithapv, Craigacp, BowenBao, askhade, zhangxiang1993, gramalingam, weixingzhang, natke, tlh20, codemzs, ryanlai2, raviskolli, pranav-prakash, faxu, adtsai, fdwr, wenbingl, jcwchen, neginraoof, cschreib-ibex
This release contains fixes and key updates for 1.8.0. For all package installation details, please refer to https://www.onnxruntime.ai.
This release contains fixes and key updates for 1.8.0. For all package installation details, please refer to https://www.onnxruntime.ai.
Performance updates for ONNX Runtime for PyTorch (training acceleration for PyTorch models)
Building onnxruntime from source now requires a C++ compiler with full C++14 support.
--enable_type_reduction flag, and perform minimal build --enable_reduced_operator_type_support flagMemory footprint reduction through shared pre-packed weights for shared initializers
Memory footprint reduction through arena shrinkage
Quantization
Transformers
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
snnn, gwang-msft, baijumeswani, fs-eire, edgchen1, zhanghuanrong, yufenglee, thiagocrepaldi, hariharans29, skottmckay, weixingzhang, tianleiwu, SherlockNoMad, ashbhandare, tracysh, satyajandhyala, liqunfu, iK1D, RandySheriffH, suffiank, hanbitmyths, wangyems, askhade, stevenlix, chilo-ms, smk2007, kit1980, codemzs, raviskolli, pranav-prakash, chenfucn, xadupre, gramalingam, harshithapv, oliviajain, xzhu1900, ytaous, MaajidKhan, RyanUnderhill, mrry, orilevari, jingyanwangms, sfatimar, KeDengMS, jywu-msft, souptc, adtsai, tlh20, yuslepukhin, duli2012, pranavsharma, faxu, georgen117, jeffbloo, Tixxx, wschin, YUNQIUGUO, tiagoshibata, martinb35, alberto-magni, ryanlai2, Craigacp, suryasidd, fdwr, jcwchen, neginraoof, natke, BowenBao
Deprecated support for OpenVINO 2020.2
Starting from this release, all ONNX Runtime CPU packages are now built without OpenMP. A version with OpenMP is available on Nuget (Microsoft.ML.OnnxRuntime.OpenMP) and PyPi (onnxruntime-openmp). Please report any issues in GH Issues.
Note: The 1.7.0 GPU package is uploaded on this Azure DevOps Feed due to the size limit on Nuget.org. Please use 1.7.1 for the GPU package through Nuget.
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members: edgchen1, snnn, skottmckay, gwang-msft, hariharans29, tianleiwu, xadupre, yufenglee, ryanlai2, wangyems, suffiank, liqunfu, orilevari, baijumeswani, weixingzhang, pranavsharma, RandySheriffH, ashbhandare, oliviajain, smk2007, tracysh, stevenlix, fs-eire, Craigacp, faxu, mrry, codemzs, chilo-ms, jcwchen, zhanghuanrong, SherlockNoMad, iK1D, askhade, zhangxiang1993, yuslepukhin, tlh20, MaajidKhan, wschin, smkarlap, wenbingl, pengwa, duli2012, natke, alberto-magni, Tixxx, HectorSVC, jingyanwangms, jstoecker, kit1980, suryasidd, RandyShuai, sfatimar, jywu-msft, liuziyue, mosdav, thiagocrepaldi, souptc, fdwr
OpenMP will be disabled in future official builds (build option will still be available). A NoOpenMP version of ONNX Runtime is now available with thi
_mm_pause() for spin loop to improve performance and power consumptionOrtValue data structure
OrtValue (check Scenario 4 in the IOBinding section of the documentation for an example)OrtValue instances can be used to bind inputs/outputs. This is in addition to existing interfaces that allows binding a piece of memory directly/using numpy arrays that can be bound and may be particularly useful when binding ORT allocated device memory.ReleaseEnv from DLLMain or put OrtEnv in global variables. It is not safe to call FreeLibrary from DllMain. - referenceContributors to ONNX Runtime include members across teams at Microsoft, along with our community members: gwang-msft, snnn, skottmckay, edgchen1, hariharans29, wangyems, yufenglee, yuslepukhin, tianleiwu, SherlockNoMad, tracysh, ryanlai2, askhade, xadupre, liqunfu, RandySheriffH, jywu-msft, KeDengMS, pranavsharma, mrry, ashbhandare, iK1D, RyanUnderhill, MaajidKhan, wenbingl, kit1980, weixingzhang, tlh20, suffiank, Craigacp, smkarlap, stevenlix, zhanghuanrong, sfatimar, ytaous, tiagoshibata, fdwr, oliviajain, alberto-magni, jcwchen, mosdav, xzhu1900, wschin, codemzs, duli2012, smk2007, natke, zhijxu-MS, manashgoswami, zhangxiang1993, faxu, HectorSVC, take-cheeze, jingyanwangms, chilo-ms, YUNQIUGUO, jgbradley1, jessebenson, martinb35, Andrews548, souptc, pengwa, liuziyue, orilevari, BowenBao, thiagocrepaldi, jeffbloo
This is a minor patch release on 1.5.1 with the following changes:
This is a minor patch release on 1.5.1 with the following changes:
Reduced Operator Kernel build allows ORT binaries to be built with only required operators in the model(s) - learn more
MacOS binaries now rely on openmp to be installed. See this for reference.
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
gwang-msft, snnn, skottmckay, hariharans29, thiagocrepaldi, tianleiwu, wangyems, RandySheriffH, yufenglee, SherlockNoMad, smk2007, jywu-msft, liqunfu, edgchen1, yuslepukhin, tiagoshibata, fdwr, ashbhandare, iK1D, wschin, BowenBao, zhanghuanrong, RyanUnderhill, ryanlai2, askhade, pranavsharma, martinb35, suffiank, ytaous, KeDengMS, rayankrish, natke, YUNQIUGUO, range4life, smkarlap, zhangxiang1993, xzhu1900, codemzs, weixingzhang, stevenlix, tracysh, mosdav, jingyanwangms, tlh20, souptc, orilevari, kit1980, yangchen-MS, faxu, fs-eire, wenbingl, chilo-ms, xkszltl, Andrews548, yuzawa-san, MaximKalininMS, jgbradley1, nickfeeney, zhijxu-MS, Tixxx, suryasidd, Craigacp, duli2012, jeffbloo
Performance optimizations for Transformer models
New Execution Providers available for preview:
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
snnn, tianleiwu, edgchen1, hariharans29, skottmckay, tracysh, yufenglee, fs-eire, codemzs, tiagoshibata, yuslepukhin, gwang-msft, wschin, smk2007, prabhat00155, liuziyue, liqunfu, ytaous, iK1D, BowenBao, askhade, pranavsharma, faxu, jywu-msft, ryanlai2, xzhu1900, KeDengMS, tlh20, smkarlap, weixingzhang, jeffbloo, RyanUnderhill, mrry, jgbradley1, stevenlix, zhanghuanrong, suffiank, Andrews548, pengwa, SherlockNoMad, orilevari, duli2012, yangchen-MS, yan12125, jornt-xilinx, ashbhandare, neginraoof, Tixxx, thiagocrepaldi, Craigacp, mayeut, chilo-ms, prasanthpul, martinb35, manashgoswami, zhangxiang1993, suryasidd, wangyems, kit1980, RandySheriffH, fdwr
Function expansion support that enables several new ONNX 1.7 ops such as NegativeLogLikelihoodLoss, GreaterOrEqual, LessOrEqual, Celu to run without a
Please note that this version of onnxruntime depends on Visual C++ 2019 runtime. Previous versions depended on Visual C++ 2017. Please also refer https://github.com/microsoft/onnxruntime/tree/rel-1.3.0#system-requirements for the full set of system requirements.
We'd like to recognize our community members across various teams at Microsoft and other companies for all their valuable contributions. Our community contributors in this release include: Adam Pocock, pranavm-nvidia, Andrew Kane, Takeshi Watanabe, Jianhao Zhang, Colin Jermain, Andrews548, Jan Scholz, Pranav Prakash, suryasidd, and S. Manohar Karlapalem.
The ONNX Runtime Training code was originally developed internally at Microsoft, before being ported to Github. We’d like to recognize the original contributors: Aishwarya Bhandare, Ashwin Kumar, Cheng Tang, Du Li, Edward Chen, Ethan Tao, Fanny Nina Paravecino, Ganesan Ramalingam, Harshitha Parnandi Venkata, Jesse Benson, Jorgen Thelin, Ke Deng, Liqun Fu, Li-Wen Chang, Peng Wang, Sergii Dymchenko, Sherlock Huang, Stuart Schaefer, Tao Qin, Thiago Crepaldi, Tianju Xu, Weichun Wang, Wei Zuo, Wei-Sheng Chin, Weixing Zhang, Xiaowan Dong, Xueyun Zhu, Zeeshan Siddiqui, and Zixuan Jiang.
[Preview] Availability of Windows Machine Learning (WinML) APIs in Windows builds of ONNX Runtime, with DirectML for GPU acceleration
We'd like to recognize our community members across various teams at Microsoft and other companies for all their valuable contributions. Our community contributors in this release include: Eric Cousineau (Toyota Research Institute), Adam Pocock (Oracle), tinchi, Changyoung Koh, Andrews548, Jianhao Zhang, nicklas-mohr-jas, James Yuzawa, William Tambellini, Maher Jendoubi, Mina Asham, Saquib Nadeem Hashmi, Sanster, and Takeshi Watanabe.
This is a minor patch release on 1.1.1.
This is a minor patch release on 1.1.1.
This fixes the a minor issue that some logging in execution_frame.cc cannot be controlled by SessionLogVerbosityLevel in SessionOptions. PR #3043
This is a minor patch release on 1.1.0.
This is a minor patch release on 1.1.0.
Performance improvements to accelerate BERT model inference latency on both GPU and CPU. Updates include:
We'd like to thank our community members across various teams at Microsoft and other companies for all the valuable contributions.
We'd like to extend special recognition to these individuals for their contributions in this release: Jianhao Zhang (JD AI), Adam Pocock (Oracle), nihui (Tencent), and Nick Groszewski. From the Intel teams, we'd like to thank Patrick Foley, Akhila Vidiyala, Ilya Lavrenov, Manohar Karlapalem, Surya Siddharth Pemmaraju, Sreekanth Yalachigere, Michal Karzynski, Thomas V Trimeloni, Tomasz Dolbniak, Amy Zhuang, Scott Cyphers, Alexander Slepko and other team members on their valuable work to support the Intel Execution Providers for ONNX Runtime.
ONNX 1.6 compatibility - operator support for all opset11 ops on CPU, including Sequence ops.
MKL-DNN provider (subgraph based execution) for improved performance
ONNX Runtime WebGPU Plugin EP 0.5.0 expands attention, mixture-of-experts, and quantized operator support, improves GPU kernel performance, and strengthens runtime integration and correctness.
Concat, Slice, Transpose, and pooling, and vectorized NHWC depthwise convolution across channels (#32497, #32498).GroupQueryAttention (#32462).GatedDeltaNet execution (#32645, #33143).SparsePagedAttention softmax, and cached sparse-indexer QSA visibility and scores (#32613, #32656, #33017).GatedDeltaNet and VarlenCausalConvWithState kernels, including subsequent state-update-capacity support for GatedDeltaNet (#32510, #32477, #32767).SparseAttentionIndexer, DynamicSparseAttention, and SparsePagedAttention, plus a packed sparse attention indexer for continuous batching (#32528, #32529, #32533, #32618).PagedAttention with full-context and local non-causal attention, and per-head Q/K normalization (#32763, #32792, #32808).MoE and expanded QMoE activation, gating, routing, and packed-input support (#32447).QMoE, INT2 expert weights, and mixed-width integer QMoE execution (#32794, #32878).VarlenNGramHashMapping for packed and variable-length batching (#32285, #32358, #32467).GatherBlockQuantized to FP8 and FP4 data. Out-of-range indices now produce zero output slices for both floating-point and existing integer formats (#32480).GroupNorm, SkipGroupNorm, and 3D ConvTranspose support (#32463, #32688).BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix (#32687).GatedRMSNorm and QuickGelu fusion in the im2col-matmul convolution path (#32512, #32560).adapterIndex to select a physical Dawn adapter independently of the ORT WebGPU context identifier, deviceId (#32319).enableMatmulFp32Accumulation provider option for MatMulNBits and its fused variants; the default remains false (#29599).MatMulNBits and added bounds checks for writes based on seqlens_k (#32582, #32623).LayerNormalization shaders and retained FP32 arithmetic in LayerNormalization (#32629, #32896).DequantizeLinear scalar/vector shader-cache collisions and 1D block_size handling (#33122, #32734).OrtMemoryInfo ABI mismatches across the plugin boundary and bundled Dawn platform cleanup (#33571, #33526).The minimum ORT version is 1.24.4. Concurrent independent Sessions are enabled on ORT 1.28.3 and later in the 1.28 series, 1.30.1 and later in the 1.30 series, and 1.31 or later. Other supported versions, including 1.29.x, use serialized compatibility. In serialized mode, callers must serialize all WebGPU operations on the same device, including Session creation/destruction, allocation, I/O binding, transfers, and Run; the plugin does not enforce this serialization. Concurrent mode does not permit overlapping I/O binding and Run on the same Session (#33082).
Known compatibility caveat: ORT 1.26.0 was reported to reject the plugin during kernel creation because of its host-side plugin-version check. Forcing serialized execution does not resolve that restriction (#33082).
Thanks to everyone who contributed to this release:
@aciddelgado, @adrastogi, @AkZGhost, @apsonawane, @edgchen1, @feich-ms, @gramalingam, @gyagp, @jchen10, @jiafatom, @justinchuby, @kunal-vaishnavi, @kylo5aby, @linzuojian, @mei1127, @mustjab, @prathikr, @qjia7, @sayanshaw24, @SharmaRithik, @skottmckay, @tianleiwu, @xadupre, @xhcao, @xiaofeihan1.
This release covers changes affecting ONNX Runtime WebGPU Plugin EP code and packaging since 0.4.0.
These release notes were prepared with AI assistance from commit history and pull request metadata.
New execution providers for improved performance on specialized hardware
This release brings kernel performance improvements, expanded operator support, and reliability fixes to the ONNX Runtime WebGPU Plugin EP.
MatMulNBits wide-tile execution using subgroup shuffle (#31703)Split when every output segment is vec4-aligned (#32251)EngramGate, NGramHashMapping) (#32106, #32268)MatMulNaiveProgram's pipeline cache key cover everything it bakes into WGSL (#32048)copy_tensors misuse instead of terminating the process, and rejected foreign GPU handles in the built-in data transfer (#32315, #32317)onnxruntime_providers compilation from its archive (#32162)wgsl_template Python tests in CI (#32214)Thanks to everyone who contributed to this release:
@4n4ny4, @daijh, @edgchen1, @hariharans29, @jchen10, @kunal-vaishnavi, @qjia7, @rvandermeulen, @sushraja-msft, @tianleiwu, @xhcao, @xiaofeihan1
This release covers commits affecting WebGPU Plugin EP code and packaging.
This summary was drafted with AI assistance from commit history and PR metadata, and reviewed before publishing.
Custom op registration via C API
ONNX 1.4 compatibility
New build packages
Custom op registration via C API
Non-Tensor type support for input/output for C and C# API
ONNX Runtime WebGPU Plugin EP 0.3.0 expands model and data-type coverage, improves generative-model performance, and strengthens configuration, reliability, and release tooling.
These release notes were drafted with AI assistance.
int64 for Add, Cast, Clip, Concat, Equal, Gather, Min, Max, ReduceSum, Reshape, Sub, Tile, and Where; uint8 for Cast, Expand, Gather, and Reshape; and int32/uint32 for CumSum and Tile. (#28804, #29392, #29830, #29834, #29839, #29844, #29847, #29854, #29861, #29897, #31049, #31702, #31709, #31714)GatherBlockQuantized support and integrated ONNX 1.22 with opset 27. (#29054, #28754)GatherBlockQuantized, and fixed WebGPU data-transfer callbacks on Windows x86. (#28704, #29030, #29255, #29595, #31568)Thank you to everyone who contributed to this release:
@AngelGalindo7, @daijh, @danielsongmicrosoft, @edgchen1, @fanchenkong1, @feich-ms, @guschmue, @haoxli, @hariharans29, @Honry, @huningxin, @jchen10, @Jiawei-Shao, @miaobin, @mingmingtasd, @mirounga, @mustjab, @nicholascelestin, @prathikr, @qjia7, @Reranko05, @Shivani767, @skottmckay, @ssam18, @sushraja-msft, @tairenpiao, @tianleiwu, @titaiwangms, @wuisabel-gif, @xhcao, and @xiaofeihan1.
Scope: commits affecting ONNX Runtime WebGPU Plugin EP code, tests, build integration, and packaging since plugin-ep-webgpu/v0.2.1.
This release brings sparse attention and packed continuous-batching support, INT2 and mixed-width QMoE execution,
decode-kernel tuning, and reliability fixes to the ONNX Runtime CUDA Plugin EP. Highlights include new QSA/CSA
attention operators, expanded paged speculative decoding, opt-in MatMul auto-tuning, and improved MXFP4 support
on Blackwell consumer GPUs.
The minimum ONNX Runtime core version remains 1.24.4, but meeting that minimum alone does not guarantee
compatibility for contributed operators. Core and the plugin must use the same contributed-operator schemas;
building both from the same ONNX Runtime revision is recommended. Schema mismatches can cause incorrect execution
or crashes and may not be detected when registering the plugin (#32855).
The matched ONNX Runtime version is 1.31.0 for this release.
GroupQueryAttention workspace recipes for KV-cache preparation and XQA, FlashAttention, memory-efficient, and unfused routes, plus conservative workspace estimation for resource-constrained graph partitioning. cuDNN workspace is reported as unavailable rather than incorrectly estimated as zero (#32446, #32453, #32454, #32617).fpA_intB split-K semaphores, paged XQA semaphores, and compact convolution state updates, reducing CUDA Graph dependency overhead (#32884).Gemm and DecoderAttention with direct broadcasts on the consuming CUDA stream, removed the constant-ones cache, and corrected zero-K Gemm bias scaling (#32852).PagedAttention for eligible FP16/BF16 causal workloads with unquantized KV caches. It is selected ahead of FlashAttention when XQA is not the winner, automatically considered on SM90+, and can be disabled with ORT_ENABLE_CUDNN_FLASH_ATTENTION=0. Follow-up fixes stabilize default-scale cache keys and backend selection during CUDA Graph execution (#32493, #32624).PagedAttention beyond FlashAttention, including portable paths for small cache pages and supported XQA speculative paths; added coverage for head size 128 with 16/32/64-token pages and ragged batches (#32619).Attention when the outputs alias the external KV-cache inputs (#31150).SparsePagedAttention and DynamicSparseAttention for externally selected attention indices over paged and contiguous KV caches. These support Qwen4-Exp QSA selected-main-cache attention and DeepSeek V4 CSA joint local/selected-auxiliary attention, with one softmax across participating entries and optional attention sinks (#32524, #32525).SparseAttentionIndexer for QSA token selection and CSA compressed-block selection, including rotary preparation, scoring, deterministic TopK, and cache updates (#32526).PackedSparseAttentionIndexer for packed, variable-length continuous batching, with per-request metadata, fixed-capacity state, and selection outputs compatible with SparsePagedAttention (#32618).DynamicSparseAttention across selected-token tiles, bounded split workspace, replaced quadratic duplicate validation with a GPU bitmap, and added single-split fast paths for large-context inference (#32671).EngramGate and NGramHashMapping for Qwen4-Exp, including normalized gated-value output, autoregressive history, per-head offsets, and EOS/segment resets (#32285).VarlenNGramHashMapping for packed DeepSeek Engram batching, preserving request-local history and preventing n-gram mixing across request boundaries, then extended it with Qwen4-Exp hashing semantics (#32358, #32467).activation attribute to GatedRMSNorm for SiLU, its Swish alias, and Sigmoid gating. SiLU remains the default (#32512).BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix (#32687).GatherBlockQuantized to FP8 and FP4 data, including full-axis blocks and scale broadcasting. Behavior change: out-of-range indices now produce zero output slices for both integer and floating-point quantized formats, rather than following ONNX Gather error semantics (#32480).(2,2), (2,4), and (4,2), with packed decode paths and a scratch-bounded dequantization fallback for supported configurations (#32697, #32743, #32761).FLOAT8E8M0 type mapping, and accelerated FP4 conversion in decode and grouped GEMM (#33067).MatMul auto-tuning via ep.cuda.enable_gemm_auto_tune=1 or ORT_CUDA_GEMM_AUTO_TUNE=1, choosing among cuBLAS, small-N GEMV, and eligible tinygemm2 kernels on SM90+. cuBLAS remains the default. Graph-enabled sessions validate candidates with CUDA Graph replay before caching the winner; warm-up inference is required before capture (#32876, #32901, #33085).MatMulNBits small-M GEMV tiling, biased near-tie tactic selection toward GEMV for small decode batches, and enabled the W8 M=6-8 fast path for the qualified SM121 FP16 shape K=3584, N=200064, block size 64 (#32876, #32884, #32742).MatMulBlockQuantizedFp8Weight workloads with a_scale and block size 128, avoiding the FP16/BF16 weight-dequantization buffer. Enable with ORT_FP8_MATMUL_DEEPGEMM=1 in a supported non-Windows DeepGEMM build; it is disabled by default and native FP8 rounding requires model-accuracy validation (#32599).MatMulNBits GEMV experiments: ep.cuda.fpa_intb_gemv_wave_aware=1 for eligible SM12x FP16/INT4 M=8 tiles, and ep.cuda.fpa_intb_gemv_paired_k=1 (or ORT_FPA_INTB_GEMV_PAIRED_K=1) for eligible FP16/INT4 block-32 M=5-8 profiling buckets. Both are disabled by default. The paired-K tactic is experimental: FP16 partial sums can overflow even when the default GEMV remains finite, so numerical validation is required; tactic profiling checks speed, not numerical safety (#33186, #33188).Split fast path for 2-4 outputs, passing metadata by value and eliminating per-invocation pinned-memory allocations and metadata uploads (#32706).MatMulNBits optional-input detection, including omitted edges and empty placeholders, and restored zero-point/scale type comparisons (#32970).MatMulInteger, Conv/FusedConv/ConvTranspose, Squeeze, and GemmFloat8, and corrected the GemmFloat8 C-operand descriptor type (#32971, #31967).GatherND index bounds checks (#31641, #31646).PackedMultiHeadAttention and RestorePadding token offsets before invalid values can participate in device address calculations (#33098, #33099).Split dimension products and rejected generation-buffer expansion when the input sequence exceeds the maximum sequence length (#32551, #32553).Resize handling for out-of-range coordinates and bounded coefficient windows (#33056, #33095).GatedDeltaNet shape inference for rank-2/rank-3 packed QKV with omitted key/value inputs, while preserving separate-Q/K/V behavior (#32708).onnxruntime_USE_FLASH_ATTENTION=OFF by excluding FlashAttention CUDA sources, and fixed builds with memory-efficient attention disabled (#32580, #32588).Thanks to the 17 human contributors who contributed to this release:
@apsonawane,
@baijumeswani,
@bheu,
@chilo-ms,
@edgchen1,
@eserscor,
@gramalingam,
@hanbitmyths,
@hariharans29,
@jiafatom,
@justinchuby,
@kunal-vaishnavi,
@SIDDARTHAREDDY8,
@skottmckay,
@tianleiwu,
@titaiwangms,
@xadupre
This release covers changes from origin/plugin-ep-cuda/rel-0.2 to origin/plugin-ep-cuda/rel-0.3 affecting CUDA Plugin
EP code, shared CUDA kernels, build integration, and packaging. The highlights focus on user-facing changes rather
than listing every infrastructure-only update.
This summary was drafted with AI assistance from commit history and PR metadata.
ONNX Runtime C# packages are now available for Linux, with GPU support for both Windows and Linux. Find the APIs and package downloads here.
Major performance work for attention-heavy LLMs.
max_k_step was enabled for NVIDIA (#28511).Qwen3 and Gemma 4 model-path improvements.
LinearAttention and quantized-path optimizations.
Reliability and hardening fixes.
past_state == present_state buffer handling (#28753).Graph-capture and buffer-management improvements.
Note: This section was AI-generated. It may have inaccuracies.
Thanks to everyone who contributed to the WebGPU EP (human contributors, alphabetical):
@apsonawane, @daijh, @edgchen1, @feich-ms, @GopalakrishnanN, @guschmue, @hariharans29, @HectorSVC, @jchen10, @qjia7, @tianleiwu, @xiaofeihan1, @xenova, @yuslepukhin.
Note: This list was compiled on a best-effort basis from PRs that touched WebGPU EP-specific paths and
intentionally includes human contributors only, so it may not capture every contribution. If yours was
missed, the omission is unintentional. Your work is no less appreciated.
This is the first open source release of ONNX Runtime.
This is the first open source release of ONNX Runtime.
Nothing published for this version
Nothing published for this version
One column per quarter.