NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
NuGet
NuGet, #2082 most downloaded on NuGet
This package contains ONNX Runtime for .Net platforms
Latest stable
1.32.0
Last release 9 days ago
13 Nov 2026
Ships fairly regularly
a new release about every 3 weeks
Nearly every release is documented
notes for 55 of 58 stable releases
5 versions withdrawn
withdrawn after publishing
127 years old
65 releases · first in 1900
Drag across the ruler to compare, or tap a release to open it.
Release Def: Branch: refs/heads/rel-1.32.0 Commit: fab30a46491a40f5e4c968b80e8a59acbbb91b53 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.32.0 Commit: fab30a46491a40f5e4c968b80e8a59acbbb91b53 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1544378
ONNX Runtime 1.32.0 reintegrates ONNX 1.23.2 with opset 28 support, introduces CPU/GPU hybrid MoE offloading and shared-layout CPU QMoE kernels, expands device-memory, weightless-compilation, and Python model-package APIs, dramatically accelerates session initialization on large graphs, and strengthens runtime and optimizer reliability. These notes cover changes since ONNX Runtime 1.31.1. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases.
SpaceToDepth, Mod, signed BitShift, and BFloat16 extensions) and updated DLPack to v1.3 across all implemented ORT numeric and boolean tensor types (#33134, #32362, #33848).QMoE (NVFP4, INT4, INT3, and INT2) and block-packed MoEPacked models, backed by shared-layout AVX-512/AVX2 CPU QMoE kernels and memory-mapped expert storage (#33450, #33451, #33452, #33973).OrtApi::EpDevice_GetMemoryCapacity, explicit multi-mode weightless compilation (ModelCompilationOptions_SetWeightlessMode and ep.enable_weightless_mode), Python bindings for the Model Package APIs, and zero-copy CPU buffer borrowing in AddExternalInitializers (#33447, #32899, #33640, #33877).GatedDeltaNet per-key-dimension decay), right-padded chunked prefill in GroupQueryAttention, native GGUF/block-packed operators (GatherPacked, MatMulPacked, MoEPacked), and sparse-attention index sharing (#33817, #33503, #33953, #33224).ACLExecutionProvider) execution provider, its build options, and its standalone C registration API have been removed. Python's get_all_providers() no longer lists ACLExecutionProvider; applications should remove it from explicit provider lists (#33851).NvTensorRTRTXExecutionProvider removed: In-tree provider-bridge NvTensorRTRTXExecutionProvider sources have been removed in favor of the standalone NVIDIA TensorRT RTX plugin EP registered via OrtApi::RegisterExecutionProviderLibrary. NvTensorRtRtxProviderOptions is retained as a deprecated declaration for source compatibility, and legacy AppendExecutionProvider calls now report a deprecation error directing callers to the plugin EP (#32276).ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT6E2M3 and ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT6E3M2 public element-type mappings and Cast conversions (#33134).kDLBool metadata and bidirectional support for all implemented ORT numeric and boolean types, including all five float8 formats, packed float4, int4/uint4, and int2/uint2 (float6 remains unsupported in DLPack). The low-level is_bool_tensor override was removed, and uint8 DLPack metadata now always imports as uint8 (#33848).OrtCompileApi::ModelCompilationOptions_SetWeightlessEnabled and the ep.enable_weightless / ep.enable_weightless_ep_context_nodes session options are deprecated in favor of ModelCompilationOptions_SetWeightlessMode and ep.enable_weightless_mode ("0" = NONE, "1" = EXTERNAL_ONLY, "2" = ALL). EPs can now advertise multi-mode capability via OrtWeightlessSupport_ALL_OR_EXTERNAL_ONLY and weightless_supported_modes (#32899).OrtApi::EpDevice_GetMemoryCapacity (C++ EpDevice::GetMemoryCapacity()) and OrtEpFactory::GetDeviceMemoryCapacity to query device-global total memory and estimated currently available memory (DEFAULT or HOST_ACCESSIBLE), with built-in and plugin CUDA EP support (#33447).weightless_supported_modes), applications request an explicit mode (ModelCompilationOptions_SetWeightlessMode / ep.enable_weightless_mode), and compiled EPContext models record weightless_mode and external_dependencies (source_model, external_weights, ep_context_binary) in model metadata (#32899).session.use_external_initializer_file_buffers_directly="1" to AddExternalInitializers for zero-copy borrowing of naturally aligned CPU buffers larger than 127 bytes on little-endian hosts, and documented the read-only guarantee for external initializers added via OrtModelEditorApi::AddInitializerToGraph with data_is_external=true (#33877, #33717).session.external_data_loader.*) to accelerate external initializer loading into GPU memory across built-in and plugin CUDA EPs (#33839).session.moe.offload.*), generic session.kernel_dispatch_policy (auto, latency, memory, safe) configuration plumbing, Level-1 workspace/prepack memory accounting fallback for nodes absent from memory-profile CSVs, and internal capability-planning protocol types (#33451, #33042, #33311, #33841).<profile_file>_compile) in onnxruntime_perf_test when -p is used with --compile_ep_context (#33053, #33694).onnxruntime_DISABLE_EXCEPTIONS (--disable_exceptions) usable without onnxruntime_MINIMAL_BUILD and added the onnxruntime_DISABLE_STREAM=ON CMake option to disable device-stream support in non-minimal builds (#33120, #33482).These updates affect core ONNX and contributed operator schemas and cross-provider behavior, not only the separately released GPU plugins.
SpaceToDepth (DCR and CRD) on CPU and CUDA with version-split registrations across DML, JS, WebGPU, and QNN; opset-28 floating floor-modulus Mod on CPU, CUDA, and DML; opset-28 BFloat16 support for Compress, OneHot, ReverseSequence, Unique, Celu, and Einsum on CPU (plus Compress, OneHot, and ReverseSequence on CUDA); and BFloat16 Pad (opsets 21–25) on CPU and CUDA (#32362, #32369, #32378, #34062).GatedDeltaNet with Kimi Delta Attention (KDA) per-key-dimension decay across CPU, CUDA, and WebGPU, and extended GatedDeltaNet and CausalConvWithState state/gate handling for multi-token speculative decoding (#33817, #33471).query_seqlens input to GroupQueryAttention to skip right-padded prefill tokens and idle batch entries (query_seqlens[b] = 0) on CPU and CUDA, and added q_norm_epsilon / k_norm_epsilon attributes to preserve distinct Q and K normalization epsilons during GQA pre-norm fusion (#33503, #33505, #33832, #33363).QMoE v1 packed-code contract, added opt-in post-SiLU clipping (swiglu_limit_after_activation) and tanh-GEGLU (activation_type="geglu") support to MoE/QMoE schemas, and added GatherPacked (CPU/CUDA), MatMulPacked (CUDA, including native Bolt global_scale), and MoEPacked (CUDA) to consume native GGUF/block-packed quantized weights directly (#33394, #33954, #33633, #33953, #33962).PackedSparseAttentionIndexerMerge for multi-token prediction index sharing and optional state_status / mid-state outputs on PackedSparseAttentionIndexer across CUDA and WebGPU; extended GatedAdd with an optional projected gate_weight across CPU, CUDA, and WebGPU; and supported empty unpacked weights in CPU and WebGPU MatMulNBits (#33224, #33383, #33472, #33470, #33559).SwiGLU, RMSNormRotary, HyperConnectionLowRankMix, RouterGateUpProj, SpeculativePrefixCommit, GatedDeltaNetStateReplay, ConvStateReplay, and MatMulNBitsQLoRA (#32126, #33469, #33384, #33650, #33474, #33213, #33493, #33370).OrtMemoryInfo ABI mismatch across the ORT/plugin EP boundary by converting OrtMemoryInfo instances exclusively through the C API (#33571).OrtEpFactory structs in provider-bridge EP factories so unimplemented optional callbacks are null, and prevented double DLL initialization and handle leaks when probing and registering plugin EP libraries (#33117, #33701).shared_assets path confinement and symlink validation (#32790, #33826).CUDA updates include CPU/GPU hybrid expert offloading (session.moe.offload.*) for QMoE (NVFP4, INT4, INT3, INT2, and mixed-width) and MoEPacked; offline K-packed NVFP4 and Tensor-Core INT4/INT3/INT2 QMoE kernels; native GGUF/block-packed GatherPacked, MatMulPacked, and MoEPacked execution; long-context (up to 256K) and speculative-decoding acceleration across DynamicSparseAttention, PackedSparseAttentionIndexer/Merge, GatedDeltaNet, HyperConnectionLowRankMix, SwiGLU, RMSNormRotary, RouterGateUpProj, SpeculativePrefixCommit, H512 XQA decode, and H256 BF16/INT8 paged speculative XQA; an opt-in persistent fpA_intB MatMulNBits tactic cache and deterministic tactic selection; route-complete Level-1 workspace estimation; cuDNN 9 frontend Conv plan caching; opt-in EP-owned runtime fusions; standard ONNX integer bitwise/BitShift and BFloat16 ArgMax/ArgMin/Pad kernels; and numerical and synchronization fixes (#33452, #33973, #33094, #33689, #33802, #29588, #33078, #33123, #33757, #34021).
Detailed CUDA features, configuration options, limitations, and fixes will be covered in the separate CUDA Plugin EP v0.4.0 release notes. Most CUDA-specific PRs are covered there rather than repeated here. Use the matched core/plugin combination and observe the contrib-op schema compatibility requirement above.
GatedDeltaNet KDA, GatedDeltaNetStateReplay and ConvStateReplay for speculative decoding, PackedSparseAttentionIndexerMerge, fused MatMulNBitsQLoRA, dense and routed block-scaled FP8 MatMul/QMoE, 16x16x16 and NVIDIA shared-memory subgroup-matrix MatMul, AMD wave64 and wide-tile Intel subgroup-matrix MatMulNBits, Qualcomm wide-subgroup DP4A, subgroup-matrix Conv, Intel dense-prefill FlashAttention, and M=1–8 FP16 GEMV (#33213, #33493, #33370, #33074, #33075, #32937, #33639, #32796, #32252, #32702).OrtDawnCacheInterface), enableFeatures/disableFeatures EP feature controls, MatMulExecutionPlanner, batched transfers and graph-capture copy/clear replay, synchronized per-session command recording, bundled D3D12 Agility SDK Python wheel DLLs, and expanded QuantizeLinear/INT4/INT8/FP16 RNN/WebNN operator coverage (#33055, #33936, #33481, #32630, #33491, #33362, #33082, #32795, #33683, #33792).Detailed WebGPU kernel additions, optimizations, and correctness fixes will be covered in the separate WebGPU Plugin EP v0.6.0 release notes.
Reshape and Unsqueeze schema registrations for opset 25+ models and flushed pending JSEP command encoders before CPU-to-GPU uploads onto reused buffers (#33749, #33361).QMoE execution over shared-layout INT4, INT3, INT2, and FP4 expert banks with AVX-512 and AVX2+FMA+F16C kernels, register-spill elimination, byte-preserving memory-mapped FileExpertStore reads, and opt-in ep.cpu.qmoe_deferred_scale (#33450, #33624, #33785, #33935).MatMulNBits execution for block sizes above 256 (including 16-bit activation QDQ conversion), supported 1-D kernels in float channels-last NhwcFusedConv, improved MLAS NCHWc convolution thread load balancing, and parallelized CPU MultiHeadAttention QKV transposes (#32885, #33690, #32335, #33887).precision attribute in CPU QuantizeLinear, hardened AVX2 M=1 SQNBit kernels, and moved GetCpuPreferredNodes element-type checks to a compile-time switch (#32735, #29886, #29647).onnxruntime::rewrite) and region-scoped EP capability rewriting infrastructure (#33631, #33652).Slice + Concat to SpaceToDepth fusion for YOLOX focus blocks (#32335).PadFusion changing MaxPool results on negative inputs, MatmulBNFusion when intermediate Reshape/Transpose moves the channel axis, GemmTransposeFusion folding public graph outputs, repeated NHWC ConvActivationFusion, MatMulAddFusion on zero-sized dimensions, and lossy DOUBLE-to-float constant conversions in LayerNorm and MatMulScale fusions; reverted an unnecessary constant-folding determinism change (#33206, #33392, #33484, #33666, #33766, #32983, #32471).QLinear* and QDQ-drop rewrites for float8/float4 quantized types, validating quantization parameter shapes and types before QDQ Add/Mul/Concat/MatMul, MatMulIntegerToFloat, and QLinearSoftmax fusions, and handling zero-channel Conv and empty-batch Add in NCHWc transforms (#33663, #33397, #33616, #33485, #33568, #33346, #33518).float16 quantization scales for small-range weights, fixed dynamic Gemm(transB=1) quantization with shared or shape-annotated weights, reused Cast nodes for shared always-float inputs in float16.py, and fixed Windows file-locking and relative external-weight resolution during float16 model-path conversion (#33239, #33415, #33090, #33193).SoftmaxCrossEntropyLoss graph optimizers, and updated training CUDA stream and runtime cuDNN loader call sites (#32982, #32987, #32981, #33744).100) for model-local functions, validated inline sparse payloads and index types before dense allocation, and checked equivalence of serialized and cross-session shared prepacked weights (#32200, #32339, #32863, #33499).Status instead of throwing under ORT_NO_EXCEPTIONS, and included the originating C API function name in catch-all error messages (#33700, #33828, #29561).GroupQueryAttention and MulInteger type/shape inference for growing KV caches, shape-derived int32 sequence lengths, and omitted zero points (#32717, #33884, #33623).{1} single-element tensor buffer reuse across rank representations, avoided module-load dynamic initialization for OrtEnv's mutex, and resolved Windows 1DS telemetry shutdown deadlocks and late-worker SDK lifetime races (#33881, #29863, #33419, #33291, #33321, #33993).Attention (projection/cache dimensions and positive head counts), PagedAttention, PackedAttention, LinearAttention, MoE hidden dimensions, MatMulNBits FP16 prepacking, Resize linear index bounds, Concat axis overflow, Whisper timestamp token bounds, TreeEnsemble category set IDs, and runtime tensor ranks in CPU Conv/ConvInteger/QLinearConv/CropAndResize/Einsum/AffineGrid (#32946, #33498, #31651, #33330, #33821, #33820, #33375, #33497, #33235, #33824, #33825, #33517).GroupQueryAttention reading device-produced total_sequence_length tensors, rank-reduced broadcast scales in CUDA LayerNormalization/RMSNormalization, Split with num_outputs and an empty optional split input, zero-divisor validation in CUDA integer Mod, and Arm64 FP16 grouped Conv packed-weight offsets and per-group bias (#33945, #28971, #32972, #32546, #33598).MoE input corruption on aliased buffers, ordered NaN as the largest value in CPU TopK, fixed fractional floating-point indices in CPU OneHot, prevented QLinearSoftmax requantization overflow and empty-input failures, and fixed integer Range boundary counts, zero-K Gemm signed-zero/bias scaling, empty negative-axis reductions, and empty batch-transposed or zero-rank MatMul operands (#33885, #33543, #33273, #33277, #33401, #33399, #33669, #33232, #33512, #33171).ModelPackage, ModelPackageContext, ModelPackageOptions, ModelPackageComponentContext), along with ModelCompiler.set_ep_context_binary_information (#33640).sm_87 (Jetson Orin), sm_100, sm_103 (GB300), and sm_110 (Jetson Thor) to Linux aarch64 CUDA 13 packages; switched the Linux aarch64 wheel to Blackwell family targets (100f, 120f, saving ~82 MB); reused sm_80 FlashAttention cubins on sm_86/sm_89 by default (saving ~30 MB in x64 GPU packages); added native sm_120 SASS to the CUDA plugin LLM target under MSVC; and added an experimental unified multi-ABI CUDA 13 Python wheel pipeline (#33369, #33315, #33649, #33301, #34089, #34088, #33173, #33034, #33771).ORT_BUILD_INFO provenance from shared onnxruntime_config.h, and fixed CMAKE_CUDA_ARCHITECTURES=all cache handling, Windows --disable_contrib_ops CUDA builds, system cuDNN include-directory filtering, Windows GPU package validation, and Clang/no-exceptions portability (#28897, #33687, #28019, #33702, #33736, #33889, #33408, #33532, #33583, #33830).cpp_client_telemetry to 3.10.309.1 and the vcpkg baseline, libprotobuf-mutator, Python tooling pins (transformers 5.10.1, onnx >= 1.21.0 / 1.22.0, Pillow >= 13.0.0, setuptools >= 84.2.1), and JavaScript/Web dependencies including security updates for browserslist, shell-quote, nanoid, undici, fast-uri, next, electron, and follow-redirects; removed the stale Intel MKL third-party notice (#33390, #33949, #33210, #33501, #33591, #33734, #33656, #33903, #33587, #33588, #33589, #33608, #33610, #33233).Thanks to our 94 human contributors for this release!
@aciddelgado, @adrastogi, @akasakariko, @AlekseiNikiforovIBM, @alexk201, @apsonawane, @arnej27959, @aswanth-07, @baijumeswani, @BBaditi, @bmehta001, @cboseak, @cbourjau, @chilo-ms, @ConnorBaker, @cursoragent, @daijh, @danielsongmicrosoft, @DavidLuong98, @DevChuriwala, @edgchen1, @El1as42, @eserscor, @galleonli, @GopalakrishnanN, @gramalingam, @gyagp, @hanbitmyths, @haoxiz-nvidia, @HuzaifaChaudary, @iBreaker, @javier-intel, @Jayashree-mcw, @Jayashree-VR, @jchen10, @jiafatom, @Jiawei-Shao, @jmcarcell, @justinchuby, @KafKafrnZ, @keshavv27, @kunal-vaishnavi, @kylo5aby, @leninworld, @LinCanNerd, @lukas-zaborsky, @Manogna-Sree, @maxim-davgalev, @MayureshV1, @mei1127, @mirounga, @mohdmahmodi, @Mohit-25-tech, @MokiMeow, @mustjab, @n1harika, @nazanin-beheshti, @nietras, @ninglinLiu, @patryk-kaiser-ARM, @prathikr, @preetha-intel, @professor-moody, @qjia7, @rookieLiu2018, @rui-ren, @rvandermeulen, @samridh10exe, @sankalpsthakur, @sayanshaw24, @selenayang888, @SharmaRithik, @silviot, @sisleyli, @sushraja-msft, @swetha097, @TANGBUDU, @the0cp, @tianleiwu, @timothyanderson096-ocdealcheck, @titaiwangms, @Trisha-Sarvam, @TrishaWadhwani, @usmanmateen, @vinay-852, @xadupre, @xenova, @xgbtxy, @xhcao, @xiaofeihan1, @xiaoyu-work, @yairbenavraham, @ykhrustalev, @Yuhx141
Full Changelog: rel-1.31.1...rel-1.32.0
Release highlights were prepared with AI assistance.
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Release Def: Branch: refs/heads/dev/erscor/2026/10/23-final-cherry-pick Commit: 41d4d61f14b9c1dd0b2ccda1e9b85dd415fd0496 Build: https://aiinfra.visual
Release Def: Branch: refs/heads/dev/erscor/2026/10/23-final-cherry-pick Commit: 41d4d61f14b9c1dd0b2ccda1e9b85dd415fd0496 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1501575
ONNX Runtime 1.31.1 is a patch release that fixes an OpenVINO-enabled Python build regression and corrects WASM Debug external-data tests. These notes cover changes since ONNX Runtime 1.31.0.
Thanks to our 3 human contributors for this release!
@eserscor, @mustjab, @tianleiwu
Full Changelog: rel-1.31.0...rel-1.31.1
Release highlights were prepared with AI assistance.
Release Def: Branch: refs/heads/rel-1.31.0 Commit: 402979ac1bef89dcd55add41eaa7277c407599eb Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.31.0 Commit: 402979ac1bef89dcd55add41eaa7277c407599eb Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1468078
ONNX Runtime 1.31.0 stabilizes model-package and EPContext data APIs, improves CPU model loading and quantized MoE inference, expands device-based execution-provider selection, and strengthens model-loading and runtime reliability. These notes cover changes since ONNX Runtime 1.30.0. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases.
PREFER_NPU (#31975).GatherBlockQuantized behavior change: out-of-range indices now produce zero output slices across CPU, CUDA, and WebGPU for both integer and floating-point quantized formats. This deliberately differs from ONNX Gather error semantics (#32480).OrtApi::GetModelPackageApi and the stable OrtModelPackageApi table. EPContext callback registration now uses stable APIs; the corresponding experimental lookup entries were removed. The stable native callback contract carries callback/state pointers, with payload limits enforced by application callbacks, bindings, or EPs rather than a native read-options policy (#33166, #32265).--use_acl now receive a deprecation warning (#32564).--use_windows_telemetry or onnxruntime_USE_WINDOWS_TELEMETRY=ON; telemetry can be disabled with --no_telemetry (#32384).session.strict_workspace_verification=1 to fail initialization on workspace declaration overruns or nonzero orphaned reservations; strict verification is not supported for ORT-format model loads (#31962, #32189).onnxruntime_DISABLE_DEVICE_DISCOVERY=ON to skip platform accelerator probing and report only the CPU device (#32592).These updates affect core operator schemas and cross-provider behavior, not only the separately released GPU plugins.
EngramGate and NGramHashMapping for Qwen4-Exp, and added VarlenNGramHashMapping for packed DeepSeek Engram batching with request-local history. Packed hashing also supports Qwen4-Exp head offsets, EOS resets, and segment boundaries (#32285, #32358, #32467).activation attribute to GatedRMSNorm for SiLU, its Swish alias, and Sigmoid gating across CPU, CUDA, and WebGPU. SiLU remains the default (#32512).BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix, with CPU, CUDA, and WebGPU implementations (#32687).GatherBlockQuantized to FP8/FP4 data, full-axis blocks, and scale broadcasting, and improved CPU UINT8 dequantization. See the out-of-range-index behavior change above (#32480, #32621).CUDA updates include sparse attention and packed continuous batching, expanded paged speculative decoding, INT2 and mixed-width QMoE execution, opt-in MatMul auto-tuning, improved MXFP4 support on Blackwell consumer GPUs, and reliability fixes.
See the CUDA Plugin EP v0.3.0 release notes for detailed features, configuration options, limitations, and fixes. Most CUDA-specific PRs are covered there rather than repeated here. Use the matched core/plugin combination and observe the contrib-op schema compatibility requirements above.
Detailed WebGPU kernel additions, optimizations, and correctness fixes will be covered in the separate WebGPU Plugin EP v0.5.0 release notes.
OrtEpFactory, device enumeration, SessionOptionsAppendExecutionProvider_V2, and EP selection policies. NPU selection requires iOS 16/macOS 13 or later; GPU selection requires iOS 15/macOS 12 or later. In builds advertising both CoreML and WebGPU for the same Apple GPU, PREFER_GPU/MAX_PERFORMANCE select CoreML through the existing EP-name tie-breaker; WebGPU remains available through explicit device selection (#31975).GroupNormalization, GroupNorm, and SkipGroupNorm support (#32606).Resize by inferring resample2d axes and fixed ConvTranspose shape-arithmetic overflow (#32628, #32661).Resize ROI axis expansion (#32745, #32747).QMoE accuracy_level attribute and the ORT_QMOE_CPU_QNBIT_GEMM override; INT4 uses FP32 activations by default, while accuracy_level=4 opts into INT8 activation computation. INT8-weight experts use INT8 activations on the new path (#32644, #32668).MatMulNBits FP32 throughput mode with mlas.qnbit.force_fp32=1 for eligible x86/x64 FP32-input, 4-bit, block-size-32, accuracy-level-4 nodes. It selects one numerical path for the whole session, is disabled by default, and trades single-request latency for batched throughput (#31723).Col2Im/Im2col, accelerated adjacent-axis-group Transpose, reduced repeated scans in 1D MaxPool, and optimized BatchNormalization for NC inputs (#32698, #32800, #32817, #32891, #32936, #33113).Where kernels and independent scale/output types in DequantizeLinear; avoided unused dynamic QGEMM packed-buffer allocations (#32846, #32728, #32879).ConvTranspose (#29542).MatMulNBits bias fusion with prepacked weights (#33079, #33091, #33115, #32973).Shape start/end, missing ConstantOfShape values during FP16 conversion, and packed GatedDeltaNet inputs. Preserved nested-graph serialization and FP16 Cast fallback consistency (#33157, #33089, #32708, #32727).Constant nodes, ORT-format input argument counts, and contrib-op shape inference; fixed out-of-bounds reads involving Loop/Scan initializers and shape-inference input data (#32940, #32865, #32607, #32609, #32681).UINT2, INT2, and FLOAT8E8M0, without renumbering public enum values. Fixed reuse of packed sub-byte buffers for full-byte tensors (#33066, #32611).Resize scale validation; used checked arithmetic for CPU LSTM and DynamicQuantizeLSTM state extents (#32348, #32555, #32552, #32557, #33060, #32654, #32722, #33128).TopK/DFT inputs, zero-channel ConvTranspose, and real-window handling for complex STFT signals (#32723, #32725, #32724, #33064, #33136).QuantizeLinear output on the last axis, and avoided degenerate CPU GQA FlashAttention tiles when L2 cache size is unknown (#31761, #32631, #33040, #32786, #33202).--enable_arm_neon_nchwc handling and added Windows ARM64 GitHub Actions coverage (#32094, #32791).Thanks to our 80 human contributors for this release!
@1duo, @abeclowicz, @aciddelgado, @adrastogi, @AkZGhost, @andife, @anubhavagr, @apsonawane, @artyomrabosh, @baijumeswani, @bbernhar, @bheu, @bingcheng1998, @bmehta001, @chilo-ms, @crvineeth97, @cursoragent, @damdoo001-arm, @danielsongmicrosoft, @edgchen1, @eserscor, @feich-ms, @galleonli, @GopalakrishnanN, @gramalingam, @gyagp, @hanbitmyths, @hariharans29, @Honry, @Hosi121, @hulkbig, @javier-intel, @jchen10, @jiafatom, @justinchuby, @kjg0724, @kuanyul-qti, @kunal-vaishnavi, @kylo5aby, @linzuojian, @maxim-davgalev, @mei1127, @melkap01-Arm, @miaobin, @mirounga, @mustjab, @NVMadhava, @prathikr, @pratikgx, @qjia7, @qti-yuduo, @raashish1601, @rohan-patnaik, @rvandermeulen, @saiyambharara, @saravanan-a-r, @sayanshaw24, @SharmaRithik, @shiyi9801, @SIDDARTHAREDDY8, @SiluPanda, @skottmckay, @sylvesterkaczmarek, @TANGBUDU, @the0cp, @tianleiwu, @titaiwangms, @velonica0, @vijay-kapse, @wangw-1991, @xadupre, @xhcao, @xiaofeihan1, @xiaohanAMD, @xiaoyu-work, @yairbenavraham, @ykhrustalev, @Yuhx141, Zapakio6, @Zestion
Full Changelog: rel-1.30.0...rel-1.31.0
Release highlights were prepared with AI assistance.
Release Def: Branch: refs/heads/rel-1.30.0 Commit: f2c39fe2f838cf35ce7da92824f5a5e3ee6e88a7 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.30.0 Commit: f2c39fe2f838cf35ce7da92824f5a5e3ee6e88a7 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1412493
ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1.
-Donnxruntime_USE_FP4_QMOE=OFF (#32096, #32163).block_size=32. Set -Donnxruntime_USE_FPA_INTB_GEMM_FULL=ON when building from source to retain the full kernel set, including BF16, zero-point, bias, larger-block-size, and native Hopper variants (#32324).Gemm and MatMul execution is gated on hardware acceleration. CPU-assigned FP16 nodes without a matching kernel now fall back to FP32 (#32301, #32197).Split, Scan, GatherND, ScatterND, SpaceToDepth/DepthToSpace, Crop, Conv, Normalizer, and pooling (#29461, #31668, #32034, #32039, #32076, #32157, #32160, #32161, #32345, #32349).BifurcationDetector inputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage (#31648, #31701, #32009, #32078, #32144).TreeEnsemble node references and bounded subtree comparison, rejected non-finite CPU RoiAlign coordinates, and required ImageScaler bias to match the channel count (#32031, #32043, #32011, #32002).MatMulFpQ4 shape inputs, and checked MLAS blockwise quantization/dequantization index ranges (#31682, #32032, #32007).MatMulNBits, RemovePadding, RotaryEmbedding, SparseAttention, Whisper beam search, NMS, QDQ, and GatherElements (#31643, #31994, #31995, #31996, #31998, #32014, #32029, #32030).CudaAsyncBuffer staging storage alive across CUDA graph replay (#31968, #32121).js-yaml, joi, fast-uri, and the Next.js end-to-end fixture (#32397, #32486, #32488, #32505, #32508).KernelContext::GetPreallocatedOutput (#29726, #32089).EngramGate and NGramHashMapping, and expanded kernel coverage for Qwen-3.5 operators (#32268, #32106).is_causal attribute to PagedAttention (#32515, #32225).VarlenCausalConvWithState for continuous batching and compact variable-length causal-convolution state updates (#32168, #32290).GatedDeltaNet operator and BFloat16 support for CUDA GatedDeltaNet (#32282, #32307).ORT_QMOE_FP4_DEEPGEMM=1, default off). This path is disabled on Windows (#32122, #32485).MatMul shapes and refined fpA-intB GEMV support checks (#31478, #32338).TopK, ArgMax, and ArgMin performance for wide last axes, and accelerated low-lane INT64 CumSum (#32404, #32092, #32238).Slice fast path for contiguous subregions and removed pinned-buffer use from Split and Concat fast paths (#28902, #32410).ReduceMean kernels and fixed ScatterElements reduction dispatch by element type and signed-zero handling in Abs (#32326, #29879, #31477).Gather support and optimized MatMulNBits wide tiles with subgroup shuffle (#31714, #31703).Split when all output segments are vec4-aligned, and selected pooling paths based on occupancy (#29820, #32251, #32313).onnxruntime_perf_test -i options to WebGPU (#32074, #31971, #32316).GPUDevice instances, corrected MatMul pipeline-cache keys and the 1D-dispatch shader fast path, and changed copy_tensors misuse to report errors instead of terminating the process (#32259, #32048, #32343, #32315).SkipLayerNormalization support and corrected output-rank validation and fallback data-type support checks (#32377, #31708, #32067, #32293).kernel_shape and output_padding lengths during DirectML kernel setup (#31999).QuantizeLinear rounding, prevented CPU TensorScatter index overflow, serialized ScatterND string updates, and widened Compress loop counters (#32452, #32012, #32033, #32008).LpNormalization inputs, zero-element BiasGelu/FastGelu, extreme Trilu diagonals, and empty reduction axes (#32020, #31698, #32013, #32156).Slice starts rank (#31670, #31678, #32018, #32044).Gemm (#32038, #32143, #32426, #32435).NodeAttrHelper string-default lifetimes (#32051, #32138, #32019).Env, and clarified how external-initializer paths interact with EP context paths (#32502, #32503, #32442).RunAsync arguments until completion (#32041, #32015).Conv3DNaive shader (#32469, #32357).Thanks to our 57 human contributors for this release!
@4n4ny4, @apsonawane, @arnej27959, @baijumeswani, @bmehta001, @chilo-ms, @crvineeth97, @daijh, @danfiedler-msft, @danielsongmicrosoft, @dannyota, @DKAIN-py, @edgchen1, @ericcraw, @eserscor, @fanchenkong1, @hanbitmyths, @hariharans29, @hdharpure9922, @Honry, @jambayk, @javier-intel, @jchen10, @jiafatom, @jnagi-intel, @justinchuby, @kadyrbekovhamit-cyber, @kunal-vaishnavi, @Lapis0x0, @LOGO127, @Manogna-Sree, @martin-klacer-arm, @mei1127, @miaobin, @mirounga, @miyanyan, @MohamedElashri, @mustjab, @Nikhi00718, @Noperi0r, @Novestars, @pkubaj, @preetha-intel, @qjia7, @rvandermeulen, @sanaa-hamel-microsoft, @skottmckay, @sushraja-msft, @swetha097, @sylvesterkaczmarek, @tianleiwu, @titaiwangms, @toothache, @xadupre, @xhcao, @xiaofeihan1, @Zestion
Full Changelog: rel-1.29.1...rel-1.30.0
Release highlights were prepared with AI assistance.
Release Def: Branch: refs/heads/rel-1.29.0 Commit: 2e2543fbe9fae542f921d47a72d21d5a4ef0b710 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.29.0 Commit: 2e2543fbe9fae542f921d47a72d21d5a4ef0b710 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1354516
ORT_DISABLE_TELEMETRY=1 before initialization disables non-Windows telemetry for the process (#27379, #29872).onnxruntime/python/tools/tensorrt dashboard tooling was removed. This does not affect the TensorRT Execution Provider APIs (#29395).k attribute against the number of experts and fixed a CPU TensorScatter security issue (#29907, #29916).Range, and CropAndResize (#29254, #29255, #29265, #29579, #29595, #29605, #29871, #31636, #31671, #31675, #31676, #31684).OrtApi::GetValue and validated DML constant tensor byte sizes (#29157, #31665).adm-zip for onnxruntime-node (#29827, #29926, #31192).ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS. Explicit thread settings still take precedence, and 0 preserves machine-sized defaults (#29688).EpContext nodes, and wired maximum-shape inference into workspace estimation (#29607, #29799, #31613).MRotaryEmbedding contrib operator for Qwen mRoPE variants (#29261, #31728).onnxruntime_perf_test through --data_shape, plus verbose graph-transformer tracing and broader inference-session error-path coverage (#29555, #29558, #29569, #29571).PagedAttention with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support (#29912).Attention CUDA kernel and enabled cuDNN SDPA for contrib Attention (#29715, #29717).attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP (#29525, #31157).MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight, plus block-scaled tensor-core/GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ (#29818, #29850, #29896, #31155, #31481).LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators (#31158, #31835).AllReduce, AllGather, and AllToAll (#31571).GatherBlockQuantized (#31693).GatherBlockQuantized and LpNormalization, reused the shared WASM loader for Blob-backed external data, and fixed per-axis QDQ and MatMulNBits edge cases (#29475, #29801, #31151, #31152, #31197).M from the input tensor at compute time (#31189).Cos and int32 support to CPU Trilu, and fixed int8 QLinearSoftmax saturation and AvgPool ceil_mode/count_include_pad behavior (#28975, #29476, #29629, #29728).TfIdfVectorizer weight indexing and skipped MinLength logits-processor construction when eos_token_id is negative (#29604, #31649).GetOverridableInitializerNames() (#29349, #29589, #29616).Split axes now produce an error instead of being accepted (#31149).ceil_mode, allowed DFT to ignore excess input data, and fixed a webpack/Terser release-build crash (#29627, #29680, #31652).CUDA package architecture selections are now aligned across plugin EP, Python, C API, TensorRT, and Node.js pipelines. Windows arm64 is only available in CUDA plugin EP (#31992):
| OS | CUDA | CUDA architectures (all in -real form) |
|---|---|---|
| Linux x64 | 12.8 | 60;70;75;80;86;89;90a;120a |
| Linux x64 | 13.x | 75;80;86;89;90a;120a |
| Linux aarch64 | 13.x | 89;90a;120a;121a |
| Windows x64 | 12.8 | 61;75;86;89;120a |
| Windows x64 | 13.x | 75;80;86;89;120a |
| Windows arm64 | 13.x | 120a;121a |
Reduced CUDA compilation time and memory usage by splitting generated SM80 MoE, fpA_intB, and MatMulNBits translation units and adding two-level workspace estimation (#29614, #29699, #29811, #31834, #31837).
Fixed CUDA 13 plugin and packaging builds on Windows, Linux, and Windows ARM64, including MSVC/TMA compatibility and CI memory limits (#31608, #31609, #31615, #31616, #31617, #31622, #31729, #31748).
Fixed MLAS AVX2 builds on toolchains without AVX-VNNI assembler support, GCC 15 -Werror builds, and an MSVC C1001 issue in the W2 AVX-512-VNNI dispatch path (#28767, #29679, #29885).
Improved Windows compatibility by skipping DXGI discovery when Win32k system calls are unavailable and delay-loading shell32 (#29755, #30889).
Fixed Dawn parallel-build races, and GPU discovery in build/test environments (#29858, #29866).
Thanks to our 63 contributors for this release!
@adrastogi, @ahsan-ca, @AngelGalindo7, @ankitm3k, @apsonawane, @blazingphoenix7, @bmehta001, @chilo-ms, @claude, @daijh, @ducviet00, @edgchen1, @elwhyjay, @eserscor, @GopalakrishnanN, @guptaishaan, @hariharans29, @Honry, @huningxin, @jchen10, @jiafatom, @jiangzhuo, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @kjg0724, @kunal-vaishnavi, @kylo5aby, @Laan33, @martin-klacer-arm, @mastryukov1990, @mcollinswisc, @miaobin, @mingmingtasd, @mirounga, @mustjab, @n1harika, @namgyu-youn, @neilmsft, @nenad1002, @nicholascelestin, @OscarFree, @prathikr, @qjia7, @quic-muchhsu, @Sammy-Dabbas, @sanaa-hamel-microsoft, @shiyi9801, @skottmckay, @tairenpiao, @TedThemistokleous, @the0cp, @tianleiwu, @titaiwangms, @velonica0, @wangw-1991, @wuisabel-gif, @xadupre, @xhcao, @xiaofeihan1, @xiaoyu-work, @yen-shi, @zlma7001
Full Changelog: v1.28.0...v1.29.0
Release Def: Branch: refs/heads/rel-1.28.0 Commit: da9b5e364c465de65c49d91e696cd6485270757f Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.28.0 Commit: da9b5e364c465de65c49d91e696cd6485270757f Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1324944
nvrtc is no longer linked, which significantly reduces the required CUDA redistributable footprint (#29252, #29808, #29705, #29620).OrtModelPackageApi now lives in the experimental C API and may change in future releases (#28746, #29142, #28990).wgsl-gen implementation (#29141, #28355).CUDA_QUANT_PREPROCESS is off by default (#29687).bind_input causing an out-of-bounds write (#28839)TensorAt for sub-byte packed types (#28973)Col2Im inputs to prevent heap over-read (#28706)CropAndResize against malformed crop_size tensors (#28766)BeamSearch vocab_size against logits width (#28774)WhisperDecoderSubgraph::CreateInitialFeeds (#29239)SparseAttention CSR indices/key lengths and rejected zero-dimension block_row_indices (#29015, #29242)mask_index to valid bounds (#29449)MaxpoolWithMask kernel rank against input spatial rank (#29253)EmbedLayerNorm/SkipLayerNorm shapes exceeding 32-bit output indexing (#29264)DecoderAttention/MultiHeadAttention shape inference and negative-axis handling in ExpandDims shape inference (#29268, #29448)TreeEnsemble target id validation and added input validation to LinearClassifier (#29293, #29060)DynamicQuantizeLSTM zero-point/scale validation typos (#29462)Loop/Scan output concatenation (#29397)raw_data to {0, 1} on unpack (#29238)Resize, PadFusion, and LoRA handling (#28779, #28780, #28801)WithOutputTensor in the Rust bindings (#29251)MlasConvPrepare working-buffer products and ConvTranspose pad computation with SafeInt (#29444, #29446)SamplingState::Init that could cause a heap buffer overflow (#29443)B/scales/zero-points shape in MatMulNBits::PrePack (#29445)ConstantOfShape output size against the input initializer before constant folding (#28751)Pad (int64/int32 truncation), Slice, and GatherBlockQuantized (#28721, #28704, #28718)subprocess (#28776, #28775)shell-quote, esbuild, tmp, ws, protobufjs, js-yaml, tar, markdown-it, @babel/core (#29022, #29044, #29055, #29057, #29061, #29062, #29063, #29079, #29090, #29156)external_data into session options (#28271, #28989, #29501)CompileModel validation to accept zero-input OrtModel graphs (#28771)OrtErrorCode documentation, single-sourced the values so StatusCode stays in sync, and added OrtErrorCode::ORT_DEVICE_RESET (#29018, #29065, #29748)EpDeviceUsage event, and ORT version logging (#28794)model_external_initializers_file_folder_path is now honored for file-path model loads (#29459)HOST_ACCESSIBLE OrtValue allocation (#28038)CudaQuantizer to onnxruntime.quantization (#29509)Flatten as a Direct8Bit op in the Python QDQ static quantizer (#28340)MaxPool during FP8 static quantization and fixed the FP8 (FLOAT8E4M3FN) scale reference distribution (#28488, #29350)TensorArray custom op (#28335)Attention & LLM decode
cudnn_frontend to 1.24 and enabled cuDNN SDPA for MHA/GQA (#28849)MoE & quantized GEMM
PrePack hook, symmetric with MatMulNBits, and fixed the prepack to always use the SM80 layout (#28749, #28978, #28965)MatMulNBits GEMV (#28980, #29167, #29166, #29170)MatMulNBits (#29451)block_size=32, and fused bias (#29622, #29499, #29585)Coverage, dependencies & fixes
ArgMax/ArgMin/ReduceSum and fixed LogSoftmax on the plugin EP (#29620)Softplus and Softsign up to opset 22 (#28982)cudaErrorInvalidValue on Blackwell (sm_120) (#29706)libcudart.so.13 hard dependency that broke import on CPU-only Linux (#29202, #29590)PrePack, and plugin EP allocator deleter lifetime (#29658, #29663)moe_kernels.cu (#29295)Equal/Sub/Where/ReduceSum under the enable_int64 flag (#29236, #29392)max_num_pending_dispatches configurable (#28761, #28894)GatherBlockQuantized (#29054)Cast to int64 by default and switched to naive reduction (#28804, #28174)do_rotary (#29247, #29002)head_size out-of-bounds race, the past_state == present_state buffer aliasing case, and GatherBlockQuantized dispatch failure for empty indices (#29593, #28753, #29030)round_prefer_ceil/round_prefer_floor (#28757)MatMulNBitsMlpFusion when the kernel is unavailable (#29089)Where and And builders (#28597)"model_path" must not be empty error (#29394)PadNodeGroupSelector::Check when dq_nodes is empty (#28733)Gather handler for the transpose optimizer (#28755)MatMulNBits (#29025)process_ext_address (#29248)sbgemm_neon_kernel and fixed failing KleidiAI NHWC unit tests (#28394, #29010)MatMulNBits (#29163)SpaceToDepth and int8 for DepthToSpace (#29154)is_causal bottom-right alignment for external KV cache (#29050, #28958)ReverseSequence returning zeros for zero-length sequences (#28759)RandomForestClassifier binary classification predictions (#28685)Gemm → QGemm fusion when alpha != 1 with bias, and validated DQ scale/zero-point shapes before QGemm fusion (#28131, #28714)Resize with non-nearest interpolation modes under ORT_ENABLE_ALL (#28454)Shape → Gather → TopK rank-1 regression (#28778, #29084)TransposeOptimizer type error on zero-point-less DequantizeLinear (#29192)SimplifiedLayerNorm fusion with a node-produced Pow exponent (#29196)MaxPool (#29201)Conv/ConvTranspose rank from weights when the input shape is unknown (#29149)STFT complex input frame offsets (#28961)wgsl-gen and removed the dynamic WGSL generator path (#28355, #29141)wasm_Release / build-wasm CI (#29040)HOST_ACCESSIBLE OrtValue allocation (#28038)requests package (#28825)outputHandlesArr in the OrtTrainingSession.evalStep JNI binding (#29576)WithOutputTensor (#29251)cpuinfo_deinitialize() (#28245)PIP_INDEX_URL piping for container build stages (#29803, #29823)LinearRegressor test coverage (#28979, #29083)Thanks to our 103 contributors for this release!
@adrastogi, @ankitm3k, @apsonawane, @ArsalanShakil, @bachelor-dou, @bopeng1234, @cbourjau, @chilo-ms, @chunghow-qti, @chwarr, @claude, @Craigacp, @crvineeth97, @dabhattimsft, @daijh, @danielsongmicrosoft, @derdeljan-msft, @edgchen1, @elwhyjay, @ericcraw, @eserscor, @fanchenkong1, @feich-ms, @fs-eire, @FuZoe, @gaugarg-nv, @gblong1, @GopalakrishnanN, @grybouilli, @guschmue, @haoxli, @hariharans29, @hnsyprst, @Honry, @intbf, @ishwar-raut1, @jambayk, @Jaswanth51, @jatinwadhwa921, @javier-intel, @jiafatom, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @jwludzik, @Kotomi-Du, @kpkbandi, @lucka-me, @maoger, @martin-klacer-arm, @maxwbuckley, @MayureshV1, @mc-nv, @mcollinswisc, @mdvoretc-intel, @mingyueliuh, @mklimenk, @n1harika, @namgyu-youn, @neilmsft, @orlmon01, @Osamaali313, @prathikr, @preetha-intel, @psakhamoori, @qiurui144, @qjia7, @qti-hungjuiw, @qti-yuduo, @quic-muchhsu, @raagrawal, @RajeevSekar, @Reranko05, @Rishi-Dave, @RyanMetcalfeInt8, @Sammy-Dabbas, @sanaa-hamel-microsoft, @sayanshaw24, @selenayang888, @sfatimar, @sgbihu, @Shivani767, @ssam18, @susbhere, @sushraja-msft, @tairenpiao, @the0cp, @tianleiwu, @titaiwangms, @umireon, @vthaniel, @wangw-1991, @wenqinI, @xadupre, @xenova, @xhcao, @xiaofeihan1, @xieofxie, @xuke537, @yen-shi, @yinli-systems, @yuslepukhin, @ZackyLake
Full Changelog: v1.27.1...v1.28.0
Release Def: Branch: refs/heads/rel-1.27.1 Commit: df2ba1cf8108aa63627cf4cdf8f807880b938616 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.27.1 Commit: df2ba1cf8108aa63627cf4cdf8f807880b938616 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1300459
This is a patch release on top of v1.27.0, containing targeted bug fixes, a CUDA QMoE decode-path optimization, and CI/build infrastructure fixes.
igemm regression in the KleidiAI path (#28571)azcopy (#29274)brew install applesimutils failure by trusting the wix/brew tap (#29450)mac-cpu-packing-jobs.yml (#29575)Thanks to our 8 contributors for this release!
@tianleiwu, @chilo-ms, @edgchen1, @adrastogi, @damdoo01-arm, @JonathanC-ARM, @martin-klacer-arm, @sanaa-hamel-microsoft
Full Changelog: v1.27.0...v1.27.1
Release Def: Branch: refs/heads/rel-1.27.0 Commit: 8f0278c77bf44b0cc83c098c6c722b92a36ac4b5 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.27.0 Commit: 8f0278c77bf44b0cc83c098c6c722b92a36ac4b5 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1263613
n.b. This release is targeting ONNX 1.21. ONNX 1.22 will be supported in ORT 1.28.
n.b. This changelog was generated via LLM. Only the contributor list has been verified. As always, only trust the commit history.
SoftmaxCrossEntropyLoss via label bounds validation (#28004)OneHot input validation and output-size computation (#28014)Expand and capped constant-folding output sizes (#28055)Tile kernel (#28070)MaxpoolWithMask::Compute (#28223)BitShift UB for shift amounts greater than or equal to bit width (#28272)seqlens_k vs cos_cache) (#28277)WordConvEmbedding to prevent OOB reads (#28279)CropBase scale handling (#28399)torch.load() calls to weights_only=True (#28421)OrtEp::OnSessionInitializationEnd() callback (#28319)kOrtEpDevice_EpMetadataKey_OSDriverVersion example and docs (#28282)quantize_static (#28221)ActivationRestrictedAsymmetric quantization option (#28237)block_size attribute support to QDQ quantization (#28522)FusedAdam optimizer in ORT Training (#28233)ConvTranspose-22 support (#27710)Split (split attribute path) and scalar Gather indices (#28270, #28278)FusedConv, Identity, Ceil, Tile, Cast(bool), Sin, Cos, and GatherND support (#28289, #28293, #28595, #28596, #28598)next, postcss, tmp, qs, body-parser, and other npm packages) (#27705, #27894, #28304, #28547, #28644, #28683, [#28694]present_key/present_value outputs in GQA and Gemma4 support (#28242)Session and InferenceSession (#27802)sympy an optional runtime dependency (#28141)py.typed marker to the onnxruntime package (#28438)UserLoggingFunction is used (#28314)OrtModelEditorApi ownership transfer (#28123)allowzero=1 handling for chained zero-size tensors (#28455)Resize nearest-mode rounding bug for negative halfway values (#28345)Thanks to our 106 contributors for this release!
@adrastogi, @adrianlizarraga, @AIFrameworksIntegration, @AlekseiNikiforovIBM, @angelser, @angelserMS, @ankitm3k, @anzzraju1997-glitch, @apsonawane, @arajendra, @ayappanec, @bachelor-dou, @badranX, @baijumeswani, @BoarQing, @bopeng1234, @bsosnader, @cbourjau, @chilo-ms, @chunghow-qti, @chwarr, @Craigacp, @daijh, @derdeljan-msft, @dparikh79, @edgchen1, @elwhyjay, @ericcraw, @eserscor, @feich-ms, @fs-eire, @gaugarg-nv, @gblong1, @GopalakrishnanN, @gramalingam, @guschmue, @hariharans29, @HectorSVC, @intbf, @ishwar-raut1, @Jaswanth51, @jatinwadhwa921, @javier-intel, @jchen10, @jiafatom, @Jiawei-Shao, @jnagi-intel, @jocelyn-stericker, @JonathanC-ARM, @justinchuby, @jwludzik, @kevinch-nv, @Kotomi-Du, @kpkbandi, @KV2773, @Laan33, @lhrios, @maxwbuckley, @MayureshV1, @mdvoretc-intel, @mingyueliuh, @mklimenk, @mustjab, @n1harika, @nazanin-beheshti, @orlmon01, @prathikr, @preetha-intel, @psakhamoori, @qiurui144, @qjia7, @qti-ashwshan, @qti-hungjuiw, @qti-yuduo, @rajatmonga, @RajeevSekar, @Rishi-Dave, @rvandermeulen, @RyanMetcalfeInt8, @SamuelLess, @sanaa-hamel-microsoft, @sfatimar, @sgbihu, @shiyi9801, @simonbyrne, @skottmckay, @susbhere, @sushraja-msft, @tairenpiao, @TejalKhade28, @theHamsta, @tianleiwu, @titaiwangms, @umangb-09, @velonica0, @vraspar, @vthaniel, @wenqinI, @xadupre, @xenova, @xhan65, @xiaofeihan1, @xieofxie, @yuslepukhin, @ZackyLake, @zejianzhang1982, @zz002
Full Changelog: v0.1.4...v1.27.0
Release Def: Branch: refs/heads/rel-1.26.0 Commit: 8c546c37b43caaca1fa25db430dab94b901cf277 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.26.0 Commit: 8c546c37b43caaca1fa25db430dab94b901cf277 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1209449
n.b. The following was generated via LLM from Git history. Only the contributor list has been verified.
onnxruntime-<os>-<arch>-gpu_cuda13-<version>.<ext>.ort model loads (#28164).setattr configuration with an allowlist (#28083).roundingType -> outputShapeRounding (#28172).@tianleiwu, @yuslepukhin, @edgchen1, @vraspar, @hariharans29, @skottmckay, @eserscor, @xadupre, @sanaa-hamel-microsoft, @elwhyjay, @Rishi-Dave, @titaiwangms, @adrianlizarraga, @jatinwadhwa921, @jchen10, @Jiawei-Shao, @maxwbuckley, @preetha-intel, @qjia7, @qti-hungjuiw, @RajeevSekar, @umangb-09, @adrastogi, @akote123, @amd-genmingz, @ankitm3k, @apsonawane, @bachelor-dou, @baijumeswani, @bopeng1234, @chilo-ms, @chwarr, @Craigacp, @dccarmo, @derdeljan-msft, @ericcraw, @fdwr, @fs-eire, @gaugarg-nv, @gblong1, @GopalakrishnanN, @Honry, @intbf, @ishwar-raut1, @Jaswanth51, @javier-intel, @JonathanC-ARM, @julia-thorn, @justinchuby, @jwludzik, @Kevin-Taha, @Kotomi-Du, @MayureshV1, @mdvoretc-intel, @miaobin, @milpuz01, @mingyueliuh, @mklimenk, @n1harika, @prathikr, @psakhamoori, @qti-yuduo, @quic-calvnguy, @RyanMetcalfeInt8, @sfatimar, @sgbihu, @ShirasawaSama, @ssam18, @susbhere, @sushraja-msft, @TejalKhade28, @theHamsta, @TomCrypto, @TsofnatMaman, @velonica0, @vthaniel, @wenqinI, @xhan65, @xhcao
Release Def: Branch: refs/heads/rel-1.25.1 Commit: 8a77e459420f58fb946fd9067285cfa719f10bdd Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.25.1 Commit: 8a77e459420f58fb946fd9067285cfa719f10bdd Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1190117
n.b. This changelog is LLM generated. Only the contributor listing has been verified.
SetRawDataInTensorProto in NVIDIA TensorRT RTX tests (#28065)Thanks to our 7 contributors for this release:
@guschmue, @sanaa-hamel-microsoft, @apsonawane, @eserscor, @ishwar-raut1, @qjia7, @theHamsta
Full Changelog: v1.25.0...v1.25.1
Release Def: Branch: refs/heads/rel-1.25.0 Commit: 7a71bc575b189cdedea7fa2c0f87389f870bd10e Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.25.0 Commit: 7a71bc575b189cdedea7fa2c0f87389f870bd10e Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1175068
--use_armnn build flags and migrate to the MLAS/KleidiAI-backed CPU EP or QNN EP for Qualcomm hardware. (#27447)onnx::TensorProto data size before allocation (#27547)ORT 1.25.0 introduces the CUDA Plugin EP — the first core implementation that enables third-party CUDA-backed EPs to be delivered as dynamically loaded plugins without rebuilding ORT.
OpKernelInfo::GetConfigOptions, LoggingManager::HasDefaultLogger() (#26879, #26919, #27540, #27541, #27587)enable_profiling in RunOptions (#26846)OrtModel input support for Compile API (#27332)router_weights input to QMoE (#27687)Extended to cover significantly more quantized LLM inference scenarios on CPU:
OrtDeviceVendorId enum for vendor-aware OrtDevice aliases (#27594)GetCompatibilityInfoFromModel / GetCompatibilityInfoFromModelBytes (#27565)OrtValue.from_dlpack rejecting zero-size tensors as non-contiguous (#27451)GetCompatibilityInfoFromModel / GetCompatibilityInfoFromModelBytes (#27565)onnxruntime.native.path (#27668)count_include_pad computation — silent correctness issue (#27375)RemoveDuplicateCastTransformer (#27363)--build_wasm_static_lib implicitly enabling --build_wasm (#27342)Thanks to our 72 contributors for this release!
@tianleiwu, @fs-eire, @edgchen1, @titaiwangms, @hariharans29, @eserscor, @Rishi-Dave, @guschmue, @adrianlizarraga, @jambayk, @qjia7, @skottmckay, @adrastogi, @sanaa-hamel-microsoft, @yuslepukhin, @ingyukoh, @Jiawei-Shao, @vraspar, @xhcao, @chilo-ms, @Honry, @JonathanC-ARM, @kunal-vaishnavi, @ShirasawaSama, @chaya2350, @derdeljan-msft, @gedoensmax, @HectorSVC, @milpuz01, @quic-calvnguy, @xenova, @akholodnamdcom, @AlekseiNikiforovIBM, @amd-genmingz, @ashrit-ms, @bachelor-dou, @BODAPATIMAHESH, @Colm-in-Arm, @daijh, @dodokw, @fanchenkong1, @ivarusic-amd, @JanSellner, @jchen10, @jiafatom, @jnagi-intel, @johannes-rehm-snkeos, @justinchuby, @keshavv27, @Kevin-Taha, @kevinlam92, @kpkbandi, @Laan33, @melkap01-Arm, @miaobin, @n-v-k, @nico-martin, @patryk-kaiser-ARM, @praneshgo, @prathikr, @qc-tbhardwa, @sagarbhure-msft, @sdotpeng, @the0cp, @TsofnatMaman, @umangb-09, @walidbr, @wenqinI, @xadupre, @xhan65, @xiaofeihan1
Full Changelog: v1.24.4...v1.25.0
Release Def: Branch: refs/heads/rel-1.24.4 Commit: 2d924974ef147392ced8409d36bd6d2e7fcc8a74 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.24.4 Commit: 2d924974ef147392ced8409d36bd6d2e7fcc8a74 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1124809
This is a patch release for ONNX Runtime 1.24, containing bug fixes and execution provider updates.
nvidia-drm is not loaded but GPU PCI devices are still exposed via sysfs. (#27591)GetOutputIndex. (#27644)python_requires from >=3.10 to >=3.11 to reflect dropped Python 3.10 support. (#27354)__builtin_ia32_tpause with the compiler-portable _tpause intrinsic to fix cross-compiler portability issues between GCC and LLVM. (#27607)Full Changelog: v1.24.3...v1.24.4
@derdeljan-msft, @adrianlizarraga, @apwojcik, @baijumeswani, @edgchen1, @mocknen, @tianleiwu, @XXXXRT666
Release Def: Branch: refs/heads/rel-1.24.3 Commit: 3a728b75062256951b6e19ce718907cf1a1d4cf0 Build: https://aiinfra.visualstudio.com/Lotus/_build/resul
Release Def: Branch: refs/heads/rel-1.24.3 Commit: 3a728b75062256951b6e19ce718907cf1a1d4cf0 Build: https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=1108362
This is a patch release for ONNX Runtime 1.24, containing bug fixes, security improvements, performance enhancements, and execution provider updates.
OrtEnv.DisableDllImportResolver to prevent fatal error on resolver conflict. (#27535)s_kernel_registry_vitisaiep.reset() in deinitialize_vitisai_ep(). (#27295)OrtEpDevice instances for plugin and provider bridge EPs. (#27522)-Warray-bounds build error in MLAS on clang 17+. (#27499)kMaxValueLength to 8192. (#27521)Full Changelog: v1.24.2...v1.24.3
@tianleiwu, @fs-eire, @adrianlizarraga, @yuslepukhin, @0-don, @anujj, @chaya2350, @chilo-ms, @dabhattimsft, @edgchen1, @eserscor, @hariharans29, @JonathanC-ARM, @lukas-folle-snkeos, @patryk-kaiser-ARM, @praneshgo, @skottmckay, @theHamsta, @vektah, @vishalpandya1990, @vthaniel, @xieofxie, @zz002
Security: Fixed an out-of-bounds read vulnerability in ArrayFeatureExtractor.
This is a patch release for ONNX Runtime 1.24, containing several bug fixes, security improvements, and execution provider updates.
SparseTensorProtoToDenseTensorProto to improve robustness. (#27323)ArrayFeatureExtractor. (#27275)BaseTester to support plugin EPs with both compiled nodes and registered kernels. (#27176)Full Changelog: v1.24.1...v1.24.2
@tianleiwu, @hariharans29, @edgchen1, @xiaofeihan1, @adrianlizarraga, @angelser, @angelserMS, @ankitm3k, @baijumeswani, @bmehta001, @ericcraw, @eserscor, @fs-eire, @guschmue, @mc-nv, @qjia7, @qti-monumeen, @titaiwangms, @yuslepukhin
DoS vulnerability in FuseReluClip
A major infrastructure enhancement enabling plugin-based EPs with dynamic loading:
OrtKernelInfo APIs for kernel-based plugin EPs (#26803)OrtApi::CreateEnvWithOptions() and OrtEpApi::GetEnvConfigEntries() (#26971)KernelInfo (#26589)Arm is formally deprecating the Arm NN Execution Provider (EP) in ONNX Runtime. The Arm NN EP is still experimental and depends on technology that is no longer actively maintained. Keeping it available now only adds complexity and potential confusion for users.
What to expect:
--enable_arm_neon_nchwc to enable this feature (#25580 #26838 #26691 #26171). This feature may be turned ON by default in a future release based on community feedback.SiLU activation perf improvement (#26753)add_external_initializers_from_files (#26012)FuseReluClip (#26878)KernelContext_GetAllocator (#26883)Thanks to our 170 contributors for this release!
@fs-eire, @tianleiwu, @edgchen1, @qjia7, @yuslepukhin, @hariharans29, @Honry, @qti-yuduo, @adrianlizarraga, @snnn, @eserscor, @vraspar, @xiaofeihan1, @guschmue, @daijh, @quic-muchhsu, @qti-jkilpatrick, @tirupath-qti, @Jiawei-Shao, @qti-hungjuiw, @quic-ashwshan, @titaiwangms, @qti-mattsinc, @chilo-ms, @jchen10, @xhcao, @skottmckay, @quic-calvnguy, @JonathanC-ARM, @Rohanjames1997, @sushraja-msft, @jambayk, @adrastogi, @xenova, @quic-tirupath, @justinchuby, @HectorSVC, @kunal-vaishnavi, @wenqinI, @prathikr, @baijumeswani, @preetha-intel, @jatinwadhwa921, @umangb-09, @qti-ashwshan, @carzh, @bachelor-dou, @ranjitshs, @gedoensmax, @xadupre, @nenad1002, @TedThemistokleous, @keshavv27, @zpye, @jnagi-intel, @jiafatom, @mingyueliuh, @Colm-in-Arm, @borg323, @chunghow-qti, @Craigacp, @BODAPATIMAHESH, @AlekseiNikiforovIBM, @hans00, @thevishalagarwal, @MaanavD, @qti-kromero, @damdoo01-arm, @BoarQing, @naomiOvad, @yuhuchua-qti, @hadiFute, @vishalpandya1990, @rivkastroh, @minfhong-qti, @kuanyul-qti, @xieofxie, @ankitm3k, @RyanMetcalfeInt8, @MayureshV1, @bopeng1234, @vthaniel, @mdvoretc-intel, @ericcraw, @javier-intel, @saurabhkale17, @sfatimar, @Kotomi-Du, @intbf, @n1harika, @TejalKhade28, @gupta-pallavi, @cbourjau, @nieubank, @r-devulap, @wszqkzqk, @sanketkaleoss, @amancini-N, @fanchenkong1, @meakbiyik, @hisham-hchowdhu, @shaoboyan091, @Stonesjtu, @qwu16, @wangw-1991, @bonktree, @naetherm, @nikhilfujitsu, @Panxuefeng-loongson, @selenayang888, @moyo1997, @chwarr, @patryk-kaiser-ARM, @fdwr, @SavaLione, @shiyi9801, @mcost45, @aciddelgado, @prudhvi-qti, @Jonahcb, @lifang-zhang, @zhaoxul-qti, @gaugarg-nv, @cocotdf, @WangFengtu1996, @orlmon01, @weidu-tpvision, @theHamsta, @kevinch-nv, @XXXXRT666, @movedancer, @melkap01-Arm, @KingSora, @urpetkov-amd, @junchao-loongson, @jixiongdeng, @wcy123, @GrigoryEvko, @anujj, @peishenyan, @quic-ankus, @jchen351, @yihonglyu, @satyajandhyala, @co63oc, @mschofie, @quic-ashigarg, @asoldano, @nproshun, @jiangzhaoming, @seungtaek94, @liqunfu, @jaholme, @hanbitmyths, @quic-boyuc, @rM-planet, @qti-vaiskv, @AndreyOrb, @pkubaj, @xhan65, @Jaswanth51, @quic-hungjuiw, @jywu-msft, @mklimenk, @derdeljan-msft, @ianfhunter, @NingW101, @feich-ms, @Akupadhye, @wschin
Full Changelog: v1.23.2...rel-1.24.1
Nothing published for this version
Fix Attention GQA implementation on CPU
Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.23.0...v1.23.1
This release introduces Execution Provider (EP) Plugin API, which is a new infrastructure for building plugin-based EPs. (#24887 , #25137, #25124, #25
This release introduces Execution Provider (EP) Plugin API, which is a new infrastructure for building plugin-based EPs. (#24887 , #25137, #25124, #25147, #25127, #25159, #25191, #2524)
This release introduces the ability to dynamically download and install execution providers. This feature is exclusively available in the WinML build and requires Windows 11 version 25H2 or later. To leverage this new capability, C/C++/C# users should use the builds distributed through the Windows App SDK, and Python users should install the onnxruntime-winml package(will be published soon). We encourage users who can upgrade to the latest Windows 11 to utilize the WinML build to take advantage of this enhancement.
Now on Windows some global object will be not destroyed if we detect that the process is being shutting down(#24891) . It will not cause memory leak as when a process ends all the memory will be returned to the operating system. This change can reduce the chance of having crashes on process exit.
Now ONNX Runtime has the ability to automatically discovery computing devices and select the best EPs to download and register. The EP downloading feature currently only works on Windows 11 version 25H2 or later.
ROCM EP was removed from the source tree. Users are recommended to use Migraphx or Vitis AI EPs from AMD. A new EP, Nvidia TensorRT RTX, was added.
EMDSK is upgraded from 4.0.4 to 4.0.8
Added WGSL template support.
SDK Update: Added support for QNN SDK 2.37.
Enhanced performance for SGEMM, IGEMM, and Dynamic Quantized MatMul operations, especially for Conv2D operators on hardware that supports SME2 (Scalable Matrix Extension v2).
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
@1duo, @Akupadhye, @amarin16, @AndreyOrb, @ankan-ban, @ankitm3k, @anujj, @aparmp-quic, @arnej27959, @bachelor-dou, @benjamin-hodgson, @Bonoy0328, @chenweng-quic, @chuteng-quic, @clementperon, @co63oc, @daijh, @damdoo01-arm, @danyue333, @fanchenkong1, @gedoensmax, @genarks, @gnedanur, @Honry, @huaychou, @ianfhunter, @ishwar-raut1, @jing-bao, @joeyearsley, @johnpaultaken, @jordanozang, @JulienMaille, @keshavv27, @kevinch-nv, @khoover, @krahenbuhl, @kuanyul-quic, @mauriciocm9, @mc-nv, @minfhong-quic, @mingyueliuh, @MQ-mengqing, @NingW101, @notken12, @omarhass47, @peishenyan, @pkubaj, @qc-tbhardwa, @qti-jkilpatrick, @qti-yuduo, @quic-ankus, @quic-ashigarg, @quic-ashwshan, @quic-calvnguy, @quic-hungjuiw, @quic-tirupath, @qwu16, @ranjitshs, @saurabhkale17, @schuermans-slx, @sfatimar, @stefantalpalaru, @sunnyshu-intel, @TedThemistokleous, @thevishalagarwal, @toothache, @umangb-09, @vatlark, @VishalX, @wcy123, @xhcao, @xuke537, @zhaoxul-qti
This release adds an optimized CPU/MLAS implementation of DequantizeLinear (8 bit) and introduces the build option client_package_build, which enables
This release adds an optimized CPU/MLAS implementation of DequantizeLinear (8 bit) and introduces the build option client_package_build, which enables default options that are more appropriate for client/on-device workloads (e.g., disable thread spinning by default).
This release replaces static linking of dxcore.lib with optional runtime loading, lowering the minimum supported version from Windows 10 22H2 (10.0.22
This release replaces static linking of dxcore.lib with optional runtime loading, lowering the minimum supported version from Windows 10 22H2 (10.0.22621) to 20H1 (10.0.19041). This enables compatibility with Windows Server 2019 (10.0.17763), where dxcore.dll may be absent.
This release introduces new API's for Model Editor, Auto EP infrastructure, and AOT Compile
MatMulNBits, enabling matrix multiplication with weights quantized to 8 bits.MatMulNBits.Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
Yulong Wang, Jian Chen, Changming Sun, Satya Kumar Jandhyala, Hector Li, Prathik Rao, Adrian Lizarraga, Jiajia Qin, Scott McKay, Jie Chen, Tianlei Wu, Edward Chen, Wanming Lin, xhcao, vraspar, Dmitri Smirnov, Jing Fang, Yifan Li, Caroline Zhu, Jianhui Dai, Chi Lo, Guenther Schmuelling, Ryan Hill, Sushanth Rajasankar, Yi-Hong Lyu, Ankit Maheshkar, Artur Wojcik, Baiju Meswani, David Fan, Enrico Galli, Hans, Jambay Kinley, John Paul, Peishen Yan, Yateng Hong, amarin16, chuteng-quic, kunal-vaishnavi, quic-hungjuiw, Alessio Soldano, Andreas Hussing, Ashish Garg, Ashwath Shankarnarayan, Chengdong Liang, Clément Péron, Erick Muñoz, Fanchen Kong, George Wu, Haik Silm, Jagadish Krishnamoorthy, Justin Chu, Karim Vadsariya, Kevin Chen, Mark Schofield, Masaya, Kato, Michael Tyler, Nenad Banfic, Ningxin Hu, Praveen G, Preetha Veeramalai, Ranjit Ranjan, Seungtaek Kim, Ti-Tai Wang, Xiaofei Han, Yueqing Zhang, co63oc, derdeljan-msft, genmingz@AMD, jiangzhaoming, jing-bao, kuanyul-quic, liqun Fu, minfhong-quic, mingyue, quic-tirupath, quic-zhaoxul, saurabh, selenayang888, sfatimar, sheetalarkadam, virajwad, zz002, Ștefan Talpalaru
Nothing published for this version
Extend CMAKE_CUDA_FLAGS with all Blackwell compute capacity #23928 - @yf711
Chat mode introduced breaking changes in the API (see migration guide).
top_k on CPU.NMS, RoiAlign, NonZero) to TensorRT by default.trt_op_types_to_exclude to exclude specific ops from TensorRT assignment.--use_qnn static_lib.SkipLayerNormalization, MatMulNBits, FusedGemm, FusedConv, EmbedLayerNormalization, BiasGelu, Attention, DynamicQuantizeMatMul, FusedMatMul, QuickGelu, SkipSimplifiedLayerNormalizationChatGLM, Baichuan2, Phi-4, etc.Phi-4 pre/post-processing support for text, vision, and audio.tokenizer.json.ImageCodec now links to native APIs if available; otherwise, falls back to built-in libraries.All the prebuilt Windows packages now require VC++ Runtime version >= 14.40(instead of 14.38). If your VC++ runtime version is lower than that, you may see a crash when ONNX Runtime was initializing. See https://github.com/microsoft/STL/wiki/Changelog#vs-2022-1710 for more details.
Updated minimum iOS and Android SDK requirements to align with React Native 0.76:
All macOS packages now require macOS version >= 13.3.
CMake Version: Increased the minimum required CMake version from 3.26 to 3.28. Added support for CMake 4.0. Python Version: Increased the minimum required Python version from 3.8 to 3.10 for building ONNX Runtime from source. Improved VCPKG support
Added the following cmake options for WebGPU EP
Added cmake option onnxruntime_BUILD_QNN_EP_STATIC_LIB for building with QNN EP as a static library. Removed cmake option onnxruntime_USE_PREINSTALLED_EIGEN.
Fixed a build issue with Visual Studio 2022 17.3 (#23911)
onnxruntime_USE_CUDA_NHWC_OPS by default for CUDA builds.nsync from dependencies.Updated Node.js installation script to support network proxy usage (#23231)
Contributors to ONNX Runtime include members across teams at Microsoft, along with our community members:
Changming Sun, Yulong Wang, Tianlei Wu, Jian Chen, Wanming Lin, Adrian Lizarraga, Hector Li, Jiajia Qin, Yifan Li, Edward Chen, Prathik Rao, Jing Fang, shiyi, Vincent Wang, Yi Zhang, Dmitri Smirnov, Satya Kumar Jandhyala, Caroline Zhu, Chi Lo, Justin Chu, Scott McKay, Enrico Galli, Kyle, Ted Themistokleous, dtang317, wejoncy, Bin Miao, Jambay Kinley, Sushanth Rajasankar, Yueqing Zhang, amancini-N, ivberg, kunal-vaishnavi, liqun Fu, Corentin Maravat, Peishen Yan, Preetha Veeramalai, Ranjit Ranjan, Xavier Dupré, amarin16, jzm-intel, kailums, xhcao, A-Satti, Aleksei Nikiforov, Ankit Maheshkar, Javier Martinez, Jianhui Dai, Jie Chen, Jon Campbell, Karim Vadsariya, Michael Tyler, PARK DongHa, Patrice Vignola, Pranav Sharma, Sam Webster, Sophie Schoenmeyer, Ti-Tai Wang, Xu Xing, Yi-Hong Lyu, genmingz@AMD, junchao-zhao, sheetalarkadam, sushraja-msft, Akshay Sonawane, Alexis Tsogias, Ashrit Shetty, Bilyana Indzheva, Chen Feiyue, Christian Larson, David Fan, David Hotham, Dmitry Deshevoy, Frank Dong, Gavin Kinsey, George Wu, Grégoire, Guenther Schmuelling, Indy Zhu, Jean-Michaël Celerier, Jeff Daily, Joshua Lochner, Kee, Malik Shahzad Muzaffar, Matthieu Darbois, Michael Cho, Michael Sharp, Misha Chornyi, Po-Wei (Vincent), Sevag H, Takeshi Watanabe, Wu, Junze, Xiang Zhang, Xiaoyu, Xinpeng Dou, Xinya Zhang, Yang Gu, Yateng Hong, mindest, mingyue, raoanag, saurabh, shaoboyan091, sstamenk, tianf-fff, wonchung-microsoft, xieofxie, zz002
Deprecate macOS 12 in packaging pipeline (#23017) - @mszhanyi
Prevent int32 quantized bias from clipping by adjusting the weight's scale (#22020) - @adrianlizarraga
Big thank you to the release manager @yf711, along with @adrianlizarraga, @HectorSVC, @jywu-msft, and everyone else who helped to make this patch release process a smooth one!
All ONNX Runtime Training packages have been deprecated. ORT 1.19.2 was the last release for which onnxruntime-training (PyPI), onnxruntime-training-c…
Release Manager: @apsonawane
Full release notes for ONNX Runtime generate() API v0.5.0 can be found here.
Full release notes for ONNX Runtime Extensions v0.13 can be found here.
Big thank you to the release manager @apsonawane, as well as @snnn, @jchen351, @sheetalarkadam, and everyone else who made this release possible!
Tianlei Wu, Yi Zhang, Yulong Wang, Scott McKay, Edward Chen, Adrian Lizarraga, Wanming Lin, Changming Sun, Dmitri Smirnov, Jian Chen, Jiajia Qin, Jing Fang, George Wu, Caroline Zhu, Hector Li, Ted Themistokleous, mindest, Yang Gu, jingyanwangms, liqun Fu, Adam Pocock, Patrice Vignola, Yueqing Zhang, Prathik Rao, Satya Kumar Jandhyala, Sumit Agarwal, Xu Xing, aciddelgado, duanshengliu, Guenther Schmuelling, Kyle, Ranjit Ranjan, Sheil Kumar, Ye Wang, kunal-vaishnavi, mingyueliuh, xhcao, zz002, 0xdr3dd, Adam Reeve, Arne H Juul, Atanas Dimitrov, Chen Feiyue, Chester Liu, Chi Lo, Erick Muñoz, Frank Dong, Jake Mathern, Julius Tischbein, Justin Chu, Xavier Dupré, Yifan Li, amarin16, anujj, chenduan-amd, saurabh, sfatimar, sheetalarkadam, wejoncy, Akshay Sonawane, AlbertGuan9527, Bin Miao, Christian Bourjau, Claude, Clément Péron, Emmanuel, Enrico Galli, Fangjun Kuang, Hann Wang, Indy Zhu, Jagadish Krishnamoorthy, Javier Martinez, Jeff Daily, Justin Beavers, Kevin Chen, Krishna Bindumadhavan, Lennart Hannink, Luis E. P., Mauricio A Rovira Galvez, Michael Tyler, PARK DongHa, Peishen Yan, PeixuanZuo, Po-Wei (Vincent), Pranav Sharma, Preetha Veeramalai, Sophie Schoenmeyer, Vishnudas Thaniel S, Xiang Zhang, Yi-Hong Lyu, Yufeng Li, goldsteinn, mcollinswisc, mguynn-intc, mingmingtasd, raoanag, shiyi, stsokolo, vraspar, wangshuai09
Full changelog: v1.19.2...v1.20.0
Nothing published for this version
Nothing published for this version
ORT 1.19.2 is a small patch release, fixing some broken workflows and introducing bug fixes.
@prathikr, @mszhanyi, @edgchen1, @tianleiwu, @wangyems, @aciddelgado, @mindest, @snnn, @baijumeswani, @MaanavD
Thanks to everyone who helped ship this release smoothly!
Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.19.0...v1.19.2
Nothing published for this version
Remove calls to deprecated api’s
TensorRT
CUDA
CPU
QNN
OpenVINO
DirectML
Changming Sun, Baiju Meswani, Scott McKay, Edward Chen, Jian Chen, Wanming Lin, Tianlei Wu, Adrian Lizarraga, Chester Liu, Yi Zhang, Yulong Wang, Hector Li, kunal-vaishnavi, pengwa, aciddelgado, Yifan Li, Xu Xing, Yufeng Li, Patrice Vignola, Yueqing Zhang, Jing Fang, Chi Lo, Dmitri Smirnov, mingyueliuh, cloudhan, Yi-Hong Lyu, Ye Wang, Ted Themistokleous, Guenther Schmuelling, George Wu, mindest, liqun Fu, Preetha Veeramalai, Justin Chu, Xiang Zhang, zz002, vraspar, kailums, guyang3532, Satya Kumar Jandhyala, Rachel Guo, Prathik Rao, Maximilian Müller, Sophie Schoenmeyer, zhijiang, maggie1059, ivberg, glen-amd, aamajumder, Xavier Dupré, Vincent Wang, Suryaprakash Shanmugam, Sheil Kumar, Ranjit Ranjan, Peishen Yan, Frank Dong, Chen Feiyue, Caroline Zhu, Adam Louly, Ștefan Talpalaru, zkep, winskuo-quic, wejoncy, vividsnow, vivianw-amd, moyo1997, mcollinswisc, jingyanwangms, Yang Gu, Tom McDonald, Sunghoon, Shubham Bhokare, RuomeiMS, Qingnan Duan, PeixuanZuo, Pavan Goyal, Nikolai Svakhin, KnightYao, Jon Campbell, Johan MEJIA, Jake Mathern, Hans, Hann Wang, Enrico Galli, Dwayne Robinson, Clément Péron, Chip Kerchner, Chen Fu, Carson M, Adam Reeve, Adam Pocock.
Big thank you to everyone who contributed to this release!
Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.18.1...v1.19.0
ONNX Runtime Python packages now have numpy dependency >=1.21.6, <2.0. Support for numpy 2.0 will be added in a future release.
Announcements:
TensorRT EP:
Packages:
This patch release also features various other bug fixes, including a CUDA 12.5 build error fix.
Big thank you to @yf711 for driving this release as the release manager and to all our contributors!
@yf711 @jchen351 @mszhanyi @snnn @wangyems @jywu-msft @skottmckay @chilo-ms @moraxu @kevinch-nv @pengwa @wejoncy @pranavsharma @Craigacp @jslhcl @adrianlizarraga @inisis @jeffbloo @mo-ja @kunal-vaishnavi @sumitsays @neNasko1 @yufenglee @dhruvbird @wangshuai09 @xiaoyu-work @axinging @yuslepukhin @YUNQIUGUO @shubhambhokare1 @fs-eire @afantino951 @tboby @HectorSVC @baijumeswani
…iOS cocoapods are being deprecated. Please use the onnxruntime-android Android package, and onnxruntime-c/onnxruntime-objc cocoapods, which support ON…
--rv64, --riscv_toolchain_root, and --riscv_qemu_path.--use_binskim_compliant_compile_flags. Note: All our release binaries are built with this flag, but when building ONNX Runtime from source, this flag is default OFF.TensorRT
CUDA
QNN
OpenVINO
AUTO:GPU,CPU will only create GPU blob, not CPU blob.DirectML
Yi Zhang, Yulong Wang, Adrian Lizarraga, Changming Sun, Scott McKay, Tianlei Wu, Peng Wang, Hector Li, Edward Chen, Dmitri Smirnov, Patrice Vignola, Guenther Schmuelling, Ye Wang, Chi Lo, Wanming Lin, Xu Xing, Baiju Meswani, Peixuan Zuo, Vincent Wang, Markus Tavenrath, Lei Cao, Kunal Vaishnavi, Rachel Guo, Satya Kumar Jandhyala, Sheil Kumar, Yifan Li, Jiajia Qin, Maximilian Müller, Xavier Dupré, Yi-Hong Lyu, Yufeng Li, Alejandro Cid Delgado, Adam Louly, Prathik Rao, wejoncy, Zesong Wang, Adam Pocock, George Wu, Jian Chen, Justin Chu, Xiaoyu, guyang3532, Jingyan Wang, raoanag, Satya Jandhyala, Hariharan Seshadri, Jiajie Hu, Sumit Agarwal, Peter Mcaughan, Zhijiang Xu, Abhishek Jindal, Jake Mathern, Jeff Bloomfield, Jeff Daily, Linnea May, Phoebe Chen, Preetha Veeramalai, Shubham Bhokare, Wei-Sheng Chin, Yang Gu, Yueqing Zhang, Guangyun Han, inisis, ironman, Ivan Berg, Liqun Fu, Yu Luo, Rui Ren, Sahar Fatima, snadampal, wangshuai09, Zhenze Wang, Andrew Fantino, Andrew Grigorev, Ashwini Khade, Atanas Dimitrov, AtomicVar, Belem Zhang, Bowen Bao, Chen Fu, Dhruv Matani, Fangrui Song, Francesco, Frank Dong, Hans Chen, He Li, Heflin Stephen Raj, Jambay Kinley, Masayoshi Tsutsui, Matttttt, Nanashi, Phoebe Chen, Pranav Sharma, Segev Finer, Sophie Schoenmeyer, TP Boudreau, Ted Themistokleous, Thomas Boby, Xiang Zhang, Yongxin Wang, Zhang Lei, aamajumder, danyue, Duansheng Liu, enximi, fxmarty, kailums, maggie1059, mindest, mo-ja, moyo1997 Big thank you to everyone who contributed to this release!
Update copying API header files to make Linux logic consistent with Windows (#19736) - @mszhanyi
General:
Build System & Packages:
Core:
CUDA EP:
TensorRT EP:
Web:
Windows:
Kernel Optimizations:
Models:
This patch release also includes additional fixes by @spampana95 and @enximi. Big thank you to all our contributors!
One column per quarter.