Summary of major features and improvements  

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported:
    • On CPU: Gemma-3n
    • On CPU, GPU: Kokoro-82M, Qwen3-VL-4B with EAGLE-3, Qwen3-ASR, Muse Glimmer 30B, Qwen3.8 27B, Gemma 4 12B, Hy-MT2-1.8B, DeepSeek OCR-2, and Granite 4.0 H Micro
    • On NPUs: FLUX.2-Klein 4B and Kokoro-82M
  • Additional CPU and GPU-enabled models available as early releases: Qwen-Image, Z-Image-Turbo, Granite 4.0 H Tiny, Fun-ASR-Nano, LFM2.5-8B-A1B, MiniCPM5-2B, RF-DETR, BGE Reranker-V2-M3, and BGE M3

• Broader LLM model support and more model compression techniques

  • OpenVINO™ GenAI adds Multi-Token Prediction (MTP) speculative decoding for Gemma 4, Qwen3.5, and Qwen3.6 on CPUs and GPUs, enabling higher throughput and lower latency without sacrificing accuracy.
  • Preview: OpenVINO™ GenAI introduces DFlash acceleration for Qwen models on GPUs, and visual-token support to reduce latency and speed up GenAI pipelines on Intel® Core™ Ultra Series 3 processors.
  • With Tree Drafting (Top-K) for EAGLE-3 now supported in OpenVINO™ GenAI, developers can unlock higher throughput on VLM pipelines compared to Chain Drafting (Top-1).
  • Xe3 integrated graphics optimizations in OpenVINO™ GenAI improve AI inference performance for Gemma 4 models processing long-context inputs on Intel® Core™ Ultra Series 3 processors.
  • OpenVINO extends Instrumentation and Tracing Technology (ITT) profiling support to the NPU, enabling developers to use Intel® VTune™ Profiler to analyze CPU, GPU, and NPU execution through one consistent toolchain.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • OpenVINO™ GenAI introduces support for ASRPipeline in Node.js, enabling JavaScript developers to run automatic speech recognition (such as Whisper and Qwen3-ASR) with streaming and performance metrics using a pipeline API similar to C++ and Python.
  • Preview: Bounded dynamic-shape support on NPUs now extends to vision workloads such as image super-resolution. This capability has been validated with the ESPCN model.
  • Preview: OpenVINO™ Model Server now includes preview support for idle model management, which unloads models when they are not in use to reduce memory usage.
  • OpenVINO™ Model Server adds support for new agentic models such as Muse Glimmer 30B and Qwen3.8 27B.

Support Change and Deprecation Notices

• Discontinued in 2026.0:

  • The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  • The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  • Support for Debian 10 has been discontinued due to the end of its standard support.
  • The PostponedConstant constructor signature has been updated for improved usability:
    • Old (removed): Callable[[Tensor], None]
    • New: Callable[[], Tensor]
  • The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  • The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  • The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  • NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • CPU plugin now requires support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • Dropped support for the TensorFlow Serving (TFS) API in OpenVINO Model Server 2026.3. KServe API is recommended for classic model deployments.
  • Support for Stateful Models was removed in OpenVINO Model Server 2026.3. These capabilities were originally introduced for Kaldi audio models which are no longer applicable. Current audio model support relies on the OpenAI API and pipelines implemented with the OpenVINO™ GenAI library.
  • Support for Python 3.10 will be discontinued in OpenVINO 2026.5, due to its end-of-life (EOL) status.

• Deprecated and to be removed in the future:

  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • Starting with 2026.0 release major internal refactoring of the graph iteration mechanism has been implemented for improved performance and maintainability. The legacy path can be enabled by setting the ONNX_ITERATOR=0 environment variable. This legacy path is deprecated and will be removed in future releases.
  • Modifying runtime information (RTMap) through read-only ConstOutput objects in the Python API is deprecated and will be removed in a 2027.0 release. Users should use non-const outputs when updating runtime metadata; attempts to modify RTMap through ConstOutput now emit a deprecation warning to ease migration.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO™ Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.
    • OpenVINO™ GenAI:
      • start_chat() / finish_chat() APIs are deprecated and will be removed in a future major release. Pass a ChatHistory object directly to generate() instead.

Acknowledgements

Thanks for contributions from the OpenVINO developer community:
@ainergiz
@Akihiro-Tabuchi
@aobolensk
@ash1shb1sht
@da3dsoul
@deepak0x
@goyaladitya05
@hrushitfujitsu
@jatinwadhwa921
@Lagmator22
@lxcxjxhx
@mostafafaheem
@NaiveTomcat
@nuclearcat
@odili
@Passavee-Losripat
@samwebster
@vtanisha

You can find OpenVINO™ toolkit 2026.4 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.4.0

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

Summary of major features and improvements  

• Functionally enabled Muse-Glimmer-30B model:
  • Model card and IR available: OpenVINO/Muse-Glimmer-30B-int4-ov · Hugging Face
  • Notebooks to try:
    • Muse Glimmer-30B multimodal reasoning with OpenVINO™
    • Create VLM-powered Chatbot using OpenVINO Generate API
  • Blogs:
    • Running Meta’s Muse Glimmer 30B Locally with OpenVINO™ GenAI;
    • Day 0 Support for Muse Glimmer from Meta on Intel Platforms
• Functionally enabled Qwen3.8-27B model:
  • Model card and IR available: OpenVINO/Qwen3.8-27B-int4-ov · Hugging Face
  • Notebook to try: Create VLM-powered Chatbot using OpenVINO Generate API
  • Blog: Day 0 Support for Qwen3.8–27B: Run the New Model on Intel Agentic PCs with OpenVINO™ from Day One
• Fixed issues:
  • Property ENABLE_CPU_PINNING supported now on dual-socket Intel® Xeon® 6+ processors (formerly codenamed Clearwater Forest) Windows Servers.
  • Chat template was updated for Gemma 4 models. Now it can be parced by the Minja templating engine.
  • Fixed accuracy issue for the Gemma 4 26B A4B model when the input prompt is long.
  • Fixed accuracy issue for the Gemma 4 31B model when an image is included in the input.
  • Fixed accuracy issue for lfm2-1.2b, lfm2.5-1.2b-instruct models for NPU device target.

You can find OpenVINO™ toolkit 2026.3.1 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.3.1

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

Summary of major features and improvements  

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported:
    • On CPU, GPU, and NPU: SmolLM3-3B, LFM2-1.2B, LFM2.5-1.2B
    • On CPU and GPU: Harrier OSS-v1-0.6B, Qwen3-8B with EAGLE-3, MiniCPM5-1B, FLUX.2-klein-4B
    • Extended to GPU and NPU: YOLO26
    • Additional models available as early releases on CPU and GPU: Qwen3-ASR, Qwen3-Omni, Gemma-3n, Qwen3-VL-Embedding-8B, Kokoro-82M
  • Support for Hugging Face Transformers v5.5, ensuring compatibility with the latest model architectures on Hugging Face

• Broader LLM model support and more model compression techniques

  • OpenVINO™ GenAI extends the EAGLE-3 speculative decoding pipeline to LLMs and VLMs, enhancing existing continuous batching and adding Top-K sampling to deliver additional token-generation speedups on CPUs, GPUs, and NPUs.
  • Lazy weight loading is enabled for IR and ONNX models to automatically select an optimal loading and compilation path, minimizing peak memory usage during model initialization.
  • Neural Network Compression Framework (NNCF) now supports FP8 quantization for ONNX models, helping developers realize FP8 performance, accuracy, and memory gains while expanding low-precision inference options for production deployments.
  • OpenVINO™ GenAI now supports three additional pipelines: Omni for multimodal workloads, ASR for speech recognition, and Embedding for multimodal embedding generation.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Introducing support for Intel® Xeon® 6+ processors (formerly codenamed Clearwater Forest)
  • MoE offloading to disk enabled, allowing 30B MoE models like Qwen3-30B-A3B to run even on devices with 16 GB of memory while maintaining acceptable tokens-per-second (TPS) generation rates.
  • OpenVINO™ Model Server simplifies model deployment and unifies REST API endpoints, reducing command complexity while providing standard v1/chat/completions support for easier integration with other serving frameworks.
  • OpenVINO™ Model Server further improves stability, performance, and accuracy for LLMs such as Qwen3.5/3.6 with linear attention and extends tool-parser support to MiniCPM5-1B and LFM2.5, improving accuracy and reliability.

Support Change and Deprecation Notices

• Discontinued in 2026.0:

  • The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  • The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  • Support for Debian 10 has been discontinued due to the end of its standard support.
  • The PostponedConstant constructor signature has been updated for improved usability:
    • Old (removed): Callable[[Tensor], None]
    • New: Callable[[], Tensor]
  • The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  • The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  • The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  • NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • CPU plugin now requires support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • OpenVINO™ migrated builds based on RHEL 8 to RHEL 9.
  • Dropped support for the TensorFlow Serving (TFS) API in OpenVINO Model Server 2026.3. KServe API is recommended for classic model deployments.

• Deprecated and to be removed in the future:

  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • Starting with 2026.0 release major internal refactoring of the graph iteration mechanism has been implemented for improved performance and maintainability. The legacy path can be enabled by setting the ONNX_ITERATOR=0 environment variable. This legacy path is deprecated and will be removed in future releases.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO™ Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • Support for Stateful models will be deprecated. These capabilities were originally introduced for Kaldi audio models which is no longer relevant. Current audio models support relies on the OpenAI API, and pipelines implemented via OpenVINO GenAI library.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.
    • OpenVINO™ GenAI:
      • start_chat() / finish_chat() APIs are deprecated and will be removed in a future major release. Pass a ChatHistory object directly to generate() instead.

You can find OpenVINO™ toolkit 2026.3 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.3.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community:
@abhijain1204fujitsu
@Akihiro-Tabuchi
@ankitm3k
@aobolensk
@AshutoshSinghIntel
@daehyun99
@ggladilov
@goyaladitya05
@Lagmator22
@pj3iL
@pjordanandrsn
@ssam18
@sshekhar563
@Sujanian1304
@tknose
@vmadananth
@wine99

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

Summary of improvements  

• Fixed issue ID 187077: YOLO26 fails to compile on GPU
• Resolved an issue in NPU plugin with shared L0 command queue causing violation of app/inference priorities

You can find OpenVINO™ toolkit 2026.2.1 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.2.1

Summary of major features and improvements  

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported: Gemma 4 E2B and Gemma 4 E4B
    • Only on CPUs & GPUs: Qwen3-Coder-Next, Qwen3.5, Qwen3.6, Trinity-mini, LFM2-24B-A2B, LFM2-8B-A1B, LFM2.5-350M​
    • Only on CPUs: YOLO26
    • Only on GPUs: Gemma 4 31B and Gemma 4 26B-A4B
    • Extended to GPUs: GPT-OSS-120B
  • Scaled Dot-Product Attention (SDPA) path support added for LFM2 models
  • Support for Hugging Face Transformers v5.0, ensuring compatibility with the latest model architecture for enhanced interoperability.

• Broader LLM model support and more model compression techniques

  • OpenVINO™ GenAI introduces extension support for loading custom extension libraries and registering unsupported operations via the extensions property. This gives developers the flexibility to run models with custom ops that OpenVINO doesn't support out of the box.
  • INT4 KV-cache compression is enabled for GPUs, with substantial memory reduction when KV cache size is significant, such as with large input prompts exceeding 32K tokens.
  • OpenVINO GenAI significantly reduces model loading times on GPU when using cache blobs — preventing bottlenecks for multi-stage AI pipelines, including agentic use cases that rely on multiple models.
  • Optimized IR read mode with independently managed constant buffers to reduce peak memory usage by avoiding unnecessary duplication of weight data unless required for correctness (Linux support added in this release).
  • Preview: Enhanced XAttention accuracy on CPUs and GPUs through by-channel INT8 KV-cache quantization (compared to by-token INT8 KV-cache), matching the default by-channel INT8 KV cache quantization when XAttention is not enabled.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • OpenVINO™ GenAI extends its JavaScript API to include a Text-to-Speech pipeline and VLM samples for browser and Node.js developers.
  • Prompt Lookup Decoding extended to vision-language pipelines, delivering significantly faster token generation for multimodal workloads on Intel CPUs and GPUs.
  • OpenVINO™ GenAI now has a smaller runtime footprint after eliminating ICU DLL dependencies from tokenization, leading to reduced memory usage, faster startup, and easier deployment.
  • OpenVINO™ Model Server extends tool-calling support to Qwen 3.5 and 3.6 models to enable agentic AI use cases.
  • OpenVINO™ Model Server adds streaming transcription support for speech-to-text, reducing latency for real-time voice applications.

Support Change and Deprecation Notices

• Discontinued in 2026.0:

  • The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  • The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  • The PostponedConstant constructor signature has been updated for improved usability:
    • Old (removed): Callable[[Tensor], None]
    • New: Callable[[], Tensor]
  • The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  • The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  • The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  • NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • CPU plugin now requires support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • OpenVINO™ migrated builds based on RHEL 8 to RHEL 9.
  • manylinux2014 upgraded to manylinux_2_28. This aligns with modern toolchain requirements but also means that CentOS 7 will no longer be supported due to glibc incompatibility.

• Deprecated and to be removed in the future:

  • Support for Ubuntu 20.04 has been discontinued due to the end of its standard support.
  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • With the release of Node.js v22, updated Node.js bindings are now available and compatible with the latest LTS version. These bindings do not support CentOS 7, as they rely on newer system libraries unavailable on legacy systems.
  • Starting with 2026.0 release major internal refactoring of the graph iteration mechanism has been implemented for improved performance and maintainability. The legacy path can be enabled by setting the ONNX_ITERATOR=0 environment variable. This legacy path is deprecated and will be removed in future releases.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • TensorFlow Serving (TFS) API support is planned for deprecation. With increasing adoption of the KServe API for classic models and the OpenAI API for generative workloads, usage of the TFS API has significantly declined. Dropping date is to be determined based on the feedback, with a tentative target of mid-2026.
    • Support for Stateful models will be deprecated. These capabilities were originally introduced for Kaldi audio models which is no longer relevant. Current audio models support relies on the OpenAI API, and pipelines implemented via OpenVINO GenAI library.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.
    • OpenVINO™ GenAI:
      • start_chat() / finish_chat() APIs are deprecated and will be removed in a future major release. Pass a ChatHistory object directly to generate() instead.

You can find OpenVINO™ toolkit 2026.2 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.2.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community:
@abbyssoul
@aobolensk
@AshutoshSinghIntel
@dbermond
@desertfury
@FarseenSh
@goyaladitya05
@KarSri7694
@Lagmator22
@mostafafaheem
@om4rrr
@pjordanandrsn
@reeseliao
@Sahilbhatane
@ssam18
@sshekhar563
@Vishwa2684
@vmadananth

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

Summary of major features and improvements  

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported on CPUs & GPUs: Qwen3 VL
  • New models supported on CPUs: GPT-OSS 120B
  • Preview: Introducing the OpenVINO backend for llama.cpp, which enables optimized inference on Intel CPUs, GPUs, and NPUs. Validated on GGUF models such as Llama-3.2-1B-Instruct-GGUF, Phi-3-mini-4k-instruct-gguf, Qwen2.5-1.5B-Instruct-GGUF, and Mistral-7B-Instruct-v0.3.
  • New notebook: Unified VLM chatbot with video file support and interactive model switching across Qwen3-VL, Qwen2.5-VL, and LLaVa-NeXT-Video.

• Broader LLM model support and more model compression techniques

  • OpenVINO™ GenAI adds TaylorSeer Lite caching for image and video generation, accelerating diffusion-transformer inference across Flux, SD3, and LTX-Video pipelines, aligned with Hugging Face Diffusers.
  • LTX-Video generation on GPU achieves end-to-end acceleration through fusion of RMSNorm and RoPE operators, significantly improving video generation performance.
  • OpenVINO™ GenAI adds dynamic LoRA support for Qwen3-VL and VL models with LLM, allowing developers to swap adapters at runtime for efficient serving of multiple model variants in production without reloading the base model.
  • Preview: The release-weights API for ov::Model enables memory reclamation during model compilation on NPUs, delivering dramatically lower peak memory consumption for edge and client deployments. Users must set this property in ov::Model, and it will be applied during compilation.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Introducing support for Intel® Core™ Series 3 processors (formerly codenamed Wildcat Lake) and Intel® Arc™ Pro B70 Graphics with 32GB memory for single-GPU inference on 20-30B parameter LLMs
  • Prompt Lookup Decoding extended to vision-language pipelines, delivering significantly faster token generation for multimodal workloads on Intel CPUs and GPUs.
  • OpenVINO™ GenAI now has a smaller runtime footprint after eliminating ICU DLL dependencies from tokenization, leading to reduced memory usage, faster startup, and easier deployment.
  • OpenVINO GenAI introduces WhisperPipeline for Node.js via its NPM package, delivering production-ready speech recognition with word-level audio-to-text transcription.
  • OpenVINO™ Model Server enhances support for Qwen3-MOE and GPT-OSS-20b models, delivering improved performance, accuracy, and robust concurrent request handling with continuous batching. These pre-optimized models are available on Hugging Face for easy deployment. Additionally, the Model Server introduces image inpainting and outpainting capabilities via the /image endpoint for AI image editing.

Support Change and Deprecation Notices

• Discontinued in 2026.0:

  • The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  • The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  • The PostponedConstant constructor signature has been updated for improved usability:
    • Old (removed): Callable[[Tensor], None]
    • New: Callable[[], Tensor]
  • The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  • The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  • The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  • NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • CPU plugin now requires support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • OpenVINO™ migrated builds based on RHEL 8 to RHEL 9.
  • manylinux2014 upgraded to manylinux_2_28. This aligns with modern toolchain requirements but also means that CentOS 7 will no longer be supported due to glibc incompatibility.
  • MacOS x86 is no longer supported.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure is unavailable starting 2026.0. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries removed from Docker images.

• Deprecated and to be removed in the future:

  • Support for Ubuntu 20.04 has been discontinued due to the end of its standard support.
  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • With the release of Node.js v22, updated Node.js bindings are now available and compatible with the latest LTS version. These bindings do not support CentOS 7, as they rely on newer system libraries unavailable on legacy systems.
  • Starting with 2026.0 release major internal refactoring of the graph iteration mechanism has been implemented for improved performance and maintainability. The legacy path can be enabled by setting the ONNX_ITERATOR=0 environment variable. This legacy path is deprecated and will be removed in future releases.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • TensorFlow Serving (TFS) API support is planned for deprecation. With increasing adoption of the KServe API for classic models and the OpenAI API for generative workloads, usage of the TFS API has significantly declined. Dropping date is to be determined based on the feedback, with a tentative target of mid-2026.
    • Support for Stateful models will be deprecated. These capabilities were originally introduced for Kaldi audio models which is no longer relevant. Current audio models support relies on the OpenAI API, and pipelines implemented via OpenVINO GenAI library.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.
    • OpenVINO™ GenAI:
      • start_chat() / finish_chat() APIs are deprecated and will be removed in a future major release. Pass a ChatHistory object directly to generate() instead.

You can find OpenVINO™ toolkit 2026.1 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.1.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community:
@andersendsa
@arunthakur009
@ItzCobaltboy
@KarSri7694
@Rahuldrabit
@samthakur587
@satheeshbhukya
@Shi-pra-19
@Sujanian1304

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

Summary of major features and improvements  

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported on CPUs & GPUs: GPT-OSS-20B, MiniCPM-V-4_5-8B, and MiniCPM-o-2.6​
  • New models supported on NPUs: MiniCPM-o-2.6. In addition, NPU support is now available on Qwen2.5-1.5B-Instruct, Qwen3-Embedding-0.6B, Qwen-2.5-coder-0.5B.​
  • OpenVINO™ GenAI now adds word-level timestamp functionality to the Whisper Pipeline on CPUs, GPUs, and NPUs, enabling more accurate transcriptions and subtitling in line with OpenAI and FasterWhisper implementations.​
  • Phi-3-mini FastDraft model is now available on Hugging Face to accelerate LLM inference on NPUs. FastDraft optimizes speculative decoding for LLMs.

• Broader LLM model support and more model compression techniques

  • With the new int4 data-aware weight compression for 3D MatMuls, the Neural Network Compression Framework enables MoE LLMs to run with reduced memory, bandwidth, and improved accuracy compared to data-free schemes-delivering faster, more efficient deployment on resource-constrained devices.​
  • Preview: the Neural Network Compression Framework now supports per-layer and per-group Look-Up Tables (LUT) for FP8-4BLUT quantization. This enables fine-grained, codebook-based compression that reduces model size and bandwidth while improving inference speed and accuracy for LLMs and transformer workloads.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Preview: OpenVINO™ GenAI adds VLM pipeline support to enhance Agentic AI framework integration.​
  • OpenVINO GenAI now supports speculative decoding for NPUs, delivering improved performance and efficient text generation through a small draft model that is periodically validated by the full-size model.​
  • Preview: NPU compiler integration with the NPU plugin enables ahead-of-time and on-device compilation without relying on OEM driver updates. Developers can enable this feature for a single, ready-to-ship package that reduces integration friction and accelerates time-to-value.​
  • OpenVINO™ Model Server adds enhanced support for audio endpoint plus agentic continuous batching and concurrent runs for improved LLM performance in agentic workflows on Intel CPUs and GPUs.

Support Change and Deprecation Notices

• Discontinued in 2026.0:

  • The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  • The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  • The PostponedConstant constructor signature has been updated for improved usability:
    • Old (removed): Callable[[Tensor], None]
    • New: Callable[[], Tensor]
  • The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  • The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  • The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  • NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • CPU plugin now requires support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • OpenVINO™ migrated builds based on RHEL 8 to RHEL 9.
  • manylinux2014 upgraded to manylinux_2_28. This aligns with modern toolchain requirements but also means that CentOS 7 will no longer be supported due to glibc incompatibility.
  • MacOS x86 is no longer supported.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure is unavailable starting 2026.0. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries removed from Docker images.

• Deprecated and to be removed in the future:

  • Support for Ubuntu 20.04 has been discontinued due to the end of its standard support.
  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • With the release of Node.js v22, updated Node.js bindings are now available and compatible with the latest LTS version. These bindings do not support CentOS 7, as they rely on newer system libraries unavailable on legacy systems.
  • Starting with 2026.0 release major internal refactoring of the graph iteration mechanism has been implemented for improved performance and maintainability. The legacy path can be enabled by setting the ONNX_ITERATOR=0 environment variable. This legacy path is deprecated and will be removed in future releases.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • TensorFlow Serving (TFS) API support is planned for deprecation. With increasing adoption of the KServe API for classic models and the OpenAI API for generative workloads, usage of the TFS API has significantly declined. Dropping date is to be determined based on the feedback, with a tentative target of mid-2026.
    • Support for Stateful models will be deprecated. These capabilities were originally introduced for Kaldi audio models which is no longer relevant. Current audio models support relies on the OpenAI API, and pipelines implemented via OpenVINO GenAI library.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.

You can find OpenVINO™ toolkit 2026.0 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2026.0.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community:
@Sujanian1304
@Nishant-ZFYII
@srinjoydutta03
@samthakur587
@IamRam3

Release documentation is available here: https://docs.openvino.ai/2026
Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

NOTE: Please continue using OpenVINO 2025.4 release unless you require the specific bug fixes addressed in the 2025.4.1 version.

Summary of improvements

• Preview: Mixture of Experts (MoE) models optimized for CPUs and GPUs, validated for GPT-OSS 20B model. How to convert model: optimum-cli export openvino -m "openai/gpt-oss-20b" out_dir --weight-format int4
• Fixed issue ID 174531: Accuracy regression of Mistral-7b-instruct-v0.2 and Mistral-7b-instruct-v0.3 on all devices when executed with OpenVINO GenAI. As a workaround, use the IR converted with OpenVINO 2025.3.
• Fixed issue ID 176777: Using the callback parameter with the Python API call generate() in Text2ImagePipeline, Image2ImagePipeline, InpaintingPipeline may cause the process to hang. As a workaround, do not use the callback parameter. C++ implementations was not affected.
• Resolved an issue in the NPU plugin where the Level Zero (L0) context was implemented as a static global object and only destroyed during DLL unload, even after unload_plugin() was called. This behavior prevented the driver from spawning threads required for certain optimizations and features.

You can find OpenVINO™ toolkit 2025.4.1 release here:

• Download archives* with OpenVINO™
• OpenVINO™ for Python: pip install openvino==2025.4.1

Summary of major features and improvements

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported:​
    • On CPUs & GPUs: Qwen3-Embedding-0.6B, Qwen3-Reranker-0.6B, Mistral-Small-24B-Instruct-2501.
    • On NPUs: Gemma-3-4b-it and Qwen2.5-VL-3B-Instruct.
  • Preview: Mixture of Experts (MoE) models optimized for CPUs and GPUs, validated for Qwen3-30B-A3B.
  • GenAI pipeline integrations: Qwen3-Embedding-0.6B and Qwen3-Reranker-0.6B for enhanced retrieval/ranking, and Qwen2.5VL-7B for video pipeline.

• Broader LLM model support and more model compression techniques

  • Gold support for Windows ML* enables developers to deploy AI models and applications effortlessly across CPUs, GPUs, and NPUs on Intel® Core™ Ultra processor-powered AI PCs.
  • The Neural Network Compression Framework (NNCF) ONNX backend now supports INT8 static post-training quantization (PTQ) and INT8/INT4 weight-only compression to ensure accuracy parity with OpenVINO IR format models. SmoothQuant algorithm support added for INT8 quantization.
  • Accelerated multi-token generation for GenAI, leveraging optimized GPU kernels to deliver faster inference, smarter KV-cache reuse, and scalable LLM performance.
  • GPU plugin updates include improved performance with prefix caching for chat history scenarios and enhanced LLM accuracy with dynamic quantization support for INT8.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Announcing support for Intel® Core™ Ultra Processor Series 3.
  • Encrypted blob format support added for secure model deployment with OpenVINO™ GenAI. Model weights and artifacts are stored and transmitted in an encrypted format, reducing risks of IP theft during deployment. Developers can deploy with minimal code changes using OpenVINO GenAI pipelines.
  • OpenVINO™ Model Server and OpenVINO™ GenAI now extend support for Agentic AI scenarios with new features such as output parsing and improved chat templates for reliable multi-turn interactions, and preview functionality for the Qwen3-30B-A3B model. OVMS also introduces a preview for audio endpoints.
  • NPU deployment is simplified with batch support, enabling seamless model execution across Intel® Core™ Ultra processors while eliminating driver dependencies. Models are reshaped to batch_size=1 before compilation.
  • The improved NVIDIA Triton Server* integration with OpenVINO backend now enables developers to utilize Intel GPUs or NPUs for deployment.

Support Change and Deprecation Notices

• Discontinued in 2025:

  • Runtime components:

    • The OpenVINO property of Affinity API is no longer available. It has been replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
    • The runtime namespace for Python API has been marked as deprecated and designated to be removed for 2026.0. The new namespace structure has been delivered, and migration is possible immediately. Details will be communicated through warnings and via documentation.
    • Binary operations Node API has been removed from Python API after previous deprecation.
    • PostponedConstant Python API Update: The PostponedConstant constructor signature is changing for better usability. Update maker from Callable[[Tensor], None] to Callable[[], Tensor]. The old signature will be removed in version 2026.0.

  • Tools:

    • The OpenVINO™ Development Tools package (pip install openvino-dev) is no longer available for OpenVINO releases in 2025.
    • Model Optimizer is no longer available. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
    • Intel® Streaming SIMD Extensions (Intel® SSE) are currently not enabled in the binary package by default. They are still supported in the source code form.
    • Legacy prefixes: l_, w_, and m_ have been removed from OpenVINO archive names.

  • OpenVINO GenAI:

    • StreamerBase::put(int64_t token)
    • The Bool value for Callback streamer is no longer accepted. It must now return one of three values of StreamingStatus enum.
    • ChunkStreamerBase is deprecated. Use StreamerBase instead.
  • Deprecated OpenVINO Model Server (OVMS) benchmark client in C++ using TensorFlow Serving API.

  • NPU Device Plugin:

    • Removed logic to detect and handle Intel® Core™ Ultra Processors (Series 1) drivers older than v1688. Since v1688 is the earliest officially supported driver, older versions (e.g., v1477) are no longer recommended or supported.
  • Python 3.9 support will be discontinued starting with the OpenVINO 2025.4 and Neural Network Compression Framework (NNCF) 2.19.0.
• Deprecated and to be removed in the future:
  • openvino.Type.undefined is now deprecated and will be removed with version 2026.0. openvino.Type.dynamic should be used instead.
  • The openvino-nightly PyPI module has been discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.
  • auto shape and auto batch size (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • MacOS x86 is no longer recommended for use due to the discontinuation of validation. Full support will be removed later in 2025.
  • The openvino namespace of the OpenVINO Python API has been redesigned, removing the nested openvino.runtime module. The old namespace is now considered deprecated and will be discontinued in 2026.0. A new namespace structure is available for immediate migration. Details will be provided through warnings and documentation.
  • Starting with OpenVINO release 2026.0, the CPU plugin will require support for the AVX2 instruction set as a minimum system requirement. The SSE instruction set will no longer be supported.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure will still be available until 2026, when the change will be finalized. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries will be removed from Docker images in 2026.
  • Starting with the 2026.0 release, OpenVINO will migrate builds based on RHEL 8 to RHEL 9.
  • NNCF create_compressed_model() method is now deprecated and will be removed in 2026. nncf.quantize() method is recommended for Quantization-Aware Training of PyTorch models.
  • NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and will be removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  • The following experimental NNCF methods are deprecated and will be removed in 2026: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  • Starting with the 2026.0 release, manylinux2014 will be upgraded to manylinux_2_28. This aligns with modern toolchain requirements but also means that CentOS 7 will no longer be supported due to glibc incompatibility.
  • With the release of Node.js v22, updated Node.js bindings are now available and compatible with the latest LTS version. These bindings do not support CentOS 7, as they rely on newer system libraries unavailable on legacy systems.
  • OpenVINO Model Server:
    • The dedicated OpenVINO operator for Kubernetes and OpenShift is now deprecated in favor of the recommended KServe operator. The OpenVINO operator will remain functional in upcoming OpenVINO Model Server releases but will no longer be actively developed. Since KServe provides broader capabilities, no loss of functionality is expected. On the contrary, more functionalities will be accessible and migration between other serving solutions and OpenVINO Model Server will be much easier.
    • TensorFlow Serving (TFS) API support is planned for deprecation. With increasing adoption of the KServe API for classic models and the OpenAI API for generative workloads, usage of the TFS API has significantly declined. Dropping date is to be determined based on the feedback, with a tentative target of mid-2026.
    • Support for Stateful models will be deprecated. These capabilities were originally introduced for Kaldi audio models which is no longer relevant. Current audio models support relies on the OpenAI API, and pipelines implemented via OpenVINO GenAI library.
    • Directed Acyclic Graph Scheduler will be deprecated in favor of pipelines managed by MediaPipe scheduler and will be removed in 2026.3. That approach gives more flexibility, includes wider range of calculators and has support for using processing accelerators.

You can find OpenVINO™ toolkit 2025.4 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2025.4.0
• OpenVINO™ for Python: pip install openvino==2025.4.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @arunthakur009 @mahdi-jfri @nashez @RudraCodesForU @Sujanian1304 @Vladislav-Denisov

Release documentation is available here: https://docs.openvino.ai/2025 Release Notes are available here: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported: Phi-4-mini-reasoning, AFM-4.5B, Gemma-3-1B-it, Gemma-3-4B-it, and Gemma-3-12B.​
  • NPU support added for: Qwen3-1.7B, Qwen3-4B, and Qwen3-8B.
  • LLMs optimized for NPU now available on OpenVINO Hugging Face collection.
  • Preview: Intel® Core™ Ultra Processor and Windows-based AI PCs can now leverage the OpenVINO™ Execution Provider for Windows* ML for high-performance, off-the-shelf starting experience on Windows*.

• Broader LLM model support and more model compression techniques

  • The NPU plug-in adds support for longer contexts of up to 8K tokens, dynamic prompts, and dynamic LoRA for improved LLM performance.
  • The NPU plug-in now supports dynamic batch sizes by reshaping the model to a batch size of 1 and concurrently managing multiple inference requests, enhancing performance and optimizing memory utilization.
  • Accuracy improvements for GenAI models on both built-in and discrete graphics achieved through the implementation of the key cache compression per channel technique, in addition to the existing KV cache per-token compression method.
  • OpenVINO™ GenAI introduces TextRerankPipeline for improved retrieval relevance and RAG pipeline accuracy, plus Structured Output for enhanced response reliability and function calling while ensuring adherence to predefined formats.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Announcing support for Intel® Arc™ Pro B-Series (B50 and B60).
  • Preview: Hugging Face models that are GGUF-enabled for OpenVINO GenAI are now supported by the OpenVINO™ Model Server for popular LLM model architectures such as DeepSeek Distill, Qwen2, Qwen2.5, and Llama 3. This functionality reduces memory footprint and simplifies integration for GenAI workloads.
  • With improved reliability and tool call accuracy, the OpenVINO™ Model Server boosts support for agentic AI use cases on AI PCs, while enhancing performance on Intel CPUs, built-in GPUs, and NPUs.
  • int4 data-aware weights compression, now supported in the Neural Network Compression Framework (NNCF) for ONNX models, reduces memory footprint while maintaining accuracy and enables efficient deployment in resource-constrained environments.

Support Change and Deprecation Notices

• Discontinued in 2025:

  • Runtime components:

    • The OpenVINO property of Affinity API is no longer available. It has been replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
    • The openvino-nightly PyPI module has been discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.The openvino-nightly PyPI module has been discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.
    • Binary operations Node API has been removed from Python API after previous deprecation.

  • Tools:

    • The OpenVINO™ Development Tools package (pip install openvino-dev) is no longer available for OpenVINO releases in 2025.
    • Model Optimizer is no longer available. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
    • Intel® Streaming SIMD Extensions (Intel® SSE) are currently not enabled in the binary package by default. They are still supported in the source code form.
    • Legacy prefixes: l_, w_, and m_ have been removed from OpenVINO archive names.

  • OpenVINO GenAI:

    • StreamerBase::put(int64_t token)
    • The Bool value for Callback streamer is no longer accepted. It must now return one of three values of StreamingStatus enum.
    • ChunkStreamerBase is deprecated. Use StreamerBase instead.
  • NNCF create_compressed_model() method is now deprecated. nncf.quantize() method is recommended for Quantization-Aware Training of PyTorch and TensorFlow models.
  • Deprecated OpenVINO Model Server (OVMS) benchmark client in C++ using TensorFlow Serving API.
• Deprecated and to be removed in the future:
  • Python 3.9 is now deprecated and will be unavailable after OpenVINO version 2025.4.
  • openvino.Type.undefined is now deprecated and will be removed with version 2026.0. openvino.Type.dynamic should be used instead.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure will still be available until 2026, when the change will be finalized. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries will be removed from Docker images in 2026.
  • The openvino namespace of the OpenVINO Python API has been redesigned, removing the nested openvino.runtime module. The old namespace is now considered deprecated and will be discontinued in 2026.0. A new namespace structure is available for immediate migration. Details will be provided through warnings and documentation.
  • Starting with the next release, manylinux2014 will be upgraded to manylinux_2_28. This aligns with modern toolchain requirements but also means that CentOS 7 will no longer be supported due to glibc incompatibility.
  • With the release of Node.js v22, updated Node.js bindings are now available and compatible with the latest LTS version. These bindings do not support CentOS 7, as they rely on newer system libraries unavailable on legacy systems.

You can find OpenVINO™ toolkit 2025.3 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2025.3.0
• OpenVINO™ for Python: pip install openvino==2025.3.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @mahdi-jfri @11happy @arunthakur009 @Vladislav-Denisov @madhurthareja @mohiuddin-khan-shiam @Hmm-1224 @kuanxian1 @johnrhimawan @kinnam888

Release documentation is available here: https://docs.openvino.ai/2025 Release Notes are available here: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported on CPUs & GPUs: Phi-4, Mistral-7B-Instruct-v0.3, SD-XL Inpainting 0.1, Stable Diffusion 3.5 Large Turbo, Phi-4-reasoning, Qwen3, and Qwen2.5-VL-3B-Instruct. Mistral 7B Instruct v0.3 is also supported on NPUs.​
  • Preview: OpenVINO ™ GenAI introduces a text-to-speech pipeline for the SpeechT5 TTS model, while the new RAG backend offers developers a simplified API that delivers reduced memory usage and improved performance.​
  • Preview: OpenVINO™ GenAI offers a GGUF Reader for seamless integration of llama.cpp based LLMs, with Python and C++ pipelines that load GGUF models, build OpenVINO graphs, and run GPU inference on-the-fly. Validated for popular models: DeepSeek-R1-Distill-Qwen (1.5B, 7B), Qwen2.5 Instruct (1.5B, 3B, 7B) & llama-3.2 Instruct (1B, 3B, 8B).

• Broader LLM model support and more model compression techniques

  • Further optimization of LoRA adapters in OpenVINO GenAI for improved LLM, VLM, and text-to-image model performance on built-in GPUs. Developers can use LoRA adapters to quickly customize models for specialized tasks.​
  • KV cache compression for CPUs is enabled by default for INT8, providing a reduced memory footprint while maintaining accuracy compared to FP16. Additionally, it delivers substantial memory savings for LLMs with INT4 support compared to INT8.​
  • Optimizations for Intel® Core™ Ultra Processor Series 2 built-in GPUs and Intel® Arc™ B Series Graphics with the Intel® XMX systolic platform to enhance the performance of VLM models and hybrid quantized image generation models, as well as improve first-token latency for LLMs through dynamic quantization.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Enhanced Linux* support with the latest GPU driver for built-in GPUs on Intel® Core™ Ultra Processor Series 2 (formerly codenamed Arrow Lake H).
  • OpenVINO™ Model Server now offers a streamlined C++ version for Windows and enables improved performance for long-context models through prefix caching, and a smaller Windows package that eliminates the Python dependency. Support for Hugging Face models is now included.​
  • Support for INT4 data-free weights compression for ONNX models implemented in the Neural Network Compression Framework (NNCF)​.
  • NPU support for FP16-NF4 precision on Intel® Core™ 200V Series processors for models with up to 8B parameters is enabled through symmetrical and channel-wise quantization, improving accuracy while maintaining performance efficiency.

Support Change and Deprecation Notices

• Discontinued in 2025:

  • Runtime components:

    • The OpenVINO property of Affinity API is no longer available. It has been replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
    • The openvino-nightly PyPI module has been discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.The openvino-nightly PyPI module has been discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.

  • Tools:

    • The OpenVINO™ Development Tools package (pip install openvino-dev) is no longer available for OpenVINO releases in 2025.
    • Model Optimizer is no longer available. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
    • Intel® Streaming SIMD Extensions (Intel® SSE) are currently not enabled in the binary package by default. They are still supported in the source code form.
    • Legacy prefixes: l_, w_, and m_ have been removed from OpenVINO archive names.

  • OpenVINO GenAI:

    • StreamerBase::put(int64_t token)
    • The Bool value for Callback streamer is no longer accepted. It must now return one of three values of StreamingStatus enum.
    • ChunkStreamerBase is deprecated. Use StreamerBase instead.
  • NNCF create_compressed_model() method is now deprecated. nncf.quantize() method is recommended for Quantization-Aware Training of PyTorch and TensorFlow models.
  • OpenVINO Model Server (OVMS) benchmark client in C++ using TensorFlow Serving API.
• Deprecated and to be removed in the future:
  • Python 3.9 is now deprecated and will be unavailable after OpenVINO version 2025.4.
  • openvino.Type.undefined is now deprecated and will be removed with version 2026.0. openvino.Type.dynamic should be used instead.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure will still be available until 2026, when the change will be finalized. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries will be removed from Docker images in 2026.
  • Ubuntu 20.04 support will be deprecated in future OpenVINO releases due to the end of standard support.
  • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • MacOS x86 is no longer recommended for use due to the discontinuation of validation. Full support will be removed later in 2025.
  • The openvino namespace of the OpenVINO Python API has been redesigned, removing the nested openvino.runtime module. The old namespace is now considered deprecated and will be discontinued in 2026.0.

You can find OpenVINO™ toolkit 2025.2 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2025.2.0
• OpenVINO™ for Python: pip install openvino==2025.2.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @11happy @rahulchaphalkar @sanleo-wq @ashwins990 @NingLi670 @mohame54 @chiruu12 @SuperChamp234 @ChrisAB @kimgeonsu @code-dev05 @Mohamed-Ashraf273 @arunthakur009 @Captain-MUDIT @Simonwzm @Hmm-1224 @srinjoydutta03 @hridaya14 @victorgearhead @Huanli-Gong @Imokutmfon

Release documentation is available here: https://docs.openvino.ai/2025 Release Notes are available here: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported: Phi-4 Mini, Jina CLIP v1, and Bce Embedding Base v1.
  • OpenVINO™ Model Server now supports VLM models, including Qwen2-VL, Phi-3.5-Vision, and InternVL2.
  • OpenVINO GenAI now includes image-to-image and inpainting features for transformer-based pipelines, such as Flux.1 and Stable Diffusion 3 models, enhancing their ability to generate more realistic content.
  • Preview: AI Playground now utilizes the OpenVINO Gen AI backend to enable highly optimized inferencing performance on AI PCs.

• Broader LLM model support and more model compression techniques

  • Reduced binary size through optimization of the CPU plugin and removal of the GEMM kernel.
  • Optimization of new kernels for the GPU plugin significantly boosts the performance of Long Short-Term Memory (LSTM) models, used in many applications, including speech recognition, language modeling, and time series forecasting.
  • Preview: Token Eviction implemented in OpenVINO GenAI to reduce the memory consumption of KV Cache by eliminating unimportant tokens. This current Token Eviction implementation is beneficial for tasks where a long sequence is generated, such as chatbots and code generation.
  • NPU acceleration for text generation is now enabled in OpenVINO™ Runtime and OpenVINO™ Model Server to support the power-efficient deployment of VLM models on NPUs for AI PC use cases with low concurrency.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Support for the latest Intel® Core™ processors (Series 2, formerly codenamed Bartlett Lake), Intel® Core™ 3 Processor N-series and Intel® Processor N-series (formerly codenamed Twin Lake) on Windows.
  • Additional LLM performance optimizations on Intel® Core™ Ultra 200H series processors for improved 2nd token latency on Windows and Linux.
  • Enhanced performance and efficient resource utilization with the implementation of Paged Attention and Continuous Batching by default in the GPU plugin.
  • Preview: The new OpenVINO backend for Executorch will enable accelerated inference and improved performance on Intel hardware, including CPUs, GPUs, and NPUs.

Support Change and Deprecation Notices

• Discontinued in 2025:

  • Runtime components:

    • The OpenVINO property of Affinity API is no longer available. It has been replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).

  • Tools:

    • The OpenVINO™ Development Tools package (pip install openvino-dev) is no longer available for OpenVINO releases in 2025.
    • Model Optimizer is no longer available. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
    • Intel® Streaming SIMD Extensions (Intel® SSE) are currently not enabled in the binary package by default. They are still supported in the source code form.
    • Legacy prefixes: l_, w_, and m_ have been removed from OpenVINO archive names.

  • OpenVINO GenAI:

    • StreamerBase::put(int64_t token)
    • The Bool value for Callback streamer is no longer accepted. It must now return one of three values of StreamingStatus enum.
    • ChunkStreamerBase is deprecated. Use StreamerBase instead.
  • NNCF create_compressed_model() method is now deprecated. nncf.quantize() method is recommended for Quantization-Aware Training of PyTorch and TensorFlow models.
  • OpenVINO Model Server (OVMS) benchmark client in C++ using TensorFlow Serving API.
• Deprecated and to be removed in the future:
  • openvino.Type.undefined is now deprecated and will be removed with version 2026.0. openvino.Type.dynamic should be used instead.
  • APT & YUM Repositories Restructure: Starting with release 2025.1, users can switch to the new repository structure for APT and YUM, which no longer uses year-based subdirectories (like “2025”). The old (legacy) structure will still be available until 2026, when the change will be finalized. Detailed instructions are available on the relevant documentation pages:
    • Installation guide - yum
    • Installation guide - apt
  • OpenCV binaries will be removed from Docker images in 2026.
  • Ubuntu 20.04 support will be deprecated in future OpenVINO releases due to the end of standard support.
  • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • MacOS x86 is no longer recommended for use due to the discontinuation of validation. Full support will be removed later in 2025.
  • The openvino namespace of the OpenVINO Python API has been redesigned, removing the nested openvino.runtime module. The old namespace is now considered deprecated and will be discontinued in 2026.0.

You can find OpenVINO™ toolkit 2025.1 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2025.1.0
• OpenVINO™ for Python: pip install openvino==2025.1.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @11happy @arkhamHack @AsVoider @chiruu12 @darshil929 @geeky33 @itsbharatj @jpy794 @kuanxian1 @Mohamed-Ashraf273 @nikolasavic3 @oToToT @SaifMohammed22 @srinjoydutta03

Release documentation is available here: https://docs.openvino.ai/2025 Release Notes are available here: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More GenAI coverage and framework integrations to minimize code changes

  • New models supported: Qwen 2.5, Deepseek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-7B, and DeepSeek-R1-Distill-Qwen-1.5B, FLUX.1 Schnell and FLUX.1 Dev
  • Whisper Model: Improved performance on CPUs, built-in GPUs, and discrete GPUs with GenAI API.
  • Preview: Introducing NPU support for torch.compile, giving developers the ability to use the OpenVINO backend to run the PyTorch API on NPUs. 300+ deep learning models enabled from the TorchVision, Timm, and TorchBench repositories..

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Preview: Addition of Prompt Lookup to GenAI API improves 2nd token latency for LLMs by effectively utilizing predefined prompts that match the intended use case.
  • Preview: The GenAI API now offers image-to-image inpainting functionality. This feature enables models to generate realistic content by inpainting specified modifications and seamlessly integrating them with the original image.
  • Asymmetric KV Cache compression is now enabled for INT8 on CPUs, resulting in lower memory consumption and improved 2nd token latency, especially when dealing with long prompts that require significant memory. The option should be explicitly specified by the user.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Support for the latest Intel® Core™ Ultra 200H series processors (formerly codenamed Arrow Lake-H)
  • Integration of the OpenVINO ™ backend with the Triton Inference Server allows developers to utilize the Triton server for enhanced model serving performance when deploying on Intel CPUs.
  • Preview: A new OpenVINO ™ backend integration allows developers to leverage OpenVINO performance optimizations directly within Keras 3 workflows for faster AI inference on CPUs, built-in GPUs, discrete GPUs, and NPUs. This feature is available with the latest Keras 3.8 release.
  • The OpenVINO Model Server now supports native Windows Server deployments, allowing developers to leverage better performance by eliminating container overhead and simplifying GPU deployment.

Support Change and Deprecation Notices

• Now deprecated:
  • Legacy prefixes l_, w_, and m_ have been removed from OpenVINO archive names.
  • The runtime namespace for Python API has been marked as deprecated and designated to be removed for 2026.0. The new namespace structure has been delivered, and migration is possible immediately. Details will be communicated through warnings and via documentation.
  • NNCF create_compressed_model() method is deprecated. nncf.quantize() method is now recommended for Quantization-Aware Training of PyTorch and TensorFlow models.

You can find OpenVINO™ toolkit 2025.0 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2025.0.0
• OpenVINO™ for Python: pip install openvino==2025.0.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @0xfedcafe @11happy @cocoshe @emir05051 @geeky33 @h6197627 @hub-bla @Manideep-Kanna @nashez @nashez @shivam5522 @sumhaj @vatsalashanubhag @xyz-harshal

Release documentation is available here: https://docs.openvino.ai/2025 Release Notes are available here: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• OpenVINO 2024.6 release includes updates for enhanced stability and improved LLM performance.

• Introduced support for Intel® Arc™ B-Series Graphics (formerly known as Battlemage).

• Implemented optimizations to improve the inference time and LLM performance on NPUs.

• Improved LLM performance with GenAI API optimizations and bug fixes.

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The macOS x86_64 debug bins will no longer be provided with the OpenVINO toolkit, starting with OpenVINO 2024.5.
  • Python 3.8 is no longer supported, starting with OpenVINO 2024.5.
    • As MxNet doesn’t support Python version higher than 3.8, according to the MxNet PyPI project, it is no longer supported by OpenVINO, either.
  • Discrete Keem Bay support is no longer supported, starting with OpenVINO 2024.5.
  • Support for discrete devices (formerly codenamed Raptor Lake) is no longer available for NPU.

You can find OpenVINO™ toolkit 2024.6 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.6.0
• OpenVINO™ for Python: pip install openvino==2024.6.0

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://docs.openvino.ai/2024/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More Gen AI coverage and framework integrations to minimize code changes

  • New models supported: Llama 3.2 (1B & 3B), Gemma 2 (2B & 9B), and YOLO11.
  • LLM support on NPU: Llama 3 8B, Llama 2 7B, Mistral-v0.2-7B, Qwen2-7B-Instruct and Phi-3
  • Noteworthy notebooks added: Sam2, Llama3.2, Llama3.2 - Vision, Wav2Lip, Whisper, and Llava.
  • Preview: support for Flax, a high-performance Python neural network library based on JAX. Its modular design allows for easy customization and accelerated inference on GPUs.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Optimizations for built-in GPUs on Intel® Core™ Ultra Processors (Series 1) and Intel® Arc™ Graphics include KV Cache compression for memory reduction along with improved usability, and model load time optimizations to improve first token latency for LLMs..
  • Dynamic quantization was enabled to improve first token latency for LLMs on built-in Intel® GPUs without impacting accuracy on Intel® Core™ Ultra Processors (Series 1). Second token latency will also improve for large batch inference.
  • A new method to generate synthetic text data is implemented in the Neural Network Compression Framework (NNCF). This will allow LLMs to be compressed more accurately using data-aware methods without datasets. Coming soon: This feature will soon be accessible via Optimum Intel on Hugging Face.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Support for Intel® Xeon® 6 Processors with P-cores (formerly codenamed Granite Rapids) and Intel® Core™ Ultra 200V series processors (formerly codenamed Arrow Lake-S).
  • Preview: GenAI API enables multimodal AI deployment with support for multimodal pipelines for improved contextual awareness, transcription pipelines for easy audio-to-text conversions, and image generation pipelines for streamlined text-to-visual conversions..
  • Speculative decoding feature added to the GenAI API for improved performance and efficient text generation using a small draft model that is periodically corrected by the full-size model.
  • Preview: LoRA adapters are now supported in the GenAI API for developers to quickly and efficiently customize image and text generation models for specialized tasks.
  • The GenAI API now also supports LLMs on NPU allowing developers to specify NPU as the target device, specifically for WhisperPipeline (for whisper-base, whisper-medium, and whisper-small) and LLMPipeline (for Llama 3 8B, Llama 2 7B, Mistral-v0.2-7B, Qwen2-7B-Instruct and Phi-3 Mini-instruct). Use driver version 32.0.100.3104 or later for best performance.

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The macOS x86_64 debug bins will no longer be provided with the OpenVINO toolkit, starting with OpenVINO 2024.5.
  • Python 3.8 is no longer supported, starting with OpenVINO 2024.5.
    • As MxNet doesn’t support Python version higher than 3.8, according to the MxNet PyPI project, it is no longer supported by OpenVINO, either.
  • Discrete Keem Bay support is no longer supported, starting with OpenVINO 2024.5.
  • Support for discrete devices (formerly codenamed Raptor Lake) is no longer available for NPU.

You can find OpenVINO™ toolkit 2024.5 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.5.0
• OpenVINO™ for Python: pip install openvino==2024.5.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @aku221b @halm-zenger @hibahassan1 @hub-bla @jagadeeshmadinni @nashez @tianyiSKY1 @tiebreaker4869

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://docs.openvino.ai/2024/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More Gen AI coverage and framework integrations to minimize code changes

  • Support for GLM-4-9B Chat, MiniCPM-1B, Llama 3 and 3.1, Phi-3-Mini, Phi-3-Medium and YOLOX-s models.
  • Noteworthy notebooks added: Florence-2, NuExtract-tiny Structure Extraction, Flux.1 Image Generation, PixArt-α: Photorealistic Text-to-Image Synthesis, and Phi-3-Vision Visual Language Assistant.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • OpenVINO™ runtime optimized for Intel® Xe Matrix Extensions (Intel® XMX) systolic arrays on built-in GPUs for efficient matrix multiplication resulting in significant LLM performance boost with improved 1st and 2nd token latency, as well as a smaller memory footprint on Intel® Core™ Ultra Processors (Series 2).
  • Memory sharing enabled for NPUs on Intel® Core™ Ultra Processors (Series 2) for efficient pipeline integration without memory copy overhead.
  • Addition of the PagedAttention feature for discrete GPUs* enables a significant boost in throughput for parallel inferencing when serving LLMs on Intel® Arc™ Graphics or Intel® Data Center GPU Flex Series.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Support for Intel® Core Ultra Processors Series 2 (formerly codenamed Lunar Lake) on Windows.
  • OpenVINO™ Model Server now comes with production-quality support for OpenAI-compatible API which enables significantly higher throughput for parallel inferencing on Intel® Xeon® processors when serving LLMs to many concurrent users.
  • Improved performance and memory consumption with prefix caching, KV cache compression, and other optimizations for serving LLMs using OpenVINO™ Model Server.
  • Support for Python 3.12.
  • Support for Red Hat Enterprise Linux (RHEL) version 9

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The macOS x86_64 debug bins will no longer be provided with the OpenVINO toolkit, starting with OpenVINO 2024.5.
  • Python 3.8 is now considered deprecated, and it will not be available beyond the 2024.4 OpenVINO version.
  • dKMB support is now considered deprecated and will be fully removed with OpenVINO 2024.5
  • Intel® Streaming SIMD Extensions (Intel® SSE) will be supported in source code form, but not enabled in the binary package by default, starting with OpenVINO 2025.0
  • The openvino-nightly PyPI module will soon be discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • A number of notebooks have been deprecated. For an up-to-date listing of available notebooks, refer to the OpenVINO™ Notebook index (openvinotoolkit.github.io).

You can find OpenVINO™ toolkit 2024.4 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.4.0
• OpenVINO™ for Python: pip install openvino==2024.4.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @hub-bla @awayzjj @jvr0123 @Pey-crypto @nashez @qxprakash

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://docs.openvino.ai/2024/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More Gen AI coverage and framework integrations to minimize code changes

  • OpenVINO pre-optimized models are now available in Hugging Face making it easier for developers to get started with these models.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Significant improvement in LLM performance on Intel discrete GPUs with the addition of Multi-Head Attention (MHA) and OneDNN enhancements.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Improved CPU performance when serving LLMs with the inclusion of vLLM and continuous batching in the OpenVINO Model Server (OVMS). vLLM is an easy-to-use open-source library that supports efficient LLM inferencing and model serving.
  • Ubuntu 24.04 long-term support (LTS), 64-bit (Kernel 6.8+) (preview support)

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • A number of notebooks have been deprecated. For an up-to-date listing of available notebooks, refer to the OpenVINO™ Notebook index (openvinotoolkit.github.io).

You can find OpenVINO™ toolkit 2024.3 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.3.0
• OpenVINO™ for Python: pip install openvino==2024.3.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @rghvsh @PRATHAM-SPS @duydl @awayzjj @jvr0123 @inbasperu @DannyVlasenko @amkarn258 @kcin96 @Vladislav-Denisov

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://docs.openvino.ai/2024/about-openvino/release-notes-openvino.html

Summary of major features and improvements

• More Gen AI coverage and framework integrations to minimize code changes

  • Llama 3 optimizations for CPUs, built-in GPUs, and discrete GPUs for improved performance and efficient memory usage.
  • Support for Phi-3-mini, a family of AI models that leverages the power of small language models for faster, more accurate and cost-effective text processing.
  • Python Custom Operation is now enabled in OpenVINO making it easier for Python developers to code their custom operations instead of using C++ custom operations (also supported). Python Custom Operation empowers users to implement their own specialized operations into any model.
  • Notebooks expansion to ensure better coverage for new models. Noteworthy notebooks added: DynamiCrafter, YOLOv10, Chatbot notebook with Phi-3, and QWEN2.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • GPTQ method for 4-bit weight compression added to NNCF for more efficient inference and improved performance of compressed LLMs.
  • Significant LLM performance improvements and reduced latency for both built-in GPUs and discrete GPUs.
  • Significant improvement in 2nd token latency and memory footprint of FP16 weight LLMs on AVX2 (13th Gen Intel® Core™ processors) and AVX512 (3rd Gen Intel® Xeon® Scalable Processors) based CPU platforms, particularly for small batch sizes.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Model Serving Enhancements:
    • Preview: OpenVINO Model Server (OVMS) now supports OpenAI-compatible API along with Continuous Batching and PagedAttention, enabling significantly higher throughput for parallel inferencing, especially on Intel® Xeon® processors, when serving LLMs to many concurrent users.
    • OpenVINO backend for Triton Server now supports built-in GPUs and discrete GPUs, in addition to dynamic shapes support.
    • Integration of TorchServe through torch.compile OpenVINO backend for easy model deployment, provisioning to multiple instances, model versioning, and maintenance.
  • Preview: addition of the Generate API, a simplified API for text generation using large language models with only a few lines of code. The API is available through the newly launched OpenVINO GenAI package.
  • Support for Intel Atom® Processor X Series. For more details, see System Requirements.
  • Preview: Support for Intel® Xeon® 6 processor.

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • A number of notebooks have been deprecated. For an up-to-date listing of available notebooks, refer to the OpenVINO™ Notebook index (openvinotoolkit.github.io).

You can find OpenVINO™ toolkit 2024.2 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.2.0
• OpenVINO™ for Python: pip install openvino==2024.2.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @siddhant-0707 @adismort14 @LucaTamSapienza @hongbo-wei @awayzjj @qxprakash @keyonjie @Huanli-Gong @hegdeadithyak @inbasperu @Thodoris1999 @hongbo-wei @himanshugupta11002 @tranchung163 @SANJITH-KUMAR-20 @anzr299 @Vladislav-Denisov

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2024-2.html

Summary of major features and improvements

• More Generative AI coverage and framework integrations to minimize code changes.

  • Mixtral and URLNet models optimized for performance improvements on Intel® Xeon® processors.
  • Stable Diffusion 1.5, ChatGLM3-6B, and Qwen-7B models optimized for improved inference speed on Intel® Core™ Ultra processors with integrated GPU.
  • Support for Falcon-7B-Instruct, a GenAI Large Language Model (LLM) ready-to-use chat/instruct model with superior performance metrics.
  • New Jupyter Notebooks added: YOLO V9, YOLO V8 Oriented Bounding Boxes Detection (OOB), Stable Diffusion in Keras, MobileCLIP, RMBG-v1.4 Background Removal, Magika, TripoSR, AnimateAnyone, LLaVA-Next, and RAG system with OpenVINO and LangChain.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • LLM compilation time reduced through additional optimizations with compressed embedding. Improved 1st token performance of LLMs on 4th and 5th generations of Intel® Xeon® processors with Intel® Advanced Matrix Extensions (Intel® AMX).
  • Better LLM compression and improved performance with oneDNN, INT4, and INT8 support for Intel® Arc™ GPUs.
  • Significant memory reduction for select smaller GenAI models on Intel® Core™ Ultra processors with integrated GPU.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • The preview NPU plugin for Intel® Core™ Ultra processors is now available in the OpenVINO open-source GitHub repository, in addition to the main OpenVINO package on PyPI.
  • The JavaScript API is now more easily accessible through the npm repository, enabling JavaScript developers’ seamless access to the OpenVINO API.
  • FP16 inference on ARM processors now enabled for the Convolutional Neural Network (CNN) by default.

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using Discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA). Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.

You can find OpenVINO™ toolkit 2024.1 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.1.0
• OpenVINO™ for Python: pip install openvino==2024.1.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @LucaTamSapienza @AsakusaRinne @awayzjj @MonalSD @siddhant-0707 @qxprakash @FredBill1 @Pranshu-S @vshampor @PRATHAM-SPS @inbasperu @linzs148 @chux0519 @ccinv @Vishwa44 @rghvsh @Aryan8912 @BHbean @Vladislav-Denisov @MeeCreeps @YaritaiKoto @Godwin-T @mory91 @Bepitic @akiseakusa @kuanxian1 @himanshugupta11002 @mengbingrock

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2024-1.html

Summary of major features and improvements

• More Generative AI coverage and framework integrations to minimize code changes.

  • Improved out-of-the-box experience for TensorFlow* sentence encoding models through the installation of OpenVINO™ toolkit Tokenizers.
  • OpenVINO™ toolkit now supports Mixture of Experts (MoE), a new architecture that helps process more efficient generative models through the pipeline.
  • JavaScript developers now have seamless access to OpenVINO API. This new binding enables a smooth integration with JavaScript API.
  • New and noteworthy models validated: Mistral, StableLM-tuned-alpha-3b, and StableLM-Epoch-3B.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Improved quality on INT4 weight compression for LLMs by adding the popular technique, Activation-aware Weight Quantization, to the Neural Network Compression Framework (NNCF). This addition reduces memory requirements and helps speed up token generation.
  • Experience enhanced LLM performance on Intel® CPUs, with internal memory state enhancement, and INT8 precision for KV-cache. Specifically tailored for multi-query LLMs like ChatGLM. ​
  • Easier optimization and conversion of Hugging Face models – compress LLM models to INT8 and INT4 with Hugging Face Optimum command line interface and export models to OpenVINO format. Note this is part of Optimum-Intel which needs to be installed separately.
  • The OpenVINO™ 2024.0 release makes it easier for developers, by integrating more OpenVINO™ features with the Hugging Face* ecosystem. Store quantization configurations for popular models directly in Hugging Face to compress models into INT4 format while preserving accuracy and performance.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • A preview plugin architecture of the integrated Neural Processor Unit (NPU) as part of Intel® Core™ Ultra processor is now included in the main OpenVINO™ package on PyPI.
  • Improved performance on ARM* by enabling the ARM threading library. In addition, we now support multi-core ARM platforms and enabled FP16 precision by default on MacOS*.
  • Improved performance on ARM platforms using throughput hint, which increases efficiency in utilization of CPU cores and memory bandwidth.​
  • New and improved LLM serving samples from OpenVINO™ Model Server for multi-batch inputs and Retrieval Augmented Generation (RAG).

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using Discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA). Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • a git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using OpenVINO Model Converter (API call: OVC) instead. Follow the model conversion transition guide for more details.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • Reshaping a model in runtime based on the incoming requests (auto shape and auto batch size) is deprecated and will be removed in the future. Using OpenVINO’s dynamic shape models is recommended instead.

You can find OpenVINO™ toolkit 2024.0 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.0.0
• OpenVINO™ for Python: pip install openvino==2024.0.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @rghvsh @YaritaiKoto @Abdulrahman-Adel @jvr0123 @sami0i @guy-tamir @rupeshs @karanjakhar @abhinav231-valisetti @rajatkrishna @lukazlim @siddhant-0707 @tiger100256-hu

Release documentation is available here: https://docs.openvino.ai/2024 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2024-0.html

Summary of major features and improvements

• More Generative AI coverage and framework integrations to minimize code changes.

  • Introducing OpenVINO Gen AI repository on GitHub that demonstrates native C and C++ pipeline samples for Large Language Models (LLMs). String tensors are now supported as inputs and tokenizers natively to reduce overhead and ease production. ​
  • New and noteworthy models validated; Mistral, Zephyr, Qwen, ChatGLM3, and Baichuan.
  • New Jupyter Notebooks for Latent Consistency Models (LCM) and Distil-Whisper. Updated LLM Chatbot notebook to include LangChain, Neural Chat, TinyLlama, ChatGLM3, Qwen, Notus, and Youri models.
  • Torch.compile is now fully integrated with OpenVINO, which now includes a hardware 'options' parameter allowing for seamless inference hardware selection by leveraging the plugin architecture in OpenVINO. ​

• Broader Large Language Model (LLM) support and more model compression techniques.

  • As part of the Neural Network Compression Framework (NNCF), INT4 weight compression model formats are now fully supported on Intel® Xeon® CPUs in addition to Intel® Core™ and iGPU, adding more performance, lower memory usage, and accuracy opportunity when using LLMs.​
  • Improved performance of transformer-based LLM on CPU and GPU using stateful model technique to increase memory efficiency where internal states are shared among multiple iterations of inference. ​
  • Easier optimization and conversion of Hugging Face models – compress LLM models to INT8 and INT4 with Hugging Face Optimum command line interface and export models to OpenVINO format. Note this is part of Optimum-Intel which needs to be installed separately.
  • Tokenizer and TorchVision transform support is now available in the OpenVINO runtime (via new API) requiring less preprocessing code and enhancing performance by automatically handling this model setup. More details on Tokenizers support in the Ecosystem section.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Full support for 5th Gen Intel® Xeon® Scalable processors (codename Emerald Rapids)
  • Further optimized performance on Intel® Core™ Ultra (codename Meteor Lake) CPU with latency hint, by leveraging both P-core and E-cores.​
  • Improved performance on ARM platforms using throughput hint, which increases efficiency in utilization of CPU cores and memory bandwidth.​
  • Preview JavaScript API to enable node JS development to access JavaScript binding via source code.​ See details below.
  • Improved model serving of LLMs through OpenVINO Model Server. This not only enables LLM serving over KServe v2 gRPC and REST APIs for more flexibility but also improves throughput by running processing like tokenization on the server side.​ More details in the Ecosystem section.

Support Change and Deprecation Notices

• The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated and will be removed from installation options and distribution channels beginning with the 2025.0 release. For more details, refer to the OpenVINO Legacy Features and Components page.
• Ubuntu 18.04 support is discontinued in the 2023.3 LTS release. The recommended version of Ubuntu is 22.04.
• Starting with 2023.3 OpenVINO longer supports Python 3.7 due to the Python community discontinuing support. Update to a newer version (currently 3.8-3.11) to avoid interruptions.
• All ONNX Frontend legacy API (known as ONNX_IMPORTER_API) will no longer be available in the 2024.0 release. 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API will be discontinued in the 2024.0 release.
• Tools:
  • Deployment Manager is deprecated and will be supported for two years according to the LTS policy. Visit the selector tool to see package distribution options or the deployment guide documentation.
  • Accuracy Checker is deprecated and will be discontinued with 2024.0.
  • Post-Training Optimization Tool (POT) has been deprecated and the 2023.3 LTS is the last release that supports the tool. Developers are encouraged to use the Neural Network Compression Framework (NNCF) for this feature.
  • Model Optimizer is deprecated and will be fully supported until the 2025.0 release. We encourage developers to perform model conversion through OpenVINO Model Converter (API call: OVC). Follow the model conversion transition guide for more details.
  • Deprecated support for a git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
  • Support for Apache MXNet, Caffe, and Kaldi model formats is deprecated and will be discontinued with the 2024.0 release.
• Runtime:
  • Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future release. We encourage developers to use the Neural Processing Unit (NPU) for low-powered systems like Intel® CoreTM Ultra or 14th generation and beyond.
  • OpenVINO C++/C/Python 1.0 APIs are deprecated and will be discontinued in the 2024.0 release. Please use API 2.0 in your applications going forward to avoid disruption. OpenVINO property Affinity API will be deprecated from 2024.0 and will be discontinued in 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).

You can find OpenVINO™ toolkit 2023.3 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2023.3.0
• OpenVINO™ for Python: pip install openvino==2023.3.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @rghvsh, @YaritaiKoto, @siddhant-0707, @sydarb, @kk271kg, @ahmadchalhoub, @ma7555, @Bhaskar365

Release documentation is available here: https://docs.openvino.ai/2023.3 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-lts/2023-3.html

Summary of major features and improvements

• More Generative AI coverage and framework integrations to minimize code changes.

  • Expanded model support for direct PyTorch model conversion – automatically convert additional models directly from PyTorch or execute via torch.compile with OpenVINO as the backend.
  • New and noteworthy models supported – we have enabled models used for chatbots, instruction following, code generation, and many more, including prominent models like LLaVA, chatGLM, Bark (text to audio), and LCM (Latent Consistency Models, an optimized version of Stable Diffusion).
  • Easier optimization and conversion of Hugging Face models – compress LLM models to Int8 with the Hugging Face Optimum command line interface and export models to the OpenVINO IR format.
  • OpenVINO is now available on Conan – a package manager which enables more seamless package management for large-scale projects for C and  C++ developers.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Accelerate inference for LLM models on Intel® Core™ CPU and iGPU with the use of Int8 model weight compression.
  • Expanded model support for dynamic shapes for improved performance on GPU.
  • Preview support for Int4 model format is now included. Int4 optimized model weights are now available to try on Intel® Core™ CPU and iGPU, to accelerate models like Llama 2 and chatGLM2.
  • The following Int4 model compression formats are supported for inference in runtime:
    • Generative Pre-training Transformer Quantization (GPTQ); with GPTQ-compressed models, you can access them through the Hugging Face repositories.
    • Native Int4 compression through Neural Network Compression Framework (NNCF).

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • In 2023.1 we announced full support for ARM architecture, now we have improved performance by enabling FP16 model formats for LLMs and integrating additional acceleration libraries to improve latency.

Support Change and Deprecation Notices

• The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated and will be removed from installation options and distribution channels with 2025.0. To learn more, refer to the OpenVINO Legacy Features and Components page. To ensure optimal performance, install the OpenVINO package (pip install openvino), which includes essential components such as OpenVINO Runtime, OpenVINO Converter, and Benchmark Tool.
• Tools:
  • Deployment Manager is deprecated and will be removed in the 2024.0 release.
  • Accuracy Checker is deprecated and will be discontinued with 2024.0.
  • Post-Training Optimization Tool (POT)  is deprecated and will be discontinued with 2024.0.
  • Model Optimizer is deprecated and will be fully supported up until the 2025.0 release. Model conversion to the OpenVINO IR format should be performed through OpenVINO Model Converter which is part of the PyPI package. Follow the Model Optimizer to OpenVINO Model Converter transition guide for smoother transition. Known limitations are TensorFlow model with TF1 Control flow and object detection models. These limitations relate to the gap in TensorFlow direct conversion capabilities which will be addressed in upcoming releases.
  • PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF).
• Runtime:
  • Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future release. We encourage developers to use the Neural Processing Unit (NPU) for low powered systems like Intel® Core™ Ultra or 14th generation and beyond.
  • OpenVINO C++/C/Python 1.0 APIs will be discontinued with 2024.0.
  • PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF).

You can find OpenVINO™ toolkit 2023.2 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2023.2.0
• OpenVINO™ for Python: pip install openvino==2023.2.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @siddhant-0707, @NsdHSO, @mahimairaja, @SANTHOSH-MAMIDISETTI, @rsato10, @PRATHAM-SPS

Release documentation is available here: https://docs.openvino.ai/2023.2 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-2.html

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.2.0.dev20230922 pre-release version here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c "conda-forge/label/openvino_dev" openvino=2023.2.0.dev20230922
• OpenVINO™ for Python: pip install --pre openvino or pip install openvino==2023.2.0.dev20230922

Release notes are available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

What's Changed

• CPU runtime:
  • Optimized Yolov8n and YoloV8s models on BF16/FP32.
  • Optimized Falcon model on 4th Generation Intel® Xeon® Scalable Processors.
• GPU runtime:
  • int8 weight compression further improves LLM performance. PR #19548
  • Optimization for gemm & fc in iGPU. PR #19780
• TensorFlow FE:
  • Added support for Selu operation. PR #19528
  • Added support for XlaConvV2 operation. PR #19466
  • Added support for TensorListLength and TensorListResize operations. PR #19390
• PyTorch FE:
  • New operations supported
    • aten::minimum aten::maximum. PR #19996
    • aten::broadcast_tensors. PR #19994
    • added support aten::logical_and, aten::logical_or, aten::logical_not, aten::logical_xor. PR #19981
    • aten::scatter_reduce and extend aten::scatter. PR #19980
    • prim::TupleIndex operation. PR #19978
    • mixed precision in aten::min/max. PR #19936
    • aten::tile op PR #19645
    • aten::one_hot PR #19779
    • PReLU. PR #19515
    • aten::swapaxes. PR #19483
    • non-boolean inputs for or and and operations. PR #19268
• Torchvision NMS can accept negative scores. PR #19826

New openvino_notebooks:

• Visual Question Answering and Image Captioning using BLIP

Fixed GitHub issues

• Fixed #19784 “[Bug]: Cannot install libprotobuf-dev along with libopenvino-2023.0.2 on Ubuntu 22.04” with PR #19788
• Fixed #19617 “Add a clear error message when creating an empty Constant” with PR #19674
• Fixed #19616 “Align openvino.compile_model and openvino.Core.compile_model functions” with PR #19778
• Fixed #19469 “[Feature Request]: Add SeLu activation in the OpenVino IR (TensorFlow Conversion)” with PR #19528
• Fixed #19019 “[Bug]: Low performance of the TF quantized model.” With PR #19735
• Fixed #19018 “[Feature Request]: Support aarch64 python wheel for Linux” with PR #19594
• Fixed #18831 “Question: openvino support for Nvidia Jetson Xavier ?” with PR #19594
• Fixed #18786 “OpenVINO Wheel does not install Debug libraries when CMAKE_BUILD_TYPE is Debug #18786” with PR #19197
• Fixed #18731 “[Bug] Wrong output shapes of MaxPool” with PR #18965
• Fixed #18091 “[Bug] 2023.0 Version crashes on Jetson Nano - L4T - Ubuntu 18.04” with PR #19717
• Fixed #7194 “Conan for simplifying dependency management” with PR #17580

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @siddhant-0707, @PRATHAM-SPS, @okhovan

Full Changelog: https://github.com/openvinotoolkit/openvino/compare/2023.1.0.dev20230811...2023.2.0.dev20230922

Summary of major features and improvements

• More Generative AI options with Hugging Face and improved PyTorch model support.
  • NEW: Your PyTorch solutions are now even further enhanced with OpenVINO. You’ve got more options and you no longer need to convert to ONNX for deployment. Developers can now use their API of choice - PyTorch or OpenVINO for added performance benefits. Additionally, users can automatically import and convert PyTorch models for quicker deployment. You can continue to make the most of OpenVINO tools for advanced model compression and deployment advantages, ensuring flexibility and a range of options.
  • torch.compile (preview) – OpenVINO is now available as a backend through PyTorch torch.compile, empowering developers to utilize OpenVINO toolkit through PyTorch APIs. This feature has also been integrated into the Automatic1111 Stable Diffusion Web UI, helping developers achieve accelerated performance for Stable Diffusion 1.5 and 2.1 on Intel CPUs and GPUs in both Native Linux and Windows OS platforms.
  • Optimum Intel – Hugging Face and Intel continue to enhance top generative AI models by optimizing execution, making your models run faster and more efficiently on both CPU and GPU. OpenVINO serves as a runtime for inferencing execution. New PyTorch auto import and conversion capabilities have been enabled, along with support for weights compression to achieve further performance gains.
• Broader LLM model support and more model compression techniques
  • Enhanced performance and accessibility for Generative AI: Runtime performance and memory usage have been significantly optimized, especially for Large Language models (LLMs). Models used for chatbots, instruction following, code generation, and many more, including prominent models like BLOOM, Dolly, Llama 2, GPT-J, GPTNeoX, ChatGLM, and Open-Llama have been enabled.
  • Improved LLMs on GPU – Model coverage for dynamic shapes support has been expanded, further helping the performance of generative AI workloads on both integrated and discrete GPUs. Furthermore, memory reuse and weight memory consumption for dynamic shapes have been improved.
  • Neural Network Compression Framework (NNCF) now includes an 8-bit weights compression method, making it easier to compress and optimize LLM models. SmoothQuant method has been added for more accurate and efficient post-training quantization for Transformer-based models.
• More portability and performance to run AI at the edge, in the cloud or locally.
  • NEW: Support for Intel(R) Core(TM) Ultra (codename Meteor Lake). This new generation of Intel CPUs is tailored to excel in AI workloads with a built-in inference accelerators.
  • Integration with MediaPipe – Developers now have direct access to this framework for building multipurpose AI pipelines. Easily integrate with OpenVINO Runtime and OpenVINO Model Server to enhance performance for faster AI model execution. You also benefit from seamless model management and version control, as well as custom logic integration with additional calculators and graphs for tailored AI solutions. Lastly, you can scale faster by delegating deployment to remote hosts via gRPC/REST interfaces for distributed processing.

Support Change and Deprecation Notices

• OpenVINO™ Development Tools package (pip install openvino-dev) is currently being deprecated and will be removed from installation options and distribution channels with 2025.0. For more info, see the documentation for Legacy Features.
• Tools:
  • Accuracy Checker is deprecated and will be discontinued with 2024.0.
  • Post-Training Optimization Tool (POT)  has been deprecated and will be discontinued with 2024.0.
• Runtime:
  • Intel® Gaussian & Neural Accelerator (Intel® GNA) is being deprecated, the GNA plugin will be discontinued with 2024.0.
  • OpenVINO C++/C/Python 1.0 APIs will be discontinued with 2024.0.
  • Python 3.7 will be discontinued with 2023.2 LTS release.

You can find OpenVINO™ toolkit 2023.1 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2023.1.0
• OpenVINO™ for Python: pip install openvino==2023.1.0

Release documentation is available here: https://docs.openvino.ai/2023.1 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-1.html

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.1.0.dev20230811 pre-release version here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c "conda-forge/label/openvino_dev" openvino=2023.1.0.dev20230811
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.1.0.dev20230811
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.1.0.dev20230811

Release notes are available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

What's Changed

• CPU runtime:
  • Enabled weights decompression support for Large Language models (LLMs). The implementation supports avx2 and avx512 HW targets for Intel® Core™ processors, improving performance in the latency mode (comparison: FP32 VS FP32+INT8 weights). For 4th Generation Intel® Xeon® Scalable Processors (formerly Sapphire Rapids) this INT8 decompression feature improves performance compared to pure BF16 inference. PRs: #18915, #19111
  • Reduced memory consumption of the ‘compile model’ stage by moving constant folding of Transpose nodes to the CPU Runtime side. PR: #18877
  • Set FP16 inference precision by default for non-convolution networks on ARM. Convolution networks will be executed in FP32. PRs: #19069, #19192, #19176
• GPU runtime: Added paddings for dynamic convolutions to improve performance for models like Stable-Diffusion v2.1, PR: #19001
• Python API:
  • Added the torchvision.transforms object to OpenVINO preprocessing. PR: #17934
  • All python tools related with OpenVINO are now available via single namespace, to improve user experience by better API readability. PR: #18157
• TensorFlow FE:
  • Added support for the TensorFlow 1 Checkpoint format. All native TensorFlow formats are now enabled.
  • Added support for 8 new operations:
    • MaxPoolWithArgmax #19085
    • UnravelIndex #19060
    • AdjustContrastv2 #19029
    • InvertPermutation #19020
    • CheckNumerics #19020
    • DivNoNan #19007
    • EnsureShape #19007
    • ShapeN #18913
• PyTorch FE:
  • Added support for 7 new operations. To know how to enjoy PyTorch models conversion follow Link
    • aten::concat #19101
    • aten::masked_scatter #19038
    • aten::linspace #18998
    • aten::view_as #18960
    • aten::std #18960
    • aten::outer #18903
    • aten::broadcast_to #18899

New openvino_notebooks

• 245-typo-detector English Typo Detection in sentences with OpenVINO™
• 247-code-language-id Identify the programming language used in an arbitrary code snippet
• 121-convert-to-openvino Learn OpenVINO model conversion API
• 244-named-entity-recognition Named entity recognition with OpenVINO™
• 246-depth-estimation-videpth Monocular Visual-Inertial Depth Estimation with OpenVINO™
• 248-stable-diffusion-xl Image generation with Stable Diffusion XL
• 249-oneformer-segmentation Universal segmentation with OneFormer

Fixed GitHub issues

• Fixed #18978 "Webassembly build fails" with PR #19005
• Fixed #18847 "Debugging OpenVINO Python GIL Error" with PR #18848
• Fixed #18465 "OpenVINO can't be built in an environment that has an 'ambient' oneDNN installation" with PR #18805

Acknowledgements Thanks for contributions from the OpenVINO developer community: @DmitriyValetov, @kai-waang

Full Changelog: 2023.1.0.dev20230728...2023.1.0.dev20230811

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.1.0.dev20230728 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• Install it via Conda: conda install -c "conda-forge/label/openvino_dev" openvino=2023.1.0.dev20230728
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.1.0.dev20230728
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.1.0.dev20230728

Release notes is available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.1.0.dev20230623 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.1.0.dev20230623
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.1.0.dev20230623

Release notes is available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

This release provides functional bug fixes and capability updates for 2023.0 that enable developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit with confidence.

Note: This is a standard release intended for developers that prefer the very latest version of OpenVINO. Standard releases will continue to be made available three to four times a year. Long Term Support (LTS) releases are also available. A new LTS version is released every year and is supported for 2 years (1 year of bug fixes, and 2 years for security patches). Visit Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy to get details on the latest LTS releases.

Major changes:

• OpenVINO GNA Plugin:
  • Fixes the issue when GNA device would not work on Gemini Lake (GLK) platforms
  • Fixes the problem with memory leak during HLK test
• OpenVINO CPU Plugin:
  • Fixes the issues occurred in Multi-Threading 2.0 getting CPU mapping detail on Windows 7 platforms
• OpenVINO Core:
  • Fixes the issues occurred when compiling a Pytorch model with unfold op

You can find OpenVINO™ toolkit 2023.0.2 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2023.0.2
• OpenVINO™ Development tools: pip install openvino-dev==2023.0.2

Release documentation is available here: https://docs.openvino.ai/2023.0/home.html

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-0.html

This release provides functional bug fixes and capability updates for 2023.0 that enable developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit with confidence.

Note: This is a standard release intended for developers that prefer the very latest version of OpenVINO. Standard releases will continue to be made available three to four times a year. Long Term Support (LTS) releases are also available. A new LTS version is released every year and is supported for 2 years (1 year of bug fixes, and 2 years for security patches). Visit Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy to get details on the latest LTS releases.

Major changes:

• POT:
  • Fixes the errors caused by the default usage of the MMap allocator (enabled in 2023.0). Only Windows affected.
• OpenVINO Core
  • Fixes the issue with properly handling the directory in read_model() on Windows

You can find OpenVINO™ toolkit 2023.0.1 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2023.0.1
• OpenVINO™ Development tools: pip install openvino-dev==2023.0.1

Release documentation is available here: https://docs.openvino.ai/2023.0/home.html

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-0.html

Summary of major features and improvements

• More integrations, minimizing code changes
  • Now you can load TensorFlow and TensorFlow Lite models directly in OpenVINO Runtime and OpenVINO Model Server. Models are converted automatically. For maximum performance, it is still recommended to convert to OpenVINO Intermediate Representation or IR format before loading the model. Additionally, we’ve introduced a similar functionality with PyTorch models as a preview feature where you can convert PyTorch models directly without needing to convert to ONNX.
  • Support for Python 3.11
  • NEW: C++ developers can now install OpenVINO runtime from Conda Forge​
  • NEW: ARM processors are now supported in CPU plug-in, including dynamic shapes, full processor performance, and broad sample code/notebook coverage. Officially validated for Raspberry Pi 4 and Apple® Mac M1/M2
  • Preview: A new Python API has been introduced to allow developers to convert and optimize models directly from Python scripts
• Broader model support and optimizations
  • Expanded model support for generative AI: CLIP, BLIP, Stable Diffusion 2.0, text processing models, transformer models (i.e. S-BERT, GPT-J, etc.), and others of note: Detectron2, Paddle Slim, RNN-T, Segment Anything Model (SAM), Whisper, and YOLOv8 to name a few.
  • Initial support for dynamic shapes on GPU - you no longer need to change to static shapes when leveraging the GPU which is especially important for NLP models.
  • Neural Network Compression Framework (NNCF) is now the main quantization solution. You can use it for both post-training optimization and quantization-aware training. Try it out: pip install nncf
• Portability and performance​
  • CPU plugin now offers thread scheduling on 12th gen Intel® Core and up. You can choose to run inference on E-cores, P-cores, or both, depending on your application’s configurations. It is now possible to optimize for performance or for power savings as needed. ​
  • NEW: Default Inference Precision - no matter which device you use, OpenVINO will default to the format that enables its optimal performance. For example, FP16 for GPU or BF16 for 4th Generation Intel® Xeon®. You no longer need to convert the model beforehand to specific IR precision, and you still have the option of running in accuracy mode if needed.​
  • Model caching on GPU is now improved with more efficient model loading/compiling.

You can find OpenVINO™ toolkit 2023.0 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2023.0.0
• OpenVINO™ Development tools: pip install openvino-dev==2023.0.0

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-0.html

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.0.0.dev20230427 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.0.0.dev20230427
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.0.0.dev20230427

Release notes is available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.0.0.dev20230407 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.0.0.dev20230407
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.0.0.dev20230407

Release notes is available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.0.0.dev20230217 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.0.0.dev20230217
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.0.0.dev20230217

Release notes is available here: https://docs.openvino.ai/nightly/prerelease_information.html Release documentation is available here: https://docs.openvino.ai/nightly/

NOTE: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.

OpenVINO™ toolkit pre-release definition:

• It is introduced to get early feedback from the community.
• The scope and functionality of the pre-release version is subject to change in the future.
• Using the pre-release in production is strongly discouraged.

You can find OpenVINO™ toolkit 2023.0.0.dev20230119 pre-release version here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install --pre openvino or pip install openvino==2023.0.0.dev20230119
• OpenVINO™ Development tools: pip install --pre openvino-dev or pip install openvino-dev==2023.0.0.dev20230119

Release documentation is available here: https://docs.openvino.ai/nightly/

Major Features and Improvements Summary

This is a Long-Term Support (LTS) release. LTS versions are released every year and supported for two years (one year for bug fixes, and two years for security patches). Read Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy  v.2 for more details.

• This 2022.3.2 LTS release provides functional and security bug fixes for the previous 2022.3.1 Long-Term Support (LTS) release, enabling developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit more efficiently.
• Intel® Movidius™ VPU-based products are supported in this release.

You can find OpenVINO™ toolkit 2022.3.2 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2022.3.2
• OpenVINO™ Development tools: pip install openvino-dev==2022.3.2

Release documentation is available here: https://docs.openvino.ai/2022.3/

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-lts/2022-3.html

Major Features and Improvements Summary

This is a Long-Term Support (LTS) release. LTS versions are released every year and supported for two years (one year for bug fixes, and two years for security patches). Read Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy  v.2 for more details.

• This 2022.3.1 LTS release provides functional bug fixes and minor capability changes for the previous 2022.3 Long-Term Support (LTS) release, enabling developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit with confidence.
• Intel® Movidius ™ VPU-based products are supported in this release.

You can find OpenVINO™ toolkit 2022.3 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2022.3.1
• OpenVINO™ Development tools: pip install openvino-dev==2022.3.1

Release documentation is available here: https://docs.openvino.ai/2022.3/

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-lts/2022-3.html

Major Features and Improvements Summary

This is a Long-Term Support (LTS) release. LTS releases are released every year and supported for 2 years (1 year of bug fixes, and 2 years for security patches). Read Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy  v.2 to get details.

• 2022.3 LTS release provides functional bug fixes, and capability changes for the previous 2022.2 release. This new release empowers developers with new performance enhancements, more deep learning models, more device portability and higher inferencing performance with less code changes.
• Broader model and hardware support – Optimize & deploy with ease across an expanded range of deep learning models including NLP, and access AI acceleration across an expanded range of hardware.
  • Full support for 4th Generation Intel® Xeon® Scalable processor family (code name Sapphire Rapids) for deep learning inferencing workloads from edge to cloud.
  • Full support for Intel’s discrete graphics cards, such as Intel® Data Center GPU Flex Series, and Intel® Arc™ GPU for DL inferencing workloads in the intelligent cloud, edge, and media analytics workloads.
  • Improved performance when leveraging throughput hint on CPU plugin for 12th and 13th Generation Intel® Core™ processor family (code named Alder Lake and Raptor Lake).
  • Enhanced “Cumulative throughput” and selection of compute modes added to AUTO functionality, enabling multiple accelerators (e.g. multiple GPUs) to be used at once to maximize inferencing performance.
• Expanded model coverage - Optimize & deploy with ease across an expanded range of deep learning models.
  • Broader support for NLP models and use cases like text to speech and voice recognition.
  • Continued performance enhancements for computer vision models Including StyleGAN2, Stable Diffusion, PyTorch RAFT and YOLOv7.
  • Significant quality and model performance improvements on Intel GPUs compared to the previous OpenVINO toolkit release.
  • New Jupyter notebook tutorials for Stable Diffusion text-to-image generation, YOLOv7 optimization and 3D Point Cloud Segmentation.
• Improved API and More Integrations – Easier to adopt and maintain code. Requires fewer code changes, aligns better with frameworks, & minimizes conversion
  • Preview of TensorFlow Front End – Load TensorFlow models directly into OpenVINO Runtime and easily export OpenVINO IR format without offline conversion. New “–use_new_frontend” flag enables this preview – see further details below in Model Optimizer section of release notes.
  • NEW: Hugging Face Optimum Intel – Gain the performance benefits of OpenVINO (including NNCF) when using Hugging Face Transformers. Initial release supports PyTorch models.
  • Intel® oneAPI Deep Neural Network Library (oneDNN) has been updated to 2.7 for further refinements and significant improvements in performance for the latest Intel CPU and GPU processors.
  • Introducing C API 2.0, to support new features introduced in OpenVINO API 2.0, such as dynamic shapes with CPU, pre-processing and post-process API, unified property definition and usage. The new C API 2.0 shares the same library files as the 1.0 API, but with a different header file.
• Note: Intel® Movidius ™ VPU based products are not supported in this release, but will be added back in a future OpenVINO 2022.3.1 LTS update. In the meantime, for support on those products please use OpenVINO 2022.1.
• Note: Macintosh* computers using the M1* processor can now install OpenVINO and use the OpenVINO ARM* Device Plug-in on OpenVINO 2022.3 LTS and later. This plugin is community supported; no support is provided by Intel and it doesn't fall under the LTS 2-year support policy. Learn more here: https://docs.openvino.ai/2022.3/openvino_docs_OV_UG_supported_plugins_ARM_CPU.html

You can find OpenVINO™ toolkit 2022.3 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2022.3.0
• OpenVINO™ Development tools: pip install openvino-dev==2022.3.0

Release documentation is available here: https://docs.openvino.ai/2022.3/

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-2022-3-lts-relnotes.html

Major Features and Improvements Summary

In this standard release, we’ve fine-tuned our largest update (2022.1) in 4 years to include support for Intel’s latest CPUs and discrete GPUs for more AI innovation and opportunity.

Note: This release intended for developers that prefer the very latest features and leading performance. Standard releases will continue to be made available three to four times a year. Long Term Support (LTS) releases are released every year and supported for 2 years (1 year of bug fixes, and 2 years for security patches). Read Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy to get details. For the latest LTS release visit our selector tool.

• Broader model and hardware support - Optimize & deploy with ease across an expanded range of deep learning models including NLP, and access AI acceleration across an expanded range of hardware.

  • NEW: Support for Intel 13th Gen Core Processor for desktop (code named Raptor Lake).
  • NEW: Preview support for Intel’s discrete graphics cards, Intel® Data Center GPU Flex Series and Intel® Arc™ GPU for DL inferencing workloads in intelligent cloud, edge and media analytics workloads. Hundreds of models enabled.
  • NEW: Test your model performance with preview support for Intel 4th Generation Xeon® processors (code named Sapphire Rapids).
  • Broader support for NLP models and use cases like text to speech and voice recognition. Reduced memory consumption when using Dynamic Input Shapes on CPU. Improved efficiency for NLP applications.

• Frameworks Integrations – More options that provide minimal code changes to align with your existing frameworks

  • OpenVINO Execution Provider for ONNX Runtime gives ONNX Runtime developers more choice for performance optimizations by making it easy to add OpenVINO with minimal code changes.
  • NEW: Accelerate PyTorch models with ONNX Runtime using OpenVINO™ integration with ONNX Runtime for PyTorch (OpenVINO™ Torch-ORT). Now PyTorch developers can stay within their framework and benefit from OpenVINO performance gains.
  • OpenVINO Integration with TensorFlow now supports more deep learning models with improved inferencing performance.

  • NOTE: The above frameworks integrations are not included in the install packages. Please visit the respective github links for more information. These products are intended for those who have not yet installed native OpenVINO

• More portability and performance - See a performance boost straight away with automatic device discovery, load balancing & dynamic inference parallelism across CPU, GPU, and more.

  • NEW: Introducing new performance hint (”Cumulative throughput”) in AUTO device, enabling multiple accelerators (e.g. multiple GPUs) to be used at once to maximize inferencing performance.
  • NEW: Introducing Intel® FPGA AI Suite support which enables real-time, low-latency, and low-power deep learning inference in this easy-to-use package

    • NOTE: The Intel® FPGA AI Suite is not included in our distribution packages, please request information here to learn more.

You can find OpenVINO™ toolkit 2022.2 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2022.2.0
• OpenVINO™ Development tools: pip install openvino-dev==2022.2.0

Release documentation is available here: https://docs.openvino.ai/2022.2/

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-relnotes.html

Major Features and Improvements Summary

This release is the biggest upgrade in 3.5 years! Read the release notes below for a summary of changes.

2022.1 release provides functional bug fixes, and capability changes for the previous 2021.4.2 LTS release. This new release empowers developers with new performance enhancements, more deep learning models, more device portability, and higher inferencing performance with fewer code changes.

Note: This is a standard release intended for developers that prefer the very latest features and leading performance. Standard releases will continue to be made available three to four times a year. Long Term Support (LTS) releases are also available. A new LTS version is released every year and is supported for 2 years (1 year of bug fixes, and 2 years for security patches). ead Intel® Distribution of OpenVINO™ toolkit Long-Term Support (LTS) Policy to get details. Latest LTS releases: 2020.x LTS and 2021.x LTS.

• Updated, cleaner API:

  • New OpenVINO API 2.0 was introduced. The API aligns OpenVINO inputs/outputs with frameworks. Input and output tensors use native framework layouts and element types. Old Inference Engine and nGraph APIs are available but will be deprecated in a future release down the road.

  • inference_engine, inference_engine_transformations, inferencengine_lp_transformations and ngraph libraries were merged to common openvino library. Other libraries were renamed. Please, use common ov:: namespace inside all OpenVINO components. See how to implement Inference Pipeline using OpenVINO API v2.0 for details.

  • Model Optimizer’s API parameters have been reduced to minimize complexity. Performance has been significantly improved for model conversion on ONNX models.

  • It’s highly recommended to migrate to API 2.0 because it already has additional features and this list will be extended later. The following list of additional features is supported by API 2.0:

    • Working with dynamic shapes. The feature is quite useful for best performance for Neural Language Processing (NLP) models, super-resolution models, and other which accepts dynamic input shapes. Note: Models compiled with dynamic shapes may show reduced performance and consume more memory than models configured with a static shape on the same input tensor size. Setting upper bounds to reshape the model for dynamic shapes or splitting the input into several parts is recommended.

    • Preprocessing of the model to add preprocessing operations to the inference models and fully occupy the accelerator and free CPU resources.

  • Read the Transition Guide for migrating to the new API 2.0.

• Portability and Performance:

  • New AUTO plugin self-discovers available system inferencing capacity based on model requirements, so applications no longer need to know its compute environment in advance.

  • The OpenVINO™ performance hints are the new way to configure the performance with portability in mind. The hints “reverse” the direction of the configuration in the right fashion: rather than map the application needs to the low-level performance settings, and keep an associated application logic to configure each possible device separately, the idea is to express a target scenario with a single config key and let the device to configure itself in response. As the hints are supported by every OpenVINO™ device, this is a completely portable and future-proof solution.

  • Automatic batching functionality via code hints automatically scale batch size based on XPU and available memory.

• Broader Model Support:

  • With Dynamic Input Shapes capabilities on CPU, OpenVINO will be able to adapt to multiple input dimensions in a single model providing more complete NLP support. Dynamic Shapes support on additional XPUs expected in a future dot release.

• New Models with focus on NLP and a new category, Anomaly detection, and support for conversion and inference of select PaddlePaddle models:

  • Pre-trained Models: Anomaly segmentation focus on industrial inspection making Speech denoising trainable plus updates on speech recognition and speech synthesis

  • Combined Demo: Noise reduction + speech recognition + question answering + translation+ text to speech

  • Public Models: Focus on NLP ContextNet, Speech-Transformer, HiFi-GAN, Glow-TTS, FastSpeech2, and Wav2Vec

• Built with 12th Gen Intel® Core™ 'Alder Lake' in mind. Supports the hybrid architecture to deliver enhancements for high-performance inferencing on CPU & integrated GPU

You can find OpenVINO™ toolkit 2022.1 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2022.1.0
• OpenVINO™ Development tools: pip install openvino-dev==2022.1.0

Release documentation is available here: https://docs.openvino.ai/2022.1/

Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-relnotes.html

This 2021.4.2 LTS release provides functional bug fixes, and minor capability changes for the previous 2021.4.1 Long-Term Support (LTS) release, enabling developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit with confidence. To learn more about long-term support and maintenance, go to the Long Term Support Policy.

You can find OpenVINO™ toolkit 2021.4.2 release here:

• Download archives* with OpenVINO™ Runtime
• OpenVINO™ Runtime for Python: pip install openvino==2021.4.2
• OpenVINO™ Development tools: pip install openvino-dev==2021.4.2

Release Notes: https://software.intel.com/content/www/us/en/develop/articles/openvino-2021-4-lts-relnotes.html

This 2021.4.1 LTS release provides functional bug fixes, and minor capability changes for the previous 2021.4 Long-Term Support (LTS) release, enabling developers to deploy applications powered by Intel® Distribution of OpenVINO™ toolkit with confidence. To learn more about long-term support and maintenance, go to the Long Term Support Policy.

You can find OpenVINO™ toolkit 2021.4.1 release here:

• Download archives* with OpenVINO™ Runtime
• OpenVINO™ Runtime for Python: pip install openvino==2021.4.1
• OpenVINO™ Development tools: pip install openvino-dev==2021.4.1

Release Notes: https://software.intel.com/content/www/us/en/develop/articles/openvino-2021-4-lts-relnotes.html

What's New

• Integrates the Deep Learning Workbench with the Intel® DevCloud for the Edge as a Beta release. Graphically analyze models using the Deep Learning Workbench on the Intel® DevCloud for the Edge (instead of a local machine only) to compare, visualize and fine-tune a solution against multiple remote hardware configurations.
• Introduces support for Red Hat Enterprise Linux (RHEL) 8.2.
• Introduces per-channel quantization support in the Model Optimizer for models quantized with TensorFlow Quantization-Aware Training containing per-channel quantization for weights, which improves performance by model compression and latency reduction.
• Pre-trained models and support for public models to streamline development:
  • Public Models: Yolov4 (for object detection), AISpeech (for speech recognition), and DeepLabv3 (for semantic segmentation)
  • Pre-trained Models: Human Pose Estimation (update), Formula Recognition Polynomial Handwritten (new), Machine Translation (update), Common Sign Language Recognition (New), and Text-to-Speech (new)
• New OpenVINO™ Security Add-on, which controls access to model(s) through secure packaging and execution. Based on KVM Virtual machines and Docker* containers and compatible with the OpenVINO™ Model Server, this new add-on enables packaging for flexible deployment and controlled model access.
• PyPI project moved from openvino-python to openvino, and 2021.1 version to be removed in the default view. The specific version is still available for users depending on this exact version by using openvino-python==2021.1

You can find OpenVINO™ toolkit 2021.2 release here:

• Download archives* with OpenVINO™ Runtime for C/C++
• OpenVINO™ Runtime for Python: pip install openvino==2021.2

Release Notes: https://software.intel.com/content/www/us/en/develop/articles/openvino-relnotes.html