OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. It enables programmers to develop scalable and efficient AI solutions with relatively few lines of code. It supports several popular model formats[2] and categories, such as large language models, computer vision, and generative AI.
Actively developed by Intel, it prioritizes high-performance inference on Intel hardware but also supports ARM/ARM64 processors[2] and encourages contributors to add new devices to the portfolio.
Based in C++, it offers the following APIs: C/C++, Python, and Node.js (an early preview).
OpenVINO is cross-platform and free for use under Apache License 2.0
OpenVINO IR[4] is the default format used to run inference. It is saved as a set of two files, *.bin and *.xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application’s API or a dedicated converter.
Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference.
The supported model formats are:[5]
PyTorch
TensorFlow
TensorFlow Lite
ONNX (including formats that may be serialized to ONNX)
PaddlePaddle
The high level pipeline of OpenVINO consists of two parts: generate IR (Intermediate Representation) files via Model Optimizer using your trained model or public one and execute inference on Inference Engine on specified devices.
OpenVINO has different sample types: classification, object detection, style transfer, speech recognition, etc. It is possible to try inference on public models. There are a variety of models for tasks, such as:
classification
segmentation
object detection
face recognition
human pose estimation
monocular depth estimation
image inpainting
style transfer
action recognition
colorization
OpenVINO™ Toolkit AI Programming Workshops
Join training on GenAI, LLMs, AI PC, and more in live Q&A with Intel’s leading AI voices. DevCon’s live and on-demand webinars include code tutorial based workshops using Jupyter Notebooks—learn with like-minded AI devs.
OpenVINO™ toolkit is an open source toolkit that accelerates AI inference with lower latency and higher throughput while maintaining accuracy, reducing model footprint, and optimizing hardware use. It streamlines AI development and integration of deep learning in domains like computer vision, large language models, and generative AI.