Bump openvino from 2022.3.1 to 2024.4.0

[dependabot]

Bumps openvino from 2022.3.1 to 2024.4.0.

Release notes

Sourced from openvino's releases.

2024.4.0

Summary of major features and improvements  

• More Gen AI coverage and framework integrations to minimize code changes

  • Support for GLM-4-9B Chat, MiniCPM-1B, Llama 3 and 3.1, Phi-3-Mini, Phi-3-Medium and YOLOX-s models.
  • Noteworthy notebooks added: Florence-2, NuExtract-tiny Structure Extraction, Flux.1 Image Generation, PixArt-α: Photorealistic Text-to-Image Synthesis, and Phi-3-Vision Visual Language Assistant.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • OpenVINO™ runtime optimized for Intel® Xe Matrix Extensions (Intel® XMX) systolic arrays on built-in GPUs for efficient matrix multiplication resulting in significant LLM performance boost with improved 1st and 2nd token latency, as well as a smaller memory footprint on Intel® Core™ Ultra Processors (Series 2).
  • Memory sharing enabled for NPUs on Intel® Core™ Ultra Processors (Series 2) for efficient pipeline integration without memory copy overhead.
  • Addition of the PagedAttention feature for discrete GPUs* enables a significant boost in throughput for parallel inferencing when serving LLMs on Intel® Arc™ Graphics or Intel® Data Center GPU Flex Series.

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • Support for Intel® Core Ultra Processors Series 2 (formerly codenamed Lunar Lake) on Windows.
  • OpenVINO™ Model Server now comes with production-quality support for OpenAI-compatible API which enables significantly higher throughput for parallel inferencing on Intel® Xeon® processors when serving LLMs to many concurrent users.
  • Improved performance and memory consumption with prefix caching, KV cache compression, and other optimizations for serving LLMs using OpenVINO™ Model Server.
  • Support for Python 3.12.
  • Support for Red Hat Enterprise Linux (RHEL) version 9

Support Change and Deprecation Notices

• Using deprecated features and components is not advised. They are available to enable a smooth transition to new solutions and will be discontinued in the future. To keep using discontinued features, you will have to revert to the last LTS OpenVINO version supporting them. For more details, refer to the OpenVINO Legacy Features and Components page.
• Discontinued in 2024.0:
  • Runtime components:
    • Intel® Gaussian & Neural Accelerator (Intel® GNA)..Consider using the Neural Processing Unit (NPU) for low-powered systems like Intel® Core™ Ultra or 14th generation and beyond.
    • OpenVINO C++/C/Python 1.0 APIs (see 2023.3 API transition guide for reference).
    • All ONNX Frontend legacy API (known as ONNX_IMPORTER_API)
    • 'PerfomanceMode.UNDEFINED' property as part of the OpenVINO Python API
  • Tools:
    • Deployment Manager. See installation and deployment guides for current distribution options.
    • Accuracy Checker.
    • Post-Training Optimization Tool (POT). Neural Network Compression Framework (NNCF) should be used instead.
    • A Git patch for NNCF integration with huggingface/transformers. The recommended approach is to use huggingface/optimum-intel for applying NNCF optimization on top of models from Hugging Face.
    • Support for Apache MXNet, Caffe, and Kaldi model formats. Conversion to ONNX may be used as a solution.
• Deprecated and to be removed in the future:
  • The macOS x86_64 debug bins will no longer be provided with the OpenVINO toolkit, starting with OpenVINO 2024.5.
  • Python 3.8 is now considered deprecated, and it will not be available beyond the 2024.4 OpenVINO version.
  • dKMB support is now considered deprecated and will be fully removed with OpenVINO 2024.5
  • Intel® Streaming SIMD Extensions (Intel® SSE) will be supported in source code form, but not enabled in the binary package by default, starting with OpenVINO 2025.0
  • The openvino-nightly PyPI module will soon be discontinued. End-users should proceed with the Simple PyPI nightly repo instead. More information in Release Policy.
  • The OpenVINO™ Development Tools package (pip install openvino-dev) will be removed from installation options and distribution channels beginning with OpenVINO 2025.0.
  • Model Optimizer will be discontinued with OpenVINO 2025.0. Consider using the new conversion methods instead. For more details, see the model conversion transition guide.
  • OpenVINO property Affinity API will be discontinued with OpenVINO 2025.0. It will be replaced with CPU binding configurations (ov::hint::enable_cpu_pinning).
  • OpenVINO Model Server components:
    • “auto shape” and “auto batch size” (reshaping a model in runtime) will be removed in the future. OpenVINO’s dynamic shape models are recommended instead.
  • A number of notebooks have been deprecated. For an up-to-date listing of available notebooks, refer to the OpenVINO™ Notebook index (openvinotoolkit.github.io).

You can find OpenVINO™ toolkit 2024.4 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2024.4.0
• OpenVINO™ for Python: pip install openvino==2024.4.0

... (truncated)

Commits

• c3152d3 [GPU] Fix lack of support for 3D (and smaller) shapes in TransposeFusion pass...
• 8c9d4be [CPU] Add interface to release compiled model internal memory (#26390)
• ad3f51b [CPU] fix QKVProjection fake-matching for accuracy with bert models (#26417)
• 0bfa578 [GPU] Disable transpose fusion in case of no transpose found on both branches...
• b34295c Change log level from error to warning if backend is not found (#26418)
• a9234a9 [intel-npu] Additional documentation for NPU_MAX_TILES property's usage (#26385)
• 04e68ee Disabled LTO for gcc older than 9.0 (#26330) (#26399)
• b543ebb [GPU] out-of-bound access from convolution kernel (#26377)
• df65664 [NPU] Fix coverity issue (#26373)
• 63923c7 [GPU] Fix accuracy issue of qwen-2 model (#26389)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Dependabot tried to add @frgfm, @charlesmindee and @felixdittrich92 as reviewers to this PR, but received the following error from GitHub:

POST https://api.github.com/repos/h2oai/doctr/pulls/37/requested_reviewers: 422 - Reviews may only be requested from collaborators. One or more of the users or teams you specified is not a collaborator of the h2oai/doctr repository. // See: https://docs.github.com/rest/pulls/review-requests#request-reviewers-for-a-pull-request

[dependabot]

The following labels could not be found: topic: build.

[dependabot]

Superseded by #38.