NPU Readiness: A Pre-Ship Checklist for Intel Core Ultra

Problem

The neural processing unit introduced with Intel Core Ultra processors is a low-power device for AI inference. OpenVINO exposes it as the NPU device, but it has requirements and limits that the CPU and GPU do not.

Teams meet the same surprises: a model with dynamic shapes that will not compile, compile times that dwarf the first inference, driver differences between machines, and caching that behaves differently from other devices.

The friction points are driver and platform requirements, the static-shape constraint, and knowing which properties your driver actually exposes.

Options and trade-offs

• Platform and driver: the NPU needs a supported platform and an NPU driver. Record the driver version with every result, because available properties and behavior depend on it.
• Static shapes: Intel community guidance and published research on NPU inference state that the NPU in OpenVINO supports models with static shapes. If your model has dynamic dimensions, plan to fix them before compiling, and check the release notes of your OpenVINO version for any newer capability.
• Driver-level cache: the plugin README and documentation say a user-mode-driver model cache is enabled by default in current NPU drivers to improve time to first inference. OpenVINO's own caching is enabled when you set cache_dir. Read how the two interact in the documentation for your driver and version.
• Property discovery: instead of relying on remembered property names, ask the device. Querying the NPU device for its supported properties lists what your driver and OpenVINO version expose. Record that list with your results.
• Compile time: the first compile can take much longer than the first inference. Use compiled-model caching so a restart does not repeat it, and measure it as described in the caching article.
• Fallback: provide a CPU path for machines where the NPU is absent, the driver is outdated, or compilation fails, and log when the fallback is used.
• Recommendation: treat NPU support as something your build proves. Add a smoke test that compiles a small static-shape model on the NPU and fails loudly if it cannot, and record the platform, driver, and OpenVINO versions. Fix input shapes before compiling, enable caching, and run twice in separate processes to confirm the second start is faster.
• Decision lens: whether the NPU is the right target at all. If your model needs dynamic shapes or more memory than the device offers, the CPU or GPU may be simpler. Example setup: a laptop application that runs a small encoder on the NPU for power efficiency and falls back to the CPU if the smoke test fails.

Minimal working example

Prereqs

• An Intel Core Ultra system with an NPU and a current NPU driver.
• Python 3 with the openvino package installed.
• A static-shape model.

npu_probe.py

import sys, time
import openvino as ov

core = ov.Core()
print("Devices:", core.available_devices)
if "NPU" not in core.available_devices:
    sys.exit("No NPU found; use the CPU fallback")

print("NPU supported properties:")
for prop in core.get_property("NPU", "SUPPORTED_PROPERTIES"):
    print(" ", prop)

core.set_property({"CACHE_DIR": "./npu_cache"})
start = time.perf_counter()
compiled = core.compile_model(sys.argv[1], "NPU")  # a static-shape model
print(f"compile on NPU: {time.perf_counter() - start:.3f}s")

Adjust file names, model names, and paths to your own setup.

Test: Run npu_probe.py twice in separate processes and keep the printed property list. Compare compile time between the first and second run. Then try a model with a dynamic dimension and record what the NPU does. If there is no NPU on the machine, the script exits with a message, which is the fallback case your application also needs to handle.

Record: platform, NPU driver version, and OpenVINO version; the property list the NPU reports; compile time with an empty and a populated cache as median and 95th percentile over at least ten fresh processes; first-inference time cold versus cached; behavior with a dynamic-shape model; fallback behavior when the NPU is unavailable.

Checklist

• Record platform, driver, and OpenVINO versions.
• Add an NPU compile smoke test to your build.
• Fix input shapes before compiling.
• Enable caching and confirm the second start is faster.
• List the properties the device exposes, provide a logged CPU fallback, and re-run everything after a driver or OpenVINO upgrade.
• Avoid: compiling a dynamic-shape model and expecting it to work.
• Avoid: assuming compile time on a developer laptop matches the target machine.

Where this doesn't apply

Machines without an NPU, models that cannot be expressed with static shapes, and workloads where the GPU or CPU already meets latency and power targets.

Sources

• OpenVINO NPU device documentation — https://docs.openvino.ai/2025/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html (accessed 2026-10-18).
• Intel NPU plugin README — https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_npu/README.md (accessed 2026-10-18).
• Intel community: NPU supports static shapes — https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/How-to-accelerate-NPU-using-OpenVino/m-p/1567889 (accessed 2026-10-18).
• NITRO: LLM Inference on Intel Laptop NPUs (arXiv) — https://arxiv.org/html/2412.11053v1 (accessed 2026-10-18).
• OpenVINO Model Caching Overview — https://docs.openvino.ai/nightly/openvino-workflow/running-inference/optimize-inference/optimizing-latency/model-caching-overview.html (accessed 2026-10-18).

---