Add a new single API for Level Zero Init and Driver Retrieval

[wdamon-intel]

Add a new single API for Level Zero Init and Driver Retrieval

Metadata

Component

Core

Classification

API

Authors

Neil R. Spruit <[email protected]>

Summary

When multiple libraries exist in an application which are initializing L0, the types of drivers reported may not be consistent. We need to have a single API to both initialize level zero and filter the reported drivers.

Motivation

In a single application, one library may request GPUs and another may request VPUs. This creates a problem where calling the initialization and driver retrieval APIs in sequence may result in the wrong set of drivers being filtered for the specific library.

To fix this issue, a single API is proposed for use by all libraries such that Level Zero initialization and driver retrieval is done in a single API call.

Description

Add a new API zeInitDrivers with a new initialization flag type to allow for both the initialization of Level Zero drivers and the retrieval of Level Zero driver handles in a single call.

This enables users to initialize and retrieve the Level Zero drivers based on the driver types requested with no dependencies or interference from other calls to Level Zero initialization.

Dependencies

• None

Details

Any library needing to use NPUs and GPUs or only NPUs would be moved to use this new API zeInitDrivers to perform the Level Zero initialization and the driver retrieval. This makes it possible to retrieve the specified set of drivers based on the initialzation flags such that:

ZE_INIT_DRIVER_TYPE_FLAG Value	Resulting Drivers
ZE_INIT_DRIVER_TYPE_FLAG_GPU	GPU Drivers are Init'd and driver handles retrieved.
ZE_INIT_DRIVER_TYPE_FLAG_NPU	NPU Drivers are Init'd and driver handles retrieved.
ZE_INIT_DRIVER_TYPE_FLAG_GPU | ZE_INIT_DRIVER_TYPE_FLAG_NPU	NPU and GPU Drivers are Init'd and driver handles retrieved.
UINT32_MAX	All Drivers of any type are Init'd and driver handles retrieved.

Multiple Call Safety

Similar to the usage of zeInit, multiple calls to zeInitDrivers with the same flags must return the same driver handles in the same order.

Each call to zeInitDrivers will initialize the driver types associated with the initialization type flags, i.e.:

zeInitDrivers with ZE_INIT_DRIVER_TYPE_FLAG_GPU will initialize and retrieve GPU Drivers.
zeInitDrivers with ZE_INIT_DRIVER_TYPE_FLAG_NPU will initialize and retrieve NPU Drivers.

Subsequent calls to zeInitDrivers with different sets of initialization type flags will return different sets of drivers.

Once a given driver has been initialized by the loader, it will not be re-initialized by the loader in subsequent calls. The initialization type will then indicate whether that driver handle needs to be returned to the user. This ensures that the driver is not forced to re-initialize after each call to zeInitDrivers and it is able to maintain the global singleton object that is the driver and device handles for a Level Zero driver.

zeInit Deprecation Plan

To maintain backwards compatibility and avoid regressions, zeInit must continue to be supported in v1.x however only with support for initializing with the first "init" flags passed in the program and limited to supporting GPU initialization.

This means, to support different sets of library initializations with different sets of drivers, the new zeInitDrivers will be evangelized as the replacement for those wanting to use both GPU and NPU or want to ensure that NPUs are available to the application.

The zeInit API will be marked deprecated as part of this proposal and removed in the next major version release of the Level Zero spec.

Application Usage of zeInitDrivers

Applications which are built linking the ze_loader will either need to have a build time check for the symbol zeInitDrivers or continue to use zeInit. This can be done by adding a compile time symbol check in the code to verify the symbol is defined in the headers. For runtime, this would limit this version of these libraries to require this new version of the Level Zero loader with this symbol defined.

Applications that want a dynamic method for discovering this new API via dlopen/LoadLibrary will be able to check for the symbol's existence during runtime.

New Entry Points

zeInitDrivers

ze_result_t zeInitDrivers(
    uint32_t* pCount,
    ze_driver_handle_t* phDrivers,
    ze_init_driver_type_desc_t *desc,
);

Parameter	Description
pCount	[ ${\textsf{\color{orange}in, out}}$ ] pointer to the number of driver instances. if count is zero, then the loader shall update the value with the total number of drivers
phDrivers	[ ${\textsf{\color{orange}in, out}}$ ][optional][range(0, *pCount)] array of driver instance handles. if count is less than the number of drivers available, then the loader shall only retrieve that number of drivers.
desc	[ ${\textsf{\color{orange}in}}$ ] descriptor with driver init flags to filter which driver handles are returned.

---

New Enums

ze_init_driver_type_flags_t

typedef uint32_t ze_init_driver_type_flags_t;
typedef enum _ze_init_driver_type_flag_t
{
    ZE_INIT_DRIVER_TYPE_FLAG_GPU = ZE_BIT(0),                                     ///< initialize & retrieve GPU drivers
    ZE_INIT_DRIVER_TYPE_FLAG_NPU = ZE_BIT(1),                                     ///< initialize & retrieve NPU drivers
    ZE_INIT_DRIVER_TYPE_FLAG_FORCE_UINT32 = 0x7fffffff

} ze_init_driver_type_flag_t;

Flag	Description
GPU	Initialize and retrieve GPU drivers.
NPU	Initialize and retrieve NPU drivers.

ze_structure_type_t

typedef enum _ze_structure_type_t {
    // ...

    ZE_STRUCTURE_TYPE_INIT_DRIVER_TYPE_DESC,

    ZE_STRUCTURE_TYPE_FORCE_UINT32 = 0x7fffffff
} ze_structure_type_t;

Type	Description
ZE_STRUCTURE_TYPE_INIT_DRIVER_TYPE_DESC	enum for the init driver type descriptor structure

---

New Structures

ze_init_driver_type_desc_t

typedef struct _ze_init_driver_type_desc_t {
    ze_structure_type_t               stype;
    const void*                       pNext;
    ze_init_driver_type_flags_t       flags;
} ze_init_driver_type_desc_t;

Field	Description
stype	[ ${\textsf{\color{orange}in}}$ ] type of this structure
pNext	[ ${\textsf{\color{orange}in, out}}$ ][ ${\textsf{\color{lightblue}optional}}$ ] must be null or a pointer to an extension specific structure (i.e. contains stype and pNext).
flags	[ ${\textsf{\color{orange}in}}$ ] init driver type flags for driver initialization and retrieval.

---

Example Usage

uint32_t npu_driver_handle_count = 0;
std::vector<ze_driver_handle_t> npu_driver_handles(npu_driver_handle_count);
ze_init_driver_type_desc_t desc = {ZE_STRUCTURE_TYPE_INIT_DRIVER_TYPE_DESC};
desc.pNext = nullptr;
desc.flags = ZE_INIT_DRIVER_TYPE_NPU;
ze_result_t npu_result = zeInitDrivers(&npu_driver_handle_count, nullptr, &desc);
if (npu_driver_handle_count > 0 && npu_result == ZE_RESULT_SUCCESS) {
      npu_driver_handles.resize(npu_driver_handle_count);
      npu_result = zeInitDrivers(&npu_driver_handle_count, npu_driver_handles.data(), &desc);
}

[kevin-t-tang]

What is the current status for this case?