While JIT-compiled eBPF programs can work on systems where Hypervisor-protected Code Integrity (HVCI) is not enabled, JIT-compiled programs fail on systems where it is enabled. This document describes the mechanism for such systems which works on existing unmodified Windows.
To permit the use of eBPF programs on systems where HVCI is enabled, the eBPF programs must be in a form that Windows' Secure Kernel (SK) can verify their signature and load them. The only code format currently supported by SK is Portable Executable (PE) files in the form of Windows Drivers (.sys files) that have been signed by a trusted key.
This document covers generating signed PE files from eBPF programs in an automated fashion.
eBPF programs start in a higher level language (like C) that is then compiled using the LLVM tool chain into an ELF file containing the definition for zero or more maps and one or more code segments containing eBPF byte code. eBPF For Windows uses the PREVAIL verifier to perform safety analysis of eBPF programs. The verifier accepts an eBPF program compiled into an Executable Linux Format (ELF) file and then returns a yes/no response depending on whether it can prove the program is safe. The verified ELF file can then be used to generate a PE image that can be loaded and used as an eBPF program.
From the root of the eBPF-For-Windows project (from a VS Developer Command Prompt), after building the project, run:
powershell scripts\Convert-BpfToNative.ps1 -ProgramName my_program
Where my_program is the name of your BPF program without the extension. This will produce a native image in x64\Release with name of my_program.sys.
The following diagram shows the steps in the pipeline:
Generation of a Windows Driver in PE format from an eBPF program can be broken down into a series of steps. The first step is executed by the developer either on their own build machine or part of a pre-existing build pipeline. The remaining steps can be executed in a secure environment (for a production signed PE image) or a developer environment (for a test signed PE image).
The first step of the process is identical to the traditional build pipeline. The eBPF program is provided as a C or other high level language file and the normal LLVM / Clang tool chain is invoked, to produce an ELF file containing one or more programs and zero or more map definitions.
The second step is to use the verifier to determine if the resulting eBPF program conforms to the required safety requirements. This is identical to the process used to by eBPF for Windows today. Only once the verifier can prove the program is safe does the process proceed to the next step.
The third step is to automatically generate C code that maps one to one with the eBPF instructions in the ELF file. The eBPF registers are mapped to local variables in the generated C function and each eBPF instruction is used to generate equivalent C operations. In addition, the generated C file also contains bindings for any helper functions referenced by the eBPF program and any maps the eBPF program uses.
The fourth step is to use a standard C compiler (GCC, MSVC, or Clang) to generate an x64/ARM64 COFF file with Windows calling conventions. The C file will expose a single top-level global variable that allows the skeleton code to find the helpers, maps, and programs contained in the file.
The fifth step is to use the platform-specific linker to generate the final PE file. The linker consumes the object file from step 4, a generic DriverEntry skeleton.
The last step is to use the standard PE signing tools to produce a signed PE file from the output of step 5. For the developer workflow the signature would be based on a test signing root, whereas for production use the PE file would be signed using the Microsoft PRS signing keys.
Step 5 consumes a set of object files to generate the final PE image. The following section gives a summary of the driver skeleton.
All Windows Drivers need to export a static function labelled "DriverEntry" that is invoked by the OS (Operating System) during loading of the driver. In addition to the standard driver related functionality, the driver entry skeleton also registers as a Network Module Registrar (NMR) client to export the program to the eBPF runtime, to create any maps required, and to import the helper functions.
Every generated C file contains a single global entry point of type metadata_table_t:
typedef struct _metadata_table
{
void (*programs)(program_entry_t** programs, size_t* count);
void (*maps)(map_entry_t** maps, size_t* count);
} metadata_table_t;
metadata_table_t myprogram_metadata_table = { _get_programs, _get_maps };
The metadata table provides pointers to three functions that permit querying the list of programs and the list of maps. The table name prefix is derived from the ELF file name, with _ replacing any character that is not valid in a C variable name. This variable is the only globally visible variable in the generated C code.
Each program in the generated C file is exported via a program_entry_t:
typedef struct _program_entry
{
uint64_t (*function)(void*);
const char* section_name;
const char* program_name;
uint16_t* referenced_map_indices;
uint16_t referenced_map_count;
helper_function_entry_t* helpers;
uint16_t helper_count;
size_t bpf_instruction_count;
ebpf_program_type_t* program_type;
ebpf_attach_type_t* expected_attach_type;
} program_entry_t;
The skeleton framework then uses NMR to publish this information to the eBPF execution context.
The generated C code exposes a table containing the address of each helper function, name, ID, and additional meta-data of the helper function. The C code generator emits a table for the helper functions referenced by the program:
typedef struct _helper_function_entry
{
uint64_t (*address)(uint64_t, uint64_t, uint64_t, uint64_t, uint64_t);
uint32_t helper_id;
const char* name;
bool tail_call;
} helper_function_entry_t;
static helper_function_entry_t _helpers[] = {
{ NULL, 1, "helper_id_1"},
{ NULL, 0, "helper_id_0"},
};
The skeleton then uses NMR to query the address of each helper function in the table, after which it writes to the address field in the table and sets additional metadata (such as if this is a tail call). Calls to helper functions in the generated code are called indirectly via the address field.
Each map referenced by any of the eBPF programs is added as a map_entry_t:
typedef struct _map_entry
{
void* address;
ebpf_map_definition_in_file_t definition;
const char* name;
} map_entry_t;
static map_entry_t_maps[] = {
{ NULL, { 36, 2, 4, 8, 1, }, "dropped_packet_map" },
{ NULL, { 36, 2, 4, 4, 1, }, "interface_index_map" },
};
The skeleton then uses NMR to either create an instance of the map or obtain the address of an existing map and then write the address into the address field. References to the maps in the generated code are then indirect via the address field.
The process of loading an eBPF program is a series of interactions between the eBPF Execution Context and the generated .sys file, and is summarized in the following diagram:
-
During driver entry, the generated DriverEntry calls NmrRegisterClient for the helper contract and the map contract.
-
In response to the attach notification on the helper function NMR contract, the skeleton queries the address of each helper function the eBPF program used and writes the address into the _helpers table.
-
In response to the attach notification on the map resolve NMR contract, the skeleton calls into the eBPF execution context to create the maps based on the definitions and writes the addresses into the _maps table.
-
Once both the map contract and the helper contract have been attached, the driver registers the program NMR contract.
-
In response to the attach notification on the program contract, the skeleton returns the list of program entry points and meta-data about the programs.
-
At this point, the eBPF execution context is free to invoke the eBPF programs.
-
In response to a detach on either the helper function or the map contract, the driver unregisters its program contract.
-
When the OS calls unload on the driver, the driver unregisters all of its contracts and waits for the notification that the eBPF execution context has detached before completing unloading.