Skip to content

Home

Jump to bottom
Russ Weight edited this page Oct 7, 2022 · 51 revisions

Linux FPGA Device Feature List (DFL) Driver

The Linux DFL driver provides support for FPGA devices that are designed to support the Device Feature List. The DFL, which is implemented in RTL, consists of a self-describing data structure in the PCI BAR space that allows the DFL driver to automatically load the drivers required for a given FPGA configuration.

Branches

fpga-upstream-dev

The fpga-upstream-dev branch consists of patches that Intel is currently in the process of submitting to the upstream linux kernel. Consistent with the upstreaming process, this branch will be rebased for each new version of the mainline linux kernel. The current convention is to rebase these patches onto the latest *-rc1 kernel. The first version of the fpga-upstream-dev branch is based on 5.7-rc1. When advancing to a new version, older versions will be renamed to append the version number (e.g. fpga-upstream-dev-5.7-rc1) and will be kept around as a reference. Note that in some cases, for technical reasons, rc1 may be skipped in favor of rc2 or rc3, etc. For example, when moving to the 5.16 kernel, 5.16-rc3 was selected as the base kernel.

fpga-ofs-dev

The fpga-ofs-dev branch serves as a staging branch for patches that are being prepared for submission to the linux kernel. This branch consists of a set of patches on top of the fpga-upstream-dev branch. Most of these additional patches will eventually be submitted upstream, but should be considered "not fully baked". They will likely be refined or replaced before they are moved into fpga-upstream-dev and eventually submitted to the upstream linux community.

Note that because fpga-ofs-dev sits on top fpga-upstream-dev, it is rebased every time fpga-upstream-dev changes. To facilitate comparison with previous versions of fpga-ofs-dev, older versions are archived with the date appended to the name (e.g. fpga-ofs-dev-2021-03-04)

When advancing to a new kernel version, older versions of fpga-ofs-dev will be renamed to append the kernel version. Unlike the fpga-upstream-dev branches, saved versions of the fpga-ofs-dev branches will be based on the the *.0 kernel version. For example, when the 5.11-rc1 version of fpga-upstream-dev was saved aside as fpga-upstream-dev-5.11-rc1, fpga-ofs-dev was rebased onto kernel version 5.11.0, and was archived as fpga-ofs-dev-5.11.0.

The most recent long term support (LTS) kernel will be maintained with both bug fixes and new functionality. The naming for these branches will include "-lts" (e.g. fpga-ofs-dev-5.15-lts).

fpga-ofs-dev-5.15-lts

The fpga-ofs-dev-5.15-lts branch is provided as a service to Intel customers who are developing products based on Intel FPGAs. Patches will be applied to the LTS branch over time to maintain equivalent DFL driver code with the fpga-ofs-dev branch. This support will continue until the next LTS upstream kernel is releases, at which time support will shift to the new LTS kernel. LTS branches are non-rebasing branches: they will maintain a consistent patch history.

The fpga-ofs-dev-5.15-lts branch will be merged with the latest stable version of the 5.15 kernel on about a two week cadence. The initial base kernel version for the fpga-ofs-dev-5.15-lts branch was 5.15.5. The process of doing these merges introduces hundreds of new patches that can obscure the DFL driver patch history. In order to identify and harvest the DFL driver patches from an LTS branch, use the --first-parent option of git log. With the --first-parent option, each merge is represented with a single entry in the git log output, making it easier to track the DFL driver specific patches:

$ git log --oneline --first-parent origin/fpga-ofs-dev-5.15-lts
ec2a46c7dd21 Merge stable kernel version v5.15.6
164534e62fe3 fpga: stratix10-soc: Do not use ret uninitialized in s10_probe()
ee1f14cddd48 fpga: region: Use standard dev_release for class driver
c9f67f9c48f6 fpga: bridge: Use standard dev_release for class driver
5b29a1ac5c42 fpga: mgr: Use standard dev_release for class driver
58d951060e8d fpga: m10bmc-sec: Extend SDM cert error logging
70344438539d configs: DFL driver config options
7cbc1ed27f50 fpga: dfl: add debug to failed partial reconfiguration

In order to filter out the stable kernel updates completely, add the --no-merges option to the command line. You can get a complete listing of the DFL patches from the 5.15-lts kernel with this command:

$ git log --oneline --first-parent --no-merges v5.15.5..origin/fpga-ofs-dev-5.15-lts
164534e62fe3 fpga: stratix10-soc: Do not use ret uninitialized in s10_probe()
ee1f14cddd48 fpga: region: Use standard dev_release for class driver
c9f67f9c48f6 fpga: bridge: Use standard dev_release for class driver
5b29a1ac5c42 fpga: mgr: Use standard dev_release for class driver
58d951060e8d fpga: m10bmc-sec: Extend SDM cert error logging
70344438539d configs: DFL driver config options
7cbc1ed27f50 fpga: dfl: add debug to failed partial reconfiguration
61bf02a014a7 fpga: dfl: iopll: Add sysfs node for feature revision

Note that the port to the 5.15-lts kernel began with kernel version 5.15.5, as reflected in the above command.

Kernel Configuration

Config File

The fpga-ofs-dev* branches each contain a configs directory at the top level of the kernel source tree. The dfl-config file contains the configuration parameters that are required to enable the features that are provided by the linux DFL driver. The README file in the same directory explains how to modify your current kernel configuration to incorporate these new parameters.

The CONFIG_LOCALVERSION option can be used to modify the kernel version string to differentiate the kernel you build from a stock kernel build of the same base version. For example, if you use the following option when building a 5.15.5 kernel, the resulting kernel will have a version string of "5.15.5-dfl".

CONFIG_LOCALVERSION="-dfl"

Note: If you are starting with a kernel configuration file from a significantly different kernel version (e.g. 3.14 vs 5.15), then you may see warning messages associated with configuration parameters that have changed or been removed between kernel versions. Be sure to evaluate any warning messages to determine if additional configuration changes are required to maintain your desired set of features.

Kernel Command Line

To use SRIOV and pass a VF to a virtual machine, you must add intel_iommu=on to the kernel command line.

When using SRIOV, it may also be necessary to add pci=realloc to the kernel command line. For additional information regarding this command line option, see https://access.redhat.com/solutions/37376. configuration.

Kernel Packaging (.rpm or .deb)

The kernel Makefile supports build targets for building .rpm and .deb packages:

  • rpm-pkg: Build both source and binary RPM kernel packages
  • binrpm-pkg: Build only the binary kernel RPM package
  • deb-pkg: Build both source and binary deb kernel packages
  • bindeb-pkg: Build only the binary kernel deb package

If you are concerned about the size of the resulting package and binaries, you can significantly reduce the size of the package and object files by setting the make variable INSTALL_MOD_STRIP. For example:

make INSTALL_MOD_STRIP=1 binrpm-pkg

Documentation

DFL Feature ID Registry

Framework Documentation

Sysfs Files

FPGA Regions

DFL devices do not use device tree, but the following device tree documentation provides a lot of useful information about FPGA regions and bridges.

Clone this wiki locally