In this lesson you will see that a BPF ELF file produced by LLVM can contain more than one XDP program, and how you can select which one to load using the libxdp API. Complete each of the assignments below to complete the lesson.
The libbpf API not only provides the basic system call wrappers (which are defined in libbpf bpf/bpf.h). The API also provides ”objects” and functions to work with them (defined in include bpf/libbpf.h).
The C structs corresponding to the libbpf objects are:
- struct
bpf_object
- struct
bpf_program
- struct
bpf_map
These structs are for libbpf internal use, and you must use the API functions to interact with the opaque objects. Functions that work with an object are named after the struct, followed by a double underscore, and a descriptive name.
The libxdp API provides objects and functions for working with XDP programs as well as objects and functions for working with AF_XDP sockets, defined in xdp/libxdp.h.
The C structs corresponding to the libxdp objects are:
- struct
xdp_program
- struct
xdp_multiprog
- struct
xsk_umem
- struct
xsk_socket
Let’s look at practical usage of libxdp and libbpf.
In xdp_loader.c the function xdp_program__create()
is used to create
an xdp_program
object, in this case by loading the program from an ELF
file. The struct xdp_program
represents a single XDP program that can be
configured and attached to an XDP hook.
We use the filename
and progname
to identify the XDP program that we
want to create from the ELF file, by using struct xdp_program_opts
:
DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts); DECLARE_LIBXDP_OPTS(xdp_program_opts, xdp_opts, .open_filename = cfg->filename, .prog_name = cfg->progname, .opts = &opts);
When a struct xdp_program
is created from an ELF file, it keeps a
reference to the underlying bpf_object
which can be used to access all the
BPF programs and maps from the ELF file. We can use the function
xdp_program__bpf_obj(prog)
for getting the bpf_object
from the
xdp_program
.
XDP can also be offloaded to run in the NIC hardware (on some offload-capable NICs).
The XDP flag XDP_FLAGS_HW_MODE enables (requests) hardware offloading, and
is set with the long-option --offload-mode
. The loader code also needs to
use a more advanced libbpf API call bpf_prog_load_xattr()
, which allows us
to set the ifindex, as this is needed at load time to enable HW offloading.
There are some details on how to get hardware offloading working on Netronome’s Agilio SmartNICs in xdp_offload_nfp.org; for instance, the firmware needs updating to support eBPF.
As this lesson involves loading and selecting an XDP program that simply
drops all packets (via action XDP_DROP
), you will need to load it on a
real interface to observe what is happening. To do this, we establish a test
lab environment. In the testenv/ directory you will find a script
testenv.sh
that helps you setup a test lab based on veth
devices and
network namespaces.
E.g. run the script like:
$ sudo ../testenv/testenv.sh setup --name veth-basic02 Setting up new environment 'veth-basic02' Setup environment 'veth-basic02' with peer ip fc00:dead:cafe:1::2.
This results in the creation of an (outer) interface named: veth-basic02
.
You can test that the environment network is operational by pinging the peer
IPv6 address fc00:dead:cafe:1::2
(as seen in the script output).
The assignment is to manually load the compiled xdp program in the ELF OBJ
file xdp_prog_kern.o
, using the xdp_loader
program in this directory.
Observe the available options you can give the xdp_loader via --help
. Try
to select the program named xdp_drop_func
via --progname
, and observe
via ping that packets gets dropped.
Here are some example commands:
sudo ./xdp_loader --help sudo ./xdp_loader --dev veth-basic02 sudo ./xdp_loader --dev veth-basic02 --unload-all sudo ./xdp_loader --dev veth-basic02 --progname xdp_drop_func sudo ./xdp_loader --dev veth-basic02 --progname xdp_pass_func
When you load an XDP program on the interface visible on your host machine, it will operate on all packets arriving to that interface. And since packets that are sent from one interface in a veth pair will arrive at the other end, the packets that your XDP program will see are the ones sent from within the network namespace (netns). This means that when you are testing, you should do the ping from within the network namespace that were created by the script.
You can “enter” the namespace manually (via sudo ip netns exec veth-basic02
/bin/bash
) or via the script like:
$ sudo ../testenv/testenv.sh enter --name veth-basic02 # ping fc00:dead:cafe:1::1
To make this ping connectivity test easier, the script also has a ping
command that pings from within the netns:
$ sudo ../testenv/testenv.sh ping --name veth-basic02
You should note that, the cool thing about using netns as a testlab is that we can still “enter” the netns even-when XDP is dropping all packets.
To have faster access to the testenv.sh script, we recommend that you create
a shell alias (called t
). The testenv script even has a command helper
for this purpose:
$ ../testenv/testenv.sh alias Eval this with `eval $(../testenv/testenv.sh alias)` to create shell alias WARNING: Creating sudo alias; be careful, this script WILL execute arbitrary programs alias t='sudo /home/fedora/git/xdp-tutorial/testenv/testenv.sh'
As pointed out, run:
eval $(../testenv/testenv.sh alias)
You should now be able to run testenv commands as t <command>
(e.g., t
ping
). All subsequent examples will use this syntax.
The testenv script will save the last used testenv name, so in most cases
you can skip the --name
parameter when running the script. If you don’t
specify a name when you run t setup
, a random name will be generated for
you.
You can have several active test environments at the same time, and you can
always select a specific one using the --name
parameter. Run t status
to
see the currently selected environment (i.e., the one that will be used if
you don’t specify one with --name
), as well as the list of all currently
active environments.
Add a new program section “xdp_abort” in xdp_prog_kern.c that uses
(returns) the XDP action XDP_ABORTED
(and compile via make
). Load this
new program, e.g. similar to above:
sudo ./xdp_loader --dev veth-basic02 --unload-all sudo ./xdp_loader --dev veth-basic02 --progname xdp_abort_func
Lesson: XDP_ABORTED is different from XDP_DROP, because it triggers the
tracepoint named xdp:xdp_exception
.
While pinging from inside the namespace, record this tracepoint and observe these records. E.g with perf like this:
sudo perf record -a -e xdp:xdp_exception sleep 4 sudo perf script