feat: provide ethtool-style features

[smira]

This is WIP.

[github-actions]

This PR is stale because it has been open 45 days with no activity.

[randomizedcoder]

This is great!

Summary
Do you think we could please add:

• Ring configuration: rx,tx rx-mini, tx-mini, rx-jumbo, tx-jumbo

• Channel configuration: rx, tx, other, combined

Ring Configuration
Really interested in being able to configure the RX and TX ring sizes. I think all drivers support "rx/tx" and some support the mini/jumbo variants

       ethtool -g|--show-ring devname

       ethtool -G|--set-ring devname [rx N] [rx-mini N] [rx-jumbo N] [tx N] [rx-buf-len N] [tcp-data-split auto|on|off] [cqe-size N] [tx-push N] [rx-push N] [tx-push-buf-len N]

For example, this is configuring a NIC on NixOS machine:

[das@hp1:~]$ ethtool --show-ring enp3s0f0
Ring parameters for enp3s0f0:
Pre-set maximums:
RX:			4096
RX Mini:		n/a
RX Jumbo:		n/a
TX:			4096
TX push buff len:	n/a
Current hardware settings:
RX:			256                   <---- default is small
RX Mini:		n/a
RX Jumbo:		n/a
TX:			256
RX Buf Len:		n/a
CQE Size:		n/a
TX Push:		off
RX Push:		off
TX push buff len:	n/a
TCP data split:		n/a

[das@hp1:~]$ ethtool --driver enp3s0f0
driver: e1000e
version: 6.11.8
firmware-version: 5.6-2
expansion-rom-version: 
bus-info: 0000:03:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes


[das@hp1:~]$ sudo ethtool --set-ring enp3s0f0 rx 4096 tx 4096 

[das@hp1:~]$ ethtool --show-ring enp3s0f0
Ring parameters for enp3s0f0:
Pre-set maximums:
RX:			4096
RX Mini:		n/a
RX Jumbo:		n/a
TX:			4096
TX push buff len:	n/a
Current hardware settings:
RX:			4096              <------ yay
RX Mini:		n/a
RX Jumbo:		n/a
TX:			4096
RX Buf Len:		n/a
CQE Size:		n/a
TX Push:		off
RX Push:		off
TX push buff len:	n/a
TCP data split:		n/a

Another machine with a better NIC looks like:

[das@hp4:~]$ ethtool --show-ring  enp1s0f0np0
Ring parameters for enp1s0f0np0:
Pre-set maximums:
RX:			8160
RX Mini:		n/a
RX Jumbo:		n/a
TX:			8160
TX push buff len:	n/a
Current hardware settings:
RX:			512
RX Mini:		n/a
RX Jumbo:		n/a
TX:			512
RX Buf Len:		n/a
CQE Size:		n/a
TX Push:		off
RX Push:		off
TX push buff len:	n/a
TCP data split:		n/a


[das@hp4:~]$ ethtool --driver enp1s0f0np0
driver: i40e
version: 6.11.8
firmware-version: 9.00 0x8000cae6 21.5.9
expansion-rom-version: 
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Channel configuration
The other very important setting is controlling how many "channels" or queues there are. e.g. This controls how many IRQs there are.

       ethtool -l|--show-channels devname

       ethtool -L|--set-channels devname [rx N] [tx N] [other N] [combined N]

e.g. On a system with 8 cores, the Intel driver automagically configures to use x8 IRQs. If the system had more cores, it would have more IRQs. Usually on a modern system with many cores, you want to tune this down.

[das@hp4:~]$ ethtool --show-channels enp1s0f0np0
Channel parameters for enp1s0f0np0:
Pre-set maximums:
RX:		n/a
TX:		n/a
Other:		1
Combined:	8
Current hardware settings:
RX:		n/a
TX:		n/a
Other:		1
Combined:	8

See also:
https://access.redhat.com/sites/default/files/attachments/20150325_network_performance_tuning.pdf

https://sysadmin.miniconf.org/2016/lca2016-jamie_bainbridge-network_performance_tuning.html#1

Really in the longer term we would want to do some IRQ pinning, but that's probably comes later
https://access.redhat.com/solutions/2144921