-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[dash]: Add DPU kvm topology guidance (#1676)
- Loading branch information
Showing
1 changed file
with
93 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,12 @@ | |
- [4 Dataflow](#4-dataflow) | ||
- [4.1 Data plane](#41-data-plane) | ||
- [4.2 Control plane](#42-control-plane) | ||
- [5 KVM topology](#5-kvm-topology) | ||
- [5.1 Single device mode testbed](#51-single-device-mode-testbed) | ||
- [5.1.1 Create DPU testbed](#511-create-dpu-testbed) | ||
- [5.1.2 Connect to the DPU](#512-connect-to-the-dpu) | ||
- [5.1.3 gnmi client access](#513-gnmi-client-access) | ||
- [5.2 DPU with VPP NPU testbed](#52-dpu-with-vpp-npu-testbed) | ||
|
||
# 1 Motivation | ||
|
||
|
@@ -87,3 +93,90 @@ graph TD | |
|
||
In the physical SmartSwitch, configuration is forwarded via GNMI in the NPU. So, in the virtual SONiC environment, the SWSS module is capable of receiving configuration from an external GNMI service through the management port, eth-midplane. However, in the single device mode, the GNMI service can also be run within the KVM and directly forward the configuration to the local SWSS module. | ||
|
||
# 5 KVM topology | ||
|
||
This section describes the steps to setup a virtual DPU testbed, please check a [pre-learning document](https://github.com/sonic-net/sonic-mgmt/blob/master/docs/testbed/README.testbed.VsSetup.md) to setup your environment and download the EOS container. Beginning the following steps, I assume you have got the cEOS image with the specific version(https://github.com/sonic-net/sonic-mgmt/blob/master/ansible/group_vars/all/ceos.yml) and sonic-mgmt containers. | ||
|
||
|
||
- The normal SONiC KVM image without DPU dataplane can be downloaded from: https://sonic-build.azurewebsites.net/api/sonic/artifacts?branchName=master&platform=vs&target=target/sonic-vs.img.gz | ||
- The DPU SONiC KVM image with dataplane will be released at the next stage | ||
|
||
## 5.1 Single device mode testbed | ||
|
||
## 5.1.1 Create DPU testbed | ||
|
||
```shell | ||
|
||
# Login to your sonic-mgmt container | ||
ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt [email protected] | ||
|
||
# CD to the ansible directory | ||
cd /data/sonic-mgmt/ansible | ||
|
||
# Build the DPU testbed | ||
./testbed-cli.sh -t vtestbed.yaml -m veos_vtb add-topo vms-kvm-dpu password.txt | ||
./testbed-cli.sh -t vtestbed.yaml -m veos_vtb deploy-mg vms-kvm-dpu veos_vtb password.txt | ||
|
||
``` | ||
|
||
If you are lucky, your testbed should pass all testcases via following commands | ||
|
||
``` shell | ||
# Login to your sonic-mgmt container | ||
ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt [email protected] | ||
|
||
# CD to the tests directory | ||
cd /data/sonic-mgmt/tests | ||
|
||
# Run testcases | ||
./run_tests.sh -u -n vms-kvm-dpu -d vlab-01 -m individual -e --skip_sanity -e --disable_loganalyzer -c dash/test_dash_vnet.py -f vtestbed.yaml -i ../ansible/veos_vtb -e "--skip_dataplane_checking" | ||
``` | ||
|
||
## 5.1.2 Connect to the DPU | ||
|
||
``` shell | ||
# Login to the DPU, the default password is `password` and the default mgmt IP address is `10.250.0.101` | ||
sshpass -p 'password' ssh -o TCPKeepAlive=yes -o ServerAliveInterval=30 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no [email protected] | ||
|
||
``` | ||
|
||
## 5.1.3 gnmi client access | ||
|
||
Once dash test cases are completed, we can find required certificates in sonic-mgmt container. | ||
|
||
``` | ||
# The certificates are in /tmp directory | ||
user@mgmt:~$ ls -l /tmp/e55648be-5a9c-49da-bcf7-645b1a926fb6 | ||
total 40 | ||
-rw-r--r-- 1 user user 99 Apr 12 05:52 extfile.cnf | ||
-rw------- 1 user user 1675 Apr 12 05:52 gnmiCA.key | ||
-rw-r--r-- 1 user user 1131 Apr 12 05:52 gnmiCA.pem | ||
-rw-r--r-- 1 user user 41 Apr 12 05:52 gnmiCA.srl | ||
-rw-r--r-- 1 user user 1017 Apr 12 05:52 gnmiclient.crt | ||
-rw-r--r-- 1 user user 907 Apr 12 05:52 gnmiclient.csr | ||
-rw------- 1 user user 1679 Apr 12 05:52 gnmiclient.key | ||
-rw-r--r-- 1 user user 1070 Apr 12 05:52 gnmiserver.crt | ||
-rw-r--r-- 1 user user 907 Apr 12 05:52 gnmiserver.csr | ||
-rw------- 1 user user 1675 Apr 12 05:52 gnmiserver.key | ||
``` | ||
|
||
Please install gnmi_cli_py from https://github.com/lguohan/gnxi/tree/master/gnmi_cli_py. | ||
|
||
Restart GNMI server with server certificates: | ||
|
||
``` | ||
# Run below commands on DUT | ||
docker exec gnmi supervisorctl stop gnmi-native | ||
docker exec gnmi bash -c "/usr/bin/nohup /usr/sbin/telemetry -logtostderr --port 50052 --server_crt /etc/sonic/telemetry/gnmiserver.crt --server_key /etc/sonic/telemetry/gnmiserver.key --ca_crt /etc/sonic/telemetry/gnmiCA.pem -gnmi_native_write=true -v=10 >/root/gnmi.log 2>&1 &" | ||
``` | ||
|
||
Follow below example to run GNMI client: | ||
|
||
``` | ||
# /root/update1 and /root/update2 are protobuf value for DASH table | ||
python2 /root/gnxi/gnmi_cli_py/py_gnmicli.py --timeout 30 -t 10.0.0.88 -p 50052 -xo sonic-db -rcert /root/gnmiCA.pem -pkey /root/gnmiclient.key -cchain /root/gnmiclient.crt -m set-update --xpath /APPL_DB/localhost/DASH_APPLIANCE_TABLE[key=123] /APPL_DB/localhost/DASH_VNET_TABLE[key=Vnet1] --value $/root/update1 $/root/update2 | ||
``` | ||
|
||
## 5.2 DPU with VPP NPU testbed | ||
|
||
TBD |