[FRR] L3 EVPN is broken with latest FRR 10.0.1 upgrade.

[dgsudharsan]

Description

L3 EVPN is broken with FRR 10.0.1 upgrade. Adding the below configuration and performing config reload/reboot results in vni in the vrf missing in configuration files and thus resulting in type 5 routes not getting programmed in the vrf

Sample configuration. Add this and perform config reload. The vni field will disappear after config reload.

vrf Vrf1
vni 100
exit-vrf
!
router bgp 65001
no bgp suppress-duplicates
neighbor 12.0.0.2 remote-as 65001
neighbor 13.0.0.2 remote-as 65001
!
address-family ipv4 unicast
network 11.0.0.1/32
exit-address-family
!
address-family l2vpn evpn
neighbor 12.0.0.2 activate
neighbor 13.0.0.2 activate
advertise-all-vni
advertise-svi-ip
exit-address-family
exit
!
router bgp 65001 vrf Vrf1
no bgp suppress-duplicates
bgp bestpath as-path multipath-relax
!
address-family ipv4 unicast
redistribute connected
exit-address-family
!
address-family l2vpn evpn
advertise ipv4 unicast
exit-address-family
exit

Steps to reproduce the issue:

Describe the results you received:

Describe the results you expected:

Output of show version:

(paste your output here)

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

[dgsudharsan]

@sudhanshukumar22 @hasan-brcm Can you please investigate this?

[sudhanshukumar22]

@dgsudharsan
: Have you configured the below commands using click commands or using vtysh ? Also, I would like to know the routing mode, is it unified or split mode ?
vrf Vrf1
vni 100
exit-vrf

I see that we have a click command to configure vrf to vni map, but that command is not going to vtysh.
root@sonic:# show vxlan vlanvnimap
+---------+-------+
| VLAN | VNI |
+=========+=======+
| Vlan100 | 100 |
+---------+-------+
Total count : 1
root@sonic:# config vrf add_vrf_vni_map Vrf1 100
root@sonic:~# vtysh

Hello, this is FRRouting (version 10.0.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

sonic# exit
root@sonic:# vtysh -c "show running-config" | grep -i vrf
vrf Vrf1
exit-vrf
vrf Vrf2
exit-vrf
root@sonic:# vtysh -c "show running-config" | grep -i vrf -A3 -B3
!
ip route 50.1.1.1/32 64.1.1.2 tag 1
!
vrf Vrf1
ip nht resolve-via-default
ipv6 nht resolve-via-default
exit-vrf
!
vrf Vrf2
ip nht resolve-via-default
ipv6 nht resolve-via-default
exit-vrf
!
router bgp 65100
bgp router-id 10.1.0.1

[sudhanshukumar22]

@dgsudharsan , can you attach a working config_db.json(from 854 image)

[bradh352]

By L3 VPN, does that mean an L3 IRB interface that participates in the VXLAN EVPN? I've never gotten that to work, I was under the assumption it was unsupported in community SONiC as per #9489 where it says L3 is not supported yet.

[dgsudharsan]

@sudhanshukumar22 Here is the evpn part of the configuration

    "VRF": {
        "Vrf1": {
            "vni": "500200"
        }
    },
    "VXLAN_EVPN_NVO": {
        "my-nvo": {
            "source_vtep": "vtep101032"
        }
    },
    "VXLAN_TUNNEL": {
        "vtep101032": {
            "src_ip": "10.1.0.32"
        }
    },
    "VXLAN_TUNNEL_MAP": {
        "vtep101032|map_50020_Vlan20": {
            "vlan": "Vlan20",
            "vni": "50020"
        },
        "vtep101032|map_76543_Vlan69": {
            "vlan": "Vlan69",
            "vni": "76543"
        },
        "vtep101032|map_500100_Vlan100": {
            "vlan": "Vlan100",
            "vni": "500100"
        },
        "vtep101032|map_500101_Vlan101": {
            "vlan": "Vlan101",
            "vni": "500101"
        },
        "vtep101032|map_500200_Vlan200": {
            "vlan": "Vlan200",
            "vni": "500200"
        }
    },

    "LOOPBACK_INTERFACE": {
        "Loopback0": {},
        "Loopback0|10.1.0.32/32": {}
    },

SONiC doesn't push FRR configuration when configuring maps. EVPN is configured in split mode. Here is the FRR configuration. Please add these configurations to the device and reboot. vni will not appear in under vrf of frr configuraiton

!
vrf Vrf1
 vni 500200
exit-vrf
!
router bgp 65000
 bgp router-id 10.1.0.32
 bgp graceful-restart restart-time 300
 bgp graceful-restart
 bgp graceful-restart preserve-fw-state
 neighbor 30.0.0.2 remote-as 65000
 neighbor 40.0.0.3 remote-as 65000
 !
 address-family ipv4 unicast
  network 10.1.0.32/32
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor 30.0.0.2 activate
  neighbor 40.0.0.3 activate
  advertise-all-vni
  advertise-svi-ip
  vni 500100
   rd 10.1.0.32:100
  exit-vni
  vni 500101
   rd 10.1.0.32:101
  exit-vni
  vni 50020
   rd 10.1.0.32:20
  exit-vni
 exit-address-family
exit
!
router bgp 65000 vrf Vrf1
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
  rd 10.1.0.32:200
 exit-address-family
exit
!

[sudhanshukumar22]

config_db_evpn.json
I have uploaded the config_db.json for EVPN configuration. I see that after config reload, even in 854, the VNI config is not present in vtysh. As we can see below
root@sonic:~# vtysh

Hello, this is FRRouting (version 8.5.4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

sonic# show running-config
Building configuration...

Current configuration:
!
frr version 8.5.4
frr defaults traditional
hostname sonic
log syslog informational
log facility local4
fpm address 127.0.0.1
no fpm use-next-hop-groups
agentx
no service integrated-vtysh-config
!
password zebra
enable password zebra
!
vrf Vrf1
ip route 22.22.22.22/32 blackhole
exit-vrf
!
router bgp 1 vrf Vrf1
bgp router-id 1.1.1.1
bgp log-neighbor-changes
no bgp default ipv4-unicast
bgp bestpath as-path ignore
bgp bestpath compare-routerid
neighbor 46.1.1.3 remote-as external
!
address-family ipv4 unicast
redistribute static
neighbor 46.1.1.3 activate
maximum-paths 1
maximum-paths ibgp 1
exit-address-family
exit
!
router bgp 1
bgp router-id 1.1.1.1
bgp log-neighbor-changes
no bgp default ipv4-unicast
bgp bestpath as-path ignore
bgp bestpath compare-routerid
neighbor 64.1.1.2 remote-as external
!
address-family ipv4 unicast
redistribute connected
neighbor 64.1.1.2 activate
maximum-paths 1
maximum-paths ibgp 1
exit-address-family
!
address-family l2vpn evpn
neighbor 64.1.1.2 activate
advertise-all-vni
exit-address-family
exit
!
ip nht resolve-via-default
!
ipv6 nht resolve-via-default
!
end
sonic# exit

So, it is not a 10.0.1 regression, as it was not working in 8.5.4 also. I have also attached the complete config_db_evpn.json file.

[sudhanshukumar22]

@dgsudharsan