[BUG] 针对重启kube-ovn网络插件类型kubernetes集群所有节点场景，存在重启后EIP无法curl通异常（概率性出现，非必现），重启前可以curl通EIP

[author970]

Kube-OVN Version

v1.12.22

Kubernetes Version

v1.27.6

Operation-system/Kernel Version

5.10.0-136.12.0.86.4.hl202.x86_64

Description

重启前可以curl通EIP，重启kube-ovn网络插件类型kubernetes集群所有节点后，待集群所有kube-ovn组件正常running，无法curl通EIP（重启所有节点有一定概率触发此异常，非必现），异常与issue2749一样。

Steps To Reproduce

1. 重启集群所有节点，重启时间：2024-10-24 09:51:41；

2. 重启后待集群正常，外部通过EIP curl，发现EIP不通（重启所有节点有一定概率触发此异常，非必现）；

3. kubectl exec进入vpc网关pod vpc-nat-gw容器中，vpc-nat-gw容器中网卡net1@if10上无 多个EIP的弹性IP；
   

4. kubectl delete vpc网关pod使pod重新创建后，创建后pod vpc-nat-gw容器中网卡net1@if10上存在多个EIP的弹性IP，正常。
   

5. 搜索kube-ovn-controller pod日志"vpc_nat_"，显示如下：
   Search "vpc_nat_" (197 hits in 1 file)
   D:\文件资料\xxx\kube-ovn-controller.log.1 (197 hits)
   Line 49474: I1024 10:00:41.530660 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-x3xxhk7u
   Line 49475: I1024 10:00:41.530824 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-7jpvp699
   Line 49476: I1024 10:00:41.530843 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-r9zkzn3v
   Line 49477: I1024 10:00:41.530853 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-7bwa0m5z
   Line 49478: I1024 10:00:41.530860 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-8qfmsbkl
   Line 49479: I1024 10:00:41.530865 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-cu0kjnia
   Line 49480: I1024 10:00:41.530877 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-dvz0m5lr
   Line 49481: I1024 10:00:41.530881 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-pne0nuw5
   Line 49482: I1024 10:00:41.530889 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-yk2obf36
   Line 49483: I1024 10:00:41.531846 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-3wtszss5
   Line 49484: I1024 10:00:41.531868 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-4tyc1gw3
   Line 49485: I1024 10:00:41.531876 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-zzusfbel
   Line 49486: I1024 10:00:41.531885 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-daqwodb7
   Line 49487: I1024 10:00:41.531889 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-g3zbyw8r
   Line 49488: I1024 10:00:41.531901 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-v5w1k2j0
   Line 49494: I1024 10:00:41.531909 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-361x9nh3
   Line 49495: I1024 10:00:41.532043 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-4pjb6i2u
   Line 49496: I1024 10:00:41.532070 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-kia96no8
   Line 49497: I1024 10:00:41.532077 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-lbcwgylw
   Line 49498: I1024 10:00:41.532084 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-qm4lxmov
   Line 49499: I1024 10:00:41.532089 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-108njiom
   Line 49500: I1024 10:00:41.532095 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-a27scv76
   Line 49501: I1024 10:00:41.532101 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-uf3wqr1r
   Line 49502: I1024 10:00:41.532107 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-x8rddy7t
   Line 49503: I1024 10:00:41.532114 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-8t864sku
   Line 49504: I1024 10:00:41.532121 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-phot9clz
   Line 49505: I1024 10:00:41.532126 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-5szww1ks
   Line 49506: I1024 10:00:41.532139 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-86t3yqie
   Line 49507: I1024 10:00:41.532146 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-fzru7uvo
   Line 49510: I1024 10:00:41.532151 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-jvw39tvl
   Line 49511: I1024 10:00:41.532220 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-mqcmv0kp
   Line 49512: I1024 10:00:41.532244 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-pzl2hntv
   Line 49513: I1024 10:00:41.532255 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-27ti4824
   Line 49514: I1024 10:00:41.532269 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-2g9s6j9r
   Line 49515: I1024 10:00:41.532278 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-xs1gvx46
   Line 49517: I1024 10:00:41.532299 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-ujef7cwa
   Line 49518: I1024 10:00:41.532316 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-ummp3hw7
   Line 49520: I1024 10:00:41.532328 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-kwok61je
   Line 49521: I1024 10:00:41.532337 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-s16g1w4y
   Line 49522: I1024 10:00:41.532346 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-tu395xrm
   Line 49523: I1024 10:00:41.532354 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-v2ny3msu
   Line 49524: I1024 10:00:41.532362 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-yrpkufhr
   Line 49525: I1024 10:00:41.532368 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-cmx60tgx
   Line 49526: I1024 10:00:41.532377 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-e2m44kgt
   Line 49527: I1024 10:00:41.532383 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-g22bq8k8
   Line 49528: I1024 10:00:41.532390 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-gn9uve9h
   Line 49529: I1024 10:00:41.532398 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-hdlqbpyr
   Line 49530: I1024 10:00:41.532404 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-jrnpz0ci
   Line 49531: I1024 10:00:41.532410 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-xnqcw8n1
   Line 49532: I1024 10:00:41.532418 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-zlmwpqgp
   Line 49533: I1024 10:00:41.532424 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-5vq54jyx
   Line 49534: I1024 10:00:41.532431 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-9gfybji0
   Line 50605: I1024 10:00:42.151702 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
   Line 50607: E1024 10:00:42.151903 7 vpc_nat_gateway.go:189] process: addOrUpdateVpcNatGateway. err: error syncing 'eip-gn9uve9h': iptables nat gw not enable, requeuing
   Line 50733: I1024 10:00:43.152962 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
   Line 50734: E1024 10:00:43.152994 7 vpc_nat_gateway.go:189] process: addOrUpdateVpcNatGateway. err: error syncing 'eip-gn9uve9h': iptables nat gw not enable, requeuing
   Line 50746: I1024 10:00:45.152506 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
   Line 50747: E1024 10:00:45.152514 7 vpc_nat_gw_nat.go:241] error syncing 'eip-lbcwgylw': iptables nat gw not enable, requeuing
   Line 50748: I1024 10:00:45.152554 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-v5w1k2j0
   Line 50749: I1024 10:00:45.152573 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-yk2obf36
   Line 50751: I1024 10:00:45.152607 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-x3xxhk7u
   Line 50753: I1024 10:00:45.152631 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-7jpvp699
   Line 50754: I1024 10:00:45.152643 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-r9zkzn3v
   Line 50755: I1024 10:00:45.152656 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-7bwa0m5z
   Line 50756: I1024 10:00:45.152667 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-8qfmsbkl
   Line 50757: I1024 10:00:45.152679 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-cu0kjnia
   Line 50758: I1024 10:00:45.152690 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-dvz0m5lr
   Line 50759: I1024 10:00:45.152479 7 vpc_nat_gateway.go:91] finish establishing vpc-nat-gateway
   Line 50760: I1024 10:00:45.152701 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-pne0nuw5
   Line 50761: I1024 10:00:45.152712 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-yk2obf36
   Line 50769: I1024 10:00:45.152723 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-3wtszss5
   Line 50771: I1024 10:00:45.152880 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-4tyc1gw3
   Line 50772: I1024 10:00:45.152898 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-zzusfbel
   Line 50774: I1024 10:00:45.152908 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-daqwodb7
   Line 50775: I1024 10:00:45.152916 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-g3zbyw8r
   Line 50780: I1024 10:00:45.152924 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-v5w1k2j0
   Line 50783: I1024 10:00:45.152583 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-cmx60tgx
   Line 50784: I1024 10:00:45.152979 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-361x9nh3
   Line 50787: I1024 10:00:45.153007 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-cu0kjnia
   Line 50788: I1024 10:00:45.153093 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-dvz0m5lr
   Line 50789: I1024 10:00:45.153109 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-fzru7uvo
   Line 50790: I1024 10:00:45.153125 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-kia96no8
   Line 50791: I1024 10:00:45.153140 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-phot9clz
   Line 50792: I1024 10:00:45.153150 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-pne0nuw5
   Line 50793: I1024 10:00:45.153163 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-s16g1w4y
   Line 50794: I1024 10:00:45.153176 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-361x9nh3
   Line 50795: I1024 10:00:45.153007 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-4pjb6i2u
   Line 50796: I1024 10:00:45.153208 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-kia96no8
   Line 50797: I1024 10:00:45.153223 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-lbcwgylw
   Line 50798: I1024 10:00:45.153235 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-qm4lxmov
   Line 50799: I1024 10:00:45.153245 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-108njiom
   Line 50800: I1024 10:00:45.153254 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-a27scv76
   Line 50804: I1024 10:00:45.153187 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-8t864sku
   Line 50807: I1024 10:00:45.153341 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-9gfybji0
   Line 50810: I1024 10:00:45.153264 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-uf3wqr1r
   Line 50811: I1024 10:00:45.153429 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-g22bq8k8
   Line 50813: I1024 10:00:45.153475 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-x8rddy7t
   Line 50815: I1024 10:00:45.153490 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-8t864sku
   Line 50816: I1024 10:00:45.153500 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-phot9clz
   Line 50818: I1024 10:00:45.153512 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-5szww1ks
   Line 50819: I1024 10:00:45.153471 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-x3xxhk7u
   Line 50820: I1024 10:00:45.153531 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-86t3yqie
   Line 50821: I1024 10:00:45.153545 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-x8rddy7t
   Line 50822: I1024 10:00:45.153558 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-xnqcw8n1
   Line 50823: I1024 10:00:45.153566 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-kwok61je
   Line 50827: I1024 10:00:45.153547 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-fzru7uvo
   Line 50828: I1024 10:00:45.153771 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-jvw39tvl
   Line 50829: I1024 10:00:45.153791 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-mqcmv0kp
   Line 50830: I1024 10:00:45.153799 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-pzl2hntv
   Line 50831: I1024 10:00:45.153808 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-27ti4824
   Line 50832: I1024 10:00:45.153815 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-2g9s6j9r
   Line 50833: I1024 10:00:45.153824 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-xs1gvx46
   Line 50844: I1024 10:00:45.153574 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-ujef7cwa
   Line 50847: I1024 10:00:45.153963 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-v2ny3msu
   Line 50849: I1024 10:00:45.153973 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-27ti4824
   Line 50851: I1024 10:00:45.153983 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-4pjb6i2u
   Line 50852: I1024 10:00:45.153993 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-ujef7cwa
   Line 50853: I1024 10:00:45.153997 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-5szww1ks
   Line 50854: I1024 10:00:45.154003 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-ummp3hw7
   Line 50855: I1024 10:00:45.154018 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-kwok61je
   Line 50856: I1024 10:00:45.154026 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-s16g1w4y
   Line 50857: I1024 10:00:45.154035 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-tu395xrm
   Line 50858: I1024 10:00:45.154042 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-v2ny3msu
   Line 50859: I1024 10:00:45.154047 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-yrpkufhr
   Line 50860: I1024 10:00:45.154054 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-cmx60tgx
   Line 50861: I1024 10:00:45.154060 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-e2m44kgt
   Line 50862: I1024 10:00:45.154066 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-g22bq8k8
   Line 51005: I1024 10:00:45.160532 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
   Line 51031: I1024 10:00:45.163101 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-gn9uve9h
   Line 51032: I1024 10:00:45.163126 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-gn9uve9h
   Line 51033: I1024 10:00:45.163145 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-hdlqbpyr
   Line 51034: I1024 10:00:45.163154 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-jrnpz0ci
   Line 51035: I1024 10:00:45.163163 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-xnqcw8n1
   Line 51036: I1024 10:00:45.163173 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-zlmwpqgp
   Line 51037: I1024 10:00:45.163181 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-5vq54jyx
   Line 51038: I1024 10:00:45.163191 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-9gfybji0
   Line 51039: I1024 10:00:45.163135 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-7jpvp699
   Line 51040: I1024 10:00:45.163206 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-86t3yqie
   Line 51041: I1024 10:00:45.163214 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-tu395xrm
   Line 51042: I1024 10:00:45.163223 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-daqwodb7
   Line 51043: I1024 10:00:45.163240 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-g3zbyw8r
   Line 51044: I1024 10:00:45.163252 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-jrnpz0ci
   Line 51045: I1024 10:00:45.163260 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-uf3wqr1r
   Line 51046: I1024 10:00:45.163267 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-108njiom
   Line 51047: I1024 10:00:45.163277 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-2g9s6j9r
   Line 51048: I1024 10:00:45.163285 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-4tyc1gw3
   Line 51049: I1024 10:00:45.163294 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-5vq54jyx
   Line 51050: I1024 10:00:45.163304 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-xs1gvx46
   Line 51051: I1024 10:00:45.163314 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-8qfmsbkl
   Line 51052: I1024 10:00:45.163321 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-r9zkzn3v
   Line 51053: I1024 10:00:45.163334 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-zlmwpqgp
   Line 51054: I1024 10:00:45.163341 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-zzusfbel
   Line 51055: I1024 10:00:45.163349 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-ummp3hw7
   Line 51056: I1024 10:00:45.163355 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-yrpkufhr
   Line 51057: I1024 10:00:45.163363 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-3wtszss5
   Line 51058: I1024 10:00:45.163371 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-7bwa0m5z
   Line 51059: I1024 10:00:45.163379 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-jvw39tvl
   Line 51060: I1024 10:00:45.163385 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-mqcmv0kp
   Line 51061: I1024 10:00:45.163392 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-qm4lxmov
   Line 51062: I1024 10:00:45.163399 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-a27scv76
   Line 51063: I1024 10:00:45.163408 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-e2m44kgt
   Line 51064: I1024 10:00:45.163416 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-hdlqbpyr
   Line 51065: I1024 10:00:45.163422 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-pzl2hntv
   Line 51076: I1024 10:00:45.231712 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51095: E1024 10:00:45.287490 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51096: E1024 10:00:45.287516 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51097: E1024 10:00:45.287545 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 51390: I1024 10:00:46.230485 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-lbcwgylw
   Line 51401: I1024 10:00:46.330325 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51425: E1024 10:00:46.430322 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51426: E1024 10:00:46.430338 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51427: E1024 10:00:46.430355 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 51507: I1024 10:00:48.430937 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51508: E1024 10:00:48.457488 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51509: E1024 10:00:48.457513 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51510: E1024 10:00:48.457543 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 51511: I1024 10:00:52.458104 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51512: E1024 10:00:52.479708 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51513: E1024 10:00:52.479724 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51514: E1024 10:00:52.479742 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 51518: I1024 10:01:00.480109 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51519: E1024 10:01:00.501763 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51520: E1024 10:01:00.501800 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51521: E1024 10:01:00.501834 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 51639: I1024 10:01:16.502776 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 51641: E1024 10:01:16.538024 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51642: E1024 10:01:16.538051 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 51643: E1024 10:01:16.538077 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 54122: I1024 10:01:36.538929 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 54125: E1024 10:01:36.564674 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 54126: E1024 10:01:36.564695 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 54127: E1024 10:01:36.564716 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 55287: I1024 10:01:56.565251 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
   Line 55290: E1024 10:01:56.591380 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 55291: E1024 10:01:56.591397 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
   Line 55292: E1024 10:01:56.591413 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
   Line 56820: I1024 10:02:16.592432 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h

Current Behavior

重启集群所有节点后，EIP存在概率无法访问。

Expected Behavior

重启集群所有节点后，EIP可正常访问。

[dosubot]

Hello, @author970! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

I found some similar issues related to EIP connectivity problems after restarting the kube-ovn network plugin:

1. [BUG] NatGateway ceases working after being restarted: This issue was related to conntrack entries not performing SNAT correctly after a NAT Gateway restart. The suggested solutions were to prevent serving traffic before routes and iptables rules are configured or to flush conntrack entries without SNAT/DNAT after routes and iptables rules are configured ^[1].

2. VPC Gateway loses connectivity as soon as an EIP gets attached to it: The workaround for this issue involved disabling ARP on the network interface before and after attaching the EIP, which seemed to address the connectivity problem ^[2].

3. EIP NAT not working for few IPs in subnet: It was suggested to use the release-1.12-mc branch or the master branch of kube-ovn, as many bugs related to ovn eip dnat snat fip were fixed in these versions ^[3].

These issues might provide some insights or potential solutions to the problem you're experiencing with EIP connectivity after a full cluster restart.

To continue talking to Dosu, mention @dosu.

[bobz965]

一次性重启集群所有节点么？

[author970]

一次性重启集群所有节点么？

是的

[bobz965]

跟这个bug应该也有点关系： #4690

[bobz965]

等 #4690 修复后，可以再看看

[kldancer]

这个错误现象是所有节点重启，nat-gw pod重建，kube-ovn控制器没能成功给给gw pod添加上eip。从日志上看，是想exec 进容器执行脚本报错了。这个报错是client-go报的错：err: unable to upgrade connection: container not found ("vpc-nat-gw")。暂时不清楚为什么会报这个错，因为从操作截图上看，是能够手动进入到nat-gw pod当中的。