[Bug Report] 旁路由模式被ICMP redirect

[bohaoist]

Checks

• I have searched the existing issues
• I have read the documentation
• Is it your first time sumbitting an issue

Current Behavior

两台deban12云服务器A和B，处于一个云虚拟网络中，A的ip是192.168.8.89，B的ip是192.168.2.37，且它们都有公网ip，在A服务器上使用daed，B服务器设置默认网关和dns服务器为A服务器的ip，会收到ICMP redirect包，导致命中代理规则为direct的时候B的流量无法走A出去，而是走自己的公网ip出去。

Expected Behavior

期望B的所有流量都经过A

Steps to Reproduce

我使用daed，有两台deban12云服务器A和B，处于一个云虚拟网络中，A的ip是192.168.8.89，B的ip是192.168.2.37，且它们都有公网ip。我在A服务器上设置了透明代理dae，A的路由表如下：
root@A:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 100 0 0 eth0
169.254.169.254 192.168.0.1 255.255.255.255 UGH 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.240.0 U 100 0 0 eth0

同时,设置B的默认网关和dns服务器为192.168.8.89，路由表如下：
root@B:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.8.89 0.0.0.0 UG 0 0 0 eth0
169.254.169.254 192.168.0.1 255.255.255.255 UGH 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.240.0 U 100 0 0 eth0

但是测试发现,B上的流量还是会走自己的公网ip出口（使用curl myip.ipip.net查看ip还是B的公网ip），ping www.baidu.com发现会有From 192.168.8.89 (192.168.8.89): icmp_seq=1 Redirect Host(New nexthop: 192.168.0.1 (192.168.0.1))，发现会收到ICMP redirect包，导致以后的出口流量就不走代理了，如何才能实现让B的流量都走A呢？

配置如下：
Routing:
pname(NetworkManager, systemd-resolved, dnsmasq) -> must_direct
dip(geoip:private) -> direct
dip(geoip:cn) -> direct
domain(geosite:cn) -> direct
fallback: proxy

global:
{
"id": "Y3Vyc29yMQ",
"name": "global",
"selected": true,
"global": {
"logLevel": "trace",
"tproxyPort": 12345,
"allowInsecure": false,
"checkInterval": "30s",
"checkTolerance": "0s",
"lanInterface": [
"eth0"
],
"wanInterface": [
"auto"
],
"udpCheckDns": [
"dns.google.com:53",
"8.8.8.8",
"2001:4860:4860::8888"
],
"tcpCheckUrl": [
"http://cp.cloudflare.com",
"1.1.1.1",
"2606:4700:4700::1111"
],
"dialMode": "ip",
"tcpCheckHttpMethod": "HEAD",
"disableWaitingNetwork": false,
"autoConfigKernelParameter": true,
"sniffingTimeout": "100ms",
"tlsImplementation": "tls",
"utlsImitate": "chrome_auto",
"tproxyPortProtect": true,
"soMarkFromDae": 0
}
}

Environment

• Dae version (use dae --version):v0.8.0
• OS (e.g cat /etc/os-release):Debian GNU/Linux 12 (bookworm)
• Kernel (e.g. uname -a):Linux ecs-bffc 6.1.0-22-amd64 chore: add github actions #1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21) x86_64 GNU/Linux
• Others:

Anything else?

No response

[dae-prow]

Thanks for opening this issue!

[mzz2017]

https://github.com/daeuniverse/dae/blob/main/docs/en/user-guide/kernel-parameters.md

理论上这个 send_redirects 会被自动配置的，你可以检查一下这个的值是否正确？手动配置一下看会不会好

[bohaoist]

@mzz2017send_redirects 打开了，重新设置了B的默认网关，发现无法ping通，nslookup可以正常返回

[mzz2017]

@bohaoist send redirects 应该关闭

[bohaoist]

@mzz2017
已经设置net.ipv4.conf.all.send_redirects = 0，A服务器的配置如下：
root@A:~# sudo sysctl --system

• Applying /usr/lib/sysctl.d/50-pid-max.conf ...
• Applying /usr/lib/sysctl.d/99-protect-links.conf ...
• Applying /etc/sysctl.d/99-sysctl.conf ...
• Applying /etc/sysctl.conf ...
  kernel.pid_max = 4194304
  fs.protected_fifos = 1
  fs.protected_hardlinks = 1
  fs.protected_regular = 2
  fs.protected_symlinks = 1
  net.ipv4.ip_forward = 1
  net.ipv6.conf.all.forwarding = 1
  net.ipv4.conf.all.send_redirects = 0
  vm.swappiness = 0
  net.ipv4.tcp_max_tw_buckets = 5000
  net.ipv4.tcp_max_syn_backlog = 1024
  net.ipv4.ip_forward = 1
  net.ipv6.conf.all.forwarding = 1
  net.ipv4.conf.all.send_redirects = 0
  vm.swappiness = 0
  net.ipv4.tcp_max_tw_buckets = 5000
  net.ipv4.tcp_max_syn_backlog = 1024

在B机器上运行traceroute，第一跳到默认网关以后就没有了，全是 * * *：
root@B:~# traceroute www.baidu.com
traceroute to www.baidu.com (180.101.50.242), 30 hops max, 60 byte packets
1 192.168.8.89 (192.168.8.89) 0.168 ms 0.177 ms 0.169 ms
2 * * *
3 * * *
4 * * *
5 * * *

[pomhg]

建议给出完整topo图，没有掩码A和B是一个网段吗，A的默认网关还是192.168.0.1，如果不是一个网段AB间至少还有一个路由。还有旁路由的问题，direct流量没有snat会有非对称路由，需要在A上做snat

[bohaoist]

@pomhg A和B都是云上的服务器，它们是同一个网段，A的默认网关也是192.168.0.1。请问做了snat，dae还可以自动生效吗？