Crash with ipv6 #9
Comments
|
Not sure if that is IPv6 specific. |
|
same test, same platform, over ipv4, does work, but yes, could be something else. |
|
Perhaps a firewall preventing IPv6 UDP connectivity? |
|
no firewall in play here. |
|
@richb-hanover - you using ipv6? |
|
My configuration works fine for IPv4. I can make a crash happen when I use IPv6...
Test connectivity using address found from Server log Client log |
|
Tested with v0.0.10-testing and it seems to work fine with IPv6 (public IPv6 or local IPv6) Server: Ubuntu 22.04 Proxmox PVE 8.0 LxC container (Intel N100 mini PC connected to RT-AX86U 2.5G LAN port)
|
|
I re-ran my experiment at #9 (comment) ensuring that both server and client were the current commit - a89abd2. It ran successfully without an error/crash. I think this issue can be closed. |
|
Hmm, my previous test was done using local network test. Today I am trying to test over the internet and it has the same problem now. IPv4 passed. IPv6 failed. ping and iperf3 over IPv6 are okay. crusader v0.0.10-testing Server: OpenWRT 23.05 Proxmox PVE 8.0 virtual router (Intel N100 mini PC) -- this is on network 1 (one public IPv4 address and /56 IPv6) Client: Windows 11 x64 using wireless (Acer Windows 11 laptop with Intel AX201 WiFi 6 adapter), using wireless connection through Asus RT-AX86U -- this is on network 2 (another public IPv4 address, another /56 IPv6) The OpenWRT firewall rule should be correct. IPv4 Test results:
|
|
Just wondering if you can test over the internet to see if you can reproduce the issue or not. Thanks. |
|
Two thoughts:
|
|
@mcuee Was both UDP and TCP forwarded? There seems to be some problem with IPv6 UDP here. |
In this paticular case, I am running crusader directly under OpenWRT router. Therefore I am not using port forwarding, but rather using Firewall -- Traffic Rules. And yes both TCP and UDP are allowed. |
|
If you feel like investigating you could try to run Wireguard and see which directions send or receives UDP traffic. |
You mean to say Wireshark, right? I will need to learn Wireshark first. Haha. I will see what I can do. |
|
Yeah I meant Wireshark =P |
|
Simple things first -- to use the debug version and get the trace. |
I am using a Proxmox PVE 8.0 LxC container as the client this time. crusader v0.0.10-testing Here is the output from tcpdump. Hopefully it will give some hints about the potential issue. More detailed tcpdump output using |
|
Just wondering if you can test over the internet again to see if the issue exists on your side or not. Thanks. |
|
My setup is a bit strange that the two home networks are actually sharing the same upstream ONT and a smart switch. This is the reason I can only test either download or upload but not both. ISP ONT -- TP-Link TL-SG105E smart switch --> two networks |
|
@mcuee I am sorry but I was not paying attention to github since march... |
|
@mcuee Could you try out https://github.com/Zoxc/crusader/tree/udp-spawn on the server and see if it helps? |
Are you testing over the internet? I am not able to test crusader over the internet with IPv6 now. Hopefully there is a public Crusader server in the future. Testing over the local network has no issues (both the server and client has public IPv6 addresses).
|
|
@mcuee - No, I don't have an internet test setup yet. I'm looking at how to dockerize the server for minimal hassle (then I have to dragoon one of my VPS servers into running it...) |
|
@Zoxc and @richb-hanover I am able to test Crusader over the Internet using IPv6 today and it works fine. Using 0.012 release. Server: OpenWRT 23.05 x86_64 So I consider this issue closed. |
|
(and great news about ipv6, thx!) |
That would be great. @richb-hanover is probably more experienced in hosting servers. I am relatively new to Open Source networking related projects myself even though I am an experienced Open Source Projects on the testing and technical support side. I will happy be able to have access to and even manage one or two servers. I have lost the abitlity to have two independent networks with my ISP with the new 5Gbps Fibre internet plan (Singapore ISPs are pushing for 3Gbps/5Gbps/10Gbps plans, because the government wants 50% of the household to be on 10Gbps plan in 2028. Now I have only single public IPv4 address and single /64 IPv6 prefix delegation. Previously I had two public IPv4 addresses and two /56 IPv6 IPv6 prefix delegations with my 1Gbps Fibre Internet connections. |
|
Maybe this issue can be closed. Thanks. |
|
One successful IPv6 test using @dtaht's public Crusader test server (wired connection).
|
|
that is really miserable latency. But it is probably coming from somewhere upstream of you. A simultaneous mtr would show where. I also, on your behalf, just put up a 4Gbps capable server at singapore.starlink.taht.net |
|
this is what my starlink looks like to Singapore:
crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink.taht.net |
|
Thanks a lot for the help. Somehow the server does not work for me. IPv4 ping is very good but IPv6 ping is very bad. Bad IPv6 latency is probabl due to Singtel, my ISP. It has similar problem with another main local ISP M1 (IPv4 is good but IPv6 is bad). But I can not carry out the test with either IPv4 or IPv6.
|
|
Info about my ISP - Singtel (Singapore Telecom) You can see different routing for IPv4 and IPv6. Using the same IPv4/IPv6 capable wireless network. |
|
IPv4/IPv6 traceroute to flent-fremont.bufferbloat.net. |
|
IPv4/IPv6 traceroute to Dallas.starlink.taht.net. |
|
mtr under Ubuntu 24.04 Intel N100 mini PC, very much similar to Windows machine. |
|
I filled a bug with linode |
|
and it was an incorrect DNS record, my bad! 2600:3c15::f03c:95ff:fe7e:75a2 it is now! try again in 15 minutes or so |
|
IPv4 wired test first using my Intel N100 mini PC with Ubuntu 24.04 Linux. Very fast server. Nice. Somehow Linux chooses to use IPv4 and not IPv6 by default.
|
|
IPv6 wired test -- the server is still good. Nice.
|
Guess the DNS record for IPv6 may need some time to update. |
|
It makes me grumpy to see 8ms of latency at 4gbit. can you slap on your Linux box and try again? |
|
Here you go.
|
|
But I am not exactly sure if the tc rules work well or not. Ir seems to only limit the upload bandwith. BTW, the CPU used in the mini PC running Ubuntu 24.04 is the low powrer Intel N100 CPU. Edit: this is expected as tc only limits upload bandwidth (egress bandwidth). |
|
Just to make sure the tc rule is working. |
|
Using wondershaper.
|
|
if you want to shape inbound w/cake try the sqm-scripts. Perversely nice to see wonder shaper struggling. the more frequent latency spikes are probably tail loss. tc -s qdisc show will give you stats |
|
I just enabled ECN on Singapore. sysctl -w net.ipv4.tcp_ecn=1 on your machine and w/cake try crusader. All the latency spikes should vanish. |
|
Question: I thought crusader uses UDP, does it really evaluate ECN at all? |
|
I just clarified the README in this PR for 0.3 documentation a1e38cc Yes - the throughput measurements use TCP |
|
Hi Rich, Thanks for clearing this up! (One reason less though to take a packet capture from a crusader run though...) |
|
DNS record for IPv6 is now correct. |
@dtaht IPv4 test result
IPv6 test result
|
|
BTW, I learned something new today. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
oot@lqos:~# crusader test --stream-stagger 5 --streams 2 --load-duration 10 fd77::2
Connected to server [fd77::2]:35481
thread 'main' panicked at 'called
Result::unwrap()on anErrvalue: "Unable to measure latency to server"', crusader-lib/src/test.rs:1304:10stack backtrace:
0: 0x7fdf8f501ddd -
1: 0x7fdf8f53bc0c -
2: 0x7fdf8f4fc771 -
3: 0x7fdf8f503335 -
4: 0x7fdf8f503056 -
5: 0x7fdf8f5038c6 -
6: 0x7fdf8f5037b7 -
7: 0x7fdf8f502294 -
8: 0x7fdf8f5034e9 -
9: 0x7fdf8f388103 -
10: 0x7fdf8f3881f3 -
11: 0x7fdf8f3e6c8a -
12: 0x7fdf8f38e850 -
13: 0x7fdf8f392c63 -
Aborted (core dumped)
The text was updated successfully, but these errors were encountered: