Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Performance drop routing 1kHz publications since 0.11.0 #279

Closed
franz6ko opened this issue Jul 17, 2024 · 5 comments
Closed

[Bug] Performance drop routing 1kHz publications since 0.11.0 #279

franz6ko opened this issue Jul 17, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@franz6ko
Copy link

Describe the bug

This is related to this other bug in zenoh_plugin_ros2dds here.

The solution for me was the same, rolling back to a previous version. In this case 0.10.0-rc-1

To reproduce

Updated to last version.
Check rate of high speed messages (previously 1000Hz dropped to 200Hz)

System info

Ubuntu 20.04

@franz6ko franz6ko added the bug Something isn't working label Jul 17, 2024
@JEnoch
Copy link
Member

JEnoch commented Jul 17, 2024

Thanks for this feedback.
Could you please detail the scenario in which you see such performance drop ?
Is it some kind of throughput test ? With which size of payload ?

@franz6ko
Copy link
Author

Hi @JEnoch,

Yes, the test was the same I explain in this other issue but in this case between the release 1.0.0-alpha.4 and the 0.10.0-rc-1.

In this case, a 1000Hz topic will drop to around 200Hz on 1.0.0-alpha.4 but would stay at 1000Hz on 0.10.0-rc-1.

I'm using it with ROS2 Foxy on Ubuntu 20.04. The topic message is basic structure holding some floats.

@JEnoch JEnoch changed the title [Bug] Massive performance drop on lateast release 1.0.0-alpha.4 [Bug] Performance drop routing 1kHz publications since 0.11.0 Jul 18, 2024
@JEnoch
Copy link
Member

JEnoch commented Jul 19, 2024

Probably the same cause than explained in eclipse-zenoh/zenoh-plugin-ros2dds#192 (comment) .
The issue is caused by some weird interaction between tokio and CycloneDDS.
Investigation in progress...

@JEnoch
Copy link
Member

JEnoch commented Aug 28, 2024

Zenoh has now been fixed (in branch dev/1.0.0) to adapt its batching mechanism to the tokio behaviour:
eclipse-zenoh/zenoh#1335
I can confirm that with default configuration (i.e. without configuring the express flag for publishers) the routing of 1kHz publications is no longer suffering performance drop.

This repo will be sync to use the fixed Zenoh version in branch dev/1.0.0 in the next few days.

@JEnoch JEnoch closed this as completed Aug 28, 2024
@franz6ko
Copy link
Author

Great to hear!! Thanks for your support. I'm happy to have contributed to this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants