Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transports/tcp: simplify IfWatcher integration #2813

Merged
merged 28 commits into from
Sep 10, 2022

Conversation

elenaf9
Copy link
Contributor

@elenaf9 elenaf9 commented Aug 11, 2022

Description

Update to if-watch version v2.0.0, which allows us to simplify our integration in the tcp transport quite a bit:

  • Since IfWatcher::new is not async anymore we do not need the IfWatch wrapper and related logic anymore
  • Use if_watch::IfWatcher for both runtimes. According to this discussion, this should be ok now: transports/quic: Add implementation based on quinn-proto #2289 (comment).
    However, I just saw now that for cfg(not(any(target_os = "ios", target_os = "linux", target_os = "macos", target_os = "windows"))) if-watch still uses async-io. Given that this won't apply for the vast majority of users I'd say this is acceptable. Alternative is to do feature request: expose tokio vs async-global-executor feature flags if-watch#21 before continuing here.
  • Remove InAddr and instead manage the IfWatcher directly in the TcpListenStream. In case of InAddr::One directly push an TransportEvent::NewAddress into our event queue when creating the listener.

Related discussions: #2289 (comment).

Open Questions

  • We could remove InAddr completely and in case of InAddr::One just directly push an TransportEvent::NewAddress into our event queue when creating the listener. See elenaf9@c4f2124. But not sure if that really simplifies things. I guess for other transports that don't have whole port-reuse logic this might make more sense.
    Edit: Done now in 64a3165.

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • A changelog entry has been made in the appropriate crates

With if-watch `2.0.0` `IfWatcher::new` is not async anymore, hence the
`IfWatch` wrapping logic is obsolete.
- In case of `InAddr::One` we can use `self.listen_addr`.
- In case of `InAddr::Any` we can use `IfWatcher::iter`.
Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a really cool simplification!

transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
@mxinden
Copy link
Member

mxinden commented Aug 12, 2022

  • However, I just saw now that for cfg(not(any(target_os = "ios", target_os = "linux", target_os = "macos", target_os = "windows"))) if-watch still uses async-io. Given that this won't apply for the vast majority of users I'd say this is acceptable. Alternative is to do fix

This seems fine to me.

Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really great improvement!

I've added some stylistic comments.

transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
@thomaseizinger
Copy link
Contributor

  • We could remove InAddr completely and in case of InAddr::One just directly push an TransportEvent::NewAddress into our event queue when creating the listener. See elenaf9@c4f2124. But not sure if that really simplifies things. I guess for other transports that don't have whole port-reuse logic this might make more sense.

I've only just noticed now. I think that is a good idea!

@elenaf9 elenaf9 marked this pull request as ready for review August 20, 2022 07:34
@thomaseizinger
Copy link
Contributor

Note: I've edited your description so we don't actually close libp2p/if-watch#21 with merging this.

transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Outdated Show resolved Hide resolved
transports/tcp/src/lib.rs Show resolved Hide resolved
@mxinden
Copy link
Member

mxinden commented Aug 30, 2022

@elenaf9 let me know if you want me to take another look here.

@elenaf9 elenaf9 requested a review from mxinden August 30, 2022 07:55
@elenaf9 elenaf9 mentioned this pull request Aug 30, 2022
4 tasks
Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement!

Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Have you tested this pull request with both tokio and async-std as a runtime using a wildcard IP address?

If yes, this is ready to merge from my end.

let listener: TcpListener = socket.into();
let local_addr = listener.local_addr()?;

if local_addr.ip().is_unspecified() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we check if port is unspecified too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first thought as well. But while an unspecified IP results in one IP per (potentially future) interface, an unspecified port results in a single randomly chosen port.

@elenaf9 @melekes please correct me in case I am missing something here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an unspecified port results in a single randomly chosen port.

Well, I am not sure what the OS does when it can not find a unique port number that is free on all interfaces. I am guessing that it will choose a different port number per interface. Though still only one per interface.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first thought as well. But while an unspecified IP results in one IP per (potentially future) interface, an unspecified port results in a single randomly chosen port.

Yes exactly.

melekes added a commit to melekes/rust-libp2p that referenced this pull request Sep 5, 2022
@elenaf9
Copy link
Contributor Author

elenaf9 commented Sep 7, 2022

Have you tested this pull request with both tokio and async-std as a runtime using a wildcard IP address?

Yes, though on my local machine (linux). But our CI also tests both runtimes in the tcp test, which includes tests with wildcard addresses. But those tests only test the tcp transport directly and not within a swarm.
Lmk if you want me to add tests that integrate the TCP transport in the swarm.

@mxinden
Copy link
Member

mxinden commented Sep 9, 2022

Have you tested this pull request with both tokio and async-std as a runtime using a wildcard IP address?

Yes, though on my local machine (linux). But our CI also tests both runtimes in the tcp test, which includes tests with wildcard addresses. But those tests only test the tcp transport directly and not within a swarm. Lmk if you want me to add tests that integrate the TCP transport in the swarm.

I think this is fine as is. Thanks.

Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for thinking this all the way through!

elenaf9 added a commit to elenaf9/rust-libp2p that referenced this pull request Sep 9, 2022
See corresponding change in tcp transport: libp2p#2813.
@mxinden
Copy link
Member

mxinden commented Sep 10, 2022

CI failure is unrelated. #2869 released v0.48.0. Thus this pull request updates the version to v0.49.0. Problem is, that https://github.com/libp2p/test-plans still operates with v0.48.0.

As far as I can tell this is a chicken-and-egg problem. We can't merge with a green CI here as the Testground tests are failing. We can not merge a patch to libp2p/test-plans as the version of libp2p/rust-libp2p is not yet bumped to v0.49.0.

I don't think this is a big issue. Will need to think about how to solve it long term. //CC @laurentsenta in case you have an idea.

I will merge here. I will create a pull request against libp2p/test-plans next.

@mxinden mxinden merged commit 457fb51 into libp2p:master Sep 10, 2022
mxinden added a commit to mxinden/test-plans that referenced this pull request Sep 10, 2022
@elenaf9 elenaf9 deleted the tcp/refactor-inaddr branch September 10, 2022 14:49
laurentsenta pushed a commit to mxinden/test-plans that referenced this pull request Sep 23, 2022
laurentsenta pushed a commit to mxinden/test-plans that referenced this pull request Sep 23, 2022
mxinden added a commit to libp2p/test-plans that referenced this pull request Sep 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants