disocvery: Rework mechanism to not interfere with network stack

[xla]

It has been shown that the current discovery design is not ideal and can interfere with the memberships protocol of handling active and passive sets see #629 (comment) #696 (comment)

• drop current stream based discovery in favour of static set of bootstrap nodes
• Implement Protocol reconnect Protocol: Reconnect #211
• design pull based discovery

[kim]

The formulation as a stream is not entirely wrong, but somewhat misleading. The consumption of that stream is wrong, though. Consider:

After the initial bootstrapping, new peers are discovered via the membership protocol. In case we run out of reachable peers, we'd like to re-initiate the bootstrapping process (ie. reconnect). For this to work, we would either need to call some sort of reset on the datastructure containing the bootstrap nodes -- or keep using a stream, which, in the case of a static list, would simply cycle. The stream abstraction can also encapsulate dynamic discovery (eg. (m)DNS) more naturally (it wouldn't need to be Sync, I think).

There currently is a trait Discovery, which we never actually make useful use of:

pub trait Discovery {
    type Addr;
    type Stream: futures::Stream<Item = (PeerId, Vec<Self::Addr>)> + Send;

    fn discover(self) -> Self::Stream;
}

We could reformulate this as:

pub trait Discovery {
    type Addr;
    type Stream: futures::Stream<Item = (PeerId, Vec<Self::Addr>)> + Send;

    fn discover(&self) -> Self::Stream;
}

such that the protocol stack calls discover whenever it wants a (lazy) list of peers to connect to.

I'm not sure this would make things clearer, though, because ultimately this is isomorphic to just passing the stream.

tl;dr The only required change is to pull from the disco stream only when needed, which allows the Static discovery to become infinitary.