Initial implementations of the TPU Vortexor #3258
Conversation
lijunwangs
force-pushed
the
tpu_vortexor
branch
from
610493d
to
bf5ac84
Compare
lijunwangs
force-pushed
the
tpu_vortexor
branch
from
4e372b1
to
257e49a
Compare
| .help("IP address to bind the validator ports"), | ||
| ) | ||
| .arg( | ||
| Arg::with_name("dynamic_port_range") |
There was a problem hiding this comment.
This is similar to how it is done in the validator side which allows the user to specify the range for the tpu addresses.
| solana-logger = { workspace = true } | ||
|
|
||
| [lib] | ||
| crate-type = ["lib"] |
There was a problem hiding this comment.
Yes. The crate generate a bin solana-vortexor. It can also support generating a library which can be used by other crates to develop. For example, I see these
ls target/debug | grep vortex
libsolana_vortexor.d
libsolana_vortexor.rlib
solana-vortexor
solana-vortexor.d
lijunwangs
force-pushed
the
tpu_vortexor
branch
from
509dbad
to
f3ba6a6
Compare
| @@ -239,3 +245,46 @@ pub async fn make_client_endpoint( | |||
| .await | |||
| .expect("Test server should be already listening on 'localhost'") | |||
| } | |||
|
|
|||
| pub async fn check_multiple_streams( | |||
There was a problem hiding this comment.
This is moved from nonblocking::quic::test to here to be reused for other testing
|
Is there a design document or anything for this? |
lijunwangs
force-pushed
the
tpu_vortexor
branch
from
51b7492
to
a45a8b2
Compare
The motivation is to make TPU streaming, sig verification more scalable. The readme captures this: https://raw.githubusercontent.com/anza-xyz/agave/11908fbbcfab7194f954177c2da0646249ee308a/vortexor/Readme.md. I have set up a review meeting to go over this to give more context. The vortexor will retrieve the stake map via regular RPC service from configured validator -- the RPC can be either pairing validator or any other trusted RPC validator in the network supporting the RPC pub/sub service. The Vortexor does not need the private key to the validator. |
don't we need the private key for QUIC handshake ? |
A private key is used to sign the TLS certificate. But the client does not care about if the key is the same as the one of the validator advertising the TPU address (The vortexor address). The client code does not check that and we have no use case to enforce that check -- this is the same currently using --public-tpu-address. Without requiring using the same private key making it more secure and simplify deployment. |
How does the client prune its connection cache when it becomes too big? |
We randomly select a connection to prune -- the current connection cache is agnostic of the stake map. |
Problem
Framework for Solana TPU vortexor which allows core validator to offload TPU receiving, sigverify and deduplications.
Summary of Changes
This is the first installment of the change. This is the framework part. It starts the TPU and forward stream based on the CLI configurations. In future PRs we will have the sigverify, forwarder, subscription management. The code has no impact to the existing validator.
Added a unit test for starting the vortexor, send some transactions and stops it.
Fixes #