gossip-protocol-spec.md

Gossip protocol

Solana nodes communicate with each other and share data using the gossip protocol. They send messages in a binary form which need to be deserialized. There are 6 types of messages:

• pull request
• pull response
• push message
• prune message
• ping
• pong

Each message contains data specific to its type: values that nodes share between them, filters, pruned nodes, etc. Nodes keep their data in Cluster Replicated Data Store (crds), which is synchronized between nodes via pull requests, push messages and pull responses.

Tip

Naming conventions used in this document

• Node - a validator running the gossip
• Peer - a node sending or receiving messages from the current node we're talking about
• Entrypoint - the gossip address of the peer the node will initially connect to
• Origin - node, the original creator of the message
• Cluster - a network of validators with a leader that produces blocks
• Leader - node, the leader of the cluster in a given slot
• Shred - is the smallest portion of block produced by a leader
• Shred version - a cluster identification value
• Fork - a fork occures when two different blocks got chained to the same parent block (e.g. next block is created before the previous one was completed)
• Epoch - it is a length of certain amount of blocks (slots) in which the validator schedule is defined
• Slot - the period of time for which each leader ingests transactions and produces a block

Message format

Each message is sent in a binary form with a maximum size of 1232 bytes (1280 is a minimum IPv6 TPU, 40 bytes is the size of IPv6 header and 8 bytes is the size of the fragment header).

Data sent in the message is serialized from a Protocol type, which can be one of:

Enum ID	Message	Data	Description
0	pull request	CrdsFilter, CrdsValue	sent by node to ask for new information
1	pull response	Pubkey, CrdsValuesList	response to a pull request
2	push message	Pubkey, CrdsValuesList	sent by node to share its data
3	prune message	Pubkey, PruneData	sent to peers with a list of origin nodes that should be pruned
4	ping message	Ping	a ping message
5	pong message	Pong	response to a ping

block-beta
    block
        columns 3
        block
            columns 1
            b["Pull request"]
            c["Pull response"]
            d["Push message"]
            e["Prune message"]
            f["Ping message"]
            g["Pong message"]
        end
        right<["Serialize"]>(right)
        a["Packet\n(1232 bytes)"]
    end

Loading

Solana client Rust implementation

enum Protocol {
    PullRequest(CrdsFilter, CrdsValue),
    PullResponse(Pubkey, Vec<CrdsValue>),
    PushMessage(Pubkey, Vec<CrdsValue>),
    PruneMessage(Pubkey, PruneData),
    PingMessage(Ping),
    PongMessage(Pong)
}

Type definitions

Fields described in the tables below have their types specified using Rust notation:

• u8 - 1 byte of unsigned data (8-bit unsigned integer)
• u16 - 16-bit unsigned integer
• u32 - 32-bit unsigned integer, and so on...
• [u8] - dynamic size array of 1-byte elements
• [u8; 32] - fixed size array of 32 elements, with each element being 1 byte
• [[u8; 64]] - a two-dimensional array containing arrays of 64 1-byte elements
• b[u8] - a bit vector containing 1-byte elements
• u32 | None - an option type, meaning an element is either u32 (in this case) or None
• (u32, [u8, 16]) - a tuple that contains two elements - one is a 32-bit integer, the second one is a 16-element array of bytes
• MyStruct - a complex type (either defined as a struct or a Rust enum), consisting of many elements of different basic types

The Size column in tables contains the size of data in bytes. The size of dynamic arrays contains an additional plus (+) sign, e.g. 32+, which means the array has at least 32 bytes. Empty dynamic arrays always have 8 bytes which is the size of the array header containing array length. In case the size of a particular complex data is unknown it is marked with ?. The limit, however, is always 1232 bytes for the whole data packet (payload within the UDP packet).

Data serialization

In the Rust implementation of the Solana node, the data is serialized into a binary form using a bincode crate as follows:

• basic types, e.g. u8, u16, u64, etc. - are serialized as they are present in the memory, e.g. u8 type is serialized as 1 byte, u16 as 2 bytes, and so on,
• array elements are serialized as above, e.g. [u8; 32] array is serialized as 32 bytes, [u16; 32] will be serialized as 32 16-bit elements which are equal to 64 bytes,
• dynamically sized arrays have always an 8-byte header containing array length plus bytes of data, therefore empty arrays take 8 bytes,
• bit vectors are serialized similar to dynamic arrays - their header contains 1-byte which tells whether there is any data in the vector, followed by 8-byte array length and the data,
• enum types contain a header with a 4-byte discriminant (tells which enum variant is selected) + additional data,
• option types are serialized using a 1-byte discriminant followed by the bytes of data. If a value is None discriminant is set to 0 and the data part is empty, otherwise it is set to 1 with data serialized according to its type,
• struct fields are serialized one by one using the rules above,
• tuples are serialized like structs.

Enum types

Enum types in Rust are more advanced than in other languages. Apart from classic enum types, e.g.:

enum CompressionType {
    Gzip,
    Bzip2
}

it is also possible to create an enum which contains data fields, e.g.:

enum SomeEnum {
    Variant1(u64),
    Variant2(SomeType)
}

struct SomeType {
    x: u32,
    y: u16,
}

In the first case, the serialized object of the CompressionType enum will only contain a 4-byte header with the discriminant value set to the selected variant (0 = GZip, 1 = Bzip2). In the latter case apart from the header the serialized data will contain additional bytes according to which variant was selected:

• Variant1: 8 bytes
• Variant2: 6 bytes (the sum of x and y fields of SomeType struct)

Special care needs to be taken when deserializing such enum as according to the selected variant number of following data bytes may be different.

Push message

It is sent by nodes who want to share information with others. Nodes gather data from their crds and send push messages to their peers periodically.

A node receiving a set of push messages will:

• check for duplicate CrdsValues and drop them
• insert new CrdsValues into the crds
• transmit newly inserted CrdsValues to their peers via push message.

Data	Type	Size	Description
Pubkey	[u8; 32]	32	a public key of the origin
CrdsValuesList	[CrdsValue]	8+	a list of values to share

Solana client Rust implementation

enum Protocol {
    //...
    PushMessage(Pubkey, Vec<CrdsValue>),
    //...
}

Pull request

A node sends a pull request to ask the cluster for new information. It creates a list of bloom filters for its crds values and sends different bloom filters to different peers. The recipients of pull requests check what info the sender is missing using the received bloom filter and then construct a pull response packed with missing CrdsValues data for the pull request origin.

Data	Type	Size	Description
CrdsFilter	CrdsFilter	37+	a bloom filter representing things node already has
CrdsValue	CrdsValue	?	a value, usually a LegacyContactInfo of the node that sends the pull request containing node socket addresses for different protocols (gossip, tvu, tpu, rpc, etc.)

CrdsFilter

Data	Type	Size	Description
filter	Bloom	25+	a bloom filter
mask	u64	8	filter mask which defines the data stored in the bloom filter
mask_bits	u32	4	number of mask bits, also defines a number of bloom filters as 2^mask_bits

Bloom

Data	Type	Size	Description
keys	[u64]	8+	keys
bits	b[u64]	9+	bits
num_bits_set	u64	8	number of bits

Solana client Rust implementation

enum Protocol {
    PullRequest(CrdsFilter, CrdsValue),
    //...
}

struct CrdsFilter {
    filter: Bloom,
    mask: u64,
    mask_bits: u32,
}

struct Bloom {
    keys: Vec<u64>,
    bits: BitVec<u64>,
    num_bits_set: u64,
}

Pull response

These are sent in response to a pull request. They contain filtered values from the node's crds which pull request origin is missing.

Data	Type	Size	Description
Pubkey	[u8; 32]	32	a public key of the origin
CrdsValuesList	[CrdsValue]	8+	a list of new values

Solana client Rust implementation

enum Protocol {
    //...
    PullResponse(Pubkey, Vec<CrdsValue>),
    //...
}

Prune message

It is sent to peers with a list of origin nodes that should be pruned. No more push messages from pruned origin nodes should be sent by the recipient of this prune message to its sender.

Data	Type	Size	Description
Pubkey	[u8, 32]	32	a public key of the origin
PruneData	PruneData	144+	a structure which contains prune details

PruneData

Data	Type	Size	Description
pubkey	[u8, 32]	32	public key of the origin of this message
prunes	[[u8, 32]]	8+	public keys of origin nodes that should be pruned
signature	[u8, 64]	64	signature of this message
destination	[u8, 32]	32	a public key of the destination node for this message
wallclock	u64	8	wallclock of the node that generated the message

Solana client Rust implementation

enum Protocol {
    //...
    PruneMessage(Pubkey, PruneData),
    //...
}

struct PruneData {
    pubkey: Pubkey,
    prunes: Vec<Pubkey,
    signature: Signature,
    destination: Pubkey,
    wallclock: u64,
}

Note: for signing purposes, before serializing, the PruneData struct is prefixed with the byte array [0xff, 'S', 'O', 'L', 'A', 'N', 'A', '_', 'P', 'R', 'U', 'N', 'E', '_', 'D', 'A', 'T', 'A'].

Solana client Rust implementation

#[derive(Serialize)]
struct SignDataWithPrefix<'a> {
    prefix: &'a [u8], // Should be a b"\xffSOLANA_PRUNE_DATA"
    pubkey: &'a Pubkey,
    prunes: &'a [Pubkey],
    destination: &'a Pubkey,
    wallclock: u64,
}

Ping message

Nodes send ping messages frequently to their peers to check whether they are active. The node receiving the ping message should respond with a pong message.

Data	Type	Size	Description
from	[u8, 32]	32	public key of the origin
token	[u8, 32]	32	32 bytes token
signature	[u8, 64]	64	signature of the message

Solana client Rust implementation

enum Protocol {
    //...
    PingMessage(Ping),
    //...
}

struct Ping {
    from: Pubkey,
    token: [u8, 32],
    signature: Signature,
}

Pong message

Sent by node as a response to the ping message.

Data	Type	Size	Description
from	[u8, 32]	32	public key of the origin
hash	[u8, 32]	32	hash of the received ping token prefixed by the "SOLANA_PING_PONG" string
signature	[u8, 64]	64	signature of the message

Solana client Rust implementation

enum Protocol {
    //...
    PongMessage(Pong)
}

struct Pong {
    from: Pubkey,
    hash: Hash,
    signature: Signature,
}

Data shared between nodes

The CrdsValue values that are sent in push messages, pull requests & pull responses contain the signature and the actual shared data:

Data	Type	Size	Description
signature	[u8; 64]	64	signature of the origin
data	CrdsData	?	data

Solana client Rust implementation

struct CrdsValue {
    signature: Signature,
    data: CrdsData,
}

CrdsData

The CrdsData is an enum and can be one of:

Enum ID	Type
0	LegacyContactInfo
1	Vote
2	LowestSlot
3	LegacySnapshotHashes (deprecated)
4	AccountsHashes (deprecated)
5	EpochSlots
6	LegacyVersion
7	Version
8	NodeInstance
9	DuplicateShred
10	SnapshotHashes
11	ContactInfo
12	RestartLastVotedForkSlots
13	RestartHeaviestFork

Solana client Rust implementation

enum CrdsData {
    LegacyContactInfo(LegacyContactInfo),
    Vote(VoteIndex, Vote),
    LowestSlot(LowestSlotIndex, LowestSlot),
    LegacySnapshotHashes(LegacySnapshotHashes),
    AccountsHashes(AccountsHashes),
    EpochSlots(EpochSlotsIndex, EpochSlots),
    LegacyVersion(LegacyVersion),
    Version(Version),
    NodeInstance(NodeInstance),
    DuplicateShred(DuplicateShredIndex, DuplicateShred),
    SnapshotHashes(SnapshotHashes),
    ContactInfo(ContactInfo),
    RestartLastVotedForkSlots(RestartLastVotedForkSlots),
    RestartHeaviestFork(RestartHeaviestFork),
}

LegacyContactInfo

Basic info about the node. Nodes send this message to introduce themselves to the cluster and provide all addresses and ports that their peers can use to communicate with them.

Data	Type	Size	Description
id	[u8; 32]	32	public key of the origin
gossip	SocketAddr	10 or 22	gossip protocol address
tvu	SocketAddr	10 or 22	address to connect to for replication
tvu_quic	SocketAddr	10 or 22	TVU over QUIC protocol
serve_repair_quic	SocketAddr	10 or 22	repair service for QUIC protocol
tpu	SocketAddr	10 or 22	transactions address
tpu_forwards	SocketAddr	10 or 22	address to forward unprocessed transactions
tpu_vote	SocketAddr	10 or 22	address for sending votes
rpc	SocketAddr	10 or 22	address for JSON-RPC requests
rpc_pubsub	SocketAddr	10 or 22	websocket for JSON-RPC push notifications
serve_repair	SocketAddr	10 or 22	address for sending repair requests
wallclock	u64	8	wallclock of the node that generated the message
shred_version	u16	2	the shred version node has been configured to use

SocketAddr

An enum, can be either V4 or V6 socket address.

Enum ID	Data	Type	Size	Description
0	V4	SocketAddrV4	10	V4 socket address
1	V6	SocketAddrV6	22	V6 socket address

SocketAddrV4

Data	Type	Size	Description
ip	[u8; 4]	4	ip address
port	u16	2	port

SocketAddrV6

Data	Type	Size	Description
ip	[u8; 16]	16	ip address
port	u16	2	port

Solana client Rust implementation

struct LegacyContactInfo {
    id: Pubkey,
    gossip: SocketAddr,
    tvu: SocketAddr,
    tvu_quic: SocketAddr,
    serve_repair_quic: SocketAddr,
    tpu: SocketAddr,
    tpu_forwards: SocketAddr,
    tpu_vote: SocketAddr,
    rpc: SocketAddr,
    rpc_pubsub: SocketAddr,
    serve_repair: SocketAddr,
    wallclock: u64,
    shred_version: u16,
}

enum SocketAddr {
    V4(SocketAddrV4),
    V6(SocketAddrV6)
}

struct SocketAddrV4 {
    ip: Ipv4Addr,
    port: u16,
}

struct SocketAddrV6 {
    ip: Ipv6Addr,
    port: u16
}

struct Ipv4Addr {
    octets: [u8; 4]
}

struct Ipv6Addr {
    octets: [u8; 16]
}

Vote

It's a validator's vote on a fork. Contains a one-byte index from the vote tower (range 0 to 31) and vote transaction to be executed by the leader.

Data	Type	Size	Description
index	u8	1	vote tower index
from	[u8; 32]	32	public key of the origin
transaction	Transaction	59+	a vote transaction, an atomically-committed sequence of instructions
wallclock	u64	8	wallclock of the node that generated the message
slot	u64	8	slot in which the vote was created

Transaction

Contains a signature and a message with a sequence of instructions.

Data	Type	Size	Description
signature	[[u8; 64]]	8+	list of signatures equal to num_required_signatures for the message
message	Message	51+	transaction message containing instructions to invoke

Message

Data	Type	Size	Description
header	MessageHeader	3	message header
account_keys	[[u8; 32]]	8+	all account keys used by this transaction
recent_blockhash	[u8; 32]	32	hash of a recent ledger entry
instructions	[CompiledInstruction]	8+	list of compiled instructions to execute

Message header

Data	Type	Size	Description
num_required_signatures	u8	1	number of signatures required for this message to be considered valid
num_readonly_signed_accounts	u8	1	last num_readonly_signed_accounts of the signed keys are read-only accounts
num_readonly_unsigned_accounts	u8	1	last num_readonly_unsigned_accounts of the unsigned keys are read-only accounts

Compiled instruction

Data	Type	Size	Description
program_id_index	u8	1	index of the transaction keys array indicating the program account ID that executes the program
accounts	[u8]	8+	indices of the transaction keys array indicating the accounts that are passed to a program
data	[u8]	8+	program input data

Solana client Rust implementation

enum CrdsData {
    //...
    Vote(VoteIndex, Vote),
    //...
}

type VoteIndex = u8;

struct Vote {
    from: Pubkey,
    transaction: Transaction,
    wallclock: u64,
    slot: Option<Slot>,
}

type Slot = u64

struct Transaction {
    signature: Vec<Signature>,
    message: Message
}

struct Message {
    header: MessageHeader,
    account_keys: Vec<Pubkey>,
    recent_blockhash: Hash,
    instructions: Vec<CompiledInstruction>,
}

struct MessageHeader {
    num_required_signatures: u8,
    num_readonly_signed_accounts: u8,
    num_readonly_unsigned_accounts: u8,
}

struct CompiledInstruction {
    program_id_index: u8,
    accounts: Vec<u8>,
    data: Vec<u8>,
}

LowestSlot

It is the first available slot in Solana blockstore that contains any data. Contains a one-byte index (deprecated) and the lowest slot number.

Data	Type	Size	Description
index	u8	1	deprecated
from	[u8; 32]	32	public key of the origin
root	u64	8	deprecated
lowest	u64	8	the actual slot
slots	[u64]	8+	deprecated
stash	[EpochIncompleteSlots]	8+	deprecated
wallclock	u64	8	wallclock of the node that generated the message

EpochIncompleteSlots

Data	Type	Size	Description
first	u64	8	first slot number
compression	CompressionType	4	compression type
compressed_list	[u8]	8+	compressed slots list

CompressionType

Compression type enum.

Enum ID	Data	Description
0	Uncompressed	uncompressed
1	GZip	gzip
2	BZip2	bzip2

Solana client Rust implementation

enum CrdsData {
    //...
    LowestSlot(LowestSlotIndex, LowestSlot),
    //...
}

type LowestSlotIndex = u8;

struct LowestSlot {
    from: Pubkey,
    root: Slot,
    lowest: Slot,
    slots: BTreeSet<Slot>,
    stash: Vec<EpochIncompleteSlots>,
    wallclock: u64,
}

struct EpochIncompleteSlots {
    first: Slot,
    compression: CompressionType,
    compressed_list: Vec<u8>,
}

enum CompressionType {
    Uncompressed,
    GZip,
    BZip2,
}

LegacySnapshotHashes, AccountsHashes

Deprecated

Data	Type	Size	Description
from	[u8, 32]	32	public key of the origin
hashes	[(u64, [u8, 32])]	8+	a list of hashes grouped by slots
wallclock	u64	8	wallclock of the node that generated the message

Solana client Rust implementation

struct AccountsHashes {
    from: Pubkey,
    hashes: Vec<(Slot, Hash)>,
    wallclock: u64,
}

type LegacySnapshotHashes = AccountsHashes;

EpochSlots

Contains a one-byte index and list of all slots from an epoch (epoch consists of around 432000 slots). There can be 256 epoch slots in total.

Data	Type	Size	Description
index	u8	1	index
from	[u8, 32]	32	public key of the origin
slots	[CompressedSlots]	8+	list of slots
wallclock	u64	8	wallclock of the node that generated the message

CompressedSlots

EnumID	Data	Type	Size	Description
0	Flate2	Flate2	24+	Flate 2 compression
1	Uncompressed	Uncompressed	25+	no compression

Flate2

Data	Type	Size	Description
first_slot	u64	8	first slot number
num	u64	8	number of slots
compressed	[u8]	8+	bytes array of compressed slots

Uncompressed

Data	Type	Size	Description
first_slot	u64	8	first slot number
num	u64	8	number of slots
slots	b[u8]	9+	bits array of slots

Solana client Rust implementation

enum CrdsData {
    //...
    EpochSlots(EpochSlotsIndex, EpochSlots),
    //...
}

type EpochSlotsIndex = u8;

struct EpochSlots {
    from: Pubkey,
    slots: Vec<CompressedSlots>,
    wallclock: u64,
}

enum CompressedSlots {
   Flate2(Flate2),
   Uncompressed(Uncompressed),
}

struct Flate2 {
    first_slot: Slot,
    num: usize,
    compressed: Vec<u8>
}

struct Uncompressed {
    first_slot: Slot,
    num: usize,
    slots: BitVec<u8>,
}

LegacyVersion

The older version of the Solana client the node is using.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
version	LegacyVersion1	7 or 11	older version of the Solana used in 1.3.x and earlier releases

LegacyVersion1

Data	Type	Size	Description
major	u16	2	major part of version
minor	u16	2	minor part of version
patch	u16	2	patch
commit	u32 | None	5 or 1	commit

Solana client Rust implementation

struct LegacyVersion {
    from: Pubkey,
    wallclock: u64,
    version: LegacyVersion1,
}

struct LegacyVersion1 {
    major: u16,
    minor: u16,
    patch: u16,
    commit: Option<u32>
}

Version

The version of the Solana client the node is using.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
version	LegacyVersion2	11 or 15	version of the Solana

LegacyVersion2

Data	Type	Size	Description
major	u16	2	major part of version
minor	u16	2	minor part of version
patch	u16	2	patch
commit	u32 | None	5 or 1	commit
feature_set	u32	4	feature set

Solana client Rust implementation

struct Version {
    from: Pubkey,
    wallclock: u64,
    version: LegacyVersion2,
}

struct LegacyVersion2 {
    major: u16,
    minor: u16,
    patch: u16,
    commit: Option<u32>,
    feature_set: u32
}

NodeInstance

Contains node creation timestamp and randomly generated token.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
timestamp	u64	8	timestamp when the instance was created
token	u64	8	randomly generated value at node instantiation

Solana client Rust implementation

struct NodeInstance {
    from: Pubkey,
    wallclock: u64,
    timestamp: u64,
    token: u64,
}

DuplicateShred

A duplicated shred proof. Contains a 2-byte index followed by other data:

Data	Type	Size	Description
index	u16	2	index
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
slot	u64	8	slot when shreds where created
_unused	u32	4	unused
_unused_shred_type	ShredType	1	unused
num_chunks	u8	1	number of chunks available
chunk_index	u8	1	index of the chunk
chunk	[u8]	8+	shred data

ShredType

This enum is serialized as 1-byte data.

Enum ID	Data	Description
0b10100101	Data	data shred
0b01011010	Code	coding shred

Solana client Rust implementation

enum CrdsData {
    //...
    DuplicateShred(DuplicateShredIndex, DuplicateShred),
    //...
}

type DuplicateShredIndex = u16;

struct DuplicateShred {
    from: Pubkey,
    wallclock: u64,
    slot: Slot,
    _unused: u32,
    _unused_shred_type: ShredType,
    num_chunks: u8,
    chunk_index: u8,
    chunk: Vec<u8>,
}

#[serde(into = "u8", try_from = "u8")]
enum ShredType {
    Data = 0b1010_0101,
    Code = 0b0101_1010,
}

SnapshotHashes

Contains hashes of full and incremental snapshots.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
full	(u64, [u8, 32])	40	hash and slot number of the full snapshot
incremental	[(u64, [u8, 32])]	8+	list of hashes and slot numbers of incremental snapshots
wallclock	u64	8	wallclock of the node that generated the message

Solana client Rust implementation

struct SnapshotHashes {
    from: Pubkey,
    full: (Slot, Hash),
    incremental: Vec<(Slot, Hash)>,
    wallclock: u64,
}

ContactInfo

Contact info of the node.

Data	Type	Size	Description
pubkey	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
outset	u64	8	timestamp when node instance was first created
shred_version	u16	2	the shred version the node has been configured to use
version	Version	13+	Solana version
addrs	[IpAddr]	8+	list of unique IP addresses
sockets	[SocketEntry]	8+	list of unique sockets
extensions	[Extension]	8+	future additions to ContactInfo will be added to Extensions instead of modifying ContactInfo, currently unused
cache	[SocketAddr, 12]	120 or 264	cache of nodes socket addresses

Version

Data	Type	Size	Description
major	u16	2	major part of version
minor	u16	2	minor part of version
patch	u16	2	patch
commit	u32 | None	5 or 1	commit
feature_set	u32	4	feature set
client	u16	2	client

IpAddr

Enum ID	Data	Type	Size	Description
0	V4	[u8; 4]	4	IP v4 addr
1	V6	[u8, 16]	16	IP v6 addr

SocketEntry

Data	Type	Size	Description
key	u8	1	protocol identifier (tvu, tpu, etc.)
index	u8	1	IpAddr index in the addrs list
offset	u16	2	port offset in respect to previous entry

Extension

Currently empty

Solana client Rust implementation

struct ContactInfo {
    pubkey: Pubkey,
    wallclock: u64,
    outset: u64,
    shred_version: u16,
    version: Version,
    addrs: Vec<IpAddr>,
    sockets: Vec<SocketEntry>,
    extensions: Vec<Extension>,
    cache: [SocketAddr; 12],
}

enum Extension {}

enum IpAddr {
    V4(Ipv4Addr),
    V6(Ipv4Addr)
}

struct Ipv4Addr {
    octets: [u8; 4]
}

struct Ipv6Addr {
    octets: [u8; 16]
}

struct SocketEntry {
    key: u8,
    index: u8,
    offset: u16
}

struct Version {
    major: u16,
    minor: u16,
    patch: u16,
    commit: Option<u32>,
    feature_set: u32,
    client: u16
}

RestartLastVotedForkSlots

Contains a list of last-voted fork slots.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
offsets	SlotsOffsets	12+	list of slots
last_voted_slot	u64	8	last voted slot
last_voted_hash	[u8, 32]	32
shred_version	u16	2	the shred version node has been configured to use

SlotsOffsets

Offsets are stored either in binary form (RawOffsets) or encoded as numbers of consecutive 1's and 0's, e.g. 110001110 is [2, 3, 3, 1].

Enum ID	Data	Type	Size	Description
0	RunLengthEncoding	[u16]	8+	encoded offsets
1	RawOffsets	b[u8]	9+	raw offsets

Solana client Rust implementation

struct RestartLastVotedForkSlots {
    from: Pubkey,
    wallclock: u64,
    offsets: SlotsOffsets,
    last_voted_slot: Slot,
    last_voted_hash: Hash,
    shred_version: u16,
}

enum SlotsOffsets {
    RunLengthEncoding(RunLengthEncoding),
    RawOffsets(RawOffsets),
}

struct RunLengthEncoding(Vec<u16>);
struct RawOffsets(BitVec<u8>);

RestartHeaviestFork

Contains the heaviest fork.

Data	Type	Size	Description
from	[u8, 32]	32	public key of origin
wallclock	u64	8	wallclock of the node that generated the message
last_slot	u64	8	last slot of the fork
last_hash	[u8, 32]	32	hash of the last slot
observed_stake	u64	8
shred_version	u16	2	the shred version node has been configured to use

Solana client Rust implementation

struct RestartHeaviestFork {
    from: Pubkey,
    wallclock: u64,
    last_slot: Slot,
    last_slot_hash: Hash,
    observed_stake: u64,
    shred_version: u16,
}

Addendum

IP Echo Server

The IP Echo Server is a server running on the TCP socket on the same gossip address. (e.g. if the node is running the gossip service on UDP socket 192.0.0.1:9000, then the IP server port is running on the same TCP socket 192.0.0.1:9000).

Before the node starts the gossip service, the node first needs to:

• find out the shred version of the cluster,
• discover its public IP address,
• check TCP/UDP port reachability.

All of these are discovered via the IP echo server running on one of the provided entrypoint nodes. Note: all validators run an IP Echo Server. The node should create sockets with ports that need to be checked and then send an IP echo server request message to one of the entrypoint nodes. The entrypoint node will then check reachability for all ports listed in the request and then it will respond with an IP echo server response containing the shred_version and public IP of the node.

IpEchoServerMessage

IP echo server message request containing a list of ports whose reachability should be checked by the server:

• UDP ports - the server should send a single byte [0x00] packet to the socket.
• TCP ports - the server should establish a TCP connection with every TCP socket.

Data	Type	Size	Description
tcp_ports	[u16, 4]	64	TCP ports that should be checked
udp_ports	[u16, 4]	64	UDP ports that should be checked

Solana client Rust implementation

/// Echo server message request.
pub struct IpEchoServerMessage {
    pub tcp_ports: [u16; 4],
    pub udp_ports: [u16; 4],
}

IpEchoServerResponse

IP echo server message response.

Data	Type	Size	Description
address	IpAddr	4 or 16	public IP of the node that sent the request
shred_version	u16	2	shred verion of the cluster

Solana client Rust implementation

/// Echo server response.
pub struct IpEchoServerResponse {
    /// Public IP address of request echoed back to the node.
    pub address: IpAddr,
    /// Cluster shred-version of the node running the server.
    pub shred_version: Option<u16>,
}