[Remote State] All the nodes should download full/ diff cluster state from remote store

[shwetathareja]

Is your feature request related to a problem? Please describe

Today, Cluster state publishing is done by the leader to all the other nodes in the cluster in a centralized fashion and it can overload the leader when there are large no. of nodes, too many cluster state metadata changes in the cluster.

Describe the solution you'd like

The cluster state is now backed to remote state. This opens up the opportunity to reduce the overhead on the leader node to publish the full/ diff cluster state to all the other nodes in the cluster. This would provide huge benefits in terms of less work done by the leader node to send serialized state over the transport to every node. Every time there is cluster state metadata change, leader will send the term, version and remote store location to download the metadata from remote storage. In case of any new cluster updates, leader should store the diff to remote storage and all the nodes should download from remote store in decentralized manner.

Related component

Cluster Manager

Describe alternatives you've considered

No response

Additional context

No response

[shwetathareja]

Detailed proposal is captured here - #13257