Support for multi path data directories per server to utilize multiple disks present in a node

[chrajeshbabu]

Currently under pinot.server.instance.dataDir we can only configure single directory path with that we cannot utilize the all the disks present in the system. Because of this limitation we need to add more nodes to launch servers and launch servers per disk to accommodate the data.

By having simple of map of segment vs directory path we can use multiple directories under data directories. It would be better to introduce the new configuration pinot.server.instance.dataDirs and while allocating directory to a segment we can have round robin based or size availability based approaches. During the start up also we can build the map by looking at the segments present in the directories configured and update when ever required.

Working on the design document to support the same and will share in few days.

[ilamhs]

@chrajeshbabu I came across this feature of using multi-disks using instance tierConfigs (#8843) that would achieve not exactly the same request but something similar. Does that satisfy the above request?