[META] Translog Optimisation: Upload .ckp file data as object metadata of .tlog file

[skumawat2025]

Please describe the end goal of this project

Proposing a optimisation for translog files uploads. In this optimisation we purpose a solution to store checkpoint file data as object metadata/header of translog.tlog file for supported cloud providers. The end goal of this is to reduce the rate or number of calls to remote store during translog uploads. In this meta, we will breakdown the issue to smaller issues that we would be working on to complete this feature request.

Supporting References

Feature request - #13022

Issues

• [Remote Store] Introduce interface changes to support read/write blob with object metadata #13092
• [Remote Store] Implement introduced interface changes for s3 plugin #13093
• [Remote Store] Introduce new translog upload/download (with object metadata) flow while maintaining dual mode handling with existing upload/download flow. #13094

Related component

Storage:Performance