Use S3 for historical queries instead of DynamoDB #3839
Labels
estimate/weeks
It will take more than 7 8-hour work days to implement
performance
Excessive resource usage and latency; usually a bug or chore
Scope has an optional multitenant mode, where reports are saved to S3 and indexed in DynamoDB.
Once #3783 is done, live rendering will not use the store, so we will have far less time pressure. I think we can drop the index and just use an S3 'list' API call to find objects.
However we will need to change the object path-name to include the time as a prefix.
Current paths are like
s3://bucket-name/00002140a76ed46df4956c4af4004160/1554123600273225527
, where the first part is a MD5 hash of the tenant ID and hour number, and the second part is the Unix timestamp in nanoseconds.Steps to complete:
The text was updated successfully, but these errors were encountered: