Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

timescaledb vs redis #3

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

feierabend654
Copy link

@feierabend654 feierabend654 commented Jul 23, 2021

This changeset adds timescaledb integration for storing and loading candles.

There are some dev scripts for one to try it out locally - ./run-dev-timescale.sh; ./run-dev-timescale-migrations.s will start timescale with an external data volume and also run the necessary setup migrations. Currently I have setup a compression bucket interval of a day and also uses the symbol column. Would appreciate some input on common access patterns to refine this.

There is another script yarn timescale-perf-test-bulk-load to bulk load one month old random data for stress testing.

And finally there are some logs sprinkled which do basic timing of storing and fetching from redis and timescale.

Also, maybe, we want to do some error handling around timescale interaction, or fail hard, not decided, lets discuss

Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
Signed-off-by: feierabend654 <[email protected]>
@feierabend654
Copy link
Author

Extended the redis script to mount a dump, see run-dev-redis.sh

Also, added a simple migration (redis->timescale) script, can be run so ./run-dev-timescale.sh; ./run-dev-timescale-migrations.sh; yarn timescale-migrate-from-redis;, this basically resets timescale instance, and migrates redis data to timescale

@BunnyNomics101
Copy link

@feierabend654 We have been trying to run this for our dex as web socket API. It deployed successfully on port 8900 but there is no signal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants