Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arctic library (mongodb collections) - Best way to store 1000's of symbols for 15 years of history #891

Open
DataCT2020 opened this issue Mar 10, 2021 · 2 comments

Comments

@DataCT2020
Copy link

Arctic Version

# Arctic version here

Arctic Store

# VersionStore, TickStore, or ChunkStore
```ChunkStore

#### Platform and version

Put here

#### Description of problem and/or code sample that reproduces the issue

Hi
We are using Arctic library data to process trades/quotes historical data. For these data we have 1000's of symbols and we are expecting to load last 15 years of history. So far we have loaded 1 year of history and everything is working as expected. 

We are using Arctic Chunkstore with Date Chunker to store these data.

We would like to know is it ok to store all data in one collection (arctic library)? or should we split collections by based on date range of data ? (e.g 2017 to 2020 data in collection1? 2013 to 2016 data in collection2).

Thanks for your time

Dhru

@shashank88
Copy link
Contributor

I will let @bmoscon reply on the Chunkstore stuff, but it should be fine to store multi-year history in one collection if it's sharded properly (if you do have shards) and the indexes in Mongo look okay which most of the stores create for you.

@DataCT2020
Copy link
Author

DataCT2020 commented Mar 12, 2021

thank for reply shashank and I will wait for further confirmation from @bmoscon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants