Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert store and param registry to sqlite #114

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Convert store and param registry to sqlite #114

wants to merge 10 commits into from

Conversation

WarmCyan
Copy link
Collaborator

This has previously been an issue because large stores take a second to write out new information - if the experiment gets ctrl-C'd before this completes, the store corrupts and you have to delete it (resetting all run numbers to zero)

Closes #25

@WarmCyan
Copy link
Collaborator Author

WarmCyan commented Nov 9, 2023

Consider if using duckdb is a better alternative to sqlalchemy - I don't love sqlalchemy's API so far and I think duckdb is a little more intuitive. Weight pros and cons.

This makes it a lot easier to interface with the sql database since we
can use dataclasses' asdict, and have a function to sanitize if needed.

Also this partially begins to address #65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use sqlite for experiment store instead of json
1 participant