-
-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a compressed output format for rethinkdb export / import #249
Comments
This is an interesting (and useful) feature that surely deserves to be implemented. As of now, we're in the middle of a revamp of this repo, because the project is flooded with old code and bad mixes of python 2 and 3. We're already working on it, in a separate repo, where we're cleaning up the messy code and start supporting only python 3 since the EOL of python 2. Our first goal is to not break any existing code. We will then start adding features as well, like the one requested in this issue. |
This is something I'm experimenting with at the moment. Does it make sense to open a PR with a proposed implementation to get some feedback? |
If there's already the implementation, we could try to make room for it in the new repo (temporarily private since early WIP). You can use the |
Cool, I opened a PR against master for now, that's what I was working on. If it's a go I can switch to |
I checked the PR rapidly, but it looks really neat. Thanks! It will need a proper code review and we will then merge it into the new repo (as soon as we're done with the basis of it). I would keep the PR open until then, if you don't mind, do you? |
Sure no problem 👍 |
Is your feature request related to a problem? Please describe.
Dumping large databases / tables involves writing very large files to the disk, is expensive in terms of disk io.
Describe the solution you'd like
Add a new compressed output format json.gz to rethinkdb export.
Enable on the fly streaming compression of data files using zlib.
The text was updated successfully, but these errors were encountered: