Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Your computer is low on memory #28

Open
sjb554 opened this issue Aug 4, 2016 · 2 comments
Open

Your computer is low on memory #28

sjb554 opened this issue Aug 4, 2016 · 2 comments

Comments

@sjb554
Copy link

sjb554 commented Aug 4, 2016

Hi - I am running some large json files (some as large as 2.5GB) and am getting a windows memory error popup like this:
"
Close programs to prevent information loss
Your computer is low on memory. Save your files and close these programs:
python.exe
Windows will only close enough programs to restore needed memory.
"

json2csv stops at that point and no CSV is created. Is there a way to have python continue parsing, even against memory issues?

Note that I keep all other programs closed while running the program.

Thanks,
SJB

@evidens
Copy link
Owner

evidens commented Aug 17, 2016

Hi, as you can tell I don't have much time for open source projects these days.
I'd started a fix for this issue in https://github.com/evidens/json2csv/tree/13-direct-transcription and I've been told it works. I didn't get around to writing tests for it, which is why I didn't merge it into master.

Sorry if this reaches you too late for your project. Good luck.

@sjb554
Copy link
Author

sjb554 commented Aug 29, 2016

Your fix seems to have worked perfectly. Instead of maxing out my memory, the program stayed at around 7Gb the entire time. I was able to parse json files as big as 2.2Gb that have fairly deep nests.

I really appreciate your help!

Best,
SJB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants