Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor reader-file handling in StdFileManager #8

Open
ecton opened this issue Sep 28, 2021 · 1 comment
Open

Refactor reader-file handling in StdFileManager #8

ecton opened this issue Sep 28, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@ecton
Copy link
Member

ecton commented Sep 28, 2021

The current implementation of read-file caching isn't great -- it never closes any files. This is fine for now, but we'll eventually run into open-file limits.

The read-files (and maybe write-files) should be switched to an Lru cache to automatically close file handles that haven't been needed as we need to open new files.

@ecton ecton added the enhancement New feature or request label Sep 28, 2021
@ecton ecton moved this to Todo in Beta Roadmap Oct 27, 2021
@ecton
Copy link
Member Author

ecton commented Apr 11, 2022

I noticed yesterday that this is causing open file issues on Mac OS when running the core-suite unit test in BonsaiDb. That test, with all features enabled, runs 99 unit tests, each which create a database with multiple views. The default max-open-files limit on Mac is only 256.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

1 participant