Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug in loading logic #366

Open
mgdenno opened this issue Jan 10, 2025 · 0 comments · May be fixed by #373
Open

Bug in loading logic #366

mgdenno opened this issue Jan 10, 2025 · 0 comments · May be fixed by #373
Assignees

Comments

@mgdenno
Copy link
Contributor

mgdenno commented Jan 10, 2025

From Teams message exchange:

so we first convert the parquet files to the the required format and save to the cache dir

but when we read from the cache dir we read all the files in the cache dir

but we don't clear the cache dir each time

so the cache dir currently has both the good file in it but also the old failed files

so it is trying to load the failed ones too...that is the issue

...so we either need to clear the cache before running or use explicit file names...

@samlamont samlamont linked a pull request Jan 16, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants