Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving slugs results in memory exhaustion #2693

Open
Tofandel opened this issue Nov 13, 2024 · 3 comments
Open

Saving slugs results in memory exhaustion #2693

Tofandel opened this issue Nov 13, 2024 · 3 comments

Comments

@Tofandel
Copy link
Contributor

Tofandel commented Nov 13, 2024

Description

Just saving a non unique slug in a database that has a lot of entries (Not like an insane amount, just 500) seems to result in a lot of queries and then memory exhaustion

I already increased the memory to 512Mo and it was ok for a few weeks, but now it exceeds the 512Mo limit again

Something is terribly wrong in there, here is the query trace from sentry as we can see there is some huge queries being run

image

image

image

I will try to investigate the cause

@antonioribeiro antonioribeiro changed the title Saving sluts results in memory exhaustion Saving slugs results in memory exhaustion Nov 13, 2024
@Tofandel
Copy link
Contributor Author

image

So apparently the slug table has grown to an incredible size (70Mb) with each entry having hundreds of duplicated non active slugs

@ifox
Copy link
Member

ifox commented Nov 19, 2024

Ugh, this sounds really bad, we'll be looking into it asap.

Does this PR help by any chance? I have not reviewed it yet.

@Tofandel
Copy link
Contributor Author

Tofandel commented Nov 19, 2024

Very likely yes, not sure if the PR solves it, but the issue should definitely because of the same thing, I have been looking at the query log and it seems 3 queries are duplicated (because a function is called multiple times when the result could be cached) in the code path and the one retrieving the list of all the existing ids is not even needed (it can be inlined with a not condition on the fk), I will send in a PR with those improvements

I think we also should add a migration to remove all the duplicated records

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants