Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enumerating release jobset names is single-threaded and slow #269357

Closed
ghost opened this issue Nov 23, 2023 · 0 comments · Fixed by #269356
Closed

enumerating release jobset names is single-threaded and slow #269357

ghost opened this issue Nov 23, 2023 · 0 comments · Fixed by #269356
Labels
0.kind: bug Something is broken

Comments

@ghost
Copy link

ghost commented Nov 23, 2023

The only known way to enumerate the names of all packages/jobs in a release is the one used by ofborg for the eval-checks:

This is obscenely slow and requires an absurd amount of memory. It is entirely single-threaded with zero parallelism. It peaks at 60gbytes of memory and takes 41 minutes on a 3ghz cpu.

This accounts for the vast majority of the cycle time for our CI.

@ghost ghost added the 0.kind: bug Something is broken label Nov 23, 2023
@ghost ghost self-assigned this Nov 23, 2023
@ghost ghost linked a pull request Nov 23, 2023 that will close this issue
@ghost ghost changed the title outpath-listing / enumerating release jobset is single-threaded and slow enumerating release jobset names is single-threaded and slow Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken
Projects
None yet
Development

Successfully merging a pull request may close this issue.

0 participants