Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add -gsplit-dwarf to LTO build to cut memory utilization in linker #73812

Merged
merged 1 commit into from
May 16, 2024

Conversation

kevingranade
Copy link
Member

This will break debugging for this build for now, but it's currently failing so ehhhh...

Summary

None

Purpose of change

Some chatter on discord brought some build failures like https://github.com/CleverRaven/Cataclysm-DDA/actions/runs/9078930225/job/24950004752#step:14:746 to my attention, looks like memory exhaustion?
Attempted to avoid overlapping linker invocations in #73793 but that didn't do it.

Describe the solution

This passes a flag to the compiler (in github LTO builds only) to output debugging data to a separate file instead of to the .o file.
As a result, at link time the linker doesn't have to load and process that (massive) debug data and doesn't consume all the memory on the system.

Describe alternatives you've considered

There are various overhauls we could look into, but the debug data really is absolutely dominating memoy usage, so I think this is the way to go.

Testing

I ran the build locally using the same build arguments and a similar gcc version (mine: g++ (Ubuntu 9.4.0-1ubuntu120.04.2) 9.4.0, CI: g++-9 (Ubuntu 9.5.0-1ubuntu122.04) 9.5.0) as in the failing CI job.
Before this change:
39.46user 12.12system 1:08.21elapsed 75%CPU (0avgtext+0avgdata 9189096maxresident)k

After this change:
7.80user 1.81system 0:09.89elapsed 97%CPU (0avgtext+0avgdata 1950676maxresident)k

The final test will be if the CI job breaks, but I'm very optomistic.

Additional context

This will break debugging for now since we aren't archiving and storing the debug data, but I think that's a better state to be in temporally than all the builds failing...
This article outlines what the whole process looks like: https://www.tweag.io/blog/2023-11-23-debug-fission/
Once we have that working, I suspect we'll want to switch all the CI builds over to this.

@github-actions github-actions bot added Code: Build Issues regarding different builds and build environments astyled astyled PR, label is assigned by github actions json-styled JSON lint passed, label assigned by github actions labels May 15, 2024
@kevingranade
Copy link
Member Author

Noticed a bug, this does not ONLY get added to LTO builds as I had intended, it should still help diagnose whether this approach will fix the GCC 9 LTO build.

This will break debugging for this build for now, but it's currently failing so ehhhh...
@kevingranade kevingranade force-pushed the split-debug-for-LTO branch from 370747a to d0f7ed7 Compare May 15, 2024 23:28
@github-actions github-actions bot added Code: Tooling Tooling that is not part of the main game but is part of the repo. BasicBuildPassed This PR builds correctly, label assigned by github actions labels May 15, 2024
@kevingranade
Copy link
Member Author

After the latest change this no longer supplies the flag to the BasicBuild job at least.

@dseguin dseguin merged commit df4c499 into master May 16, 2024
19 checks passed
@kevingranade kevingranade deleted the split-debug-for-LTO branch May 16, 2024 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
astyled astyled PR, label is assigned by github actions BasicBuildPassed This PR builds correctly, label assigned by github actions Code: Build Issues regarding different builds and build environments Code: Tooling Tooling that is not part of the main game but is part of the repo. json-styled JSON lint passed, label assigned by github actions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants