Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reset Go settings for protoc dependencies #3005

Merged
merged 4 commits into from
May 5, 2022

Conversation

fmeum
Copy link
Member

@fmeum fmeum commented Nov 14, 2021

What type of PR is this?

Bug fix

What does this PR do? Why is it needed?

Reduces the number of essentially distinct configurations in which protoc as a dependency of go_proto_library is analyzed.

Note that this may not (yet) result in a significant reduction of rebuilds since Bazel is not particular friendly to reset transitions at the moment: It encodes some information about the path of transitions taken into the build configuration, which e.g. prevents transitions from ever returning to the top-level target configuration. But this will be solved by bazelbuild/bazel#14023, which is very actively being worked on. In the meantime, this change should at least not regress the current behavior further.

Which issues(s) does this PR fix?

Fixes a part of #2999.

Other notes for review

@robfig If you think that this PR is useful, I would also look into a follow-up that resets the non-Go attrs, e.g. data and embedsrcs, of rules such as go_library to their values before go_transition. Ideally, rules_go settings set by transitions would only ever propagate to rules that provide a Go provider. Since this requires memoizing the state of settings before a transition, this will be a bit more complicated, which is why I would like to get your feedback on this simpler case first.

@google-cla google-cla bot added the cla: yes label Nov 14, 2021
@blico blico self-requested a review December 1, 2021 18:52
@robfig
Copy link
Contributor

robfig commented Dec 8, 2021

Hi @fmeum , thanks for the contribution. We would definitely like to avoid cache thrashing. I don't personally have any knowledge about transitions, but @blico volunteered to study up and review. Apologies for the delay, but be assured we haven't forgotten and would like to get this merged.

@fmeum
Copy link
Member Author

fmeum commented Dec 8, 2021

No worries, let me know if you need any additional explanations.

@fmeum fmeum force-pushed the hermeticity-tests branch from cd757ef to 6f570a7 Compare January 16, 2022 15:30
@fmeum
Copy link
Member Author

fmeum commented Jan 16, 2022

I rebased the PR onto master after my other changes were merged.

@fmeum
Copy link
Member Author

fmeum commented Mar 20, 2022

@blico Friendly ping as the required commit has landed in Bazel: bazelbuild/bazel@52d1d4a

@fmeum
Copy link
Member Author

fmeum commented Apr 9, 2022

@robfig I just submitted #3108, which is based on this PR and finished the leftover work. Along the way, it fixes a rules_go correctness bug reported in bazelbuild/bazel#14626. Do you think one of the maintainers will have time to review either of those PRs in the near future?

Copy link
Contributor

@linzhp linzhp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passed the tests in Uber's Go monorepo. Left a few stylish comments. Looks good overall. Sorry for taking so long to review.

tests/core/transition/hermeticity_test.go Show resolved Hide resolved
go/private/rules/transition.bzl Outdated Show resolved Hide resolved
go/private/rules/transition.bzl Outdated Show resolved Hide resolved
go/private/rules/transition.bzl Outdated Show resolved Hide resolved
proto/BUILD.bazel Outdated Show resolved Hide resolved

go_reset_target depends on a single target, built using go_reset_transition.
go_exec_reset_target depends on a single target, built using go_exec_reset_transition.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc seems wrong

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the chance to improve the docs, but am not sure I understood this particular point.

go/private/rules/transition.bzl Outdated Show resolved Hide resolved
fmeum added 4 commits May 4, 2022 22:28
This transition should be applied to all non-Go tools (e.g. protoc) to
prevent unnecessary rebuilds.
The test verifies that Go Starlark settings such as bootstrap_nogo and
pure are reset before they reach protoc, which is a dependency of
go_proto_library in multiple ways.
@fmeum fmeum force-pushed the hermeticity-tests branch from 8bba4e1 to 4efa730 Compare May 5, 2022 08:17
@fmeum
Copy link
Member Author

fmeum commented May 5, 2022

@linzhp Thanks for the review. I pushed a commit that should address your comments.

@linzhp linzhp merged commit a20046b into bazel-contrib:master May 5, 2022
@fmeum fmeum deleted the hermeticity-tests branch May 5, 2022 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants