-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crate_universe
: Enable modext isolation.
#17088
Conversation
Marking as draft, as this PR requires renaming the I still think this is worth it, but this requires a bit more discussion than just enabling modext isolation. |
6191d86
to
cdbb036
Compare
cdbb036
to
798efde
Compare
388dc2c
to
a72c526
Compare
java/kotlin-extractor/BUILD.bazel
Outdated
load( | ||
"@codeql//java/kotlin-extractor:versions.bzl", | ||
"@ql//java/kotlin-extractor:versions.bzl", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was some @codeql
s in the comments at the start of this file; presumably they need to be changed too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I changed the relevant ones - when you build from the internal repo, we have an alias in place that you can still call the build targets @codeql//something, only from the external repo it's now (unfortunately) @ql//something, which is why I changed the wording of the comment there to remove the repo name alltogether when building from the external repo.
That said, there might still be an incorrect comment there, so if you find one, please shout!
MODULE.bazel
Outdated
module( | ||
name = "codeql", | ||
name = "ql", | ||
version = "0.0", | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can avoid having to change @codeql
to @ql
by using repo_name
here as well:
module( | |
name = "codeql", | |
name = "ql", | |
version = "0.0", | |
) | |
module( | |
name = "ql", | |
version = "0.0", | |
repo_name = "codeql", | |
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an excellent and unexpected find, thanks! This indeed makes things much nicer.
This should allow us to build our python and ruby code independently - in particular, we can now do shallow checkouts of one without the other. Previously, the modext introduced cross-dependency. This also reduces the amount of work we do in the crate universe processing for the other language, even though it's unused. This does need renaming the module, as otherwise the generated paths from rules_rust get too long for Windows :(
a72c526
to
f9bc97b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm good with these changes, thanks! I'd prefer to keep ruby_deps
as an alias of rd
though, as you can do that with use_repo
(see the kwargs
parameter in the docs).
@@ -30,27 +31,37 @@ bazel_dep(name = "rules_rust", version = "0.49.1") | |||
|
|||
bazel_dep(name = "buildifier_prebuilt", version = "6.4.0", dev_dependency = True) | |||
|
|||
crate = use_extension( | |||
# crate_py but shortened due to Windows file path considerations | |||
cp = use_extension( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
really minor cosmetic nit, but is there any reason to use cp
here and just r
for ruby? If cr
for ruby is already too much, maybe then also use p
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I wanted to limit the use of single-letter repo names to where we really need them (as there's only 26 of them available). The more letters, the easier to search for the repo name in our source code, too.
MODULE.bazel
Outdated
cargo_lockfile = "//ruby/extractor:Cargo.lock", | ||
manifests = [ | ||
"//ruby/extractor:Cargo.toml", | ||
"//ruby/extractor/codeql-extractor-fake-crate:Cargo.toml", | ||
], | ||
) | ||
use_repo(crate, "py_deps", "ruby_deps") | ||
use_repo(r, "rd") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can also assign an alias here, avoiding the changes in the two build files:
use_repo(r, "rd") | |
use_repo(r, ruby_deps="rd") |
This should allow us to build our python and ruby
code independently - in particular, we can now do shallow checkouts of one without the other.
Previously, the modext introduced a cross-dependency. This also reduces the amount of work we do in the
crate universe processing for the other language, even though it's unused.
This also gives us a speed bump of about 4min on all python-only jobs, as the main slowness of
rules_rust
is due to our ruby git dependencies, which we now don't pull in any longer.The new file paths look like this: