Skip to content

Commit

Permalink
Fix non-determinism in bootstrap scalac actions when executing outsid…
Browse files Browse the repository at this point in the history
…e of a sandbox

This fixes a bug where every bootstrap scalac action was writing class
files to the same directory: tmp/classes. Each action would run scalac
with the output directory set to tmp/classes and then create a jar from
all the class files in that directory.

If you're only running a single bootstrap scalac action, then this works
fine.

The bug happens when you run multiple bootstrap scalac actions because
you build up class files in the tmp/classes. This causes the jars you
produce to contain not only the current target's class files, but also
any class files from previous bootstrap scalac actions. Which specific
class files are in your jar depend on what actions have run up until
that point in time. This is not deterministic.

When executing with the sandboxed strategy this bug wasn't an issue
because the sandbox kept actions sufficiently isolated from each other.
Meaning you'd get only your action's classes in the output jar.
  • Loading branch information
James Judd committed Jul 26, 2024
1 parent 4077774 commit 2ba3db6
Showing 1 changed file with 24 additions and 20 deletions.
44 changes: 24 additions & 20 deletions rules/private/phases/phase_bootstrap_compile.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ def phase_bootstrap_compile(ctx, g):
transitive = [ctx.attr._jdk[java_common.JavaRuntimeInfo].files, g.classpaths.compile, g.classpaths.compiler],
)

tmp = ctx.actions.declare_directory("{}/tmp/classes".format(ctx.label.name))

compiler_classpath = ":".join([f.path for f in g.classpaths.compiler.to_list()])
compile_classpath = ":".join([f.path for f in g.classpaths.compile.to_list()])
srcs = " ".join([f.path for f in g.classpaths.srcs])
Expand All @@ -31,37 +33,39 @@ def phase_bootstrap_compile(ctx, g):
if int(scala_configuration.version[0]) >= 3:
main_class = "dotty.tools.dotc.Main"

ctx.actions.run_shell(
inputs = inputs,
tools = [ctx.executable._jar_creator],
outputs = [g.classpaths.jar],
command = _strip_margin(
"""
command = _strip_margin(
"""
|set -eo pipefail
|
|mkdir -p tmp/classes
|
|{java} \\
| -cp {compiler_classpath} \\
| {main_class} \\
| -cp {compile_classpath} \\
| -d tmp/classes \\
| -d {tmp} \\
| {global_scalacopts} \\
| {scalacopts} \\
| {srcs}
|
|{jar_creator} {output_jar} tmp/classes 2> /dev/null
|{jar_creator} {output_jar} {tmp} 2> /dev/null
|""".format(
compiler_classpath = compiler_classpath,
compile_classpath = compile_classpath,
global_scalacopts = " ".join(scala_configuration.global_scalacopts),
java = ctx.attr._jdk[java_common.JavaRuntimeInfo].java_executable_exec_path,
jar_creator = ctx.executable._jar_creator.path,
main_class = main_class,
output_jar = g.classpaths.jar.path,
scalacopts = " ".join(ctx.attr.scalacopts),
srcs = srcs,
),
compiler_classpath = compiler_classpath,
compile_classpath = compile_classpath,
global_scalacopts = " ".join(scala_configuration.global_scalacopts),
java = ctx.attr._jdk[java_common.JavaRuntimeInfo].java_executable_exec_path,
jar_creator = ctx.executable._jar_creator.path,
main_class = main_class,
output_jar = g.classpaths.jar.path,
scalacopts = " ".join(ctx.attr.scalacopts),
srcs = srcs,
tmp = tmp.path,
),
)

ctx.actions.run_shell(
inputs = inputs,
tools = [ctx.executable._jar_creator],
mnemonic = "BootstrapScalacompile",
outputs = [g.classpaths.jar, tmp],
command = command,
execution_requirements = _resolve_execution_reqs(ctx, {}),
)

0 comments on commit 2ba3db6

Please sign in to comment.