Build (after clean) with disk cache is missing empty subdirectories

[george-enf]

Description of the bug:

I build a rule with disk cache enabled. I've added a 5 second delay in rule1.sh to prove that it is executing:

$ bazel build //:rule1 --disk_cache=diskcache
Target //:rule1 up-to-date:
  bazel-bin/tree
  bazel-bin/empty_dir
INFO: Elapsed time: 5.092s, Critical Path: 5.02s

The rule generates 2 directory trees. The empty_dir is completely empty. And the tree has an empty subdirectory.

$ find bazel-bin/
bazel-bin/
bazel-bin/empty_dir
bazel-bin/tree
bazel-bin/tree/subdir
bazel-bin/tree/subdir/file
bazel-bin/tree/empty_subdir

I do:

$ bazel clean

Then, I rebuild. rule1.sh is not re-executed.

$ bazel build //:rule1 --disk_cache=diskcache
Target //:rule1 up-to-date:
  bazel-bin/tree
  bazel-bin/empty_dir
INFO: Elapsed time: 0.163s, Critical Path: 0.01s

The empty subdirectory empty_subdir is missing:

$ find bazel-bin/
bazel-bin/
bazel-bin/empty_dir
bazel-bin/tree
bazel-bin/tree/subdir
bazel-bin/tree/subdir/file

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

BUILD

genrule(
    name = "rule1",
    cmd = "./rule1.sh $(RULEDIR)",
    tools = [
        "rule1.sh",
    ],
    outs = [
        "tree",
        "empty_tree",
    ],
)

rule1.sh

#!/bin/bash

set -e

RULEDIR="$1"
mkdir -p "$RULEDIR"
cd "$RULEDIR"

sleep 5

mkdir -p tree/subdir tree/empty_subdir
echo test > tree/subdir/file

mkdir -p empty_tree

Which operating system are you running Bazel on?

Ubuntu 20.04.1 LTS

What is the output of bazel info release?

release 5.2.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

It looks like there is special handing for empty directories here:

bazel/src/main/java/com/google/devtools/build/lib/remote/RemoteExecutionService.java

Lines 1086 to 1090 in 24a4941

	for (Entry<Path, DirectoryMetadata> entry : metadata.directories()) {
	// For empty output directories, create the empty directory itself
	entry.getKey().createDirectoryAndParents();
	symlinksInDirectories.addAll(entry.getValue().symlinks());
	}

I am not sure how empty subdirectories are/should be handled.

[fmeum]

@coeuvre Resolving this would probably require generalizing what I did in #15276 to apply to all empty (leaf) directories, not just the top-level one.

What do you think of the approach to add a new artifact type representing an empty directory? I think that this could even clean up some of the existing logic introduced for #15276.

[meisterT]

@tjgq did you look into this recently?

[tjgq]

I had a chat with @coeuvre. We believe the issue is that we fail to create empty directories here when downloading from the remote cache.

[tjgq]

I think this issue actually runs deeper: we don't track empty directories inside tree artifacts properly (see #15901). However, it should be possible to fix the disk cache issue separately.

[coeuvre]

Duplicated of #15789.

[brentleyjones]

@coeuvre duplicated what? Did you mean #15901?

[coeuvre]

Argh, yes, that's what I had in my mind.