Argument globbing (expansion) on Windows causes arguments intended as patterns to be prematurely expanded

[ackerleytng]

In ls for example, the --ignore flag expects a pattern.

In windows, these flags are prematurely expanded, so patterns become actual files, which are then ignored when they should not be.

For example, in test_ls_ignore_explicit_period, two files are created:

• .hidden.yml
• regular.yml

When this command is executed: ls -a --ignore ?hidden.yml, the desired outcome is that both .hidden.yml and regular.yml are listed, because filename matching in ls requires explicit .s for hidden files.

On windows, the ?hidden.yml is expanded by wild::args() to .hidden.yml. When matching, the new argument .hidden.yml is a perfect match, and has a leading ., which causes it to be hidden (not the expected output)

More details here: #3803

The premature globbing causes problems in some cases. We should investigate how the globbing can be refactored.

[tertsdiepraam]

This is mostly an issue with testing I believe. In both unix and windows, the expansion happens, but for one it happens in the shell and for the other in the util. Hence, it only happens in the tests for windows. So if we can disable it when calling it from the test suite, I think we're good. Apart from the tests, we could look into using an env var or something to control the expansion.

[ackerleytng]

We don't want the expansion for all arguments. For one, when the argument passed is meant to be a pattern, the expansion of the pattern messes up the user's intention.

I believe the expansion happens outside of tests too.

[tertsdiepraam]

But the same happens on unix shells, doesn't it? The shell doesn't have any idea about what an argument is for and whether to apply the expansion.

I believe the expansion happens outside of tests too.

It does, I'm not questioning that. But in my opinion, there's only a problem when it comes to testing. In normal use, the expansion is expected behavior.

[ackerleytng]

It does happen on unix shells, but that expansion can be controlled by escaping, so a user could do ls -a --ignore '?hidden.yml' to pass the pattern to ls.

By applying expansion through wild, the user's intention is modified (in the case where passing patterns are the intention). The example in the issue description is an example of how the glob expansion causes .hidden.yml to be excluded from the listing when this file, according to behavior on Linux and the man pages, should not be excluded from the listing.

[tertsdiepraam]

Quoting also stops the expansion in wild, at least according to the docs, I don't have a Windows machine to verify.

[ackerleytng]

I verified that the expansion is excluded when quotes are applied: (I hacked ls to just print arguments)

PS C:\Users\Quickemu\Documents\coreutils> .\target\debug\coreutils.exe ls -a --ignore '?hidden.yml'
["ls", "-a", "--ignore", "?hidden.yml"]
PS C:\Users\Quickemu\Documents\coreutils> .\target\debug\coreutils.exe ls -a --ignore "?hidden.yml"
["ls", "-a", "--ignore", "?hidden.yml"]
PS C:\Users\Quickemu\Documents\coreutils> .\target\debug\coreutils.exe ls -a --ignore "?"hidden.yml
["ls", "-a", "--ignore", "?", "hidden.yml"]
PS C:\Users\Quickemu\Documents\coreutils>

With .arg, the quotes get passed all the way through:

   ... some test output elided ...
current_directory_resolved:
touch: C:\Users\Quickemu\AppData\Local\Temp\.tmpETTgWG\.hidden.yml
touch: C:\Users\Quickemu\AppData\Local\Temp\.tmpETTgWG\regular.yml
run: C:\Users\Quickemu\Documents\coreutils\target\debug\coreutils.exe ls -a --ignore '?hidden.yml'
thread 'test_ls::test_ls_ignore_explicit_period' panicked at ''["ls", "-a", "--ignore", "\'?hidden.yml\'"
]
' does not contain '.hidden.yml'', tests\common\util.rs:410:9
   ... some test output elided ...