Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistencies with globs in MatchSpec parsing #3601

Open
3 tasks done
jaimergp opened this issue Nov 13, 2024 · 7 comments
Open
3 tasks done

Inconsistencies with globs in MatchSpec parsing #3601

jaimergp opened this issue Nov 13, 2024 · 7 comments

Comments

@jaimergp
Copy link
Contributor

Troubleshooting docs

  • My problem is not solved in the Troubleshooting docs

Anaconda default channels

  • I do NOT use the Anaconda default channels (pkgs/* etc.)

How did you install Mamba?

Other (please describe)

Search tried in issue tracker

matchspec glob

Latest version of Mamba

  • My problem is not solved with the latest version

Tried in Conda?

I do not have this problem with Conda, just with Mamba

Describe your issue

I have detected some parsing / roundtrip issues with libmamba MatchSpec objects:

This should be valid (although it's a redundant version glob):

>>> print(MatchSpec.parse("name *,*.* build*"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
libmambapy.bindings.specs.ParseError: Found invalid version predicate in "*.*"
>>> print(MatchSpecConda("name *,*.* build*"))
name[version='*,*.*',build=build*]

Also, when passing this spec to the solver, the error message is lossy. Note how the trailing * is lost in the build string:

 {
  "Install": [
    "accelerate[version=\"=*,=*\",build=\"np17py27*\"]"
  ]
}
Could not solve for environment specs
The following package could not be installed
└─ accelerate =*,=* np17py27 does not exist (perhaps a typo or a missing channel).

mamba info / micromamba info

libmamba version : 2.0.3
          mamba version : 2.0.3
           curl version : libcurl/8.10.1 OpenSSL/3.4.0 zlib/1.3.1 zstd/1.5.6 libssh2/1.11.0 nghttp2/1.58.0
     libarchive version : libarchive 3.7.4 zlib/1.2.13 liblzma/5.2.6 bz2lib/1.0.8 liblz4/1.9.3 libzstd/1.5.6
       envs directories : /opt/conda/envs
          package cache : /opt/conda/pkgs
                          /root/.mamba/pkgs
            environment : base (active)
           env location : /opt/conda
      user config files : /root/.mambarc
 populated config files : /opt/conda/.condarc
       virtual packages : __unix=0=0
                          __linux=5.15.49=0
                          __glibc=2.31=0
                          __archspec=1=aarch64
               channels : https://conda.anaconda.org/conda-forge/linux-aarch64
                          https://conda.anaconda.org/conda-forge/noarch
       base environment : /opt/conda

Logs

No response

environment.yml

No response

~/.condarc

No response

@Hind-M
Copy link
Member

Hind-M commented Nov 20, 2024

Is *.* supposed to be equivalent to * in this case? Or is it supposed to be handled as a regex?

@jaimergp
Copy link
Contributor Author

If you ask me, I understanding as "any version that has at least two components". So * would match [1, 2, 1.2, 2.3.4], but *.* would only match [1.2, 2.3.4] in the same list.

I checked and conda seems to do that:

$ conda search -c bioconda "anvio=*"
Loading channels: done
# Name                       Version           Build  Channel             
anvio                              6               0  bioconda            
anvio                            6.1               0  bioconda            
anvio                            6.1               1  bioconda            
anvio                            6.2               0  bioconda            
anvio                              7               0  bioconda            
anvio                              7      hdfd78af_1  bioconda            
$ conda search -c bioconda "anvio=*.*"
Loading channels: done
# Name                       Version           Build  Channel             
anvio                            6.1               0  bioconda            
anvio                            6.1               1  bioconda            
anvio                            6.2               0  bioconda            

@Hind-M
Copy link
Member

Hind-M commented Nov 21, 2024

Hmm I see. In mamba 2, it seems that the version is not handled as a regex at all. Antoine suggested this CEP and was pushing IINM to not use regex in versions to avoid complexifying things.
I don't know if it's always possible to express a version as described in the mentioned CEP.
Or said differently, what are the concrete cases where a regex expression cannot be expressed with an operator and numbers to be handled?
I see that *.* is one case for example...

EDIT: Maybe this is just a missing part that we need to implement in mamba 2...

@jdblischak
Copy link
Contributor

In mamba 2, it seems that the version is not handled as a regex at all.

This seems like a substantial backwards incompatibility that should be added to the list of breaking changes introduced in mamba 2. Should I submit a PR?

@jdblischak
Copy link
Contributor

jdblischak commented Dec 3, 2024

Or said differently, what are the concrete cases where a regex expression cannot be expressed with an operator and numbers to be handled?

My use case is nightly feedstock builds (xref: TileDB-Inc/conda-forge-nightly-controller#151 (comment)).

We build the latest version of our C++ library and upload it as a conda binary with the version string X.X.X.YYYY_MM_DD. Then we build the latest version of our Python client, and the recipe pins the C++ library with the requirement *.YYYY_MM_DD. This allows us to always install the nightly version of the C++ library without worrying about the current version of the C++ library (which is always changing).

@Hind-M
Copy link
Member

Hind-M commented Dec 4, 2024

In mamba 2, it seems that the version is not handled as a regex at all.

This seems like a substantial backwards incompatibility that should be added to the list of breaking changes introduced in mamba 2. Should I submit a PR?

Yes a PR is most welcome, thank you!

@jdblischak
Copy link
Contributor

Actually I am a little confused. I was going to write that the breaking change was that you could no longer use globs in version restrictions because the version parsing was no longer handled as a regex. However, looking at the documentation for VersionSpec, it appears that it isn't that straightforward.

From my experiments (code below), I found that:

  • Trailing globs work as expected
  • Internal globs are accepted but do not work
  • Leading globs fail with invalid version predicate
import libmambapy

libmambapy.version.version_info
## ('2', '0', '4')

import libmambapy.specs as specs

specs.VersionSpec.parse("0.0.*")
specs.VersionSpec.parse("0.*.0")
specs.VersionSpec.parse("*.0.0")
## Traceback (most recent call last):
##   File "<stdin>", line 1, in <module>
## libmambapy.bindings.specs.ParseError: Found invalid version predicate in "*.0.0"

# Trailing glob works as expected
specs.VersionSpec.parse("0.0.*").contains(specs.Version.parse("0.0.1"))
## True
specs.VersionSpec.parse("0.0.*").contains(specs.Version.parse("1.0.0"))
## False

# Internal glob accepted but does not work
specs.VersionSpec.parse("0.*.0").contains(specs.Version.parse("0.1.0"))
## False
specs.VersionSpec.parse("0.*.0").contains(specs.Version.parse("1.0.0"))
## False

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants