apache-airflow[all] resolution. #74
Assignees
Labels
Comments
tdejager
changed the title
|
So #84 now actually gets a lot further regarding this issue. We now get this error Error reading metadata from artifact 'snowflake-connector-python-2.9.0.tar.gz' skipping (Could not resolve environment for wheel building)
thread 'main' panicked at 'could not find metadata for any sdist or wheel for snowflake-connector-python 2.9.0. The following artifacts are available:
- snowflake-connector-python-2.9.0.tar.gz', crates/rattler_installs_packages/src/resolve.rs:479:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtracewhich is caused by needing recursive sdist support for which we need to solve #79 so we can install the built wheel. |
|
So now I'm having the problem where on my machine I need to think of a good way to debug this, maybe @notatallshaw has an idea. I think it being caused by |
|
I would love to help but I don't really have any techniques or tools to investigaate why rip/resolvo made a certain choice like I do with pip/resolvelib. By the way I get a very different error on Linux and pinning the version: $ cargo r -- apache-airflow[all]==2.7.3
...
2023-12-05T23:57:13.796410Z INFO rattler_installs_packages::resolve::dependency_provider: collecting apache-airflow[kerberos]
× Could not solve for the requested requirements:
│ The following packages are incompatible
│ |-- apache-airflow[all] ==2.7.3 cannot be installed because there are no viable options:
│ |-- apache-airflow[all] 2.7.3 would require
│ |-- opentelemetry-exporter-prometheus *, which cannot be installed because there are no viable options:
│ |-- opentelemetry-exporter-prometheus 1.12.0rc1 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 1.10a0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.42b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.41b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.40b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.39b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.38b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.37b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.36b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.35b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.34b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.33b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.32b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.30b1 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.30b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.29b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.17b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.16b1 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.16b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.15b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.14b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.13b0 is excluded because prereleases are not allowed
│ |-- opentelemetry-exporter-prometheus 0.12b0 is excluded because prereleases are not allowedI beleive according to the spec an installer can (should?) choose a pre release package if there is no other option. |
|
Here's the spec on handling pre-releases: https://packaging.python.org/en/latest/specifications/version-specifiers/#handling-of-pre-releases By the way, I think Pip doesn't handle this consistently between selecting packages initially (will select a pre release package if no other option is available even if |
|
I definetly think this contributes to why rip backtracks to such old versions of packages when trying the requirement |
|
Thanks so much for looking into this! I'm first working on integrating it back into pixi and will look at this next I think 👍 |
I wanted to ask as well, what tools do you use to debug these things with pip/resolvelib? |
Ah, I think you're going to be fairly disappointed. I usually start by putting some print statements around here that print out all unique cause names: https://github.com/pypa/pip/blob/23.3.1/src/pip/_vendor/resolvelib/resolvers.py#L430. This gives me a sense for what packages are changing the path of resolution from resolvelib's natural DFS, I also see when they're printed in the context of the logs, so it gives me a general idea what requirements have just been collected. From that point it depends what the problem is. For a complex backtracking I might add more information in the prints, such as the package versions of the causes, and also their parents name and version, this will allow me to construct a chain of what led to the final result. If the issue is something that used to work but no longer does I might use pypi-timemachine and run the resolution on two different dates and see what the differences are. And otherwise I will just change the requirements by adding upper bounds, changing the order, adding transitive dependencies into the requirements, and see how it affects it. Often though this is more to help a user who is stuck and needs to find a solution so they can continue to work. |
|
Cool! Thanks for giving insight into you problem-solving, I think for resolvo we also do a lot of print statements, when debugging. The large number of clauses can still make this pretty difficult though :) I didn't know about |
1 task
|
With #141 (which will hopefully be merged soon), I can resolve My results: |
|
It's broken again on main because of #178 and I get to a point where I cannot build for the same reason as mentioned in that thread. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Apache airflow seems to be a good package to test because it has a lot of dependencies.
Currently, it resolve to this version:
with no dependencies listed, which is incorrect, because of this probably:
#71 (comment)
Let's track this issue for when we can actually solve it. We need #32 for this as well with sdist building.
The text was updated successfully, but these errors were encountered: