Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use current and requested Python versions in requires-python incompatibility errors #986

Merged
merged 8 commits into from
Jan 22, 2024
14 changes: 14 additions & 0 deletions crates/puffin-resolver/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,10 @@ impl From<pubgrub::error::PubGrubError<PubGrubPackage, Range<Version>, Infallibl
pubgrub::error::PubGrubError::NoSolution(derivation_tree) => {
ResolveError::NoSolution(NoSolutionError {
derivation_tree,
// The following should be populated before display for the best error messages
available_versions: FxHashMap::default(),
selector: None,
python_requirement: None,
})
}
pubgrub::error::PubGrubError::SelfDependency { package, version } => {
Expand All @@ -128,6 +130,7 @@ pub struct NoSolutionError {
derivation_tree: DerivationTree<PubGrubPackage, Range<Version>>,
available_versions: FxHashMap<PubGrubPackage, Vec<Version>>,
selector: Option<CandidateSelector>,
python_requirement: Option<PythonRequirement>,
}

impl std::error::Error for NoSolutionError {}
Expand All @@ -137,6 +140,7 @@ impl std::fmt::Display for NoSolutionError {
// Write the derivation report.
let formatter = PubGrubReportFormatter {
available_versions: &self.available_versions,
python_requirement: self.python_requirement.as_ref(),
};
let report =
DefaultStringReporter::report_with_formatter(&self.derivation_tree, &formatter);
Expand Down Expand Up @@ -201,4 +205,14 @@ impl NoSolutionError {
self.selector = Some(selector);
self
}

/// Update the Python requirements attached to the error.
#[must_use]
pub(crate) fn with_python_requirement(
mut self,
python_requirement: &PythonRequirement,
) -> Self {
self.python_requirement = Some(python_requirement.clone());
self
}
}
51 changes: 51 additions & 0 deletions crates/puffin-resolver/src/pubgrub/report.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,17 @@ use rustc_hash::{FxHashMap, FxHashSet};

use crate::candidate_selector::CandidateSelector;
use crate::prerelease_mode::PreReleaseStrategy;
use crate::python_requirement::PythonRequirement;

use super::PubGrubPackage;

#[derive(Debug)]
pub(crate) struct PubGrubReportFormatter<'a> {
/// The versions that were available for each package
pub(crate) available_versions: &'a FxHashMap<PubGrubPackage, Vec<Version>>,

/// The versions that were available for each package
pub(crate) python_requirement: Option<&'a PythonRequirement>,
}

impl ReportFormatter<PubGrubPackage, Range<Version>> for PubGrubReportFormatter<'_> {
Expand All @@ -31,6 +35,53 @@ impl ReportFormatter<PubGrubPackage, Range<Version>> for PubGrubReportFormatter<
format!("we are solving dependencies of {package} {version}")
}
External::NoVersions(package, set) => {
if matches!(package, PubGrubPackage::Python(_)) {
if let Some(python) = self.python_requirement {
if python.target().release().iter().eq(python
.installed()
.release()
.iter()
.take(2))
{
// Simple case, the installed version is the same as the target version
// N.B. Usually the target version does not include anything past the
// minor version mumber so we only compare to part of the installed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are they not exactly the same when not provided? I think I would've expected --python-version 3.10 to be treated as --python-version 3.10.0. I'm just wondering if this will be correct if the user uses a different patch version in the resolution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a target Python version is provided e.g. --python-version 3.11

DEBUG puffin_resolver::resolver Solving with target Python 3.11.6 and installed Python 3.12.0

If the target Python version is inferred from the current version

DEBUG puffin_resolver::resolver Solving with target Python 3.12 and installed Python 3.12.0

If a target Python version is provided with patch version e.g. --python-version 3.11.3

DEBUG puffin_resolver::resolver Solving with target Python 3.11.3 and installed Python 3.12.0

If a target Python version is provided but not installed e.g. --python-version 3.13

DEBUG puffin_resolver::resolver Solving with target Python 3.13 and installed Python 3.12.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We compare the full target to the first two numbers of the installed so we should only have equality here when --python-version is not provided.

Let me know if I can make the this clearer in my comment.

Copy link
Member

@charliermarsh charliermarsh Jan 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are helpful...

(So first, I am assuming that 3.12 and 3.12.0 are considered equal when you use == with Version. I'm pretty sure this is true and per spec. In that case, we shouldn't need to do the .take(2) thing.)

If a target Python version is provided e.g. --python-version 3.11

I initially thought this one was a bug, because 3.11 should've resolved to 3.11.0. But then I realized we do a thing whereby we assume the highest known minor. That behavior kind of feels wrong in hindsight... I think it exists because --python-version 3.7 would otherwise use 3.7.0, and a lot of things aren't compatible with the first few patch releases in 3.7, and that was a confusing user experience in my testing. But it might be wrong...

If the target Python version is inferred from the current version

So these should match based on Version equality, I think.

If a target Python version is provided with patch version e.g. --python-version 3.11.3

These should be considered unequal based on Version equality.

If a target Python version is provided but not installed e.g. --python-version 3.13

This seems right to me, they'd be considered unequal based on version equality. But it shouldn't have to do with whether the target is installed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which case specifically would be wrong if we used == instead of take(2)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would we see 3.12.1 != 3.12 though? Why is one of the versions not inclusive of the patch, when you don't provide a --python-version? Isn't that wrong?

Copy link
Member Author

@zanieb zanieb Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea why it's implemented that way, but yes as noted above if the target version is inferred from the current Python version it will not include the patch version. I'm all for changing that behavior if it was not intentional, but it doesn't seem like we should do it here.

Here's another commit showing the installed version alongside the target one: 43910a8e

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I think the right order of operations is to fix the missing patch version (in a separate PR) and then rebase this on top of that. Otherwise, we're adding a workaround for a bug that we need to fix immediately. But feel free to merge, I see the issue and can fix now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tossed something up here 🤞 #1033

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing it!

// version. If the target version is longer, we'll do the complex
// display instead.
return format!(
"the current {package} version ({}) does not satisfy {}",
python.target(),
PackageRange::compatibility(package, set)
);
}
// Complex case, the target was provided and differs from the installed one
// Determine which Python version requirement was not met
if !set.contains(python.target()) {
return format!(
"the requested {package} version ({}) does not satisfy {}",
python.target(),
PackageRange::compatibility(package, set)
);
}
// TODO(zanieb): Explain to the user why the installed version is relevant
// when they provided a target version; probably via a "hint"
debug_assert!(
!set.contains(python.installed()),
"There should not be an incompatibility where the range is satisfied by both Python requirements"
);
return format!(
"the current {package} version ({}) does not satisfy {}",
python.installed(),
PackageRange::compatibility(package, set)
);
}
// We should always have the required Python versions, if we don't we'll fall back
// to a less helpful message in production
debug_assert!(
false,
"Error reporting should always be provided with Python versions"
);
}
let set = self.simplify_set(set, package);
if set.as_ref() == &Range::full() {
format!("there are no versions of {package}")
Expand Down
22 changes: 11 additions & 11 deletions crates/puffin-resolver/src/python_requirement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,30 @@ use pep508_rs::MarkerEnvironment;
use puffin_interpreter::Interpreter;

#[derive(Debug, Clone)]
pub struct PythonRequirement<'a> {
pub struct PythonRequirement {
/// The installed version of Python.
installed: &'a Version,
installed: Version,
/// The target version of Python; that is, the version of Python for which we are resolving
/// dependencies. This is typically the same as the installed version, but may be different
/// when specifying an alternate Python version for the resolution.
target: &'a Version,
target: Version,
}

impl<'a> PythonRequirement<'a> {
pub fn new(interpreter: &'a Interpreter, markers: &'a MarkerEnvironment) -> Self {
impl PythonRequirement {
pub fn new(interpreter: &Interpreter, markers: &MarkerEnvironment) -> Self {
Self {
installed: interpreter.version(),
target: &markers.python_version.version,
installed: interpreter.version().clone(),
target: markers.python_version.version.clone(),
}
}

/// Return the installed version of Python.
pub(crate) fn installed(&self) -> &'a Version {
self.installed
pub(crate) fn installed(&self) -> &Version {
&self.installed
}

/// Return the target version of Python.
pub(crate) fn target(&self) -> &'a Version {
self.target
pub(crate) fn target(&self) -> &Version {
&self.target
}
}
11 changes: 8 additions & 3 deletions crates/puffin-resolver/src/resolver/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ pub struct Resolver<'a, Provider: ResolverProvider> {
overrides: Overrides,
allowed_urls: AllowedUrls,
markers: &'a MarkerEnvironment,
python_requirement: PythonRequirement<'a>,
python_requirement: PythonRequirement,
selector: CandidateSelector,
index: &'a InMemoryIndex,
/// A map from [`PackageId`] to the `Requires-Python` version specifiers for that package.
Expand Down Expand Up @@ -120,7 +120,7 @@ impl<'a, Provider: ResolverProvider> Resolver<'a, Provider> {
manifest: Manifest,
options: ResolutionOptions,
markers: &'a MarkerEnvironment,
python_requirement: PythonRequirement<'a>,
python_requirement: PythonRequirement,
index: &'a InMemoryIndex,
provider: Provider,
) -> Self {
Expand Down Expand Up @@ -221,7 +221,12 @@ impl<'a, Provider: ResolverProvider> Resolver<'a, Provider> {

// Add version information to improve unsat error messages.
if let ResolveError::NoSolution(err) = err {
ResolveError::NoSolution(err.with_available_versions(&self.python_requirement, &self.index.packages).with_selector(self.selector.clone()))
ResolveError::NoSolution(
err
.with_available_versions(&self.python_requirement, &self.index.packages)
.with_selector(self.selector.clone())
.with_python_requirement(&self.python_requirement)
)
} else {
err
}
Expand Down
4 changes: 2 additions & 2 deletions crates/puffin-resolver/src/resolver/provider.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ pub struct DefaultResolverProvider<'a, Context: BuildContext + Send + Sync> {
/// These are the entries from `--find-links` that act as overrides for index responses.
flat_index: &'a FlatIndex,
tags: &'a Tags,
python_requirement: PythonRequirement<'a>,
python_requirement: PythonRequirement,
exclude_newer: Option<DateTime<Utc>>,
allowed_yanks: AllowedYanks,
no_binary: &'a NoBinary,
Expand All @@ -66,7 +66,7 @@ impl<'a, Context: BuildContext + Send + Sync> DefaultResolverProvider<'a, Contex
fetcher: DistributionDatabase<'a, Context>,
flat_index: &'a FlatIndex,
tags: &'a Tags,
python_requirement: PythonRequirement<'a>,
python_requirement: PythonRequirement,
exclude_newer: Option<DateTime<Utc>>,
allowed_yanks: AllowedYanks,
no_binary: &'a NoBinary,
Expand Down
5 changes: 3 additions & 2 deletions crates/puffin/tests/pip_compile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -687,8 +687,9 @@ fn compile_python_37() -> Result<()> {

----- stderr -----
× No solution found when resolving dependencies:
╰─▶ Because only Python<3.8 is available and black==23.10.1 depends on
Python>=3.8, we can conclude that black==23.10.1 cannot be used.
╰─▶ Because the requested Python version (3.7.17) does not satisfy
Python>=3.8 and black==23.10.1 depends on Python>=3.8, we can conclude
that black==23.10.1 cannot be used.
And because you require black==23.10.1 we can conclude that the
requirements are unsatisfiable.
"###);
Expand Down
14 changes: 7 additions & 7 deletions crates/puffin/tests/pip_install_scenarios.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2554,7 +2554,7 @@ fn requires_python_version_does_not_exist() -> Result<()> {

----- stderr -----
× No solution found when resolving dependencies:
╰─▶ Because only Python<4.0 is available and albatross==1.0.0 depends on Python>=4.0, we can conclude that albatross==1.0.0 cannot be used.
╰─▶ Because the current Python version (3.7) does not satisfy Python>=4.0 and albatross==1.0.0 depends on Python>=4.0, we can conclude that albatross==1.0.0 cannot be used.
And because you require albatross==1.0.0 we can conclude that the requirements are unsatisfiable.
"###);
});
Expand Down Expand Up @@ -2611,7 +2611,7 @@ fn requires_python_version_less_than_current() -> Result<()> {

----- stderr -----
× No solution found when resolving dependencies:
╰─▶ Because only Python>3.8 is available and albatross==1.0.0 depends on Python<=3.8, we can conclude that albatross==1.0.0 cannot be used.
╰─▶ Because the current Python version (3.9) does not satisfy Python<=3.8 and albatross==1.0.0 depends on Python<=3.8, we can conclude that albatross==1.0.0 cannot be used.
And because you require albatross==1.0.0 we can conclude that the requirements are unsatisfiable.
"###);
});
Expand Down Expand Up @@ -2668,7 +2668,7 @@ fn requires_python_version_greater_than_current() -> Result<()> {

----- stderr -----
× No solution found when resolving dependencies:
╰─▶ Because only Python<3.10 is available and albatross==1.0.0 depends on Python>=3.10, we can conclude that albatross==1.0.0 cannot be used.
╰─▶ Because the current Python version (3.9) does not satisfy Python>=3.10 and albatross==1.0.0 depends on Python>=3.10, we can conclude that albatross==1.0.0 cannot be used.
And because you require albatross==1.0.0 we can conclude that the requirements are unsatisfiable.
"###);
});
Expand Down Expand Up @@ -2876,22 +2876,22 @@ fn requires_python_version_greater_than_current_excluded() -> Result<()> {

----- stderr -----
× No solution found when resolving dependencies:
╰─▶ Because there are no versions of Python that satisfy Python>=3.10,<3.11 and only Python<3.12 is available, we can conclude that any of:
╰─▶ Because the current Python version (3.9) does not satisfy Python>=3.10,<3.11 and the current Python version (3.9) does not satisfy Python>=3.12, we can conclude that any of:
Python>=3.10,<3.11
Python>=3.12
are incompatible.
And because there are no versions of Python that satisfy Python>=3.11,<3.12 we can conclude that Python>=3.10 are incompatible.
And because the current Python version (3.9) does not satisfy Python>=3.11,<3.12 we can conclude that Python>=3.10 are incompatible.
And because albatross==2.0.0 depends on Python>=3.10 and there are no versions of albatross that satisfy any of:
albatross>2.0.0,<3.0.0
albatross>3.0.0,<4.0.0
albatross>4.0.0
we can conclude that albatross>=2.0.0,<3.0.0 cannot be used. (1)

Because there are no versions of Python that satisfy Python>=3.11,<3.12 and only Python<3.12 is available, we can conclude that Python>=3.11 are incompatible.
Because the current Python version (3.9) does not satisfy Python>=3.11,<3.12 and the current Python version (3.9) does not satisfy Python>=3.12, we can conclude that Python>=3.11 are incompatible.
And because albatross==3.0.0 depends on Python>=3.11 we can conclude that albatross==3.0.0 cannot be used.
And because we know from (1) that albatross>=2.0.0,<3.0.0 cannot be used, we can conclude that albatross>=2.0.0,<4.0.0 cannot be used. (2)

Because only Python<3.12 is available and albatross==4.0.0 depends on Python>=3.12, we can conclude that albatross==4.0.0 cannot be used.
Because the current Python version (3.9) does not satisfy Python>=3.12 and albatross==4.0.0 depends on Python>=3.12, we can conclude that albatross==4.0.0 cannot be used.
And because we know from (2) that albatross>=2.0.0,<4.0.0 cannot be used, we can conclude that albatross>=2.0.0 cannot be used.
And because you require albatross>=2.0.0 we can conclude that the requirements are unsatisfiable.
"###);
Expand Down