Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 23.10 #511

Closed
jakirkham opened this issue Nov 7, 2023 · 18 comments · Fixed by #518 or #523
Closed

Release 23.10 #511

jakirkham opened this issue Nov 7, 2023 · 18 comments · Fixed by #518 or #523

Comments

@jakirkham
Copy link
Member

Now that Conda 23.10 is released and the blogpost is up. Congratulations to the team for making that happen 👏

What would be our next steps for packaging it in Miniforge?

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 7, 2023

we used to have troubles with boa compatibility.

if the tests pass. I'm ok with a new release

@jakirkham
Copy link
Member Author

Yes this is a good point

Raised this item at a previous core meeting, the feedback we got there was to switch to using conda-build + conda-libmamba-solver, which ships (and is default) in this release

So maybe this provides a way forward?

cc @jaimergp @wolfv (in case you have anything to add here)

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 7, 2023

i'm ok with removing the boa test. A year ago, without it, it would cause a bunch of downgrades to happen during the build process, which can sometimes be problematic.

@isuruf
Copy link
Member

isuruf commented Nov 7, 2023

The plan me and @jaimergp talked about is to first release conda-smithy which will switch to conda-build+conda-libmamba-solver by default, wait to see if there are any issues and if there are no issues, release a new mambaforge.

@isuruf
Copy link
Member

isuruf commented Nov 7, 2023

The other solution was to wait for a new boa release. See mamba-org/boa#381

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 7, 2023

great sounds awesome!

@jakirkham
Copy link
Member Author

Thanks Isuru! 🙏

This helps answer the question of what the steps would be

first release conda-smithy which will switch to conda-build+conda-libmamba-solver by default, wait to see if there are any issues and if there are no issues, release a new mambaforge

If we want to do more testing first, that sounds reasonable

The other solution was to wait for a new boa release. See mamba-org/boa#381

If we think there will be movement there, great!

That said, it looks like that issue has been open for a week without response

Also this was brought up at the core meeting a few weeks ago ( after months of being in a holding pattern around various incompatibilities: conda-forge/boa-feedstock#73 )

To be clear this is not intended as criticism towards anyone. There have been lots of changes in tooling and it takes work to keep things in a compatible state

That said, think taking a practical approach to resolving this (like moving forward here and letting boa catch up when they are able) may be a better approach for all involved. We can always add back the boa test once things are in a working state

@jaimergp
Copy link
Member

jaimergp commented Nov 7, 2023

The plan me and @jaimergp talked about is to first release conda-smithy

Yep! Anything holding back a conda-smithy release this week?

@jakirkham
Copy link
Member Author

Updating the thread, conda-smithy release 3.28.0 is packaged ( conda-forge/conda-smithy-feedstock#267 )

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 8, 2023

I just really don't understand the constraints of boa. it says it can support newer mamba, but then the Miniforge builder complains
https://github.com/conda-forge/boa-feedstock/blob/main/recipe/meta.yaml#L27

@isuruf
Copy link
Member

isuruf commented Nov 8, 2023

@hmaarrfk, the latest builds were marked broken due to an issue with building recipes on conda-forge.

@jakirkham
Copy link
Member Author

Yep the PR that made that change has a comment at the end that provides more context: conda-forge/boa-feedstock#74 (comment)

Agree the situation around boa is confusing. Have been confused myself at times in trying to understand what is going on there

That said, as the newest conda defaults to libmamba as the solver and conda-build will use it too. Just switching to conda-build should give us a path forward in the interim

This approach was tested in staged-recipes and is now being tested in feedstocks with the latest conda-smithy. If all goes smoothly, we can use a similar approach here

@jakirkham
Copy link
Member Author

cc @jezdez (for awareness)

@jaimergp
Copy link
Member

The problem right now is that the latest conda-build is not compatible with boa. And if we update the Miniforge distro here with the latest stuff, boa will cause a conflict when it's installed in-feedstock.

That's not a problem if we switch to conda-build + conda-libmamba-solver. IN fact, we have done so I think? But some feedstocks are incompatible due to a bug I'm chasing in conda/conda-libmamba-solver#386. These feedstocks need boa for mambabuild.

So I see three solutions:

  • Make boa compatible with the latest conda-build. This just needs a release, but Wolf detect some last minute bugs so not sure what's going to take.
  • Make the Miniforge version configurable in conda-forge.yml. Instead of defaulting to latest we can let users pick an old one. This would allow us to release a new Miniforge with 23.10.
  • We wait until I fix the CLS bug.

boa users would need to pin to 23.3 or something via conda-forge.yml. e.g.

conda_build_tool: mambabuild
conda_install_tool: mamba
miniforge_version: 23.3.1-0  # this key doesn't exist yet

Thoughts?

@hmaarrfk
Copy link
Contributor

Make boa compatible with the latest conda-build. This just needs a release, but Wolf detect some last minute bugs so not sure what's going to take.

I feel like we should choose this. though it is unclear to me exactly what bugs exist.

they can't be irreconcilable....

@jakirkham
Copy link
Member Author

Thanks for summarizing the situation Jaime! 🙏

That is also my understanding. Hope that helps for other folks running into this issue without context


While in principle I understand why folks would want boa to be compatible, it is practically out-of-our-hands

In a prior core meeting, we discussed options and decided to drop the boa dependency (Wolf supported this)

The next step we decided was to start using conda-build with the libmamba solver more broadly (as Isuru explained above #511 (comment) ). That way we can test and find issues

We have been doing that for a little bit. We found a few bugs (expected given we went from testing in staged-recipes to testing in all of conda-forge). The conda-libmamba-solver team has been prompt in resolving them

A new Miniforge would use the new solver by default. So think we want to be confident that is working as expected. See the current bug as just part of that effort we had committed to

Think once we are comfortable with the conda-libmamba-solver situation. We can go ahead with a release


To summarize:

If 1 happens, great! Just don't think we should block on that any more

If we need to go faster for some reason, would also be ok with 2, but would need to be convinced as to why we need to go faster (since this will also result in an uptick in Miniforge user issues)

Would much prefer 3. It is consistent with what we have done so far and should minimize disruption (while letting us to continue to make progress)

@h-vetinari
Copy link
Member

Make boa compatible with the latest conda-build. It needs a release, but Wolf detect some last minute bugs so not sure what's going to take.

Presumably that release was 0.16? In any case, both before and after conda-forge/boa-feedstock#77, the boa recipe did not specify a cap on the conda-build version.

@hmaarrfk hmaarrfk reopened this Nov 26, 2023
@hmaarrfk
Copy link
Contributor

Reopenning since #518 was merged in haste and will likely be reverted (or at the very least, not marked as the latest release)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
5 participants