Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new flags to facilitate automated forge #1175

Closed
hadim opened this issue Nov 9, 2024 · 3 comments
Closed

Add new flags to facilitate automated forge #1175

hadim opened this issue Nov 9, 2024 · 3 comments

Comments

@hadim
Copy link
Contributor

hadim commented Nov 9, 2024

I am trying to setup a private forge with a mix of inter-dependents noarch and arch-specific packages. The setup is similar to https://github.com/prefix-dev/ai-forge/.

The variants file looks like this.
zip_keys:
  - - ${{ "cxx_compiler_version" if unix }}
    - ${{ "c_compiler_version" if unix }}
  - - python
    - numpy
    - python_impl

c_stdlib:
  - ${{ "sysroot" if linux }}
  - ${{ "macosx_deployment_target" if osx }}
  - ${{ "vs" if win }}

c_stdlib_version:
  - ${{ "2.17" if linux }}
  - ${{ "10.13" if osx and x86_64 }}
  - ${{ "11.0" if osx and arm64 }}

c_compiler:
  - ${{ "gcc" if linux }}
  - ${{ "clang" if osx }}
  - ${{ "vs2019" if win and x86_64 }}
  - ${{ "vs2022" if win and arm64 }}

c_compiler_version:
  - ${{ "13" if linux }}
  - ${{ "17" if osx }}

cxx_compiler:
  - ${{ "gxx" if linux }}
  - ${{ "clangxx" if osx }}
  - ${{ "vs2019" if win and x86_64 }}
  - ${{ "vs2022" if win and arm64 }}

cxx_compiler_version:
  - ${{ "13" if linux }}
  - ${{ "17" if osx }}

rust_compiler:
  - rust

rust_compiler_version:
  - 1.80.0

python:
  # part of a zip_keys: python, python_impl, numpy
  - 3.12.* *_cpython
python_impl:
  # part of a zip_keys: python, python_impl, numpy
  - cpython
python_min:
  # minimum supported python version per CFEP-25
  # bump to next minor version when we drop python versions
  - "3.12"

# we build for the oldest version possible of numpy for forward compatibility
# we roughly follow NEP29 in choosing the oldest version
numpy:
  # part of a zip_keys: python, python_impl, numpy
  - 1.26

Here are two flags I think could improve the single-repo based way of setting up a forge.

  • rattler-build build --no-test-if-emulate:

Do not run tests if target-platform != build-platform. Sometimes a test-time dep is not available on osx-arm64 while the package has been built on osx-64.

  • rattler-build build --noarch-platform=linux-64

Only buildnoarch packages if build-platform == noarch-platform (set to linux-64 on c-f for example). It guarantees that at least one platform can build a noarch package (for example, many c-f noarch packages do not work on Windows because of missing pytorch).

That flag could also be something like --skip-noarch but the former one is easier to parametrized in the GHA CI.

@wolfv
Copy link
Member

wolfv commented Nov 13, 2024

Hi @hadim thank you for the two suggestions. They both make a lot of sense to me. I think both could be achieved with what's there right now:

For building noarch packages only on e.g. linux-64 we could do:

build:
  skip:
      - build_platform != "linux-64"

For skipping tests on emulation we coudl do:

tests:
  - if: build_platform == target_platform
     then: ...

But your suggestions also make total sense to me and we should implement them!

@hadim
Copy link
Contributor Author

hadim commented Nov 13, 2024

I like your workarounds. Good idea.

But your suggestions also make total sense to me and we should implement them!

I might give it a try at some point then.

@hadim
Copy link
Contributor Author

hadim commented Nov 15, 2024

closed by #1192 and #1190

@hadim hadim closed this as completed Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants