Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add pytorch integration guide. #2711

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
aad6eb4
docs: add pytorch integration guide.
ruben-arts Dec 16, 2024
45a6c67
Update docs/features/pytorch.md
ruben-arts Dec 16, 2024
bff97ec
Update docs/features/pytorch.md
ruben-arts Dec 16, 2024
6e466de
Update docs/features/pytorch.md
ruben-arts Dec 16, 2024
bc9c713
docs: improve docs further
ruben-arts Dec 16, 2024
9061c64
docs: fix pytorch examples on windows
ruben-arts Dec 17, 2024
4410131
docs: cleanup example snippets
ruben-arts Dec 17, 2024
6b07f5b
docs: improve system requirements docs
ruben-arts Dec 17, 2024
d721ef1
docs: improve admonitions size
ruben-arts Dec 17, 2024
4d43665
docs: improve Troubleshooting
ruben-arts Dec 17, 2024
92e10ab
docs: improve Troubleshooting
ruben-arts Dec 17, 2024
c00c63c
docs: add platform mix note for mac and pypi
ruben-arts Dec 17, 2024
156f1fc
test: test the pytorch documentation examples
ruben-arts Dec 17, 2024
b3b7c52
ci: add extra slow tests to run only on main or label
ruben-arts Dec 17, 2024
e9bd5ac
lint
ruben-arts Dec 17, 2024
fc6d294
ci: add long running tests to multiple platforms
ruben-arts Dec 17, 2024
44b8b66
ci: manage timeout only in CI
ruben-arts Dec 17, 2024
075588a
ci: typo in label name
ruben-arts Dec 17, 2024
bd9515b
ci debug labels
ruben-arts Dec 17, 2024
55bd68a
remove label debug
ruben-arts Dec 17, 2024
0491de3
improve cuda-version docs
ruben-arts Dec 17, 2024
3bf81bb
add win to pytorch channel
ruben-arts Dec 17, 2024
2ab21f2
Merge branch 'main' into docs/add_pytorch_integration_guide
ruben-arts Dec 18, 2024
946ef3b
debug ci test
ruben-arts Dec 18, 2024
9657dd5
more debugging
ruben-arts Dec 18, 2024
5b3db81
fix: ci and move around the tests
ruben-arts Dec 18, 2024
aac34be
ci: clean cache on windows to avoid running out of it
ruben-arts Dec 18, 2024
1ce4400
Merge branch 'main' into docs/add_pytorch_integration_guide
ruben-arts Dec 18, 2024
b548079
Update tests/integration_python/common.py
ruben-arts Dec 18, 2024
93af9f3
fix: typo
ruben-arts Dec 18, 2024
4c3845a
test: move and add clean
ruben-arts Dec 19, 2024
ffe7be1
Merge remote-tracking branch 'upstream/main' into docs/add_pytorch_in…
ruben-arts Dec 19, 2024
45804be
make sure pixi clean is run
ruben-arts Dec 19, 2024
6540cec
make ci use bash
ruben-arts Dec 19, 2024
7b8d8b7
split slow and fast tests
ruben-arts Dec 19, 2024
866c25c
split downstream from tests
ruben-arts Dec 19, 2024
3971add
undo removing in test
ruben-arts Dec 19, 2024
8c5b138
Merge remote-tracking branch 'upstream/main' into docs/add_pytorch_in…
ruben-arts Dec 19, 2024
3482962
ci: improve names
ruben-arts Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 201 additions & 43 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -351,9 +351,9 @@ jobs:
# Run integration tests on important platforms
#

test-integration-windows-x86_64:
timeout-minutes: 30
name: pytest integration | windows x86_64
test-pytest-windows-x86_64:
timeout-minutes: 10
name: Pytest | windows x86_64
runs-on: windows-latest
needs: build-binary-windows-x86_64
env:
Expand All @@ -372,48 +372,223 @@ jobs:
name: pixi-windows-x86_64-${{ github.sha }}
path: ${{ env.PIXI_WORKSPACE }}/${{ env.TARGET_RELEASE }}
- name: Verify pixi installation
run: pixi --version
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi info

- name: Install pixi
- name: Run pytests
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi install -v
run: pixi run --locked test-integration-ci

test-pytest-macos-aarch64:
timeout-minutes: 10
name: Pytest | macos aarch64
runs-on: macos-14
needs: build-binary-macos-aarch64
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-macos-aarch64-${{ github.sha }}
path: ${{ env.TARGET_RELEASE }}
- name: Setup unix binary, add to github path
run: |
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi info

- name: Run integration tests
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi run --locked test-integration-ci

test-pytest-linux-x86_64:
timeout-minutes: 10
name: Pytest | linux x86_64
runs-on: 8core_ubuntu_latest_runner
needs: build-binary-linux-x86_64
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-linux-x86_64-${{ github.sha }}
path: ${{ env.TARGET_RELEASE }}
- name: Setup unix binary, add to github path
run: |
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi info

- name: Run integration tests
run: pixi run --locked test-integration-ci

test-integration-windows-x86_64:
timeout-minutes: 30
name: Integration tests | windows x86_64
runs-on: windows-latest
needs: build-binary-windows-x86_64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Create Dev Drive using ReFS
run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1
- name: Copy Git Repo to Dev Drive
run: |
Copy-Item -Path "${{ github.workspace }}" -Destination "${{ env.PIXI_WORKSPACE }}" -Recurse
echo "${{ env.PIXI_WORKSPACE }}/${{ env.TARGET_RELEASE }}" | Out-File -Append -Encoding utf8 -FilePath $env:GITHUB_PATH
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-windows-x86_64-${{ github.sha }}
path: ${{ env.PIXI_WORKSPACE }}/${{ env.TARGET_RELEASE }}

- name: Verify pixi installation
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi info

- name: Run long running integration tests
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi run --locked test-integration-extra-slow-ci

- name: Test examples
shell: bash
working-directory: ${{ env.PIXI_WORKSPACE }}
run: bash tests/scripts/test-examples.sh

- name: "Checkout Deltares/Ribasim"

test-integration-macos-aarch64:
timeout-minutes: 30
name: Integration tests | macos aarch64
runs-on: macos-14
needs: build-binary-macos-aarch64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-macos-aarch64-${{ github.sha }}
path: ${{ env.TARGET_RELEASE }}
- name: Setup unix binary, add to github path
run: |
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi info

- name: Run long running integration tests
run: pixi run --locked test-integration-extra-slow-ci

- name: Run integration tests
run: pixi run --locked test-integration-ci

- name: Test examples
run: bash tests/scripts/test-examples.sh

- name: Test export
run: pixi run --locked test-export

test-integration-linux-x86_64:
timeout-minutes: 30
name: Integration tests | linux x86_64
runs-on: 8core_ubuntu_latest_runner
needs: build-binary-linux-x86_64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-linux-x86_64-${{ github.sha }}
path: ${{ env.TARGET_RELEASE }}
- name: Setup unix binary, add to github path
run: |
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi info

- name: Run long running integration tests
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
run: pixi run --locked test-integration-ci -m "extra_slow"

- name: Run integration tests
run: pixi run --locked test-integration-ci

- name: "Test examples"
run: bash tests/scripts/test-examples.sh

- name: "Test export"
run: pixi run --locked test-export

test-downstream-windows-x86_64:
timeout-minutes: 30
name: Downstream tests | windows x86_64
runs-on: windows-latest
needs: build-binary-windows-x86_64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "target/pixi/release"
steps:
- uses: actions/checkout@v4
- name: Create Dev Drive using ReFS
run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1
- name: Copy Git Repo to Dev Drive
run: |
Copy-Item -Path "${{ github.workspace }}" -Destination "${{ env.PIXI_WORKSPACE }}" -Recurse
echo "${{ env.PIXI_WORKSPACE }}/${{ env.TARGET_RELEASE }}" | Out-File -Append -Encoding utf8 -FilePath $env:GITHUB_PATH
- name: Download binary from build
uses: actions/download-artifact@v4
with:
name: pixi-windows-x86_64-${{ github.sha }}
path: ${{ env.PIXI_WORKSPACE }}/${{ env.TARGET_RELEASE }}
- name: Verify pixi installation
run: pixi info

- name: Install pixi's pixi project
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi install -vv

- name: Checkout Deltares/Ribasim
uses: actions/checkout@v4
with:
repository: Deltares/Ribasim
path: ribasim
- name: "Copy Deltares/Ribasim to Dev Drive"
- name: Copy Deltares/Ribasim to Dev Drive
run: Copy-Item -Path "${{ github.workspace }}/ribasim" -Destination "${{ env.PIXI_WORKSPACE }}/ribasim" -Recurse
- name: "Install Deltares/Ribasim"
- name: Install Deltares/Ribasim
run: pixi install
working-directory: ${{ env.PIXI_WORKSPACE }}/ribasim

- name: "Checkout quantco/polarify"
- name: Checkout quantco/polarify
uses: actions/checkout@v4
with:
repository: quantco/polarify
path: polarify
- name: "Copy quantco/polarify to Dev Drive"
- name: Copy quantco/polarify to Dev Drive
run: Copy-Item -Path "${{ github.workspace }}/polarify" -Destination "${{ env.PIXI_WORKSPACE }}/polarify" -Recurse
- name: "Install quantco/polarify"
- name: Install quantco/polarify
run: pixi install
working-directory: ${{ env.PIXI_WORKSPACE }}/polarify

test-integration-macos-aarch64:
timeout-minutes: 30
name: pytest integration | macos aarch64

test-downstream-macos-aarch64:
timeout-minutes: 15
name: Downstream tests | macos aarch64
runs-on: macos-14
needs: build-binary-macos-aarch64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
Expand All @@ -428,20 +603,11 @@ jobs:
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi --version
run: pixi info

- name: Install pixi
run: pixi install -v

- name: Run integration tests
run: pixi run --locked test-integration-ci

- name: Test examples
run: bash tests/scripts/test-examples.sh

- name: Test export
run: pixi run --locked test-export

- name: "Checkout Deltares/Ribasim"
uses: actions/checkout@v4
with:
Expand All @@ -460,11 +626,12 @@ jobs:
run: pixi install
working-directory: polarify

test-integration-linux-x86_64:
timeout-minutes: 30
name: pytest integration | linux x86_64
test-downstream-linux-x86_64:
timeout-minutes: 15
name: Downstream tests | linux x86_64
runs-on: 8core_ubuntu_latest_runner
needs: build-binary-linux-x86_64
if: ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'test:extra_slow') }}
env:
TARGET_RELEASE: "${{ github.workspace }}/target/pixi/release"
steps:
Expand All @@ -479,20 +646,12 @@ jobs:
chmod a+x ${{ env.TARGET_RELEASE }}/pixi
echo "${{ env.TARGET_RELEASE }}" >> $GITHUB_PATH
- name: Verify pixi installation
run: pixi --version
run: pixi info

- name: Install pixi
working-directory: ${{ env.PIXI_WORKSPACE }}
run: pixi install -v

- name: Run integration tests
run: pixi run --locked test-integration-ci

- name: "Test examples"
run: bash tests/scripts/test-examples.sh

- name: "Test export"
run: pixi run --locked test-export

- name: "Checkout nerfstudio-project/nerfstudio"
uses: actions/checkout@v4
with:
Expand All @@ -519,13 +678,12 @@ jobs:
- name: "Install quantco/polarify"
run: pixi install
working-directory: polarify

#
# Install a number of common wheels on some platforms
#

test-common-wheels-linux-x86_64:
name: "test wheel installation | linux x86_64"
name: Wheel Tests | linux x86_64
needs:
- build-binary-linux-x86_64
uses: ./.github/workflows/test_common_wheels.yml
Expand All @@ -535,7 +693,7 @@ jobs:
runs-on: 8core_ubuntu_latest_runner

test-common-wheels-windows-x86_64:
name: "test wheel installation | windows x86_64"
name: Wheel Tests | windows x86_64
needs:
- build-binary-windows-x86_64
uses: ./.github/workflows/test_common_wheels.yml
Expand All @@ -545,7 +703,7 @@ jobs:
runs-on: windows-latest

test-common-wheels-macos-aarch64:
name: "test wheel installation | macos aarch64"
name: Wheel Tests | macos aarch64
needs:
- build-binary-macos-aarch64
uses: ./.github/workflows/test_common_wheels.yml
Expand Down
Loading
Loading