Skip to content

Commit

Permalink
ci: speed up Python test (#3776)
Browse files Browse the repository at this point in the history
Reduce installation time from 2.5 min to 1.5 min.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
- Enhanced Python environment setup and package installation processes
to improve performance and compatibility.
- **Chores**
- Updated package management commands to ensure the latest versions of
necessary libraries are used.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
  • Loading branch information
njzjz authored May 13, 2024
1 parent 4ab6436 commit dac3a3c
Showing 1 changed file with 3 additions and 10 deletions.
13 changes: 3 additions & 10 deletions .github/workflows/test_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,23 +23,16 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- uses: mpi4py/setup-mpi@v1
with:
mpi: openmpi
- run: python -m pip install -U uv
- run: uv pip install --system -e .[cpu,test,torch]
- run: uv pip install --system --only-binary=horovod -e .[cpu,test,torch] horovod[tensorflow-cpu] mpi4py mpich
env:
# Please note that uv has some issues with finding
# existing TensorFlow package. Currently, it uses
# TensorFlow in the build dependency, but if it
# changes, setting `TENSORFLOW_ROOT`.
TENSORFLOW_VERSION: ${{ matrix.tf }}
TENSORFLOW_VERSION: ${{ matrix.python == '3.8' && '2.13.1' || '2.16.1' }}
DP_BUILD_TESTING: 1
- run: uv pip install --system --no-build-isolation horovod mpi4py
env:
HOROVOD_WITH_TENSORFLOW: 1
HOROVOD_WITHOUT_PYTORCH: 1
HOROVOD_WITHOUT_GLOO: 1
UV_EXTRA_INDEX_URL: "https://pypi.anaconda.org/njzjz/simple https://pypi.anaconda.org/mpi4py/simple"
- run: dp --version
- name: Get durations from cache
uses: actions/cache@v4
Expand Down

0 comments on commit dac3a3c

Please sign in to comment.