Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.github/workflows: Refactor CI jobs #3090

Merged
merged 31 commits into from
Nov 18, 2022
Merged

.github/workflows: Refactor CI jobs #3090

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
fe5ee71
Set MSRV correctly for all crates
thomaseizinger Nov 7, 2022
a696fb5
Don't include TLS on wasm platforms
thomaseizinger Nov 8, 2022
e3c62ab
Fix clippy warnings
thomaseizinger Nov 8, 2022
071e7db
Activate necessary features for test
thomaseizinger Nov 7, 2022
3a2b0ed
Run yaml formatter
thomaseizinger Nov 7, 2022
8c36289
Run clippy on stable and beta
thomaseizinger Nov 7, 2022
46a1bf5
Don't install things that are already present
thomaseizinger Nov 7, 2022
342b175
Rename CI job
thomaseizinger Nov 7, 2022
569cbf6
Refactor test and build jobs
thomaseizinger Nov 7, 2022
5ef7549
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 8, 2022
636c873
Pin protoc action
thomaseizinger Nov 9, 2022
24a42df
Actually adjust name
thomaseizinger Nov 9, 2022
79b3740
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 10, 2022
9c83d21
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 13, 2022
e51e883
Ensure we can compile without any features
thomaseizinger Nov 15, 2022
be659b1
Cache build artefacts for MSRV check
thomaseizinger Nov 15, 2022
3060f0b
Add comment explaining why this caching works
thomaseizinger Nov 15, 2022
4314979
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 15, 2022
f753b16
Remove unnecessary cast (clippy beta lint)
thomaseizinger Nov 15, 2022
05d6690
Activate default features to make it compile
thomaseizinger Nov 15, 2022
e0ab8cd
Add crate name to cache key
thomaseizinger Nov 15, 2022
0f3a561
Set missing MSRV for libp2p-quic
thomaseizinger Nov 15, 2022
990c321
Add changelog entries
thomaseizinger Nov 15, 2022
2a1b2ca
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 15, 2022
a7a0dae
Merge branch 'master' into 2951-ci-matrix
mxinden Nov 16, 2022
6d61142
Make `libp2p-swarm` wasm compatible again
thomaseizinger Nov 17, 2022
c85cae9
Merge branch 'master' into 2951-ci-matrix
thomaseizinger Nov 18, 2022
5e8164d
Set MSRV for webrtc
thomaseizinger Nov 18, 2022
ca8dbee
Fix clippy beta lints
thomaseizinger Nov 18, 2022
489e16f
Don't compile libp2p-webrtc for wasm
thomaseizinger Nov 18, 2022
5ee2e24
Merge branch 'master' into 2951-ci-matrix
mxinden Nov 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
321 changes: 172 additions & 149 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,175 +11,229 @@ concurrency:
cancel-in-progress: true

jobs:
test-desktop:
name: Build and test
test:
name: Test ${{ matrix.crate }}
runs-on: ubuntu-latest
needs: gather_published_crates
strategy:
fail-fast: false
matrix:
args: [
"--no-default-features",
"--all-features",
"--benches --all-features",
]
crate: ${{ fromJSON(needs.gather_published_crates.outputs.members) }}
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1
- name: Install Protoc
uses: arduino/setup-protoc@v1
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

- uses: actions/checkout@v3
- uses: actions/checkout@v3

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
with:
key: ${{ matrix.args }}
- name: Get MSRV for ${{ matrix.crate }}
id: parse-msrv
run: |
RUST_VERSION=$(cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.name == "${{ matrix.crate }}") | .rust_version')
echo "version=${RUST_VERSION}" >> $GITHUB_OUTPUT

- name: Switch to Rust ${{ steps.parse-msrv.outputs.version }} for MSRV check
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: ${{ steps.parse-msrv.outputs.version }}
override: true

- name: Check if ${{ matrix.crate }} compiles on MSRV (Rust ${{ steps.parse-msrv.outputs.version }})
run: cargo check --package ${{ matrix.crate }} --all-features

- name: Switch to latest Rust stable for tests
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
with:
key: ${{ matrix.crate }}

- run: cargo test --package ${{ matrix.crate }} --all-features

- name: Check if crate has been released
id: check-released
run: |
RESPONSE_CODE=$(curl https://crates.io/api/v1/crates/${{ matrix.crate }} --silent --write-out "%{http_code}" --output /dev/null)
echo "code=${RESPONSE_CODE}"
echo "code=${RESPONSE_CODE}" >> $GITHUB_OUTPUT

- run: cargo test --workspace ${{ matrix.args }}
- name: Check public API for semver violations
if: steps.check-released.outputs.code == 200 # Workaround until https://github.com/obi1kenobi/cargo-semver-check/issues/146 is shipped.
run: |
cargo install cargo-semver-checks
cargo semver-checks check-release -p ${{ matrix.crate }}

test-wasm:
name: Build on WASM
runs-on: ubuntu-latest
cross:
name: Compile on ${{ matrix.target }}
strategy:
matrix:
toolchain: [
wasm32-unknown-emscripten,
wasm32-wasi
]
include:
- toolchain: wasm32-unknown-unknown
args: "--features wasm-bindgen"
env:
CC: clang-11
defaults:
run:
shell: bash
- target: "wasm32-unknown-unknown"
os: ubuntu-latest
- target: "wasm32-unknown-emscripten"
os: ubuntu-latest
- target: "wasm32-wasi"
os: ubuntu-latest
- target: "x86_64-apple-darwin"
os: macos-latest
- target: "x86_64-pc-windows-msvc"
os: windows-latest
runs-on: ${{ matrix.os }}
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1
- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: actions/checkout@v3
- uses: actions/checkout@v3

- name: Install Rust ${{ matrix.toolchain }}
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
toolchain: stable
target: ${{ matrix.toolchain }}
override: true
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
target: ${{ matrix.target }}

- name: Install a recent version of clang
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0

- name: Install CMake
run: sudo apt-get install -y cmake
- run: cargo check --package libp2p --all-features --target=${{ matrix.target }}

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
with:
key: ${{ matrix.toolchain }}
feature_matrix: # Test various feature combinations work correctly
name: Compile with select features (${{ matrix.features }})
runs-on: ubuntu-latest
strategy:
matrix:
include:
- features: "mdns tcp dns tokio"
- features: "mdns tcp dns async-std"
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: actions/checkout@v3

- name: Build on ${{ matrix.toolchain }}
# TODO: also run `cargo test`
# TODO: ideally we would build `--workspace`, but not all crates compile for WASM
run: cargo build --target=${{ matrix.toolchain }} ${{ matrix.args }}
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
with:
key: ${{ matrix.runtime }}

- run: cargo check --package libp2p --features="${{ matrix.features }}"

check-rustdoc-links:
name: Check rustdoc intra-doc links
runs-on: ubuntu-latest
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1
- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: actions/checkout@v3
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0

- name: Check rustdoc links
run: RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links --deny warnings" cargo doc --verbose --workspace --no-deps --all-features --document-private-items
- name: Check rustdoc links
run: RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links --deny warnings" cargo doc --verbose --workspace --no-deps --all-features --document-private-items

check-clippy:
clippy:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
rust-version: [
stable,
beta
jxs marked this conversation as resolved.
Show resolved Hide resolved
]
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1
- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: actions/checkout@v3
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
components: clippy
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: ${{ matrix.rust-version }}
override: true
components: clippy

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0

- name: Run cargo clippy
uses: actions-rs/cargo@844f36862e911db73fe0815f00a4a2602c279505 # v1.0.3
with:
command: custom-clippy # cargo alias to allow reuse of config locally
- name: Run cargo clippy
uses: actions-rs/cargo@844f36862e911db73fe0815f00a4a2602c279505 # v1.0.3
with:
command: custom-clippy # cargo alias to allow reuse of config locally

integration-test:
ipfs-integration-test:
name: Integration tests
runs-on: ubuntu-latest
steps:
- name: Install Protoc
uses: arduino/setup-protoc@v1
- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: actions/checkout@v3
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0

- name: Run ipfs-kad example
run: RUST_LOG=libp2p_swarm=debug,libp2p_kad=trace,libp2p_tcp=debug cargo run --example ipfs-kad --features full
- name: Run ipfs-kad example
run: RUST_LOG=libp2p_swarm=debug,libp2p_kad=trace,libp2p_tcp=debug cargo run --example ipfs-kad --features full

rustfmt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
components: rustfmt
- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true
components: rustfmt

- name: Check formatting
run: cargo fmt -- --check
- name: Check formatting
run: cargo fmt -- --check

manifest_lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- name: Ensure `full` feature contains all features
run: |
ALL_FEATURES=$(cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.name == "libp2p") | .features | keys | map(select(. != "full")) | sort | join(" ")')
FULL_FEATURE=$(cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.name == "libp2p") | .features["full"] | sort | join(" ")')

test "$ALL_FEATURES = $FULL_FEATURE"

echo "$ALL_FEATURES";
echo "$FULL_FEATURE";

test "$ALL_FEATURES" = "$FULL_FEATURE"

gather_crates_for_semver_checks:
- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- name: Ensure `full` feature contains all features
run: |
ALL_FEATURES=$(cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.name == "libp2p") | .features | keys | map(select(. != "full")) | sort | join(" ")')
FULL_FEATURE=$(cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.name == "libp2p") | .features["full"] | sort | join(" ")')
test "$ALL_FEATURES = $FULL_FEATURE"
echo "$ALL_FEATURES";
echo "$FULL_FEATURE";
test "$ALL_FEATURES" = "$FULL_FEATURE"

gather_published_crates:
Copy link
Contributor

@galargh galargh Nov 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since test depends on it, my suggestion would be to place this job above it. Not a requirement but it'd make it easier to read.

Copy link
Contributor Author

@thomaseizinger thomaseizinger Nov 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually follow a different philosophy:

The most important functions jobs should be at the top. Assuming sufficiently descriptive names, you shouldn't need the implementation of a function job to continue reading and understanding what is happening.

If you do the reverse - putting all "leaf"1 jobs at the top - you end up reading a lot of unrelated snippets before getting to the parts that actually matter.

Footnotes

  1. As in, the leafs in a dependency-tree. ↩

runs-on: ubuntu-latest
outputs:
members: ${{ steps.cargo-metadata.outputs.members }}
Expand All @@ -189,35 +243,4 @@ jobs:
- id: cargo-metadata
run: |
WORKSPACE_MEMBERS=$(cargo metadata --format-version=1 --no-deps | jq -c '.packages | .[] | select(.publish == null) | .name' | jq -s '.' | jq -c '.')
echo "::set-output name=members::${WORKSPACE_MEMBERS}"

semver-check:
runs-on: ubuntu-latest
needs: gather_crates_for_semver_checks
strategy:
fail-fast: false
matrix:
crate: ${{ fromJSON(needs.gather_crates_for_semver_checks.outputs.members) }}
steps:
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@bb6001c4ea612bf59c3abfc4756fbceee4f870c7 # 0.10.0
with:
access_token: ${{ github.token }}

- uses: actions/checkout@v3

- uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af # v1.0.7
with:
profile: minimal
toolchain: stable
override: true

- name: Install Protoc
uses: arduino/setup-protoc@v1

- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd # v2.1.0

- run: cargo install cargo-semver-checks

- name: Semver Check
run: cargo semver-checks check-release -p ${{ matrix.crate }}
echo "members=${WORKSPACE_MEMBERS}" >> $GITHUB_OUTPUT
2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "libp2p"
edition = "2021"
rust-version = "1.60.0"
rust-version = "1.62.0"
description = "Peer-to-peer networking library"
version = "0.50.0"
authors = ["Parity Technologies <[email protected]>"]
Expand Down
Loading