Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce Cypress Execution Time with Github Actions Parallel Jobs #3579

Draft
wants to merge 94 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
cacbfe5
Initial WIP version of resource creation test
antowaddle Nov 20, 2024
0fa2201
Experimental changes to poll the UI for updates
antowaddle Nov 21, 2024
af87ad1
Working version if resource is present
antowaddle Nov 21, 2024
0e9868a
increase card timeout and delete active wait
FedeAlonso Nov 21, 2024
429f086
Merge pull request #1 from FedeAlonso/fix/active_wait
antowaddle Nov 21, 2024
443e530
Added changes to find namespace from variables
antowaddle Nov 22, 2024
966917f
Final changes to read variables, cleaned up utils
antowaddle Nov 22, 2024
69db944
Merge branch 'main' into cypress-RHOAIENG-14368
antowaddle Nov 22, 2024
a9ecfa3
Small change to a comment
antowaddle Nov 22, 2024
6cd6a3e
Dummy change to trigger mocks
antowaddle Nov 22, 2024
fcc6d18
Merge branch 'opendatahub-io:main' into main
antowaddle Nov 22, 2024
c7d471a
Merge branch 'main' into cypress-RHOAIENG-14368
antowaddle Nov 22, 2024
7628316
Save changes on cypress-RHOAIENG-12649
antowaddle Nov 25, 2024
847ad11
Changed file directories and names as requested on a PR comment
antowaddle Nov 25, 2024
552a516
Merge branch 'main' into cypress-RHOAIENG-14368
antowaddle Nov 25, 2024
80cc894
Merge branch 'cypress-RHOAIENG-14368' of https://github.com/antowaddl…
antowaddle Nov 25, 2024
163ed94
Merge remote-tracking branch 'origin/main' into cypress-RHOAIENG-12649
antowaddle Nov 25, 2024
0f7eead
Saving changes to current branch
antowaddle Nov 25, 2024
48c7435
Additional directory/file name changes
antowaddle Nov 25, 2024
14a39de
Additional changes to save
antowaddle Nov 25, 2024
6a93ec2
Resolving timeout issue breaking mock tests, also resolved latest PR …
antowaddle Nov 25, 2024
31b9723
Further changes for this test
antowaddle Nov 25, 2024
f2b636d
Changes to revert the exist method appended to getCardView.
antowaddle Nov 25, 2024
a9feb1b
Fixed linting
antowaddle Nov 25, 2024
aa9430e
Linting fixes
antowaddle Nov 25, 2024
eddd7d8
Merge branch 'main' into cypress-RHOAIENG-12649Merge branch 'main' of…
antowaddle Nov 25, 2024
680a0ee
Final comments added
antowaddle Nov 25, 2024
b6bc5b3
Merge branch 'main' into cypress-RHOAIENG-12649
antowaddle Nov 26, 2024
781e952
Fixed merge conflict
antowaddle Nov 26, 2024
b05b676
Small change to page object name
antowaddle Nov 26, 2024
ae51479
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 9, 2024
dac0574
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 10, 2024
2cd1f7e
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 11, 2024
645dca1
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 13, 2024
f044a29
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 16, 2024
7aa9059
Merge branch 'main' of github.com:opendatahub-io/odh-dashboard
antowaddle Dec 16, 2024
1836395
Testing changes to parallelism in Mock tests
antowaddle Dec 16, 2024
ce3216a
Fixed syntax
antowaddle Dec 16, 2024
4fa98dd
Removed parallel matrix from entire file, made changes to Cypress exe…
antowaddle Dec 16, 2024
0369cbe
Created a new job for Mocks, which allows to control threads and para…
antowaddle Dec 16, 2024
c69f2b4
Naming change
antowaddle Dec 16, 2024
54fbb94
Syntax fix
antowaddle Dec 16, 2024
c127896
Removed reference to cypress cloud and added headless execution
antowaddle Dec 16, 2024
15e6da6
Merge branch 'main' into RHOAIENG-15752
antowaddle Dec 16, 2024
a2263b1
Indentation fix
antowaddle Dec 16, 2024
797c7a3
Merge branch 'RHOAIENG-15752' of https://github.com/antowaddle/odh-da…
antowaddle Dec 16, 2024
dafd8de
changes to split files
antowaddle Dec 16, 2024
aac612d
Reverting to previous version which executed tests in parallel. Clean…
antowaddle Dec 16, 2024
80ca9ac
Changes to artifacts
antowaddle Dec 17, 2024
97fca1d
1 Container to test single execution speed with a split job
antowaddle Dec 17, 2024
0d14270
Removed redundent test, added logic to split mocks up and run them in…
antowaddle Dec 17, 2024
3f8d63c
Fixed syntax issue
antowaddle Dec 17, 2024
7268075
Moved Code Coverage to correct position
antowaddle Dec 17, 2024
53d9bac
Makes fixes to kick tests off
antowaddle Dec 17, 2024
516a781
More tweaks to make tests execute
antowaddle Dec 17, 2024
0e36378
Tweaked command to launch server which I omitted last time - d'oh
antowaddle Dec 17, 2024
7f95c0e
Tweak to the execution comman
antowaddle Dec 17, 2024
15de92f
Further tweaking the execution command
antowaddle Dec 17, 2024
0910c0a
More tweaks
antowaddle Dec 17, 2024
6d1598b
Changes to Get test files to work out pathing issue
antowaddle Dec 17, 2024
d5f3254
Pathing debugging and removed Unit tests for debugging
antowaddle Dec 17, 2024
50577cc
Removed unit test req
antowaddle Dec 17, 2024
2b24021
Further pathing debugging
antowaddle Dec 17, 2024
4201f2f
Further pathing debugging part 2
antowaddle Dec 17, 2024
f0bbe90
Further pathing debugging part 3
antowaddle Dec 17, 2024
3d08973
List contents of mocked dir
antowaddle Dec 17, 2024
530a722
Further pathing changes
antowaddle Dec 17, 2024
bfee475
Added recursive check
antowaddle Dec 17, 2024
193ac58
Syntax fix
antowaddle Dec 17, 2024
1e2b81b
Change to Cypress execution command
antowaddle Dec 17, 2024
e89ee81
Container debug 1
antowaddle Dec 17, 2024
796dacf
Container debug 2
antowaddle Dec 17, 2024
62583fd
Container debug 3
antowaddle Dec 17, 2024
cb1d40a
Container debug 3
antowaddle Dec 17, 2024
9467c1f
Container debug 5
antowaddle Dec 17, 2024
144c3bb
Spec file debugger 1
antowaddle Dec 17, 2024
14accb3
Spec file debugger 2
antowaddle Dec 17, 2024
1b06a58
Spec file debugger 3
antowaddle Dec 18, 2024
434750c
Spec file debugger 4
antowaddle Dec 18, 2024
48152f5
Spec file debugger 5
antowaddle Dec 18, 2024
5d5fc56
Spec file debugger 6
antowaddle Dec 18, 2024
52bf0f4
Spec file debugger 7
antowaddle Dec 18, 2024
fa3a968
Spec file debugger 8
antowaddle Dec 18, 2024
9d2490a
Spec file debugger 9
antowaddle Dec 18, 2024
b78c024
Spec file debugger 10
antowaddle Dec 18, 2024
64b47a7
Spec file debugger 11
antowaddle Dec 18, 2024
0043c92
Spec file debugger 12
antowaddle Dec 18, 2024
2a6ad14
Spec file debugger 13
antowaddle Dec 18, 2024
074f3d9
Spec file debugger 14
antowaddle Dec 18, 2024
6a3b1bc
Spec file debugger 15
antowaddle Dec 18, 2024
d565378
Spec file debugger 16
antowaddle Dec 18, 2024
4846ba0
Spec file debugger 17
antowaddle Dec 18, 2024
17e612f
Spec file debugger 18
antowaddle Dec 18, 2024
39ea4dd
Removed references to CYPRESS_
antowaddle Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 56 additions & 50 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,62 +1,68 @@
name: Test
on: [push, pull_request]
jobs:
Tests:
CypressMockTests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version: [18.x]
container: [1, 2, 3, 4, 5] # Creates 5 parallel executions for Cypress tests.
steps:
- uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}

- name: Setup Node.js
uses: actions/[email protected]
with:
node-version: ${{ matrix.node-version }}
- name: Node.js modules cache, repository
uses: actions/cache@v4
id: repo-cache
with:
path: ${{ github.workspace }}/node_modules
key: ${{ runner.os }}-${{ matrix.node-version }}-modules-repo-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-${{ matrix.node-version }}-modules-repo
- name: Node.js modules cache, backend
uses: actions/cache@v4
id: backend-cache
with:
path: ${{ github.workspace }}/backend/node_modules
key: ${{ runner.os }}-${{ matrix.node-version }}-modules-backend-${{ hashFiles('**/backend/package-lock.json') }}
restore-keys: |
${{ runner.os }}-${{ matrix.node-version }}-modules-backend
- name: Node.js modules cache, frontend
uses: actions/cache@v4
id: frontend-cache
with:
path: |
~/.cache/Cypress
${{ github.workspace }}/frontend/node_modules
key: ${{ runner.os }}-${{ matrix.node-version }}-modules-frontend-${{ hashFiles('**/frontend/package-lock.json') }}
restore-keys: |
${{ runner.os }}-${{ matrix.node-version }}-modules-frontend
- name: Install Node.js packages
if: ${{ steps.repo-cache.outputs.cache-hit != 'true' || steps.backend-cache.outputs.cache-hit != 'true' || steps.frontend-cache.outputs.cache-hit != 'true' }}
node-version: '18.x'

- name: Install Node.js packages for Cypress
run: npm install
- name: Check for uncomitted changes

- name: Get test files
id: test-files
run: |
git diff --exit-code
- name: Test backend & test frontend with code coverage
if: ${{ success() }}
run: npm run test:backend && npm run test:frontend:coverage
- name: Upload coverage reports to Codecov
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
name: frontend
directory: ./frontend/coverage
verbose: true
- uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
name: cypress-results
path: ./frontend/src/__tests__/cypress/results/
echo "Current working directory:"
pwd

TEST_DIR="frontend/src/__tests__/cypress/cypress/tests/mocked"
echo "Listing contents of the current directory:"
ls -la
echo "Checking for test files in $TEST_DIR"

echo "Listing contents of frontend/src/__tests__/cypress/cypress/tests:"
ls -la frontend/src/__tests__/cypress/cypress/tests/

if [ -d "$TEST_DIR" ]; then
echo "Listing contents of mocked directory:"
ls -la "$TEST_DIR"

TEST_FILES=$(find "$TEST_DIR" -name "*.cy.ts" | tr '\n' ',')
echo "Found test files: $TEST_FILES"

echo "files=$TEST_FILES" >> $GITHUB_OUTPUT
else
echo "Directory does not exist. No test files found." >> $GITHUB_OUTPUT
echo "files=" >> $GITHUB_OUTPUT # Set empty output if no files found
fi

- name: Run Cypress Mock Tests in Parallel
env:
CONTAINER_INDEX: ${{ matrix.container }}
TEST_FILES: ${{ steps.test-files.outputs.files }}
run: |
echo "Running Cypress tests for container $CONTAINER_INDEX..."
echo "Test Files: $TEST_FILES"

# Generate the list of spec files for this container and trim trailing comma
SPEC_FILES=$(echo "$TEST_FILES" | tr ',' '\n' | awk -v container="$CONTAINER_INDEX" 'NR % 5 == container - 1' | sed 's|^frontend/||' | tr '\n' ',' | sed 's/,$//')

# Check if SPEC_FILES is empty
if [ -z "$SPEC_FILES" ]; then
echo "No spec files found for container $CONTAINER_INDEX. Skipping Cypress tests."
exit 0
else
echo "Spec files for this container: $SPEC_FILES"
export SPEC_PATTERN="$SPEC_FILES"
echo "Running Cypress with spec pattern: ${SPEC_PATTERN}"
cd frontend && npm run test:cypress-ci
fi
4 changes: 3 additions & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,16 @@
"test:jest": "jest",
"test:jest:coverage": "rimraf jest-coverage && jest --silent --coverage --coverageReporters=json --coverageReporters=lcov",
"test:unit": "npm run test:jest -- --silent",
"test:cypress-ci": "npx concurrently -P -k -s first \"npm run cypress:server:build && npm run cypress:server\" \"npx wait-on tcp:127.0.0.1:9001 && npm run cypress:run:mock -- {@}\" -- ",
"test:unit-coverage": "run-s test:lint test:type-check test:jest:coverage",
"test:cypress-ci": "npx concurrently -P -k -s first \"npm run cypress:server:build && npm run cypress:server\" \"npx wait-on tcp:127.0.0.1:9001 && CY_MOCK=1 CY_WS_PORT=9002 npm run cypress:run -- --headless --project src/__tests__/cypress --spec '${SPEC_PATTERN}'\"",
"test:cypress-ci:coverage": "npx concurrently -P -k -s first \"npm run cypress:server:build:coverage && npm run cypress:server\" \"npx wait-on tcp:127.0.0.1:9001 && npm run cypress:run:mock:coverage\"",
"coverage:merge": "rimraf coverage && istanbul-merge --out coverage/coverage-final.json jest-coverage/coverage-final.json src/__tests__/cypress/coverage/coverage-final.json && nyc report --reporter html -t coverage --report-dir coverage/report && nyc report --reporter json-summary -t coverage --report-dir coverage",
"cypress:open": "cypress open --project src/__tests__/cypress",
"cypress:run": "cypress run -b chrome --project src/__tests__/cypress",
"cypress:open:mock": "CY_MOCK=1 CY_WS_PORT=9002 npm run cypress:open -- ",
"cypress:run:mock": "CY_MOCK=1 CY_WS_PORT=9002 npm run cypress:run -- ",
"cypress:run:mock:coverage": "rimraf src/__tests__/cypress/coverage src/__tests__/cypress/.nyc_output && CY_COVERAGE=true npm run cypress:run:mock",
"cypress:run:mock:coverage:parallel": "cypress run --coverage",
"cypress:open:record": "CY_RECORD=1 npm run cypress:open",
"cypress:run:record": "CY_RECORD=1 npm run cypress:run && npm run cypress:format",
"cypress:server:build": "ODH_DIST_DIR=./public-cypress POLL_INTERVAL=9999999 FAST_POLL_INTERVAL=9999999 WS_HOSTNAME=localhost:9002 npm run build",
Expand Down
8 changes: 6 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,12 @@
"test": "run-s test:backend test:frontend",
"test:backend": "cd ./backend && npm run test",
"test:frontend": "cd ./frontend && npm run test",
"test:frontend:coverage": "cd ./frontend && npm run test:coverage",
"test:cypress-ci": "cd ./frontend && npm run test:cypress-ci -- ",
"test:frontend:unit-coverage": "cd ./frontend && npm run test:unit-coverage",

"test:cypress-ci": "CY_MOCK=1 CY_WS_PORT=9002 npm run cypress:run",
"test:cypress-ci:parallel": "cd ./frontend && npm run cypress:server:build:coverage && npm run cypress:server & npx wait-on tcp:127.0.0.1:9001 && npm run cypress:run:mock:coverage",
"cypress:run:mock:coverage": "cypress run --coverage",
"cypress:run:mock:coverage:parallel": "cypress run --coverage",
"test:unit": "cd ./frontend && npm run test:unit",
"test:fix": "run-s test:fix-backend test:fix-frontend",
"test:fix-backend": "cd ./backend && npm run test:fix",
Expand Down
Loading