Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Move tests from CircleCI to GitHub #558

Merged
merged 40 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
07e6a1d
github tests trigger
l0uden Jul 1, 2024
4f53259
new tests trigger
l0uden Jul 2, 2024
e4572c7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 2, 2024
6931cfe
fix new tests trigger
l0uden Jul 2, 2024
bd7e755
Merge remote-tracking branch 'origin/circleci_to_github' into circlec…
l0uden Jul 2, 2024
b0c3ff2
another fix new tests trigger
l0uden Jul 2, 2024
956d179
another fix new tests trigger
l0uden Jul 2, 2024
06d52ff
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 2, 2024
394af24
added checkout
l0uden Jul 2, 2024
d97b04a
Merge remote-tracking branch 'origin/circleci_to_github' into circlec…
l0uden Jul 2, 2024
cfbee47
gdate to date
l0uden Jul 2, 2024
0c1a2ea
branch from variable
l0uden Jul 2, 2024
4c95993
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 9, 2024
36d22cf
integration tests trigger
l0uden Jul 9, 2024
70aa321
delete circleci tests trigger
l0uden Jul 15, 2024
0ff68f0
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 15, 2024
80ecf9e
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 16, 2024
e4fbc0c
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 17, 2024
932c8ec
vizro-ai slack notifications
l0uden Jul 17, 2024
d581859
workflows check
l0uden Jul 19, 2024
9a30b8a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
1369e0f
deleted default directory
l0uden Jul 19, 2024
9da4611
install yaml
l0uden Jul 19, 2024
b85d1e6
install pyyaml
l0uden Jul 19, 2024
1541a2e
delete test content
l0uden Jul 19, 2024
a90758e
branch notifications
l0uden Jul 19, 2024
cc5a92d
move secrets from env to general storage
l0uden Jul 19, 2024
8dedc45
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 19, 2024
d4b5305
review fixes
l0uden Jul 22, 2024
94f78e4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2024
3699b74
fix for matrix
l0uden Jul 22, 2024
6ca1035
fix for matrix file
l0uden Jul 22, 2024
028e151
fix for matrix file with if
l0uden Jul 22, 2024
a43b0b9
linting fix
l0uden Jul 22, 2024
11c81bc
linting fix
l0uden Jul 22, 2024
ba1b1aa
small review fix
l0uden Jul 25, 2024
fbb6066
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 26, 2024
a624d80
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Jul 30, 2024
011ade9
delete circleci-trigger.yml
l0uden Jul 30, 2024
e49856f
Merge branch 'main' of https://github.com/mckinsey/vizro into circlec…
l0uden Aug 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/checks-workflows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Checks for GitHub workflows

on:
push:
branches: [main]
pull_request:
branches:
- main

env:
PYTHONUNBUFFERED: 1
FORCE_COLOR: 1
PYTHON_VERSION: "3.11"

jobs:
checks-workflows:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Checks for GitHub workflows
l0uden marked this conversation as resolved.
Show resolved Hide resolved
run: |
python tools/scan_yaml_for_risky_text.py.py .github/workflows
l0uden marked this conversation as resolved.
Show resolved Hide resolved
75 changes: 0 additions & 75 deletions .github/workflows/circleci-trigger.yml

This file was deleted.

13 changes: 13 additions & 0 deletions .github/workflows/test-integration-vizro-ai.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,16 @@ jobs:
cd ../vizro-ai
hatch run ${{ matrix.hatch-env }}:pip install ../vizro-core/dist/vizro*.tar.gz
hatch run ${{ matrix.hatch-env }}:test-integration

- name: Send custom JSON data to Slack
id: slack
uses: slackapi/[email protected]
if: failure()
with:
payload: |
{
"text": "Vizro-ai ${{ matrix.hatch-env }} integration tests build result: ${{ job.status }}\nBranch: ${{ github.head_ref }}\n${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
52 changes: 52 additions & 0 deletions .github/workflows/vizro-qa-tests-trigger.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: Vizro QA tests trigger

on:
push:
branches: [main]
pull_request:
branches:
- main

env:
PYTHONUNBUFFERED: 1
FORCE_COLOR: 1

jobs:
vizro-qa-test-trigger-fork:
if: ${{ github.event.pull_request.head.repo.fork }}
name: Vizro QA ${{ matrix.label }} trigger
runs-on: ubuntu-latest
strategy:
matrix:
include:
- label: integration tests
- label: notebooks tests
steps:
- name: Passed fork step
run: echo "Success!"

vizro-qa-tests-trigger:
if: ${{ ! github.event.pull_request.head.repo.fork }}
name: Vizro QA ${{ matrix.label }} trigger
runs-on: ubuntu-latest
strategy:
matrix:
include:
- label: integration tests
file: "VIZRO_QA_INTEGRATION_TESTS_WORKFLOW"
- label: notebooks test
file: "VIZRO_QA_NOTEBOOKS_TESTS_WORKFLOW"
steps:
- uses: actions/checkout@v4
- name: Tests trigger
run: |
export INPUT_OWNER=${{ secrets.VIZRO_QA_ORG }}
export INPUT_REPO=${{ secrets.VIZRO_QA_REPO }}
if [ "${{ matrix.file }}" == "VIZRO_QA_INTEGRATION_TESTS_WORKFLOW" ]; then
export INPUT_WORKFLOW_FILE_NAME=${{ secrets.VIZRO_QA_INTEGRATION_TESTS_WORKFLOW }}
elif [ "${{ matrix.file }}" == "VIZRO_QA_NOTEBOOKS_TESTS_WORKFLOW" ]; then
export INPUT_WORKFLOW_FILE_NAME=${{ secrets.VIZRO_QA_NOTEBOOKS_TESTS_WORKFLOW }}
fi
l0uden marked this conversation as resolved.
Show resolved Hide resolved
export INPUT_GITHUB_TOKEN=${{ secrets.VIZRO_SVC_PAT }}
export INPUT_REF=${{ github.head_ref }}
tools/trigger-workflow-and-wait.sh
19 changes: 19 additions & 0 deletions tools/scan_yaml_for_risky_text.py.py
l0uden marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
"""Check for security issues in workflows files."""

import sys
from pathlib import Path

# according to this article: https://nathandavison.com/blog/github-actions-and-the-threat-of-malicious-pull-requests
# we should avoid using `pull_request_target` for security reasons
risky_text = "pull_request_target"


def find_risky_files(path: str):
"""Searching for risky text in yml files for given path."""
return {str(file) for file in Path(path).rglob("*.yml") if f"{risky_text}" in file.read_text()}
l0uden marked this conversation as resolved.
Show resolved Hide resolved


if __name__ == "__main__":
risky_files = find_risky_files(sys.argv[1])
if risky_files:
sys.exit(f"{risky_text} found in files {risky_files}.")
Loading
Loading