-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Download GIN Data for Pipeline Tests #795
Merged
Merged
Changes from 84 commits
Commits
Show all changes
101 commits
Select commit
Hold shift + click to select a range
1bbc780
Update testing-pipelines.yml
garrettmflynn cc563d7
Create example_data_cache.yml
CodyCBakerPhD 79aec20
Create testing_pipelines.yml
CodyCBakerPhD e70ba8e
Rename testing_pipelines.yml to deploy_testing_pipelines.yml
CodyCBakerPhD 5f6a4fc
Update and rename testing-pipelines.yml to testing_pipelines.yml
CodyCBakerPhD 5deda1e
Update deploy_testing_pipelines.yml
CodyCBakerPhD 130e546
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD c238e6f
snake case
CodyCBakerPhD 2e76258
Merge branch 'download-gin-data-for-tests' of https://github.com/neur…
CodyCBakerPhD 8fffa11
fix yaml
CodyCBakerPhD b9895a6
fix yaml
CodyCBakerPhD 4f4e32c
remove unused action
CodyCBakerPhD 34a48e7
Rename pyflask_build_and_dist_tests.yml to testing_pyflask_build_and_…
CodyCBakerPhD c9f147b
Rename project_action.yml to add_to_nwb_dashboard.yml
CodyCBakerPhD 5a7e7df
move codespell from action to pre-commit
CodyCBakerPhD 9bd42e6
Delete .github/workflows/codespell.yml
CodyCBakerPhD ca546be
Update testing_pipelines.yml
CodyCBakerPhD 633f66c
remove concurrency from triggered
CodyCBakerPhD 886fad2
Update testing_pipelines.yml
CodyCBakerPhD 1c79fad
streamline CI deployment; adjust all names
CodyCBakerPhD 4ac8d0b
try again
CodyCBakerPhD 4197ab9
try again
CodyCBakerPhD 2cf529c
try again
CodyCBakerPhD 68e971e
try again
CodyCBakerPhD 7f5e19d
Fail if no path
garrettmflynn e21df99
Update testing_pipelines.yml
garrettmflynn 0d23d60
Archive screenshots from the test pipelines
garrettmflynn 6b67a34
Remove pipeline screenshots
garrettmflynn 4c99f5f
Update example_data_cache.yml
garrettmflynn b7e40c3
Update example_data_cache.yml
garrettmflynn 8245334
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD 6bdc053
Update example_data_cache.yml
CodyCBakerPhD 0c8ea73
Update example_data_cache.yml
CodyCBakerPhD d361629
try without cd
CodyCBakerPhD e224ec1
Update example_data_cache.yml
CodyCBakerPhD 093eff3
Update example_data_cache.yml
CodyCBakerPhD 110ecbc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 7a33def
Update pipelines.test.ts
garrettmflynn ca70bad
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn fd3d193
Update example_pipelines.yml
garrettmflynn 3aa7ac9
Update pipelines.test.ts
garrettmflynn b2a3627
Merge branch 'main' into download-gin-data-for-tests
garrettmflynn 06cdfab
Add additional assertion
garrettmflynn c5254ff
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 498018c
Update pipelines.test.ts
garrettmflynn 0b4eec8
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn 8c07aa5
Update pipelines.test.ts
garrettmflynn e726e82
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] c5acde9
Overwrite previous screenshots
garrettmflynn f441b60
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn 0e4cc99
Use an actual assertion to test if test data exists
garrettmflynn b65f1e2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] c9889fe
Properly dismiss notifications
garrettmflynn 7b30236
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD 0731f0f
Run a test that provides insight into the errors occuring for each te…
garrettmflynn 98a34ed
Use relative paths for cache
garrettmflynn 2be2f00
Update testing_pipelines.yml
garrettmflynn 67134a5
Update testing_pipelines.yml
CodyCBakerPhD be020b4
Show env file
garrettmflynn a637d0a
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn 06fb327
Merge branch 'main' into download-gin-data-for-tests
garrettmflynn d3bc6c8
Update testing_pipelines.yml
garrettmflynn f84d191
Update testing_pipelines.yml
garrettmflynn 3474c6d
Update testing_pipelines.yml
garrettmflynn cfba1fc
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD b898150
separate printout for cleaner view
CodyCBakerPhD 7223dd1
Update pipelines.test.ts
garrettmflynn fa0ee12
Update testing_pipelines.yml
garrettmflynn a3e070f
try 'more' instead of 'type'
CodyCBakerPhD 61f74cd
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD f86a6d5
try using action
CodyCBakerPhD ac47699
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] d94dcdb
Update testing_pipelines.yml
CodyCBakerPhD 7fe5cb1
revert action
CodyCBakerPhD 6f3e300
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD b54c06d
Create test workflow
garrettmflynn 41f60f2
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD 69e82fd
Update test_env_creation.yml
garrettmflynn f2083c6
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn 4965cbe
Update deploy_tests_on_pull_request.yml
garrettmflynn 43851d7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 7cb4587
Update testing_pipelines.yml
garrettmflynn ab3162c
Merge branch 'download-gin-data-for-tests' of https://github.com/Neur…
garrettmflynn 82e85f2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] ce64563
Update to match test workflow exactly
garrettmflynn b2503a7
Update test_env_creation.yml
garrettmflynn 487676c
Update testing_pipelines.yml
garrettmflynn 0e6a899
Explicitly specify bash shell
garrettmflynn 3ec5164
Update testing_dev_with_live_services.yml
garrettmflynn aee6993
Fix dev tests
garrettmflynn d1b86d6
propagate secret
CodyCBakerPhD e5ff2c7
Merge branch 'main' into download-gin-data-for-tests
CodyCBakerPhD e7714ba
suppress windows on pipeline CI
CodyCBakerPhD a631c25
suppress inclusion as well
CodyCBakerPhD 98d6af2
suppress on dev as well
CodyCBakerPhD 58de00d
attempt another rename
CodyCBakerPhD 761de27
suppress inclusion; comment out os
CodyCBakerPhD f6593b1
restore os
CodyCBakerPhD 4ece0ed
try empty os list in dev
CodyCBakerPhD f6154e3
fix yaml; try os removal again
CodyCBakerPhD 67f5803
complete removal
CodyCBakerPhD File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
name: Daily Tests | ||
|
||
on: | ||
workflow_dispatch: | ||
schedule: | ||
- cron: "0 14 * * *" # Daily at 10am EST | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
jobs: | ||
|
||
testing_dev: | ||
uses: ./.github/workflows/testing_dev.yml | ||
secrets: | ||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
||
testing_dev_with_live_services: | ||
uses: ./.github/workflows/testing_dev_with_live_services.yml | ||
secrets: | ||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
||
testing_flask_build_and_dist: | ||
uses: ./.github/workflows/testing_flask_build_and_dist.yml | ||
|
||
example_data_cache: | ||
uses: ./.github/workflows/example_data_cache.yml | ||
secrets: | ||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
S3_GIN_BUCKET: ${{ secrets.S3_GIN_BUCKET }} | ||
|
||
testing_pipelines: | ||
needs: example_data_cache | ||
uses: ./.github/workflows/testing_pipelines.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
name: Deploy | ||
|
||
on: | ||
pull_request: | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
jobs: | ||
|
||
Dev: | ||
uses: ./.github/workflows/testing_dev.yml | ||
secrets: | ||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
||
Live: | ||
uses: ./.github/workflows/testing_dev_with_live_services.yml | ||
secrets: | ||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
||
Build: | ||
uses: ./.github/workflows/testing_flask_build_and_dist.yml | ||
|
||
Data: | ||
uses: ./.github/workflows/example_data_cache.yml | ||
secrets: | ||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
S3_GIN_BUCKET: ${{ secrets.S3_GIN_BUCKET }} | ||
|
||
Temp: | ||
uses: ./.github/workflows/test_env_creation.yml | ||
|
||
Pipelines: | ||
needs: Data | ||
uses: ./.github/workflows/testing_pipelines.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
name: Example data cache | ||
on: | ||
workflow_call: | ||
secrets: | ||
AWS_ACCESS_KEY_ID: | ||
required: true | ||
AWS_SECRET_ACCESS_KEY: | ||
required: true | ||
S3_GIN_BUCKET: | ||
required: true | ||
|
||
jobs: | ||
|
||
run: | ||
# Will read on PR dashboard as 'Deploy / Data / {os} cache' | ||
# Action dashboard identified by 'Deploy' | ||
# Requirement settings identified as '{os} cache' | ||
name: ${{ matrix.os }} cache | ||
runs-on: ${{ matrix.os }} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ["3.12"] | ||
os: [ubuntu-latest, macos-latest, macos-13, windows-latest] | ||
|
||
steps: | ||
|
||
- name: Get ephy_testing_data current head hash | ||
id: ephys | ||
run: echo "::set-output name=HASH_EPHY_DATASET::$(git ls-remote https://gin.g-node.org/NeuralEnsemble/ephy_testing_data.git HEAD | cut -f1)" | ||
- name: Get cached ephys example data - ${{ steps.ephys.outputs.HASH_EPHY_DATASET }} | ||
uses: actions/cache@v4 | ||
id: cache-ephys-datasets | ||
with: | ||
path: ./ephy_testing_data | ||
key: ephys-datasets-${{ matrix.os }}-${{ steps.ephys.outputs.HASH_EPHY_DATASET }} | ||
- name: Get ophys_testing_data current head hash | ||
id: ophys | ||
run: echo "::set-output name=HASH_OPHYS_DATASET::$(git ls-remote https://gin.g-node.org/CatalystNeuro/ophys_testing_data.git HEAD | cut -f1)" | ||
- name: Get cached ophys example data - ${{ steps.ophys.outputs.HASH_OPHYS_DATASET }} | ||
uses: actions/cache@v4 | ||
id: cache-ophys-datasets | ||
with: | ||
path: ./ophys_testing_data | ||
key: ophys-datasets-${{ matrix.os }}-${{ steps.ophys.outputs.HASH_OPHYS_DATASET }} | ||
- name: Get behavior_testing_data current head hash | ||
id: behavior | ||
run: echo "::set-output name=HASH_BEHAVIOR_DATASET::$(git ls-remote https://gin.g-node.org/CatalystNeuro/behavior_testing_data.git HEAD | cut -f1)" | ||
- name: Get cached behavior example data - ${{ steps.behavior.outputs.HASH_BEHAVIOR_DATASET }} | ||
uses: actions/cache@v4 | ||
id: cache-behavior-datasets | ||
with: | ||
path: ./behavior_testing_data | ||
key: behavior-datasets-${{ matrix.os }}-${{ steps.behavior.outputs.HASH_behavior_DATASET }} | ||
|
||
- if: steps.cache-ephys-datasets.outputs.cache-hit != 'true' || steps.cache-ophys-datasets.outputs.cache-hit != 'true' || steps.cache-behavior-datasets.outputs.cache-hit != 'true' | ||
name: Install and configure AWS CLI | ||
run: | | ||
pip install awscli | ||
aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
|
||
- if: steps.cache-ephys-datasets.outputs.cache-hit != 'true' | ||
name: Download ephys dataset from S3 | ||
run: | | ||
aws s3 cp --region=us-east-2 ${{ secrets.S3_GIN_BUCKET }}/ephy_testing_data ./ephy_testing_data --recursive | ||
- if: steps.cache-ophys-datasets.outputs.cache-hit != 'true' | ||
name: Download ophys dataset from S3 | ||
run: | | ||
aws s3 cp --region=us-east-2 ${{ secrets.S3_GIN_BUCKET }}/ophys_testing_data ./ophys_testing_data --recursive | ||
- if: steps.cache-behavior-datasets.outputs.cache-hit != 'true' | ||
name: Download behavior dataset from S3 | ||
run: | | ||
aws s3 cp --region=us-east-2 ${{ secrets.S3_GIN_BUCKET }}/behavior_testing_data ./behavior_testing_data --recursive |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
name: Test environment creation | ||
on: | ||
workflow_call: | ||
|
||
jobs: | ||
|
||
run: | ||
|
||
name: ${{ matrix.os }} environment test | ||
runs-on: ${{ matrix.os }} | ||
defaults: | ||
run: | ||
shell: bash -l {0} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
os: [ubuntu-latest, macos-latest, macos-13, windows-latest] | ||
include: | ||
- os: ubuntu-latest | ||
label: environments/environment-Linux.yml | ||
|
||
- os: macos-latest # Mac arm64 runner | ||
label: environments/environment-MAC-apple-silicon.yml | ||
|
||
- os: macos-13 # Mac x64 runner | ||
label: environments/environment-MAC-intel.yml | ||
|
||
- os: windows-latest | ||
label: environments/environment-Windows.yml | ||
|
||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set working directory as environment variable | ||
run: echo "CURRENT_WORKING_DIR=$(pwd)" >> $GITHUB_ENV | ||
if: runner.os != 'Windows' | ||
|
||
- name: Set working directory as environment variable (Windows) | ||
run: echo "CURRENT_WORKING_DIR=%cd%" >> $env:GITHUB_ENV | ||
shell: cmd | ||
if: runner.os == 'Windows' | ||
|
||
- name: Set working directory as environment file | ||
run: echo "CURRENT_WORKING_DIR=$(pwd)" >> .env | ||
if: runner.os != 'Windows' | ||
|
||
- name: Set working directory as environment file (Windows) | ||
run: echo "CURRENT_WORKING_DIR=%cd%" >> .env | ||
shell: cmd | ||
if: runner.os == 'Windows' | ||
|
||
|
||
- name: Print working directory | ||
run: echo "The current working directory is $CURRENT_WORKING_DIR" | ||
|
||
|
||
- name: Print environment file | ||
run: cat .env | ||
if: runner.os != 'Windows' | ||
|
||
- name: Print environment file | ||
run: type .env | ||
shell: cmd | ||
if: runner.os == 'Windows' |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For this style, I would recommend this syntax at the top of the file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh sure. I'm hoping to remove this ASAP.
It shows that we can, in fact, save a .env file and display the value on Windows. The
Set working directory as environment variable
option was simply to see if we can access something like this cross-platform. Will try this global env variable out!Just switched the test pipeline workflow to use the exact same configuration for saving the
.env
file. Should tell us whether we're configuring GitHub Actions wrong or the test pipeline action is somehow not compatible with saving an.env
file.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw that... CI can be really confusing sometimes
One thing it might be is the conda activation step, which swaps to a different shell type (and maybe the
.env
thing is only available from the defaultbash
??)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep looks like it. Now that we're digging into this, it looks like the
Live
service tests are also not behaving as expected—possibly as a consequence of these inconsistencies.In both
Dev
andLive
tests,No DANDI API key provided
is printed to the console—despite the latter doing a similar dump to an environment file.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm although is this only happening in this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nevermind, this was just a consequence of not registering the DANDI API Key secret