chore: add pre-commit downstream workflows

[maxArturo]

Overview

• If there's a pre-commit update:
  • "require" and run the lint and test workflows, avoiding any direct recursive calls to GH actions

Description

Based on our previous conversation I looked at how to get this running with dependabot, but I quickly realized it wouldn't work as expected.

It would only run when a pip dependency changed, so effectively it wouldn't run the specific code that the current GH action runs with a daily frequency.

Also, using dependabot with pre-commit is very low on the GH team's priority.

So I decided to "unfurl" the needed downstream actions as direct dependencies on the "pre-commit" job, by "requiring" them and conditionally running linting and testing if there's a pre-commit update. This works because we can see if the pr was created or updated in the outputs of the create-pr action.

PROOF:

You can see the successful run and then the lint/test actions being triggered afterwards.

Additional Details

The one thing to keep in mind is if you want something else to conditionally run based on pre-commit update, you have to add it here as a direct downstream dependency. That's it!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[clavedeluna]

You can see the successful run and then the lint/test actions being triggered afterwards.

But the unit tests did not run ?

[maxArturo]

I cancelled the tests because I didn’t  want to wait for them to finish, but otherwise it should work as expected. You can see that in the logs. https://productionresultssa12.blob.core.windows.net/actions-results/1b5b6d3c-d7b0-4923-873d-84837550d205/workflow-job-run-bed211fd-7dd0-5134-e289-e2d60b68d8e4/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-03-18T14%3A27%3A30Z&sig=NY%2FOB170%2BMjtxbE0KxgD3WU4KXsTnGCRnnzRJvCzWzY%3D&sp=r&spr=https&sr=b&st=2024-03-18T14%3A17%3A25Z&sv=2021-12-02MaxOn Mar 18, 2024, at 10:27 AM, Dani Alcala ***@***.***> wrote: You can see the successful run and then the lint/test actions being triggered afterwards. But the unit tests did not run ? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.47%. Comparing base (e33e8ea) to head (053cebd).
Report is 119 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #386      +/-   ##
==========================================
- Coverage   96.60%   94.47%   -2.13%     
==========================================
  Files         101      126      +25     
  Lines        4275     5543    +1268     
==========================================
+ Hits         4130     5237    +1107     
- Misses        145      306     +161     

see 8 files with indirect coverage changes

[clavedeluna]

IFF integration tests fail it's because of timeout, fixing it here #387

[clavedeluna]

This is a reasonable approach to solve for the minor problem of having to manually pull and force push that auto-update commit.

Do want @drdavella to give this approval before merge though.

[drdavella]

@maxArturo thanks again for the contribution. I think this is pretty cool and it looks like it works.

I recently discovered an action for checking diffs and I wonder whether it would help simplify this action at all? It's not a blocker but I'm curious looking ahead: https://github.com/technote-space/get-diff-action

I also wonder whether we can use a similar approach to make the PyPI release conditional on test execution. It's hard to do that right now because you can't filter a workflow_run action on a specific tag (only on a branch). Anyway, just an idle thought, maybe it doesn't make much sense.

[maxArturo]

Hi @drdavella my pleasure!

I recently discovered an action for checking diffs and I wonder whether it would help simplify this action at all? It's not a blocker but I'm curious looking ahead: https://github.com/technote-space/get-diff-action

Oh that's neat! Looks like it's very full featured. I do worry about the fact that it's archived/frozen, and it might be heavy-handed for this use case. I think the git diff work is already being done by peter-evans/create-pull-request: if there's no changes to the repo I believe it won't create/update any PR's, which is why this is the conditional:

       if: |
          (steps.pr-create.outputs.pull-request-operation == 'created' ||
          steps.pr-create.outputs.pull-request-operation == 'updated') &&
          steps.pr-create.outputs.pull-request-number

If we ever need a more advanced filter criteria (e.g a subset of files/folders) to conditionally execute, I'd recommend running a "straight up" custom git diff command and avoid pulling in an entire node/NPM project that will do almost the same! 😄

I also wonder whether we can use a similar approach to make the PyPI release conditional on test execution. It's hard to do that right now because you can't filter a workflow_run action on a specific tag (only on a branch). Anyway, just an idle thought, maybe it doesn't make much sense.

Could be! Looks like the releases are handled manually, so you could just call a uses from this line and include whatever workflows you'd like to run before the pypi release happens.

I haven't had to do this before, but there's a good opportunity keep things DRY and group your code sanity GH workflows in an "omnibus" yml file. Then, you'd uses that in all the spots where you want a gate (at this point, here and on release it seems). There seems to be a balance between autotriggers (on: push, etc) and "manual" calling, but only the team members know best which fits where.