test-coverage.yaml should fail on error

[Moohan]

debendabot recommended updating the codecov action from 4 to 5 in our test-coverage.yaml. In the release notes, there are some breaking changes that will affect the current r-lib example: file: becomes files: and plugin: becomes plugins: I was surprised to see the workflow had passed on the PR though... Looking at the job details - https://github.com/Public-Health-Scotland/phsopendata/actions/runs/11883091545/job/33109379336 we can see that the action reported a warning and an error but the overall workflow still reported passing for some reason.

I'm not sure if this is something you can update or if it's part of the codecov action itself.

[gaborcsardi]

Well, you modified our example, and actually ended up with a broken step, so I am not sure where to go from here.

I am also not sure behind the intent of

          fail_ci_if_error: ${{ github.event_name != 'pull_request' && true || false }}

which I would think is the same as

          fail_ci_if_error: ${{ github.event_name != 'pull_request' }}

but maybe I am missing something.
Cf. https://github.com/Public-Health-Scotland/phsopendata/actions/runs/11883091545/workflow#L43

We have

          fail_ci_if_error: ${{ github.event_name != 'pull_request' || secrets.CODECOV_TOKEN }}

in the example workflow, because in a pull request there might not be a token in open source projects, so if we don't do this, then all such PRs will fail.

In any case, the workflows in the examples folder are just that, examples. If you don't like them, modify them as you like. But marking this as a bug because you don't like the default seems a bit excessive to me. :)

[Moohan]

Hey, sorry if this came across wrong. I was sure I hadn't edited the workflow, so I didn't check it where I might have noticed the fail_ci_if_error param!

I looked back at the blame, and I must have taken the example at this point in history: 76c974b. I'll update our workflow with the current content.

I guess, as with this issue: codecov/codecov-action#1348, I'd expect it to fail by default and was surprised that it hadn't. When posting an issue, the templates only allow Bug / Feature Request / Security Vulnerability, and choosing the 'Bug' template will automatically tag it as a bug.

[gaborcsardi]

I'd expect it to fail by default and was surprised that it hadn't.

It fails by default, except in PRs without a token.

only allow Bug / Feature Request / Security Vulnerability

Ah, right, makes sense, I'll update the template.