This repo demonstrates differences in coverage reporting between vitest and vite-plugin-istanbul (see vitest-dev/vitest#7109)
- Clone the repo and
npm install
- Do
npm run cypress
andnpm run report-cypress
. This generates coverage JSON in.nyc_output/out.json
and LCOV report incoverage/cypress
. - Do
npm run vitest
. This generates coverage JSON and an LCOV report incoverage/vitest
. - Inspecting both LCOV reports, it is easy to verify that both branches of
UnderTest.tsx
are exercised - one by Cypress and one by vitest
- Inspecting both JSON files, it is slightly less easy to see that the statement maps do not agree with each other on where the statements are.
- Do
npm run merge-coverage
andnpm run report-merged-coverage
. This generates merged coverage JSON and an LCOV report incoverage/merged
.
Inspecting the merged LCOV report, the merge has clearly failed:
- we expect both branches of
UnderTest.tsx
to be neatly covered, but the report shows that the cypress branch is not covered - there is some sort of non-covered statement on line 3, where the input type is declared (so no statement is present)
My hypothesis is that istanbul is configured differently in vitest and vite-istanbul-plugin, making the JSON coverage files non-compatible.
One easy way to see this is in the Cypress JSON coverage file, which includes the following block:
"inputSourceMap": {
"version": 3,
"sources": [
"<REDACTED>/src/UnderTest.tsx"
],
"mappings": "AAAA,OAAO,WAAW;AAUlB,wBAAwB,UAAU,EAAE,MAAM,GAAmB;AACzD,MAAI,MAAO,QAAO,oCAAC,cAAK,UAAQ;AAChC,SAAO,oCAAC,cAAK,UAAQ;AACzB;",
"names": []
}
...which is not present in the vitest JSON output. I don't know how the mapping process works but this is clearly not respected during merge.
I thought originally this might be a bug in the istanbul-merge
library, which I only use because it has much better ergonomics than nyc merge
. Using nyc merge
gives the same result.