feat: Use osv-scalibr SBOM extractors #1380
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
google#1286 adds support for Maven registry during resolution. As a follow up, this PR updates the documentation for transitive scanning about specifying data source during resolution as well as specifying Maven registry. This PR also corrects the deps.dev API version we are using. We also need to update the documentation in google#1181.
…l-no-resolve` flags (google#1342) Closes google#1339 and closes google#1121 Adds flags to use offline mode for vulnerabilities (`--experimental-offline-vulnerabilities`) and transitive resolution separately (`--experimental-no-resolve`) The original `--experimental-offline` flag retains the same behaviour by functionally setting both of these flags.
Currently we can't reliably merge into v2 because checks don't automatically trigger
Another day, another snapshot change 😄
I'm not sure exactly where this file came from but we're not using it so it can go
…er group (google#1349) Bumps the bundler group in /docs with 1 update: [rexml](https://github.com/ruby/rexml). Updates `rexml` from 3.3.8 to 3.3.9 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/ruby/rexml/releases">rexml's releases</a>.</em></p> <blockquote> <h2>REXML 3.3.9 - 2024-10-24</h2> <h3>Improvements</h3> <ul> <li>Improved performance. <ul> <li><a href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li> <li>Patch by NAITOH Jun.</li> </ul> </li> </ul> <h3>Fixes</h3> <ul> <li> <p>Fixed a parse bug for text only invalid XML.</p> <ul> <li><a href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li> <li>Patch by NAITOH Jun.</li> </ul> </li> <li> <p>Fixed a parse bug that <code>&#0x...;</code> is accepted as a character reference.</p> </li> </ul> <h3>Thanks</h3> <ul> <li>NAITOH Jun</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/ruby/rexml/blob/master/NEWS.md">rexml's changelog</a>.</em></p> <blockquote> <h2>3.3.9 - 2024-10-24 {#version-3-3-9}</h2> <h3>Improvements</h3> <ul> <li>Improved performance. <ul> <li><a href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li> <li>Patch by NAITOH Jun.</li> </ul> </li> </ul> <h3>Fixes</h3> <ul> <li> <p>Fixed a parse bug for text only invalid XML.</p> <ul> <li><a href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li> <li>Patch by NAITOH Jun.</li> </ul> </li> <li> <p>Fixed a parse bug that <code>&#0x...;</code> is accepted as a character reference.</p> </li> </ul> <h3>Thanks</h3> <ul> <li>NAITOH Jun</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/ruby/rexml/commit/38eaa86ac7abe0d31cf49d8df57ad239fdeb80e9"><code>38eaa86</code></a> Add 3.3.9 entry</li> <li><a href="https://github.com/ruby/rexml/commit/ce59f2eb1aeb371fe1643414f06618dbe031979f"><code>ce59f2e</code></a> parser: fix a bug that &#0x...; is accepted as a character reference</li> <li><a href="https://github.com/ruby/rexml/commit/a09646d395a07399cbf9bc3bc8d6d8bb1d13ecea"><code>a09646d</code></a> test: fix indent</li> <li><a href="https://github.com/ruby/rexml/commit/cf0fb9c9ca3dc0d725c8e4644aa0e728025f42ce"><code>cf0fb9c</code></a> Fix <code>IOSource#readline</code> for <code>@pending_buffer</code> (<a href="https://redirect.github.com/ruby/rexml/issues/215">#215</a>)</li> <li><a href="https://github.com/ruby/rexml/commit/1d0c362526f6e25e2abcd13e2fcefcc718c20e78"><code>1d0c362</code></a> Optimize <code>IOSource#read_until</code> method (<a href="https://redirect.github.com/ruby/rexml/issues/210">#210</a>)</li> <li><a href="https://github.com/ruby/rexml/commit/622011f25ac1519fd553d6c56da52d7eba14a787"><code>622011f</code></a> Bump version</li> <li>See full diff in <a href="https://github.com/ruby/rexml/compare/v3.3.8...v3.3.9">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/google/osv-scanner/network/alerts). </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xueqin Cui <[email protected]>
# PR Summary Small PR - adjusts the sources to use the correct `usage.md` file. Signed-off-by: Emmanuel Ferdman <[email protected]> Co-authored-by: Rex P <[email protected]>
Scalibr is crashing with a SIGSEGV while trying to parse this directory: https://github.com/semgrep/semgrep/tree/develop/cli/tests/default/e2e/targets/dependency_aware This is due to the lack of an array length check after the dependencyPath split. Added failing lockfile as a test. ## Steps to reproduce it ``` $ git clone https://github.com/semgrep/semgrep $ osv-scanner semgrep/cli/tests/default/e2e/targets/dependency_aware/pnpm-error-key/pnpm-lock.yaml panic: runtime error: index out of range [0] with length 0 goroutine 1 [running]: github.com/google/osv-scanner/pkg/lockfile.extractPnpmPackageNameAndVersion({0xc000328fd0?, 0xb?}, 0x401599999999999a?) /tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:95 +0x34f github.com/google/osv-scanner/pkg/lockfile.parsePnpmLock({0xc000814230?, 0xc0004a6fc0?}) /tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:141 +0x145 github.com/google/osv-scanner/pkg/lockfile.PnpmLockExtractor.Extract({}, {0x7f545125f268, 0xc00015de40}) /tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:208 +0x25b github.com/google/osv-scanner/pkg/lockfile.ExtractDeps({0x7f545125f268, 0xc00015de40}, {0x0, 0x0}) /tmp/osv-scanner/pkg/lockfile/extract.go:61 +0x24a github.com/google/osv-scanner/pkg/osvscanner.scanLockfile({0x19374a8, 0xc000359e00}, {0xc00037e540, 0x69}, {0x0, 0x0}, 0x0) /tmp/osv-scanner/pkg/osvscanner/osvscanner.go:371 +0x8aa github.com/google/osv-scanner/pkg/osvscanner.scanDir.func1({0x7ffd822dfd35?, 0x100?}, {0x1932558, 0xc0004a4b90}, {0x0?, 0x0?}) /tmp/osv-scanner/pkg/osvscanner/osvscanner.go:168 +0x745 path/filepath.walkDir({0x7ffd822dfd35, 0x54}, {0x1932558, 0xc0004a4b90}, 0xc000814970) /usr/lib/golang/src/path/filepath/path.go:443 +0x50 path/filepath.WalkDir({0x7ffd822dfd35, 0x54}, 0xc000814970) /usr/lib/golang/src/path/filepath/path.go:533 +0x7b github.com/google/osv-scanner/pkg/osvscanner.scanDir({0x19374a8?, 0xc000359e00?}, {0x7ffd822dfd35?, 0x54?}, 0x0?, 0x0?, 0x1?, 0x0?) /tmp/osv-scanner/pkg/osvscanner/osvscanner.go:126 +0x212 github.com/google/osv-scanner/pkg/osvscanner.DoScan({{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0xc0004a4a80, 0x1, 0x1}, {0x0, ...}, ...}, ...) /tmp/osv-scanner/pkg/osvscanner/osvscanner.go:906 +0xf73 github.com/google/osv-scanner/cmd/osv-scanner/scan.action(0xc000359500, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030}) /tmp/osv-scanner/cmd/osv-scanner/scan/main.go:208 +0xca8 github.com/google/osv-scanner/cmd/osv-scanner/scan.Command.func2(0xc000815ab8?) /tmp/osv-scanner/cmd/osv-scanner/scan/main.go:145 +0x2c github.com/urfave/cli/v2.(*Command).Run(0xc0005d2160, 0xc000359500, {0xc000688080, 0x2, 0x2}) GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:276 +0x97d github.com/urfave/cli/v2.(*Command).Run(0xc0005d2840, 0xc0003593c0, {0xc0004a6060, 0x3, 0x3}) GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:269 +0xbb7 github.com/urfave/cli/v2.(*App).RunContext(0xc00024c600, {0x1932328, 0x2397500}, {0xc0004a6060, 0x3, 0x3}) GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:333 +0x5a5 github.com/urfave/cli/v2.(*App).Run(...) GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:307 main.run({0xc0000400a0, 0x2, 0x2}, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030}) /tmp/osv-scanner/cmd/osv-scanner/main.go:52 +0x678 main.main() /tmp/osv-scanner/cmd/osv-scanner/main.go:126 +0x45 ``` --------- Co-authored-by: Gareth Jones <[email protected]> Co-authored-by: Xueqin Cui <[email protected]>
Having ecosystems sorted by their name makes it easier to review this section of code
…oogle#1365) While I'm pretty these are technically invalid, they're easy to support and without this Alpine is the only comparator that panics when parsing an empty string which I think is a little sad.
…ions (google#1360) While I'm a little reluctant to do this as I'm sure I included these for a reason, they're apparently not covered by any tests and with `semantic` expected to go public soon, I think it's a bit nicer to have them gone until someone can prove they're needed as some of these could arguably be a breaking change to remove. I'm pretty sure the bulk of this was present as part of having `semantic` rebuild the parsed version for debugging when I was writing the implementation, but that's not actually a feature so we don't explicitly need to be doing it - this won't stop us from reintroducing the logic in future if we decide we want it
…oogle#1361) I realised that I have overly complicated the act of returning the result of comparing components in this function, and that the last two conditions will never be true as we loop over the largest number of components 😅
…tor (google#1362) Currently this return is not covered because we're explicitly doing all three possible comparisons, so we might as well move the equality check to the end as the default
…ons (google#1366) These letters naturally compare in the right order, so we can just compare them directly which saves us a loop and an untestable `panic`. (this will conflict with google#1362)
There's no reason for us to retain these in the internal packages, so they can just go Co-authored-by: Rex P <[email protected]>
Since this is an internal package it's not a breaking change to rename this Co-authored-by: Rex P <[email protected]>
…e#1364) Notably this adds coverage over the Ubuntu entry, since that reuses Debian so we don't have any explicit tests for that in the comparison tests. We also no longer need to be manually adding `CRAN` since `lockfile` supports it
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use osv-scalibr SBOM extractors rather than the internal ones.
Can't be merged yet as it is blocked on #1379