feat: check PyPI packages for presence of a wheel file in malware metadata analysis #930
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Automatically generated by Commitizen.
This PR adds support for the detection of Dockerfiles, so as to cover scenarios where Docker gets used as a build tool. * Docker presence is detected by finding files either named Dockerfile or in the formats *.Dockerfile or Dockerfile.* to cover different naming conventions of dockerfiles, e.g. dev.Dockerfile or like Macaron's own Dockerfile.base and Dockerfile.final. This is defined in defaults.ini under [builder.docker] * The supported build command keyword is build, and supported deploy command keyword is push, defined in defaults.ini under [builder.docker] *For CI deploy commands the GitHub action docker/build-push-action is supported, defined in defaults.ini under [builder.docker.ci.deploy] Signed-off-by: Tim Yarkov <[email protected]>
…#461) Signed-off-by: Nathan Nguyen <[email protected]>
This PR sets `encode=True` to encode qualifiers of a PURL string as a normalized string while converting it to a dictionary and storing it to the SQLite database because SQLite doesn't support dict type. It also adds exception handling for deserializing a PURL string while initializing a Component instance. Signed-off-by: behnazh-w <[email protected]>
…467) If a repository is not available for an artifact/analysis target identified by a PURL string, the `mcn_provenance_available_1` check throws an exception. This PR fixes this bug by checking if the repository is available before running the check. Signed-off-by: behnazh-w <[email protected]>
The micronaut-core release is generating provenances again and our provenance checks pass now. This PR updates the expected result for micronaut-core. Signed-off-by: behnazh-w <[email protected]>
…find the check result (#473) This PR fixes the following bug in the policy engine: Bug description: the policy test failed to apply the policy because it was using the `repo_id` instead the `component_id`, and the related check result could not be found by the policy engine. Signed-off-by: behnazh-w <[email protected]>
…to >=6.21.0,<6.84.4 (#470)
… >=3.0.0,<3.5.0 (#462)
… list (#459) GitPython 3.1.35 fixes CVE-2023-40590 and CVE-2023-41040. This PR removes these CVEs from the pip-audit ignore list. See https://github.com/gitpython-developers/GitPython/releases/tag/3.1.35 Signed-off-by: behnazh-w <[email protected]>
This PR adds a new check, `mcn_infer_artifact_pipeline_1` to detect a potential pipeline from which an artifact is published. When a verifiable provenance is found for an artifact, the result of this check can be discarded. Otherwise, we check whether a CI workflow run has automatically published the artifact. This check supports Maven artifacts built using Gradle or Maven and published on Maven Central only. Support for other registries and ecosystems will be added in the future. Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
Automatically generated by Commitizen.
#480) Signed-off-by: Ben Selwyn-Smith <[email protected]>
…sights (#388) This feature modifies the Repo Finder, so that it can: be usable from anywhere within Macaron; accept PURL strings as input; and, support more languages via Google's Open Source Insights (deps.dev) This enables Macaron to accept artifact PURLs as input, whereby the Repo Finder will be used to attempt to retrieve the related repository. Additional languages include those supported by deps.dev: Python, NodeJS, .Net, and Rust. Note that currently these will only work when specifying an artifact PURL as input, or providing an SBOM. Full support for these extra languages will require the addition of new dependency analyzers. A new config option is also provided to disable API calls to Google's Open Source Insights, if desired. Signed-off-by: Ben Selwyn-Smith <[email protected]>
Signed-off-by: Nathan Nguyen <[email protected]>
Signed-off-by: Nathan Nguyen <[email protected]>
This PR adds detection capabilities for projects using Go, npm and Yarn as their build tools. Note this PR does not add dependency resolution of these tools, only detection, but dependencies can be provided as a CycloneDX SBOM using `--sbom-path` CLI argument to analyze their dependencies. The `defaults.ini` file defines the specification for detection of Go, npm and Yarn projects in the relevant sections. Signed-off-by: Tim Yarkov <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
…mponent with no repository (#165) Core engine: * For all software components (main target and dependencies), the analysis will not be skipped if the repository URL is not found. * Collect and run the analysis for dependencies from the SBOM (if provided) even when the repository URL is not available for the main target. HTML reports: * Display a small message in the Target Information section when the repository is not available. * Collapse the check report table when all checks fail. Signed-off-by: Trong Nhan Mai <[email protected]>
This PR adds `changesets/action` to the list of GitHub Actions used to publish npm packages. An example project that uses this Action to automate releases is `sigstore/sigstore-js`. Note that there is a TODO to also check the `publish` input provided to this Action to improve detection accuracy. This PR also fixes a bug for using the deploy `github_actions` for build tools. Previously, only the `github_actions` of pip build tool was read from `defaults.ini` and used in `build_as_code_check.py`. This fix removes this hard-coded read from `defaults.ini` and makes it applicable to all build tools. Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: Nicholas Allen <[email protected]>
…#509) Signed-off-by: Nicholas Allen <[email protected]>
Signed-off-by: Nathan Nguyen <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: Nathan Nguyen <[email protected]>
This PR renames `mcn_infer_artifact_pipeline_1` to `mcn_find_artifact_pipeline_1`. This check can support all the package registries now. When a verifiable provenance is found for an artifact, we use it to obtain the pipeline trigger. Otherwise, we use heuristics to find the triggering pipeline. Signed-off-by: behnazh-w <[email protected]>
…923) The GitHub API for some reason does not anymore return the steps information of the job that has published pkg:maven/io.micronaut.test/[email protected] even though it was published in Aug 2024, which is much earlier than the 400 retention policy. This PR raises a new exception to handle this case and allows the corresponding integration test to fail. Signed-off-by: behnazh-w <[email protected]>
If a package is already known to be malicious, this PR reports it as part of the mcn_detect_malicious_metadata_1 check. Additionally, two new integration tests for known Python and npm malware have been added. Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
Signed-off-by: behnazh-w <[email protected]>
Automatically generated by Commitizen.
oracle-contributor-agreement
bot
added
the
OCA Verified
art1f1c3R
changed the title
art1f1c3R
changed the title
art1f1c3R
changed the title
nicallen
reviewed
Nov 29, 2024
behnazh-w
reviewed
Nov 29, 2024
behnazh-w
approved these changes
Nov 29, 2024
nicallen
approved these changes
Nov 29, 2024
… in the pypi malware analyzer, which checks for whether a wheel file is available with the package.
…quester, which now does not report a false-positive Signed-off-by: Carl Flottmann <[email protected]>
…onvention of other heuristics. Cleaned up code for the invalid test case for the heuristic's test Signed-off-by: Carl Flottmann <[email protected]>
Signed-off-by: Carl Flottmann <[email protected]>
art1f1c3R
force-pushed
the
art1f1c3R/download-file-presence
branch
from
fe0b375
to
4f0ffde
Compare
oracle-contributor-agreement
bot
added
OCA Required
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A new heuristic has been added to the PyPI package malware analyzer heuristics that checks if the specified package and version is available as a wheel (
.whl) file. The rationale behind this is that, if a malicious package has a suspicious setup file (setup.py), then it will omit a wheel file in the package so that when installed, the setup file will be run automatically. This new heuristic passes if there is a wheel file available, and fails if there is not. It has been added to the suspicious combinations (SUSPICIOUS_COMBO) such that, when the suspicious setup heuristic fails, this heuristic must fail. When the suspicious setup heuristic passes, this heuristic is indifferent and has no effect.New files added to the project are:
src/macaron/malware_analyzer/pypi_heuristics/metadata/wheel_presence.py: the new heuristic and its analyze functionalitytests/malware_analyzer/pypi/test_wheel_presence.py: a test file for the new heuristic.