Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support scanning multiple uv.lock files #9

Merged
merged 7 commits into from
Dec 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ repos:
# --write-changes (Don't use this to stop typos making auto-corrections)
]
- repo: https://github.com/owenlamont/uv-secure
rev: 0.1.1
rev: 0.1.2
hooks:
- id: uv-secure
- repo: https://github.com/adrienverge/yamllint.git
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-hooks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
description: "Run 'uv-secure' to check uv.lock dependencies for known vulnerabilities"
entry: uv-secure
language: python
pass_filenames: false
files: (^|[/\\])uv\.lock$
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Scan your uv.lock file for dependencies with known vulnerabilities
## Installation

I recommend installing uv-secure as a uv tool or with pipx as it's intended to be used
as a CLI tool and it probably only makes sense to have one version installed globally.
as a CLI tool, and it probably only makes sense to have one version installed globally.

Installing with uv tool as follows:

Expand All @@ -24,17 +24,19 @@ environment.

## Usage

After installation you can run uv-secure --help to see the options.
After installation, you can run uv-secure --help to see the options.

```text
>> uv-secure --help

Usage: uv-secure [OPTIONS]
Usage: run.py [OPTIONS] [UV_LOCK_PATHS]...

Parse a uv.lock file, check vulnerabilities, and display summary.
Parse uv.lock files, check vulnerabilities, and display summary.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────╮
│ uv_lock_paths [UV_LOCK_PATHS]... Paths to the uv.lock files [default: None] │
╰──────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────╮
│ --uv-lock-path -p PATH Path to the uv.lock file [default: uv.lock] │
│ --ignore -i TEXT Comma-separated list of vulnerability IDs to │
│ ignore, e.g. VULN-123,VULN-456 │
│ --version Show the application's version │
Expand All @@ -45,8 +47,8 @@ After installation you can run uv-secure --help to see the options.
╰──────────────────────────────────────────────────────────────────────────────────────╯
```

By default if run with no options uv-secure will look for a uv.lock file in the current
working directory and scan that for known vulnerabilities. E.g.
By default, if run with no arguments uv-secure will look for a uv.lock file in the
current working directory and scan that for known vulnerabilities. E.g.

```text
>> uv-secure
Expand All @@ -65,7 +67,7 @@ uv-secure can be run as a pre-commit hook by adding this configuration to your

```yaml
- repo: https://github.com/owenlamont/uv-secure
rev: 0.1.1
rev: 0.2.0
hooks:
- id: uv-secure
```
Expand All @@ -85,8 +87,8 @@ from where pre-commit is run.

Below are some ideas (in no particular order) I have for improving uv-secure:

- Update the command line arguments and pre-commit configuration to support finding and
testing all uv.lock files.
- Support reading configuration from pyproject.toml
- Support reading configuration for multiple pyproject.toml files for mono repos
- Add rate limiting on how hard the PyPi json API is hit to query package
vulnerabilities (this hasn't been a problem yet but I suspect may be for uv.lock files
with many dependencies).
Expand Down
13 changes: 0 additions & 13 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,19 +38,6 @@ dependencies = [
"typer>=0.15.1",
]

[project.optional-dependencies]
jupyter = [
"ipywidgets>=8.1.5",
"jupyterlab-code-formatter>=3.0.2",
"jupyterlab>=4.2.5",
"lckr-jupyterlab-variableinspector>=3.2.4",
"nbdime>=4.0.2",
"nbstripout>=0.7.1",
"notebook>=7.2.2",
"ruff>=0.6.4",
"tqdm>=4.66.6",
]

[project.scripts]
uv-secure = "uv_secure.run:app"

Expand Down
4 changes: 2 additions & 2 deletions src/uv_secure/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from uv_secure.__version__ import __version__
from uv_secure.run import app, check_dependencies
from uv_secure.run import app


__all__ = ["__version__", "app", "check_dependencies"]
__all__ = ["__version__", "app"]
2 changes: 1 addition & 1 deletion src/uv_secure/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.1.2"
__version__ = "0.2.0"
4 changes: 4 additions & 0 deletions src/uv_secure/dependency_checker/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from uv_secure.dependency_checker.dependency_checker import check_dependencies


__all__ = ["check_dependencies"]
84 changes: 84 additions & 0 deletions src/uv_secure/dependency_checker/dependency_checker.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import asyncio
from pathlib import Path

import inflect
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
import typer

from uv_secure.package_info import download_vulnerabilities, parse_uv_lock_file


def check_dependencies(uv_lock_path: Path, ignore_ids: list[str]) -> int:
"""Checks dependencies for vulnerabilities and summarizes the results."""
console = Console()

if not uv_lock_path.exists():
console.print(f"[bold red]Error:[/] File {uv_lock_path} does not exist.")
raise typer.Exit(1)

dependencies = parse_uv_lock_file(uv_lock_path)
console.print(
f"[bold cyan]Checking {uv_lock_path} dependencies for vulnerabilities...[/]"
)

results = asyncio.run(download_vulnerabilities(dependencies))

total_dependencies = len(results)
vulnerable_count = 0
vulnerabilities_found = []

for dep, vulnerabilities in results:
# Filter out ignored vulnerabilities
filtered_vulnerabilities = [
vuln for vuln in vulnerabilities if vuln.id not in ignore_ids
]
if filtered_vulnerabilities:
vulnerable_count += 1
vulnerabilities_found.append((dep, filtered_vulnerabilities))

inf = inflect.engine()
total_plural = inf.plural("dependency", total_dependencies)
vulnerable_plural = inf.plural("dependency", vulnerable_count)

if vulnerable_count > 0:
console.print(
Panel.fit(
f"[bold red]Vulnerabilities detected![/]\n"
f"Checked: [bold]{total_dependencies}[/] {total_plural}\n"
f"Vulnerable: [bold]{vulnerable_count}[/] {vulnerable_plural}"
)
)

table = Table(
title="Vulnerable Dependencies",
show_header=True,
header_style="bold magenta",
)
table.add_column("Package", style="dim", width=20)
table.add_column("Version", style="dim", width=10)
table.add_column("Vulnerability ID", style="bold cyan", width=25)
table.add_column("Details", width=40)

for dep, vulnerabilities in vulnerabilities_found:
for vuln in vulnerabilities:
vuln_id_hyperlink = (
Text.assemble((vuln.id, f"link {vuln.link}"))
if vuln.link
else Text(vuln.id)
)
table.add_row(dep.name, dep.version, vuln_id_hyperlink, vuln.details)

console.print(table)
return 1 # Exit with failure status

console.print(
Panel.fit(
f"[bold green]No vulnerabilities detected![/]\n"
f"Checked: [bold]{total_dependencies}[/] {total_plural}\n"
f"All dependencies appear safe!"
)
)
return 0 # Exit successfully
5 changes: 5 additions & 0 deletions src/uv_secure/package_info/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from uv_secure.package_info.lock_file_parser import parse_uv_lock_file
from uv_secure.package_info.vulnerability_downloader import download_vulnerabilities


__all__ = ["download_vulnerabilities", "parse_uv_lock_file"]
29 changes: 29 additions & 0 deletions src/uv_secure/package_info/lock_file_parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
from pathlib import Path
import sys

from pydantic import BaseModel


# Conditional import for toml
if sys.version_info >= (3, 11):
import tomllib as toml
else:
import tomli as toml


class Dependency(BaseModel):
name: str
version: str


def parse_uv_lock_file(file_path: Path) -> list[Dependency]:
"""Parses a uv.lock TOML file and extracts package PyPi dependencies"""
with file_path.open("rb") as f:
data = toml.load(f)

package_data = data.get("package", [])
return [
Dependency(name=package["name"], version=package["version"])
for package in package_data
if package.get("source", {}).get("registry") == "https://pypi.org/simple"
]
56 changes: 56 additions & 0 deletions src/uv_secure/package_info/vulnerability_downloader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import asyncio
import re
from typing import Optional

import httpx
from pydantic import BaseModel
import typer

from uv_secure.package_info.lock_file_parser import Dependency


class Vulnerability(BaseModel):
id: str
details: str
fixed_in: Optional[list[str]] = None
aliases: Optional[list[str]] = None
link: Optional[str] = None
source: Optional[str] = None
summary: Optional[str] = None
withdrawn: Optional[str] = None


def _canonicalize_name(name: str) -> str:
"""Converts a package name to its canonical form for PyPI URLs"""
return re.sub(r"[_.]+", "-", name).lower()


async def _download_package_vulnerabilities(
client: httpx.AsyncClient, dependency: Dependency
) -> tuple[Dependency, list[Vulnerability]]:
"""Queries the PyPi JSON API for vulnerabilities of a given dependency."""
canonical_name = _canonicalize_name(dependency.name)
url = f"https://pypi.org/pypi/{canonical_name}/{dependency.version}/json"
try:
response = await client.get(url)
if response.status_code == 200:
data = response.json()
vulnerabilities = [
Vulnerability(**v) for v in data.get("vulnerabilities", [])
]
return dependency, vulnerabilities
typer.echo(
f"Warning: Could not fetch data for {dependency.name}=={dependency.version}"
)
except httpx.RequestError as e:
typer.echo(f"Error fetching {dependency.name}=={dependency.version}: {e}")
return dependency, []


async def download_vulnerabilities(
dependencies: list[Dependency],
) -> list[tuple[Dependency, list[Vulnerability]]]:
"""Fetch vulnerabilities for all dependencies concurrently."""
async with httpx.AsyncClient(timeout=10) as client:
tasks = [_download_package_vulnerabilities(client, dep) for dep in dependencies]
return await asyncio.gather(*tasks)
Loading
Loading