feat: add initial support for uv (#816)

* refactor(dependency): move `pep621` to `pep621.base` * refactor(dependency): move `pdm` to `pep621.pdm` * refactor(pep621): simplify logic * refactor(pep621): get dev dependencies from main `get` * refactor(pep621): remove useless argument for PEP 508 extraction * style(pdm): move misplaced dataclass attribute * feat(pep621): add uv dependency getter * feat(dependency): detect uv * test(functional): test against uv * docs: mention uv support * docs(uv): better document dev dependencies
fpgmaas · Aug 25, 2024 · 673086e · 673086e
1 parent 4e82f7a
commit 673086e
Show file tree

Hide file tree

Showing 19 changed files with 423 additions and 100 deletions.
diff --git a/README.md b/README.md
@@ -9,14 +9,13 @@
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/deptry)](https://pypistats.org/packages/deptry)
 [![License](https://img.shields.io/github/license/fpgmaas/deptry)](https://img.shields.io/github/license/fpgmaas/deptry)
 
-_deptry_ is a command line tool to check for issues with dependencies in a Python project, such as unused or missing dependencies. It supports the following types of projects:
+_deptry_ is a command line tool to check for issues with dependencies in a Python project, such as unused or missing
+dependencies. It supports projects
+using [Poetry](https://python-poetry.org/), [pip](https://pip.pypa.io/), [PDM](https://pdm-project.org/), [uv](https://docs.astral.sh/uv/),
+and more generally any project supporting [PEP 621](https://peps.python.org/pep-0621/) specification.
 
-- Projects that use [Poetry](https://python-poetry.org/) and a corresponding _pyproject.toml_ file
-- Projects that use [PDM](https://pdm.fming.dev/latest/) and a corresponding _pyproject.toml_ file
-- Projects that use any package manager that strictly follows [PEP 621](https://peps.python.org/pep-0621/) dependency specification
-- Projects that use a _requirements.txt_ file according to the [pip](https://pip.pypa.io/en/stable/user_guide/) standards
-
-Dependency issues are detected by scanning for imported modules within all Python files in a directory and its subdirectories, and comparing those to the dependencies listed in the project's requirements.
+Dependency issues are detected by scanning for imported modules within all Python files in a directory and its
+subdirectories, and comparing those to the dependencies listed in the project's requirements.
 
 ---
 <p align="center">

diff --git a/docs/index.md b/docs/index.md
@@ -17,12 +17,10 @@
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/deptry)](https://pypistats.org/packages/deptry)
 [![License](https://img.shields.io/github/license/fpgmaas/deptry)](https://img.shields.io/github/license/fpgmaas/deptry)
 
-_deptry_ is a command line tool to check for issues with dependencies in a Python project, such as unused or missing dependencies. It supports the following types of projects:
-
-- Projects that use [Poetry](https://python-poetry.org/) and a corresponding `pyproject.toml` file
-- Projects that use [PDM](https://pdm.fming.dev/latest/) and a corresponding `pyproject.toml` file
-- Projects that use any package manager that strictly follows [PEP 621](https://peps.python.org/pep-0621/) dependency specification
-- Projects that use a `requirements.txt` file according to the [pip](https://pip.pypa.io/en/stable/user_guide/) standards
+_deptry_ is a command line tool to check for issues with dependencies in a Python project, such as unused or missing
+dependencies. It supports projects
+using [Poetry](https://python-poetry.org/), [pip](https://pip.pypa.io/), [PDM](https://pdm-project.org/), [uv](https://docs.astral.sh/uv/),
+and more generally any project supporting [PEP 621](https://peps.python.org/pep-0621/) specification.
 
 Dependency issues are detected by scanning for imported modules within all Python files in a directory and its subdirectories, and comparing those to the dependencies listed in the project's requirements.
 

diff --git a/docs/usage.md b/docs/usage.md
@@ -34,13 +34,16 @@ To determine the project's dependencies, _deptry_ will scan the directory it is
 1. If a `pyproject.toml` file with a `[tool.poetry.dependencies]` section is found, _deptry_ will assume it uses Poetry and extract:
     - dependencies from `[tool.poetry.dependencies]` section
     - development dependencies from `[tool.poetry.group.dev.dependencies]` or `[tool.poetry.dev-dependencies]` section
-2. If a `pyproject.toml` file with a `[tool.pdm.dev-dependencies]` section is found, _deptry_ will assume it uses PDM and extract:
+1. If a `pyproject.toml` file with a `[tool.pdm.dev-dependencies]` section is found, _deptry_ will assume it uses PDM and extract:
     - dependencies from `[project.dependencies]` and `[project.optional-dependencies]` sections
     - development dependencies from `[tool.pdm.dev-dependencies]` section and from the groups under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
-3. If a `pyproject.toml` file with a `[project]` section is found, _deptry_ will assume it uses [PEP 621](https://peps.python.org/pep-0621/) for dependency specification and extract:
+1. If a `pyproject.toml` file with a `[tool.uv.dev-dependencies]` section is found, _deptry_ will assume it uses uv and extract:
+    - dependencies from `[project.dependencies]` and `[project.optional-dependencies]` sections
+    - development dependencies from `[tool.uv.dev-dependencies]` section and from the groups under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
+1. If a `pyproject.toml` file with a `[project]` section is found, _deptry_ will assume it uses [PEP 621](https://peps.python.org/pep-0621/) for dependency specification and extract:
     - dependencies from `[project.dependencies]` and `[project.optional-dependencies]`.
     - development dependencies from the groups under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
-4. If a `requirements.in` or `requirements.txt` file is found, _deptry_ will:
+1. If a `requirements.in` or `requirements.txt` file is found, _deptry_ will:
     - extract dependencies from that file.
     - extract development dependencies from `dev-dependencies.txt` and `dependencies-dev.txt`, if any exist
 

diff --git a/python/deptry/dependency_getter/builder.py b/python/deptry/dependency_getter/builder.py
@@ -5,8 +5,9 @@
 from pathlib import Path
 from typing import TYPE_CHECKING, Mapping
 
-from deptry.dependency_getter.pdm import PDMDependencyGetter
-from deptry.dependency_getter.pep_621 import PEP621DependencyGetter
+from deptry.dependency_getter.pep621.base import PEP621DependencyGetter
+from deptry.dependency_getter.pep621.pdm import PDMDependencyGetter
+from deptry.dependency_getter.pep621.uv import UvDependencyGetter
 from deptry.dependency_getter.poetry import PoetryDependencyGetter
 from deptry.dependency_getter.requirements_files import RequirementsTxtDependencyGetter
 from deptry.exceptions import DependencySpecificationNotFoundError
@@ -47,6 +48,9 @@ def build(self) -> DependencyGetter:
             if self._project_uses_pdm(pyproject_toml):
                 return PDMDependencyGetter(self.config, self.package_module_name_map, self.pep621_dev_dependency_groups)
 
+            if self._project_uses_uv(pyproject_toml):
+                return UvDependencyGetter(self.config, self.package_module_name_map, self.pep621_dev_dependency_groups)
+
             if self._project_uses_pep_621(pyproject_toml):
                 return PEP621DependencyGetter(
                     self.config, self.package_module_name_map, self.pep621_dev_dependency_groups
@@ -102,6 +106,23 @@ def _project_uses_pdm(pyproject_toml: dict[str, Any]) -> bool:
         else:
             return True
 
+    @staticmethod
+    def _project_uses_uv(pyproject_toml: dict[str, Any]) -> bool:
+        try:
+            pyproject_toml["tool"]["uv"]["dev-dependencies"]
+            logging.debug(
+                "pyproject.toml contains a [tool.uv.dev-dependencies] section, so uv is used to specify the project's"
+                " dependencies."
+            )
+        except KeyError:
+            logging.debug(
+                "pyproject.toml does not contain a [tool.uv.dev-dependencies] section, so uv is not used to specify the"
+                " project's dependencies."
+            )
+            return False
+        else:
+            return True
+
     @staticmethod
     def _project_uses_pep_621(pyproject_toml: dict[str, Any]) -> bool:
         if pyproject_toml.get("project"):

diff --git a/python/deptry/dependency_getter/pep621/__init__.py b/python/deptry/dependency_getter/pep621/__init__.py
diff --git a/python/deptry/dependency_getter/pep_621.py → ...n/deptry/dependency_getter/pep621/base.py b/python/deptry/dependency_getter/pep_621.py → ...n/deptry/dependency_getter/pep621/base.py
@@ -1,22 +1,16 @@
 from __future__ import annotations
 
-import itertools
 import logging
 import re
 from dataclasses import dataclass
-from typing import TYPE_CHECKING
 
 from deptry.dependency import Dependency
 from deptry.dependency_getter.base import DependenciesExtract, DependencyGetter
 from deptry.utils import load_pyproject_toml
 
-if TYPE_CHECKING:
-    from collections.abc import Mapping, Sequence
-
 
 @dataclass
 class PEP621DependencyGetter(DependencyGetter):
-    pep621_dev_dependency_groups: tuple[str, ...] = ()
     """
     Class to extract dependencies from a pyproject.toml file in which dependencies are specified according to PEP 621. For example:
 
@@ -41,34 +35,36 @@ class PEP621DependencyGetter(DependencyGetter):
     `pep621_dev_dependency_groups=(test,)`, both `pytest` and `pytest-cov` are returned as development dependencies.
     """
 
+    pep621_dev_dependency_groups: tuple[str, ...] = ()
+
     def get(self) -> DependenciesExtract:
         dependencies = self._get_dependencies()
         optional_dependencies = self._get_optional_dependencies()
 
-        if self.pep621_dev_dependency_groups:
-            self._check_for_invalid_group_names(optional_dependencies)
-            dev_dependencies, leftover_optional_dependencies = (
-                self._split_development_dependencies_from_optional_dependencies(optional_dependencies)
-            )
-            dependencies = [*dependencies, *leftover_optional_dependencies]
-            return DependenciesExtract(dependencies, dev_dependencies)
-
-        dependencies = [*dependencies, *itertools.chain(*optional_dependencies.values())]
-        return DependenciesExtract(dependencies, [])
+        dev_dependencies_from_optional, remaining_optional_dependencies = (
+            self._split_development_dependencies_from_optional_dependencies(optional_dependencies)
+        )
+        return DependenciesExtract(
+            [*dependencies, *remaining_optional_dependencies],
+            self._get_dev_dependencies(dev_dependencies_from_optional),
+        )
 
     def _get_dependencies(self) -> list[Dependency]:
         pyproject_data = load_pyproject_toml(self.config)
         dependency_strings: list[str] = pyproject_data["project"].get("dependencies", [])
-        return self._extract_pep_508_dependencies(dependency_strings, self.package_module_name_map)
+        return self._extract_pep_508_dependencies(dependency_strings)
 
     def _get_optional_dependencies(self) -> dict[str, list[Dependency]]:
         pyproject_data = load_pyproject_toml(self.config)
 
         return {
-            group: self._extract_pep_508_dependencies(dependencies, self.package_module_name_map)
+            group: self._extract_pep_508_dependencies(dependencies)
             for group, dependencies in pyproject_data["project"].get("optional-dependencies", {}).items()
         }
 
+    def _get_dev_dependencies(self, dev_dependencies_from_optional: list[Dependency]) -> list[Dependency]:
+        return dev_dependencies_from_optional
+
     def _check_for_invalid_group_names(self, optional_dependencies: dict[str, list[Dependency]]) -> None:
         missing_groups = set(self.pep621_dev_dependency_groups) - set(optional_dependencies.keys())
         if missing_groups:
@@ -86,21 +82,21 @@ def _split_development_dependencies_from_optional_dependencies(
         Split the optional dependencies into optional dependencies and development dependencies based on the `pep621_dev_dependency_groups`
         parameter. Return a tuple with two values: a list of the development dependencies and a list of the remaining 'true' optional dependencies.
         """
-        dev_dependencies = list(
-            itertools.chain.from_iterable(
-                deps for group, deps in optional_dependencies.items() if group in self.pep621_dev_dependency_groups
-            )
-        )
-        regular_dependencies = list(
-            itertools.chain.from_iterable(
-                deps for group, deps in optional_dependencies.items() if group not in self.pep621_dev_dependency_groups
-            )
-        )
+        dev_dependencies: list[Dependency] = []
+        regular_dependencies: list[Dependency] = []
+
+        if self.pep621_dev_dependency_groups:
+            self._check_for_invalid_group_names(optional_dependencies)
+
+        for group, dependencies in optional_dependencies.items():
+            if group in self.pep621_dev_dependency_groups:
+                dev_dependencies.extend(dependencies)
+            else:
+                regular_dependencies.extend(dependencies)
+
         return dev_dependencies, regular_dependencies
 
-    def _extract_pep_508_dependencies(
-        self, dependencies: list[str], package_module_name_map: Mapping[str, Sequence[str]]
-    ) -> list[Dependency]:
+    def _extract_pep_508_dependencies(self, dependencies: list[str]) -> list[Dependency]:
         """
         Given a list of dependency specifications (e.g. "django>2.1; os_name != 'nt'"), convert them to Dependency objects.
         """
@@ -114,7 +110,7 @@ def _extract_pep_508_dependencies(
                     Dependency(
                         name,
                         self.config,
-                        module_names=package_module_name_map.get(name),
+                        module_names=self.package_module_name_map.get(name),
                     )
                 )
 

diff --git a/python/deptry/dependency_getter/pdm.py → ...on/deptry/dependency_getter/pep621/pdm.py b/python/deptry/dependency_getter/pdm.py → ...on/deptry/dependency_getter/pep621/pdm.py
@@ -4,8 +4,7 @@
 from dataclasses import dataclass
 from typing import TYPE_CHECKING
 
-from deptry.dependency_getter.base import DependenciesExtract
-from deptry.dependency_getter.pep_621 import PEP621DependencyGetter
+from deptry.dependency_getter.pep621.base import PEP621DependencyGetter
 from deptry.utils import load_pyproject_toml
 
 if TYPE_CHECKING:
@@ -15,20 +14,13 @@
 @dataclass
 class PDMDependencyGetter(PEP621DependencyGetter):
     """
-    Class to get dependencies that are specified according to PEP 621 from a `pyproject.toml` file for a project that uses PDM for its dependency management.
+    Class to get dependencies that are specified according to PEP 621 from a `pyproject.toml` file for a project that
+    uses PDM for its dependency management.
     """
 
-    def get(self) -> DependenciesExtract:
-        pep_621_dependencies_extract = super().get()
-
-        return DependenciesExtract(
-            pep_621_dependencies_extract.dependencies,
-            [*pep_621_dependencies_extract.dev_dependencies, *self._get_pdm_dev_dependencies()],
-        )
-
-    def _get_pdm_dev_dependencies(self) -> list[Dependency]:
+    def _get_dev_dependencies(self, dev_dependencies_from_optional: list[Dependency]) -> list[Dependency]:
         """
-        Try to get development dependencies from pyproject.toml, which with PDM are specified as:
+        Retrieve dev dependencies from pyproject.toml, which in PDM are specified as:
 
         [tool.pdm.dev-dependencies]
         test = [
@@ -40,6 +32,8 @@ def _get_pdm_dev_dependencies(self) -> list[Dependency]:
             "tox-pdm>=0.5",
         ]
         """
+        dev_dependencies = super()._get_dev_dependencies(dev_dependencies_from_optional)
+
         pyproject_data = load_pyproject_toml(self.config)
 
         dev_dependency_strings: list[str] = []
@@ -50,4 +44,4 @@ def _get_pdm_dev_dependencies(self) -> list[Dependency]:
         except KeyError:
             logging.debug("No section [tool.pdm.dev-dependencies] found in pyproject.toml")
 
-        return self._extract_pep_508_dependencies(dev_dependency_strings, self.package_module_name_map)
+        return [*dev_dependencies, *self._extract_pep_508_dependencies(dev_dependency_strings)]
diff --git a/python/deptry/dependency_getter/pep621/uv.py b/python/deptry/dependency_getter/pep621/uv.py
@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+from deptry.dependency_getter.pep621.base import PEP621DependencyGetter
+from deptry.utils import load_pyproject_toml
+
+if TYPE_CHECKING:
+    from deptry.dependency import Dependency
+
+
+@dataclass
+class UvDependencyGetter(PEP621DependencyGetter):
+    """
+    Class to get dependencies that are specified according to PEP 621 from a `pyproject.toml` file for a project that
+    uses uv for its dependency management.
+    """
+
+    def _get_dev_dependencies(self, dev_dependencies_from_optional: list[Dependency]) -> list[Dependency]:
+        """
+        Retrieve dev dependencies from pyproject.toml, which in uv are specified as:
+
+        [tool.uv]
+        dev-dependencies = [
+            "pytest==8.3.2",
+            "pytest-cov==5.0.0",
+            "tox",
+        ]
+
+        Dev dependencies marked as such from optional dependencies are also added to the list of dev dependencies found.
+        """
+        dev_dependencies = super()._get_dev_dependencies(dev_dependencies_from_optional)
+
+        pyproject_data = load_pyproject_toml(self.config)
+
+        dev_dependency_strings: list[str] = []
+        try:
+            dev_dependency_strings = pyproject_data["tool"]["uv"]["dev-dependencies"]
+        except KeyError:
+            logging.debug("No section [tool.uv.dev-dependencies] found in pyproject.toml")
+
+        return [*dev_dependencies, *self._extract_pep_508_dependencies(dev_dependency_strings)]
diff --git a/tests/data/project_with_uv/pyproject.toml b/tests/data/project_with_uv/pyproject.toml
@@ -0,0 +1,32 @@
+[project]
+# PEP 621 project metadata
+# See https://www.python.org/dev/peps/pep-0621/
+name = "foo"
+version = "0.0.1"
+requires-python = ">=3.8"
+dependencies = [
+    "pkginfo==1.11.1",
+    "tomli==2.0.1",
+    "urllib3==2.2.2",
+]
+
+[project.optional-dependencies]
+foo = [
+    "click==8.1.7",
+    "isort==5.13.2",
+]
+bar = ["requests==2.32.3"]
+
+[tool.uv]
+dev-dependencies = [
+    "black==24.8.0",
+    "mypy==1.11.1",
+    "pytest==8.2.0",
+    "pytest-cov==5.0.0",
+]
+
+[tool.deptry]
+pep621_dev_dependency_groups = ["bar"]
+
+[tool.deptry.per_rule_ignores]
+DEP002 = ["pkginfo"]
diff --git a/tests/data/project_with_uv/src/main.py b/tests/data/project_with_uv/src/main.py
@@ -0,0 +1,10 @@
+from os import chdir, walk
+from pathlib import Path
+
+import black
+import click
+import mypy
+import pytest
+import pytest_cov
+import white as w
+from urllib3 import contrib
diff --git a/tests/data/project_with_uv/src/notebook.ipynb b/tests/data/project_with_uv/src/notebook.ipynb
@@ -0,0 +1,37 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "9f4924ec-2200-4801-9d49-d4833651cbc4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import click\n",
+    "from urllib3 import contrib\n",
+    "import tomli"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}