Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate from pylint and flake8 to ruff #248

Merged
merged 7 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions .flake8

This file was deleted.

30 changes: 6 additions & 24 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
exclude: ^vendor/|^tests/.*/fixtures/.*
Expand All @@ -10,39 +10,21 @@ repos:

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.6.4
rev: v0.7.0
hooks:
# Run the linter.
- id: ruff
args: [ --fix ]
# Run the formatter.
- id: ruff-format

- repo: https://github.com/timothycrosley/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort
files: \.py$
exclude: ^vendor/

- repo: https://github.com/pycqa/flake8
rev: 7.0.0
hooks:
- id: flake8
files: \.py$
args:
- --config=.flake8

# https://pre-commit.com/#repository-local-hooks
- repo: local
hooks:
- id: pylint
name: pylint
entry: pylint
language: system
types: [python]
files: \.py$
exclude: ^vendor/
args:
- --rcfile=.pylintrc

- repo: local
hooks:
- id: mypy
Expand Down
28 changes: 0 additions & 28 deletions .pylintrc

This file was deleted.

13 changes: 3 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,6 @@ test-dep-ubuntu:
tar vxf apache-zookeeper-$(ZOOKEEPER_VERSION)-bin.tar.gz --wildcards apache-zookeeper-$(ZOOKEEPER_VERSION)-bin/lib/*.jar
sudo cp -r apache-zookeeper-$(ZOOKEEPER_VERSION)-bin/lib /usr/share/zookeeper

.PHONY: pylint
pylint: $(GENERATED)
pre-commit run pylint --all-files

.PHONY: flake8
flake8: $(GENERATED)
pre-commit run flake8 --all-files

.PHONY: copyright
copyright:
Expand All @@ -69,15 +62,15 @@ unittest: $(GENERATED)
test: lint copyright unittest

.PHONY: isort
isort:
isort: $(GENERATED)
pre-commit run isort --all-files
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need isort? Ruff sorts imports.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, just removing isort b/c the current ruff config should cover sorting AFAICT


.PHONY: ruff
ruff:
ruff: $(GENERATED)
pre-commit run ruff-format --all-files

.PHONY: mypy
mypy:
mypy: $(GENERATED)
pre-commit run mypy --all-files

.PHONY: reformat
Expand Down
5 changes: 2 additions & 3 deletions astacus/client.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.
Client commands for Astacus tool
Expand Down
5 changes: 2 additions & 3 deletions astacus/common/access_log.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2024 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2024 Aiven Ltd
See LICENSE for details.
"""

from collections.abc import Sequence
Expand Down
10 changes: 4 additions & 6 deletions astacus/common/asyncstorage.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""
Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.
"""

Expand All @@ -10,7 +8,7 @@


class AsyncHexDigestStorage:
"""Subset of the HexDigestStorage API proxied async -> sync via starlette threadpool
"""Subset of the HexDigestStorage API proxied async -> sync via starlette threadpool.
Note that the access is not intentionally locked; therefore even
synchronous API can be used in parallel (at least if it is safe to
Expand All @@ -29,7 +27,7 @@ async def list_hexdigests(self) -> list[str]:


class AsyncJsonStorage:
"""Subset of the JsonStorage API proxied async -> sync via starlette threadpool
"""Subset of the JsonStorage API proxied async -> sync via starlette threadpool.
Note that the access is not intentionally locked; therefore even
synchronous API can be used in parallel (at least if it is safe to
Expand Down
6 changes: 2 additions & 4 deletions astacus/common/cachingjsonstorage.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""

Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.

Caching layer on top of storage.

Expand Down
5 changes: 2 additions & 3 deletions astacus/common/cassandra/client.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2021 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details.

This client code is originally from Cashew

Expand Down
2 changes: 1 addition & 1 deletion astacus/common/cassandra/config.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details
See LICENSE for details.

Client configuration data model

Expand Down
13 changes: 5 additions & 8 deletions astacus/common/cassandra/schema.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2021 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details.

Schema-related parts are based on basebackup_schema.py of Cashew

Expand Down Expand Up @@ -106,10 +105,9 @@ def __lt__(self, o: CassandraNamed) -> bool:

@classmethod
def get_create_statement(cls, metadata: cm.Function) -> str:
"""
Based on Function.as_cql_query, but doesn't strip `frozen` from arguments as they need to be specified on restoration
"""Based on Function.as_cql_query, but doesn't strip `frozen` from arguments as they need to be specified on restoration
With default cassandra-driver the function fails to create with an error message like "Non-frozen UDTs are not
allowed inside collections: ..."
allowed inside collections: ...".
"""
sep = " "
keyspace = cm.protect_name(metadata.keyspace)
Expand Down Expand Up @@ -306,7 +304,7 @@ class CassandraSchema(AstacusModel):

@classmethod
def from_cassandra_session(cls, cas: CassandraSession) -> "CassandraSchema":
"""Retrieve the schema using CassandraSession"""
"""Retrieve the schema using CassandraSession."""
# Cassandra driver updates the schema in the background. Right after connect it's possible the schema hasn't been
# retrieved yet, and it appears to be empty. This call should block until the schema has been retrieved, and raise
# an exception if the nodes aren't in sync
Expand Down Expand Up @@ -346,7 +344,6 @@ def restore_pre_data(self, cas: CassandraSession) -> None:
- restore_post_data is done after the restore_pre_data, and
actual snapshot restoration + moving of files has been done
"""

for keyspace in self.keyspaces:
# These may be version dependant, and in general should be recreated
# during restore (and-or other configuration applying)
Expand Down
5 changes: 2 additions & 3 deletions astacus/common/cassandra/utils.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2021 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details.

Miscellaneous utilities related to CassandraSession

Expand Down
6 changes: 2 additions & 4 deletions astacus/common/dependencies.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""
Copyright (c) 2021 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details.
Dependency injection helper functions.
"""
Expand Down
6 changes: 2 additions & 4 deletions astacus/common/etcd.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""

Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.

Minimal etcdv3 client library on top of httpx

Expand Down
5 changes: 2 additions & 3 deletions astacus/common/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.
"""


Expand Down
9 changes: 3 additions & 6 deletions astacus/common/ipc.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.
"""

from .magic import DEFAULT_EMBEDDED_FILE_SIZE, StrEnum
Expand All @@ -18,7 +17,6 @@
import socket

# pydantic validators are class methods in disguise
# pylint: disable=no-self-argument


# These are the database plugins we support; list is intentionally
Expand Down Expand Up @@ -146,8 +144,7 @@ def create_snapshot_request(


class SnapshotHash(msgspec.Struct, kw_only=True, frozen=True):
"""
This class represents something that is to be stored in the object storage.
"""This class represents something that is to be stored in the object storage.
size is provided mainly to allow for even loading of nodes in case
same hexdigest is available from multiple nodes.
Expand Down
5 changes: 2 additions & 3 deletions astacus/common/limiter.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2021 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2021 Aiven Ltd
See LICENSE for details.
"""

from collections.abc import Awaitable, Iterable
Expand Down
10 changes: 4 additions & 6 deletions astacus/common/m3placement.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""

Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.

M3 placement handling code

Expand Down Expand Up @@ -55,7 +53,7 @@ class M3PlacementNodeReplacement(AstacusModel):
def rewrite_single_m3_placement_node(
placement, *, src_pnode: M3PlacementNode, dst_pnode: M3PlacementNode, dst_isolation_group=""
):
"""rewrite single m3 placement entry in-place in protobuf
"""Rewrite single m3 placement entry in-place in protobuf.

Relevant places ( see m3db
src/cluster/generated/proto/placementpb/placement.proto which is
Expand Down Expand Up @@ -85,7 +83,7 @@ def rewrite_single_m3_placement_node(


def rewrite_m3_placement_bytes(value: bytes, replacements: Sequence[M3PlacementNodeReplacement]):
"""rewrite whole binary m3 placement, with the set of node replacements"""
"""Rewrite whole binary m3 placement, with the set of node replacements."""
placement = m3_placement_pb2.Placement()
placement.ParseFromString(value)
for replacement in replacements:
Expand Down
5 changes: 2 additions & 3 deletions astacus/common/magic.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2020 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2020 Aiven Ltd
See LICENSE for details.
"""

from enum import Enum
Expand Down
5 changes: 2 additions & 3 deletions astacus/common/msgspec_glue.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""
Copyright (c) 2024 Aiven Ltd
See LICENSE for details
"""Copyright (c) 2024 Aiven Ltd
See LICENSE for details.
"""

from pydantic import PydanticValueError
Expand Down
Loading
Loading