Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

up to stream #1

Merged
merged 3 commits into from
Dec 4, 2023
Merged

up to stream #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c916564
ci: Change the GitHub Actions trigger, drop Celery 3 support (#261)
Mulugruntz Dec 3, 2023
9990a60
Add support for CPython 3.12, Pypy 3.10 (#263)
Mulugruntz Dec 3, 2023
4342175
ci: Add PR types (#265)
Mulugruntz Dec 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,16 @@ name: Build and Test

on:
push:
branches:
- master
pull_request:
branches:
- master
types: [closed]
types:
- opened
- synchronize
- reopened
- closed

jobs:
matrix-builder:
Expand All @@ -22,9 +28,9 @@ jobs:
architecture: x64
- id: set-matrix
env:
PY_VERSIONS: "[\"3.7\", \"3.8\", \"3.9\", \"3.10\", \"3.11\", \"pypy-3.8-v7.3.11\", \"pypy-3.9-v7.3.11\"]"
PY_VERSIONS: "[\"3.7\", \"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"pypy-3.9\", \"pypy-3.10\"]"
OS_NAMES: "[\"ubuntu-20.04\"]" #, windows-latest, macOS-latest]
CELERY_VERSIONS: "[\">=3.1.0,<4.0.0\", \">=4.0.0,<5.0.0\", \">=5.0.0,<6.0.0\"]"
CELERY_VERSIONS: "[\">=4.0.0,<5.0.0\", \">=5.0.0,<6.0.0\"]"
run: |
python -m pip install --upgrade pip
MATRIX=$(python .github/workflows/resolve_versions.py matrix -c "$PY_VERSIONS" -o "$OS_NAMES" -n "celery" -s "$CELERY_VERSIONS")
Expand Down Expand Up @@ -109,6 +115,9 @@ jobs:
pip install "celery${{ matrix.celery }}"
pip install pytest-github-actions-annotate-failures

- name: Check with black
run: black --check .

- name: Check with mypy
if: ${{ !startsWith(matrix.python-version, 'pypy') && matrix.python-version >= '3.7' }}
run: mypy
Expand Down
17 changes: 4 additions & 13 deletions .github/workflows/resolve_versions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,23 +127,14 @@ def process_matrix(args: Namespace) -> None:
for c, o, n, s in matrix:
compatible_versions = get_compatible_versions(c, o, n, s)
if compatible_versions:
# This is because celery 3 depends on use_2to3,
# which is no longer supported by 3.9, 3.10 and 3.11
# and by newer pypy versions.
if (
n == "celery"
and (
c.replace(".", "").startswith("39")
or c.replace(".", "").startswith("310")
or c.replace(".", "").startswith("311")
or c.startswith("pypy-3")
)
and all(cv.startswith("3") for cv in compatible_versions)
) or (
# This is because inspect.getfullargspec has been removed in Python 3.11
# and `vine.five` 4.x was depending on it. This is fixed in 5.x.
n == "celery"
and c.replace(".", "").startswith("311")
and (
c.replace(".", "").startswith("311")
or c.replace(".", "").startswith("312")
)
and all(cv.startswith("4") for cv in compatible_versions)
):
if is_inverted:
Expand Down
44 changes: 27 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,16 @@ Publish and Subscribe with Celery

## Supported dependencies

| Python | Celery 3 | Celery 4 | Celery 5 |
|----------|-----------------------------------------------------------------------|-----------------------------------------------------------------------|-----------------------------------------------------------------------|
| 3.7 | ![][badge-m_linux_3.7_celery3] ![][badge-t_linux_3.7_celery3] | ![][badge-m_linux_3.7_celery4] ![][badge-t_linux_3.7_celery4] | ![][badge-m_linux_3.7_celery5] ![][badge-t_linux_3.7_celery5] |
| 3.8 | ![][badge-m_linux_3.8_celery3] ![][badge-t_linux_3.8_celery3] | ![][badge-m_linux_3.8_celery4] ![][badge-t_linux_3.8_celery4] | ![][badge-m_linux_3.8_celery5] ![][badge-t_linux_3.8_celery5] |
| 3.9 | ![][badge-m_linux_3.9_celery3] ![][badge-t_linux_3.9_celery3] | ![][badge-m_linux_3.9_celery4] ![][badge-t_linux_3.9_celery4] | ![][badge-m_linux_3.9_celery5] ![][badge-t_linux_3.9_celery5] |
| 3.10 | ![][badge-m_linux_3.10_celery3] ![][badge-t_linux_3.10_celery3] | ![][badge-m_linux_3.10_celery4] ![][badge-t_linux_3.10_celery4] | ![][badge-m_linux_3.10_celery5] ![][badge-t_linux_3.10_celery5] |
| 3.11 | ![][badge-m_linux_3.11_celery3] ![][badge-t_linux_3.11_celery3] | ![][badge-m_linux_3.11_celery4] ![][badge-t_linux_3.11_celery4] | ![][badge-m_linux_3.11_celery5] ![][badge-t_linux_3.11_celery5] |
| pypy 3.8 | ![][badge-m_linux_pypy3.8_celery3] ![][badge-t_linux_pypy3.8_celery3] | ![][badge-m_linux_pypy3.8_celery4] ![][badge-t_linux_pypy3.8_celery4] | ![][badge-m_linux_pypy3.8_celery5] ![][badge-t_linux_pypy3.8_celery5] |
| pypy 3.9 | ![][badge-m_linux_pypy3.9_celery3] ![][badge-t_linux_pypy3.9_celery3] | ![][badge-m_linux_pypy3.9_celery4] ![][badge-t_linux_pypy3.9_celery4] | ![][badge-m_linux_pypy3.9_celery5] ![][badge-t_linux_pypy3.9_celery5] |
| Python | Celery 3 | Celery 4 | Celery 5 |
|-----------|-------------------------------------------------------------------------|-------------------------------------------------------------------------|-------------------------------------------------------------------------|
| 3.7 | ![][badge-m_linux_3.7_celery3] ![][badge-t_linux_3.7_celery3] | ![][badge-m_linux_3.7_celery4] ![][badge-t_linux_3.7_celery4] | ![][badge-m_linux_3.7_celery5] ![][badge-t_linux_3.7_celery5] |
| 3.8 | ![][badge-m_linux_3.8_celery3] ![][badge-t_linux_3.8_celery3] | ![][badge-m_linux_3.8_celery4] ![][badge-t_linux_3.8_celery4] | ![][badge-m_linux_3.8_celery5] ![][badge-t_linux_3.8_celery5] |
| 3.9 | ![][badge-m_linux_3.9_celery3] ![][badge-t_linux_3.9_celery3] | ![][badge-m_linux_3.9_celery4] ![][badge-t_linux_3.9_celery4] | ![][badge-m_linux_3.9_celery5] ![][badge-t_linux_3.9_celery5] |
| 3.10 | ![][badge-m_linux_3.10_celery3] ![][badge-t_linux_3.10_celery3] | ![][badge-m_linux_3.10_celery4] ![][badge-t_linux_3.10_celery4] | ![][badge-m_linux_3.10_celery5] ![][badge-t_linux_3.10_celery5] |
| 3.11 | ![][badge-m_linux_3.11_celery3] ![][badge-t_linux_3.11_celery3] | ![][badge-m_linux_3.11_celery4] ![][badge-t_linux_3.11_celery4] | ![][badge-m_linux_3.11_celery5] ![][badge-t_linux_3.11_celery5] |
| 3.12 | ![][badge-m_linux_3.12_celery3] ![][badge-t_linux_3.12_celery3] | ![][badge-m_linux_3.12_celery4] ![][badge-t_linux_3.12_celery4] | ![][badge-m_linux_3.12_celery5] ![][badge-t_linux_3.12_celery5] |
| pypy 3.9 | ![][badge-m_linux_pypy3.9_celery3] ![][badge-t_linux_pypy3.9_celery3] | ![][badge-m_linux_pypy3.9_celery4] ![][badge-t_linux_pypy3.9_celery4] | ![][badge-m_linux_pypy3.9_celery5] ![][badge-t_linux_pypy3.9_celery5] |
| pypy 3.10 | ![][badge-m_linux_pypy3.10_celery3] ![][badge-t_linux_pypy3.10_celery3] | ![][badge-m_linux_pypy3.10_celery4] ![][badge-t_linux_pypy3.10_celery4] | ![][badge-m_linux_pypy3.10_celery5] ![][badge-t_linux_pypy3.10_celery5] |


## Basic usage
Expand Down Expand Up @@ -103,8 +104,9 @@ celery_pubsub.publish('some.very.good.test', 42) # task 3 only
* 2.0.0
* Drop support for CPython 2.7, 3.4, 3.5, 3.6
* Drop support for Pypy 2.7 and 3.6.
* Add support for Pypy 3.8 and 3.9.
* Add support for CPython 3.11.
* Drop support for Celery 3.
* Add support for Pypy 3.9 and 3.10.
* Add support for CPython 3.11 and 3.12.
* Type hints are now directly in the code. No more stubs files.
* 1.0.2
* Add stubs file for type hinting.
Expand Down Expand Up @@ -182,14 +184,18 @@ celery_pubsub.publish('some.very.good.test', 42) # task 3 only
[badge-m_linux_3.11_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_3.11_celery4/shields
[badge-m_linux_3.11_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_3.11_celery5/shields

[badge-m_linux_pypy3.8_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.8_celery3/shields
[badge-m_linux_pypy3.8_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.8_celery4/shields
[badge-m_linux_pypy3.8_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.8_celery5/shields
[badge-m_linux_3.12_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_3.12_celery3/shields
[badge-m_linux_3.12_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_3.12_celery4/shields
[badge-m_linux_3.12_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_3.12_celery5/shields

[badge-m_linux_pypy3.9_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.9_celery3/shields
[badge-m_linux_pypy3.9_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.9_celery4/shields
[badge-m_linux_pypy3.9_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.9_celery5/shields

[badge-m_linux_pypy3.10_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.10_celery3/shields
[badge-m_linux_pypy3.10_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.10_celery4/shields
[badge-m_linux_pypy3.10_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/m_linux_pypy-3.10_celery5/shields

[//]: # (Status in tagged version)
[badge-t_linux_3.7_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.7_celery3/shields
[badge-t_linux_3.7_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.7_celery4/shields
Expand All @@ -211,10 +217,14 @@ celery_pubsub.publish('some.very.good.test', 42) # task 3 only
[badge-t_linux_3.11_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.11_celery4/shields
[badge-t_linux_3.11_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.11_celery5/shields

[badge-t_linux_pypy3.8_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.8_celery3/shields
[badge-t_linux_pypy3.8_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.8_celery4/shields
[badge-t_linux_pypy3.8_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.8_celery5/shields
[badge-t_linux_3.12_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.12_celery3/shields
[badge-t_linux_3.12_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.12_celery4/shields
[badge-t_linux_3.12_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_3.12_celery5/shields

[badge-t_linux_pypy3.9_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.9_celery3/shields
[badge-t_linux_pypy3.9_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.9_celery4/shields
[badge-t_linux_pypy3.9_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.9_celery5/shields

[badge-t_linux_pypy3.10_celery3]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.10_celery3/shields
[badge-t_linux_pypy3.10_celery4]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.10_celery4/shields
[badge-t_linux_pypy3.10_celery5]: https://byob.yarr.is/Mulugruntz/celery-pubsub/2.0.0-beta3_linux_pypy-3.10_celery5/shields
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
celery>=3.1.0
celery>=4.0.0
61 changes: 35 additions & 26 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,39 +20,48 @@

import celery
from celery import Task
from celery.worker import WorkController

from celery_pubsub import subscribe, unsubscribe
from pkg_resources import get_distribution, parse_version
from celery_pubsub import subscribe
from packaging.version import parse, Version


# TODO: Simplify this when we drop support for Python 3.7.
if typing.TYPE_CHECKING:

def get_distribution_version(distribution_name: str) -> Version:
...

else:

def get_distribution_version(distribution_name: str) -> Version:
try:
from importlib.metadata import distribution

return parse(distribution(distribution_name).version)
except ImportError:
# Fallback for Python < 3.8. Remove when we drop support for Python 3.7.
from pkg_resources import get_distribution

return parse(get_distribution(distribution_name).version)


P = ParamSpec("P")
R = TypeVar("R")
task: Callable[..., Callable[[Callable[P, R]], Task[P, R]]]

if not typing.TYPE_CHECKING:
if get_distribution("celery").parsed_version < parse_version("4.0.0"):
celery.current_app.conf.update(
CELERY_ALWAYS_EAGER=True,
)
task = celery.task

@pytest.fixture
def celery_worker() -> WorkController:
pass

else: # pragma: no cover
task = celery.shared_task

@pytest.fixture(scope="session")
def celery_config():
return {
"broker_url": "memory://",
"result_backend": "rpc://",
"broker_transport_options": {"polling_interval": 0.05},
}

if get_distribution("celery").parsed_version >= parse_version("5.0.0"):
pytest_plugins = ["celery.contrib.pytest"]
task = celery.shared_task

@pytest.fixture(scope="session")
def celery_config():
return {
"broker_url": "memory://",
"result_backend": "rpc://",
"broker_transport_options": {"polling_interval": 0.05},
}

if get_distribution_version("celery") >= parse("5.0.0"):
pytest_plugins = ["celery.contrib.pytest"]


@pytest.fixture(scope="session")
Expand Down
Loading