Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/task builder #131

Merged
merged 48 commits into from
Mar 27, 2021
Merged
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
ff459bd
Add task builder
JonatanMartens Feb 20, 2021
bd11a09
Add single_value function to dict conversion to task_builder
JonatanMartens Feb 20, 2021
7095d9e
Move task_builder from pyzeebe/worker to pyzeebe/task
JonatanMartens Feb 20, 2021
0f659fd
Add type annotations to build_task
JonatanMartens Feb 20, 2021
dc57eff
Add type annotations to build_task
JonatanMartens Feb 20, 2021
0ab0445
Add log when failing to execute job
JonatanMartens Feb 20, 2021
89b1667
Refactor Task to be a dataclass containing the job_handler and the co…
JonatanMartens Feb 20, 2021
67b0a0a
Add pyzeebe/task/types
JonatanMartens Feb 20, 2021
2c3f931
Move TaskDecorator from pyzeebe/task/task_decorator to pyzeebe/task/t…
JonatanMartens Feb 20, 2021
d3f2aba
Refactor task_handler to accept TaskConfig
JonatanMartens Feb 21, 2021
b8c3a94
Rename TaskHandler to ZeebeTaskRouter
JonatanMartens Feb 21, 2021
59131d0
Refactor ZeebeTaskRouter tests
JonatanMartens Feb 21, 2021
3bb171d
Refactor ZeebeWorker to use task_builder
JonatanMartens Feb 21, 2021
f52ac70
Fix worker tests
JonatanMartens Feb 21, 2021
cb03ed4
Rename functions named _ to dummy_function
JonatanMartens Feb 21, 2021
1057465
Add type annotations to ZeebeWorker tests
JonatanMartens Feb 21, 2021
d920ed9
Export TaskConfig, default_exception_handler
JonatanMartens Feb 21, 2021
d2c4d04
Remove redundant import from zeebe_worker.py
JonatanMartens Feb 21, 2021
48cb320
Remove ZeebeDecoratorBase
JonatanMartens Feb 21, 2021
d02ae2b
Refactor integration tests to use pytest fixtures and use new ZeebeWo…
JonatanMartens Feb 21, 2021
579055d
Use inspect to get function parameters
JonatanMartens Feb 21, 2021
67710f1
Import Callable from typing instead of collections
JonatanMartens Feb 23, 2021
409d114
Remove usage of dataclasses
JonatanMartens Mar 6, 2021
37f1431
Rename NoVariableNameGiven to NoVariableNameGivenError
JonatanMartens Mar 6, 2021
6a35d71
Get index with 1 instead of -1
JonatanMartens Mar 6, 2021
1797841
Change test_remove_task_from_many to have at least one task
JonatanMartens Mar 6, 2021
f7e7987
Rename TaskNotFound to TaskNotFoundError
JonatanMartens Mar 6, 2021
a154a85
Rename all pyzeebe exceptions to end with Error
JonatanMartens Mar 6, 2021
5010b12
Rename all exceptions files to errors
JonatanMartens Mar 6, 2021
d861280
Refactor build_job_handler tests to fit arrange, act, assert structure
JonatanMartens Mar 6, 2021
1da1d0e
Use pytest parameterize to simplify TestGetFunctionParameters
JonatanMartens Mar 6, 2021
2a2eaf7
Improve default_exception_handler test name
JonatanMartens Mar 6, 2021
f746baa
Add happy path to task_config tests
JonatanMartens Mar 6, 2021
8d7547b
Add docstrings to before and after methods
JonatanMartens Mar 14, 2021
f63f1d0
Rename timeout field in TaskConfig to timeout_ms
JonatanMartens Mar 14, 2021
62c6002
Clean get_parameters_from_function function inner names
JonatanMartens Mar 14, 2021
2b8ddb2
Add repr to Task and TaskConfig objects
JonatanMartens Mar 14, 2021
bd568e2
Rename wrapper to task_wrapper
JonatanMartens Mar 14, 2021
3483cf1
Rename fixture from task_handler_mock to job_handler_spy
JonatanMartens Mar 14, 2021
83c4ce6
Call route._is_task_duplicate in test_no_duplicate_task_type_error_is…
JonatanMartens Mar 15, 2021
3c57237
Receive regular params instead of task config
JonatanMartens Mar 20, 2021
876d819
Add test case for no variables added to result
JonatanMartens Mar 20, 2021
6d9e85e
Run tests on every push
JonatanMartens Mar 20, 2021
c29addb
Merge branch 'pre-release/3.0.0' into refactor/task-builder
JonatanMartens Mar 20, 2021
34e7f26
Fix integration tests to work without task config
JonatanMartens Mar 20, 2021
4a8ba92
Update ZeebeWorker documentation
JonatanMartens Mar 20, 2021
22042f4
Remove docstring whitespace in ZeebeTaskRouter
JonatanMartens Mar 27, 2021
1621369
Move variable name check back to TaskConfig
JonatanMartens Mar 27, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/test-python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ name: Test pyzeebe

on:
push:
branches: [ master, development, feature/*, bugfix/*, maintenance/*, pre-release/* ]
pull_request:
branches: [ master, development, feature/*, bugfix/*, maintenance/*, pre-release/* ]
branches: [ master, development ]

jobs:
build:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/test-zeebe-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ name: Integration test pyzeebe

on:
push:
branches: [ master, development, feature/*, bugfix/*, maintenance/*, pre-release/* ]
pull_request:
branches: [ master, development, feature/*, bugfix/*, maintenance/*, pre-release/* ]
branches: [ master, development ]

jobs:
test:
Expand Down
42 changes: 21 additions & 21 deletions docs/exceptions.rst
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@
==========
Exceptions
errors
==========

All ``pyzeebe`` exceptions inherit from :py:class:`PyZeebeException`
All ``pyzeebe`` exceptions inherit from :py:class:`PyZeebeError`

.. autoexception:: pyzeebe.exceptions.PyZeebeException
.. autoexception:: pyzeebe.errors.PyZeebeError

.. autoexception:: pyzeebe.exceptions.TaskNotFound
.. autoexception:: pyzeebe.errors.TaskNotFoundError

.. autoexception:: pyzeebe.exceptions.NoVariableNameGiven
.. autoexception:: pyzeebe.errors.NoVariableNameGivenError

.. autoexception:: pyzeebe.exceptions.NoZeebeAdapter
.. autoexception:: pyzeebe.errors.NoZeebeAdapterError

.. autoexception:: pyzeebe.exceptions.DuplicateTaskType
.. autoexception:: pyzeebe.errors.DuplicateTaskTypeError

.. autoexception:: pyzeebe.exceptions.ActivateJobsRequestInvalid
.. autoexception:: pyzeebe.errors.ActivateJobsRequestInvalidError

.. autoexception:: pyzeebe.exceptions.JobAlreadyDeactivated
.. autoexception:: pyzeebe.errors.JobAlreadyDeactivatedError

.. autoexception:: pyzeebe.exceptions.JobNotFound
.. autoexception:: pyzeebe.errors.JobNotFoundError

.. autoexception:: pyzeebe.exceptions.MessageAlreadyExists
.. autoexception:: pyzeebe.errors.MessageAlreadyExistsError

.. autoexception:: pyzeebe.exceptions.WorkflowNotFound
.. autoexception:: pyzeebe.errors.WorkflowNotFoundError

.. autoexception:: pyzeebe.exceptions.WorkflowInstanceNotFound
.. autoexception:: pyzeebe.errors.WorkflowInstanceNotFoundError

.. autoexception:: pyzeebe.exceptions.WorkflowHasNoStartEvent
.. autoexception:: pyzeebe.errors.WorkflowHasNoStartEventError

.. autoexception:: pyzeebe.exceptions.WorkflowInvalid
.. autoexception:: pyzeebe.errors.WorkflowInvalidError

.. autoexception:: pyzeebe.exceptions.InvalidJSON
.. autoexception:: pyzeebe.errors.InvalidJSONError

.. autoexception:: pyzeebe.exceptions.ZeebeBackPressure
.. autoexception:: pyzeebe.errors.ZeebeBackPressureError

.. autoexception:: pyzeebe.exceptions.ZeebeGatewayUnavailable
.. autoexception:: pyzeebe.errors.ZeebeGatewayUnavailableError

.. autoexception:: pyzeebe.exceptions.ZeebeInternalError
.. autoexception:: pyzeebe.errors.ZeebeInternalError

.. autoexception:: pyzeebe.exceptions.InvalidOAuthCredentials
.. autoexception:: pyzeebe.errors.InvalidOAuthCredentialsError

.. autoexception:: pyzeebe.exceptions.InvalidCamundaCloudCredentials
.. autoexception:: pyzeebe.errors.InvalidCamundaCloudCredentialsError
8 changes: 3 additions & 5 deletions docs/worker_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
Worker Reference
================

The :py:class:`ZeebeTaskHandler` class from which both :py:class:`ZeebeWorker` and :py:class:`ZeebeTaskRouter` inherit.
The :py:class:`ZeebeWorker` class inherits from :py:class:`ZeebeTaskRouter` class.
This means that all methods that :py:class:`ZeebeTaskRouter` has will also appear in :py:class:`ZeebeWorker`.

.. autoclass:: pyzeebe.worker.task_handler.ZeebeTaskHandler
.. autoclass:: pyzeebe.ZeebeTaskRouter
:members:
:undoc-members:

Expand All @@ -13,9 +14,6 @@ The :py:class:`ZeebeTaskHandler` class from which both :py:class:`ZeebeWorker` a
:members:
:undoc-members:

.. autoclass:: pyzeebe.ZeebeTaskRouter
:members:
:undoc-members:

.. autoclass:: pyzeebe.Job
:members:
Expand Down
7 changes: 4 additions & 3 deletions pyzeebe/__init__.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
__version__ = "2.3.1"

from pyzeebe import exceptions
from pyzeebe import errors
from pyzeebe.client.client import ZeebeClient
from pyzeebe.credentials.camunda_cloud_credentials import CamundaCloudCredentials
from pyzeebe.credentials.oauth_credentials import OAuthCredentials
from pyzeebe.job.job import Job
from pyzeebe.job.job_status import JobStatus
from pyzeebe.task.exception_handler import ExceptionHandler
from pyzeebe.task.task_decorator import TaskDecorator
from pyzeebe.worker.task_router import ZeebeTaskRouter
from pyzeebe.task.task_config import TaskConfig
from pyzeebe.task.types import TaskDecorator
from pyzeebe.worker.task_router import ZeebeTaskRouter, default_exception_handler
from pyzeebe.worker.worker import ZeebeWorker
38 changes: 19 additions & 19 deletions pyzeebe/client/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@ def run_workflow(self, bpmn_process_id: str, variables: Dict = None, version: in
int: workflow_instance_key, the unique id of the running workflow generated by Zeebe.

Raises:
WorkflowNotFound: No workflow with bpmn_process_id exists
InvalidJSON: variables is not JSONable
WorkflowHasNoStartEvent: The specified workflow does not have a start event
ZeebeBackPressure: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailable: If the Zeebe gateway is unavailable
WorkflowNotFoundError: No workflow with bpmn_process_id exists
InvalidJSONError: variables is not JSONable
WorkflowHasNoStartEventError: The specified workflow does not have a start event
ZeebeBackPressureError: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailableError: If the Zeebe gateway is unavailable
ZeebeInternalError: If Zeebe experiences an internal error

"""
Expand All @@ -62,11 +62,11 @@ def run_workflow_with_result(self, bpmn_process_id: str, variables: Dict = None,
tuple: (The workflow instance key, A dictionary of the end state of the workflow instance)

Raises:
WorkflowNotFound: No workflow with bpmn_process_id exists
InvalidJSON: variables is not JSONable
WorkflowHasNoStartEvent: The specified workflow does not have a start event
ZeebeBackPressure: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailable: If the Zeebe gateway is unavailable
WorkflowNotFoundError: No workflow with bpmn_process_id exists
InvalidJSONError: variables is not JSONable
WorkflowHasNoStartEventError: The specified workflow does not have a start event
ZeebeBackPressureError: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailableError: If the Zeebe gateway is unavailable
ZeebeInternalError: If Zeebe experiences an internal error

"""
Expand All @@ -86,9 +86,9 @@ def cancel_workflow_instance(self, workflow_instance_key: int) -> int:
int: The workflow_instance_key

Raises:
WorkflowInstanceNotFound: If no workflow instance with workflow_instance_key exists
ZeebeBackPressure: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailable: If the Zeebe gateway is unavailable
WorkflowInstanceNotFoundError: If no workflow instance with workflow_instance_key exists
ZeebeBackPressureError: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailableError: If the Zeebe gateway is unavailable
ZeebeInternalError: If Zeebe experiences an internal error

"""
Expand All @@ -103,9 +103,9 @@ def deploy_workflow(self, *workflow_file_path: str) -> None:
workflow_file_path (str): The file path to a workflow definition file (bpmn/yaml)

Raises:
WorkflowInvalid: If one of the workflow file definitions is invalid
ZeebeBackPressure: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailable: If the Zeebe gateway is unavailable
WorkflowInvalidError: If one of the workflow file definitions is invalid
ZeebeBackPressureError: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailableError: If the Zeebe gateway is unavailable
ZeebeInternalError: If Zeebe experiences an internal error

"""
Expand All @@ -125,9 +125,9 @@ def publish_message(self, name: str, correlation_key: str, variables: Dict = Non
active, a MessageAlreadyExists will be raised.

Raises:
MessageAlreadyExist: If a message with message_id already exists
ZeebeBackPressure: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailable: If the Zeebe gateway is unavailable
MessageAlreadyExistError: If a message with message_id already exists
ZeebeBackPressureError: If Zeebe is currently in back pressure (too many requests)
ZeebeGatewayUnavailableError: If the Zeebe gateway is unavailable
ZeebeInternalError: If Zeebe experiences an internal error

"""
Expand Down
6 changes: 3 additions & 3 deletions pyzeebe/credentials/camunda_cloud_credentials.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from pyzeebe.credentials.oauth_credentials import OAuthCredentials
from pyzeebe.exceptions import InvalidOAuthCredentials, InvalidCamundaCloudCredentials
from pyzeebe.errors import InvalidOAuthCredentialsError, InvalidCamundaCloudCredentialsError


class CamundaCloudCredentials(OAuthCredentials):
def __init__(self, client_id: str, client_secret: str, cluster_id: str):
try:
super().__init__(url="https://login.cloud.camunda.io/oauth/token", client_id=client_id,
client_secret=client_secret, audience=f"{cluster_id}.zeebe.camunda.io")
except InvalidOAuthCredentials:
raise InvalidCamundaCloudCredentials(client_id=client_id, cluster_id=cluster_id)
except InvalidOAuthCredentialsError:
raise InvalidCamundaCloudCredentialsError(client_id=client_id, cluster_id=cluster_id)

def get_connection_uri(self) -> str:
return f"{self.audience}:443"
4 changes: 2 additions & 2 deletions pyzeebe/credentials/oauth_credentials.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from requests_oauthlib import OAuth2Session

from pyzeebe.credentials.base_credentials import BaseCredentials
from pyzeebe.exceptions import InvalidOAuthCredentials
from pyzeebe.errors import InvalidOAuthCredentialsError


class OAuthCredentials(BaseCredentials):
Expand Down Expand Up @@ -34,7 +34,7 @@ def get_access_token(url: str, client_id: str, client_secret: str, audience: str
response.raise_for_status()
return response.json()["access_token"]
except HTTPError:
raise InvalidOAuthCredentials(url=url, client_id=client_id, audience=audience)
raise InvalidOAuthCredentialsError(url=url, client_id=client_id, audience=audience)

def get_connection_uri(self) -> str:
return None
Empty file removed pyzeebe/decorators/__init__.py
Empty file.
15 changes: 0 additions & 15 deletions pyzeebe/decorators/zeebe_decorator_base.py

This file was deleted.

6 changes: 6 additions & 0 deletions pyzeebe/errors/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from .credentials_errors import *
from .job_errors import *
from .message_errors import *
from .pyzeebe_errors import *
from .workflow_errors import *
from .zeebe_errors import *
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from pyzeebe.exceptions.pyzeebe_exceptions import PyZeebeException
from pyzeebe.errors.pyzeebe_errors import PyZeebeError


class InvalidOAuthCredentials(PyZeebeException):
class InvalidOAuthCredentialsError(PyZeebeError):
def __init__(self, url: str, client_id: str, audience: str):
super().__init__(
f"Invalid OAuth credentials supplied for {url} with audience {audience} and client id {client_id}")


class InvalidCamundaCloudCredentials(PyZeebeException):
class InvalidCamundaCloudCredentialsError(PyZeebeError):
def __init__(self, client_id: str, cluster_id: str):
super().__init__(f"Invalid credentials supplied for cluster {cluster_id} with client {client_id}")
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from pyzeebe.exceptions.pyzeebe_exceptions import PyZeebeException
from pyzeebe.errors.pyzeebe_errors import PyZeebeError


class ActivateJobsRequestInvalid(PyZeebeException):
class ActivateJobsRequestInvalidError(PyZeebeError):
def __init__(self, task_type: str, worker: str, timeout: int, max_jobs_to_activate: int):
msg = "Failed to activate jobs. Reasons:"
if task_type == "" or task_type is None:
Expand All @@ -16,13 +16,13 @@ def __init__(self, task_type: str, worker: str, timeout: int, max_jobs_to_activa
super().__init__(msg)


class JobAlreadyDeactivated(PyZeebeException):
class JobAlreadyDeactivatedError(PyZeebeError):
def __init__(self, job_key: int):
super().__init__(f"Job {job_key} was already stopped (Completed/Failed/Error)")
self.job_key = job_key


class JobNotFound(PyZeebeException):
class JobNotFoundError(PyZeebeError):
def __init__(self, job_key: int):
super().__init__(f"Job {job_key} not found")
self.job_key = job_key
5 changes: 5 additions & 0 deletions pyzeebe/errors/message_errors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pyzeebe.errors.pyzeebe_errors import PyZeebeError


class MessageAlreadyExistsError(PyZeebeError):
pass
Original file line number Diff line number Diff line change
@@ -1,25 +1,26 @@
class PyZeebeException(Exception):
class PyZeebeError(Exception):
pass


class TaskNotFound(PyZeebeException):
class TaskNotFoundError(PyZeebeError):
pass


class NoVariableNameGiven(PyZeebeException):
class NoVariableNameGivenError(PyZeebeError):
def __init__(self, task_type: str):
super().__init__(f"No variable name given for single_value task {task_type}")
self.task_type = task_type


class NoZeebeAdapter(PyZeebeException):
class NoZeebeAdapterError(PyZeebeError):
pass


class DuplicateTaskType(PyZeebeException):
class DuplicateTaskTypeError(PyZeebeError):
def __init__(self, task_type: str):
super().__init__(f"Task with type {task_type} already exists")
self.task_type = task_type

class MaxConsecutiveTaskThreadError(PyZeebeException):

class MaxConsecutiveTaskThreadError(PyZeebeError):
pass
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
from pyzeebe.exceptions.pyzeebe_exceptions import PyZeebeException
from pyzeebe.errors.pyzeebe_errors import PyZeebeError


class WorkflowNotFound(PyZeebeException):
class WorkflowNotFoundError(PyZeebeError):
def __init__(self, bpmn_process_id: str, version: int):
super().__init__(
f"Workflow definition: {bpmn_process_id} with {version} was not found")
self.bpmn_process_id = bpmn_process_id
self.version = version


class WorkflowInstanceNotFound(PyZeebeException):
class WorkflowInstanceNotFoundError(PyZeebeError):
def __init__(self, workflow_instance_key: int):
super().__init__(f"Workflow instance key: {workflow_instance_key} was not found")
self.workflow_instance_key = workflow_instance_key


class WorkflowHasNoStartEvent(PyZeebeException):
class WorkflowHasNoStartEventError(PyZeebeError):
def __init__(self, bpmn_process_id: str):
super().__init__(f"Workflow {bpmn_process_id} has no start event that can be called manually")
self.bpmn_process_id = bpmn_process_id


class WorkflowInvalid(PyZeebeException):
class WorkflowInvalidError(PyZeebeError):
pass


class InvalidJSON(PyZeebeException):
class InvalidJSONError(PyZeebeError):
pass
13 changes: 13 additions & 0 deletions pyzeebe/errors/zeebe_errors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from pyzeebe.errors.pyzeebe_errors import PyZeebeError


class ZeebeBackPressureError(PyZeebeError):
pass


class ZeebeGatewayUnavailableError(PyZeebeError):
pass


class ZeebeInternalError(PyZeebeError):
pass
6 changes: 0 additions & 6 deletions pyzeebe/exceptions/__init__.py

This file was deleted.

Loading