Skip to content

Commit

Permalink
Remove FlowProject.add_operation (#487)
Browse files Browse the repository at this point in the history
* Fix FlowProject.add_operation docstring error

* Make FlowProject.add_operation a classmethod

* Add test for FlowProject.add_operation classmethod

* Update changelog for `FlowProject.add_operation`.

* Remove ``FlowProject.add_operation``

* Add documentation for directives. (#480)

* Add documentation for directives.

* Fix directives docs, add _FORK directive.

* Update changelog.

* Bump pre-commit from 2.10.1 to 2.11.1 (#489)

Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 2.10.1 to 2.11.1.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/master/CHANGELOG.md)
- [Commits](pre-commit/pre-commit@v2.10.1...v2.11.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Change default status_parallelization to "none". (#486)

* Change default status_parallelization to process.

* Update changelog.

* Change default status_parallelization to "none".

* Bump ruamel-yaml from 0.16.12 to 0.17.2 (#490)

Bumps [ruamel-yaml](https://sourceforge.net/p/ruamel-yaml/code/ci/default/tree) from 0.16.12 to 0.17.2.

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bradley Dice <[email protected]>

* Deprecate and remove references to add_operation.

* Fix import.

* Default memory requests to integer values (#484)

* converting to int

* updating docstring

* switching to int output for M and G

* Update flow/util/template_filters.py

Co-authored-by: Bradley Dice <[email protected]>

* fixing formatting

* Update template reference data.s

* Minor edits.

* Update changelog.

* Add support for "GB" suffixes.

* Update memory tests.

Co-authored-by: Bradley Dice <[email protected]>

* Changed TORQUE to PBS (#491)

Renamed TORQUE scheduler related classes, objects, templates, etc to PBS in code and documentation.

* changed torque to pbs

* fixed isort error

* changed torque to pbs

* rename PBSScheduler

* rename PBSScheduler

* rename PBSScheduler

* Add changelog entry.

* Rename Torque -> PBS in more places.

* Update contributors.yaml

* Added back deprecated class for DefaultTorqueEnvironment.

* Remove extra line in contributors.yaml.

* Fix import.

Co-authored-by: Bradley Dice <[email protected]>

* Add submission summary feature. (#488)

* Add submission summary feature.

* Update changelog.

* Sort status counts by status code.

Co-authored-by: Bradley Dice <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alyssa Travitz <[email protected]>
Co-authored-by: Charlottez112 <[email protected]>
  • Loading branch information
5 people authored Apr 12, 2021
1 parent 9990950 commit c51bbc2
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 72 deletions.
1 change: 1 addition & 0 deletions changelog.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Fixed
Removed
+++++++

- Removed ``FlowProject.add_operation`` (#487).
- Removed deprecated ``--walltime`` argument (#478).
- Removed deprecated ``flow.run`` interface (#478).
- Removed deprecated ``FlowProject.export_job_statuses`` (#478).
Expand Down
1 change: 0 additions & 1 deletion doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ The FlowProject
.. autosummary::

FlowProject.ALIASES
FlowProject.add_operation
FlowProject.completed_operations
FlowProject.get_job_status
FlowProject.label
Expand Down
85 changes: 14 additions & 71 deletions flow/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import cloudpickle
import jinja2
import signac
from deprecation import deprecated
from jinja2 import TemplateNotFound as Jinja2TemplateNotFound
from signac.contrib.filterparse import parse_filter_arg
from tqdm.auto import tqdm
Expand Down Expand Up @@ -62,6 +63,7 @@
switch_to_directory,
)
from .util.translate import abbreviate, shorten
from .version import __version__

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -3927,75 +3929,18 @@ def labels(self, job):
elif bool(label_value) is True:
yield label_name

def add_operation(self, name, cmd, pre=None, post=None, **kwargs):
r"""Add an operation to the workflow.
This method will add an instance of :class:`~.FlowOperation` to the
operations of this project.
.. seealso::
A Python function may be defined as an operation function directly
using the :meth:`~.operation` decorator.
Any FlowOperation is associated with a specific command, which should
be a function of :class:`~signac.contrib.job.Job`. The command (cmd)
can be stated as function, either by using str-substitution based on a
job's attributes, or by providing a unary callable, which expects an
instance of job as its first and only positional argument.
For example, if we wanted to define a command for a program called
'hello', which expects a job id as its first argument, we could
construct the following two equivalent operations:
.. code-block:: python
op = FlowOperation('hello', cmd='hello {job.id}')
op = FlowOperation('hello', cmd=lambda 'hello {}'.format(job.id))
Here are some more useful examples for str-substitutions:
.. code-block:: python
# Substitute job state point parameters:
op = FlowOperation('hello', cmd='cd {job.ws}; hello {job.sp.a}')
Preconditions (pre) and postconditions (post) can be used to trigger an
operation only when certain conditions are met. Conditions are unary
callables, which expect an instance of job as their first and only
positional argument and return either True or False.
An operation is considered "eligible" for execution when all
preconditions are met and when at least one of the postconditions is
not met. Preconditions are always met when the list of preconditions
is empty. Postconditions are never met when the list of postconditions
is empty.
Please note, eligibility in this contexts refers only to the workflow
pipeline and not to other contributing factors, such as whether the
job-operation is currently running or queued.
Parameters
----------
name : str
A unique identifier for this operation, which may be freely chosen.
cmd : str or callable
The command to execute operation; should be a function of job.
pre : sequence of callables
List of preconditions. (Default value = None)
post : sequence of callables
List of postconditions. (Default value = None)
\*\*kwargs
Keyword arguments passed as directives.
"""
if name in self.operations:
raise KeyError("An operation with this identifier is already added.")
op = self.operations[name] = FlowCmdOperation(cmd=cmd, pre=pre, post=post)
if name in self._groups:
raise KeyError("A group with this identifier already exists.")
self._groups[name] = FlowGroup(
name, operations={name: op}, operation_directives=dict(name=kwargs)
@deprecated(
deprecated_in="0.14",
removed_in="0.16",
current_version=__version__,
details="Method has been removed.",
)
def add_operation(*args, **kwargs): # noqa: D102
raise AttributeError(
"The add_operation() method was removed in version 0.14. "
"Please see https://docs.signac.io/en/latest/flow-project.html#defining-a-workflow "
"for instructions on how to define operations using the current API. This message "
"will be removed in version 0.16."
)

def completed_operations(self, job):
Expand Down Expand Up @@ -4070,8 +4015,6 @@ def operation(cls, func, name=None):
def hello(job):
print('Hello', job)
See also: :meth:`~.flow.FlowProject.add_operation`.
Parameters
----------
func : callable
Expand Down

0 comments on commit c51bbc2

Please sign in to comment.