Proposal for operation hooks

[csadorf]

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

About

This is a proposal for a feature that would allow users to specify hook functions that are executed either before or after the execution of an operation.
The purpose is to simplify the execution of specific functions before and after the execution of a job-operation, such as logging or data upload functions.

A hook function takes an instance of JobOperation as first argument which allows to access the name of the operation, the job it operates on, the command, and optional execution directives.
For example:

#!python
def log_operation(op):
    log("Operation {} started.".format(op))

This log function could then be specified to be executed on operation start:

#!python

@Project.operation
@Project.hook.on_start(log_operation)
def hello(job):
    print('hello', job)

API

Hooks are in general defined as lists of functions, as part of the flow.hooks.Hooks class.

• on_start: Executed before a operation is executed.
• on_finish: Executed after a operation has completed (with or without error).
• on_success: Executed after a operation has completed without error.
• on_fail: Executed after a operation has completed with error.

Low-level API

The low-level API allows the installation of hooks on the project instance level via the manipulation of the hooks instance variable:

#!python
project = Project()
project.hooks.on_start.append(my_log_function)

And during the adding of operations, again, via a dictionary:

#!python
project.add_operation(
    name='hello',
    cmd='echo hello {job._id}',
    hooks={'on_start': [my_log_function]})

The hooks are executed in order of definition, where project-wide hooks are executed prior to operation-wide hooks.

Decorator API

Alternatively, hooks can be installed via the decorator API:

#!python
from flow import hook

@Project.hook.on_start(log_operation)
def my_op(job):
    pass

Note, that the hooks are only executed if the operation is executed through the FlowProject interface.

Configurational API

Finally, hooks can be installed directly in the signac configuration by specifying a function that installs the hooks during project initialization.

#!bash
# signac.rc

[flow]
hooks = my_hooks.install_hooks

The hooks config key is interpreted as list, multiple modules would be provided as a comma-separated list.
The my_hooks module must be directly importable and should have a install_hooks() function in the global namespace similar to this:

#!python
# my_hooks.py

def log_operation(op):
    log("Operation {} started.".format(op))

def install_hooks(project):
    project.hooks.on_start.append(log_operation)
    return project

The install_hooks() function should return the project argument so that we can use the following idiom: install_hooks(Project()).main().

The default hook installation function is called install_hooks so in this case the following configuration would be equivalent:

#!bash
[flow]
hooks = my_hooks

As a final example, assuming that the flow package had a module that would install hooks for the automatic generation of snapshots after execution, it would likely be installed like this:

#!bash
[flow]
hooks = flow.snapshots

[csadorf]

Original comment by Bradley Dice (Bitbucket: bdice, GitHub: bdice).

---

Good idea, and the API looks reasonable. It seems like we can usually do most customizations in flow by altering the FlowProject class, but in this case we have to work on an instance of the class. I'd have to think more about it to determine if this is a problem, and if so, whether it can be solved.

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

Yes, that is true, but that is actually intentional. My reasoning is that the class defines a specific workflow, while the instance is more strongly related to the actual execution. I see the hooks execution-related, less workflow related.

One edge case is the decorator API which behaves more like a class definition...

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

I think one thing we need to decide is whether those hooks are expected to always be executed, i.e., also when we just call the function, (that would be problematic for @cmd operations) or just when using FlowProject.run*().

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

Slightly updated the description and the API definition.

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

A prototype for the proposed API is implemented in the execution-hooks branch.

[csadorf]

Original comment by Bradley Dice (Bitbucket: bdice, GitHub: bdice).

---

If I define a hook, I want it to run any time I call the function, no matter if that's in a Python script or the shell interface for flow. This is why I'm confused by its separation from the class workflow into a specific instance. That sounds kind of annoying to me, I'd rather it stays as part of the class. Can you explain why that would be problematic for @cmd operations? Do you mean if the user called the operation outside of signac, just on the command line?

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

Well, right now, if you call a function foo() that happens to be a FlowProject operation it will just call it as a "normal function". So if you had a @cmd function like this:

@FlowProject.operaton
@flow.cmd
def foo(job):
    return 'echo foo {job._id}'

it will just print echo foo abc....

[csadorf]

Original comment by Vyas Ramasubramani (Bitbucket: vramasub, GitHub: vyasr).

---

I had the same reaction as Bradley; everything looks good to me aside from the class/instance question. I'll think on it a bit more.

[csadorf]

Original comment by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).

---

Project-wide hooks

As per offline discussion with @vyasr and @bdice, I'm going to update this proposal to promote a decorator-based API for project-wide hooks, similar to:

class Project(FlowProject):
    pass


@Project.hooks.on_start
def log_operation(op):
    print(op)

Hooks and cmd-operations

Furthermore, we will need to account for @cmd-decorated operations, like:

@Project.operation
@flow.cmd
def foo(job):
    return "echo hello {}".format(job)

by padding the execution in (submission-)scripts, with something like:

$ flow _exec_pre_hooks foo abc123
$ echo hello abc123
$ flow _exec_post_hooks foo abc123

Alternatively, we just disable hooks for cmd-operations or require them to be executed through the flow exec command.

[bdice]

This is resolved by #508.