Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

👌 IMPROVE: Set default extras for FromGroupSubmissionController #5

Open
mbercx opened this issue Sep 17, 2021 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@mbercx
Copy link
Member

mbercx commented Sep 17, 2021

Problem description

The current implementation of the submission controller relies on extras in the parent_group that uniquely define each node. This means that in order to use the submission controller, there is always some management of these extras involved. This means that in case e.g. new nodes are added to the parent_group, one must make sure that these extras aren't forgotten, adding extra overhead.

Desired change

I think we should consider having a default approach where the UUID of the node in the parent_group is used to check which nodes have already been submitted. This will require a bit of tweaking of the get_process_extra_projections and get_extra_unique_keys in the FromGroupSubmissionController class. Perhaps the latter method will return a reserved key, e.g. _parent_uuid, which means the former will return a projection that grabs the UUID instead of anything from the extras.

Other considerations

One thing that's definitely nice about the extras is that they are passed on in case the submitted work chains are then in turn inputs for a next submission controller. So it's very easy and fast to connect these different runs, especially if you branch at some point in your submission workflow. One way to preserve this would be to assign an _ancestor_uuid instead. If this key isn't present in the extras of the node in the parent_group, it is generated from the actual UUID of the node. Else it is just passed on.

@mbercx mbercx added the enhancement New feature or request label Sep 17, 2021
@mbercx
Copy link
Member Author

mbercx commented Sep 28, 2021

Additionally: why was it decided to make the get_extra_unique_keys a method that needs to be specified for the submission controller, instead of just an input argument?

I'm finding myself writing e.g. a generic PwRelaxSubmissionController, and then another subclass for this one that only specifies the get_extra_unique_keys method for a certain project. I could avoid writing that extra class if the extra keys were simply an input to the work chain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant