Brainstorming on the migration of Nipype 1.0 interfaces (to Pydra)

After some playing around the input/output specs etc. during #3124, it seems that migrating the interfaces is a quite large undertaking.

Problem statement

Migrating the existing interfaces into Python callables. Although defining the __call__() method could work out, it would be desirable to make these ported functions more lightweight (and hopefully dropping many dependencies).

Input specs

Input specs seem relatively easy to port:

• The mandatory metadata indicates positional arguments
• The usedefault metadata indicates when the input should be converted to a positional or a keyword argument.
• The position metadata could be passed in as a dictionary.
• Trait types can be easily used to generate Python type-annotations.
• Filename generating metadata (name_{template,source}, genfile) is probably not hard to translate to keyword arguments with naming templates, or integrated with the ports for _parse_inputs()+_format_arg().
• Other behaviors:
  • _parse_inputs()+_format_arg(): will require a port, although it does not seem excessively hard. This could use a better design.
  • exists: we might just go ahead without this one.
  • traits.List/traits.Range: min/max etc could probably be dropped.
  • desc: with #3124, these will be easy to integrate into the docstrings.

Output Specs

These seem trickier - there is generally very little metadata associated with the outputs. Moreover, the major issue is how to port the aggregate_outputs()+_list_outputs() functioning.

• aggregate_outputs() seems easier to implement, as very few interfaces override it
• _list_outputs() is a lot harder to port, as most of the interfaces use it. One possibility, which would substantially improve Nipype 1.0 in the worst-case scenario, would be to add more metadata to the output specs (e.g., glob and re to File and Directory so that they can collect patterns, and templated default values to build up names; or stdout/err metadata to trigger parsing outputs). An early implementation of these new metadata could just point to functions (e.g., gen_filename())

_run_interface()

This should be fairly easy for CommandLine interfaces, but could be tricky for pure python interfaces.

Satra's thoughts

• as of a few versions back in nipype, we created an option for interfaces to have minimal representation (just inputspec and outputspec). those should be the easiest to convert. 
• the interfaces that overwrite parse (e.g., SPM, ANTs), format arg, aggregate and list outputs are going to be the hardest to do. these are also the hardest to convert to CWL/Boutiques
• our use of dataclasses in pydra is also going to be a bit cumbersome. python's implementation of limitations in dataclasses makes them harder to work with (they don't do dynamic stuff with inheritance well).
• perhaps we can still have a nipype BaseInterface class derived from the pydra base tasks, but one that allow for us to move code more easily
• pydra does not do any type checking at this point (other than File type for hashing). to regain that functionality we would have to do additional bits. i have always thought that these would be extensions in pydra that can be enabled or disabled for optimization.