Honour always tag even when step selection is in use.

[JSKenyon]

Currently, invoking stimela run recipe.yml -s dependent_step will fail if dependent_step depends on the outputs of a preceding step. The user can specify stimela run recipe.yml -s dependency_step -s dependent_step, but this can become tedious and may require more in-depth recipe knowledge. The natural mechanism for handling this case is the always tag i.e. steps tagged as always are integral to recipe functionality and as such should always be run. Currently, -s will completely ignore any tags. This PR instead augments the selected steps with the always steps. One downside of this approach is that the always steps may be run even when they are unnecessary i.e. if a users only runs steps prior to the always step, they cannot be a requirement. @o-smirnov thoughts?

[JSKenyon]

My feeling is that always should be reserved for steps which can be run independently of any steps outside the always group. An example of this would be the ms-info step of the parrot pipeline. It should always be run as it is necessary for other steps. The alternative is to support a step_requires field but that may become quite involved.

[o-smirnov]

Thanks @JSKenyon I like that idea. Do we need an extra command-line option to explicitly skip (--skip/-k) a step tagged with always?

P.S. I've already been thinking about auto-dependencies for a future version: #254 (comment)

[JSKenyon]

Thanks @JSKenyon I like that idea. Do we need an extra command-line option to explicitly skip (--skip/-k) a step tagged with always?

I have added a simple (--skip-step/-k) command for explicitly disabling specific steps. This should also work when you want to skip a step inside a tagged group.

[o-smirnov]

Should -k/--skip also not apply inside a range selected with -s FROM:TO?

[JSKenyon]

I have decided to take a stab at rewriting the restrict_steps function. Will resubmit for review when I am done.

[JSKenyon]

@o-smirnov I have rewritten some of the restrict logic and I believe that it is now a little clearer. This probably results in slight changes in behaviour but I think the new approach is more powerful than the old. Specifically, -t and -s can now be combined e.g. stimela run recipe.yml -t some_steps -s extra_step. This will run all the tagged steps in addition to the extra step. Steps tagged as always will always run unless explicitly skipped with -k. -k now supports the same behaviour as -s for ranges of steps. Let me know what you think.

[o-smirnov]

Looks good and is extremely useful, thanks @JSKenyon!