You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When RAIL is reading the yaml to set up the pipeline, it should check (at the very beginning!) if the stated output files already exist and refuse to overwrite them, unless flag OVERWRITE=True
The text was updated successfully, but these errors were encountered:
I want to clarify how this should behave with respect to "resume" mode, which I understand to be a flag one can set so the pipeline will "ignore stages if all of their output already exists" (described in "Initialize the pipeline" step of Pipe_Example.ipynb, and included as a value set in pipe_example.yml). This mode seems pretty similar to me.
If we add a flag a user can toggle when reading and running a pipeline, should this just completely override the value of resume mode specified in the pipeline's .yml when the OVERWRITE flag has been set to either true or false, but default to the pipeline's resume mode if we chose not to include overwrite as a keyword?
@aimalz I remember we spoke about this but only briefly - I was wondering if you have any thoughts.
Edit to add: maybe this just needs unit tests to confirm that this flag is actually working?
This might be a good place to also check if input files exist where they're expected to, and it might be related to resolving occurrences of stages needing their input files to be in src/rail.
When RAIL is reading the yaml to set up the pipeline, it should check (at the very beginning!) if the stated output files already exist and refuse to overwrite them, unless flag
OVERWRITE=True
The text was updated successfully, but these errors were encountered: