Add details of skip mode

move workflow run modes description back to simulation-modes.rst

change simulation-modes to workflow-run-modes

src/7-to-8/major-changes/cylc-install.rst

@@ -125,7 +125,7 @@ See :ref:`the user guide <RemoteInit>` for more details.
 
       cylc config -i '[install][symlink dirs][<platform-name>]'
 
-   The recommended workaround is to use a "dummy" task that runs on the
+   The recommended workaround is to use a :term:`dummy task` that runs on the
    particular platform before any such mirror tasks in order to setup symlink
    directories, but without running anything.
 

src/glossary.rst

@@ -1767,3 +1767,17 @@ Glossary
       :term:`incomplete outputs <output completion>`, then the workflow is
       not complete and the scheduler will :term:`stall` pending manual intervention.
 
+   dummy task
+      A task which runs a trivially simple script such as ``sleep 1``,
+      ``exit 0`` or ``true``, or which uses alternative :ref:`task-run-modes`
+      or :ref:`workflow-run-modes` to run such a script or avoid running a
+      script at all.
+
+      .. important::
+
+         All tasks in a workflow run in dummy or simulation mode are dummy
+         tasks, but the methods outlined above will create dummy tasks in a
+         live workflow.
+
+   dummy mode
+      Modifies all tasks to be :ref:`workflow-run-modes.dummy` tasks.

src/user-guide/running-workflows/index.rst

@@ -12,7 +12,7 @@ Running Workflows
    tracking-task-state
    workflow-completion
    reflow
-   simulation-modes
+   workflow-run-modes
    scheduler-log-files
    dynamic-behaviour
    authentication-files

src/user-guide/running-workflows/workflow-run-modes.rst

@@ -1,33 +1,60 @@
-.. _SimulationMode:
+.. _workflow-run-modes:
 
-Simulation Modes
-----------------
+Workflow Run Modes
+------------------
 
 Cylc can run a workflow without running the real jobs, which may be
 long-running and resource-hungry.
 
-**Dummy mode** replaces real jobs with background ``sleep`` jobs on the
-scheduler host. This avoids :term:`job runner` directives that request compute
-resources for real workflow tasks, and it allows any workflow configuration to
-be run locally in dummy mode.
+This is useful when developing workflows. Tasks submitted
+using :ref:`workflow-run-modes.dummy` and
+:ref:`workflow-run-modes.simulation` can be used while
+developing workflows to understand how different outcomes
+and run lengths will affect the workflow.
+
+To apply a run mode to a workflow:
 
 .. code-block:: console
 
-   $ cylc play --mode=dummy <workflow-id>  # real dummy jobs
+   $ cylc play --mode=dummy <workflow-id>
+   $ cylc play --mode=simulation <workflow-id>
 
-.. note::
+.. admonition:: Limitations
 
-   All configured task ``script`` items including ``init-script`` are ignored
-   in dummy mode. If your ``init-script`` is required to run even local dummy
-   jobs, the relevant environment setup should be done elsewhere.
+   * Workflow run mode is recorded in the workflow run database. Cylc will not let you
+     *restart* a dummy mode workflow in live mode, or vice versa. Instead,
+     install a new instance of the workflow and run it from scratch in the new mode.
+   * Workflows run in Dummy and Simulation mode do not allow the run mode of
+     individual tasks to be overridden by :ref:`task-run-modes.skip`
 
 
-**Simulation mode** does not run real jobs at all, and does not generate job
-log files.
+.. _workflow-run-modes.dummy:
 
-.. code-block:: console
+Dummy Mode
+----------
+
+**Dummy mode** replaces real jobs with background jobs on the
+scheduler host which use ``sleep`` to simulate the run length according to
+the settings described for simulation mode.
+This avoids :term:`job runner` directives that request compute
+resources for real workflow tasks, and it allows any workflow configuration to
+be run locally in dummy mode.
+
+.. admonition:: Limitations
+
+   * Dummy mode can only be applied to a whole workflow.
+   * Dummy tasks run locally, so dummy mode does not test communication with remote
+     job platforms. However, it is easy to write a live-mode test workflow with
+     simple ``sleep 10`` tasks that submit to a remote platform.
 
-   $ cylc play --mode=simulation <workflow-id>  # no real jobs
+
+.. _workflow-run-modes.simulation:
+
+Simulation Mode
+---------------
+
+**Simulation mode** does not run real jobs, and does not generate job
+log files.
 
 
 Simulated Run Length
@@ -38,19 +65,17 @@ changed with :cylc:conf:`[runtime][<namespace>][simulation]default run length`.
 
 If :cylc:conf:`[runtime][<namespace>]execution time limit` and
 :cylc:conf:`[runtime][<namespace>][simulation]speedup factor` are both set,
-run length is computed by dividing the time limit by the speedup factor.
-
+run length is computed by dividing the
+:cylc:conf:`[runtime][<namespace>]execution time limit` by
+:cylc:conf:`[runtime][<namespace>][simulation]` speedup factor.
 
 Simulated Failure
 ^^^^^^^^^^^^^^^^^
 
-Tasks always complete custom outputs, by default they succeed.
+Tasks always complete all custom outputs, by default they succeed.
 
-.. warning::
-
-   In simulation mode a succeeded output may not imply that
-   submitted and/or started outputs are generated - so you will not
-   be able to test graph pathways such as ``task:started => do_something``.
+   If you want to test individual workflow pathways, use
+   :ref:`skip mode <task-run-modes.skip>`.
 
 You can set some or all instances of a task to fail using
 :cylc:conf:`[runtime][<namespace>][simulation]fail cycle points`.
@@ -81,21 +106,11 @@ whether your failure handling works correctly:
           fail try 1 only = false
           fail cycle points = 2022-01-01T03:00Z
 
-Limitations
-^^^^^^^^^^^
-
-Dummy tasks run locally, so dummy mode does not test communication with remote
-job platforms. However, it is easy to write a live-mode test workflow with
-simple ``sleep 10`` tasks that submit to a remote platform.
-
-Alternate path branching is difficult to simulate effectively. You can
-configure certain tasks to fail via
-:cylc:conf:`[runtime][<namespace>][simulation]`, but all branches based
-on mutually exclusive custom outputs will run because all custom outputs get
-artificially completed in dummy mode and in simulation mode.
-
-.. note::
+.. admonition:: Limitations
 
-   Run mode is recorded in the workflow run database. Cylc will not let you
-   *restart* a dummy mode workflow in live mode, or vice versa. Instead,
-   install a new instance of the workflow and run it from scratch in the new mode.
+   * Simulation mode can only be applied to a whole workflow.
+   * Alternate path branching is difficult to simulate effectively. You can
+     configure certain tasks to fail via
+     :cylc:conf:`[runtime][<namespace>][simulation]`, but all branches based
+     on mutually exclusive custom outputs will run because all custom outputs get
+     artificially completed in dummy mode and in simulation mode.

src/user-guide/task-implementation/index.rst

@@ -16,3 +16,4 @@ job fails and the task is re-run).
    job-scripts
    job-submission
    ssh-job-management
+   task-run-modes

src/user-guide/task-implementation/task-run-modes.rst

@@ -0,0 +1,112 @@
+.. _task-run-modes:
+
+Task Run Modes
+==============
+
+.. versionadded:: 8.4.0
+
+   * Before 8.4.0 run modes could only be set for the workflow, not
+     for individual tasks.
+   * Skip mode.
+
+
+As well as the default task implementation ("live" mode)
+other implementations can be selected:
+
+* By setting the scheduler run mode as an argument to the
+  ``cylc play --mode`` command (:ref:`workflow-run-modes.simulation`
+  and :ref:`workflow-run-modes.dummy`).
+* By setting :cylc:conf:`[runtime][<namespace>]run mode` for
+  "skip" mode.
+
+.. note::
+
+   The simulation and dummy workflow run modes ignore the
+   :cylc:conf:`[runtime][<namespace>]run mode` setting.
+
+
+.. _task-run-modes.skip:
+
+Skip Mode
+---------
+
+Skip mode is designed as an aid to workflow control:
+
+* Allows creation of :term:`dummy tasks<dummy task>` as part of workflow
+  design.
+* Allows skipping of tasks in a running workflow using 
+  ``cylc broadcast -s 'run mode = skip'``.
+
+Skip mode allows the user to specify which task outputs
+will be emitted using :cylc:conf:`[runtime][<namespace>][skip]outputs`.
+
+By default task event handlers are disabled by skip mode, but they
+can be enabled using
+:cylc:conf:`[runtime][<namespace>][skip]disable task event handlers`.
+
+Set task to succeeded
+^^^^^^^^^^^^^^^^^^^^^
+
+.. admonition:: Scenario
+
+   We don't need a task to run, but want to set it to succeed.
+
+Broadcast :cylc:conf:`[runtime][<namespace>]run mode` setting the
+value to ``skip``.
+
+Create a Graph Control Task
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. admonition:: Scenario
+
+   We have a large family to large family trigger.
+
+   .. code-block:: cylc
+
+      [task parameters]
+          x = 1..N   # Where N is big.
+      [scheduling]
+          [[graph]]
+              R1 = FAMILY:succeed-all => FAMILY2
+      [runtime]
+         [[FAMILY, FAMILY2]]
+         [[task1<x>]]
+            inherit = FAMILY
+         [[task2<x>]]
+            inherit = FAMILY2
+
+   If we increase N, the number of dependencies being tracked
+   by the scheduler is N^2.
+
+Adding a skip mode task between the families:
+
+.. code-block:: cylc
+
+   FAMILY:succeed-all => skiptask => FAMILY2
+
+Will reduce the number of dependencies to 2N.
+
+Parameter Exclusion
+^^^^^^^^^^^^^^^^^^^
+
+.. admonition:: Scenario
+
+   We want to skip a small number of tasks from a parameterized
+   group of tasks:
+
+   .. code-block:: cylc
+
+      [task parameters]
+          # House number 13 doesn't actually exist on this street...
+          house_number = 1..20
+
+We can use skip mode to make sure that a parameter task always
+succeeds without running anything:
+
+.. code-block:: cylc
+
+   [runtime]
+       [[post parcel<house_number>]]
+           script = send letter
+       [[post parcel<house_number=13>]]
+           run mode = skip

src/user-guide/troubleshooting.rst

@@ -309,12 +309,12 @@ logged a file in this directory:
 
 If you need to access files from a remote platform (e.g. 2-stage ``fcm_make``),
 ensure that a task has submitted to it before you do so. If needed you can use
-a blank "dummy" task to ensure that remote installation is completed *before*
+a blank "non-functional" task to ensure that remote installation is completed *before*
 you run any tasks which require this e.g:
 
 .. code-block:: cylc-graph
 
-   dummy => fetch_files
+   non_functional => fetch_files
 
 
 Conda / Mamba environment activation fails

src/user-guide/writing-workflows/runtime.rst

@@ -468,7 +468,7 @@ An implicit task is one that appears in the graph but is not defined under
 
 Depending on the value of :cylc:conf:`flow.cylc[scheduler]allow implicit tasks`,
 Cylc can automatically create default task definitions for these, to submit
-local dummy jobs that just return the standard job status messages.
+dummy jobs that just return the standard job status messages.
 
 Implicit tasks can be used to mock up functional workflows very quickly. A
 default ``script`` can be added to the root family, e.g. to slow job execution

src/user-guide/writing-workflows/scheduling.rst

@@ -1287,9 +1287,10 @@ number of members of ``FAM1`` and ``M`` is the number of members of ``FAM2``.
 This can result in high memory use as the number of family members grows.
 
 You can greatly reduce the number of dependencies generated here by putting
-dummy tasks in the graph to represent the state of the upstream family. For
-example, if ``FAM2`` should trigger off any member of ``FAM1`` succeeding you
-can use a dummy task ``FAM1_done`` like this:
+:term:`dummy tasks<dummy task>` in the graph to represent the state of the 
+upstream family. For example, if ``FAM2`` should trigger off any member of
+``FAM1`` succeeding you can use a :term:`dummy tasks<dummy task>`
+``FAM1_done`` like this:
 
 .. code-block:: cylc
 
@@ -1301,12 +1302,18 @@ can use a dummy task ``FAM1_done`` like this:
    [runtime]
    # ...
        [[FAM1_done]]
-           script = true
+           run mode = skip
    # ...
 
 This :term:`graph` generates only ``N + M`` dependencies, which takes
 significantly less memory and CPU to store and evaluate.
 
+.. note::
+
+   By setting ``run mode = skip`` the task will be follow the
+   :ref:`task-run-modes.skip` submission pathway, which
+   is significantly more efficient than a conventional task.
+
 
 .. _InterCyclePointTriggers: