prefect caching misleading

[siligam]

The following error is misleading and potentially a side effect of prefect caching. 'thetao' units in both model and tables are 'degC'. The units set on rule for this variable is indeed 'degC' (rule.data_request_variable.unit) but in the error message some other units appear. Those units are for 'fgco2' variable. I guess prefect caching is unable to differentiate rules.

ValueError("Cannot convert variables:\n    incompatible units for variable 'thetao': Cannot convert from 'degree_Celsius' ([temperature]) to 'kilogram / meter ** 2 / second' ([mass] / [length] ** 2 / [time])")))

To reproduce this error:
use examples/sample.yaml
branch: feat/dimensionless-units

[siligam]

Every thing works fine after turning off cache.

diff --git a/src/pymorize/pipeline.py b/src/pymorize/pipeline.py
index 879be8d..1fa25f8 100644
--- a/src/pymorize/pipeline.py
+++ b/src/pymorize/pipeline.py
@@ -65,8 +65,8 @@ class Pipeline:
                 Task(
                     fn=step,
                     # cache_key_fn=generate_cache_key,
-                    cache_expiration=self._cache_expiration,
-                    cache_policy=TASK_SOURCE + INPUTS,
+                    # cache_expiration=self._cache_expiration,
+                    # cache_policy=TASK_SOURCE + INPUTS,
                 )
             )

The log file shows:

grep Converting slurm-13789839.out
Converting units: (thetao -> thetao) degC -> degC
Converting units: (CO2f -> fgco2) mmolC/m2/d -> kg m-2 s-1
Converting units: (so -> so) psu -> g/kg (0.001)

[pgierz]

Hmm...The cache_policy takes care of looking at the inputs, and each step gets one of the Rule objects as an input, so I am unsure why the cache is not unambiguously set up for each one separately.

I'll have a look. @siligam, you can focus on #55 for right now.