I/E specification in SysSettings

[olejandro]

Additional records (representing i/e options) are generated for TIMES-NZ in #256 for the attributes that have i/e options specified in VT files. The discrepancy appears to be due to Veda behaviour described here (i.e. redefinition (via any tag in SysSettings) of interpolation options defined in Base/SubRES is ignored in Veda2.0).

Should the tool do the same or should we introduce a switch to control this behaviour?

[Antti-L]

Comment: I don't think it is due to that VEDA2 "things that is different", which is only related to using TFM_INS in SysSettings. The SysSettings facility for IE options has always been meant to provide default values, which should be overridden by any IE options explicitly defined by the user elsewhere. VEDA has thus always prevented the SysSettings IE defaults from overriding what has been defined in both VT or SubRes files. And in my view that is the only sensible convention: Values explicitly defined for specific attribute instances in VT or Subres files should not be overridden by those generic bulk definitions made in SysSettings. For regular scenarios this usually works automatically (because SysSettings is best placed before regular scenarios in the scenario order).

[olejandro]

Well, I am only quoting the official source. 🤷 I guess, it must be true what is written there... At least at some point.

Imho, using the transform tags, meant for data manipulation, to specify defaults is not a good design choice, as it overcomplicates things. Also, why should the placement of syssettings matter?

In any case, we could implement the same behaviour and use it by default, but allow turning it off. In my view, the main benefit of having the switch is that it will make it more explicit that there is an exception to the general data manipulation routine.

[Antti-L]

Being able to define default IE options like we can in SysSettings is in my view extremely valuable, and I don't see any way how this capability would be complicating things for the user. If you would take that away, it would be a fundamental regression for the users.

The NZ model does explicitly state the default nature by the sheet name: "Interpol_Extrapol_Defaults". So, it should be clear to anyone that they are meant to be defaults.

Of course the placement of SySsettings matters, like the placement of all regular scenarios exactly in the scenario order defined by the user is critical for reproducing any model run case. As you should know, GAMS overwrites parameter values defined earlier when the same parameter instances are defined in data statements defined later. And so, if SysSettings would be included as the last scenario, the defaults would not work for any regular scenarios. On the other hand, putting SysSettings as the first scenario of all model data would be ideal (then no special handling would be needed in VEDA for the default IEs), but it is not possible with VEDA, because VEDA does not have a separate file (or section) for defining all the domains first. The best placing is thus basically right after the Base and SubRes scenarios.

[Antti-L]

Imho, using the transform tags, meant for data manipulation, to specify defaults is not a good design choice

Looks like you may have adopted a meaning for TFM ("transform") tags somewhat different from VEDA. Under VEDA, the TFM_ tags are, in general, not doing any data manipulation which would change any existing data. They basically only generate new scenario-specific data into the scenario where the tags are, and do not change the data in other scenarios at all. As far as I can recall, with the exception of the cases where TS_Filter is being used, the existing data is left untouched in all other scenarios, just like they should be when generating the default IE options. However, if in xl2times the TFM tables are, in fact, causing some additional data manipulation, I understand it might cause consistency issues with VEDA models.

[olejandro]

@Antti-L it is probably just the language issue! Neither you nor I is a native english speaker which now and then seems to cause misunderstandings. 🤷