Feature attrdict and model definition declutter (#719)

* Move YAML functionality to calliope.io (pre AttrDict YAML removal).
* Add YAML templating w/tests (pre AttrDict YAML removal).
* Simplify model initialisation to two main cases (pre-removal AttrDict YAML / inheritance functions
* Replace AttrDict and remove redundant 'inheritance' helper functionality
* Rename scenarios to model_definition and rearrange test location.
* Remove redundant AttrDict call in AttrDict.union(). 
* Update documentation: templates now in YAML section
* Remove references to the 'inheritance' helper function from the documentation
* Remove unused load function in attrdict
* Move to_yaml from AttrDict to io

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ### User-facing changes
 
+|changed| `template:` can now be used anywhere within YAML definition files, not just in the `nodes`, `techs` and `data_tables` sections.
+
 |changed| "An overview of the Calliope terminology" information admonition to remove self-references and improve understandability.
 Now also includes a visual depiction of how the different defined components connect together (#699).
 
@@ -31,6 +33,10 @@ This change has occurred to avoid confusion between data "sources" and model ene
 
 ### Internal changes
 
+|changed| Model definition reading is now defined in a single place (preprocess/model_definition.py).
+
+|changed| Moved YAML reading/importing functionality out of `AttrDict`. It is now part of our `io` functionality.
+
 |fixed| Avoided gurobi 12.0 incompatibility with pyomo by setting the lower bound to v6.8.2.
 
 ## 0.7.0.dev4 (2024-09-10)

docs/creating/data_tables.md

@@ -18,7 +18,6 @@ In brief it is:
 * [**drop**](#selecting-dimension-values-and-dropping-dimensions): dimensions to drop from your rows/columns, e.g., a "comment" row.
 * [**add_dims**](#adding-dimensions): dimensions to add to the table after loading it in, with the corresponding value(s) to assign to the dimension index.
 * [**rename_dims**](#renaming-dimensions-on-load): dimension names to map from those defined in the data table (e.g `time`) to those used in the Calliope model (e.g. `timesteps`).
-* [**template**](#using-a-template): Reference to a [template](templates.md) from which to inherit common configuration options.
 
 When we refer to "dimensions", we mean the sets over which data is indexed in the model: `nodes`, `techs`, `timesteps`, `carriers`, `costs`.
 In addition, when loading from file, there is the _required_ dimension `parameters`.
@@ -271,7 +270,7 @@ In this section we will show some examples of loading data and provide the equiv
         cost_storage_cap.data: 150
     ```
 
-    1. To limit repetition, we have defined [templates](templates.md) for our costs.
+    1. To limit repetition, we have defined [templates](yaml.md#reusing-definitions-through-templates) for our costs.
 
 !!! info "See also"
     Our [data table loading tutorial][loading-tabular-data] has more examples of loading tabular data into your model.

docs/creating/index.md

@@ -32,7 +32,7 @@ We distinguish between:
 - the model **definition** (your representation of a physical system in YAML).
 
 Model configuration is everything under the top-level YAML key [`config`](config.md).
-Model definition is everything else, under the top-level YAML keys [`parameters`](parameters.md), [`techs`](techs.md), [`nodes`](nodes.md), [`templates`](templates.md), and [`data_tables`](data_tables.md).
+Model definition is everything else, under the top-level YAML keys [`parameters`](parameters.md), [`techs`](techs.md), [`nodes`](nodes.md), and [`data_tables`](data_tables.md).
 
 It is possible to define alternatives to the model configuration/definition that you can refer to when you initialise your model.
 These are defined under the top-level YAML keys [`scenarios` and `overrides`](scenarios.md).
@@ -81,5 +81,4 @@ The rest of this section discusses everything you need to know to set up a model
 - An overview of [YAML as it is used in Calliope](yaml.md) - though this comes first here, you can also safely skip it and refer back to it as a reference as questions arise when you go through the model configuration and definition examples.
 - More details on the [model configuration](config.md).
 - The key parts of the model definition, first, the [technologies](techs.md), then, the [nodes](nodes.md), the locations in space where technologies can be placed.
-- How to use [technology and node templates](templates.md) to reduce repetition in the model definition.
 - Other important features to be aware of when defining your model: defining [indexed parameters](parameters.md), i.e. parameter which are not indexed over technologies and nodes, [loading tabular data](data_tables.md), and defining [scenarios and overrides](scenarios.md).

docs/creating/techs.md

@@ -15,7 +15,7 @@ This establishes the basic characteristics in the optimisation model (decision v
 ??? info "Sharing configuration with templates"
 
     To share definitions between technologies and/or nodes, you can use configuration templates (the `template` key).
-    This allows a technology/node to inherit definitions from [`template` definitions](templates.md).
+    This allows a technology/node to inherit definitions from [`template` definitions](yaml.md#reusing-definitions-through-templates).
     Note that `template` is different to setting a `base_tech`.
     Setting a base_tech does not entail any configuration options being inherited;
     `base_tech` is only used when building the optimisation problem (i.e., in the `math`).