Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydro horizons missing from temporal causes tab file to disappear #1102

Open
milindsmart opened this issue Feb 27, 2024 · 0 comments
Open

Hydro horizons missing from temporal causes tab file to disappear #1102

milindsmart opened this issue Feb 27, 2024 · 0 comments

Comments

@milindsmart
Copy link
Contributor

Relevant to #819 and ultimately #6 .

gen_hydro projects are linked to horizons in inputs_project_hydro_operational_characteristics. The way hydro data is read from the database is to pick the intersection of these horizons, with those specified in the inputs_temporal_horizons_timepoints table :

-- Limit to bt-horizons from this temporal scenario ID
JOIN (
SELECT DISTINCT balancing_type_horizon, horizon
FROM inputs_temporal_horizon_timepoints
WHERE temporal_scenario_id = {subscenarios.TEMPORAL_SCENARIO_ID}
AND subproblem_id = {subproblem}
AND stage_id = {stage}
) as hrz_tbl
ON (
balancing_type_project = balancing_type_horizon
AND prj_tbl.horizon = hrz_tbl.horizon
)

When horizons are missing from the temporal table, the intersection is also the null set and therefore the tab files aren't even written:

# Only write if we have data
data_list = [row for row in data.fetchall()]
if data_list:
with open(out_file, append_mode, newline="") as f:

This results in a file not found error:

ERROR:root:[Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' and FileNotFoundError: [Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' .

I see three equally valid ways to handle this:

  1. Make sure all empty files are written, with (perhaps?) the result that the hydro projects are never simulated. This aligns with existing practice of silently dropping mismatched horizons.
  2. Make mismatched horizons an error, and use LEFT JOIN instead of INNER JOIN while fetching the hydro opchar data. This entails a lot of changes throughout the test suite.
  3. Keep existing behaviour as by-design.

I am in the process of writing validation to catch this case, but before that wanted to understand if current operational behaviour is intended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant