Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve predictability of DataQuery, DataID, and dependency tree #3018

Draft
wants to merge 22 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
5db7d99
Cleanup dependency tree tests
djhoese Dec 3, 2024
66b714a
Update dep tree tests to be more realistic
djhoese Dec 3, 2024
cab16b9
Convert combine metadata tests to pytest
djhoese Dec 3, 2024
1e88269
Fix inconsistency with DataID "resolution" transitive property
djhoese Dec 7, 2024
dd46a4e
Convert dataquery tests to pytest
djhoese Dec 9, 2024
251524b
Fix DataQuery equality to require query keys to match
djhoese Dec 9, 2024
ef07c55
Fix inconsistency with DataID "resolution" transitive property
djhoese Dec 9, 2024
438de2c
Split data query tests to be more explicit
djhoese Dec 9, 2024
ce01aa6
Cleanup satpy internals documentation regarding DataQuery equality
djhoese Dec 9, 2024
ed2867a
Change DataQuery equality to require all query keys to be equal
djhoese Dec 13, 2024
d223f10
Add test querying for a wavelength on no-wavelength DataIDs
djhoese Dec 16, 2024
aba5dc5
Merge combine times test cases
djhoese Dec 16, 2024
24a1068
Refactor ID key types to separate module
djhoese Dec 16, 2024
2085b50
Extract ID update with query function
djhoese Dec 16, 2024
6afc26d
Remove unused _match_query_value method
djhoese Dec 16, 2024
633802f
Add shared_key option to DataQuery equality checks
djhoese Dec 17, 2024
0f77b4e
Refactor get_keys_from_config for simpler conditionals
djhoese Dec 17, 2024
67cb550
Attempt to refactor DataQuery equality
djhoese Dec 17, 2024
52469d4
Refactor DataQuery equality checks
djhoese Dec 17, 2024
68880a4
Another try at making CodeScene happy with get_keys_from_config
djhoese Dec 17, 2024
de36fb8
Remove duplicated test code in test_satpy_cf_nc.py
djhoese Dec 17, 2024
c45ed8d
Remove accidental holoviews import in dependency tree
djhoese Dec 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Remove accidental holoviews import in dependency tree
djhoese committed Dec 17, 2024

Verified

This commit was signed with the committer’s verified signature.
djhoese David Hoese
commit c45ed8d831284405826d53bc919b5f09ccfa2d00
8 changes: 5 additions & 3 deletions satpy/dependency_tree.py
Original file line number Diff line number Diff line change
@@ -20,17 +20,19 @@
from __future__ import annotations

import warnings
from typing import Container, Iterable, Optional
from typing import TYPE_CHECKING, Container, Iterable, Optional

import numpy as np
from holoviews.core.options import Compositor

from satpy import DataID, DataQuery, DatasetDict
from satpy.dataset import ModifierTuple, create_filtered_query
from satpy.dataset.data_dict import TooManyResults, get_key
from satpy.dataset.dataid import update_id_with_query
from satpy.node import EMPTY_LEAF_NAME, LOG, CompositorNode, MissingDependencies, Node, ReaderNode

if TYPE_CHECKING:
from satpy.composites import CompositeBase

Check warning on line 34 in satpy/dependency_tree.py

Codecov / codecov/patch

satpy/dependency_tree.py#L34

Added line #L34 was not covered by tests


class Tree:
"""A tree implementation."""
@@ -511,23 +513,23 @@

return self._get_compositor_by_name(key)

def _get_compositor_by_name(self, key: DataQuery) -> Compositor | None:
def _get_compositor_by_name(self, key: DataQuery) -> CompositeBase | None:
name_query = DataQuery(name=key["name"])
for sensor_name in sorted(self.compositors):
sensor_data_dict = self.compositors[sensor_name]
# get all IDs that have the minimum "distance" for our composite name
all_comp_ids = sensor_data_dict.get_key(name_query, num_results=0)
if len(all_comp_ids) == 0:
continue

Check warning on line 523 in satpy/dependency_tree.py

Codecov / codecov/patch

satpy/dependency_tree.py#L523

Added line #L523 was not covered by tests

# Filter to those that don't disagree with the original query
matching_comp_ids = key.filter_dataids(all_comp_ids, shared_keys=True)
if len(matching_comp_ids) > 1:
warnings.warn("Multiple compositors matching {name_query} to create {key} variant. "

Check warning on line 528 in satpy/dependency_tree.py

Codecov / codecov/patch

satpy/dependency_tree.py#L528

Added line #L528 was not covered by tests
"Going to use the name-only 'base' compositor definition.")
matching_comp_ids = matching_comp_ids[:1]

Check warning on line 530 in satpy/dependency_tree.py

Codecov / codecov/patch

satpy/dependency_tree.py#L530

Added line #L530 was not covered by tests
if len(matching_comp_ids) != 1:
raise KeyError("Can't find compositor {key['name']} by name only.")

Check warning on line 532 in satpy/dependency_tree.py

Codecov / codecov/patch

satpy/dependency_tree.py#L532

Added line #L532 was not covered by tests
comp_id = matching_comp_ids[0]
# should use the "short-circuit" path and find the exact name-only compositor by DataID
return sensor_data_dict[comp_id]
Loading