Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrospective change of a result through filter operation #55

Open
fkroeber opened this issue Sep 16, 2024 · 0 comments
Open

Retrospective change of a result through filter operation #55

fkroeber opened this issue Sep 16, 2024 · 0 comments
Labels
bug 🐛 Something isn't working

Comments

@fkroeber
Copy link
Collaborator

Description

Due to the usage of filterer.values = utils.null_as_zero(filterer), recipe results are changed in a way that NaN values are converted to zero values if they are used in a subsequent filter operation.

Reproducible example

import json
import geopandas as gpd
import semantique as sq
from semantique.processor.core import QueryProcessor

# define recipe
recipe = sq.QueryRecipe()

recipe["res_a"] = sq.reflectance("s2_band02").\
    filter_time("before", sq.time_instant("2019-12-31"))

recipe["res_b"] = sq.reflectance("s2_band02").\
    filter_time("before", sq.time_instant("2019-12-31"))

recipe["res_c"] = sq.reflectance("s2_band02").\
    filter(sq.result("res_a"))

# create context
with open("files/mapping.json", "r") as file:
    mapping = sq.mapping.Semantique(json.load(file))

with open("files/layout_gtiff.json", "r") as file:
    dc = sq.datacube.GeotiffArchive(json.load(file), src = "files/layers_gtiff.zip")

space = sq.SpatialExtent(gpd.read_file("files/footprint.geojson"))
time = sq.TemporalExtent("2019-01-01", "2020-12-31")

context = {
    "datacube": dc, 
    "mapping": mapping,
    "space": space,
    "time": time,
    "crs": 3035, 
    "tz": "UTC", 
    "spatial_resolution": [-10, 10],
    "track_types": False
}

# execute recipe
fp = QueryProcessor.parse(recipe, **context)
response = fp.optimize().execute()

# evaluate equivalance of both results -> leads to False
# res_a contains zero values at places where res_b contains NaN values 
response["res_a"].equals(response["res_b"])

Expected behavior / Proposed solution

The filter operation should not alter previous results. A deep copy of the filterer needs to be created to ensure this. All occurrences of utils.null_as_zero() within semantique (find . -type f -exec grep -i 'null_as_zero' {} +) should be checked for if they incorrectly mutate an existing object instead of creating a new copy.

@fkroeber fkroeber added the bug 🐛 Something isn't working label Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant