Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to specify equality predicate for compare_values and provide a more sensible default (IS) #387

Merged
merged 2 commits into from
Aug 12, 2024

Conversation

GabrielKS
Copy link
Contributor

Previously, at the leaves of compare_values's recursion tree, it would mostly test equality using the == operator, sometimes the Base.isequal function. This PR lets a sophisticated user specify a custom equality predicate match_fn to override that behavior, such as to test a more approximate equality on a system when it is exported to a certain format and reimported. It also standardizes the default match_fn to

isequivalent(x, y) = isequal(x, y) || (x == y)

, which is true for NaN, NaN (unlike ==) and for -0.0, 0.0 (unlike isequal). Is that a violation of SemVer?

@GabrielKS GabrielKS requested review from daniel-thom and jd-lara July 19, 2024 03:09
@GabrielKS GabrielKS changed the title Add option to specify equality predicate for compare_values and provide a more sensible default Add option to specify equality predicate for compare_values and provide a more sensible default (IS) Jul 19, 2024
Copy link

codecov bot commented Jul 19, 2024

Codecov Report

Attention: Patch coverage is 86.04651% with 6 lines in your changes missing coverage. Please review.

Project coverage is 78.28%. Comparing base (fb1cf85) to head (295e365).

Files Patch % Lines
src/utils/utils.jl 86.66% 4 Missing ⚠️
src/in_memory_time_series_storage.jl 0.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #387      +/-   ##
==========================================
+ Coverage   78.20%   78.28%   +0.08%     
==========================================
  Files          71       71              
  Lines        5473     5484      +11     
==========================================
+ Hits         4280     4293      +13     
+ Misses       1193     1191       -2     
Flag Coverage Δ
unittests 78.28% <86.04%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/common.jl 60.00% <100.00%> (+10.00%) ⬆️
src/components.jl 86.39% <100.00%> (ø)
src/hdf5_time_series_storage.jl 96.32% <100.00%> (+0.01%) ⬆️
src/internal.jl 84.37% <100.00%> (ø)
src/supplemental_attribute_associations.jl 93.28% <100.00%> (+0.04%) ⬆️
src/system_data.jl 91.28% <100.00%> (ø)
src/time_series_manager.jl 95.86% <100.00%> (ø)
src/time_series_metadata_store.jl 93.49% <100.00%> (+0.01%) ⬆️
src/in_memory_time_series_storage.jl 69.23% <0.00%> (-0.77%) ⬇️
src/utils/utils.jl 63.00% <86.66%> (+1.77%) ⬆️

Copy link
Member

@jd-lara jd-lara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with this PR but I don't understand why is breaking PSY. We need to make sure this change is backwards compatible.

@GabrielKS GabrielKS force-pushed the gks/compare_values_match_fn branch from 3a397f3 to 295e365 Compare August 6, 2024 03:49
@GabrielKS
Copy link
Contributor Author

Okay, the interface should be backwards compatible now.

return compare_values(x, y; compare_uuids = compare_uuids,
exclude = union(exclude, [COMPARE_VALUES_SENTINEL]))
end
exclude = setdiff(exclude, [COMPARE_VALUES_SENTINEL])
Copy link
Contributor Author

@GabrielKS GabrielKS Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach is a little hacky but I am convinced something along these lines is the least bad way to accomplish the task without changing the interface at all (and it's temporary)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ask me what I learned about Core.kwcall today)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol ...

@GabrielKS GabrielKS requested a review from jd-lara August 6, 2024 03:53
@@ -1040,6 +1040,7 @@ function _try_time_series_metadata_by_full_params(
end

function compare_values(
match_fn::Union{Function, Nothing},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, what is supposed to happen here is a user passes a nothing?

Copy link
Contributor Author

@GabrielKS GabrielKS Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't just do

compare_values(x, y; kwargs...) = compare_values(isequivalent, x, y; kwargs...)

because we need to be able to keep track of the fact that the two-argument version was initially called so that an old two-argument version in PSY can be called where relevant (I think this was the backwards compatibility issue with my first attempt). So we do

compare_values(x, y; kwargs...) = compare_values(nothing, x, y; kwargs...)

and define that nothing will eventually default to isequivalent if it isn't intercepted by a two-argument version. This is documented here:

case of the recursion. If `nothing` or not specified, the default implementation uses

compare_values(x, y; kwargs...) = compare_values(nothing, x, y; kwargs...)

# Get the default match_fn if necessary. Only call when you know you're done recursing
_fetch_match_fn(match_fn::Function) = match_fn
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite a complicated workaround... I understand it avoids code repetition but it makes the code harder to read

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would

(isnothing(match_fn) ? isequivalent : match_fn)(a, b)

be more readable than

_fetch_match_fn(match_fn)(a, b)

?

Copy link
Member

@jd-lara jd-lara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks good to me as it is. But I need you to document this function and its use somewhere in the docs because it isn't obvious what to use it for.

@GabrielKS
Copy link
Contributor Author

The PR looks good to me as it is. But I need you to document this function and its use somewhere in the docs because it isn't obvious what to use it for.

There is a docstring at

Recursively compares struct values. Prints all mismatched values to stdout.
which will end up here, are you talking about something more than that?

@jd-lara jd-lara merged commit dd48d53 into main Aug 12, 2024
10 checks passed
compare_uuids = false,
exclude = Set{Symbol}(),
) where {T}
function compare_values(match_fn::Union{Function, Nothing}, x::T, y::U;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GabrielKS In what cases do we pass different types to this function? How can two values that are of different types be considered equivalent?

Copy link
Contributor Author

@GabrielKS GabrielKS Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want for instance isequivalent(0, 0.0), both because it makes sense and for backwards compatibility. In a broader sense we want to impose minimal constraints on how match_fn works, if it wants to specify that things of different type don't match then it is free to do so (and the default does generally do this).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the purpose of compare_values to "Recursively compares struct values."? We should always be comparing identical types. Are we ever calling it to test integers and floats?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants