Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add strict argument to audformat.utils.hash() #454

Merged
merged 3 commits into from
Jul 17, 2024
Merged

Add strict argument to audformat.utils.hash() #454

merged 3 commits into from
Jul 17, 2024

Conversation

hagenw
Copy link
Member

@hagenw hagenw commented Jul 15, 2024

Closes #447

Adds the strict argument to audformat.utils.hash().
If True it will take the order of rows and the names of index levels and columns into account. The returned hash is then identical to the one attached to parquet table files.

A hashing variant that takes into account the order of rows, name of columns and levels, and the data types was needed for calculating a hash for parquet tables as introduced in #419. Before, this was handled by a private method, independent of audformat.utils.hash().

image

Copy link

codecov bot commented Jul 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.0%. Comparing base (a8cd511) to head (7514c60).

Additional details and impacted files
Files Coverage Δ
audformat/core/table.py 100.0% <100.0%> (ø)
audformat/core/utils.py 100.0% <100.0%> (ø)

@hagenw hagenw requested a review from ChristianGeng July 15, 2024 10:15
@hagenw
Copy link
Member Author

hagenw commented Jul 15, 2024

include_order_and_names is verbose, but also slightly long as an argument name. strict might be a better choice?

Copy link
Member

@ChristianGeng ChristianGeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apiwise I think that having only one utils.hash as before and distinguishing calculations based on the include_order_and_names kwarg is nice.

I cannot do a technical review of the md5 algorithm though, so I am not attempting.
The test coverage is fairly extensive. This will help a lot in the discovery of implementation changes on the pandas side, or even in the unlikely case that sth changes in hashlib.md5.

I would not be passionate about the name of the keyword argument, be it "strict" or "include_order_and_names" as long as the motivation becomes clear. The code and the issue description contain already links to the pandas issue where this was discussed. I wonder whether links should even be extended to point to the audformat issues that led to the implementation as is (the pyarrow metdatata has, effectively clarifying why this was needed)?

Apart from that I think that this is good and can be approved.

@hagenw hagenw changed the title Add include_order_and_names argument to hash() Add strict argument to audformat.utils.hash() Jul 17, 2024
@hagenw
Copy link
Member Author

hagenw commented Jul 17, 2024

I added a reference to #419 in the description and renamed the argument to strict, thanks for the suggestions.

@hagenw hagenw merged commit 79d6246 into main Jul 17, 2024
10 checks passed
@hagenw hagenw deleted the hash-argument branch July 17, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide table dataframe hashing in public API?
2 participants