Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1653121: Support some Timedelta aggregations on axis=0. #2248

Merged

Conversation

sfc-gh-mvashishtha
Copy link
Contributor

Fixes SNOW-1653121

Test and support aggregation on axis=0. We still raise NotImplementedError if:

  1. the aggregation requires concatenating a frame with timedelta types
  2. the aggregation requires transposing a row containing a timedelta type and other types.

To facilitate this change, introduce the class SnowparkPandasAggFunc, which represents a Snowpark pandas aggregation, together with the bool preserves_snowpark_pandas_type. We can use SnowparkPandasAggFunc instead of the separate mappings GROUPBY_AGG_PRESERVES_SNOWPARK_PANDAS_TYPE and GROUPBY_AGG_WITH_NONE_SNOWPARK_PANDAS_TYPES.

This change also fixes the bug that timedelta aggregations like mean would produce the wrong type (and the wrong result) by truncating the float result if preserves_snowpark_pandas_type.

@sfc-gh-mvashishtha sfc-gh-mvashishtha added the NO-PANDAS-CHANGEDOC-UPDATES This PR does not update Snowpark pandas docs label Sep 6, 2024
@sfc-gh-mvashishtha sfc-gh-mvashishtha marked this pull request as ready for review September 6, 2024 21:44
@sfc-gh-mvashishtha sfc-gh-mvashishtha requested a review from a team as a code owner September 6, 2024 21:44
@sfc-gh-mvashishtha
Copy link
Contributor Author

marking as draft till I fix SNOW-1654730

@sfc-gh-mvashishtha sfc-gh-mvashishtha marked this pull request as draft September 7, 2024 00:42
@sfc-gh-mvashishtha sfc-gh-mvashishtha marked this pull request as ready for review September 13, 2024 03:08
Copy link
Contributor

@sfc-gh-nkrishna sfc-gh-nkrishna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, LGTM!

Signed-off-by: sfc-gh-mvashishtha <[email protected]>
@sfc-gh-mvashishtha sfc-gh-mvashishtha enabled auto-merge (squash) September 13, 2024 20:19
@sfc-gh-mvashishtha sfc-gh-mvashishtha merged commit c7be18c into main Sep 13, 2024
35 checks passed
@sfc-gh-mvashishtha sfc-gh-mvashishtha deleted the mvashishtha/SNOW-1653121/timedelta-axis-0-aggregations branch September 13, 2024 20:55
@github-actions github-actions bot locked and limited conversation to collaborators Sep 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
NO-PANDAS-CHANGEDOC-UPDATES This PR does not update Snowpark pandas docs snowpark-pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants