Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-1852934: Add support for DataFrame.map method #2754

Merged
merged 3 commits into from
Dec 13, 2024
Merged

SNOW-1852934: Add support for DataFrame.map method #2754

merged 3 commits into from
Dec 13, 2024
+33 −20

Conversation

sfc-gh-nkumar
Copy link
Contributor

@sfc-gh-nkumar sfc-gh-nkumar commented Dec 12, 2024 •
edited by sfc-gh-helmeleegy
Loading

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-1852934

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines

  3. Please describe how your code solves the related issue.

In pandas 2.1.0 DataFrame.applymap was renamed to DataFrame.map. We already have support for DataFrame.applymap, added redirect to support DataFrame.map as well.

@sfc-gh-nkumar sfc-gh-nkumar requested a review from a team as a code owner December 12, 2024 19:55
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1852934-frame-map branch from d365518 to 710f233 Compare December 12, 2024 20:53
sfc-gh-helmeleegy
sfc-gh-helmeleegy approved these changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@sfc-gh-helmeleegy sfc-gh-helmeleegy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

sfc-gh-jkew
sfc-gh-jkew requested changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@sfc-gh-jkew sfc-gh-jkew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extraneous comment it seems

tests/integ/modin/frame/test_applymap.py Outdated Show resolved Hide resolved
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1852934-frame-map branch from 653310a to 001bf50 Compare December 12, 2024 23:49
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1852934-frame-map branch 2 times, most recently from fec6641 to 1cdcf66 Compare December 13, 2024 20:49
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1852934-frame-map branch from 1cdcf66 to 887cb80 Compare December 13, 2024 22:28
sfc-gh-jkew
sfc-gh-jkew approved these changes Dec 13, 2024
View reviewed changes
Copy link
Contributor

@sfc-gh-jkew sfc-gh-jkew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. LGTM

@sfc-gh-nkumar sfc-gh-nkumar merged commit 9e418dc into main Dec 13, 2024
41 checks passed
@sfc-gh-nkumar sfc-gh-nkumar deleted the nkumar-SNOW-1852934-frame-map branch December 13, 2024 23:41
@github-actions github-actions bot locked and limited conversation to collaborators Dec 13, 2024
sfc-gh-jjiao
sfc-gh-jjiao reviewed Dec 13, 2024
View reviewed changes
src/snowflake/snowpark/modin/plugin/extensions/dataframe_overrides.py
def _map(self, func, na_action: str | None = None, **kwargs) -> DataFrame:
pass # pragma: no cover
def _map(self, func: PythonFuncType, na_action: str | None = None, **kwargs):
# TODO: SNOW-1063346: Modin upgrade - modin.pandas.DataFrame functions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this ticket SNOW-1063346 (https://snowflakecomputing.atlassian.net/browse/SNOW-1063346) is done. what's the TODO here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this line of comment is copied over and it is no longer valid since it is done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question. @sfc-gh-joshi might have insights into what work is left to remove these TODOs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sfc-gh-dpetersohn originally left these comments in the code to identify which functions have differing implementations from the upstream modin versions, but I ended up using the ticket to track moving these methods into dataframe_overrides.py from the old snowflake/snowpark/modin/pandas/dataframe.py file. We can remove the TODO when we remove the override definition (i.e. we match the upstream modin implementation), though this might not be possible or desirable in all cases.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sfc-gh-jjiao sfc-gh-jjiao sfc-gh-jjiao left review comments

@sfc-gh-joshi sfc-gh-joshi sfc-gh-joshi left review comments

@sfc-gh-helmeleegy sfc-gh-helmeleegy sfc-gh-helmeleegy approved these changes

@sfc-gh-jkew sfc-gh-jkew sfc-gh-jkew approved these changes

Assignees
No one assigned
Labels
snowpark-pandas
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sfc-gh-nkumar @sfc-gh-joshi @sfc-gh-jkew @sfc-gh-helmeleegy @sfc-gh-jjiao