-
Notifications
You must be signed in to change notification settings - Fork 116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SNOW-1620436 Improved to_datetime to handle all local input cases and improved time series notebook tests #2184
Conversation
156b732
to
ce50b6f
Compare
…me series notebook tests
ce50b6f
to
0c28a0c
Compare
@@ -1784,8 +1781,28 @@ def to_datetime( | |||
# TODO: SNOW-1063345: Modin upgrade - modin.pandas functions in general.py | |||
raise_if_native_pandas_objects(arg) | |||
|
|||
if arg is None: | |||
return None # same as pandas | |||
if not isinstance(arg, (DataFrame, Series, pd.Index)): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
match with what we do today for DatetimeIndex and timedelta.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor nit, but LGTM!
Are we sure we don't want to break up this PR? |
It's only two changes on src:
Others are just test updated based on the above two. I did include most of tests from the pandas time series doc in our notebook in this PR. But should be fine if something can be improved there, we can use followup PRs |
This makes sense. |
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-1620436
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.
This pull request includes several changes to improve the handling of datetime conversions and related tests in the Snowpark Modin integration. The most important changes include enhancements to the
to_datetime
function, updates to internal utility messages, and adjustments to test cases for better accuracy and performance.Improvements to
to_datetime
function:pd.to_datetime
to handle all local input cases. (CHANGELOG.md
)to_datetime
to handle non-Modin objects usingpandas.to_datetime
and to return appropriate Modin objects. (src/snowflake/snowpark/modin/pandas/general.py
) [1] [2]to_datetime
. (src/snowflake/snowpark/modin/pandas/general.py
) [1] [2]Updates to internal utilities:
AUTO_FORMAT_WARNING_MSG
for consistent warning messages about Snowflake's automatic format detection. (src/snowflake/snowpark/modin/plugin/_internal/timestamp_utils.py
)src/snowflake/snowpark/modin/plugin/utils/warning_message.py
)Adjustments to test cases:
pd.Index
for datetime inputs to ensure consistency and accuracy in results. (tests/integ/modin/tools/test_to_datetime.py
) [1] [2] [3]to_datetime
. (tests/integ/modin/tools/test_to_datetime.py
) [1] [2] [3]These changes enhance the robustness and performance of datetime handling in the Snowpark Modin integration, ensuring better alignment with pandas behavior and more accurate test coverage.