Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-1418523: make udf and sproc registration thread safe #2289

Merged
merged 29 commits into from
Sep 25, 2024
Merged

SNOW-1418523: make udf and sproc registration thread safe #2289

merged 29 commits into from
Sep 25, 2024

Conversation

sfc-gh-aalam
Copy link
Contributor

@sfc-gh-aalam sfc-gh-aalam commented Sep 12, 2024 •
edited
Loading

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-1663723

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.

  3. Please describe how your code solves the related issue.

    This PR makes UDF/Sproc related update to make calling, invoking, and registering UDXFs and sprocs thread-safe. Along with these changes, adding packages and imports is also made thread-safe in this PR. The change involves

    • Protecting read/write of Session._packages and Session._import_paths
    • Refactor registration for UDF/SP where we read _runtime_version_from_requirement once and use the value throughout the registration process.

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@sfc-gh-aalam sfc-gh-aalam changed the base branch from main to aalam-SNOW-1418523-concurrent-file-operations September 12, 2024 22:19
sfc-gh-aalam
sfc-gh-aalam commented Sep 13, 2024
View reviewed changes
src/snowflake/snowpark/_internal/udf_utils.py
) -> None:
if session is not None and session._runtime_version_from_requirement:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change is to ensure the same value for runtime_version is read from session througout udf/sp registration process.

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@sfc-gh-aalam sfc-gh-aalam marked this pull request as ready for review September 17, 2024 18:17
@sfc-gh-aalam sfc-gh-aalam requested a review from a team as a code owner September 17, 2024 18:17
@sfc-gh-aalam sfc-gh-aalam requested review from sfc-gh-jdu, sfc-gh-yixie and sfc-gh-jrose and removed request for a team September 17, 2024 18:17
@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@sfc-gh-aalam sfc-gh-aalam mentioned this pull request Sep 17, 2024
Merged
8 tasks
@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@sfc-gh-aalam sfc-gh-aalam force-pushed the aalam-SNOW-1418523-concurrent-file-operations branch from fde612e to df3263c Compare September 17, 2024 21:52
@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

Base automatically changed from aalam-SNOW-1418523-concurrent-file-operations to aalam-SNOW-1418523-make-internal-session-variables-thread-safe September 25, 2024 17:00
@github-actions GitHub Actions
Copy link

Seems like your changes contain some Local Testing changes, please request review from @snowflakedb/local-testing

@sfc-gh-aalam sfc-gh-aalam merged commit 42d6e19 into aalam-SNOW-1418523-make-internal-session-variables-thread-safe Sep 25, 2024
32 checks passed
@sfc-gh-aalam sfc-gh-aalam deleted the aalam-SNOW-1418523-make-udf-sproc-thread-safe branch September 25, 2024 17:06
@github-actions github-actions bot locked and limited conversation to collaborators Sep 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sfc-gh-jrose sfc-gh-jrose sfc-gh-jrose approved these changes

@sfc-gh-jdu sfc-gh-jdu sfc-gh-jdu approved these changes

@sfc-gh-yixie sfc-gh-yixie Awaiting requested review from sfc-gh-yixie sfc-gh-yixie was automatically assigned from snowflakedb/snowpark-python-api-reviewers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sfc-gh-aalam @sfc-gh-jdu @sfc-gh-jrose