Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ingestion): Explicitly set requirement on snowflake-connector-python to be newer or equal to 3.4.0 #10445

Conversation

skrydal
Copy link
Collaborator

@skrydal skrydal commented May 7, 2024

There was a problem appearing in snowflake-connector-python before version 3.4.0 due to its internal dependency on oscrypto library which malfunctioned if openssl version contained more than 2 digits for either major, minor or patch number (see details). Version 3.4.0 of snowflake-connector-python drops dependency on oscrypto and therefor fixes the problem. There was already an issue raised last year to address this: #9192 and image which was built and released back then indeed did not suffer from this problem. At this moment head version of datahub-ingestion image has openssl version 3.0.11 installed and snowflake-connector-python version 3.9.1 which means it does not suffer from this problem. Nevertheless when installing acryl-datahub pip package with snowflake support it might happen that version older than 3.4.0 of the snowflake-connector-python is installed - current setup.py just requires installed version to be different than 2.8.2. This PR imposes additional requirement on pip to make sure snowflake-connector-python version is newer or equal to 3.4.0 to avoid this problem (it did happen to me when working with acryl-datahub pip package recently).

@github-actions github-actions bot added ingestion PR or Issue related to the ingestion of metadata community-contribution PR or Issue raised by member(s) of DataHub Community datahub-community-champion PRs authored by DataHub Community Champions labels May 7, 2024
metadata-ingestion/setup.py Outdated Show resolved Hide resolved
metadata-ingestion/setup.py Outdated Show resolved Hide resolved
@hsheth2 hsheth2 added the merge-pending-ci A PR that has passed review and should be merged once CI is green. label May 8, 2024
@anshbansal anshbansal merged commit 9debbdd into datahub-project:master May 8, 2024
57 of 81 checks passed
sleeperdeep pushed a commit to sleeperdeep/datahub that referenced this pull request Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-contribution PR or Issue raised by member(s) of DataHub Community datahub-community-champion PRs authored by DataHub Community Champions ingestion PR or Issue related to the ingestion of metadata merge-pending-ci A PR that has passed review and should be merged once CI is green.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants