Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Make automatic use of Azure storage account keys opt-in #20652

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

nameexhaustion
Copy link
Collaborator

@nameexhaustion nameexhaustion commented Jan 10, 2025

ref #20634

This makes automatic authentication using inferred storage account keys opt-in using an environment variable to reduce security concerns.

If this environment variable is unset and we observe an error, we print a message to hint to the user that this is an available option:

[CloudOptions::build_azure]: Warning: Azure authentication check failed, subsequent cloud operations may return an error. Note that polars was able to retrieve the storage account key for this URL from the Azure CLI - to allow polars to automatically use storage account keys for authentication, set POLARS_AUTO_AZURE_ACCOUNT_KEY=1 in the environment. To silence this warning, set POLARS_AUTO_AZURE_ACCOUNT_KEY=0.

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Jan 10, 2025
@nameexhaustion nameexhaustion changed the title feat: Make automatic authentication using Azure storage account keys is opt-in feat: Make automatic authentication using Azure storage account keys opt-in Jan 10, 2025
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 0% with 68 lines in your changes missing coverage. Please review.

Project coverage is 78.96%. Comparing base (2e0b3a3) to head (056089f).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-io/src/cloud/options.rs 0.00% 66 Missing ⚠️
py-polars/polars/io/cloud/credential_provider.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #20652      +/-   ##
==========================================
- Coverage   79.02%   78.96%   -0.06%     
==========================================
  Files        1557     1557              
  Lines      220551   220727     +176     
  Branches     2514     2513       -1     
==========================================
+ Hits       174283   174301      +18     
- Misses      45695    45852     +157     
- Partials      573      574       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nameexhaustion nameexhaustion marked this pull request as ready for review January 10, 2025 08:10
@nameexhaustion nameexhaustion marked this pull request as draft January 10, 2025 08:10
@nameexhaustion nameexhaustion marked this pull request as ready for review January 10, 2025 08:11
@nameexhaustion nameexhaustion changed the title feat: Make automatic authentication using Azure storage account keys opt-in feat: Make automatic use of Azure storage account keys opt-in Jan 10, 2025
eprintln!("[CloudOptions::build_azure]: Permission check OK");
}
return Ok(store);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Authentication will now prioritize Entra ID (via the Python-side DefaultAzureCredential if azure.identity is installed). Using the storage account key will only happen if we get a permission error (above).

@nameexhaustion nameexhaustion marked this pull request as draft January 10, 2025 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants