Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking ticket for smart_open() - wasb support #1562

Open
9 tasks
utkarsharma2 opened this issue Jan 10, 2023 · 2 comments
Open
9 tasks

Tracking ticket for smart_open() - wasb support #1562

utkarsharma2 opened this issue Jan 10, 2023 · 2 comments
Labels
feature New feature or request product/python-sdk Label describing products

Comments

@utkarsharma2
Copy link
Collaborator

Please describe the feature you'd like to see
In our code base, we should remove the following lines of code:

@property
def smartopen_uri(self) -> str:
"""
SmartOpen does not support URIs prefixed with wasb, so we need to change them to azure.
:return: URI compatible with SmartOpen for Azure BlobStorage.
"""
parsed_url = urlparse(self.path)
if parsed_url.scheme == "wasbs":
return self.path.replace("wasbs", "azure")
elif parsed_url.scheme == "wasb":
return self.path.replace("wasb", "azure")
else:
return self.path

once the ticket is resolved on smart open end - piskvorky/smart_open#755

Describe the solution you'd like
We should change it back to location.path

Are there any alternatives to this feature?
Is there another way we could solve this problem or enable this use-case?

Acceptance Criteria

  • All checks and tests in the CI should pass
  • Unit tests (90% code coverage or more, once available)
  • Integration tests (if the feature relates to a new database or external service)
  • Example DAG
  • Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
  • Exception handling in case of errors
  • Logging (are we exposing useful information to the user? e.g. source and destination)
  • Improve the documentation (README, Sphinx, and any other relevant)
  • How to use Guide for the feature (example)
@utkarsharma2 utkarsharma2 added the feature New feature or request label Jan 10, 2023
@pankajastro pankajastro added the product/python-sdk Label describing products label Jan 11, 2023
@phanikumv phanikumv changed the title Tacking ticket for smart_open() - wasb support Tracking ticket for smart_open() - wasb support Jan 16, 2023
@phanikumv
Copy link
Collaborator

Waiting on piskvorky/smart_open#755 to be implemented

@sunank200
Copy link
Contributor

Added comment on the thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request product/python-sdk Label describing products
Projects
None yet
Development

No branches or pull requests

4 participants