#38: Ingest an instrument from a URL

[evelynnchen-cmu]

Description

util/url_loader.py includes back-end functionality for securely downloading and validating files (PDF, .xlsx, .txt, CSV, and .docx) from user-provided URLs. Implementation ensures that all downloads adhere to security, compliance, and performance standards. It includes URL validation, rate limiting, SSL validation, and content & file size checks to prevent abuse and maintain system integrity.

Fixes #38

Type of change

• New feature (non-breaking change which adds functionality)

Testing

I added tests in test_url_loader.py and utilized Mock objects to simulate responses and test edge cases effectively.

Since the Harmony Python package is used by the Harmony API (which is itself used by the R library and the web app), we need to avoid making any changes that break the Harmony API. Please also run the Harmony API unit tests and check that the API still runs with your changes to the Python package: https://github.com/harmonydata/harmonyapi

Test Configuration

• Library version: v1.0.1
• OS: MacOS 14.6
• Toolchain: Python 3.10, PyCharm

Checklist

• My code follows the style guidelines of this project. I have applied a Linter (recommended: Pycharm's code formatter) to make my whitespace consistent with the rest of the project.
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have checked my code and corrected any misspellings
• The Harmony API is not broken by my change to the Harmony Python library
• I add third party dependencies only when necessary. If I changed the requirements, it changes in requirements.txt, pyproject.toml and also in the requirements.txt in the API repo
• If I introduced a new feature, I documented it (e.g. making a script example in the script examples repository so that people will know how to use it.