Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Azure Blob Storage SDK to v12 #2573

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
bf69e03
Upgrade Azure to v12
pamelafox Nov 21, 2024
e106bc8
Update changelog
pamelafox Nov 21, 2024
073b7e7
New lines
pamelafox Nov 21, 2024
e50a3b8
Remove comment
pamelafox Nov 21, 2024
11b1c7d
Why is test failing
pamelafox Nov 21, 2024
30a6304
Rename dev containers
pamelafox Nov 22, 2024
8c87d91
Fix dev container config
pamelafox Nov 22, 2024
ff6f3dc
More remote print debugging
pamelafox Nov 23, 2024
f78c728
Update Azurite emulator for tests
pamelafox Nov 23, 2024
95dd260
update minimum reqs for azure-storage-blob
pamelafox Nov 23, 2024
a8af6dc
Apply pre-commit
pamelafox Nov 23, 2024
70508d5
Change how import error is handled
pamelafox Nov 23, 2024
a7a8e5f
Update the min reqs for 3.9
pamelafox Nov 23, 2024
da835a8
Revert unneeded tox change
pamelafox Nov 23, 2024
cde5d70
Lower numpy version
pamelafox Nov 23, 2024
c526059
Update tests, readme, dev container
pamelafox Nov 25, 2024
96c6048
Update devcontainer
pamelafox Nov 25, 2024
d7167eb
Make devcontainer work for all services
pamelafox Nov 25, 2024
4d39865
Add ports to devcontainer
pamelafox Nov 25, 2024
c700069
Add link to the admin
pamelafox Nov 25, 2024
8dc71bc
Run pre-commit
pamelafox Nov 25, 2024
f22343e
Change constructor to receive client, change download to send_file in…
pamelafox Nov 26, 2024
18d1c33
Update changelog
pamelafox Nov 26, 2024
71e6889
Remove type annotations to match S3 client
pamelafox Nov 26, 2024
59c752a
Merge branch 'master' into azureupgrade
pamelafox Nov 30, 2024
4e611de
Add to latest version
pamelafox Nov 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// For format details, see https://aka.ms/devcontainer.json
{
"name": "flask-admin (Python 3.12)",
"image": "mcr.microsoft.com/devcontainers/python:3.12-bullseye",

// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "vscode"
}
6 changes: 6 additions & 0 deletions .devcontainer/tests/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
ARG IMAGE=bullseye
FROM mcr.microsoft.com/devcontainers/${IMAGE}

RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends postgresql-client \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
17 changes: 17 additions & 0 deletions .devcontainer/tests/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// For format details, see https://aka.ms/devcontainer.json.
{
"name": "flask-admin tests (Postgres + Azurite + Mongo)",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"workspaceFolder": "/workspace",
"forwardPorts": [10000, 10001, 5432, 27017],
"portsAttributes": {
"10000": {"label": "Azurite Blob Storage Emulator", "onAutoForward": "silent"},
"10001": {"label": "Azurite Blob Storage Emulator HTTPS", "onAutoForward": "silent"},
"5432": {"label": "PostgreSQL port", "onAutoForward": "silent"},
"27017": {"label": "MongoDB port", "onAutoForward": "silent"},
},
// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "vscode",
"postAttachCommand": "psql -U postgres -h localhost -c 'CREATE EXTENSION IF NOT EXISTS hstore;' flask_admin_test && pip install -e \".[all]\" && pip install --use-pep517 -r requirements/dev.txt"
}
44 changes: 44 additions & 0 deletions .devcontainer/tests/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
version: '3'

services:
app:
build:
context: .
dockerfile: Dockerfile
args:
IMAGE: python:3.12

volumes:
- ../..:/workspace:cached

# Overrides default command so things don't shut down after the process ends.
command: sleep infinity
environment:
AZURE_STORAGE_CONNECTION_STRING: DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;

postgres:
image: postgis/postgis:16-3.4
restart: unless-stopped
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: flask_admin_test
volumes:
- postgres-data:/var/lib/postgresql/data
network_mode: service:app

azurite:
container_name: azurite
image: mcr.microsoft.com/azure-storage/azurite:latest
restart: unless-stopped
volumes:
- azurite-data:/data
network_mode: service:app

mongo:
image: mongo:5.0.14-focal
restart: unless-stopped
network_mode: service:app

volumes:
postgres-data:
azurite-data:
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
ports:
- 27017:27017
azurite:
image: arafato/azurite:2.6.5
image: mcr.microsoft.com/azure-storage/azurite:latest
env:
executable: blob
ports:
Expand Down
21 changes: 17 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,11 +130,13 @@ You should see output similar to:

OK

**NOTE!** For all the tests to pass successfully, you\'ll need Postgres (with
the postgis and hstore extension) & MongoDB to be running locally. You'll
also need *libgeos* available.
**NOTE!** For all the tests to pass successfully, you\'ll need several services running locally:
Postgres (with the postgis and hstore extension), MongoDB, and Azurite.
You'll also need *libgeos* available.
See tests.yaml for Docker configuration and follow service-specific setup below.

## Setting up local Postgres for tests

For Postgres:
```bash
psql postgres
> CREATE DATABASE flask_admin_test;
Expand All @@ -143,6 +145,7 @@ psql postgres
> CREATE EXTENSION postgis;
> CREATE EXTENSION hstore;
```

If you\'re using Homebrew on MacOS, you might need this:

```bash
Expand All @@ -155,6 +158,16 @@ createuser -s postgresql
brew services restart postgresql
```

## Setting up Azure Blob Storage emulator for tests

1. Run the [Azurite emulator](https://learn.microsoft.com/azure/storage/common/storage-use-azurite?tabs=visual-studio%2Cblob-storage)

2. Set the connection string for the emulator:

```bash
export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"
```

You can also run the tests on multiple environments using *tox*.

## 3rd Party Stuff
Expand Down
4 changes: 4 additions & 0 deletions doc/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ Changelog
2.0.0a3
-------

Breaking changes:

* Azure Blob Storage SDK has been upgraded from the legacy version (v2) to the latest version (v12). AzureFileAdmin now accept `blob_service_client` rather than `connection_string` to give more flexibility with connection types.
samuelhwilliams marked this conversation as resolved.
Show resolved Hide resolved

Fixes:

* Jinja templates can now be loaded in StrictUndefined mode.
Expand Down
29 changes: 29 additions & 0 deletions examples/azure-blob-storage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Azure Blob Storage Example

Flask-Admin example for an Azure Blob Storage account.

To run this example:

1. Clone the repository and navigate to this example::

git clone https://github.com/pallets-eco/flask-admin.git
cd flask-admin/examples/azure-blob-storage

2. Create and activate a virtual environment::

python -m venv venv
source venv/bin/activate

3. Install requirements::

pip install -r requirements.txt

4. Either run the Azurite Blob Storage emulator or create an actual Azure Blob Storage account. Set this environment variable:

export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"

The value below is the default for the Azurite emulator.

4. Run the application::

python app.py
Empty file.
40 changes: 40 additions & 0 deletions examples/azure-blob-storage/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import logging
import os

from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient
from flask import Flask
from flask_admin import Admin
from flask_admin.contrib.fileadmin.azure import AzureFileAdmin
from flask_babel import Babel

logging.basicConfig(level=logging.INFO)
app = Flask(__name__)
app.config["SECRET_KEY"] = "secret"


@app.route("/")
def index():
return '<a href="/admin/">Click me to get to Admin!</a>'


admin = Admin(app)
babel = Babel(app)


if conn_str := os.getenv("AZURE_STORAGE_CONNECTION_STRING"):
logging.info("Connecting to Azure Blob storage with connection string.")
client = BlobServiceClient.from_connection_string(conn_str)
elif account_name := os.getenv("AZURE_STORAGE_ACCOUNT_URL"):
# https://learn.microsoft.com/azure/storage/blobs/storage-blob-python-get-started?tabs=azure-ad#authorize-access-and-connect-to-blob-storage
logging.info("Connecting to Azure Blob storage with keyless auth")
client = BlobServiceClient(account_name, credential=DefaultAzureCredential())

file_admin = AzureFileAdmin(
blob_service_client=client,
container_name="fileadmin-tests",
)
admin.add_view(file_admin)

if __name__ == "__main__":
app.run(debug=True)
2 changes: 2 additions & 0 deletions examples/azure-blob-storage/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
../..[azure-blob-storage]
azure-identity
Loading