Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #44: custom postgres docker image #45

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/custom-dockerfile-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
type: choice
options:
- nginx
- postgres
tag:
required: true
description: Version a tag l'image
Expand Down
5 changes: 5 additions & 0 deletions dockerfiles/postgres/.env.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
BB_URL=
BB_SERVICE_ACCOUNT=
BB_SERVICE_KEY=
BB_INSTANCE_ID=
BB_DATABASE_ID=
17 changes: 17 additions & 0 deletions dockerfiles/postgres/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM postgres:latest

RUN apt-get update && apt-get install -y python3 python3-requests curl
RUN mkdir -p /app && chown postgres:postgres /app

WORKDIR /app

COPY fetch_schema.py /app/fetch_schema.py
COPY entrypoint.sh /app/entrypoint.sh

RUN chmod +x /app/entrypoint.sh

RUN chmod 700 /app

USER postgres

ENTRYPOINT ["/app/entrypoint.sh"]
55 changes: 55 additions & 0 deletions dockerfiles/postgres/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Custom Postgres image

This repository contains a custom PostgreSQL Docker image that dynamically
fetches the latest database schema from Bytebase during container startup. The
container dynamically retrieves the latest schema for a specified `INSTANCE_ID`
and `DATABASE_ID` from Bytebase at runtime, ensuring the PostgreSQL database
is always up-to-date.

## Environment Variables

To configure the schema-fetching process, the following environment variables
are required:

- `BB_URL` – The base URL to access the Bytebase API.
- `BB_SERVICE_ACCOUNT` – The service account email used for authentication with
Bytebase.
- `BB_SERVICE_KEY` – The API key or password associated with the service
account.
- `BB_INSTANCE_ID` – The ID of the Bytebase instance containing the desired
schema.
- `BB_DATABASE_ID` – The ID of the specific database schema to be fetched from
Bytebase.

## Usage example

The following is an example of how this custom image can be used in a Docker
Compose setup for the **Fertiscan** project:

```yaml
services:
postgres:
image: ghcr.io/ai-cfia/postgres:latest
container_name: fertiscan-postgres
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=fertiscan
- BB_URL=${BB_URL}
- BB_SERVICE_ACCOUNT=${BB_SERVICE_ACCOUNT}
- BB_SERVICE_KEY=${BB_SERVICE_KEY}
- INSTANCE_ID=${BB_INSTANCE_ID}
- DATABASE_ID=${BB_DATABASE_ID}
env_file:
- .env
ports:
- "5432:5432"
healthcheck:
test: ["CMD", "pg_isready", "-U", "postgres"]
timeout: 10s
retries: 5
networks:
- fertiscan-network
volumes:
- postgres-data:/var/lib/postgresql/data
```
5 changes: 5 additions & 0 deletions dockerfiles/postgres/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

python3 /app/fetch_schema.py

exec postgres
49 changes: 49 additions & 0 deletions dockerfiles/postgres/fetch_schema.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import os
import requests

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use https://pypi.org/project/python-dotenv/ to fetch env values from Dockerfile

e.g

from dotenv import load_dotenv

if __name__ == "__main__":
    load_dotenv()

    BB_URL = os.getenv("BB_URL")
    BB_SERVICE_ACCOUNT = os.getenv("BB_SERVICE_ACCOUNT")
    BB_SERVICE_KEY = os.getenv("BB_SERVICE_KEY")
    INSTANCE_ID = os.getenv("BB_INSTANCE_ID")
    DATABASE_ID = os.getenv("BB_DATABASE_ID")

BB_URL = os.getenv("BB_URL")
BB_SERVICE_ACCOUNT = os.getenv("BB_SERVICE_ACCOUNT")
BB_SERVICE_KEY = os.getenv("BB_SERVICE_KEY")
INSTANCE_ID = os.getenv("BB_INSTANCE_ID")
DATABASE_ID = os.getenv("BB_DATABASE_ID")

def generate_token():
login_url = f"{BB_URL}/v1/auth/login"
headers = {
'Content-Type': 'application/json',
'Accept-Encoding': 'deflate, gzip'
}
data = {
"email": BB_SERVICE_ACCOUNT,
"password": BB_SERVICE_KEY,
"web": True
}

response = requests.post(login_url, json=data, headers=headers)
if response.status_code == 200:
return response.json().get('token')
else:
print(f"Failed to generate token: {response.status_code}, {response.text}")
return None

def fetch_fertiscan_dev_schema():
token = generate_token()
if token:
schema_url = f"{BB_URL}/v1/instances/{INSTANCE_ID}/databases/{DATABASE_ID}/schema"
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
}

response = requests.get(schema_url, headers=headers)
if response.status_code == 200:
with open('schema.sql', 'w') as file:
file.write(response.json()['schema'])
print("Schema saved to schema.sql")
else:
print(f"Failed to fetch schema: {response.status_code}, {response.text}")
else:
print("No token generated, cannot fetch schema.")

if __name__ == "__main__":
fetch_fertiscan_dev_schema()
Loading