-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b0a9185
commit bc5566a
Showing
6 changed files
with
119 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| name: Metabase Export | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| push: | ||
| branches: ## TODO retirer avant de merger | ||
| - feat/metabase-user | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v1 | ||
|
|
||
| - name: Install Scalingo CLI | ||
| run: curl -O https://cli-dl.scalingo.com/install && bash install | ||
|
|
||
| - name: Install SSH key | ||
| # Credit: https://stackoverflow.com/a/69234389 | ||
| run: | | ||
| mkdir -p ~/.ssh | ||
| install -m 600 -D /dev/null ~/.ssh/id_rsa | ||
| echo "${{ secrets.GH_SCALINGO_SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa | ||
| - name: Add Scalingo as a known host | ||
| run: | | ||
| ssh-keyscan -H ssh.osc-fr1.scalingo.com >> ~/.ssh/known_hosts | ||
| - name: Init environment variables | ||
| run: | | ||
| echo "SRC_DATABASE_URL=${{ secrets.METABASE_EXPORT_SRC_DATABASE_URL }}" >> .env.metabase | ||
| echo "DEST_METABASE_DATABASE_URL=${{ secrets.METABASE_EXPORT_DEST_METABASE_DATABASE_URL }}" >> .env.metabase | ||
| - name: Run export | ||
| run: make ci_metabase_export | ||
| env: | ||
| METABASE_SRC_APP: ${{ vars.METABASE_EXPORT_SRC_APP }} | ||
| METABASE_DEST_APP: ${{ vars.METABASE_EXPORT_DEST_APP }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Metabase | ||
|
|
||
| Conformément aux usages de la communauté beta.gouv.fr, DiaLog utilise Metabase pour la collecte de statistiques relatives à son utilisation. | ||
|
|
||
| L'instance Metabase est accessible ici : https://dialog-metabase.osc-fr1.scalingo.io/ | ||
|
|
||
| Vous devez disposer d'un compte pour y accéder. Demandez pour cela à un membre de l'équipe. | ||
|
|
||
| ## Aperçu de l'installation | ||
|
|
||
| Le Metabase de DiaLog est hébergé sur Scalingo sous l'application `dialog-metabase`. (Demandez à un membre de l'équipe de vous ajouter à cette application pour y avoir accès.) | ||
|
|
||
| Conformément aux recommendations Beta, une base de données dédiée aux données Metabase a été créée afin de la séparer des données de production. Elle est hébergée sur l'application `dialog-metabase` via un add-on PostgreSQL, comme indiqué dans [cette doc](https://doc.incubateur.net/communaute/les-outils-de-la-communaute/autres-services/metabase/metabase#connecter-metabase-a-une-base-de-donnees-anonymisee). | ||
|
|
||
| ## Lancer l'export | ||
|
|
||
| L'export Metabase peut être déclenché via [GitHub Actions](./github_actions.md) à l'aide du workflow [`metabase_export.yml`](../../.github/workflows/metabase_export.yml). | ||
|
|
||
| La configuration passe par diverses variables d'environnement listées ci-dessous : | ||
|
|
||
| | Variable d'environnement | Configuration | Description | | ||
| |---|---|---| | ||
| | `METABASE_EXPORT_SRC_APP` | [Variable](https://docs.github.com/fr/actions/learn-github-actions/variables) au sens GitHub Actions | `dialog` (pour la production) | | ||
| | `METABASE_EXPORT_DEST_APP` | [Variable](https://docs.github.com/fr/actions/learn-github-actions/variables) au sens GitHub Actions | `dialog-metabase` | | ||
| | `METABASE_EXPORT_SRC_DATABASE_URL` | [Secret](https://docs.github.com/fr/actions/security-guides/using-secrets-in-github-actions) au sens GitHub Actions | L'URL d'accès à la base de données applicative par la CI (`./tools/scalingodbtunnel dialog --host-url`) | | ||
| | `METABASE_EXPORT_DEST_METABASE_DATABASE_URL` | [Secret](https://docs.github.com/fr/actions/security-guides/using-secrets-in-github-actions) au sens GitHub Actions | L'URL d'accès à la base de données Metabase par la CI (`./tools/scalingodbtunnel dialog-metabase --host-url --port 10001`) | | ||
| | `GH_SCALINGO_SSH_PRIVATE_KEY` | Secret | Clé SSH privée permettant l'accès à Scalingo par la CI | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| #!/bin/bash | ||
| # Inspiré de : https://doc.incubateur.net/communaute/les-outils-de-la-communaute/autres-services/metabase/metabase#connecter-metabase-a-une-base-de-donnees-anonymisee | ||
| set -euxo pipefail | ||
|
|
||
| source .env.metabase ## Créé dans le workflow GitHub Actions | ||
|
|
||
| # Création des tables (persistera dans la DB source jusqu'à prochaine exécution de ce script) | ||
|
|
||
| psql $SRC_DATABASE_URL -c "DROP TABLE IF EXISTS analytics_user" | ||
| psql $SRC_DATABASE_URL -c " | ||
| CREATE TABLE analytics_user AS | ||
| SELECT | ||
| uuid_generate_v4() AS id, | ||
| u.registration_date | ||
| FROM \"user\" AS u" | ||
| psql $SRC_DATABASE_URL -c "ALTER TABLE analytics_user ADD PRIMARY KEY (id)" | ||
|
|
||
| # Création des index | ||
|
|
||
| # Copie vers la DB Metabase | ||
|
|
||
| pg_dump $SRC_DATABASE_URL -O -x -t analytics_user -c | psql "$DEST_METABASE_DATABASE_URL" |