-
-
Notifications
You must be signed in to change notification settings - Fork 57
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: extra info on db backup / restore
- Loading branch information
1 parent
b12a9d3
commit d9005da
Showing
1 changed file
with
36 additions
and
75 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,72 +10,24 @@ your own cloud server. | |
- Get a cloud server (tested with Ubuntu 22.04). | ||
- Set up a domain name, and point the DNS to your cloud server. | ||
- SSH into your server. Set up a user with sudo called | ||
fmtm. [this](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04) | ||
svcfmtm. [this](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04) | ||
is a good guide for basic server setup including creation of a | ||
user. | ||
|
||
### Install some stuff it'll need | ||
|
||
#### Docker | ||
|
||
- Install | ||
Docker. [Here](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-22-04) | ||
is a good tutorial for that; do steps 1 and 2. At the time of | ||
writing that consisted of: | ||
|
||
sudo apt update | ||
sudo apt install apt-transport-https ca-certificates curl software-properties-common | ||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg | ||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null | ||
sudo apt update | ||
sudo apt install docker-ce | ||
sudo usermod -aG docker ${USER} | ||
su - ${USER} | ||
|
||
- Now install Docker Compose (as per [this | ||
tutorial](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-22-04)). At | ||
the time of writing (the latest version of Docker Compose may | ||
change, so the version number might be out of date, but the rest | ||
shouldn't change) this consisted of: | ||
|
||
mkdir -p ~/.docker/cli-plugins/ | ||
curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose | ||
sudo chmod +x ~/.docker/cli-plugins/docker-compose | ||
|
||
### Grab the FMTM code | ||
|
||
Clone the Git repo for the fmtm with `git clone https://github.com/hotosm/fmtm.git`. Step into the resulting directory | ||
with `cd fmtm`. | ||
|
||
### Set up the environment and utilities to launch | ||
|
||
Create the env file interactively with: | ||
### Run the install script | ||
|
||
```bash | ||
bash scripts/gen-env.sh | ||
``` | ||
|
||
OR | ||
curl -L https://get.fmtm.dev -o install.sh | ||
bash install.sh | ||
# Alternative URL: https://fmtm.hotosm.org/install.sh | ||
|
||
```bash | ||
cp .env.example .env | ||
|
||
# Then edit values manually | ||
# Then follow the prompts | ||
``` | ||
|
||
Main variables of note to update: | ||
#### Additional Environment Variables | ||
|
||
```dotenv | ||
ODK_CENTRAL_USER=`<CHANGEME>` | ||
ODK_CENTRAL_PASSWD=`<CHANGEME>` | ||
CERT_EMAIL=`<EMAIL_ADDRESS_TO_GENERATE_CERT_FOR>` | ||
OSM_CLIENT_ID=`<CHANGEME>` | ||
OSM_CLIENT_SECRET=`<CHANGEME>` | ||
S3_ACCESS_KEY=`<CHANGEME>` | ||
S3_SECRET_KEY=`<CHANGEME>` | ||
``` | ||
Variables are set in `.env`. | ||
Some can be updated manually, as required. | ||
|
||
#### EXTRA_CORS_ORIGINS | ||
|
||
|
@@ -113,7 +65,7 @@ stack, and variables should be set accordingly. | |
If you run FMTM with ODK and Minio (S3) included, then the | ||
domains will default to: | ||
|
||
``` | ||
```dotenv | ||
${FMTM_DOMAIN} --> Frontend | ||
api.${FMTM_DOMAIN} --> Backend | ||
odk.${FMTM_DOMAIN} --> ODK Central | ||
|
@@ -122,26 +74,17 @@ s3.${FMTM_DOMAIN} --> S3 / Minio | |
|
||
These defaults can be overriden with respective environment variables: | ||
|
||
``` | ||
```dotenv | ||
FMTM_API_DOMAIN | ||
FMTM_ODK_DOMAIN | ||
FMTM_S3_DOMAIN | ||
``` | ||
|
||
### Start the Compose Stack | ||
|
||
Run the production docker-compose config: | ||
`docker compose -f docker-compose.main.yml up -d` | ||
|
||
> Note: The images should be built already on Github. | ||
With any luck, this will launch the docker container where the project | ||
runs, and you can access the working website from the domain name! | ||
|
||
### Connecting to a remote database | ||
|
||
- A database may be located on a headless Linux server in the cloud. | ||
- To access the database via GUI tool such as PGAdmin, it is possible using port tunneling. | ||
- To access the database via GUI tool such as PGAdmin, | ||
it is possible using port tunneling. | ||
|
||
```bash | ||
ssh [email protected] -N -f -L {local_port}:localhost:{remote_port} | ||
|
@@ -156,18 +99,36 @@ This will map port 5432 on the remote machine to port 5430 on your local machine | |
|
||
```bash | ||
GIT_BRANCH=development | ||
backup_filename="fmtm-db-backup-$(date +'%Y-%m-%d').sql.gz" | ||
backup_filename="fmtm-db-${GIT_BRANCH}-$(date +'%Y-%m-%d').sql.gz" | ||
echo $backup_filename | ||
|
||
docker exec -i -e PGPASSWORD=PASSWORD_HERE fmtm-db-${GIT_BRANCH} pg_dump --verbose --format c -U fmtm fmtm | gzip -9 > "$backup_filename" | ||
docker exec -i -e PGPASSWORD=PASSWORD_HERE \ | ||
fmtm-db-${GIT_BRANCH} \ | ||
pg_dump --verbose --format c -U fmtm fmtm \ | ||
| gzip -9 > "$backup_filename" | ||
|
||
# For ODK | ||
docker exec -i -e PGPASSWORD=PASSWORD_HERE \ | ||
fmtm-central-db-${GIT_BRANCH} \ | ||
pg_dump --verbose --format c -U odk odk | \ | ||
gzip -9 > "$backup_filename" | ||
``` | ||
|
||
## Manual Database Restores | ||
|
||
```bash | ||
# On a different machine (else change the container name) | ||
GIT_BRANCH=development | ||
backup_filename=fmtm-db-backup-XXXX-XX-XX-sql.gz | ||
|
||
cat "$backup_filename" | gunzip | docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE fmtm-db-${GIT_BRANCH} pg_restore --verbose -U fmtm -d fmtm | ||
backup_filename=fmtm-db-${GIT_BRANCH}-XXXX-XX-XX-sql.gz | ||
|
||
cat "$backup_filename" | gunzip | \ | ||
docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE \ | ||
fmtm-db-${GIT_BRANCH} \ | ||
pg_restore --verbose -U fmtm -d fmtm | ||
|
||
# For ODK | ||
cat "$backup_filename" | gunzip | \ | ||
docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE \ | ||
fmtm-central-db-${GIT_BRANCH} \ | ||
pg_restore --verbose -U odk -d odk | ||
``` |