Skip to content

Commit

Permalink
docs: extra info on db backup / restore
Browse files Browse the repository at this point in the history
  • Loading branch information
spwoodcock committed Nov 23, 2023
1 parent b12a9d3 commit d9005da
Showing 1 changed file with 36 additions and 75 deletions.
111 changes: 36 additions & 75 deletions docs/dev/Production.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,72 +10,24 @@ your own cloud server.
- Get a cloud server (tested with Ubuntu 22.04).
- Set up a domain name, and point the DNS to your cloud server.
- SSH into your server. Set up a user with sudo called
fmtm. [this](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04)
svcfmtm. [this](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04)
is a good guide for basic server setup including creation of a
user.

### Install some stuff it'll need

#### Docker

- Install
Docker. [Here](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-22-04)
is a good tutorial for that; do steps 1 and 2. At the time of
writing that consisted of:

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker ${USER}
su - ${USER}

- Now install Docker Compose (as per [this
tutorial](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-22-04)). At
the time of writing (the latest version of Docker Compose may
change, so the version number might be out of date, but the rest
shouldn't change) this consisted of:

mkdir -p ~/.docker/cli-plugins/
curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose
sudo chmod +x ~/.docker/cli-plugins/docker-compose

### Grab the FMTM code

Clone the Git repo for the fmtm with `git clone https://github.com/hotosm/fmtm.git`. Step into the resulting directory
with `cd fmtm`.

### Set up the environment and utilities to launch

Create the env file interactively with:
### Run the install script

```bash
bash scripts/gen-env.sh
```

OR
curl -L https://get.fmtm.dev -o install.sh
bash install.sh
# Alternative URL: https://fmtm.hotosm.org/install.sh

```bash
cp .env.example .env

# Then edit values manually
# Then follow the prompts
```

Main variables of note to update:
#### Additional Environment Variables

```dotenv
ODK_CENTRAL_USER=`<CHANGEME>`
ODK_CENTRAL_PASSWD=`<CHANGEME>`
CERT_EMAIL=`<EMAIL_ADDRESS_TO_GENERATE_CERT_FOR>`
OSM_CLIENT_ID=`<CHANGEME>`
OSM_CLIENT_SECRET=`<CHANGEME>`
S3_ACCESS_KEY=`<CHANGEME>`
S3_SECRET_KEY=`<CHANGEME>`
```
Variables are set in `.env`.
Some can be updated manually, as required.

#### EXTRA_CORS_ORIGINS

Expand Down Expand Up @@ -113,7 +65,7 @@ stack, and variables should be set accordingly.
If you run FMTM with ODK and Minio (S3) included, then the
domains will default to:

```
```dotenv
${FMTM_DOMAIN} --> Frontend
api.${FMTM_DOMAIN} --> Backend
odk.${FMTM_DOMAIN} --> ODK Central
Expand All @@ -122,26 +74,17 @@ s3.${FMTM_DOMAIN} --> S3 / Minio

These defaults can be overriden with respective environment variables:

```
```dotenv
FMTM_API_DOMAIN
FMTM_ODK_DOMAIN
FMTM_S3_DOMAIN
```

### Start the Compose Stack

Run the production docker-compose config:
`docker compose -f docker-compose.main.yml up -d`

> Note: The images should be built already on Github.
With any luck, this will launch the docker container where the project
runs, and you can access the working website from the domain name!

### Connecting to a remote database

- A database may be located on a headless Linux server in the cloud.
- To access the database via GUI tool such as PGAdmin, it is possible using port tunneling.
- To access the database via GUI tool such as PGAdmin,
it is possible using port tunneling.

```bash
ssh [email protected] -N -f -L {local_port}:localhost:{remote_port}
Expand All @@ -156,18 +99,36 @@ This will map port 5432 on the remote machine to port 5430 on your local machine

```bash
GIT_BRANCH=development
backup_filename="fmtm-db-backup-$(date +'%Y-%m-%d').sql.gz"
backup_filename="fmtm-db-${GIT_BRANCH}-$(date +'%Y-%m-%d').sql.gz"
echo $backup_filename

docker exec -i -e PGPASSWORD=PASSWORD_HERE fmtm-db-${GIT_BRANCH} pg_dump --verbose --format c -U fmtm fmtm | gzip -9 > "$backup_filename"
docker exec -i -e PGPASSWORD=PASSWORD_HERE \
fmtm-db-${GIT_BRANCH} \
pg_dump --verbose --format c -U fmtm fmtm \
| gzip -9 > "$backup_filename"

# For ODK
docker exec -i -e PGPASSWORD=PASSWORD_HERE \
fmtm-central-db-${GIT_BRANCH} \
pg_dump --verbose --format c -U odk odk | \
gzip -9 > "$backup_filename"
```

## Manual Database Restores

```bash
# On a different machine (else change the container name)
GIT_BRANCH=development
backup_filename=fmtm-db-backup-XXXX-XX-XX-sql.gz

cat "$backup_filename" | gunzip | docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE fmtm-db-${GIT_BRANCH} pg_restore --verbose -U fmtm -d fmtm
backup_filename=fmtm-db-${GIT_BRANCH}-XXXX-XX-XX-sql.gz

cat "$backup_filename" | gunzip | \
docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE \
fmtm-db-${GIT_BRANCH} \
pg_restore --verbose -U fmtm -d fmtm

# For ODK
cat "$backup_filename" | gunzip | \
docker exec -i -e PGPASSWORD=NEW_PASSWORD_HERE \
fmtm-central-db-${GIT_BRANCH} \
pg_restore --verbose -U odk -d odk
```

0 comments on commit d9005da

Please sign in to comment.