Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

Install + set up PgBackRest in postgresql-base #813

Merged
merged 3 commits into from
Oct 8, 2021
Merged

Install + set up PgBackRest in postgresql-base #813

merged 3 commits into from
Oct 8, 2021

Conversation

pypt
Copy link
Contributor

@pypt pypt commented Oct 8, 2021

Install and configure PgBackRest to postgresql-base so that images that derive from it - postgresql-server and temporal-postgresql - could use it.

The only supported repository for backups in this PR is Amazon S3 and S3-compatible object stores. Repository's credentials are configured through (optional) environment variables.

There's a terse doc on how to do full / incremental backups too.

As per #812, I intend to deploy this, do a full backup, see how long does it take + how much space does it occupy, and then come up with a backup strategy.

Unrelated, but in:

https://github.com/mediacloud/backend/compare/pgbackrest?expand=1#diff-2c08109dfb859a1eab9191db0d9aab0b4b330e09c5666f15e5654ca0be3b6933R1930-R1936

/dev/shm's size for temporal-postgresql gets increased too. PostgreSQL used to rely on /dev/shm heavily, I'm not sure if that's the case anymore, but it doesn't hurt to hike it up a bit.

References #812.

@pypt pypt requested a review from jtotoole October 8, 2021 13:39
@pypt pypt self-assigned this Oct 8, 2021

## Usage

Create stanza:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh also, from where should these commands be run? woodward?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently woodward, but the official definition is "whichever server has a role-postgresql-server Docker Swarm label set to it in production Ansible inventory (hosts.yml)".

Also it has to be run within a running postgresql-server container.

@jtotoole
Copy link
Contributor

jtotoole commented Oct 8, 2021

This looks really good! Questions just for my understanding:

  • From where does one configure backup frequency?
  • Is there a way to conditionally increase MC_PGBACKREST_RETENTION_FULL if the latest backup has failed? Is that something we'd want to do?

@pypt
Copy link
Contributor Author

pypt commented Oct 8, 2021

From where does one configure backup frequency?

At the moment backups happen when we run pgbackrest backup manually. Later when we know long do they take and how much disk space does a typical backup occupy, we'll add them as a Cron job somewhere.

Is there a way to conditionally increase MC_PGBACKREST_RETENTION_FULL if the latest backup has failed? Is that something we'd want to do?

Could you clarify this question?

@pypt pypt merged commit cfa447c into master Oct 8, 2021
@pypt pypt deleted the pgbackrest branch October 8, 2021 21:38
@pypt
Copy link
Contributor Author

pypt commented Oct 8, 2021

Merging this one then, but let me know if you have any more improvements to suggest / questions.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants