Skip to content

Commit

Permalink
fix(postgresql): add a missing argument to the stop logical replicati…
Browse files Browse the repository at this point in the history
…on command (#355)
  • Loading branch information
wojcik-dorota authored Aug 1, 2024
1 parent 62e1e60 commit 3ba3e03
Showing 1 changed file with 45 additions and 45 deletions.
90 changes: 45 additions & 45 deletions docs/products/postgresql/howto/setup-logical-replication.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,13 @@
---
title: Set up logical replication to Aiven for PostgreSQL®
sidebar_label: Set up logical replication
---

Aiven for PostgreSQL® represents an ideal managed solution for a variety
of use cases; remote production systems can be completely migrated to
Aiven using different methods including
[using Aiven-db-migrate](migrate-aiven-db-migrate) or the standard
[dump and restore method](migrate-pg-dump-restore).
Aiven for PostgreSQL® represents an ideal managed solution for a variety of use cases; remote production systems can be completely migrated to Aiven using different methods including [using Aiven-db-migrate](migrate-aiven-db-migrate) or the standard [dump and restore method](migrate-pg-dump-restore).

Whether you are migrating or have another use case to keep an existing
system in sync with an Aiven for PostgreSQL service, setting up a
**logical replica** is a good way to achieve that. This article goes
through the steps of replicating some tables from a self-managed
PostgreSQL cluster to Aiven.
system in sync with an Aiven for PostgreSQL service, you can address that by setting up a
**logical replica** and replicating tables from a self-managed PostgreSQL cluster to Aiven.

:::note
These instructions work also with AWS RDS PostgreSQL 10+ and [Google
Expand All @@ -35,19 +30,17 @@ These are the placeholders you will need to replace in the code sample:

## Requirements

You will need:

- PostgreSQL version 10 or newer.
- PostgreSQL version 10 or newer
- Connection between the source cluster's PostgreSQL port and Aiven
for PostgreSQL cluster.
- Access to an superuser role on the source cluster.
for PostgreSQL cluster
- Access to an superuser role on the source cluster
- `wal_level` setting to `logical` on the source cluster. To verify
and change the `wal_level` setting check
and change the `wal_level` setting, see
[the instructions on setting this configuration](/docs/products/postgresql/howto/migrate-aiven-db-migrate#pg_migrate_wal).

:::note
If you are using an AWS RDS PostgreSQL cluster as source, the
`rds.logical_replication` parameter must be set to `1` (true) in the
`rds.logical_replication` parameter must be set to `1` (`true`) in the
parameter group.
:::

Expand All @@ -59,16 +52,16 @@ extensions on the source cluster, but a superuser account is required.
:::tip
The `aiven_extras` extension enables the creation of a
publish/subscribe-style logical replication without a superuser account,
and it is preinstalled on Aiven for PostgreSQL servers. For more info on
`aiven_extras` check the dedicated [GitHub
and it is preinstalled on Aiven for PostgreSQL servers. For more information on
`aiven_extras`, see the dedicated [GitHub
repository](https://github.com/aiven/aiven-extras). The following
example will assume `aiven_extras` extension is not available in the
example assumes the `aiven_extras` extension is not available in the
source PostgreSQL database.
:::

This example assumes a source database called `origin_database` on a
self-managed PostgreSQL cluster. The replication will mirror three
tables, named `test_table`, `test_table_2` and `test_table_3`, to the
tables, named `test_table`, `test_table_2`, and `test_table_3`, to the
`defaultdb` database on Aiven for PostgreSQL. The process to setup the
logical replication is the following:

Expand All @@ -77,7 +70,7 @@ logical replication is the following:
1. Create the `PUBLICATION` entry, named `pub_source_tables`, for the
test tables:

```
```sql
CREATE PUBLICATION pub_source_tables
FOR TABLE test_table,test_table_2,test_table_3
WITH (publish='insert,update,delete');
Expand All @@ -89,15 +82,15 @@ logical replication is the following:
database.

When creating a publication entry, the `publish` parameter defines
the operations to transfer. In the above example, all the `INSERT`,
`UPDATE` or `DELETE` operations will be transferred.
the operations to transfer. In this example, all the `INSERT`,
`UPDATE`, or `DELETE` operations will be transferred.
:::

1. PostgreSQL's logical replication doesn't copy table definitions,
that can be extracted from the `origin_database` with `pg_dump` and
included in a `origin-database-schema.sql` file with:

```
```bash
pg_dump --schema-only --no-publications \
SRC_CONN_URI \
-t test_table -t test_table_2 -t test_table_3 > origin-database-schema.sql
Expand All @@ -106,22 +99,22 @@ logical replication is the following:
1. Connect via `psql` to the destination Aiven for PostgreSQL database
and create the new `aiven_extras` extension:

```
```sql
CREATE EXTENSION aiven_extras CASCADE;
```

1. Create the table definitions in the Aiven for PostgreSQL destination
database within `psql`:

```
```sql
\i origin-database-schema.sql
```

1. Create a `SUBSCRIPTION` entry, named `dest_subscription`, in the
Aiven for PostgreSQL destination database to start replicating
changes from the source `pub_source_tables` publication:

```
```sql
SELECT * FROM
aiven_extras.pg_create_subscription(
'dest_subscription',
Expand All @@ -134,10 +127,10 @@ logical replication is the following:

1. Verify that the subscription has been created successfully. As the
`pg_subscription` catalog is superuser-only, you can use the
`aiven_extras.pg_list_all_subscriptions()` function from
`aiven_extras.pg_list_all_subscriptions()` function from the
`aiven_extras` extension:

```
```sql
SELECT subdbid, subname, subowner, subenabled, subslotname
FROM aiven_extras.pg_list_all_subscriptions();
Expand All @@ -149,7 +142,7 @@ logical replication is the following:

1. Verify the subscription status:

```
```sql
SELECT * FROM pg_stat_subscription;
subid | subname | pid | relid | received_lsn | last_msg_send_time | last_msg_receipt_time | latest_end_lsn | latest_end_time
Expand All @@ -158,12 +151,11 @@ logical replication is the following:
(1 row)
```

1. Verify the data is correctly copied over the Aiven for PostgreSQL
target tables
1. Verify the data is correctly copied over the Aiven for PostgreSQL target tables.

## Remove unused replication setup

It is important to remove unused replication setups, since the
It is important to remove unused replication setups since the
underlying replication slots in PostgreSQL forces the server to keep all
the data needed to replicate since the publication creation time. If the
data stream has no readers, there will be an ever-growing amount of data
Expand All @@ -172,13 +164,21 @@ on disk until it becomes full.
To remove an unused subscription, essentially stopping the replication,
run the following command in the Aiven for PostgreSQL target database:

```
SELECT * FROM aiven_extras.pg_drop_subscription('dest_subscription');
```
- When the source database is accessible

```sql
SELECT * FROM aiven_extras.pg_drop_subscription('dest_subscription');
```

- When there is no access to the source database

```sql
SELECT * FROM aiven_extras.pg_drop_subscription('dest_subscription', FALSE);
```

Verify the replication removal with:

```
```sql
SELECT * FROM aiven_extras.pg_list_all_subscriptions();
subdbid | subname | subowner | subenabled | subconninfo | subslotname | subsynccommit | subpublications
Expand All @@ -188,22 +188,22 @@ subdbid | subname | subowner | subenabled | subconninfo | subslotname | subsync

## Manage inactive or lagging replication slots

Inactive or lagging replication could cause problems in a database, like
Inactive or lagging replication can cause problems in a database, like
an ever-increasing disk usage not associated to any growth of the amount
of data in the database. Filling the disk causes the database instance
to stop serving clients and thus a loss of service.
to stop serving clients and a loss of service.

1. Assess the replication slots status via `psql`:

```
```sql
SELECT slot_name,restart_lsn FROM pg_replication_slots;
```

The command output is like:

```
slot_name │ restart_lsn
──────────────┼─────────────
```text
slot_name │ restart_lsn
──────────────┼─────────────
pghoard_local │ 6E/16000000
dest_slot | 5B/8B0
(2 rows)
Expand All @@ -227,15 +227,15 @@ to stop serving clients and thus a loss of service.
problem. You can disable and enable the associated subscription
using `aiven_extras`:

```
```sql
SELECT * FROM aiven_extras.pg_alter_subscription_disable('dest_subscription');
SELECT * FROM aiven_extras.pg_alter_subscription_enable('dest_subscription');
```

- If the `dest_slot` connector is no longer needed, run the
following command to remove it:

```
```sql
SELECT pg_drop_replication_slot('dest_slot');
```

Expand Down

0 comments on commit 3ba3e03

Please sign in to comment.