diff --git a/product_docs/docs/tpa/23/architecture-M1.mdx b/product_docs/docs/tpa/23/architecture-M1.mdx index 64718467585..848001483ce 100644 --- a/product_docs/docs/tpa/23/architecture-M1.mdx +++ b/product_docs/docs/tpa/23/architecture-M1.mdx @@ -5,11 +5,8 @@ originalFilePath: architecture-M1.md --- -A Postgres cluster with one or more active locations, each with the same -number of Postgres nodes and an extra Barman node. Optionally, there can -also be a location containing only a witness node, or a location -containing only a single node, even if the active locations have more -than one. +A Postgres cluster with a single primary node and physical replication +to a number of standby nodes including backup and failover management. This architecture is suitable for production and is also suited to testing, demonstrating and learning due to its simplicity and ability to @@ -19,25 +16,53 @@ If you select subscription-only EDB software with this architecture it will be sourced from EDB Repos 2.0 and you will need to [provide a token](reference/edb_repositories/). -## Application and backup failover - -The M1 architecture implements failover management in that it ensures -that a replica will be promoted to take the place of the primary should -the primary become unavailable. However it *does not provide any -automatic facility to reroute application traffic to the primary*. If -you require, automatic failover of application traffic you will need to -configure this at the application itself (for example using multi-host -connections) or by using an appropriate proxy or load balancer and the -facilities offered by your selected failover manager. - -The above is also true of the connection between the backup node and the -primary created by TPA. The backup will not be automatically adjusted to -target the new primary in the event of failover, instead it will remain -connected to the original primary. If you are performing a manual -failover and wish to connect the backup to the new primary, you may -simply re-run `tpaexec deploy`. If you wish to automatically change the -backup source, you should implement this using your selected failover -manager as noted above. +## Failover management + +The M1 architecture always includes a failover manager. Supported +options are repmgr, EDB Failover Manager (EFM) and Patroni. In all +cases, the failover manager will be configured by default to ensure that +a replica will be promoted to take the place of the primary should the +primary become unavailable. + +### Application failover + +The M1 architecture does not generally provide an automatic facility to +reroute application traffic to the primary. There are several ways you +can add this capability to your cluster. + +In TPA: + +- If you choose repmgr as the failover manager and enable PgBouncer, you + can include the `repmgr_redirect_pgbouncer: true` hash under + `cluster_vars` in `config.yml`. This causes repmgr to automatically + reconfigure PgBouncer to route traffic to the new primary on failover. + +- If you choose Patroni as the failover manager and enable PgBouncer, + Patroni will automatically reconfigure PgBouncer to route traffic to + the new primary on failover. + +- If you choose EFM as the failover manager, you can use the + `efm_conf_settings` hash under `cluster_vars` in `config.yml` to + [configure EFM to use a virtual IP address + (VIP)](/efm/latest/04_configuring_efm/05_using_vip_addresses/). This + is an additional IP address which will always route to the primary + node. + +- Place an appropriate proxy or load balancer between the cluster and + you application and use a [TPA hook](tpaexec-hooks/) to configure + your selected failover manager to update it with the route to the new + primary on failover. + +- Handle failover at the application itself, for example by using + multi-host connection strings. + +### Backup failover + +TPA does not configure any kind of 'backup failover'. If the Postgres +node from which you are backing up is down, backups will simply halt +until the node is back online. To manually connect the backup to the new +primary, edit `config.yml` to add the `backup` hash to the new primary +instance and re-run `tpaexec deploy`. ## Cluster configuration @@ -78,18 +103,18 @@ More detail on the options is provided in the following section. #### Additional Options -| Parameter | Description | Behaviour if omitted | -| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | -| `--platform` | One of `aws`, `docker`, `bare`. | Defaults to `aws`. | -| `--location-names` | A space-separated list of location names. The number of active locations is equal to the number of names supplied, minus one for each of the witness-only location and the single-node location if they are requested. | A single location called "main" is used. | -| `--primary-location` | The location where the primary server will be. Must be a member of `location-names`. | The first listed location is used. | -| `--data-nodes-per-location` | A number from 1 upwards. In each location, one node will be configured to stream directly from the cluster's primary node, and the other nodes, if present, will stream from that one. | Defaults to 2. | -| `--witness-only-location` | A location name, must be a member of `location-names`. | No witness-only location is added. | -| `--single-node-location` | A location name, must be a member of `location-names`. | No single-node location is added. | -| `--enable-haproxy` | 2 additional nodes will be added as a load balancer layer.
Only supported with Patroni as the failover manager. | HAproxy nodes will not be added to the cluster. | -| `--enable-pgbouncer` | PgBouncer will be configured in the Postgres nodes to pool connections for the primary. | PgBouncer will not be configured in the cluster. | -| `--patroni-dcs` | Select the Distributed Configuration Store backend for patroni.
Only option is `etcd` at this time.
Only supported with Patroni as the failover manager. | Defaults to `etcd`. | -| `--efm-bind-by-hostname` | Enable efm to use hostnames instead of IP addresses to configure the cluster `bind.address`. | Defaults to use IP addresses | +| Parameter | Description | Behaviour if omitted | +| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | +| `--platform` | One of `aws`, `docker`, `bare`. | Defaults to `aws`. | +| `--location-names` | A space-separated list of location names. The number of locations is equal to the number of names supplied. | A single location called "main" is used. | +| `--primary-location` | The location where the primary server will be. Must be a member of `location-names`. | The first listed location is used. | +| `--data-nodes-per-location` | A number from 1 upwards. In each location, one node will be configured to stream directly from the cluster's primary node, and the other nodes, if present, will stream from that one. | Defaults to 2. | +| `--witness-only-location` | A location name, must be a member of `location-names`. This location will be populated with a single witness node only. | No witness-only location is added. | +| `--single-node-location` | A location name, must be a member of `location-names`. This location will be populated with a single data node only. | No single-node location is added. | +| `--enable-haproxy` | Two additional nodes will be added as a load balancer layer.
Only supported with Patroni as the failover manager. | HAproxy nodes will not be added to the cluster. | +| `--enable-pgbouncer` | PgBouncer will be configured in the Postgres nodes to pool connections for the primary. | PgBouncer will not be configured in the cluster. | +| `--patroni-dcs` | Select the Distributed Configuration Store backend for patroni.
Only option is `etcd` at this time.
Only supported with Patroni as the failover manager. | Defaults to `etcd`. | +| `--efm-bind-by-hostname` | Enable efm to use hostnames instead of IP addresses to configure the cluster `bind.address`. | Defaults to use IP addresses |

diff --git a/product_docs/docs/tpa/23/architecture-PGD-Always-ON.mdx b/product_docs/docs/tpa/23/architecture-PGD-Always-ON.mdx index 948cf5d1aba..375317fa225 100644 --- a/product_docs/docs/tpa/23/architecture-PGD-Always-ON.mdx +++ b/product_docs/docs/tpa/23/architecture-PGD-Always-ON.mdx @@ -5,10 +5,10 @@ originalFilePath: architecture-PGD-Always-ON.md --- -!!! Note +!!!Note + This architecture is for Postgres Distributed 5 only. -If you require PGD 4 or 3.7 please use [BDR-Always-ON](architecture-BDR-Always-ON/). -!!! +If you require PGD 4 or 3.7 please use [BDR-Always-ON](BDR-Always-ON/). EDB Postgres Distributed 5 in an Always-ON configuration, suitable for use in test and production. @@ -85,9 +85,9 @@ data centre that provides a level of redundancy, in whatever way this definition makes sense to your use case. For example, AWS regions, your own data centres, or any other designation to identify where your servers are hosted. +!!! - -!!! Note Note for AWS users +!!! Note for AWS users If you are using TPA to provision an AWS cluster, the locations will be mapped to separate availability zones within the `--region` you diff --git a/product_docs/docs/tpa/23/reference/barman.mdx b/product_docs/docs/tpa/23/reference/barman.mdx index 80f7546e683..f0fd713c9d0 100644 --- a/product_docs/docs/tpa/23/reference/barman.mdx +++ b/product_docs/docs/tpa/23/reference/barman.mdx @@ -90,3 +90,17 @@ them to each other's authorized_keys file. The postgres user must be able to ssh to the barman server in order to archive WAL segments (if configured), and the barman user must be able to ssh to the Postgres instance to take or restore backups. + +## `barman` and `barman_role` Postgres users + +TPA will create two Postgres users, `barman` and `barman_role`. + +TPA versions `<23.35` created the `barman` Postgres user as a `superuser`. + +Beginning with `23.35` the `barman` user is created with `NOSUPERUSER`, +so any re-deploys on existing clusters will remove the `superuser` attribute +from the `barman` Postgres user. Instead, the `barman_role` is granted the +required set of privileges and the `barman` user is granted `barman_role` membership. + +This avoids granting the `superuser` attribute to the `barman` user, using the set +of privileges provided in the [Barman Manual](https://docs.pgbarman.org/release/latest/#postgresql-connection). diff --git a/product_docs/docs/tpa/23/reference/harp.mdx b/product_docs/docs/tpa/23/reference/harp.mdx index 281f9e5515a..27dfdc5a9ed 100644 --- a/product_docs/docs/tpa/23/reference/harp.mdx +++ b/product_docs/docs/tpa/23/reference/harp.mdx @@ -13,7 +13,7 @@ to `harp`, which is the default for BDR-Always-ON clusters. You must provide the `harp-manager` and `harp-proxy` packages. Please contact EDB to obtain access to these packages. -## Configuring HARP +## Variables for HARP configuration See the [HARP documentation](https://www.enterprisedb.com/docs/pgd/4/harp/04_configuration/) for more details on HARP configuration. @@ -41,6 +41,7 @@ for more details on HARP configuration. | `harp_proxy_max_client_conn` | `75` | Maximum number of client connections accepted by harp-proxy (`max_client_conn`) | | `harp_ssl_password_command` | None | a custom command that should receive the obfuscated sslpassword in the stdin and provide the handled sslpassword via stdout. | | `harp_db_request_timeout` | `10s` | similar to dcs -> request_timeout, but for connection to the database itself. | +| `harp_local_etcd_only` | None | limit harp manager endpoints list to only contain the local etcd node instead of all etcd nodes | You can use the [harp-config hook](../tpaexec-hooks/#harp-config) @@ -114,7 +115,7 @@ provide api endpoints to monitor service's health. The variable can contain these keys: -``` +```yaml enable: false secure: false cert_file: "/etc/tpa/harp_proxy/harp_proxy.crt" diff --git a/product_docs/docs/tpa/23/reference/pgbouncer.mdx b/product_docs/docs/tpa/23/reference/pgbouncer.mdx index 567d840caac..8e36b4528d6 100644 --- a/product_docs/docs/tpa/23/reference/pgbouncer.mdx +++ b/product_docs/docs/tpa/23/reference/pgbouncer.mdx @@ -1,16 +1,28 @@ --- -description: Adding pgbouncer to your Postgres cluster. -title: Configuring pgbouncer +description: Adding PgBouncer to your Postgres cluster. +title: Configuring PgBouncer originalFilePath: pgbouncer.md --- -TPA will install and configure pgbouncer on instances whose `role` +TPA will install and configure PgBouncer on instances whose `role` contains `pgbouncer`. -By default, pgbouncer listens for connections on port 6432 and forwards -connections to `127.0.0.1:5432` (which may be either Postgres or -[haproxy](haproxy/), depending on the architecture). +By default, PgBouncer listens for connections on port 6432 and, if no +`pgbouncer_backend` is specified, forwards connections to +`127.0.0.1:5432` (which may be either Postgres or [haproxy](haproxy/), +depending on the architecture). + +!!!Note Using PgBouncer to route traffic to the primary + +If you are using the M1 architecture with repmgr you can set +`repmgr_redirect_pgbouncer: true` hash under `cluster_vars` to have +PgBouncer connections directed to the primary. The PgBouncer will be +automatically updated on failover to route to the new primary. You +should use this option in combination with setting `pgbouncer_backend` +to the primary instance name to ensure that the cluster is initially +deployed with PgBouncer configured to route to the primary. +!!! You can set the following variables on any `pgbouncer` instance. diff --git a/product_docs/docs/tpa/23/reference/postgres_extension_configuration.mdx b/product_docs/docs/tpa/23/reference/postgres_extension_configuration.mdx index 6986d73ecc8..52ee6a54fa8 100644 --- a/product_docs/docs/tpa/23/reference/postgres_extension_configuration.mdx +++ b/product_docs/docs/tpa/23/reference/postgres_extension_configuration.mdx @@ -25,7 +25,7 @@ the package containing the extension. - [Adding the *vector* extension through configuration](reconciling-local-changes/) - [Specifying extensions for configured databases](postgres_databases/) -- [Including shared preload entries for extensions](postgresql.conf/#shared_preload_libraries) +- [Including shared preload entries for extensions](postgresql.conf/#shared-preload-libraries) - [Installing Postgres-related packages](postgres_installation_method_pkg/) ## TPA recognized extensions diff --git a/product_docs/docs/tpa/23/reference/tpaexec-download-packages.mdx b/product_docs/docs/tpa/23/reference/tpaexec-download-packages.mdx index 801d59d2ece..b548d2c526e 100644 --- a/product_docs/docs/tpa/23/reference/tpaexec-download-packages.mdx +++ b/product_docs/docs/tpa/23/reference/tpaexec-download-packages.mdx @@ -24,7 +24,7 @@ are supported. container of the target operating system and uses that system's package manager to resolve dependencies and download all necessary packages. The required Docker setup for download-packages is the same as that for - [using Docker as a deployment platform](../platform-docker). + [using Docker as a deployment platform](#platform-docker). ## Usage diff --git a/product_docs/docs/tpa/23/rel_notes/index.mdx b/product_docs/docs/tpa/23/rel_notes/index.mdx index 9a3fd7f6f59..651be5a8139 100644 --- a/product_docs/docs/tpa/23/rel_notes/index.mdx +++ b/product_docs/docs/tpa/23/rel_notes/index.mdx @@ -3,6 +3,7 @@ title: Trusted Postgres Architect release notes navTitle: Release notes description: Release notes for Trusted Postgres Architect and later navigation: + - tpa_23.35_rel_notes - tpa_23.34.1_rel_notes - tpa_23.34_rel_notes - tpa_23.33_rel_notes @@ -36,6 +37,7 @@ The Trusted Postgres Architect documentation describes the latest version of Tru | Trusted Postgres Architect version | Release Date | |---|---| +| [23.35](./tpa_23.35_rel_notes) | 25 Nov 2024 | | [23.34.1](./tpa_23.34.1_rel_notes) | 09 Sep 2024 | | [23.34](./tpa_23.34_rel_notes) | 22 Aug 2024 | | [23.33](./tpa_23.33_rel_notes) | 24 Jun 2024 | diff --git a/product_docs/docs/tpa/23/rel_notes/src/draft/tpa_23.35_rel_notes.yml b/product_docs/docs/tpa/23/rel_notes/src/draft/tpa_23.35_rel_notes.yml deleted file mode 100644 index a3c080918cd..00000000000 --- a/product_docs/docs/tpa/23/rel_notes/src/draft/tpa_23.35_rel_notes.yml +++ /dev/null @@ -1,142 +0,0 @@ -product: Trusted Postgres Architect -version: 23.35 -date: 29 October 2024 -intro: | - New features, enhancements, bug fixes, and other changes in Trusted Postgres Architect 23.35 include the following: -highlights: | - - More Trust - - More Postgres - - More Architect -relnotes: -- details: | - `PermissionsStartOnly` has been deprecated and is now achieved via `ExecStartPost=+/bin/bash...` syntax. - relnote: Remove deprecated `PermissionStartOnly` in postgres.service.j2 template - impact: low - jira: - - TPA-762 - type: Change -- details: | - Fixed a bug whereby the test that ensures the current pgd-proxy configuration - matches the expected configuration would fail for version < 5.5.0. This fix ensures - that TPA won't try to query configuration keys added in version 5.5.0. - impact: low - jira: - - TPA-819 - relnote: Fix tpaexec test for pgd-proxy config verification - type: Bug Fix -- details: | - The PostGIS package will automatically be added when a user specifies `postgis` - as an entry in either `postgres_extensions` or the list of extensions named under - `postgres_databases`. Also enables the CRB (Code Ready Builder) repository for - RHEL-compatible distributions so PostGIS dependencies can be installed. - impact: low - jira: - - TPA-771 - relnote: Add `postgis` to list of recognized extensions - type: Change -- details: | - Certain required privileges are granted to Postgres role, `barman_role`, - which is then granted to the `barman` Postgres user. This avoids creating the - `barman` user as a superuser. This role can also be granted to other Postgres - users by adding it to their `granted_roles` list using `postgres/createuser`. The - `barman_role` is created as part of the Barman tasks; if Barman is not used, this - role will not be created. Therefore, the task that grants privileges to this role - is only executed if the `barman_role` username is in the list of Postgres users - that are created. The ''barman'' user now has `NOSUPERUSER` explicitly specified - as a role attribute. If a cluster was deployed with a previous TPA version (which - created the ''barman'' user as a superuser), deploying with this version will - remove the `superuser` role attribute from the `barman` user. - impact: low - jira: - - TPA-148 - - TPA-818 - relnote: The `barman` Postgres user should not be a superuser - type: Change -- details: | - A `primary_slot_name` is configured on the primary node to ensure the - old primary uses a physical slot for replication during an EFM switchover. However, - ''bdr_init_physical'' attempts to use it for node initialisation and hangs indefinitely - since the slot does not exist in a PGD installation. This `primary_slot_name` - is now conditionally set explicitly when the `failover_manager` is EFM to avoid - setting it unnecessarily. - impact: low - jira: - - TPA-712 - relnote: '`primary_slot_name` added for EFM compatibility interferes with `bdr_init_physical` ' - type: Bug Fix -- details: | - If the `pgdcli_package_version` is specified in `config.yml`, the `bash-completion` - package is incorrectly named because the `packages_for` filter erroneously appends - the `pgdcli_package_version` to the package name. This results in an attempt - to download a nonexistant package. The `bash-completion` package is now appended - to the list after the `packages_for` filter, since it''s version is independent - from the `pgdcli_package_version`. - impact: low - jira: - - TPA-794 - relnote: Download correct `bash-completion` package version - type: Bug Fix -- details: | - Fixed an issue whereby in some cases error messages would be repeated - even after successful tasks. - impact: low - jira: - - TPA-812 - relnote: Clear error message stack after each task - type: Bug Fix -- details: | - Improve postgres-monitor script to better manage recoverable errors and - add retries on network errors to ensure that it won''t return failure when it - just didn't allow enough time for postgres service to be fully started. - impact: low - jira: - - TPA-796 - relnote: Improve postgres-monitor script - type: Change -- details: | - Fixed an issue whereby new replicas in patroni clusters would fail with - errors related to replication slots. - impact: low - jira: - - TPA-792 - - TPA-781 - relnote: Enable new replicas in patroni clusters - type: Bug Fix -- details: | - A support ticket questioned why the `pemserver` and `barman` nodes are - added to the `Allowed node host list` in EFM when they are not relevant to EFM - functions. Refactored the task that writes the `efm.node` configuration to only - include those nodes that have `efm` in their list of roles. - impact: low - jira: - - TPA-817 - relnote: Only add nodes with `efm` role to cluster `efm.nodes` file - type: Change -- details: | - 'If `--enable-pem` and `--enable-pg-backup-api` are passed to `tpaexec - configure`, `pem-agent` is added twice to the `barman` node if it is also a `witness`. Fixed - by by consolidating both `if` statements together to only evaluate the conditions - once. ' - impact: low - jira: - - TPA-793 - relnote: Add `pem-agent` role on barman nodes at most once for M1 architecture - type: Bug Fix -- details: | - Fixed a bug whereby if the user excluded the `pkg` selector, later pem-related - tasks would fail because the pem_python_executable fact had not been set. - impact: low - jira: - - TPA-814 - relnote: set pem_python_executable outside pkg role - type: Bug Fix -- details: | - The `--efm-install-path` and `--efm-cluster-name` flags are set when a PEM - server is registered on an EFM node. The `Streaming Replication`, `Failover - Manager Node Status` and `Failover Manager Cluster Info` probes are enabled when - a PEM agent is registered on an EFM node. ' - impact: low - jira: - - TPA-586 - relnote: Enable EFM probes when a PEM agent is registered on an EFM node - type: Change \ No newline at end of file diff --git a/product_docs/docs/tpa/23/rel_notes/src/tpa_23.35_rel_notes.yml b/product_docs/docs/tpa/23/rel_notes/src/tpa_23.35_rel_notes.yml new file mode 100644 index 00000000000..76c5f0fa288 --- /dev/null +++ b/product_docs/docs/tpa/23/rel_notes/src/tpa_23.35_rel_notes.yml @@ -0,0 +1,232 @@ +product: Trusted Postgres Architect +version: 23.35 +date: 25 November 2024 +intro: | + New features, enhancements, bug fixes, and other changes in Trusted Postgres Architect 23.35 include the following: +highlights: | + - Support for PGD Lightweight architecture + - Postgis is now a recognized extension. + - Docker `configure` creates named networks with static IP addresses. + - Support for RedHat Enterprise Linux 9 for ARM architectures. + - Support for PostgreSQL, EDB Postgres Extended, and EDB Postgres Advanced Server 17. + - Options for STIG/CIS compliance. + +relnotes: + - details: + "`PermissionsStartOnly` has been deprecated and is now achieved via `ExecStartPost=+/bin/bash...` + syntax" + impact: low + jira: + - TPA-762 + relnote: Remove deprecated `PermissionStartOnly` in postgres.service.j2 template + type: Change + - details: + Fixed a bug whereby the test that ensures the current pgd-proxy configuration + matches the expected configuration would fail for version < 5.5.0. This fix ensures + that TPA won't try to query configuration keys added in version 5.5.0. + impact: low + jira: + - TPA-819 + relnote: Fix tpaexec test for pgd-proxy config verification + type: Bug Fix + - details: + The PostGIS package will automatically be added when a user specifies `postgis` + as an entry in either `postgres_extensions` or the list of extensions named under + `postgres_databases`. Also enables the CRB (Code Ready Builder) repository for + RHEL-compatible distributions so PostGIS dependencies can be installed. + impact: low + jira: + - TPA-771 + relnote: Add `postgis` to list of recognized extensions + type: Enhancement + - details: + Certain required privileges are granted to Postgres role, `barman_role`, + which is then granted to the `barman` Postgres user. This avoids creating the + `barman` user as a superuser. This role can also be granted to other Postgres + users by adding it to their `granted_roles` list using `postgres/createuser`. + The `barman_role` is created as part of the Barman tasks; if Barman is not used, + this role will not be created. Therefore, the task that grants privileges to this + role is only executed if the `barman_role` username is in the list of Postgres + users that are created. The 'barman' user now has `NOSUPERUSER` explicitly specified + as a role attribute. If a cluster was deployed with a previous TPA version (which + created the 'barman' user as a superuser), deploying with this version will remove + the `superuser` role attribute from the `barman` user. + impact: low + jira: + - TPA-148 + - TPA-818 + relnote: The `barman` Postgres user is no longer a superuser + type: Change + - details: + Add new optional var `harp_local_etcd_only` available when using etcd with + HARP. This option tells HARP manager to connect to local etcd node. This recommendation + follows the best practices learnt by doing the same when `bdr` as consensus procotol + is being used. The default mode of adding multiple endpoints can lead to performance + issues in some cases. This option is added to give more control to the user. + impact: low + jira: + - TPA-821 + relnote: Add new option `harp_local_etcd_only` when using etcd with HARP + type: Change + addresses: 41931 + - details: + A `primary_slot_name` is configured on the primary node to ensure the old + primary uses a physical slot for replication during an EFM switchover. However, + 'bdr_init_physical' attempts to use it for node initialisation and hangs indefinitely + since the slot does not exist in a PGD installation. This `primary_slot_name` + is now conditionally set explicitly when the `failover_manager` is EFM to avoid + setting it unnecessarily. + impact: low + jira: + - TPA-712 + relnote: "Fix case where `primary_slot_name` added for EFM compatibility interferes with `bdr_init_physical`" + type: Bug Fix + addresses: 36064 + - details: + If the `pgdcli_package_version` is specified in `config.yml`, the `bash-completion` + package is incorrectly named because the `packages_for` filter erroneously appends + the `pgdcli_package_version` to the package name. This results in an attempt to + download a nonexistant package. The `bash-completion` package is now appended + to the list after the `packages_for` filter, since it's version is independent + from the `pgdcli_package_version`. + impact: low + jira: + - TPA-794 + relnote: Download correct `bash-completion` package version + type: Bug Fix + addresses: 38773 + - details: + TPA is now able to generate a PGD Lightweight architecture comprised of + three nodes in two locations (2 nodes in Primary and one in Disaster Recovery) + designed to ease migrations from physical replication. Users can now run `tpaexec + configure lw -a Lightweight --postgresql 15`. + impact: low + jira: + - TPA-838 + relnote: Add support for PGD Lightweight architecture + type: Enhancement + - details: + TPA now clears the error message stack after each task to ensure messages + are not spuriously repeated + impact: low + jira: + - TPA-812 + relnote: + Fix an issue whereby in some cases error messages would be repeated even + after successful tasks. + type: Bug Fix + - details: + Improve postgres-monitor script to better manage recoverable errors and + add retries on network errors to ensure that it won't return failure when it just + didn't allow enough time for postgres service to be fully started. + impact: low + jira: + - TPA-796 + relnote: Improve postgres-monitor script + type: Change + addresses: 39191 + - details: + Fixed an issue whereby new replicas in Patroni clusters would fail with + errors related to replication slots. + impact: low + jira: + - TPA-792 + - TPA-781 + relnote: Fix issue that prevented the addition of replicas to Patroni clusters + type: Bug Fix + - details: Previously the `pemserver` and `barman` nodes were + added to the `Allowed node host list` in EFM when they were not relevant to EFM + functions. Refactored the task that writes the `efm.node` configuration to only + include those nodes that have `efm` in their list of roles. + impact: low + jira: + - TPA-817 + relnote: Only add nodes with `efm` role to cluster `efm.nodes` file + type: Change + - details: + If `--enable-pem` and `--enable-pg-backup-api` are passed to `tpaexec configure`, + `pem-agent` is added twice to the `barman` node if it is also a `witness`. Fixed + by consolidating both `if` statements together to only evaluate the conditions + once. + impact: low + jira: + - TPA-793 + relnote: Add `pem-agent` role on barman nodes at most once for M1 architecture + type: Bug Fix + - details: + Fixed a bug whereby if the user excluded the `pkg` selector, later PEM-related + tasks would fail because the `pem_python_executable` fact had not been set. + impact: low + jira: + - TPA-814 + relnote: Set `pem_python_executable` outside of the `pkg` role + type: Bug Fix + - details: + The `--efm-install-path` and `--efm-cluster-name` flags are set when a + PEM server is registered on an EFM node. The `Streaming Replication`, `Failover + Manager Node Status` and `Failover Manager Cluster Info` probes are enabled when + a PEM agent is registered on an EFM node. + impact: low + jira: + - TPA-586 + relnote: Enable EFM probes when a PEM agent is registered on an EFM node + type: Enhancement + - details: + TPA now supports command-line options to create a cluster configured to + conform to many of the requirements of the STIG and CIS security standards. These + options cause TPA to set postgresql.conf settings as defined in the relevant + standards, to install required extensions, to configure other aspects of system + behaviour such as filesystem permissions and user connection limits, and to check + for other requirements such as FIPS crypto standards which TPA can't directly + impose. The clusters thus generated are not certified by TPA to conform to the + standards, but much of the groundwork of creating a conforming cluster is now + automated. + impact: low + jira: + - TPA-366 + - TPA-836 + - TPA-837 + relnote: Support STIG/CIS compliance + type: Enhancement + - details: + The configure command will now automatically add a named network and static + IP addresses to config.yml when Docker is the selected platform. The network name + is the same as the cluster name and the address range follows the existing semantics + of the --network option with the exception that only one subnet is used for the + whole cluster rather than one per location. If a subnet prefix is not specified + by the user, TPA will attempt to select a prefix which results in a subnet large + enough to fit the whole cluster. The key `ip_address` may now be used to specify + a static IP for a Docker instance as long as a named network is specified in the + config.yml. + impact: low + jira: + - TPA-261 + - TPA-407 + - TPA-434 + relnote: Have `configure` create a user-defined network on Docker + type: Enhancement + - details: Packages are now published targeting RHEL 9 ARM64, and TPA supports + deployments using this architecture and OS. Also updated the list of supported + AWS images to include the RedHat 9 ARM64 AMI provided by Amazon. The default `instance_type` + for ARM64 EC2 instances has been updated from `a1` to `t4g`, which is the current + generation processor available for burstable general purpose workloads. + impact: low + jira: + - TPA-780 + relnote: Support RedHat Enterprise Linux 9 for ARM architectures + type: Enhancement + - details: + Clusters can be configured to use PostgreSQL, EDB Postgres Extended and + EDB Postgres Advanced Server version 17. Barman no longer needs to install the + postgres server package to get the `pg_receivewal` binary when using EDB Postgres + Advanced Server 17 or EDB Postgres Extended 17 since the binary has been added + to the client package for these versions. TPA raises an architecture error when a cluster + is configured with `repmgr` as the failover_manager as it is not available for + Postgres 17. Updated documentation to reflect supported versions. + impact: low + jira: + - TPA-803 + relnote: + Support PostgreSQL, EDB Postgres Extended, and EDB Postgres Advanced Server + 17 + type: Enhancement \ No newline at end of file diff --git a/product_docs/docs/tpa/23/rel_notes/tpa_23.35_rel_notes.mdx b/product_docs/docs/tpa/23/rel_notes/tpa_23.35_rel_notes.mdx new file mode 100644 index 00000000000..bb3646a6512 --- /dev/null +++ b/product_docs/docs/tpa/23/rel_notes/tpa_23.35_rel_notes.mdx @@ -0,0 +1,55 @@ +--- +title: Trusted Postgres Architect 23.35 release notes +navTitle: Version 23.35 +--- + +Released: 25 November 2024 + +New features, enhancements, bug fixes, and other changes in Trusted Postgres Architect 23.35 include the following: + +## Highlights + +- Support for PGD Lightweight architecture +- Postgis is now a recognized extension. +- Docker `configure` creates named networks with static IP addresses. +- Support for RedHat Enterprise Linux 9 for ARM architectures. +- Support for PostgreSQL, EDB Postgres Extended, and EDB Postgres Advanced Server 17. +- Options for STIG/CIS compliance. + +## Enhancements + + + + + + + + + +
DescriptionAddresses
Add postgis to list of recognized extensions

The PostGIS package will automatically be added when a user specifies postgis as an entry in either postgres_extensions or the list of extensions named under postgres_databases. Also enables the CRB (Code Ready Builder) repository for RHEL-compatible distributions so PostGIS dependencies can be installed.

Add support for PGD Lightweight architecture

TPA is now able to generate a PGD Lightweight architecture comprised of three nodes in two locations (2 nodes in Primary and one in Disaster Recovery) designed to ease migrations from physical replication. Users can now run tpaexec configure lw -a Lightweight --postgresql 15.

Enable EFM probes when a PEM agent is registered on an EFM node

The --efm-install-path and --efm-cluster-name flags are set when a PEM server is registered on an EFM node. The Streaming Replication, Failover Manager Node Status and Failover Manager Cluster Info probes are enabled when a PEM agent is registered on an EFM node.

Support STIG/CIS compliance

TPA now supports command-line options to create a cluster configured to conform to many of the requirements of the STIG and CIS security standards. These options cause TPA to set postgresql.conf settings as defined in the relevant standards, to install required extensions, to configure other aspects of system behaviour such as filesystem permissions and user connection limits, and to check for other requirements such as FIPS crypto standards which TPA can't directly impose. The clusters thus generated are not certified by TPA to conform to the standards, but much of the groundwork of creating a conforming cluster is now automated.

Have configure create a user-defined network on Docker

The configure command will now automatically add a named network and static IP addresses to config.yml when Docker is the selected platform. The network name is the same as the cluster name and the address range follows the existing semantics of the --network option with the exception that only one subnet is used for the whole cluster rather than one per location. If a subnet prefix is not specified by the user, TPA will attempt to select a prefix which results in a subnet large enough to fit the whole cluster. The key ip_address may now be used to specify a static IP for a Docker instance as long as a named network is specified in the config.yml.

Support RedHat Enterprise Linux 9 for ARM architectures

Packages are now published targeting RHEL 9 ARM64, and TPA supports deployments using this architecture and OS. Also updated the list of supported AWS images to include the RedHat 9 ARM64 AMI provided by Amazon. The default instance_type for ARM64 EC2 instances has been updated from a1 to t4g, which is the current generation processor available for burstable general purpose workloads.

Support PostgreSQL, EDB Postgres Extended, and EDB Postgres Advanced Server 17

Clusters can be configured to use PostgreSQL, EDB Postgres Extended and EDB Postgres Advanced Server version 17. Barman no longer needs to install the postgres server package to get the pg_receivewal binary when using EDB Postgres Advanced Server 17 or EDB Postgres Extended 17 since the binary has been added to the client package for these versions. TPA raises an architecture error when a cluster is configured with repmgr as the failover_manager as it is not available for Postgres 17. Updated documentation to reflect supported versions.

+ + +## Changes + + + + + + + +
DescriptionAddresses
Remove deprecated PermissionStartOnly in postgres.service.j2 template

PermissionsStartOnly has been deprecated and is now achieved via ExecStartPost=+/bin/bash... syntax

The barman Postgres user is no longer a superuser

Certain required privileges are granted to Postgres role, barman_role, which is then granted to the barman Postgres user. This avoids creating the barman user as a superuser. This role can also be granted to other Postgres users by adding it to their granted_roles list using postgres/createuser. The barman_role is created as part of the Barman tasks; if Barman is not used, this role will not be created. Therefore, the task that grants privileges to this role is only executed if the barman_role username is in the list of Postgres users that are created. The 'barman' user now has NOSUPERUSER explicitly specified as a role attribute. If a cluster was deployed with a previous TPA version (which created the 'barman' user as a superuser), deploying with this version will remove the superuser role attribute from the barman user.

Add new option harp_local_etcd_only when using etcd with HARP

Add new optional var harp_local_etcd_only available when using etcd with HARP. This option tells HARP manager to connect to local etcd node. This recommendation follows the best practices learnt by doing the same when bdr as consensus procotol is being used. The default mode of adding multiple endpoints can lead to performance issues in some cases. This option is added to give more control to the user.

41931
Improve postgres-monitor script

Improve postgres-monitor script to better manage recoverable errors and add retries on network errors to ensure that it won't return failure when it just didn't allow enough time for postgres service to be fully started.

39191
Only add nodes with efm role to cluster efm.nodes file

Previously the pemserver and barman nodes were added to the Allowed node host list in EFM when they were not relevant to EFM functions. Refactored the task that writes the efm.node configuration to only include those nodes that have efm in their list of roles.

+ + +## Bug Fixes + + + + + + + + + +
DescriptionAddresses
Fix tpaexec test for pgd-proxy config verification

Fixed a bug whereby the test that ensures the current pgd-proxy configuration matches the expected configuration would fail for version < 5.5.0. This fix ensures that TPA won't try to query configuration keys added in version 5.5.0.

Fix case where primary_slot_name added for EFM compatibility interferes with bdr_init_physical

A primary_slot_name is configured on the primary node to ensure the old primary uses a physical slot for replication during an EFM switchover. However, 'bdr_init_physical' attempts to use it for node initialisation and hangs indefinitely since the slot does not exist in a PGD installation. This primary_slot_name is now conditionally set explicitly when the failover_manager is EFM to avoid setting it unnecessarily.

36064
Download correct bash-completion package version

If the pgdcli_package_version is specified in config.yml, the bash-completion package is incorrectly named because the packages_for filter erroneously appends the pgdcli_package_version to the package name. This results in an attempt to download a nonexistant package. The bash-completion package is now appended to the list after the packages_for filter, since it's version is independent from the pgdcli_package_version.

38773
Fix an issue whereby in some cases error messages would be repeated even after successful tasks.

TPA now clears the error message stack after each task to ensure messages are not spuriously repeated

Fix issue that prevented the addition of replicas to Patroni clusters

Fixed an issue whereby new replicas in Patroni clusters would fail with errors related to replication slots.

Add pem-agent role on barman nodes at most once for M1 architecture

If --enable-pem and --enable-pg-backup-api are passed to tpaexec configure, pem-agent is added twice to the barman node if it is also a witness. Fixed by consolidating both if statements together to only evaluate the conditions once.

Set pem_python_executable outside of the pkg role

Fixed a bug whereby if the user excluded the pkg selector, later PEM-related tasks would fail because the pem_python_executable fact had not been set.

+ + diff --git a/product_docs/docs/tpa/23/tpaexec-configure.mdx b/product_docs/docs/tpa/23/tpaexec-configure.mdx index a7988cc550c..4dae9a1fc8c 100644 --- a/product_docs/docs/tpa/23/tpaexec-configure.mdx +++ b/product_docs/docs/tpa/23/tpaexec-configure.mdx @@ -118,18 +118,34 @@ configuration. You must edit config.yml to specify multiple regions. ### Network configuration +!!!Note + +These options are not meaningful for the "bare" platform, where +TPA will not alter the network configuration of existing servers. +!!! + By default, each cluster will be configured with a number of randomly selected `/28` subnets from the CIDR range `10.33.0.0/16`, depending on the selected architecture. Specify `--network 192.168.0.0/16` to assign subnets from a different network. - -**Note:** On AWS clusters, this corresponds to the VPC CIDR. +On AWS clusters, this corresponds to the VPC CIDR. See [aws](platform-aws/#vpc-required) documentation for details. Specify `--subnet-prefix 26` to assign subnets of a different size, /26 instead of /28 in this case. +!!!Note + +When the "docker" platform is selected, TPA will always place the +entire cluster in a single subnet regardless of the architecture. This +subnet is generated according to the logic described here with the +exception that if the `subnet-prefix` is not specified, TPA will +automatically select a subnet size large enough to accomodate the number +of instances in +`config.yaml`. +!!! + Specify `--no-shuffle-subnets` to allocate subnets from the start of the network CIDR range, without randomisation, e.g. `10.33.0.0/28`, then `10.33.0.16/28` and so on. @@ -138,9 +154,6 @@ Specify `--exclude-subnets-from ` to exclude subnets that are already used in existing cluster config.yml files. You can specify this argument multiple times for each directory. -**Note:** These options are not meaningful for the "bare" platform, where -TPA will not alter the network configuration of existing servers. - ### Instance type Specify `--instance-type ` to select an instance type. @@ -247,7 +260,7 @@ details. #### Postgres flavour and version TPA supports PostgreSQL, EDB Postgres Extended, and EDB Postgres -Advanced Server (EPAS) versions 11 through 16. +Advanced Server (EPAS) versions 11 through 17. You must specify both the flavour (or distribution) and major version of Postgres to install, for example: diff --git a/product_docs/docs/tpa/23/tpaexec-hooks.mdx b/product_docs/docs/tpa/23/tpaexec-hooks.mdx index 5fccd1f9597..d9df23785c8 100644 --- a/product_docs/docs/tpa/23/tpaexec-hooks.mdx +++ b/product_docs/docs/tpa/23/tpaexec-hooks.mdx @@ -1,6 +1,6 @@ --- navTitle: Deployment hooks -description: Extending TPA with hooks to execute arbitrary Anisible tasks. +description: Extending TPA with hooks to execute arbitrary Ansible tasks. title: TPA hooks originalFilePath: tpaexec-hooks.md