Skip to content

Commit

Permalink
Merge branch 'branch/v15' into bot/backport-38494-branch/v15
Browse files Browse the repository at this point in the history
  • Loading branch information
stevenGravy authored Feb 22, 2024
2 parents ea65529 + 13ae43a commit f4d8a3b
Show file tree
Hide file tree
Showing 11 changed files with 960 additions and 19 deletions.
12 changes: 9 additions & 3 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,10 @@
"title": "Cert Authority Rotation",
"slug": "/management/operations/ca-rotation/"
},
{
"title": "Database Cert Authority Rotation",
"slug": "/management/operations/db-ca-rotation/"
},
{
"title": "TLS Routing Migration",
"slug": "/management/operations/tls-routing/",
Expand All @@ -951,9 +955,11 @@
{
"title": "Proxy Peering Migration",
"slug": "/management/operations/proxy-peering/",
"forScopes": [
"enterprise"
]
"forScopes": ["enterprise"]
},
{
"title": "Database CA Migrations",
"slug": "/management/operations/db-ca-migrations/"
}
]
},
Expand Down
1 change: 1 addition & 0 deletions docs/cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,7 @@
"nodename",
"nohup",
"nologin",
"noout",
"noprompt",
"nosql",
"nowait",
Expand Down
21 changes: 16 additions & 5 deletions docs/pages/database-access/reference/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,22 @@ spec:
# Database connection endpoint.
uri: "localhost:5432"
# Optional CA for validating the database certificate.
ca_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
# Optional TLS configuration.
tls:
# TLS verification mode. Valid options are:
# 'verify-full' - performs full certificate validation (default).
# 'verify-ca' - the same as `verify-full`, but skips the server name validation.
# 'insecure' - accepts any certificate provided by database (not recommended).
mode: verify-full
# Optional database DNS server name. It allows to override the DNS name on
# a client certificate when connecting to a database.
# Use only with 'verify-full' mode.
server_name: db.example.com
# Optional CA for validating the database certificate.
ca_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
# Database admin user for automatic user provisioning.
admin_user:
Expand Down
2 changes: 2 additions & 0 deletions docs/pages/management/operations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,7 @@ the [Cluster Administration Guides](./admin.mdx) section.
- [Scaling](./operations/scaling.mdx): How to configure Teleport for large-scale deployments.
- [Backup and Restore](./operations/backup-restore.mdx): Backing up and restoring the cluster.
- [CA Rotation](./operations/ca-rotation.mdx): Rotating Teleport certificate authorities.
- [Database CA Rotation](./operations/db-ca-rotation.mdx): Rotating Teleport's `db` or `db_client` certificate authorities.
- [TLS Routing Migration](./operations/tls-routing.mdx): Migrating your Teleport cluster to single-port TLS routing mode.
- [Proxy Peering Migration](./operations/proxy-peering.mdx): Migrating your Teleport cluster to Proxy Peering mode.
- [Database CA Migrations](./operations/db-ca-migrations.mdx): Completing Teleport's Database CA migrations.
29 changes: 18 additions & 11 deletions docs/pages/management/operations/ca-rotation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,20 @@ This section will show you how to rotate Teleport's certificate authority.
If you are joining Teleport processes to a cluster via the Teleport Auth Service
using a [join token](../../agents/join-services-to-your-cluster/join-token.mdx), each
Teleport process will need a CA pin to trust the Auth Service. The CA pin will
change after each CA rotation. Make sure you use the *new* CA pin when adding
Teleport services after rotation.
change after each `host` CA rotation. Make sure you use the *new* CA pin when adding
Teleport services after `host` CA rotation.

<Admonition type="warning" title="Desktop Access">
Teleport signs Windows Desktop certificates with the user certificate authority.
If the user CA is rotated, the new CA certificate must be exported and
<Admonition type="warning" title="Exported CAs">
Teleport signs Windows Desktop client certificates with the `user` certificate
authority.
If the `user` CA is rotated, the new CA certificate must be exported and
configured in group policy.

Teleport signs self-hosted database host certificates with the `db` certificate
authority and signs database client certificates with the `db_client` CA.
If either of these CAs is rotated, then self-hosted databases must be
reconfigured.
Refer to [the Database CA Rotation Guide](./db-ca-rotation.mdx).
</Admonition>

### Rotation phases
Expand Down Expand Up @@ -54,7 +61,7 @@ There are two kinds of certificate rotations:
- **Semi-automatic:** Teleport automatically transitions between phases of the
rotation after some amount of time (known as a *grace period*) elapses.

For both types of rotations, the cluster goes through the phases in the
For both types of rotation, the cluster goes through the phases in the
following order:

- `standby` -> `init` -> `update_clients` -> `update_servers` -> `standby`
Expand All @@ -63,7 +70,7 @@ Administrators can abort the rotation and revert all changes any time before
the rotation is completed by entering the `rollback` phase.

```code
$ tctl auth rotate --phase=rollback --type=<Var name="type" description="Certificate authority to rotate"/> --manual
$ tctl auth rotate --manual --type=<Var name="type" description="Certificate authority to rotate"/> --phase=rollback
```

For example, if an admin has detected that some nodes failed to upgrade during
Expand All @@ -90,7 +97,7 @@ of the cluster.
Initiate the manual rotation of host certificate authorities:

```code
$ tctl auth rotate --phase=init --type=<Var name="type" description="Certificate authority to rotate"/> --manual
$ tctl auth rotate --manual --type=<Var name="type" description="Certificate authority to rotate"/> --phase=init
Updated rotation phase to "init". To check status use 'tctl status'
```

Expand Down Expand Up @@ -134,7 +141,7 @@ transitions.
Execute the transition from `init` to `update_clients`:

```code
$ tctl auth rotate --phase=update_clients --type=<Var name="type" description="Certificate authority to rotate"/> --manual
$ tctl auth rotate --manual --type=<Var name="type" description="Certificate authority to rotate"/> --phase=update_clients
# Updated rotation phase to "update_clients". To check status use 'tctl status'
$ tctl status
# Cluster acme.cluster
Expand Down Expand Up @@ -164,7 +171,7 @@ Now that all nodes have caught up, execute the transition from `update_clients`
to `update_servers`:

```code
$ tctl auth rotate --phase=update_servers --type=<Var name="type" description="Certificate authority to rotate"/> --manual
$ tctl auth rotate --manual --type=<Var name="type" description="Certificate authority to rotate"/> --phase=update_servers
# Updated rotation phase to "update_servers". To check status use 'tctl status'
$ tctl status
Expand Down Expand Up @@ -205,7 +212,7 @@ $ tsh ssh hello@terminal
</Admonition>

```code
$ tctl auth rotate --phase=standby --type=<Var name="type" description="Certificate authority to rotate"/> --manual
$ tctl auth rotate --manual --type=<Var name="type" description="Certificate authority to rotate"/> --phase=standby
```

Verify that the rotation has completed with `tctl`:
Expand Down
176 changes: 176 additions & 0 deletions docs/pages/management/operations/db-ca-migrations.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
---
title: Database CA Migrations
description: How to complete Teleport Database CA migrations.
---

In Teleport, self-hosted databases must be configured with certificates to
enable mTLS authentication via the Teleport Database Service.

Teleport 15 introduced a new `db_client` certificate authority (CA) to split the
responsibilities of the Teleport `db` CA, which was acting as both host and
client CA for Teleport self-hosted database access.

Teleport's host/client database CA split is intended to limit the potential for
lateral movement to other resources in the event that a database instance's
private key is compromised.

The `db` and `db_client` CAs were both introduced as an automatic migration
after upgrading to Teleport 10 and Teleport 15, respectively.

This guide will explain why these CAs were created and how to determine if your
CA(s) should be rotated to complete the migration process.

## Prerequisites

(!docs/pages/includes/edition-prereqs-tabs.mdx!)

- (!docs/pages/includes/tctl.mdx!)
- A Teleport cluster that was created prior to Teleport 15.
If your Teleport cluster was created with Teleport 15+, then this guide does
not apply to your cluster, because your `db` and `db_client` CAs were not
migrated.

## Teleport `db` CA migration

Your Teleport cluster's `db` CA can be used to issue certificates to self-hosted
databases.
This is convenient, because the Teleport Database Service trusts certificates
issued by the `db` CA by default, so there is no additional TLS configuration
in Teleport required.

Alternatively, you can issue certificates to your self-hosted databases using
an external CA - you just need to configure the Teleport Database Service to
trust that CA when connecting to your database(s).

<Details title="How to make Teleport trust an external database CA?">
For a static database defined in your Teleport Database Service `teleport.yaml`
configuration file, set `tls.ca_cert_file` to a file containing your CA's root
certificate.

For a dynamic database, put your CA's root certificate in `spec.tls.ca_cert`.

For examples and more information, consult the
[Database Access Configuration Reference](../../database-access/reference/configuration.mdx).
</Details>

Prior to Teleport 10, the Teleport `host` CA was used to issue certificates to
self-hosted databases (via `tctl auth sign`).
The Teleport `db` CA was introduced to decouple self-hosted database CA rotation
from the rest of your Teleport cluster.
The idea is that you should be able to rotate the CA used for self-hosted
databases without affecting other resources connected to your cluster.
Likewise, when you rotate your cluster's `host` CA, you should not have to worry
about affecting self-hosted databases.

To avoid breaking database access after upgrading to Teleport 10, Teleport
clusters are automatically migrated to create the `db` CA as a copy of
the `host` CA.

If your cluster was upgraded to Teleport 10 and you use Teleport to issue
certificates to your self-hosted databases, then you should ensure that you have
completed the `db` CA migration.
Otherwise, if you later rotate just one CA for any reason, a copy of the old CA
will still exist.
While this does not necessarily lead to a vulnerability in your cluster, it is
bad security practice to keep an old CA around after rotating it.

To complete the `db` CA migration:
- we recommend rotating your `host` CA
- we **strongly recommend** rotating your `db` CA

## Teleport `db_client` CA migration

The Teleport Database Service needs to authenticate itself to self-hosted
database(s) using a client certificate, which requires that you configure your
database(s) to trust Teleport's `db_client` CA.
Prior to the introduction of the `db_client` CA in Teleport 15, self-hosted
had to be configured to trust the Teleport `db` CA for client authentication.

With the old approach - trusting the `db` CA for client connections - if a
database's private key is compromised, and a `db` certificate was issued for
that key, then it could be used to gain access to other databases.

Not all self-hosted databases are vulnerable to lateral movement after a private
key compromise.
For example, MySQL and PostgreSQL both verify that a client's certificate
subject matches the client's database user.
Other databases only verify that a client's certificate is trusted, but do not
match the certificate subject to the database username.
For example, Cassandra, ScyllaDB, and Redis do not verify the client cert
subject.
All of these databases can be configured to require password authentication
after a successful mTLS handshake.
However, for defense in depth, these databases should only mTLS handshake with
a client that presents a `db_client` CA-issued certificate.

If your Teleport cluster was upgraded to Teleport 15, then you should ensure
that you have completed the `db_client` migration.
To complete the `db_client` CA migration:
- we recommend rotating your `db` CA
- we **strongly recommend** rotating your `db_client` CA.
- we **strongly recommend** reconfiguring your databases' certificates after
you complete the `db_client` CA rotation.

<Admonition type="note" title="reconfiguring certs after db_client CA rotation">
If you use `tctl auth sign` to reconfigure a database's certificates during
a `db_client` CA rotation, then the trusted certificate output will include
both the old and the new CA certificates.
To complete the migration, you should reconfigure those databases again after
the rotation - that way they will only trust the new CA.

If you don't want to reconfigure each database both during and after the
`db_client` CA rotation, and you do not mind temporarily losing connectivity
to your databases via Teleport, then you can just complete the `db_client` CA
rotation and reconfigure your databases afterward.
</Admonition>

## 1/2. Check for Teleport CA migrations

If you upgraded your cluster to Teleport 10 and you have never rotated your
`host` or `db` CAs, then you should complete the `db` CA migration.

If you upgraded your cluster to Teleport 15 and you have never rotated your
`db` or `db_client` CAs, then you should complete the `db_client` CA migration.

If you are unsure whether you need to complete the migration for either the `db`
or `db_client` CAs, you can check for duplicated CAs.
Use these commands to print the X.509 certificate serial number for your `host`,
`db`, and `db_client` CAs (in that order):

```code
$ tctl auth export --type=tls-host | openssl x509 -noout -serial
$ tctl auth export --type=db | openssl x509 -noout -serial
$ tctl auth export --type=db-client | openssl x509 -noout -serial
```

If the `db` CA serial number matches the `host` CA serial number, then you
need to complete the `db` CA migration.

If the `db_client` CA serial number matches the `db` CA serial number, then you
need to complete the `db_client` CA migration.

## 2/2. Rotate CAs

If you need to complete both the `db` and `db_client` migrations, then a single
rotation of each of the `host`, `db`, and `db_client` CAs is enough: you do not
need to rotate the `db` CA twice.

If you need to rotate the `host` CA, we recommend completing that rotation
before starting either of the `db` or `db_client` CA rotations: do not rotate
other CAs in parallel with a `host` CA rotation.
For information about `host` CA rotation, refer to the
[CA Rotation Guide](./ca-rotation.mdx).

Database CA rotations are a little different, because they involve configuring
external resources (self-hosted databases) with new certificates during the
rotation.
You can (and should) rotate the `db` and `db_client` CAs at the same time to
avoid repeating the database certificate reconfiguration steps.

For details on rotating the `db` or `db_client` CA, refer to the
[Database CA Rotation Guide](./db-ca-rotation.mdx).

## Further reading

- How the [Teleport Certificate Authority](../../architecture/authentication.mdx) works.
- How [Teleport Database Access](../../database-access/architecture.mdx) works.
Loading

0 comments on commit f4d8a3b

Please sign in to comment.