backend: add migration tool to migrate between any two backends

[dboslee]

This adds a new hidden command teleport migrate.

The command takes a config file which allows a source and destination backend to be specified. It then copies all the data from the source to the destination backend.

An example config looks like

src:
  type: dynamodb
  table_name: teleport-main
  region: us-west-2
dst:
  type: cockroachdb
  conn_string: postgresql://teleport@cockroachdb:26257/teleport_backend
parallel: 100

We plan to use this on cloud to migrate between dynamodb and cockroachdb but its generic where it could be used to migrate between any backends. For example we could also use this to migrate between dynamodb tables in different regions.

Its important to note that all Teleport Auth services should be scaled down when running this command to avoid data inconsistencies.

[github-actions]

The PR changelog entry failed validation: Changelog entry not found in the PR body. Please add a "no-changelog" label to the PR, or changelog lines starting with changelog: followed by the changelog entries for the PR.

[fspmarshall]

As mentioned elsewhere, definitely wants to be streaming and/or paginated, and I think we probably want some kind of check to make sure folks don't accidentally write to a backend with existing state in it (probably can just be a full range call with limit 1 that fails the command if something is returned). Otherwise, I think this looks great and will be very good to have. Nice work!

[dboslee]

@fspmarshall

As mentioned elsewhere, definitely wants to be streaming and/or paginated, and I think we probably want some kind of check to make sure folks don't accidentally write to a backend with existing state in it (probably can just be a full range call with limit 1 that fails the command if something is returned). Otherwise, I think this looks great and will be very good to have. Nice work!

I like the idea the only issue is if you hit an error and need to rerun the migration after some items have already been moved then you are stuck.

Perhaps we make this suggestion the default behavior with an flag to ignore this check.

[fspmarshall]

@dboslee safe default with force/override sounds reasonable to me.

[dboslee]

Apologies for the delay on this, had some higher priority things to work on.

[dboslee]

docker: Error response from daemon: Head "https://registry-1.docker.io/v2/library/fedora/manifests/latest": received unexpected HTTP status: 503 Service Unavailable.

503 from docker.io 🙄

[public-teleport-github-review-bot]

@dboslee See the table below for backport results.

Branch	Result
branch/v15	Create PR
branch/v16	Create PR