From 5002a862da2c9f69c7e913912e6b536366a7892c Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Tue, 10 Dec 2024 21:59:42 +0000 Subject: [PATCH] Additional migration assistant feedback (#8895) * Fix migration assistant bugs Signed-off-by: Archer * Fix link Signed-off-by: Archer * Additional feedback Signed-off-by: Archer * Add additional feedback. Signed-off-by: Archer * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Fix TOC. Signed-off-by: Archer * fix link Signed-off-by: Archer --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> (cherry picked from commit befc0d29a85a2c57b137e9174944aab2458805d2) Signed-off-by: github-actions[bot] --- _includes/cards.html | 2 +- .../getting-started-data-migration.md | 6 ++++-- _migration-assistant/index.md | 4 ++-- .../migration-console/index.md | 1 + _migration-assistant/migration-phases/index.md | 11 +++++++++-- .../live-traffic-migration/index.md | 18 ++++++++++++++++++ ...witching-traffic-from-the-source-cluster.md | 5 ++++- .../using-traffic-replayer.md | 5 ++++- .../assessing-your-cluster-for-migration.md | 6 ++++-- .../planning-your-migration/index.md | 15 +++++++++++++++ .../verifying-migration-tools.md | 5 ++++- 11 files changed, 66 insertions(+), 12 deletions(-) rename _migration-assistant/{ => deploying-migration-assistant}/getting-started-data-migration.md (97%) create mode 100644 _migration-assistant/migration-phases/live-traffic-migration/index.md rename _migration-assistant/migration-phases/{ => live-traffic-migration}/switching-traffic-from-the-source-cluster.md (95%) rename _migration-assistant/migration-phases/{ => live-traffic-migration}/using-traffic-replayer.md (98%) rename _migration-assistant/migration-phases/{ => planning-your-migration}/assessing-your-cluster-for-migration.md (93%) create mode 100644 _migration-assistant/migration-phases/planning-your-migration/index.md rename _migration-assistant/migration-phases/{ => planning-your-migration}/verifying-migration-tools.md (98%) diff --git a/_includes/cards.html b/_includes/cards.html index 5ab37b8c27..3fa1809506 100644 --- a/_includes/cards.html +++ b/_includes/cards.html @@ -34,7 +34,7 @@

Migration Assistant

-

Migrate to OpenSearch from other platforms

+

Migrate to OpenSearch

Documentation →

diff --git a/_migration-assistant/getting-started-data-migration.md b/_migration-assistant/deploying-migration-assistant/getting-started-data-migration.md similarity index 97% rename from _migration-assistant/getting-started-data-migration.md rename to _migration-assistant/deploying-migration-assistant/getting-started-data-migration.md index 4110f29edf..f260a28701 100644 --- a/_migration-assistant/getting-started-data-migration.md +++ b/_migration-assistant/deploying-migration-assistant/getting-started-data-migration.md @@ -1,10 +1,12 @@ --- layout: default title: Getting started with data migration +parent: Deploying Migration Assistant nav_order: 10 redirect_from: - /upgrade-to/upgrade-to/ - /upgrade-to/snapshot-migrate/ + - /migration-assistant/getting-started-with-data-migration/ --- # Getting started with data migration @@ -207,8 +209,8 @@ Run the following command to access the migration console: {% include copy.html %} -`accessContainer.sh` is located in `/opensearch-migrations/deployment/cdk/opensearch-service-migration/` on the Bootstrap instance. To learn more, see [Accessing the migration console]. -`{: .note} +`accessContainer.sh` is located in `/opensearch-migrations/deployment/cdk/opensearch-service-migration/` on the Bootstrap instance. To learn more, see [Accessing the migration console]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/migrating-metadata/). +{: .note} --- diff --git a/_migration-assistant/index.md b/_migration-assistant/index.md index f024fdb69c..8e8422f0c4 100644 --- a/_migration-assistant/index.md +++ b/_migration-assistant/index.md @@ -50,7 +50,7 @@ Acting as a traffic simulation tool, the Traffic Replayer replays recorded reque The Metadata migration tool integrated into the Migration CLI can be used independently to migrate cluster metadata, including index mappings, index configuration settings, templates, component templates, and aliases. -### reindex-from-snapshot +### Reindex-from-Snapshot `Reindex-from-Snapshot` (RFS) reindexes data from an existing snapshot. Workers on Amazon Elastic Container Service (Amazon ECS) coordinate the migration of documents from an existing snapshot, reindexing the documents in parallel to a target cluster. @@ -70,6 +70,6 @@ The design deployed in AWS is as follows: 2. An Application Load Balancer with capture proxies relays traffic to a source while replicating data to Amazon Managed Streaming for Apache Kafka (Amazon MSK). 3. Using the migration console, you can initiate metadata migration to establish indexes, templates, component templates, and aliases on the target cluster. 4. With continuous traffic capture in place, you can use a `reindex-from-snapshot` process to capture data from your current index. -4. Once `reindex-from-snapshot` is complete, captured traffic is replayed from Amazon MSK to the target cluster by the traffic replayer. +4. Once `Reindex-from-Snapshot` is complete, captured traffic is replayed from Amazon MSK to the target cluster by the traffic replayer. 5. Performance and behavior of traffic sent to the source and target clusters are compared by reviewing logs and metrics. 6. After confirming that the target cluster's functionality meets expectations, clients are redirected to the new target. diff --git a/_migration-assistant/migration-console/index.md b/_migration-assistant/migration-console/index.md index 3e08e72c5c..eab3001868 100644 --- a/_migration-assistant/migration-console/index.md +++ b/_migration-assistant/migration-console/index.md @@ -4,6 +4,7 @@ title: Migration console nav_order: 30 has_children: true permalink: /migration-console/ +has_toc: false redirect_from: - /migration-console/index/ --- diff --git a/_migration-assistant/migration-phases/index.md b/_migration-assistant/migration-phases/index.md index c3c6c14b07..63e4b574e5 100644 --- a/_migration-assistant/migration-phases/index.md +++ b/_migration-assistant/migration-phases/index.md @@ -3,14 +3,21 @@ layout: default title: Migration phases nav_order: 50 has_children: true +has_toc: false permalink: /migration-phases/ redirect_from: - /migration-phases/index/ --- -This page details how to conduct a migration with Migration Assistant. It encompasses a variety of scenarios including: +# Migration phases + +This page details how to conduct a migration with Migration Assistant. It shows you how to [plan for your migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/index/) and encompasses a variety of migration scenarios, including: - [**Metadata migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/migrating-metadata/): Migrating cluster metadata, such as index settings, aliases, and templates. - [**Backfill migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/backfill/): Migrating existing or historical data from a source to a target cluster. -- **Live traffic migration**: Replicating live ongoing traffic from a source to a target cluster. +- [**Live traffic migration**]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/using-traffic-replayer/): Replicating live ongoing traffic from [a source cluster]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/) to a target cluster. + + + + diff --git a/_migration-assistant/migration-phases/live-traffic-migration/index.md b/_migration-assistant/migration-phases/live-traffic-migration/index.md new file mode 100644 index 0000000000..f9e3d9f71f --- /dev/null +++ b/_migration-assistant/migration-phases/live-traffic-migration/index.md @@ -0,0 +1,18 @@ +--- +layout: default +title: Live traffic migration +nav_order: 99 +parent: Migration phases +has_toc: false +has_children: true +--- + +# Live traffic migration + +Live traffic migration intercepts HTTP requests to a source cluster and stores them in a durable stream before forwarding them to the source cluster. The stored requests are then duplicated and replayed to the target cluster. This process synchronizes the source and target clusters while highlighting behavioral and performance differences between them. Kafka is used to manage the data flow and reconstruct HTTP requests. You can monitor the replication process through CloudWatch metrics and the [migration console]({{site.url}}{{site.baseurl}}/migration-console/), which provides results in JSON format for analysis. + +To start with live traffic migration, use the following steps: + +1. [Using Traffic Replayer]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/using-traffic-replayer/) +2. [Switching traffic from the source cluster]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/) + diff --git a/_migration-assistant/migration-phases/switching-traffic-from-the-source-cluster.md b/_migration-assistant/migration-phases/live-traffic-migration/switching-traffic-from-the-source-cluster.md similarity index 95% rename from _migration-assistant/migration-phases/switching-traffic-from-the-source-cluster.md rename to _migration-assistant/migration-phases/live-traffic-migration/switching-traffic-from-the-source-cluster.md index c43580eef9..e9e477654a 100644 --- a/_migration-assistant/migration-phases/switching-traffic-from-the-source-cluster.md +++ b/_migration-assistant/migration-phases/live-traffic-migration/switching-traffic-from-the-source-cluster.md @@ -2,7 +2,10 @@ layout: default title: Switching traffic from the source cluster nav_order: 110 -parent: Migration phases +grand_parent: Migration phases +parent: Live traffic migration +redirect_from: + - /migration-assistant/migration-phases/switching-traffic-from-the-source-cluster/ --- # Switching traffic from the source cluster diff --git a/_migration-assistant/migration-phases/using-traffic-replayer.md b/_migration-assistant/migration-phases/live-traffic-migration/using-traffic-replayer.md similarity index 98% rename from _migration-assistant/migration-phases/using-traffic-replayer.md rename to _migration-assistant/migration-phases/live-traffic-migration/using-traffic-replayer.md index 5b7af3c3f7..05c734d7be 100644 --- a/_migration-assistant/migration-phases/using-traffic-replayer.md +++ b/_migration-assistant/migration-phases/live-traffic-migration/using-traffic-replayer.md @@ -2,7 +2,10 @@ layout: default title: Using Traffic Replayer nav_order: 100 -parent: Migration phases +grand_parent: Migration phases +parent: Live traffic migration +redirect_from: + - /migration-assistant/migration-phases/using-traffic-replayer/ --- # Using Traffic Replayer diff --git a/_migration-assistant/migration-phases/assessing-your-cluster-for-migration.md b/_migration-assistant/migration-phases/planning-your-migration/assessing-your-cluster-for-migration.md similarity index 93% rename from _migration-assistant/migration-phases/assessing-your-cluster-for-migration.md rename to _migration-assistant/migration-phases/planning-your-migration/assessing-your-cluster-for-migration.md index 5ded49eb59..23bceb7114 100644 --- a/_migration-assistant/migration-phases/assessing-your-cluster-for-migration.md +++ b/_migration-assistant/migration-phases/planning-your-migration/assessing-your-cluster-for-migration.md @@ -2,12 +2,14 @@ layout: default title: Assessing your cluster for migration nav_order: 60 -parent: Migration phases +parent: Planning your migration +grand_parent: Migration phases +redirect_from: + - /migration-assistant/migration-phases/assessing-your-cluster-for-migration/ --- # Assessing your cluster for migration - The goal of the Migration Assistant is to streamline the process of migrating from one location or version of Elasticsearch/OpenSearch to another. However, completing a migration sometimes requires resolving client compatibility issues before they can communicate directly with the target cluster. ## Understanding breaking changes diff --git a/_migration-assistant/migration-phases/planning-your-migration/index.md b/_migration-assistant/migration-phases/planning-your-migration/index.md new file mode 100644 index 0000000000..4f0d264e93 --- /dev/null +++ b/_migration-assistant/migration-phases/planning-your-migration/index.md @@ -0,0 +1,15 @@ +--- +layout: default +title: Planning your migration +nav_order: 59 +parent: Migration phases +has_toc: false +has_children: true +--- + +# Planning your migration + +This section describes how to plan for your migration to OpenSearch by: + +- [Assessing your current cluster for migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/assessing-your-cluster-for-migration/). +- [Verifying that you have the tools for migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/planning-your-migration/verifying-migration-tools/). \ No newline at end of file diff --git a/_migration-assistant/migration-phases/verifying-migration-tools.md b/_migration-assistant/migration-phases/planning-your-migration/verifying-migration-tools.md similarity index 98% rename from _migration-assistant/migration-phases/verifying-migration-tools.md rename to _migration-assistant/migration-phases/planning-your-migration/verifying-migration-tools.md index 77df2b4280..aa2c5466cd 100644 --- a/_migration-assistant/migration-phases/verifying-migration-tools.md +++ b/_migration-assistant/migration-phases/planning-your-migration/verifying-migration-tools.md @@ -2,7 +2,10 @@ layout: default title: Verifying migration tools nav_order: 70 -parent: Migration phases +parent: Planning your migration +grand_parent: Migration phases +redirect_from: + - /migration-assistant/migration-phases/verifying-migration-tools/ --- # Verifying migration tools