Skip to content

Commit

Permalink
replace dm external link with internal link (pingcap#7419)
Browse files Browse the repository at this point in the history
  • Loading branch information
ran-huang authored Jan 18, 2022
1 parent 13fa485 commit eb16fa1
Show file tree
Hide file tree
Showing 8 changed files with 64 additions and 63 deletions.
2 changes: 1 addition & 1 deletion filter-dml-event.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ When performing incremental data replication, you can use the [Binlog Event Filt

To address the issue, since v2.0.5, DM supports using `binlog value filter` in incremental data replication to filter data. Among the DM-supported and `ROW`-formatted binlog, the binlog events carry values of all columns, and you can configure SQL expressions based on these values. If the expression calculates a row change as `TRUE`, DM does not replicate this row change to the downstream.

Similar to [Binlog Event Filter](/filter-binlog-event.md), you need to configure `binlog value filter` in the task configuration file. For details, see the following configuration example. For the advanced task configuration and the description, refer to [DM advanced task configuration file](https://docs.pingcap.com/tidb-data-migration/stable/task-configuration-file-full#task-configuration-file-template-advanced).
Similar to [Binlog Event Filter](/filter-binlog-event.md), you need to configure `binlog value filter` in the task configuration file. For details, see the following configuration example. For the advanced task configuration and the description, refer to [DM advanced task configuration file](/dm/task-configuration-file-full.md#task-configuration-file-template-advanced).

```yaml
name: test
Expand Down
20 changes: 10 additions & 10 deletions migrate-aurora-to-tidb.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,8 @@ If you encounter any problem during the import, refer to [TiDB Lightning FAQ](/t

### Prerequisites

- [Install DM](https://docs.pingcap.com/tidb-data-migration/stable/deploy-a-dm-cluster-using-tiup).
- [Get the source database and target database privileges required for DM](https://docs.pingcap.com/tidb-data-migration/stable/dm-worker-intro).
- [Install DM](/dm/deploy-a-dm-cluster-using-tiup.md).
- [Get the source database and target database privileges required for DM](/dm/dm-worker-intro.md).

### Step 1: Create the data source

Expand Down Expand Up @@ -244,7 +244,7 @@ mysql-instances:
# safe-mode: true # If this field is set to true, DM changes INSERT of the data source to REPLACE for the target database, and changes UPDATE of the data source to DELETE and REPLACE for the target database. This is to ensure that when the table schema contains a primary key or unique index, DML statements can be imported repeatedly. In the first minute of starting or resuming an incremental replication task, DM automatically enables the safe mode.
```

The YAML file above is the minimum configuration required for the migration task. For more configuration items, refer to [DM Advanced Task Configuration File](https://docs.pingcap.com/tidb-data-migration/stable/task-configuration-file-full).
The YAML file above is the minimum configuration required for the migration task. For more configuration items, refer to [DM Advanced Task Configuration File](/dm/task-configuration-file-full.md).

### Step 3. Run the migration task

Expand Down Expand Up @@ -273,7 +273,7 @@ The parameters used in the command above are described as follows:

If the task fails to start, check the prompt message and fix the configuration. After that, you can re-run the command above to start the task.

If you encounter any problem, refer to [DM error handling](https://docs.pingcap.com/tidb-data-migration/stable/error-handling) and [DM FAQ](https://docs.pingcap.com/tidb-data-migration/stable/faq).
If you encounter any problem, refer to [DM error handling](/dm/dm-error-handling.md) and [DM FAQ](/dm/dm-faq.md).

### Step 4. Check the migration task status

Expand All @@ -285,7 +285,7 @@ To learn whether the DM cluster has an ongoing migration task and the task statu
tiup dmctl --master-addr ${advertise-addr} query-status ${task-name}
```

For a detailed interpretation of the results, refer to [Query Status](https://docs.pingcap.com/tidb-data-migration/stable/query-status).
For a detailed interpretation of the results, refer to [Query Status](/dm/dm-query-status.md).

### Step 5. Monitor the task and view logs

Expand All @@ -300,8 +300,8 @@ When DM is running, DM-worker, DM-master, and dmctl print the related informatio

## What's next

- [Pause the migration task](https://docs.pingcap.com/tidb-data-migration/stable/pause-task).
- [Resume the migration task](https://docs.pingcap.com/tidb-data-migration/stable/resume-task).
- [Stop the migration task](https://docs.pingcap.com/tidb-data-migration/stable/stop-task).
- [Export and import the cluster data source and task configuration](https://docs.pingcap.com/tidb-data-migration/stable/export-import-config).
- [Handle failed DDL statements](https://docs.pingcap.com/tidb-data-migration/stable/handle-failed-ddl-statements).
- [Pause the migration task](/dm/dm-pause-task.md).
- [Resume the migration task](/dm/dm-resume-task.md).
- [Stop the migration task](/dm/dm-stop-task.md).
- [Export and import the cluster data source and task configuration](/dm/dm-export-import-config.md).
- [Handle failed DDL statements](/dm/handle-failed-ddl-statements.md).
27 changes: 14 additions & 13 deletions migrate-large-mysql-shards-to-tidb.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ In this document, you can migrate data following this procedure:

Before getting started, see the following documents to prepare for the migration task.

- [Deploy a DM Cluster Using TiUP](https://docs.pingcap.com/tidb-data-migration/stable/deploy-a-dm-cluster-using-tiup)
- [Deploy a DM Cluster Using TiUP](/dm/deploy-a-dm-cluster-using-tiup.md)
- [Use TiUP to Deploy Dumpling and Lightning](/migration-tools.md)
- [Privileges required by DM-worker](https://docs.pingcap.com/tidb-data-migration/stable/dm-worker-intro#privileges-required-by-dm-worker)
- [Privileges required by DM-worker](/dm/dm-worker-intro.md)
- [Upstream Permissions for Lightning](/tidb-lightning/tidb-lightning-faq.md#what-are-the-privilege-requirements-for-the-target-database)
- [Downstream Permissions for Dumpling](/dumpling-overview.md#export-data-from-tidbmysql)

Expand Down Expand Up @@ -71,7 +71,7 @@ The target TiKV cluster must have enough disk space to store the imported data.

### Check conflicts for Sharded Tables

If the migration involves merging data from different sharded tables, primary key or unique index conflicts may occur during the merge. Therefore, before migration, you need to take a deep look at the current sharding scheme from the business point of view, and find a way to avoid the conflicts. For more details, see [Handle conflicts between primary keys or unique indexes across multiple sharded tables](https://docs.pingcap.com/tidb-data-migration/stable/shard-merge-best-practices#handle-conflicts-between-primary-keys-or-unique-indexes-across-multiple-sharded-tables). The following is a brief description.
If the migration involves merging data from different sharded tables, primary key or unique index conflicts may occur during the merge. Therefore, before migration, you need to take a deep look at the current sharding scheme from the business point of view, and find a way to avoid the conflicts. For more details, see [Handle conflicts between primary keys or unique indexes across multiple sharded tables](/dm/shard-merge-best-practices.md#handle-conflicts-between-primary-keys-or-unique-indexes-across-multiple-sharded-tables). The following is a brief description.

Assume that tables 1~4 have the same table structure as follows.

Expand Down Expand Up @@ -307,7 +307,7 @@ name: task-test # The name of the task. Should be globally unique.
task-mode: incremental # The mode of the task. "incremental" means full data migration is skipped and only incremental replication is performed.
# Required for incremental replication from sharded tables. By default, the "pessimistic" mode is used.
# If you have a deep understanding of the principles and usage limitations of the optimistic mode, you can also use the "optimistic" mode.
# For more information, see [Merge and Migrate Data from Sharded Tables](https://docs.pingcap.com/zh/tidb-data-migration/stable/feature-shard-merge).
# For more information, see [Merge and Migrate Data from Sharded Tables](https://docs.pingcap.com/zh/tidb/dev/feature-shard-merge/).

shard-mode: "pessimistic"

Expand Down Expand Up @@ -365,7 +365,7 @@ mysql-instances:
# safe-mode: true
```

For more configurations, see [DM Advanced Task Configuration File](https://docs.pingcap.com/tidb-data-migration/stable/task-configuration-file-full/).
For more configurations, see [DM Advanced Task Configuration File](/dm/task-configuration-file-full.md).

Before you start the data migration task, it is recommended to use the `check-task` subcommand in `tiup dmctl` to check if the configuration meets the DM configuration requirements.

Expand All @@ -390,7 +390,7 @@ The parameters in this command are described as follows.
|--master-addr| {advertise-addr} of any DM-master node in the cluster that dmctl connects to. For example: 172.16.10.71:8261 |
|start-task | Starts the data migration task. |

If the task fails to start, first make configuration changes according to the prompt messages from the returned result, and then run the `start-task task.yaml` subcommand in `tiup dmctl` to restart the task. If you encounter problems, see [Handle Errors](https://docs.pingcap.com/tidb-data-migration/stable/error-handling) and [TiDB Data Migration FAQ](https://docs.pingcap.com/tidb-data-migration/stable/faq).
If the task fails to start, first make configuration changes according to the prompt messages from the returned result, and then run the `start-task task.yaml` subcommand in `tiup dmctl` to restart the task. If you encounter problems, see [Handle Errors](/dm/dm-error-handling.md) and [TiDB Data Migration FAQ](/dm/dm-faq.md).

### Check the migration status

Expand All @@ -402,7 +402,7 @@ You can check if there are running migration tasks in the DM cluster and their s
tiup dmctl --master-addr ${advertise-addr} query-status ${task-name}
```

For more information, see [Query Status](https://docs.pingcap.com/zh/tidb-data-migration/stable/query-status).
For more information, see [Query Status](/dm/dm-query-status.md).

### Monitor tasks and view logs

Expand All @@ -423,9 +423,10 @@ You can view the history of a migration task and internal operational metrics th

- [Dumpling](/dumpling-overview.md)
- [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md)
- [Pessimistic mode and optimistic mode](https://docs.pingcap.com/tidb-data-migration/stable/feature-shard-merge)
- [Pause a Data Migration Task](https://docs.pingcap.com/tidb-data-migration/stable/pause-task)
- [Resume a Data Migration Task](https://docs.pingcap.com/tidb-data-migration/stable/resume-task)
- [Stop a Data Migration Task](https://docs.pingcap.com/tidb-data-migration/stable/stop-task)
- [Export and Import Data Sources and Task Configuration of Clusters](https://docs.pingcap.com/tidb-data-migration/stable/export-import-config)
- [Handle Failed DDL Statements](https://docs.pingcap.com/tidb-data-migration/stable/handle-failed-ddl-statements)
- [Pessimistic mode and optimistic mode](/dm/feature-shard-merge.md)
- [Pause a Data Migration Task](/dm/dm-pause-task.md)
- [Resume a Data Migration Task](/dm/dm-resume-task.md)
- [Stop a Data Migration Task](/dm/dm-stop-task.md)
- [Export and Import Data Sources and Task Configuration of Clusters](/dm/dm-export-import-config.md)
- [Handle Failed DDL Statements](/dm/handle-failed-ddl-statements.md)
- [Handle Errors](/dm/dm-error-handling.md)
20 changes: 10 additions & 10 deletions migrate-large-mysql-to-tidb.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ This document describes how to migrate large datasets from MySQL to TiDB. The wh

## Prerequisites

- [Install DM](https://docs.pingcap.com/tidb-data-migration/stable/deploy-a-dm-cluster-using-tiup).
- [Install DM](/dm/deploy-a-dm-cluster-using-tiup.md).
- [Install Dumpling and TiDB Lightning](/migration-tools.md).
- [Grant the source database and target database privileges required for DM](https://docs.pingcap.com/tidb-data-migration/stable/dm-worker-intro).
- [Grant the source database and target database privileges required for DM](/dm/dm-worker-intro.md).
- [Grant the target database privileges required for TiDB Lightning](/tidb-lightning/tidb-lightning-faq.md#what-are-the-privilege-requirements-for-the-target-database).
- [Grant the source database privileges required for Dumpling](/dumpling-overview.md#export-data-from-tidbmysql).

Expand Down Expand Up @@ -226,7 +226,7 @@ If the import fails, refer to [TiDB Lightning FAQ](/tidb-lightning/tidb-lightnin
# safe-mode: true # If this field is set to true, DM changes INSERT of the data source to REPLACE for the target database, and changes UPDATE of the data source to DELETE and REPLACE for the target database. This is to ensure that when the table schema contains a primary key or unique index, DML statements can be imported repeatedly. In the first minute of starting or resuming an incremental replication task, DM automatically enables the safe mode.
```

The YAML above is the minimum configuration required for the migration task. For more configuration items, refer to [DM Advanced Task Configuration File](https://docs.pingcap.com/tidb-data-migration/stable/task-configuration-file-full).
The YAML above is the minimum configuration required for the migration task. For more configuration items, refer to [DM Advanced Task Configuration File](/dm/task-configuration-file-full.md).

Before you start the migration task, to reduce the probability of errors, it is recommended to confirm that the configuration meets the requirements of DM by running the `check-task` command:

Expand All @@ -253,7 +253,7 @@ If the import fails, refer to [TiDB Lightning FAQ](/tidb-lightning/tidb-lightnin

If the task fails to start, check the prompt message and fix the configuration. After that, you can re-run the command above to start the task.

If you encounter any problem, refer to [DM error handling](https://docs.pingcap.com/tidb-data-migration/stable/error-handling) and [DM FAQ](https://docs.pingcap.com/tidb-data-migration/stable/faq).
If you encounter any problem, refer to [DM error handling](/dm/dm-error-handling.md) and [DM FAQ](/dm/dm-faq.md).

### Check the migration task status

Expand All @@ -265,7 +265,7 @@ To learn whether the DM cluster has an ongoing migration task and view the task
tiup dmctl --master-addr ${advertise-addr} query-status ${task-name}
```

For a detailed interpretation of the results, refer to [Query Status](https://docs.pingcap.com/tidb-data-migration/stable/query-status).
For a detailed interpretation of the results, refer to [Query Status](/dm/dm-query-status.md).

### Monitor the task and view logs

Expand All @@ -280,8 +280,8 @@ When DM is running, DM-worker, DM-master, and dmctl print the related informatio

## What's next

- [Pause the migration task](https://docs.pingcap.com/tidb-data-migration/stable/pause-task).
- [Resume the migration task](https://docs.pingcap.com/tidb-data-migration/stable/resume-task).
- [Stop the migration task](https://docs.pingcap.com/tidb-data-migration/stable/stop-task).
- [Export and import the cluster data source and task configuration](https://docs.pingcap.com/tidb-data-migration/stable/export-import-config).
- [Handle failed DDL statements](https://docs.pingcap.com/tidb-data-migration/stable/handle-failed-ddl-statements).
- [Pause a Data Migration Task](/dm/dm-pause-task.md)
- [Resume a Data Migration Task](/dm/dm-resume-task.md)
- [Stop a Data Migration Task](/dm/dm-stop-task.md)
- [Export and Import Data Sources and Task Configuration of Clusters](/dm/dm-export-import-config.md)
- [Handle Failed DDL Statements](/dm/handle-failed-ddl-statements.md)
Loading

0 comments on commit eb16fa1

Please sign in to comment.