Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In [[filter.event-filters]], with "rename" DDLs the matcher is filtered based on the new name rather than old name #11946

Open
kennytm opened this issue Dec 27, 2024 · 0 comments
Labels
affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. found/gs type/bug The issue is confirmed as a bug.

Comments

@kennytm
Copy link
Contributor

kennytm commented Dec 27, 2024

What did you do?

  1. Create 3 tables:

    use test;
    create table a1 (pk int primary key, val int);
    create table a2 like a1;
    create table b3 like a1;
  2. Create a changefeed with configuration:

    [filter]
    rules = ['test.a*', 'test.b*']
    [[filter.event-filters]]
    matcher = ['test.a*']
    ignore-event = ['all ddl']
    cdc cli changefeed create --sink-uri='file:///tmp/test-cdc-storage?protocol=canal-json' --config=cfg.toml -c xxx
  3. Perform the following renames:

    rename table a1 to b1;
    rename table a2 to c2;
    rename table b3 to a3;

What did you expect to see?

As explained in https://docs.pingcap.com/tidb/stable/ticdc-ddl#ddl-replication-considerations-for-renaming-tables, since filter.rules is looking at the old name, we expect filter.event-filters[].matcher is also looking at the old name too. That means the downstream should ignore the first two renames and only replicate the last.

-- rename table a1 to b1;  -- ignored
-- rename table a2 to c2;  -- ignored
rename table b3 to a3;  -- replicated

What did you see instead?

The filter.event-filters[].matcher rule actually looks at the new name, so instead the first two DDLs are replicated while the last one is ignored.

rename table a1 to b1;  -- incorrectly replicated
rename table a2 to c2;  -- incorrectly replicated
-- rename table b3 to a3;  -- incorrectly ignored

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

v8.5.0

Upstream TiKV version (execute tikv-server --version):

v8.5.0

TiCDC version (execute cdc version):

v8.5.0
@kennytm kennytm added affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. found/gs type/bug The issue is confirmed as a bug. labels Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. found/gs type/bug The issue is confirmed as a bug.
Projects
None yet
Development

No branches or pull requests

1 participant