Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TIS 实时同步Mysql 凌晨报错了，如果mysql数据库没有新数据进来时候，TIS会一直链接吗？报： Connection reset #405

Open
andyuan1000 opened this issue Dec 27, 2024 · 1 comment
Open

TIS 实时同步Mysql 凌晨报错了，如果mysql数据库没有新数据进来时候，TIS会一直链接吗？报： Connection reset #405

andyuan1000 opened this issue Dec 27, 2024 · 1 comment

Comments

@andyuan1000
Copy link

andyuan1000 commented Dec 27, 2024 •
edited by baisui1981
Loading

报错日志如下：

2024-12-27 02:22:01,309 ERROR io.debezium.connector.mysql.MySqlStreamingChangeEventSource  [] - Error during binlog processing. Last offset stored = {transaction_id=null, ts_sec=1735221945, file=mysql-bin.000332, pos=835525549, server_id=2, event=1}, binlog reader near position = mysql-bin.000332/835525549
2024-12-27 02:22:01,309 ERROR io.debezium.connector.mysql.MySqlStreamingChangeEventSource  [] - Error during binlog processing. Last offset stored = {transaction_id=null, ts_sec=1735221946, file=mysql-bin.000332, pos=835525549, server_id=2, event=1}, binlog reader near position = mysql-bin.000332/835525549
2024-12-27 02:22:01,311 ERROR io.debezium.pipeline.ErrorHandler                            [] - Producer failure
io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 3 more
2024-12-27 02:22:01,310 ERROR io.debezium.pipeline.ErrorHandler                            [] - Producer failure
io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) [tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 3 more
2024-12-27 02:22:01,313 INFO  io.debezium.connector.mysql.MySqlStreamingChangeEventSource  [] - Stopped reading binlog after 0 events, last recorded offset: {transaction_id=null, ts_sec=1735221946, file=mysql-bin.000332, pos=835525549, server_id=2, event=1}
2024-12-27 02:22:01,313 INFO  io.debezium.connector.mysql.MySqlStreamingChangeEventSource  [] - Stopped reading binlog after 0 events, last recorded offset: {transaction_id=null, ts_sec=1735221945, file=mysql-bin.000332, pos=835525549, server_id=2, event=1}
2024-12-27 02:22:01,360 ERROR org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager [] - Received uncaught exception.
java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:168) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) [flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at org.apache.flink.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:86) ~[flink-connector-mysql-cdc-3.1.0.jar:3.1.0]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
2024-12-27 02:22:01,366 INFO  io.debezium.jdbc.JdbcConnection                              [] - Connection gracefully closed
2024-12-27 02:22:01,367 INFO  com.dtstack.chunjun.sink.format.BaseRichOutputFormat         [] - taskNumber[0] close()
2024-12-27 02:22:01,367 INFO  com.dtstack.chunjun.connector.starrocks.streamload.StreamLoadManager [] - StarRocks Sink is about to close.
2024-12-27 02:22:01,370 INFO  org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Split fetcher 0 exited.
2024-12-27 02:22:01,805 ERROR org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager [] - Received uncaught exception.
java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:168) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) [flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at org.apache.flink.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:86) ~[flink-connector-mysql-cdc-3.1.0.jar:3.1.0]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
2024-12-27 02:22:01,807 INFO  com.dtstack.chunjun.sink.format.BaseRichOutputFormat         [] - taskNumber[0] close()
2024-12-27 02:22:01,807 INFO  com.dtstack.chunjun.connector.starrocks.streamload.StreamLoadManager [] - StarRocks Sink is about to close.
2024-12-27 02:22:01,808 INFO  io.debezium.jdbc.JdbcConnection                              [] - Connection gracefully closed
2024-12-27 02:22:01,867 INFO  io.debezium.metrics.Metrics                                  [] - Unable to unregister metrics MBean 'debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source' as it was not found
2024-12-27 02:22:01,867 INFO  org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Split fetcher 0 exited.
2024-12-27 02:22:21,367 INFO  com.dtstack.chunjun.dirty.log.LogDirtyDataCollector          [] - Print consumer closed.
2024-12-27 02:22:21,367 INFO  com.dtstack.chunjun.sink.format.BaseRichOutputFormat         [] - subtask[0}] close() finished
2024-12-27 02:22:21,367 INFO  org.apache.flink.connector.base.source.reader.SourceReaderBase [] - Closing Source Reader.
2024-12-27 02:22:21,369 WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: 192.168.9.54:3306:market_new2 -> Process -> t_electronic_back_order_detail_dto2Rowdata -> skipUpdateBeforeEvent -> Sink: t_electronic_back_order_detail (1/1)#0 (2452eb0dcd283e7effba799a6c32a290_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from RUNNING to FAILED with failure cause:
java.lang.RuntimeException: One or more fetchers have encountered exception
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:263) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:185) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:147) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:168) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	... 1 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at org.apache.flink.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:86) ~[flink-connector-mysql-cdc-3.1.0.jar:3.1.0]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
2024-12-27 02:22:21,370 INFO  org.apache.flink.runtime.taskmanager.Task                    [] - Freeing task resources for Source: 192.168.9.54:3306:market_new2 -> Process -> t_electronic_back_order_detail_dto2Rowdata -> skipUpdateBeforeEvent -> Sink: t_electronic_back_order_detail (1/1)#0 (2452eb0dcd283e7effba799a6c32a290_cbc357ccb763df2852fee8c4fc7d55f2_0_0).
2024-12-27 02:22:21,374 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor           [] - Un-registering task and sending final execution state FAILED to JobManager for task Source: 192.168.9.54:3306:market_new2 -> Process -> t_electronic_back_order_detail_dto2Rowdata -> skipUpdateBeforeEvent -> Sink: t_electronic_back_order_detail (1/1)#0 2452eb0dcd283e7effba799a6c32a290_cbc357ccb763df2852fee8c4fc7d55f2_0_0.
2024-12-27 02:22:21,479 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:8, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1, taskHeapMemory=96.000mb (100663293 bytes), taskOffHeapMemory=0 bytes, managedMemory=128.000mb (134217730 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: d4c5fb62ef96250a778f57d04d53439e, jobId: 09dfe39432f9fb5e51e4a04614a0c034).
2024-12-27 02:22:21,480 INFO  org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Remove job 09dfe39432f9fb5e51e4a04614a0c034 from job leader monitoring.
2024-12-27 02:22:21,480 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor           [] - Close JobManager connection for job 09dfe39432f9fb5e51e4a04614a0c034.
2024-12-27 02:22:21,807 INFO  com.dtstack.chunjun.dirty.log.LogDirtyDataCollector          [] - Print consumer closed.
2024-12-27 02:22:21,807 INFO  com.dtstack.chunjun.sink.format.BaseRichOutputFormat         [] - subtask[0}] close() finished
2024-12-27 02:22:21,808 INFO  org.apache.flink.connector.base.source.reader.SourceReaderBase [] - Closing Source Reader.
2024-12-27 02:22:21,808 WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: 192.168.9.54:3306:market_new2 -> Process -> t_electronic_back_order_dto2Rowdata -> skipUpdateBeforeEvent -> Sink: t_electronic_back_order (1/1)#0 (fbb2a8b3264d2e290e66d7de3c62f45d_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from RUNNING to FAILED with failure cause:
java.lang.RuntimeException: One or more fetchers have encountered exception
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:263) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:185) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:147) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:168) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) ~[flink-connector-files-tis-1.18.1.jar:tis-1.18.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	... 1 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at org.apache.flink.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:86) ~[flink-connector-mysql-cdc-3.1.0.jar:3.1.0]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Connection reset
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
	at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:226) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:198) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1036) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[tis-flink-cdc-shade-4-debezium-connector-mysql-4.0.1.jar:?]
	... 1 more

这是mysql配置

1735264299493
@baisui1981
Copy link
Member

wait_timeout 等待时间是 5小时，说明从晚上大概9点 左右就没有增量数据了，然后 半夜2点又开始来增量数据了，这是jdbc connection 已经关闭了，就会报以上错误。解决办法可以是 在flink-cdc 创建mysql connection 配置中 设置一个test sql 让他隔一段时间去执行一个简单的sql，保证mysql 服务端不会关闭连接。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@andyuan1000 @baisui1981