Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.3版本,cluster与cluster同步时报错 #99

Open
sookey opened this issue Feb 19, 2024 · 3 comments
Open

3.3版本,cluster与cluster同步时报错 #99

sookey opened this issue Feb 19, 2024 · 3 comments

Comments

@sookey
Copy link

sookey commented Feb 19, 2024

不清楚是否是因为key里有乱码。该报错中提到的阈值能否调整?乱码key能否跳过?

报错如下:
2024-02-19 09:32:37.843|[ WARN ]|[threadPoolTaskExecutor-63] -> MemoryAndSqliteTaskStatusManger#brokenStatusAndLog:211 | 任务Id【71D030FF17DF42E082833B9EDF508404】任务 启动/运行异常停止 ,Class【class syncer.transmission.queue.SendCommandWithOutQueue】,异常原因【被抛弃key数量到达阈值[1],exception reason[null->RdbCommandSendStrategy-> AofCommandSendStrategy->DbMappingStrategy->OffsetUpdateStrategy->DataAnalysisStrategy->SentinelCommandFileterStrategy->CommandProcessingTimeCalculationStrategy]】 2024-02-19 09:32:37.844|[ERROR ]|[threadPoolTaskExecutor-63] -> SendCommandWithOutQueue#run:188 | [71D030FF17DF42E082833B9EDF508404]抛弃 command:[STRING] key:��^@^Et^@ ^\IOT_INSTRUCT_SEQ:10019990034 value [class syncer.replica.event.iter.datatype.BatchedKeyValuePairEvent:"Nw=="] ,class:[class syncer.replica.event.iter.datatype.Batche dKeyStringValueStringEvent]:原因[null->RdbCommandSendStrategy->AofCommandSendStrategy->DbMappingStrategy->OffsetUpdateStrategy->DataAnalysisStrategy->SentinelCommandFi leterStrategy->CommandProcessingTimeCalculationStrategy] syncer.transmission.exception.StartegyNodeException: null->RdbCommandSendStrategy->AofCommandSendStrategy->DbMappingStrategy->OffsetUpdateStrategy->DataAnalysisStrateg y->SentinelCommandFileterStrategy->CommandProcessingTimeCalculationStrategy at syncer.transmission.strategy.commandprocessing.impl.CommandProcessingTimeCalculationStrategy.run(CommandProcessingTimeCalculationStrategy.java:85) at syncer.transmission.strategy.commandprocessing.ProcessingRunStrategyChain.run(ProcessingRunStrategyChain.java:57) at syncer.transmission.queue.SendCommandWithOutQueue.run(SendCommandWithOutQueue.java:78) at syncer.transmission.task.RedisDataSyncTransmissionTask$1.onEvent(RedisDataSyncTransmissionTask.java:227) at syncer.replica.listener.ValueDumpIterableEventListener.onEvent(ValueDumpIterableEventListener.java:65) at syncer.replica.listener.AbstractReplicationListener.doEventListener(AbstractReplicationListener.java:57) at syncer.replica.replication.AbstractReplication.submitEvent(AbstractReplication.java:146) at syncer.replica.parser.RedisSyncerRdbParser.parse(RedisSyncerRdbParser.java:207) at syncer.replica.protocol.DefaultSyncRedisProtocol$2.handle(DefaultSyncRedisProtocol.java:215) at syncer.replica.parser.protocol.ProtocolReplyParser.parse(ProtocolReplyParser.java:102) at syncer.replica.protocol.DefaultSyncRedisProtocol.reply(DefaultSyncRedisProtocol.java:263) at syncer.replica.protocol.DefaultSyncRedisProtocol.parseDump(DefaultSyncRedisProtocol.java:202)

@pingxingshikong
Copy link
Member

你的redis是什么版本? 可以设置阀值大小的,可以在请求参数中添加errorCount值

@sookey
Copy link
Author

sookey commented Feb 22, 2024

感谢回复!
我使用的redis是5.0.7版本。
我查看了文档:docs/api2_v3.md 的edittask接口并编辑了请求进行提交,返回该接口不存在(404),是否我请求有问题?
curl -XPOST 'http://127.0.0.1:8083/api/v2/edittask' -H 'Content-Type: application/json' -H 'X-Token:******' -d '{ "sourceRedisAddress":"0.*.*.200:7380", "targetRedisAddress": "10.*.*.100:7380", "targetRedisType":"CLUSTER", "targetRedisVersion": 5, "taskName": "redis-cluster-sit", "errorCount": 100000, "idleTimeRunsMillis": 100000, "maxPoolSize": 1000, "maxWaitTime": 10000, "minPoolSize": 100, "autostart":true}'
返回:
{"timestamp":"2024-02-22T08:16:02.905+00:00","status":404,"error":"Not Found","message":"","path":"/api/v2/edittask"}

@sookey
Copy link
Author

sookey commented Feb 22, 2024

这个请求我想达到两个目的:
1、新增errorCount参数,达到增加阈值的效果;
2、限制线程数,之前使用发现线程会不断升高最终达到服务器上限(达到过2万多),感觉默认线程池没有生效,不清楚这是不是一个bug。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants