[kv] Fix out of order exception after delete a not exist row #312
+341
−31
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: #230
This pr is aims to fix the
OutOfOrderSequenceException
after delete a not exist row or input keys withMergeEngine
set toFIRST_ROW
. The reason of this problem is that thekvRecordBatch
send from client may not generated any CDC logs,for example, when client attempts to delete some non-existent keys or
MergeEngine
set toFIRST_ROW
. In this case, the old implements will simply return but not update batchSequence, as doing so would cause aOutOfOrderSequenceException
problem. Therefore, this pr will introduce a filedlastOffsetDelta
in record batch used to calculate the lastOffset of the current batch as: [lastOffset = baseOffset + LastOffsetDelta] instead of [lastOffset = baseOffset + recordCount -1]. The reason for introducing this field is that there might be cases where the offset delta in batch does not match the recordCount. For example, when generating CDC logs for a kv table and sending a batch that only contains the deletion of non-existent kvs, no CDC logs would be generated. However, we need to increment the batchSequence for the corresponding writerId to make sure no {@link OutOfOrderSequenceException} will be thrown. In such a case, we would generate a logRecordBatch with a LastOffsetDelta of 1 but a recordCount of 0.Tests
API and Format
Documentation