fix(storage-manager): race condition on latest_updates
structure
#1399
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As the lock on the
latest_updates
structure was dropped before the operation on the Storage was performed and re-acquired afterwards, a race condition could have happened as illustrated in the following scenario.Let us assume that the Storage and
latest_updates
have(A, T_A0)
.A first operation is performed with
T_A1 > T_A0
. As the timestamp is greater,latest_updates
will allow it. The lock onlatest_updates
is released.The Storage is updated and has
(A, T_A1)
.In the meantime, another operation is received with
T_A2 > T_A0
. As the timestamp is greater thanT_A0
(latest_updates
was still not updated!), it is allowed.For scheduling reasons, the second operation is performed entirely: the Storage and
latest_updates
both have(A, T_A2)
.The scheduler goes back to finish
T_A1
and inserts inlatest_updates
the pair(A, T_A1)
.The information in the Storage (
(A, T_A2)
) and inlatest_updates
((A, T_A1)
) are no longer coherent.This commit fixes this issue by only releasing the lock on the
latest_updates
structure after having updated it — instead of after checking if the received operation is more recent.Note that this should create very little additional contention: the bulk of the processing is performed when interacting with the Storage, not when checking / updating the
latest_updates
structure.latest_updates
structure after having updated it.