[da-vinci][dvc] Dropping unassigned partitions #1332
Conversation
| @@ -154,7 +154,13 @@ synchronized CompletableFuture<Void> subscribe( | |||
| // Recreate store config that was potentially deleted by unsubscribe. | |||
| config.store(); | |||
| } | |||
|
|
|||
| ComplementSet<Integer> unassignedPartitionSet = ComplementSet.newSet(subscription); | |||
There was a problem hiding this comment.
Upon bootstrapping, it seems that there is a setting to throw a VeniceException when a version is present.
Bootstrapping is invoked in DaVinciBackend.
Additionally, subscription of partitions is invoked in Da Vinci Client where set of partitions can be passed into the subscribe function directly.
There was a problem hiding this comment.
What can happen is that when you call unsubscribe subscription can be empty (no partitions passed into subscribe), causing it to delete the current version.
Also, I don't think subscribe is the right place to perform the validation. If a user is subscribing in a for loop, not everything would have been added to subscription yet on the first n-1 invocations. It would be better to do this validation somewhere we can guarantee all subscriptions have been performed. DaVinciBackend::bootstrap might be a good place.
Summary, imperative, start upper case, don't end with a period
The objective is to remove unassigned partitions.
Applying this issue to Da Vinci Client which has information over partitions in storage.
This PR is an extension of the PR #1196 [https://github.com//pull/1196] that checks which partitions should be kept [in
StorageService] and applies the check inVeniceServer.Resolves #650
How was this PR tested?
A corresponding test will be written.
Does this PR introduce any user-facing changes?