-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compression #112
Merged
Merged
Compression #112
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
fulmicoton
commented
Feb 12, 2024
•
edited
Loading
edited
fulmicoton
force-pushed
the
compression2
branch
3 times, most recently
from
February 12, 2024 07:14
167c913
to
ce8b913
Compare
fulmicoton
commented
Feb 12, 2024
fulmicoton
commented
Feb 12, 2024
fulmicoton
force-pushed
the
compression2
branch
2 times, most recently
from
February 13, 2024 09:12
db47399
to
eaff8b9
Compare
fulmicoton
force-pushed
the
compression2
branch
11 times, most recently
from
February 13, 2024 13:17
4dab87d
to
e7c5728
Compare
fulmicoton
force-pushed
the
compression2
branch
from
February 14, 2024 07:38
e7c5728
to
7bef8e8
Compare
The difficulty in this PR is coming from the fact that we need to observe the maximum transmissible unit, and due to the fact that we want to keep the simplficity of using the same `MessageDigest` object when sending and emitting messages. The solution I went for requires to compress things twice. When building our message object, we rely on a stream compressor that makes it possible to get an upperbound of the resulting payload after we insert an arbitrary serializable object. We then decompose the delta into as many mutations. I had to refactor the delta object, so that we have a proper bijection between a delta object and the list of mutations that were used to build it. The delta object hence encodes the order in which nodes, and values were inserted. The point there is to ensure that the actual serialization will yield the same size as the "simulated one" and hence respect the mtu. The code is a bit simpler that way too. Closes #72
fulmicoton
force-pushed
the
compression2
branch
from
February 14, 2024 08:42
7bef8e8
to
5347eb8
Compare
guilload
approved these changes
Feb 14, 2024
fulmicoton
added a commit
that referenced
this pull request
Feb 15, 2024
The difficulty in this PR is coming from the fact that we need to observe the maximum transmissible unit, and due to the fact that we want to keep the simplficity of using the same `MessageDigest` object when sending and emitting messages. The solution I went for requires to compress things twice. When building our message object, we rely on a stream compressor that makes it possible to get an upperbound of the resulting payload after we insert an arbitrary serializable object. We then decompose the delta into as many mutations. I had to refactor the delta object, so that we have a proper bijection between a delta object and the list of mutations that were used to build it. The delta object hence encodes the order in which nodes, and values were inserted. The point there is to ensure that the actual serialization will yield the same size as the "simulated one" and hence respect the mtu. The code is a bit simpler that way too. Closes #72
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.