Rewrite sync Assembler to improve performance. #1530
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, a latch was used to synchronize the user thread reading messages and
the background thread reading from the network. This required two thread switches
per message.
Now, the background thread writes messages to queue, from which the user thread
reads. This allows passing several frames at each thread switch, reducing the
overhead.
With this server code:
async def test(websocket):
for i in range(int(await websocket.recv())):
await websocket.send(f"{{\"iteration\": {i}}}")
async with serve(test, "localhost", 8765) as server:
await server.serve_forever()
and this client code:
with connect("ws://localhost:8765", compression=None) as websocket:
websocket.send("1_000_000")
for message in websocket:
pass
an unscientific benchmark (running it on my laptop) shows a 2.5x speedup,
going from 11 seconds to 4.4 seconds. Setting a very large recv_bufsize
and max_size doesn't yield significant further improvement.
Flow control was tested by inserting debug logs in maybe_pause/resume()
and by measuring the wait for the recv_flow_control lock. It showed the
expected behavior of pausing and unpausing coupled with some wait time.
The new implementation mirrors the asyncio implementation and gains the
option to prevent or force decoding of frames.
Fix #1376 for the threading implementation.
aaugustin
force-pushed
the
rewrite-sync-assembler
branch
from
fa78d82 to
1387c97
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, a latch was used to synchronize the user thread reading messages and the background thread reading from the network. This required two thread switches per message.
Now, the background thread writes messages to queue, from which the user thread reads. This allows passing several frames at each thread switch, reducing the overhead.
With this server code::
and this client code::
an unscientific benchmark (running it on my laptop) shows a 2.5x speedup, going from 11 seconds to 4.4 seconds. Setting a very large recv_bufsize and max_size doesn't yield significant further improvement.
The new implementation mirrors the asyncio implementation and gains the option to prevent or force decoding of frames. Refs #1376.