Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unused inference server utils #19

Open
plstcharles opened this issue Jul 10, 2020 · 0 comments
Open

Unused inference server utils #19

plstcharles opened this issue Jul 10, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@plstcharles
Copy link
Member

Update message clustering and risk inference was previously executed asynchronously in batches of humans. The input data required for this was packaged into serializable structures and sent via ZMQ sockets to another process with various threads (broker, workers). At some point, the increase in update message counts (probably due to the fragmentation of long encounters to 15-minute chunks) made the serialization & packet exchange process longer than inline clustering+inference on the main thread. This is very sad, but lesson learned: multithreading/multiprocessing in Python SUCKS.

At some point, we will have to pick one of these options:

  • Get rid of all the remote inference logic and go back to a leaner codepath for clustering/inference. That would mean losing the possibility to do async stuff again, and getting rid of a lot of code that worked for a long time.
  • Rewrite the message batching and serialization logic to make I/O as lightweight as possible so that we can start using the inference server again. Back when it was introduced, the speedup for big simulations was ~10x, and it would be great to get that speedup again now that the rest of the code is not as slow as it once was. Delta encoding and compression might come in handy here...
  • Rewrite the entire clustering/inference pipeline in C++, and forget about pickling/multiprocessing altogether. This would require a significant time investment, but "breaking out of the GIL shackle" would definitely allow us to do big simulations.
@plstcharles plstcharles added the enhancement New feature or request label Jul 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant