Python server performance

[wangkuiyi]

Thank you for making saxml open source! I'm one of the many people who think this project is very useful.

I am interested in how well the Python server works. I know that GPU kernels can be launched well with Python to make good use of GPUs. Could Python listen to network requests and handle them efficiently?

If not, should this Python server be used as a prototype for a C++ server that loads and runs AOT-compiled JAX programs (https://jax.readthedocs.io/en/latest/aot.html)? Thanks!