How does concurrency work

[Sinnaeve]

Hi

I'm not really sure of concurrency is handle in langserve.

In the examples I see it seems usually we should do

chain = an_llm_chain
add_routes(app, chain , path="/agent", per_req_config_modifier=per_req_config_modifier)

but then if mutliple calls are made to the endpoint from mutliple users at same time, how does it work ?
are there multiple instances of my chain object created ? Do they run in same threads ?
Is there thing I should avoid in my chain like using an external object that is mutable ?
for example, I'm using langfuse and was thinking of doing something like this to pass langfuse trace id to client:
final_chain_conv_agent = RunnableParallel(passed=agent_executor_conv_agent, myid=lambda x: langfuse_handler.get_trace_id())
but if my chain use a global langfuse_handler object, does that means two calls access the same langfuse_handler and might mixed there id.
If someone could give a high level overview it would be very helpful.

[achswp]

👍 how does langserve serve concurrent requests?
I tried 16 requests at a time and it's getting 2-3 times slower to respond to any request.