Checkpoints and Recover questions

[lordofthejars]

Hi, thank you very much for developing this project. It mimics the Langgraph project well, but in Java, it is excellent.

I have some questions regarding Checkpoints and recovery, which are more related to a missing piece I don't get in Langgraph philosophy than to a missing feature or some questions.

So I've got a simple graph with two states. I inject the MemorySaver when I build the Graph before compiling. Then I compile, and I've got the instance.

Then, with the Graph compiled, my first question is: Is thread-safe? So I can put in a server and send multiple parallel requests to the Graph?

And related to my first question, I cannot reuse the Graph because it is not thread-safe, then I need to compile for each request (so it is request-scoped), which is ok, but if it is thread-safe, then the MemorySaver instance has all the requests that have happened; the problem is that I have an UUID/ThreadId to identify the executed step, but this is not helpful, as probably it is better to determine the execution with some user id, execution id to identify what a specific user did. Of course, if Graph is not thread-safe and I need one new Graph per request, it is easy to fix this problem.

This is one part; the second part, which I couldn't find an example, is ok. I have all the checkpoints, and in the middle of execution in one node, I got an exception, which I captured and so on; how can I replay from this node down to the graph? (If it is possible to do it), after I fixed the problem?

Thank you very much for your work on the library and sorry for this extensive issue. Hope I can contribute to the project in the near future either with code or with examples.

[bsorrentino]

Hi @lordofthejars, I'll try to answer to your questions as best II can:

1. The Compiled Graph Is thread safe?

   Yes. the Compiled Graph is thread safe. It work on a new (and not shared) copy of state for each execution.

2. Is the Compiled Graph + Checkpoints + MemorySaver thread safe ?

   No. the current implementation of Checkpoints + MemorySaver is not thread safe this until will be completed: Add support of ThreadId in CheckPointSaver  #20.
   Tips 👉: for server implementation the ThreadId could be set with SessionId/UserId

3. Using checkpoints, if in the middle of execution in one node, I got an exception, which I captured and so on; how can I replay from this node down to the graph after I fixed the problem?

   In the current implementation, checkpoints allow only the re-play, so you simply re-run the compiled graph (with checkpoint) and the state will not be created but recovered from the last saved checkpoint. The starting from the a particular node is up to you, this means that your nodes must be bit smarter, understanding from the state if they have to execute their job or just skip it.

Let me know if I have clarified your points in

[lordofthejars]

First of all, thank you very much for your answer.

1 and 2 clear.

3rd point: OK, so the state object is not created, but all nodes are executed, so you need to develop something in the nodes, like skipping the execution if a value has already been calculated.

Thanks

[bsorrentino]

Hi @lordofthejars, for the 3rd point currently it is so.

I've in plan to work on #14 that is support of resume feature (as in langchainjs 👀) that should avoid to implement further logic into nodes

[lordofthejars]

oh that's great. If you need help or discuss any design decision, test it. Just ping me.

Now, I am using langgraph4j in Quarkus and its langchain4j extension. It works probably some kind of Quarus extension would be required too, but for now it works