Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Checkpoint Version #2565

Merged
merged 1 commit into from
Nov 28, 2024
Merged

Update Checkpoint Version #2565

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libs/checkpoint/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]

Check notice on line 1 in libs/checkpoint/pyproject.toml

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

.........................................
fanout_to_subgraph_10x: Mean +- std dev: 63.2 ms +- 1.6 ms
.........................................
fanout_to_subgraph_10x_sync: Mean +- std dev: 53.4 ms +- 0.9 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (11.5 ms) is 12% of the mean (98.8 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 98.8 ms +- 11.5 ms
.........................................
fanout_to_subgraph_10x_checkpoint_sync: Mean +- std dev: 97.7 ms +- 1.8 ms
.........................................
fanout_to_subgraph_100x: Mean +- std dev: 687 ms +- 35 ms
.........................................
fanout_to_subgraph_100x_sync: Mean +- std dev: 524 ms +- 8 ms
.........................................
fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 1.06 sec +- 0.04 sec
.........................................
fanout_to_subgraph_100x_checkpoint_sync: Mean +- std dev: 972 ms +- 25 ms
.........................................
react_agent_10x: Mean +- std dev: 32.0 ms +- 0.7 ms
.........................................
react_agent_10x_sync: Mean +- std dev: 23.4 ms +- 0.3 ms
.........................................
react_agent_10x_checkpoint: Mean +- std dev: 49.0 ms +- 1.0 ms
.........................................
react_agent_10x_checkpoint_sync: Mean +- std dev: 39.2 ms +- 1.3 ms
.........................................
react_agent_100x: Mean +- std dev: 355 ms +- 7 ms
.........................................
react_agent_100x_sync: Mean +- std dev: 279 ms +- 4 ms
.........................................
react_agent_100x_checkpoint: Mean +- std dev: 1.02 sec +- 0.01 sec
.........................................
react_agent_100x_checkpoint_sync: Mean +- std dev: 907 ms +- 11 ms
.........................................
wide_state_25x300: Mean +- std dev: 24.8 ms +- 0.4 ms
.........................................
wide_state_25x300_sync: Mean +- std dev: 15.9 ms +- 0.2 ms
.........................................
wide_state_25x300_checkpoint: Mean +- std dev: 284 ms +- 4 ms
.........................................
wide_state_25x300_checkpoint_sync: Mean +- std dev: 268 ms +- 3 ms
.........................................
wide_state_15x600: Mean +- std dev: 29.1 ms +- 0.6 ms
.........................................
wide_state_15x600_sync: Mean +- std dev: 18.3 ms +- 0.2 ms
.........................................
wide_state_15x600_checkpoint: Mean +- std dev: 484 ms +- 5 ms
.........................................
wide_state_15x600_checkpoint_sync: Mean +- std dev: 466 ms +- 5 ms
.........................................
wide_state_9x1200: Mean +- std dev: 28.9 ms +- 0.5 ms
.........................................
wide_state_9x1200_sync: Mean +- std dev: 18.4 ms +- 0.2 ms
.........................................
wide_state_9x1200_checkpoint: Mean +- std dev: 317 ms +- 4 ms
.........................................
wide_state_9x1200_checkpoint_sync: Mean +- std dev: 300 ms +- 5 ms

Check notice on line 1 in libs/checkpoint/pyproject.toml

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+-----------------------------------------+---------+------------------------+
| Benchmark                               | main    | changes                |
+=========================================+=========+========================+
| wide_state_9x1200_checkpoint_sync       | 299 ms  | 300 ms: 1.00x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_25x300_checkpoint_sync       | 267 ms  | 268 ms: 1.01x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_15x600_checkpoint            | 481 ms  | 484 ms: 1.01x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_9x1200_checkpoint            | 313 ms  | 317 ms: 1.01x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_25x300_checkpoint            | 280 ms  | 284 ms: 1.01x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_25x300                       | 24.3 ms | 24.8 ms: 1.02x slower  |
+-----------------------------------------+---------+------------------------+
| wide_state_15x600_sync                  | 17.8 ms | 18.3 ms: 1.03x slower  |
+-----------------------------------------+---------+------------------------+
| wide_state_15x600                       | 28.2 ms | 29.1 ms: 1.03x slower  |
+-----------------------------------------+---------+------------------------+
| wide_state_9x1200                       | 28.0 ms | 28.9 ms: 1.03x slower  |
+-----------------------------------------+---------+------------------------+
| wide_state_25x300_sync                  | 15.4 ms | 15.9 ms: 1.03x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_10x                  | 61.2 ms | 63.2 ms: 1.03x slower  |
+-----------------------------------------+---------+------------------------+
| react_agent_100x_sync                   | 270 ms  | 279 ms: 1.03x slower   |
+-----------------------------------------+---------+------------------------+
| wide_state_9x1200_sync                  | 17.8 ms | 18.4 ms: 1.04x slower  |
+-----------------------------------------+---------+------------------------+
| react_agent_100x                        | 341 ms  | 355 ms: 1.04x slower   |
+-----------------------------------------+---------+------------------------+
| react_agent_10x                         | 30.7 ms | 32.0 ms: 1.04x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_10x_sync             | 51.3 ms | 53.4 ms: 1.04x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_10x_checkpoint_sync  | 93.6 ms | 97.7 ms: 1.04x slower  |
+-----------------------------------------+---------+------------------------+
| react_agent_10x_checkpoint              | 46.9 ms | 49.0 ms: 1.04x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_100x_sync            | 501 ms  | 524 ms: 1.05x slower   |
+-----------------------------------------+---------+------------------------+
| react_agent_10x_sync                    | 22.3 ms | 23.4 ms: 1.05x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_100x_checkpoint_sync | 927 ms  | 972 ms: 1.05x slower   |
+-----------------------------------------+---------+------------------------+
| react_agent_100x_checkpoint_sync        | 848 ms  | 907 ms: 1.07x slower   |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_100x_checkpoint      | 989 ms  | 1.06 sec: 1.08x slower |
+-----------------------------------------+---------+------------------------+
| react_agent_10x_checkpoint_sync         | 36.4 ms | 39.2 ms: 1.08x slower  |
+-----------------------------------------+---------+------------------------+
| fanout_to_subgraph_10x_checkpoint       | 91.7 ms | 98.8 ms: 1.08
name = "langgraph-checkpoint"
version = "2.0.6"
version = "2.0.7"
description = "Library with base interfaces for LangGraph checkpoint savers."
authors = []
license = "MIT"
Expand Down
Loading