Skip to content
bench

feat: Vector Search #958

Sign in to view logs

feat: Vector Search

feat: Vector Search #958

Triggered via pull request November 26, 2024 00:46
@hinthornwhinthornw
synchronize #2526
wfh/_/pgsearch
Status Success
Total duration 48m 0s
Artifacts –

bench.yml

on: pull_request
benchmark
47m 47s
benchmark
Fit to window
Zoom out
Zoom in

Annotations

2 notices
Benchmark results: libs/checkpoint-postgres/Makefile#L1
......................................... fanout_to_subgraph_10x: Mean +- std dev: 61.1 ms +- 1.4 ms ......................................... fanout_to_subgraph_10x_sync: Mean +- std dev: 51.2 ms +- 0.7 ms ......................................... fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 92.4 ms +- 8.8 ms ......................................... fanout_to_subgraph_10x_checkpoint_sync: Mean +- std dev: 93.4 ms +- 2.2 ms ......................................... fanout_to_subgraph_100x: Mean +- std dev: 609 ms +- 26 ms ......................................... fanout_to_subgraph_100x_sync: Mean +- std dev: 499 ms +- 7 ms ......................................... fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 943 ms +- 45 ms ......................................... fanout_to_subgraph_100x_checkpoint_sync: Mean +- std dev: 925 ms +- 18 ms ......................................... react_agent_10x: Mean +- std dev: 30.9 ms +- 0.7 ms ......................................... react_agent_10x_sync: Mean +- std dev: 22.4 ms +- 0.2 ms ......................................... react_agent_10x_checkpoint: Mean +- std dev: 46.6 ms +- 0.9 ms ......................................... react_agent_10x_checkpoint_sync: Mean +- std dev: 36.8 ms +- 0.5 ms ......................................... react_agent_100x: Mean +- std dev: 342 ms +- 7 ms ......................................... react_agent_100x_sync: Mean +- std dev: 271 ms +- 4 ms ......................................... react_agent_100x_checkpoint: Mean +- std dev: 942 ms +- 9 ms ......................................... react_agent_100x_checkpoint_sync: Mean +- std dev: 844 ms +- 11 ms ......................................... wide_state_25x300: Mean +- std dev: 24.0 ms +- 0.4 ms ......................................... wide_state_25x300_sync: Mean +- std dev: 15.3 ms +- 0.1 ms ......................................... wide_state_25x300_checkpoint: Mean +- std dev: 280 ms +- 3 ms ......................................... wide_state_25x300_checkpoint_sync: Mean +- std dev: 268 ms +- 6 ms ......................................... wide_state_15x600: Mean +- std dev: 28.3 ms +- 0.6 ms ......................................... wide_state_15x600_sync: Mean +- std dev: 17.9 ms +- 0.2 ms ......................................... wide_state_15x600_checkpoint: Mean +- std dev: 482 ms +- 6 ms ......................................... wide_state_15x600_checkpoint_sync: Mean +- std dev: 466 ms +- 6 ms ......................................... wide_state_9x1200: Mean +- std dev: 28.2 ms +- 0.5 ms ......................................... wide_state_9x1200_sync: Mean +- std dev: 17.8 ms +- 0.2 ms ......................................... wide_state_9x1200_checkpoint: Mean +- std dev: 314 ms +- 4 ms ......................................... wide_state_9x1200_checkpoint_sync: Mean +- std dev: 300 ms +- 5 ms
Comparison against main: libs/checkpoint-postgres/Makefile#L1
+----------------------------------------+---------+-----------------------+ | Benchmark | main | changes | +========================================+=========+=======================+ | fanout_to_subgraph_10x_sync | 52.1 ms | 51.2 ms: 1.02x faster | +----------------------------------------+---------+-----------------------+ | react_agent_10x_sync | 22.7 ms | 22.4 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x_checkpoint_sync | 94.6 ms | 93.4 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | wide_state_25x300_sync | 15.5 ms | 15.3 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_sync | 505 ms | 499 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | react_agent_100x_sync | 273 ms | 271 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | wide_state_9x1200_sync | 18.0 ms | 17.8 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | react_agent_10x_checkpoint | 47.0 ms | 46.6 ms: 1.01x faster | +----------------------------------------+---------+-----------------------+ | wide_state_15x600_sync | 18.0 ms | 17.9 ms: 1.00x faster | +----------------------------------------+---------+-----------------------+ | react_agent_100x_checkpoint | 944 ms | 942 ms: 1.00x faster | +----------------------------------------+---------+-----------------------+ | wide_state_15x600 | 28.1 ms | 28.3 ms: 1.01x slower | +----------------------------------------+---------+-----------------------+ | Geometric mean | (ref) | 1.00x faster | +----------------------------------------+---------+-----------------------+ Benchmark hidden because not significant (17): react_agent_10x, fanout_to_subgraph_10x, fanout_to_subgraph_10x_checkpoint, fanout_to_subgraph_100x_checkpoint_sync, react_agent_100x, wide_state_25x300, wide_state_9x1200_checkpoint_sync, wide_state_15x600_checkpoint, react_agent_100x_checkpoint_sync, fanout_to_subgraph_100x_checkpoint, wide_state_15x600_checkpoint_sync, react_agent_10x_checkpoint_sync, wide_state_9x1200, wide_state_25x300_checkpoint_sync, wide_state_25x300_checkpoint, wide_state_9x1200_checkpoint, fanout_to_subgraph_100x