Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make the KV store configurable via env vars #391

Merged
merged 8 commits into from
Nov 28, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add more docs
masci committed Nov 28, 2024
commit 6ec8ba8654ec1a5d1081677066c9b637f28fdbdf
2 changes: 2 additions & 0 deletions docs/docs/api_reference/llama_deploy/control_plane.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# `control_plane`

::: llama_deploy.control_plane
options:
show_docstring_parameters: true
15 changes: 15 additions & 0 deletions docs/docs/module_guides/llama_deploy/20_core_components.md
Original file line number Diff line number Diff line change
@@ -56,6 +56,21 @@ The control plane is responsible for managing the state of the system, including
- Handling service completion.
- Launching the control plane server.

The state of the system is persisted in a key-value store that by default consists of a simple mapping in memory.
In particular, the state store contains:

- The name and definition of the registered services.
- The active sessions and their relative tasks and event streams.
- The Context, in case the service is of type Workflow,

In case you need a more scalable storage for the system state, you can set the `state_store_uri` field in the Control
Plane configuration to point to one of the databases we support (see
[the Python API reference](../../api_reference/llama_deploy/control_plane.md)) for more details.
Using a scalable storage for the global state is mostly needed when:
- You want to scale the control plane horizontally, and you want every instance to share the same global state.
- The control plane has to deal with high traffic (many services, sessions and tasks).
- The global state needs to be persisted across restarts (for example, workflow contexts are stored in the global state).

## Service

The general structure of a service is as follows:
3 changes: 2 additions & 1 deletion examples/redis_state_store/README.md
Original file line number Diff line number Diff line change
@@ -4,7 +4,8 @@
> This example is mostly based on the [Quick Start](../quick_start/README.md), see there for more details.

We'll be deploying a simple workflow on a local instance of Llama Deploy using Redis as a scalable storage for the
global state. This is mostly needed when you have more than one control plane running concurrently.
global state. See [the Control Plane documentation](https://docs.llamaindex.ai/en/stable/module_guides/llama_deploy/20_core_components/#control-plane)
for an overview of what the global state consists of and when the default storage might not be enough.

Before starting Llama Deploy, use Docker compose to start the Redis container and run it in the background:

20 changes: 16 additions & 4 deletions llama_deploy/control_plane/config.py
Original file line number Diff line number Diff line change
@@ -2,6 +2,7 @@
from urllib.parse import urlparse

from llama_index.core.storage.kvstore.types import BaseKVStore
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict


@@ -16,14 +17,25 @@ class ControlPlaneConfig(BaseSettings):
tasks_store_key: str = "tasks"
session_store_key: str = "sessions"
step_interval: float = 0.1
host: str = "127.0.0.1"
port: int = 8000
host: str = Field(
default="127.0.0.1",
description="The host where to run the control plane server",
)
port: int = Field(
default=8000, description="The TCP port where to bind the control plane server"
)
internal_host: str | None = None
internal_port: int | None = None
running: bool = True
cors_origins: List[str] | None = None
topic_namespace: str = "llama_deploy"
state_store_uri: str | None = None
topic_namespace: str = Field(
default="llama_deploy",
description="The prefix used in the message queue topic to namespace messages from this control plane",
)
state_store_uri: str | None = Field(
default=None,
description="The connection URI of the database where to store state. If None, SimpleKVStore will be used",
)

@property
def url(self) -> str: