Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
Signed-off-by: Pavel Abramov <[email protected]>
  • Loading branch information
uncleDecart committed Oct 11, 2024
1 parent 240bea9 commit 752a95f
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 41 deletions.
60 changes: 19 additions & 41 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,27 @@
# How does `nkv` work exactly?

You can think about there are two main components in `nkv`: `Server` and `NkvStorage`.
`Server` gives asynchronous access to clients for `NkvStorage` since latter is made
synchronous by design to simplify the architecture and components. `Server` is responsible
to handle client connections.
`nkv` is 3 components:

`NkvClient` sends requests to `Server` through a connection and `Server` interacts with `NkvStorage`
struct via Rust channels and `NkvStorage` struct interacts then with `StoragePolicy` to store
value on a file system and `Notifier` which sends to `Subscribers` updates whenever value is changed and
send Close message whenever value is deleted. It also stores everything in a Trie structure as a cache to
access elements.
- core library that provides key-value storage and subscription management, implementing all of the above core functionality, communicated via rust channels, usable by rust programs
- standalone server that wraps the core library to provide a socket-based API accessible to any client that communicates with the API over the socket
- standalone client that simplifies communicating with the server over the socket

From the flow diagram you can see how `NkvStorage` processes requests.
The `NkvCore` library allows you to put, get, delete values and subscribe to recieve notifications on value updates via Rust channels.
When instantiated, it receives a parameter called `StorageEngine`. This is what stores the actual data. `StorageEngine` is a trait,
defined [here](../src/traits.rs), and can be implemented via drivers to different storage engines, for example, memory, persistent on disk,
mysql database, or anything else that can implement get, put and delete.

![nkv flow diagram](../imgs/nkv_flow.drawio.png)

`StoragePolicy` and `Notifier` are traits and `NkvStorage` is a cache mechanism for a generic
`Value` which is composed of `StoragePolicy` and `Notifier`. So one can define their own `StoragePolicy`
and/or `Notifier`. That is done because default implementation of `nkv` might not be suitable to your
particular use case, for example you want to store values in a mysql database or you do not want
to keep them at all, or you want to notify your clients via unix domain socket or zmq socket
because your existing system relies on it or you think it's better, etc. In code it will look something
like this:
Note that since it is a generic, one instance of `NkvCore` can use *only one `StorageEngine`*. So you cannot store some values in files,
ignore others and write some to sql database.

```rust
let mut none_tcp_nkv = NotifyKeyValue::<NoStorage, TcpNotifier>::new(path)?;
let mut redis_tcp_nkv = NotifyKeyValue::<RedisStorage, TcpNotifier>::new(path)?;
let mut mysql_uds_nkv = NotifyKeyValue::<MySqlStorage, UnixSockNotifier>::new(path)?;
```
Yes, for inter-thread communication you can use `nkv` only for Rust language, since it's written in Rust, if you want to connect
applications, written in other programming language together, you will need to run separate `Server` to achieve that. It uses NkvCore structure
and provides communication interface via some OS-primitive like Unix Domain Socket, TCP socket, anything *programming language agnostic*.
Current `Server` implementation is using TCP socket and can be found [here](../src/srv.rs). It is also using a language agnostic protocol to send messages.
They are marshaled in JSON and send via TCP socket, message format could be found [here](../src/srv.rs). `Server` implements the same 5 endpoints
defined above and it uses TCP sockets to handle all the 5 endpoints. In case of subscription it passes the TCP connection to a structure called `Notifier`,
which is nothing more, but a translator from a channel notifications recieved by `Notifier` from `NkvCore` are forwarded to client via TCP connection Notifier owns.

In theory, if you don't like the caching mechanism, `Trie` could be turned into a trait and then you can
implement `LRUTrie` or `NoCache`.
You can see flow diagram for put, get, subscribe below:

## Default implementaiton

The way `Server` is handling *subscribe* is different from other API calls: TCP connection is not closed,
but rather kept open to send messages to `Subscriber`. So `NkvClient` when called `subscribe()` creates a
`Subscriber` struct and stores it, which in turns in its own thread listens to messages comming from `NkvStorage`
via `Notifier` and newest value would be sent to `tokio::watch` channel.
`NkvStorage` is a map of String Key and a value containing `StoragePolicy` and `Notifier`.
`StoragePolicy` is an object, which stores your state (some variable) on file system, so that
you can restart your application without worrying about data loss. `Notifier` on the other hand
handles the channel between server and client to notify latter if anybody changed value. This
channel is a OS-primitive (sockets) so you can
write clients in any programming language you want. Last two components are `Server` and `NkvClient`.
Former creates `NkvStorage` object and manages its access from asynchronous requests. It does
so by exposing endpoints through some of the OS-primitive (for example, socket), so again, clients could
be in any programming language you like; Latter is used to connect to `Server` and implement the API
![nkv flow diagram](../imgs/nkv_flow.drawio.png)
Binary file modified imgs/nkv_flow.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 752a95f

Please sign in to comment.