-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Pavel Abramov <[email protected]>
- Loading branch information
1 parent
240bea9
commit 752a95f
Showing
2 changed files
with
19 additions
and
41 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,49 +1,27 @@ | ||
# How does `nkv` work exactly? | ||
|
||
You can think about there are two main components in `nkv`: `Server` and `NkvStorage`. | ||
`Server` gives asynchronous access to clients for `NkvStorage` since latter is made | ||
synchronous by design to simplify the architecture and components. `Server` is responsible | ||
to handle client connections. | ||
`nkv` is 3 components: | ||
|
||
`NkvClient` sends requests to `Server` through a connection and `Server` interacts with `NkvStorage` | ||
struct via Rust channels and `NkvStorage` struct interacts then with `StoragePolicy` to store | ||
value on a file system and `Notifier` which sends to `Subscribers` updates whenever value is changed and | ||
send Close message whenever value is deleted. It also stores everything in a Trie structure as a cache to | ||
access elements. | ||
- core library that provides key-value storage and subscription management, implementing all of the above core functionality, communicated via rust channels, usable by rust programs | ||
- standalone server that wraps the core library to provide a socket-based API accessible to any client that communicates with the API over the socket | ||
- standalone client that simplifies communicating with the server over the socket | ||
|
||
From the flow diagram you can see how `NkvStorage` processes requests. | ||
The `NkvCore` library allows you to put, get, delete values and subscribe to recieve notifications on value updates via Rust channels. | ||
When instantiated, it receives a parameter called `StorageEngine`. This is what stores the actual data. `StorageEngine` is a trait, | ||
defined [here](../src/traits.rs), and can be implemented via drivers to different storage engines, for example, memory, persistent on disk, | ||
mysql database, or anything else that can implement get, put and delete. | ||
|
||
![nkv flow diagram](../imgs/nkv_flow.drawio.png) | ||
|
||
`StoragePolicy` and `Notifier` are traits and `NkvStorage` is a cache mechanism for a generic | ||
`Value` which is composed of `StoragePolicy` and `Notifier`. So one can define their own `StoragePolicy` | ||
and/or `Notifier`. That is done because default implementation of `nkv` might not be suitable to your | ||
particular use case, for example you want to store values in a mysql database or you do not want | ||
to keep them at all, or you want to notify your clients via unix domain socket or zmq socket | ||
because your existing system relies on it or you think it's better, etc. In code it will look something | ||
like this: | ||
Note that since it is a generic, one instance of `NkvCore` can use *only one `StorageEngine`*. So you cannot store some values in files, | ||
ignore others and write some to sql database. | ||
|
||
```rust | ||
let mut none_tcp_nkv = NotifyKeyValue::<NoStorage, TcpNotifier>::new(path)?; | ||
let mut redis_tcp_nkv = NotifyKeyValue::<RedisStorage, TcpNotifier>::new(path)?; | ||
let mut mysql_uds_nkv = NotifyKeyValue::<MySqlStorage, UnixSockNotifier>::new(path)?; | ||
``` | ||
Yes, for inter-thread communication you can use `nkv` only for Rust language, since it's written in Rust, if you want to connect | ||
applications, written in other programming language together, you will need to run separate `Server` to achieve that. It uses NkvCore structure | ||
and provides communication interface via some OS-primitive like Unix Domain Socket, TCP socket, anything *programming language agnostic*. | ||
Current `Server` implementation is using TCP socket and can be found [here](../src/srv.rs). It is also using a language agnostic protocol to send messages. | ||
They are marshaled in JSON and send via TCP socket, message format could be found [here](../src/srv.rs). `Server` implements the same 5 endpoints | ||
defined above and it uses TCP sockets to handle all the 5 endpoints. In case of subscription it passes the TCP connection to a structure called `Notifier`, | ||
which is nothing more, but a translator from a channel notifications recieved by `Notifier` from `NkvCore` are forwarded to client via TCP connection Notifier owns. | ||
|
||
In theory, if you don't like the caching mechanism, `Trie` could be turned into a trait and then you can | ||
implement `LRUTrie` or `NoCache`. | ||
You can see flow diagram for put, get, subscribe below: | ||
|
||
## Default implementaiton | ||
|
||
The way `Server` is handling *subscribe* is different from other API calls: TCP connection is not closed, | ||
but rather kept open to send messages to `Subscriber`. So `NkvClient` when called `subscribe()` creates a | ||
`Subscriber` struct and stores it, which in turns in its own thread listens to messages comming from `NkvStorage` | ||
via `Notifier` and newest value would be sent to `tokio::watch` channel. | ||
`NkvStorage` is a map of String Key and a value containing `StoragePolicy` and `Notifier`. | ||
`StoragePolicy` is an object, which stores your state (some variable) on file system, so that | ||
you can restart your application without worrying about data loss. `Notifier` on the other hand | ||
handles the channel between server and client to notify latter if anybody changed value. This | ||
channel is a OS-primitive (sockets) so you can | ||
write clients in any programming language you want. Last two components are `Server` and `NkvClient`. | ||
Former creates `NkvStorage` object and manages its access from asynchronous requests. It does | ||
so by exposing endpoints through some of the OS-primitive (for example, socket), so again, clients could | ||
be in any programming language you like; Latter is used to connect to `Server` and implement the API | ||
![nkv flow diagram](../imgs/nkv_flow.drawio.png) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.