From 8d7351f381179e518f304afb8064761a29875934 Mon Sep 17 00:00:00 2001 From: Pavel Abramov Date: Mon, 7 Oct 2024 21:37:46 +0200 Subject: [PATCH] Add Dockerfile to run nkv-server Signed-off-by: Pavel Abramov --- Dockerfile | 32 ++++++++++++++++++++++++++++++++ README.md | 22 +++++++++++++++++++++- config.toml | 28 ++++++++++++++++++++++++++++ docs/DESIGN_DECISIONS.md | 10 ++++++++++ 4 files changed, 91 insertions(+), 1 deletion(-) create mode 100644 Dockerfile create mode 100644 config.toml diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..3fae9c4 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,32 @@ +ARG RUST_VERSION=1.80.1 + +FROM rust:${RUST_VERSION}-alpine3.20 AS toolchain +ENV CARGO_BUILD_TARGET="x86_64-unknown-linux-musl" +RUN rustup target add ${CARGO_BUILD_TARGET} +RUN apk add --no-cache musl-dev linux-headers make clang mold +RUN cargo install cargo-chef@0.1.67 + +FROM toolchain AS planer +WORKDIR /app +COPY ./Cargo.toml ./Cargo.lock ./ +COPY ./benches ./benches +RUN cargo chef prepare --recipe-path recipe.json + +FROM toolchain AS cacher +WORKDIR /app +COPY --from=planer /app/recipe.json recipe.json +RUN cargo chef cook --release --recipe-path recipe.json + +FROM toolchain AS builder +WORKDIR /app +COPY --from=cacher /app /app +COPY . /app +RUN echo "Cargo target: $CARGO_BUILD_TARGET" +RUN cargo build --release --target $CARGO_BUILD_TARGET + +FROM scratch +WORKDIR /app +COPY --from=builder /tmp /tmp +COPY --from=builder /app/target/x86_64-unknown-linux-musl/release/nkv-server . +COPY --from=builder /app/target/x86_64-unknown-linux-musl/release/nkv-client . +ENTRYPOINT [] diff --git a/README.md b/README.md index c56c2c4..ab89299 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,27 @@ For more information reffer to [this](./docs/DESIGN_DECISIONS.md) When you have some shared state between services/processes and you also want to be notified when the value is changed ### How do I use it? -In order to use it you should install Rust programming language. If you're running Linux or OSX you can do so via `rustup` + +#### Using docker containers + +You can directly pull docker containers for client and server. They are published with each release: + +Make sure that you have docker [installed](https://docs.docker.com/engine/install/) + +```sh +docker run -d --net=host uncledecart/nkv:latest ./nkv-server "0.0.0.0:4222" +docker run -it --net=host uncledecart/nkv:latest ./nkv-client "4222" +``` + +when using network other than the host, the network implementation may impact network performance, unrelated to nkv itself. +Also docker container builds nkv with musl, not glibc, which introduce slight perfomance degradation but gives way +smaller container size, for more information refer to [Design decisions doc](./docs/DESIGN_DECISIONS.md) + +#### Building locally + +If you want latest version or you want to modify `nkv` to use different storage or notification policies, +you can build and run `nkv` locally. In order to use it you should install Rust programming language. +If you're running Linux or OSX you can do so via `rustup` ```sh curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh diff --git a/config.toml b/config.toml new file mode 100644 index 0000000..b7cc60f --- /dev/null +++ b/config.toml @@ -0,0 +1,28 @@ +[target.aarch64-unknown-linux-musl] +linker = "/usr/bin/clang" +rustflags = [ + "-C", + "link-arg=--ld-path=/usr/bin/mold", + "-C", + "link-arg=--target=aarch64-unknown-linux-musl", +] + +[target.x86_64-unknown-linux-musl] +linker = "/usr/bin/clang" +rustflags = [ + "-C", + "link-arg=--ld-path=/usr/bin/mold", + "-C", + "link-arg=--target=x86_64-unknown-linux-musl", +] + +# FIXME: riscv64 is not yet available in rust:1.80.1-alpine3.20 +# but both clang and mold support it. Alos musl-dev is available in Alpine 3.20 +# [target.riscv64gc-unknown-linux-gnu] +# linker = "/usr/bin/clang" +# rustflags = [ +# "-C", +# "link-arg=--ld-path=/usr/bin/mold", +# "-C", +# "link-arg=--target=riscv64-unknown-linux-musl", +# ] diff --git a/docs/DESIGN_DECISIONS.md b/docs/DESIGN_DECISIONS.md index 8fedf53..afee21b 100644 --- a/docs/DESIGN_DECISIONS.md +++ b/docs/DESIGN_DECISIONS.md @@ -10,3 +10,13 @@ explanation, please, write it here and refer to this doc in the comment section. Send update to multiple clients can be very time-consuming operation, which will lock sending other states, we want to be able to do so in async manner, however, just putting update messages in a separate thread, firing and forgeting about them won't do the trick. Because it so may happen that during updates a new value would come in and then state of the system is undertermined, could be partially updated, could be only old value, could be new value. Having a queue might also lead to infinite queue or DDoS of the system, best way would be a lock-free queue. You can think of buffer of two as a lock free queue, which operates in a following way: when we are updating clients (consumers) with a new value, we lock that value and if any other client wants to write a new value, we do so in another element of our array (assuming that clients do not create a race condition themselves) and since we are Boxing, updating element is a cheap operation. +### Dockerfile + +#### Why use musl and link binary statically? + +We want docker image to be as small as possible, for that we want to use scratch image. +It is easier to just copy one statically linked binary to a container rather than finding out, +what libs are required and then copyting them there (although, it is doable). As of now, only +musl supports static linking. There are known issues with musl performance described in this +[article](https://andygrove.io/2020/05/why-musl-extremely-slow/) but for now we regard this +problem as premature optimisation, we need to figure out, which APIs and clients are useful.