logger: add klog functionality for segcache-rs #335

brayniac · 2021-08-19T20:45:28Z

Adds a new logger which has command logger specific functionality.

Adds command logging to pelikan_segcache_rs.

Changes to the memcache wire protocol to support providing the
necessary context for command logging.

src/rust/protocol/src/memcache/wire/response/mod.rs

Adds a new logger which has command logger specific functionality. Adds command logging to pelikan_segcache_rs. Changes to the memcache wire protocol to support providing the necessary context for command logging. Fixes `cas` operation in seg storage implementation.

src/rust/config/src/units.rs

src/rust/logger/Cargo.toml

src/rust/storage/seg/src/hashtable/mod.rs

Merge master into feature branch and resolve merge conflicts due to metrics library changes.

src/rust/config/src/klog.rs

Refactors the logger crate prior to review

src/rust/logger/src/backend/file.rs

src/rust/logger/src/receiver.rs

src/rust/logger/src/sender.rs

src/rust/logger/examples/demo-multi.rs

@@ -0,0 +1,49 @@
+// Copyright 2021 Twitter, Inc.


brayniac · 2021-09-23T23:24:08Z

src/rust/logger/src/format.rs

+) -> Result<(), std::io::Error> {
+    writeln!(
+        w,
+        "{} {}",


👍 agree. #341 opened to track that

brayniac · 2021-09-23T23:18:55Z

src/rust/logger/src/lib.rs

+#[macro_export]
+macro_rules! klog {
+    ($($arg:tt)*) => (
+        error!(target: "klog", $($arg)*);


Not strictly true - there's a LevelFilter::Off variant, which will filter Level::Error, but added a comment that conveys what you're asking for. =)

src/rust/logger/src/single.rs

+/// single `Output`.
+pub struct LogBuilder {
+    total_buffer_size: usize,
+    log_message_size: usize,


brayniac marked this pull request as draft August 19, 2021 20:45

thinkingfish reviewed Aug 19, 2021

View reviewed changes

src/rust/protocol/src/memcache/wire/response/mod.rs Outdated Show resolved Hide resolved

brayniac force-pushed the klog branch from faa4607 to f8d3b75 Compare August 31, 2021 14:57

brayniac force-pushed the klog branch from f8058ed to 88b21b3 Compare August 31, 2021 20:39

brayniac changed the title ~~logger: initial commit of klog functionality for segcache-rs~~ logger: add klog functionality Aug 31, 2021

brayniac changed the title ~~logger: add klog functionality~~ logger: add klog functionality for segcache-rs Aug 31, 2021

brayniac marked this pull request as ready for review August 31, 2021 22:04

brayniac commented Aug 31, 2021

View reviewed changes

src/rust/config/src/units.rs Show resolved Hide resolved

brayniac commented Aug 31, 2021

View reviewed changes

src/rust/logger/Cargo.toml Outdated Show resolved Hide resolved

brayniac commented Aug 31, 2021

View reviewed changes

src/rust/storage/seg/src/hashtable/mod.rs Outdated Show resolved Hide resolved

brayniac added 9 commits September 1, 2021 12:07

revert cas fix, moved to separate pr

b8987ca

change to mpmc crate and update rustcommon

c95c24c

use rustcommon-time instead of chrono

11786ae

fix fuzz target

dd5bf54

merge master into feature branch

12bac98

Merge master into feature branch and resolve merge conflicts due to metrics library changes.

address review feedback

57a1355

add license header

6fee0c4

fix pingserver metrics

9e38240

revert expiry change

e6f7028

brayniac commented Sep 9, 2021

View reviewed changes

src/rust/config/src/klog.rs Outdated Show resolved Hide resolved

brayniac added 7 commits September 9, 2021 16:15

cleanup

a543c3f

fix year in license header

084ecc9

refactor logger crate

cb9e347

Refactors the logger crate prior to review

fixes

8d1d80d

add docs and comments

49cbda5

actually use configured buffer sizes

82e9307

remove flexi_logger from Cargo manifest

8d47f45

brayniac commented Sep 16, 2021

View reviewed changes

src/rust/logger/src/backend/file.rs Outdated Show resolved Hide resolved

brayniac commented Sep 16, 2021

View reviewed changes

src/rust/logger/src/backend/file.rs Outdated Show resolved Hide resolved

This comment was marked as spam.

Sign in to view

thinkingfish reviewed Sep 17, 2021

View reviewed changes

src/rust/logger/src/receiver.rs Outdated Show resolved Hide resolved

thinkingfish reviewed Sep 17, 2021

View reviewed changes

src/rust/logger/src/sender.rs Outdated Show resolved Hide resolved

brayniac added 16 commits September 20, 2021 15:57

reimplement logger

1475c47

rustfmt

5033917

add temporary helper function in logger crate

9075aff

fix logging level and format

62a0b99

add sampling functionality

63d4538

missing file

dee9329

hide some implementation details and address clippy lints

241777f

cleanup

e1a492c

split examples

0f0b799

fix examples

530a513

cleanup

cee9702

docs / cleanup

1000896

update core after refactor of logging

9df4d40

add version numbers

3f6607f

add some crate level doc

6908000

add explanation for dropping new messages

c4d9a0d

thinkingfish reviewed Sep 23, 2021

View reviewed changes

brayniac added 3 commits September 23, 2021 16:12

add noplogger

ad638e6

add comments about choosing error level in convenience macro

db48514

helps to commit the noplogger file

6011154

brayniac mentioned this pull request Sep 23, 2021

klog tools #341

Open

fix rustdoc for nop

c065945

thinkingfish reviewed Sep 24, 2021

View reviewed changes

src/rust/logger/src/single.rs Outdated

/// single `Output`.

pub struct LogBuilder {

total_buffer_size: usize,

log_message_size: usize,

This comment was marked as spam.

Sign in to view

brayniac added 2 commits September 24, 2021 08:26

address review feedback

f1de38e

wire up config options for queue depth and message size

dcc84bd

thinkingfish approved these changes Sep 24, 2021

View reviewed changes

brayniac merged commit bea4775 into twitter:master Sep 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

logger: add klog functionality for segcache-rs #335

logger: add klog functionality for segcache-rs #335

brayniac commented Aug 19, 2021 •

edited

Loading

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

brayniac Sep 23, 2021

This comment was marked as spam.

brayniac Sep 23, 2021 •

edited

Loading

This comment was marked as spam.

logger: add klog functionality for segcache-rs #335

logger: add klog functionality for segcache-rs #335

Conversation

brayniac commented Aug 19, 2021 • edited Loading

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

brayniac Sep 23, 2021

Choose a reason for hiding this comment

This comment was marked as spam.

brayniac Sep 23, 2021 • edited Loading

Choose a reason for hiding this comment

This comment was marked as spam.

brayniac commented Aug 19, 2021 •

edited

Loading

brayniac Sep 23, 2021 •

edited

Loading