Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an easy way to implement per-entry TTL and TTI #300

Open
tatsuya6502 opened this issue Aug 10, 2023 · 7 comments
Open

Provide an easy way to implement per-entry TTL and TTI #300

tatsuya6502 opened this issue Aug 10, 2023 · 7 comments

Comments

@tatsuya6502
Copy link
Member

Provide an easy way to implement per-entry TTL and TTI using the Expiry trait.

While Expiry trait provides great flexibility to implement different per-entry expiration policies, its API is a bit low level and some users are having problems using it to implement simple policies like per-entry time-to-live and time-to-idle.

We could simply put an example struct to the examples directory. Or we could make the example struct generic enough and put it under moka::expiry module.

@jchimene
Copy link

starting with things that can expire

pub enum TokenKind {
    Stop,
    Start,
    Help,
    Point(String),
    Unknown,
}

impl From<&TokenKind> for usize {
    fn from(value: &TokenKind) -> Self {
        match value {
            TokenKind::Stop => 0,
            TokenKind::Start => 1,
            TokenKind::Help => 2,
            TokenKind::Point(_) => 3,
            TokenKind::Unknown => 4,
        }
    }
}

I define TTI & TTL tables for per-item expirations. Each contains constants that are based on the current environment.
Lather, rinse, repeat for whatever expiration durations are appropriate

const SCALE: u32 = 10000;

lazy_static! {
    pub static ref FIVE_MIN: Duration = {
        let mut duration = Duration::from_millis(1000 * 60 * 5);
        if "development" == get_run_env() {
            duration = duration / SCALE;
        };
        println!("FIVE_MIN: {:?}", duration);
        duration
    };
}

Tables referenced by the Expiry trait

lazy_static! {
    pub static ref TTI: Vec<Duration> = vec![*FIVE_MIN /* Stop */, *TWO_DAYS /* Start */, *FIVE_MIN /* Help */, *TWO_DAYS /* Point */, *ZERO, /* Unknown */];
}

lazy_static! {
    pub static ref TTL: Vec<Duration> = vec![*FIVE_MIN /* Stop */, *TWENTYEIGHT_DAYS /* Start */, *FIVE_MIN /* Help */, *SIX_HRS /* Point */, *ZERO, /* Unknown */];
}

Expiry trait iimplementation

pub struct ACExpiry {}

impl ACExpiry {
    fn tti(&self, activity: &Activity) -> Duration {
        TTI[usize::from(&activity.token)]
    }

    fn ttl(&self, activity: &Activity) -> Duration {
        TTL[usize::from(&activity.token)]
    }
}

@Swatinem
Copy link
Contributor

As another example, in Symbolicator each cache item is carrying an explicit Instant around which is the TTL. And I created a simple wrapper to make that into an Expiry:

https://github.com/getsentry/symbolicator/blob/55e950327413ffdc13b3f3f0f222c5ca94287c62/crates/symbolicator-service/src/caching/memory.rs#L70-L104

@zaidoon1
Copy link

zaidoon1 commented Sep 5, 2023

it would be great if we can call a function to set a ttl for a specific key like redis: https://redis.io/commands/expire/

use case:

i'm building an in memory rate limiter using moka as the kv store. The high level code is:

struct RateLimitState {
    pub available_tokens: u32,
    pub ttl: Duration,
}

// key: u32
// value: Arc<Mutex<RateLimitState>>

let rate_limit_state = cache.get(key)....
let mut rate_limit_state = rate_limit_state.lock();

// rate limiting logic

// update state
rate_limit_state.available_tokens = newTokens;
 rate_limit_state.ttl = newTtl;

I need to add a mutex because if concurrent requests hit my service asking to increment a counter (implementing token bucket in my case), i need to process the requests serially so I can keep track of the right state. Right now, best I can do is store the ttl in the RateLimit struct and use expire_after_read. Ideally I would use the expire_after_update but updating the struct this way is not recognized as an update to the kv so the new ttl won't be picked up.

With the feature i'm requesting, I'm thinking i would be able to do something like:

struct RateLimitState {
    pub available_tokens: u32,
}

// key: u32
// value: Arc<Mutex<RateLimitState>>

let rate_limit_state = cache.get(key)....
let mut rate_limit_state = rate_limit_state.lock();

// rate limiting logic

// update state
rate_limit_state.available_tokens = newTokens;
 
// update ttl of key
cache.updateTTL(key, newTTl)

This way I also avoid storing the ttl in the struct as I can derive it from other states i'm not showing here.

Maybe there is a better way to do what I want without the "feature i'm requesting"?

@tatsuya6502
Copy link
Member Author

it would be great if we can call a function to set a ttl for a specific key like redis: https://redis.io/commands/expire/

With the feature i'm requesting, I'm thinking i would be able to do something like:

Thank for the feedback! We had similar discussion for and_compute and and_optionally_compute: #227 (comment). I think that is the one you are looking for.

I have not had chance to finalize the spec of and_compute/and_optionally_compute. I am currently working on 6, 7 and 8 on the road map below (almost done), and then I will work on 10. Then finally I will be able to work on and_compute/and_optionally_compute.

https://github.com/orgs/moka-rs/projects/1/views/1

roadmap-2023-09-05

@zaidoon1
Copy link

zaidoon1 commented Sep 9, 2023

it would be great if we can call a function to set a ttl for a specific key like redis: https://redis.io/commands/expire/

With the feature i'm requesting, I'm thinking i would be able to do something like:

Thank for the feedback! We had similar discussion for and_compute and and_optionally_compute: #227 (comment). I think that is the one you are looking for.

I have not had chance to finalize the spec of and_compute/and_optionally_compute. I am currently working on 6, 7 and 8 on the road map below (almost done), and then I will work on 10. Then finally I will be able to work on and_compute/and_optionally_compute.

https://github.com/orgs/moka-rs/projects/1/views/1

roadmap-2023-09-05

thank you for the context!

@tatsuya6502
Copy link
Member Author

Just FYI, I brushed up the Expiry doc for clarity:

@zaidoon1
Copy link

zaidoon1 commented Apr 7, 2024

Now that and_compute is available, is there an example of how I can use this to insert/update entries AND set/update the ttl?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants