You have one of the fastest reads on cache, is this comparison a fair one?

[kolinfluence]

@maypok86 thanks for the work on s3-fifo on golang, would like your opinion on this benchmark, it's based on capacity size and not item capacities so just curious if i can make it fairer for comparison etc.

phuslu/lru and your work on otter is really impressive and each cache serves different needs in different areas.

pls give your opinion on this.
https://github.com/cloudxaas/gocache/blob/main/lrux/bytes/README.md

there's a higher hit ratio version here because it's not affected by the hash collision.
https://github.com/cloudxaas/gocache/blob/main/lru/bytes/README.md

[maypok86]

To be honest, the benchmarks look a bit dubious.

1. Why are you testing other caches on only one thread?
2. It looks like you are limiting the cache size based on the number of bytes. It seems that in these benchmarks, this will lead to your cache storing 100 entries and the rest 1024*100.
3. Why are you asking otter to collect statistics? You don't need it. And statistics collection is focused on concurrent access in otter. This will work quite slowly on a single thread.
4. Cost in your case is the size of the entry in bytes. phuslu/lru does not support this at all, as far as I remember.

[kolinfluence]

@maypok86

1. it's a lot of work to do a lot of benchmarks. i'm starting to appreciate the work you guys have done. but i will add those sharded ones too. it's so new i only had time to add non-sharded version for comparison.
2. i wanted to increase the benchmark to show it stores 1024*100 entries too. you agree it should be done that way? i have no intention whatsoever in purposefully tweaking the parameters to show skewed results.
3. oic. thx for clarification. i will do the sharded version later.
4. yes you are right.

otter definitely has its advantage. however, was wondering if you have implemented
https://github.com/scalalang2/golang-fifo before for comparison because it uses s3-fifo as comparison.

[kolinfluence]

@maypok86 hope to have your debunk on the dubious part. do mention if you have ran the test. just updated with new test using chatgpt to have hit ratio of 100% for all the cache. i understand the time and work required to test all scenarios but let's hope some other contributors can help do that.

[maypok86]

I haven't fully read your benchmarks, but you haven't fixed the main problem.

You don't need multiple types of benchmarks. You need benchmarks in which access to the cache is concurrent. You are currently checking concurrent-safe caches on a single thread. It makes absolutely no sense.

[maypok86]

Well, usually user wants to see benchmarks for a mixed load.

[kolinfluence]

thanks for the suggestion. i'm going to use the benchmark folder you have done. will be back again with results.
also, will get chatgpt4 to suggest a fairer comparison on this.

update: i'm not avoiding the benchmarks anymore at this stage, too many parameters to set for all of them. hope for others to take this forward and update the benchmarks. your "benchmark" folder is very good. will do that someday. i'm just now tied down with some very crucial stuff that i need to do. if anyone can update the benchmark for real world use case etc will be great. i will use it in production for now and report on findings from time to time when super free. thank you so much for your feedback!