-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[core] Support guava cache in CacheManager #4389
Conversation
paimon-common/src/main/java/org/apache/paimon/io/cache/CacheManager.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
Hi @FangYongs @Aitozi , if you think guava cache is better, can you provide the benchmark result? |
paimon-common/src/main/java/org/apache/paimon/io/cache/InternalCache.java
Outdated
Show resolved
Hide resolved
paimon-common/src/main/java/org/apache/paimon/io/cache/InternalCacheBuilder.java
Outdated
Show resolved
Hide resolved
There is still caffeine2 cache in |
Hi @FangYongs @Aitozi , after #4401 , can you test caffiene 2 still have problem? |
91708e7
to
6cb9cee
Compare
@JingsongLi I have rebase #4401 and add benchmark for caffiene 2 and guava. When the cache get a value after it is removed before, the results show that in this case, guava will have higher performance than caffeine 2, and due to the issues of caffeine 2, the more accesses there are, the greater the performance gap. Running benchmark: cache-benchmark cache-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) RelativeOPERATORTEST_cache-benchmark_cache-CAFFEINE 5025 / 5028 0.0 50249690.8 1.0X |
paimon-common/src/main/java/org/apache/paimon/io/cache/CacheManager.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Purpose
Linked issue: close #4388
Support guava cache type in
CacheManager
Tests
Added test case
FileBasedRandomInputViewTest
andFileBasedBloomFilterTest
API and Format
no
Documentation
no