-
Notifications
You must be signed in to change notification settings - Fork 998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[core] Avoid building the LookupFile cache repeatedly when the cache … #3859
Conversation
@@ -93,7 +126,7 @@ public static Cache<String, LookupFile> createCache( | |||
} | |||
|
|||
private static int fileWeigh(String file, LookupFile lookupFile) { | |||
return fileKibiBytes(lookupFile.localFile); | |||
return lookupFile.isUsed() ? 0 : fileKibiBytes(lookupFile.localFile); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I completely don't understand why just marking a boolean can save disk space
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marking the boolean is to make the weight to zero while this file is using. This could avoid the lookup file being evicted when putting it to a nearly full cache.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just:
LookupFile lookupFile = lookupFileCache.getIfPresent(file.fileName());
boolean newLookupFile = false;
if (lookupFile is null or lookupFile is closed) {
lookupFile = createLookupFile();
newLookupFile = true;
}
try {
..... use lookup file
} finally {
if (newLookupFile) {
lookupFileCache.put(file.fileName(), lookupFile);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lookupFileCache.put(file.fileName(), newLookupFile);
This still could evict this new create file out directly?
@JingsongLi Have updated to lookup the file before putting to cache We may update to above 3.1.2 later to solve this strange evicting behavior according to this. (In test, it's better). But, the 3.x version requires JDK11 build. |
Thanks! |
+1 |
…is nearly full.
Purpose
Linked issue: close #3855
Tests
API and Format
Documentation