Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

fix: fix bugs for considering eviction in parallel and others #66

Merged
merged 4 commits into from
Apr 30, 2024

Conversation

lanlou1554
Copy link
Collaborator

@lanlou1554 lanlou1554 commented Apr 30, 2024

  1. get_data_from_cache and put_data_to_cache aren't atomic. Think about one thread executes get_data_from_cache very early, and it is swapped out, another thread complete put_data_from_cache for the same key, and this thread wakes up, try to execute put_data_from_cache, it will poll from s3, update cache again, but there is no need. So I still want to use status_of_keys to track all the keys in the mem and disk replacer, and if it is completed, then threads can directly return, which is the most efficient way to handle it. So I fix bugs for these situations.
  2. when a key in status_of_keys is set to completed, then it will forever stay in status_of_keys even if it is evicted, so some later threads cannot find data if it is evicted. We have to remove the key from status_of_keys when it is evicted.
  3. During eviction from mem to disk, we need to use status_of_keys as a huge lock to lock it, otherwise eviction & write conflicts may happen
  4. when main insertion progress fails, it should notify waiters
  5. if no one is waiting, then we don't need to grab the lock for replacers to pin (pin_count is 0)
  6. for disk_store read_data, the key to unpin replacer is wrong

TODO: when mem_replacer wants to evict A from mem to disk, is there any possibility that the status of A is incompleted? (Currently I just use continue to aviod eviction to disk and write to disk conflicts...)

@lanlou1554
Copy link
Collaborator Author

If what I design is correct, then we can close #49

@xx01cyx xx01cyx merged commit 25f46af into main Apr 30, 2024
1 check passed
@xx01cyx xx01cyx deleted the ll/fix_parallel branch April 30, 2024 19:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants