-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Optimization] TaskBatcher to use ConcurrentHashMap for batching tasks per executor #5099
Labels
Comments
amkhar
pushed a commit
to amkhar/OpenSearch
that referenced
this issue
Jan 11, 2023
…pensearch-project#5099) Signed-off-by: Aman Khare <[email protected]>
6 tasks
amkhar
pushed a commit
to amkhar/OpenSearch
that referenced
this issue
Jan 31, 2023
…pensearch-project#5099) Signed-off-by: Aman Khare <[email protected]>
shwetathareja
pushed a commit
that referenced
this issue
Mar 7, 2023
#5827) * Use ConcurrentHashMap for batching tasks per executor in TaskBatcher (#5099) Signed-off-by: Aman Khare <[email protected]>
opensearch-trigger-bot bot
pushed a commit
that referenced
this issue
Mar 7, 2023
#5827) * Use ConcurrentHashMap for batching tasks per executor in TaskBatcher (#5099) Signed-off-by: Aman Khare <[email protected]> (cherry picked from commit 30e4e5e) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross
pushed a commit
that referenced
this issue
Mar 7, 2023
#5827) (#6566) * Use ConcurrentHashMap for batching tasks per executor in TaskBatcher (#5099) (cherry picked from commit 30e4e5e) Signed-off-by: Aman Khare <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This is closed as part of #5827 |
mingshl
pushed a commit
to mingshl/OpenSearch-Mingshl
that referenced
this issue
Mar 24, 2023
opensearch-project#5827) * Use ConcurrentHashMap for batching tasks per executor in TaskBatcher (opensearch-project#5099) Signed-off-by: Aman Khare <[email protected]> Signed-off-by: Mingshi Liu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
In TaskBatcher.java, for batching it maintains a HashMap
tasksPerBatchingKey
and uses synchronized block to find duplicates using the batching key (which is based on task executor) and update the tasks for the given key. It can use ConcurrentHashMap instead which will only lock the segment and use computeIfPresent for computing updated tasks.OpenSearch/server/src/main/java/org/opensearch/cluster/service/TaskBatcher.java
Lines 97 to 100 in 082f059
Expected behavior
The tasks belonging to different batchingKey can be submitted in parallel and will make submitTasks faster.
The text was updated successfully, but these errors were encountered: