Fix concurrent dispatcher #338
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hey there! 👋🏼 We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted.
Details: |
juan518munoz
requested changes
Nov 14, 2024
|
|
||
| let next_expected_batch = Arc::new(Mutex::new(None)); | ||
|
|
||
| let stop_receiver_clone = stop_receiver.clone(); | ||
| let pool_clone = self.pool.clone(); | ||
| let config_clone = self.config.clone(); | ||
| let next_expected_batch_clone = next_expected_batch.clone(); | ||
| let pending_blobs_reader = tokio::spawn(async move { | ||
| let mut pending_blobs = JoinSet::new(); |
There was a problem hiding this comment.
Variable name is not descriptive, I would rename it to something like dispatcher_task_set or disptacher_tasks.
|
|
||
| let next_expected_batch = Arc::new(Mutex::new(None)); | ||
|
|
||
| let stop_receiver_clone = stop_receiver.clone(); | ||
| let pool_clone = self.pool.clone(); | ||
| let config_clone = self.config.clone(); | ||
| let next_expected_batch_clone = next_expected_batch.clone(); | ||
| let pending_blobs_reader = tokio::spawn(async move { | ||
| let mut pending_blobs = JoinSet::new(); | ||
| pending_blobs.spawn(async move { |
There was a problem hiding this comment.
Considering that we now don't have a _reader variable, add a comment above this line stating that this task reads pending blobs from de database.
| @@ -120,99 +127,138 @@ impl DataAvailabilityDispatcher { | |||
| let client = self.client.clone(); | |||
| let request_semaphore = self.request_semaphore.clone(); | |||
| let notifier = Arc::new(Notify::new()); | |||
| let pending_blobs_sender = tokio::spawn(async move { | |||
| let mut spawned_requests = vec![]; | |||
| pending_blobs.spawn(async move { | |||
There was a problem hiding this comment.
Same as the other comment, but clarifying that this task is in charge sending blobs to the dispatcher.
juan518munoz
approved these changes
Nov 14, 2024
juanbono
approved these changes
Nov 14, 2024
gianbelinche
merged commit 598d6e0
into
eigen-client-m0-grafana-metrics
11 of 24 checks passed
10 tasks
juan518munoz
added a commit
that referenced
this pull request
Nov 19, 2024
* initial commit * initial commit * fix ambiguous job name & add docker restart command * fix integration test command * update readme, remove fetching lambda repo * reorganize readme instructions * Fix concurrent dispatcher (#338) * Add shutdown to dispatch batches * Add JoinSet * Format code * Fix unbounded channel breaking authenticated dispersal * Fix pr comments * feat(eigen-client-m0-implementation): optimize concurrent dispatcher (#345) * initial commit * optimize dispatch_batches fn * remove commented code * remove needless variables * optimize inclusion_poller fn * break loop if dispatch fail * remove client_lock variable * switch to retriable err * replace arbitrary value with config --------- Co-authored-by: Gianbelinche <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What ❔
This PR fixes a bug with concurrent dispatcher, where the loop was never stopped so the pending blobs sender thread never stopped, so no error was shown when a dispatched failed.
Why ❔
Checklist
zkstack dev fmtandzkstack dev lint.