Add generic thread queues, use them to implement autocue-specific queue #4151
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the way we handle queues in the scheduler.
Previously, we had the following queues:
non_blocking
, for processing fast tasks, typically for the harbor server and telnet.blocking
, for blocking tasks.maybe_blocking
for... ? This was never clear.These were maped to, resp,
non_blocking
,generic
andfast
queues,fast
queues processingmaybe_blocking
tasks.. 🤔.By default, there were
0
fast queues (again, not sure why) and2
generic queues. Generic queues process any available task.With the adoption of
autocue
internal resolution viasource.drop
, we started seeing some interesting queue deadlock:2
generic queues are created2
request.dynamic
orplaylist
are created, each requesting a request resolution.autocue
computation which, in turn, also requires a request resolution.autocue
request resolutionThis PR cleans up the scheduler notion of queues and allows for dedicated queues to fix this dependency issue.
Default queues, named queues
By default, we only have 2 type of queues:
non_blocking
queues, defaults to2
generic
, defaults to2
.We also support named queues, which are queues with any arbitrary string name value. These are created when requested by the user.
New queues can be configured as follows:
Now, each type of queue only process tasks for their own queue, making it possible to control the concurrency of each tasks sent to them.
All the
thread.run.*
functions are updated to remove thefast
parameter and add an explicitqueue
parameter instead, which controls to which queue a given task is sent.autocue
queueautocue
resolutions are not sent to a dedicatedautocue
queue. Only oneautocue
queue is created by default.This is made possible by adding a
thread_queue
parameter torequest.dynamic
and all the operators relying on it.This way, the
autocue
computations happen one at a time by default, which help controlling CPU usage with them, and never prevent request resolution from completing.