Use Channels rather than BlockingCollection for block processing

[benaadams]

Changes

• Wrapping a ConcurrentQueue in a BlockingCollection burns a ton of CPU spinning; whereas its a pattern ideally suited to Channels; and using true async for rather than TaskCompletionSource for async is lower overhead.
• Faster full sync

Types of changes

What types of changes does your code introduce?

• Optimization
• Refactoring

Testing

Requires testing

• No

[LukaszRozmej]

Recovery thread should also have boosted priority.

My main concern is that before recovery and processing had dedicated threads and now they will compete on thread pool. Should we create a LongRunning task for that or handle it with custom scheduler that will onw a thread?

Can you demonstrate the gain?

[LukaszRozmej]

Why is this needed?

[benaadams]

Is used to determine if tries should be recovered from network and whether the tx pool should be notified of invalidated accounts in its cache (both should only happen from the main processor)

[benaadams]

Recovery thread should also have boosted priority.

Is start of block; is either not doing anything or is doing catch up when main thread is processing and then its competing with the Parallel.For s in main processing

My main concern is that before recovery and processing had dedicated threads and now they will compete on thread pool.

Can't use async on dedicated thread or loose the thread on first await. Don't think is hugely significant; still boost main processing and Recovery doesn't out compete all the parallel processing of main thread.

[LukaszRozmej]

Would be good to remove other BlockingCollections too, especially the long running ones

[benaadams]

Would be good to remove other BlockingCollections too, especially the long running ones

One of them breaks CI, so going to submit as separate PRs

[benaadams]

Fixed; PRs #7846, #7845, #7844,