You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Subscribe API creates a queue to keep log entries received from each log stream. The queue, subscribedLogEntriesQueue, uses a buffered channel as an MPSC queue. Since its buffer size is as large as the subscribe span length, it can panic if the span is too broad. For example, users can subscribe to log entries without stopping by passing MaxGLSN as an end position of range to the Subscribe API. However, it makes the channel size too large, so panic occurs.
It is essential to ensure that the queue length is not too large, regardless of the subscription range. The queue size should be small enough to avoid wasting memory but not so small that it requires frequent calls to the Subscribe API. We may need to refactor the overall architecture to address this issue.
The text was updated successfully, but these errors were encountered:
ijsong
changed the title
fix panic caused by too large channel size in pkg/varlog.subscribedLogEntriesQueue
client: fix panic caused by too large channel size in pkg/varlog.subscribedLogEntriesQueue
Dec 5, 2023
Subscribe API creates a queue to keep log entries received from each log stream. The queue, subscribedLogEntriesQueue, uses a buffered channel as an MPSC queue. Since its buffer size is as large as the subscribe span length, it can panic if the span is too broad. For example, users can subscribe to log entries without stopping by passing MaxGLSN as an end position of range to the Subscribe API. However, it makes the channel size too large, so panic occurs.
varlog/pkg/varlog/subscribe.go
Lines 452 to 461 in 9e6713b
It is essential to ensure that the queue length is not too large, regardless of the subscription range. The queue size should be small enough to avoid wasting memory but not so small that it requires frequent calls to the Subscribe API. We may need to refactor the overall architecture to address this issue.
The text was updated successfully, but these errors were encountered: