Consumers unable to join group until group leader is restarted #1196
schmigware
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Duplicating issue #1195 as this may be the more appropriate forum.
Library version github.com/confluentinc/confluent-kafka-go/v2 v2.0.2
Kakfa broker version: 2.8.2 (Commit:3146c6ff4a24cc24)
Any input on the following would be greatly appreciated,
I have various consumers in consumer group FOO. The consumers are running on a number of k8s clusters and connecting to a common broker.
The consumers subscribe to various topics BAR0, BAR1, BAR2. Each of these topics has 32 partitions.
The observed behaviour is as follows:
Kafka CLI tooling does not reflect ANY consumer as a consumer group member for topic BAR0.
Consumer subscribes to topic BAR0 without error and is assigned an empty partition list.
Kafka CLI tooling does not reflect the consumer as a consumer group member for topic BAR0.
Kafka CLI tooling does not reflect ANY consumer as a consumer group member for topic BAR0.
This is the basic issue. Partitions don't appear to be assigned to any consumer.
Can be resolved by:
Identifying the consumer group leader.
Killing it.
"Stuck" consumer is now assigned partitions.
This does not appear to be an issue with partition assignment within a group. Each consumer is subscribed to an unrelated topic. We are namespacing our Kafka topics to match them 1:1 to k8s clusters and namespaces. Certainly, the "stuck" consumer and the consumer group leader are not subscribing to the same topic.
No errors or warnings are observed in broker / consumer group leader logs.
Beta Was this translation helpful? Give feedback.
All reactions