You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using a 1-node cluster, when shutting down Kafka and bringing it back up, pipeline_kafka consumers don't reconnect automatically, and don't consume any new messages:
2019-04-10 13:38:47.600 UTC,,,39,,5cadd114.27,9,,2019-04-10 11:18:44 UTC,,0,LOG,00000,"checkpoint starting: time",,,,,,,,,""
2019-04-10 13:38:48.856 UTC,,,39,,5cadd114.27,10,,2019-04-10 11:18:44 UTC,,0,LOG,00000,"checkpoint complete: wrote 12 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.217 s, sync=0.007 s, total=1.255 s; sync files=12, longest=0.001 s, average=0.000 s; distance=15 kB, estimate=212 kB",,,,,,,,,""
2019-04-10 13:39:21.585 UTC,,,110,,5cadd422.6e,2,,2019-04-10 11:31:46 UTC,13/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 110): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Disconnected (after 7654878ms in state UP)",,,,,,,,,""
2019-04-10 13:39:21.659 UTC,,,111,,5cadd422.6f,2,,2019-04-10 11:31:46 UTC,11/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 111): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Disconnected (after 7654852ms in state UP)",,,,,,,,,""
2019-04-10 13:39:21.769 UTC,,,109,,5cadd422.6d,2,,2019-04-10 11:31:46 UTC,9/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 109): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Disconnected (after 7654907ms in state UP)",,,,,,,,,""
2019-04-10 13:39:21.901 UTC,,,112,,5cadd422.70,3,,2019-04-10 11:31:46 UTC,14/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 112): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Disconnected (after 7654893ms in state UP)",,,,,,,,,""
2019-04-10 13:41:33.403 UTC,,,110,,5cadd422.6e,3,,2019-04-10 11:31:46 UTC,13/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 110): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Connect to ipv4#10.100.124.90:9092 failed: Connection timed out (after 131788ms in state CONNECT)",,,,,,,,,""
2019-04-10 13:41:33.433 UTC,,,111,,5cadd422.6f,3,,2019-04-10 11:31:46 UTC,11/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 111): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Connect to ipv4#10.100.124.90:9092 failed: Connection timed out (after 131788ms in state CONNECT)",,,,,,,,,""
2019-04-10 13:41:33.597 UTC,,,109,,5cadd422.6d,3,,2019-04-10 11:31:46 UTC,9/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 109): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Connect to ipv4#10.100.124.90:9092 failed: Connection timed out (after 131788ms in state CONNECT)",,,,,,,,,""
2019-04-10 13:41:33.727 UTC,,,112,,5cadd422.70,4,,2019-04-10 11:31:46 UTC,14/0,0,LOG,00000,"[pipeline_kafka] foobar_transactions_stream <- foobar.public.transactions (PID 112): librdkafka error: [thrd:kafka00:9092/bootstrap]: kafka00:9092/0: Connect to ipv4#10.100.124.90:9092 failed: Connection timed out (after 131789ms in state CONNECT)",,,,,,,,,""
What's worse, it seems that the last processed offset got corrupted somehow:
When using a 1-node cluster, when shutting down Kafka and bringing it back up, pipeline_kafka consumers don't reconnect automatically, and don't consume any new messages:
What's worse, it seems that the last processed offset got corrupted somehow:
Expected behavior: Consumers reconnect automatically without user's intervention, and starts processing new messages.
The text was updated successfully, but these errors were encountered: