[improve][broker] Reduce the pressure from the transaction buffer in rolling restarts #1076
BewareMyPowerpulsar-ci.yaml
on: pull_request
Preconditions
7s
Matrix: unit-tests
Matrix: Build Pulsar java-test-image docker image
CI - Unit - Upload Coverage
0s
Run CodeQL Analysis
0s
Matrix: system-tests
Matrix: flaky-system-tests
Matrix: integration-tests
Delete system test docker image artifact
0s
Delete integration test docker image artifact
0s
Pulsar CI checks completed
6s
Annotations
13 errors and 40 warnings
|
CI - Integration - Messaging
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
|
|
CI - Integration - Messaging
The operation was canceled.
|
|
CI - System - Upload Coverage
Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1
|
|
CI - System - Upload Coverage
Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1
|
|
CI - System - Upload Coverage
Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1
|
|
CI - Unit - Brokers - Broker Group 3
Process completed with exit code 1.
|
|
TransactionAndSchemaAuthZTest.testSchemaAndTransactionAuthorization:
pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TransactionAndSchemaAuthZTest.java#L127
java.lang.NullPointerException: Cannot invoke "org.apache.pulsar.client.admin.PulsarAdmin.topics()" because "this.superUserAdmin" is null
|
|
|
|
SegmentAbortedTxnProcessorTest.testClearSnapshotSegments:
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/SegmentAbortedTxnProcessorTest.java#L231
class org.apache.pulsar.broker.transaction.buffer.impl.LazySnapshotWriter cannot be cast to class org.apache.pulsar.broker.service.SystemTopicTxnBufferSnapshotService$ReferenceCountedWriter (org.apache.pulsar.broker.transaction.buffer.impl.LazySnapshotWriter and org.apache.pulsar.broker.service.SystemTopicTxnBufferSnapshotService$ReferenceCountedWriter are in unnamed module of loader 'app')
|
|
CI - Unit - Brokers - Broker Group 2
Process completed with exit code 1.
|
|
CI - Unit - Brokers - Broker Group 1
Process completed with exit code 1.
|
|
TransactionTest.testCanDeleteNamespaceWhenEnableTxnSegmentedSnapshot:
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/TransactionTest.java#L345
--- An unexpected error occurred in the server ---
Message: {"errorMsg":"","reqId":1558541934568924508, "remote":"localhost/127.0.0.1:41459", "local":"/127.0.0.1:44048"}
Stacktrace:
org.apache.pulsar.client.api.PulsarClientException$TopicDoesNotExistException: {"errorMsg":"","reqId":1558541934568924508, "remote":"localhost/127.0.0.1:41459", "local":"/127.0.0.1:44048"}
at org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:1359)
at org.apache.pulsar.client.impl.ClientCnx.handlePartitionResponse(ClientCnx.java:684)
at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:144)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:840)
|
|
Pulsar CI checks completed
Process completed with exit code 1.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.bookie.rackawareness.BookieRackAffinityMappingTest created 1 new threads. There are now 11 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryChangeLedgerPathTest created 16 new threads. There are now 26 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.bookkeeper.mledger.offload.filesystem.impl.FileSystemManagedLedgerOffloaderTest created 6 new threads. There are now 16 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.testclient.PerfClientUtilsTest created 12 new threads. There are now 22 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.broker.authentication.AuthenticationProviderAthenzTest created 1 new threads. There are now 11 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.broker.authentication.ProxySaslAuthenticationTest created 2 new threads. There are now 12 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.packages.management.storage.bookkeeper.BookKeeperPackagesStorageTest created 2 new threads. There are now 12 threads in total.
|
|
CI - Unit - Other
Summary: Tests in class org.apache.bookkeeper.mledger.impl.ManagedLedgerTest created 10 new threads. There are now 20 threads in total.
|
|
CI - Unit - Brokers - Client Impl
Summary: Tests in class org.apache.pulsar.client.impl.ReaderTest created 8 new threads. There are now 17 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8ClientTests created 7 new threads. There are now 16 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.http.HttpSinkTest created 70 new threads. There are now 79 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8SinkTests created 1 new threads. There are now 17 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.rabbitmq.source.RabbitMQSourceTest created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8AuthTests created 11 new threads. There are now 28 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.rabbitmq.sink.RabbitMQSinkTest created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.kinesis.KinesisSinkTest created 9 new threads. There are now 19 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8SinkRawDataTests created 3 new threads. There are now 31 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.text.HdfsStringSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.ElasticSearchBWCTests created 6 new threads. There are now 37 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.seq.HdfsSequentialSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.seq.HdfsTextSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.opensearch.OpenSearchClientSslTests created 18 new threads. There are now 55 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.text.HdfsStringSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.opensearch.OpenSearchClientTests created 2 new threads. There are now 57 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.ElasticSearchRawRecordTests created 96 new threads. There are now 151 threads in total.
|
|
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSinkTests created 1 new threads. There are now 10 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorRollingRestartTest created 47 new threads. There are now 57 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AutoRecoveryMainTest created 18 new threads. There are now 28 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorCheckAllLedgersTaskTest created 1 new threads. There are now 11 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorBookieTest created 28 new threads. There are now 38 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.TestReplicationWorker created 17 new threads. There are now 27 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.BookieAutoRecoveryTest created 15 new threads. There are now 25 threads in total.
|
|
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorPeriodicCheckTest created 11 new threads. There are now 21 threads in total.
|
|
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.transaction.SegmentAbortedTxnProcessorTest created 71 new threads. There are now 80 threads in total.
|
|
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.transaction.buffer.TopicTransactionBufferTest created 88 new threads. There are now 97 threads in total.
|
|
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.transaction.TopicTransactionBufferRecoverTest created 95 new threads. There are now 104 threads in total.
|
|
CI - Unit - Brokers - Broker Group 1
Summary: Tests in class org.apache.pulsar.broker.transaction.TransactionProduceTest created 95 new threads. There are now 104 threads in total.
|
|
CI - Integration - Cli
Received request to deprovision: The request was cancelled by the remote provider.
|
|
Deprecation notice: v1, v2, and v3 of the artifact actions
The following artifacts were uploaded using a version of actions/upload-artifact that is scheduled for deprecation: "coverage_and_deps_inttest_METRICS.tar.zst", "coverage_and_deps_inttest_STANDALONE.tar.zst", "coverage_and_deps_inttest_TRANSACTION.tar.zst", "coverage_and_deps_systest_FUNCTION.tar.zst", "coverage_and_deps_systest_PULSAR_CONNECTORS_PROCESS.tar.zst", "coverage_and_deps_systest_PULSAR_CONNECTORS_THREAD.tar.zst", "coverage_and_deps_systest_PULSAR_IO.tar.zst", "coverage_and_deps_systest_SCHEMA.tar.zst", "coverage_and_deps_systest_TIERED_FILESYSTEM.tar.zst", "coverage_and_deps_systest_TIERED_JCLOUD.tar.zst", "coverage_and_deps_unittest_BROKER_CLIENT_API.tar.zst", "coverage_and_deps_unittest_BROKER_CLIENT_IMPL.tar.zst", "coverage_and_deps_unittest_BROKER_GROUP_4.tar.zst", "coverage_and_deps_unittest_CLIENT.tar.zst", "coverage_and_deps_unittest_METADATA.tar.zst", "coverage_and_deps_unittest_OTHER.tar.zst", "coverage_and_deps_unittest_PROXY.tar.zst", "coverage_and_deps_unittest_PULSAR_IO.tar.zst", "coverage_and_deps_unittest_PULSAR_IO_ELASTIC.tar.zst", "coverage_and_deps_unittest_PULSAR_IO_KAFKA_CONNECT.tar.zst", "pulsar-java-test-image.zst", "pulsar-maven-repository-binaries.tar.zst", "pulsar-server-distribution.tar.zst", "pulsar-test-latest-version-image.zst".
Please update your workflow to use v4 of the artifact actions.
Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
Jacoco-coverage-report-systests
Expired
|
25.3 MB |
|
|
Unit-BROKER_GROUP_1-dumps
Expired
|
11 KB |
|
|
Unit-BROKER_GROUP_1-surefire-reports
Expired
|
13.2 MB |
|
|
Unit-BROKER_GROUP_2-dumps
Expired
|
31.2 KB |
|
|
Unit-BROKER_GROUP_2-surefire-reports
Expired
|
11.9 MB |
|
|
Unit-BROKER_GROUP_3-surefire-reports
Expired
|
10.1 MB |
|
|
coverage_and_deps_inttest_METRICS.tar.zst
Expired
|
302 KB |
|
|
coverage_and_deps_inttest_STANDALONE.tar.zst
Expired
|
298 KB |
|
|
coverage_and_deps_inttest_TRANSACTION.tar.zst
Expired
|
306 KB |
|
|
coverage_and_deps_systest_FUNCTION.tar.zst
Expired
|
306 KB |
|
|
coverage_and_deps_systest_PULSAR_CONNECTORS_PROCESS.tar.zst
Expired
|
321 KB |
|
|
coverage_and_deps_systest_PULSAR_CONNECTORS_THREAD.tar.zst
Expired
|
325 KB |
|
|
coverage_and_deps_systest_PULSAR_IO.tar.zst
Expired
|
327 KB |
|
|
coverage_and_deps_systest_SCHEMA.tar.zst
Expired
|
310 KB |
|
|
coverage_and_deps_systest_TIERED_FILESYSTEM.tar.zst
Expired
|
91.3 MB |
|
|
coverage_and_deps_systest_TIERED_JCLOUD.tar.zst
Expired
|
302 KB |
|
|
coverage_and_deps_unittest_BROKER_CLIENT_API.tar.zst
Expired
|
2.13 MB |
|
|
coverage_and_deps_unittest_BROKER_CLIENT_IMPL.tar.zst
Expired
|
2.15 MB |
|
|
coverage_and_deps_unittest_BROKER_GROUP_4.tar.zst
Expired
|
2.05 MB |
|
|
coverage_and_deps_unittest_CLIENT.tar.zst
Expired
|
645 KB |
|
|
coverage_and_deps_unittest_METADATA.tar.zst
Expired
|
251 KB |
|
|
coverage_and_deps_unittest_OTHER.tar.zst
Expired
|
9.9 MB |
|
|
coverage_and_deps_unittest_PROXY.tar.zst
Expired
|
127 KB |
|
|
coverage_and_deps_unittest_PULSAR_IO.tar.zst
Expired
|
366 KB |
|
|
coverage_and_deps_unittest_PULSAR_IO_ELASTIC.tar.zst
Expired
|
90.6 KB |
|
|
coverage_and_deps_unittest_PULSAR_IO_KAFKA_CONNECT.tar.zst
Expired
|
173 KB |
|
|
pulsar-java-test-image.zst
Expired
|
279 MB |
|
|
pulsar-maven-repository-binaries.tar.zst
Expired
|
293 MB |
|
|
pulsar-server-distribution.tar.zst
Expired
|
205 MB |
|
|
pulsar-test-latest-version-image.zst
Expired
|
1.72 GB |
|