[improve][build, test] Merge branch apache/branch-3.1 into 3.1_ds #219
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(cherry picked from commit 1bf7371)
apache#21278) (cherry picked from commit 682eb36) # Conflicts: # pom.xml
…ear (apache#21302) (cherry picked from commit 70d086f)
(cherry picked from commit e76a86e) # Conflicts: # pulsar-common/src/test/java/org/apache/pulsar/common/nar/NarUnpackerTest.java
…pache#21329) (cherry picked from commit c883f50)
…AuthorizationProvider plugin (apache#20939) Fixed apache#20938
…apache#21338) Co-authored-by: Lari Hotari <[email protected]>
…g from BookKeeper to Pulsar. (apache#21340) There is no testing for AutoRecovery replication in Pulsar's current test suite, and we need to cover it. So migrate the replication testing from BookKeeper to Pulsar.
Fix zookeeper related flacky test (cherry picked from commit 9ab7417)
…re building (apache#21365) (cherry picked from commit 421c98a)
apache#21155) The client assumed the connection was inactive, but the Broker assumed the connection was fine. The Client tried to use a new connection to reconnect a producer, then got an error `Producer with name 'st-0-5' is already connected to topic`. - In a connection, the second connection waits for the first connection to complete\. But there is a bug that causes this mechanism to fail\. - If a producer uses a default name, the second registration will override the first one. But it can not override the first one if it uses a specified producer name\. I think this mechanism is to prevent a client from creating two producers with the same name. However, method `Producer.isSuccessorTo` has checked the `producer-id`, and the `producer-id` of multiple producers created by the same client are different. So this mechanism can be deleted. - For `issue 1`: If a producer with the same name tries to use a new connection, async checks the old connection is available. The producers related to the connection that is not available are automatically cleaned up. - For `issue 2`: - Fix the bug that causes a complete producer future will be removed from `ServerCnx`. - Remove the mechanism that prevents a producer with a specified name from overriding the previous producer. (cherry picked from commit bda16b6)
(cherry picked from commit 39235ed) # Conflicts: # pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ClusterMigrationTest.java
### Motivation As https://pulsar.apache.org/docs/3.1.x/sql-overview/, Pulsar SQL is based on [Trino (formerly Presto SQL)](https://trino.io/), which supports UUID type. But now, the UUID field in Avro or JSON schema will be interpreted as VARCHAR. ### Modifications Support decoding UUID form AVRO or JSON schema. (cherry picked from commit 8c70943)
(cherry picked from commit e6cd11f)
) (cherry picked from commit ffc083b)
…le when using extensible load manager (apache#21213) (apache#21314)
…r migration tests (apache#21391) (cherry picked from commit e2c6c08)
…o after bkclient initialize (apache#20944)
…elete heartbeat topic (apache#21360) Co-authored-by: fanjianye <[email protected]> Co-authored-by: Jiwei Guo <[email protected]>
…MinNumRacksPerWriteQuorum not work problem. (apache#21327)
…d avoid recursive update error (apache#21282) (cherry picked from commit aecdb03)
…er (apache#21332) (cherry picked from commit ecd40e4)
(cherry picked from commit 643428b)
(cherry picked from commit fe2d61d)
Signed-off-by: tison <[email protected]>
…pache#21684) Fixes apache#21557 ### Motivation There is a network package loss issue after enabling `haProxyProtocolEnabled`, which leads the error `Checksum failed on the broker` and `Adjusted frame length exceeds`, you can reproduce the issue by the test `testSlowNetwork`. ### Modifications Fix the bug. (cherry picked from commit 6e18874)
…s disable. (apache#21193) Co-authored-by: Jiwe Guo <[email protected]>
… inactive bundle states (apache#21721) (cherry picked from commit 8d16580)
…rnal topics (apache#21729) (cherry picked from commit 88df040)
…#21704) Co-authored-by: Jiwe Guo <[email protected]> (cherry picked from commit 84ea1ca)
…oller (apache#21790) (cherry picked from commit 99d06b9)
…Topic failure when the leadership changes apache#21764 (apache#21801)
…zk when revoke subscription permission (apache#21696) Co-authored-by: fanjianye <[email protected]> Co-authored-by: Jiwe Guo <[email protected]>
Fixes apache#21501 ### Motivation No need to `synchronized` the method `isLeader` in LeaderService See the deadlock stack : ``` "pulsar-external-listener-44525-1": at org.apache.pulsar.functions.worker.FunctionMetaDataManager.giveupLeadership(FunctionMetaDataManager.java) - waiting to lock <0x0000100013535c90> (a org.apache.pulsar.functions.worker.FunctionMetaDataManager) at org.apache.pulsar.functions.worker.LeaderService.becameInactive(LeaderService.java:167) - locked <0x000010001344c6d8> (a org.apache.pulsar.functions.worker.LeaderService) at org.apache.pulsar.client.impl.ConsumerImpl.lambda$activeConsumerChanged$27(ConsumerImpl.java:1136) at org.apache.pulsar.client.impl.ConsumerImpl$$Lambda$2606/0x00007f854ce9cb10.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136) at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run([email protected]/Thread.java:833) "pulsar-web-44514-6": at org.apache.pulsar.functions.worker.LeaderService.isLeader(LeaderService.java) - waiting to lock <0x000010001344c6d8> (a org.apache.pulsar.functions.worker.LeaderService) at org.apache.pulsar.functions.worker.SchedulerManager.scheduleInternal(SchedulerManager.java:200) at org.apache.pulsar.functions.worker.SchedulerManager.schedule(SchedulerManager.java:229) at org.apache.pulsar.functions.worker.FunctionMetaDataManager.updateFunctionOnLeader(FunctionMetaDataManager.java:251) - locked <0x0000100013535c90> (a org.apache.pulsar.functions.worker.FunctionMetaDataManager) at org.apache.pulsar.functions.worker.rest.api.ComponentImpl.internalProcessFunctionRequest(ComponentImpl.java:1775) at org.apache.pulsar.functions.worker.rest.api.ComponentImpl.updateRequest(ComponentImpl.java:996) at org.apache.pulsar.functions.worker.rest.api.FunctionsImpl.registerFunction(FunctionsImpl.java:222) at org.apache.pulsar.broker.admin.impl.FunctionsBase.registerFunction(FunctionsBase.java:196) ```
…mpact is a marker (apache#21718)
…anch-3.1
Conflicts:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImpl.java
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateChannelImpl.java
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractBaseDispatcher.java
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorSubscriptionTest.java
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/TransactionTest.java
pulsar-io/alluxio/pom.xml
pulsar-io/hbase/pom.xml
pulsar-io/influxdb/pom.xml
pulsar-io/jdbc/clickhouse/pom.xml
pulsar-io/jdbc/core/pom.xml
pulsar-io/jdbc/mariadb/pom.xml
pulsar-io/jdbc/openmldb/pom.xml
pulsar-io/jdbc/postgres/pom.xml
pulsar-io/jdbc/sqlite/pom.xml
pulsar-io/redis/pom.xml
pulsar-io/solr/pom.xml
pulsar-package-management/bookkeeper-storage/pom.xml
pulsar-package-management/core/pom.xml
pulsar-package-management/filesystem-storage/pom.xml
pulsar-package-management/pom.xml
testmocks/pom.xml
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
This PR merges apache/branch-3.1 into 3.1_ds.
It merges the commits of branch-3.1 from 20th Dec, 23 till Jan 4th
Modifications
Describe the modifications you've done.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)