Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix a bug that didn't exclude dependencies with classifier #24

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Fix a bug that didn't exclude dependencies with classifier #24

wants to merge 1 commit into from

Conversation

guiyanakuang
Copy link

@guiyanakuang guiyanakuang commented Jun 26, 2022 •
edited
Loading

In the trino 386 version, ck connector. we define the following in the root pom.xml

https://github.com/trinodb/trino/blob/c9f881f955147d6b1a4dee93e9bb64e41e301cd3/pom.xml#L1020-L1031

            <dependency>
                <groupId>com.clickhouse</groupId>
                <artifactId>clickhouse-jdbc</artifactId>
                <version>0.3.2-patch3</version>
                <classifier>all</classifier>
                <exclusions>
                    <exclusion>
                        <groupId>*</groupId>
                        <artifactId>*</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>

From the logs during debugging development using DevelopmentServer we can see that.

2022-06-26T13:33:12.850+0800	DEBUG	startup-0	io.trino.server.PluginManager	    /Users/user/.m2/repository/com/clickhouse/clickhouse-grpc-client/0.3.2-patch3/clickhouse-grpc-client-0.3.2-patch3-netty.jar
2022-06-26T13:33:12.850+0800	DEBUG	startup-0	io.trino.server.PluginManager	    /Users/user/.m2/repository/com/clickhouse/clickhouse-http-client/0.3.2-patch3/clickhouse-http-client-0.3.2-patch3-shaded.jar

clickhouse-grpc-client-0.3.2-patch3-netty.jar and clickhouse-http-client-0.3.2-patch3-shaded.jar are not excluded because they have a classifier set.

This pr is aimed at fixing this bug.

clickhouse-jdbc-0.3.2-patch3-all.jar and clickhouse-grpc-client-0.3.2-patch3-netty.jar have classes with the same name but different contents. If com.clickhouse.client.data.ClickHouseLZ4InputStream in clickhouse-grpc-client-0.3.2-patch3-netty.jar is loaded first, an exception will be thrown, which blocks trino local development.

com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2053)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3966)
	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4863)
	at io.trino.collect.cache.EvictableCache.get(EvictableCache.java:108)
	at io.trino.plugin.jdbc.CachingJdbcClient.get(CachingJdbcClient.java:752)
	at io.trino.plugin.jdbc.CachingJdbcClient.getSchemaNames(CachingJdbcClient.java:148)
	at io.trino.plugin.jdbc.CachingJdbcClient.schemaExists(CachingJdbcClient.java:141)
	at io.trino.plugin.jdbc.DefaultJdbcMetadata.schemaExists(DefaultJdbcMetadata.java:116)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.schemaExists(ClassLoaderSafeConnectorMetadata.java:174)
	at io.trino.metadata.MetadataManager.lambda$schemaExists$1(MetadataManager.java:224)
	at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
	at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:528)
	at io.trino.metadata.MetadataManager.schemaExists(MetadataManager.java:224)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowColumns(ShowQueriesRewrite.java:474)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowColumns(ShowQueriesRewrite.java:216)
	at io.trino.sql.tree.ShowColumns.accept(ShowColumns.java:68)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.rewrite.ShowQueriesRewrite.rewrite(ShowQueriesRewrite.java:213)
	at io.trino.sql.rewrite.StatementRewrite.rewrite(StatementRewrite.java:53)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:76)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:71)
	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:269)
	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:193)
	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:808)
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:135)
	at io.trino.$gen.Trino_dev____20220626_050612_2.call(Unknown Source)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
	at com.clickhouse.client.data.ClickHouseLZ4InputStream.<clinit>(ClickHouseLZ4InputStream.java:20)
	at com.clickhouse.client.http.ClickHouseHttpConnection.getResponseInputStream(ClickHouseHttpConnection.java:245)
	at com.clickhouse.client.http.HttpUrlConnectionImpl.buildResponse(HttpUrlConnectionImpl.java:65)
	at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:203)
	at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
	at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
	at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
	at com.clickhouse.jdbc.ClickHouseDatabaseMetaData.query(ClickHouseDatabaseMetaData.java:71)
	at com.clickhouse.jdbc.ClickHouseDatabaseMetaData.query(ClickHouseDatabaseMetaData.java:54)
	at com.clickhouse.jdbc.ClickHouseDatabaseMetaData.getSchemas(ClickHouseDatabaseMetaData.java:1215)
	at io.trino.plugin.jdbc.BaseJdbcClient.listSchemas(BaseJdbcClient.java:153)
	at io.trino.plugin.jdbc.BaseJdbcClient.getSchemaNames(BaseJdbcClient.java:142)
	at io.trino.plugin.jdbc.ForwardingJdbcClient.getSchemaNames(ForwardingJdbcClient.java:72)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.lambda$getSchemaNames$1(StatisticsAwareJdbcClient.java:92)
	at io.trino.plugin.jdbc.jmx.JdbcApiStats.wrap(JdbcApiStats.java:35)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.getSchemaNames(StatisticsAwareJdbcClient.java:92)
	at io.trino.plugin.jdbc.CachingJdbcClient.lambda$getSchemaNames$1(CachingJdbcClient.java:148)
	at io.trino.collect.cache.EmptyCache.get(EmptyCache.java:66)
	at io.trino.plugin.jdbc.CachingJdbcClient.get(CachingJdbcClient.java:752)
	at io.trino.plugin.jdbc.CachingJdbcClient.getSchemaNames(CachingJdbcClient.java:148)
	at io.trino.plugin.jdbc.CachingJdbcClient.lambda$getSchemaNames$1(CachingJdbcClient.java:148)
	at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4868)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3533)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2282)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2159)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2049)
	... 40 more
Caused by: java.lang.ClassNotFoundException: net.jpountz.lz4.LZ4Factory
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at io.trino.server.PluginClassLoader.loadClass(PluginClassLoader.java:126)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 66 more

@guiyanakuang
Copy link
Author

cc @electrum

@guiyanakuang guiyanakuang mentioned this pull request Jun 29, 2022
Closed
@hashhar
Copy link

hashhar commented Aug 23, 2022

@electrum ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@guiyanakuang @hashhar