Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft: [Coral-Trino] Avoid accidental translation of Trino from_unixtime SQL call #465

Closed
wants to merge 1 commit into from
Closed

Draft: [Coral-Trino] Avoid accidental translation of Trino from_unixtime SQL call #465

wants to merge 1 commit into from

Conversation

findinpath
Copy link
Contributor

@findinpath findinpath commented Oct 17, 2023 •
edited
Loading

What changes are proposed in this pull request, and why are they necessary?

Spin-off from #464 to showcase the solution based on TrinoFromUnixtimeFunction SqlOperator implementation.

Because there is no definition of the function trino_from_unixtime in StaticHiveFunctionRegistry, the test com.linkedin.coral.trino.rel2trino.HiveToTrinoConverterTest#testSubstrWithTimestampOperator fails with the exception:

org.apache.calcite.runtime.CalciteContextException: At line 0, column 0: No match found for function signature trino_from_unixtime(<NUMERIC>)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
	at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
	at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4867)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1770)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:288)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:234)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:234)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.type.InferTypes.lambda$static$0(InferTypes.java:46)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1868)
	at com.linkedin.coral.hive.hive2rel.HiveSqlValidator.inferUnknownTypes(HiveSqlValidator.java:51)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:474)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392)
	at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
	at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965)
	at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:940)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:647)
	at com.linkedin.coral.common.utils.TypeDerivationUtil.getRelDataType(TypeDerivationUtil.java:69)
	at com.linkedin.coral.common.transformers.SqlCallTransformer.deriveRelDatatype(SqlCallTransformer.java:68)
	at com.linkedin.coral.trino.rel2trino.transformers.SubstrOperatorTransformer.transform(SubstrOperatorTransformer.java:65)
	at com.linkedin.coral.common.transformers.SqlCallTransformer.apply(SqlCallTransformer.java:53)
	at com.linkedin.coral.common.transformers.SqlCallTransformers.apply(SqlCallTransformers.java:36)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
	at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:868)
	at org.apache.calcite.sql.SqlAsOperator.acceptCall(SqlAsOperator.java:123)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:66)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:80)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:39)
	at org.apache.calcite.sql.SqlNodeList.accept(SqlNodeList.java:153)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
	at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:868)
	at org.apache.calcite.sql.SqlSelectOperator.acceptCall(SqlSelectOperator.java:129)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:66)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at com.linkedin.coral.trino.rel2trino.RelToTrinoConverter.convert(RelToTrinoConverter.java:91)
	at com.linkedin.coral.trino.rel2trino.HiveToTrinoConverterTest.testSubstrWithTimestampOperator(HiveToTrinoConverterTest.java:573)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:701)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:893)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1218)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:758)
	at org.testng.TestRunner.run(TestRunner.java:613)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
	at org.testng.SuiteRunner.run(SuiteRunner.java:240)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1170)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1095)
	at org.testng.TestNG.run(TestNG.java:1007)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:141)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:90)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match found for function signature trino_from_unixtime(<NUMERIC>)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
	at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)

How was this patch tested?

N/A

@findinpath findinpath mentioned this pull request Oct 17, 2023
Closed
@findinpath
Copy link
Contributor Author

Superseded by #467

@findinpath findinpath closed this Oct 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@findinpath