[Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps #3680

chenhaipeng · 2024-07-30T03:02:45Z

Search before asking

I had searched in the issues and found no similar issues.

What happened

version info
jdk1.8 ,flink1.14 ,scala-2.11 , dinky1.0.3
when i use command compiled package ,
mvn clean package -U -DskipTests=true -P prod,flink-single-version,scala-2.11,aliyun,flink-1.14,web
i got dinky-release-1.14-1.0.3.tgz, but i doesn't work well and there is the following error

Caused by: java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
	at org.apache.flink.table.planner.delegation.PlannerBase.<init>(PlannerBase.scala:109) ~[flink-table-planner_2.11-1.14.6.jar:1.14.6]
	at org.apache.flink.table.planner.delegation.StreamPlanner.<init>(StreamPlanner.scala:52) ~[flink-table-planner_2.11-1.14.6.jar:1.14.6]
	at org.apache.flink.table.planner.delegation.DefaultPlannerFactory.create(DefaultPlannerFactory.java:61) ~[flink-table-planner_2.11-1.14.6.jar:1.14.6]
	at org.apache.flink.table.factories.PlannerFactoryUtil.createPlanner(PlannerFactoryUtil.java:50) ~[flink-table_2.11-1.14.6.jar:1.14.6]
	at org.dinky.executor.CustomTableEnvironmentImpl.create(CustomTableEnvironmentImpl.java:159) ~[dinky-client-1.14-1.0.3.jar:?]
	at org.dinky.executor.CustomTableEnvironmentImpl.create(CustomTableEnvironmentImpl.java:122) ~[dinky-client-1.14-1.0.3.jar:?]
	at org.dinky.executor.LocalStreamExecutor.createCustomTableEnvironment(LocalStreamExecutor.java:72) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.executor.Executor.init(Executor.java:161) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.executor.LocalStreamExecutor.<init>(LocalStreamExecutor.java:67) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.executor.ExecutorFactory.buildLocalExecutor(ExecutorFactory.java:52) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.executor.ExecutorFactory.buildExecutor(ExecutorFactory.java:44) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.job.JobManager.init(JobManager.java:225) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.job.JobManager.build(JobManager.java:203) ~[dinky-core-1.0.3.jar:?]
	at org.dinky.service.task.FlinkSqlTask.getJobManager(FlinkSqlTask.java:73) ~[dinky-admin-1.0.3.jar:?]
	at org.dinky.service.task.FlinkSqlTask.<init>(FlinkSqlTask.java:52) ~[dinky-admin-1.0.3.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_392]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_392]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_392]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_392]
	at cn.hutool.core.util.ReflectUtil.newInstance(ReflectUtil.java:873) ~[hutool-all-5.8.27.jar:5.8.27]
	at org.dinky.service.task.BaseTask.getTask(BaseTask.java:67) ~[dinky-admin-1.0.3.jar:?]
	at org.dinky.service.impl.TaskServiceImpl.explainTask(TaskServiceImpl.java:481) ~[dinky-admin-1.0.3.jar:?]
	at org.dinky.service.impl.TaskServiceImpl$$FastClassBySpringCGLIB$$22087f7c.invoke(<generated>) ~[dinky-admin-1.0.3.jar:?]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.dinky.aop.UdfClassLoaderAspect.round(UdfClassLoaderAspect.java:55) ~[dinky-admin-1.0.3.jar:?]
	... 89 more

What you expected to happen

At first, I suspected it was an issue with my Scala compilation. I have already changed all the Scala versions in the pom file to 2.11, but the problem still persists.

How to reproduce

compile with scala2.11 and flink1.14 mvn clean package -U -DskipTests=true -P prod,flink-single-version,scala-2.11,aliyun,flink-1.14,web
add into extends
flink-table_2.11-1.4.2.jar
flink-scala_2.11-1.14.6.jar
flink-table-runtime_2.11-1.14.6.jar
flink-table-planner_2.11-1.14.6.jar

Anything else

No response

Version

1.0.2

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

The text was updated successfully, but these errors were encountered:

github-actions · 2024-07-30T03:02:56Z

Hello @chenhaipeng, this issue is about web, so I assign it to @Zzm0809. If you have any questions, you can comment and reply.

你好 @chenhaipeng, 这个 issue 是关于 web 的，所以我把它分配给了 @Zzm0809。如有任何问题，可以评论回复。

chenhaipeng · 2024-07-30T03:05:30Z

Hello @chenhaipeng, this issue is about web, so I assign it to @Zzm0809. If you have any questions, you can comment and reply.

你好 @chenhaipeng, 这个 issue 是关于 web 的，所以我把它分配给了 @Zzm0809。如有任何问题，可以评论回复。

不是一个web的bug， scala的兼容问题，我不知道如何解决，我已经在所有地方指定scala的版本

chenhaipeng · 2024-07-30T03:15:06Z

mvn dependency:tree there is dependency for scala

Zzm0809 · 2024-07-30T03:58:35Z

非 dinky 问题, scala 兼容性问题 , 可添加 scala-library

另: scala-1.11 将在 dinky-v1.1.0 中移除

chenhaipeng · 2024-07-30T06:27:43Z

非 dinky 问题, scala 兼容性问题 , 可添加 scala-library

另: scala-1.11 将在 dinky-v1.1.0 中移除

意思是要添加scal scala-library-2.11.12.jar吗，还是把jar 添加到pom里面，我测试的版本是v1.0.3 基于源码自己编译的

chenhaipeng · 2024-07-30T09:42:19Z

在lib 目录增加scala-library-2.11.12.jar 解决，话说pom 里面是不是需要添加才对

 <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
</dependency>

Zzm0809 · 2024-07-30T12:12:16Z

非 dinky 问题, scala 兼容性问题 , 可添加 scala-library
另: scala-1.11 将在 dinky-v1.1.0 中移除

意思是要添加scal scala-library-2.11.12.jar吗，还是把jar 添加到pom里面，我测试的版本是v1.0.3 基于源码自己编译的

放在部署 lib 下或者 extends 下即可

Zzm0809 · 2024-07-30T12:13:29Z

在lib 目录增加scala-library-2.11.12.jar 解决，话说pom 里面是不是需要添加才对
 <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
</dependency>

只有 scala-1.11 才有可能出现此问题, 在 dinky-v1.1.0 发布后不再支持切换 scala 版本, 默认只有 scala-2.12

chenhaipeng · 2024-07-31T06:15:04Z

在lib 目录增加scala-library-2.11.12.jar 解决，话说pom 里面是不是需要添加才对
 <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
</dependency>
只有 scala-1.11 才有可能出现此问题, 在 dinky-v1.1.0 发布后不再支持切换 scala 版本, 默认只有 scala-2.12

scala-2.11 也有

Zzm0809 · 2024-07-31T14:27:08Z

在lib 目录增加scala-library-2.11.12.jar 解决，话说pom 里面是不是需要添加才对
 <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
</dependency>
只有 scala-1.11 才有可能出现此问题, 在 dinky-v1.1.0 发布后不再支持切换 scala 版本, 默认只有 scala-2.12
scala-2.11 也有

不太明白你意思，官方发布包基于scala-2.12,目前不会有此问题。无法复现。

chenhaipeng · 2024-08-01T11:05:00Z

在lib 目录增加scala-library-2.11.12.jar 解决，话说pom 里面是不是需要添加才对
 <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
</dependency>
只有 scala-1.11 才有可能出现此问题, 在 dinky-v1.1.0 发布后不再支持切换 scala 版本, 默认只有 scala-2.12
scala-2.11 也有
不太明白你意思，官方发布包基于scala-2.12,目前不会有此问题。无法复现。
mvn clean package -U -DskipTests=true -P prod,flink-single-version,scala-2.11,aliyun,flink-1.14,web, 指定为scala-2.11 会出现这个问题

chenhaipeng · 2024-08-01T13:49:09Z

另外测试更新：
指定scala-2.11 没有效果，因为dinky-shaded-paimon 依赖 paimon-common， paimon-common 的包是基于scala-2.12 编译


这个问题无法闭环

Zzm0809 · 2024-08-02T04:02:39Z

另外测试更新：
指定scala-2.11 没有效果，因为dinky-shaded-paimon 依赖 paimon-common， paimon-common 的包是基于scala-2.12 编译


这个问题无法闭环

dev分支不需要指定scala版本

chenhaipeng · 2024-08-02T05:02:37Z

另外测试更新：
指定scala-2.11 没有效果，因为dinky-shaded-paimon 依赖 paimon-common， paimon-common 的包是基于scala-2.12 编译


这个问题无法闭环

dev分支不需要指定scala版本

不是这个意思，拉一个后端的进来，我的意思是，v1.0.3 版本无论是否指定scala-2.11 版本都没有意义，因为引用的paimon 是基于scala-2.12 编译的，导致功能有问题

gaoyan1998 · 2024-08-02T05:39:19Z

另外测试更新：
指定scala-2.11 没有效果，因为dinky-shaded-paimon 依赖 paimon-common， paimon-common 的包是基于scala-2.12 编译


这个问题无法闭环

dev分支不需要指定scala版本

不是这个意思，拉一个后端的进来，我的意思是，v1.0.3 版本无论是否指定scala-2.11 版本都没有意义，因为引用的paimon 是基于scala-2.12 编译的，导致功能有问题

1.1.0已经取消paimon强依赖，后续计划作为可选

chenhaipeng · 2024-08-02T08:04:37Z

另外测试更新：
指定scala-2.11 没有效果，因为dinky-shaded-paimon 依赖 paimon-common， paimon-common 的包是基于scala-2.12 编译


这个问题无法闭环

dev分支不需要指定scala版本

不是这个意思，拉一个后端的进来，我的意思是，v1.0.3 版本无论是否指定scala-2.11 版本都没有意义，因为引用的paimon 是基于scala-2.12 编译的，导致功能有问题

1.1.0已经取消paimon强依赖，后续计划作为可选

v1.1.0 官方没有提供了scala-2.11的版本，如果我修改pom.xml 的scala 依赖为scala-2.11 功能有影响吗，我这边项目比较旧，依赖的组件还是hadoop 是2.7.5 ，flink 是1.14.6-scala-2.11

aiwenmo · 2024-08-02T14:20:38Z

scala-2.12 的 dinky 往 scala-2.11 的flink 提交应该可以兼容，试试看

chenhaipeng · 2024-08-02T16:11:38Z

scala-2.12 的 dinky 往 scala-2.11 的flink 提交应该可以兼容，试试看

我重新用dinky 1.1.0, 修改里面pom scala 版本，已经测试可以了

chenhaipeng added Bug Something isn't working Waiting for reply Waiting for reply labels Jul 30, 2024

github-actions bot assigned Zzm0809 Jul 30, 2024

chenhaipeng changed the title ~~[Bug] [Module Name] Bug title~~ [Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps Jul 30, 2024

Zzm0809 added FAQ Frequently Asked Questions and removed Bug Something isn't working Waiting for reply Waiting for reply labels Jul 30, 2024

Zzm0809 closed this as completed Aug 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps #3680

[Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps #3680

chenhaipeng commented Jul 30, 2024 •

edited

Loading

github-actions bot commented Jul 30, 2024

chenhaipeng commented Jul 30, 2024

chenhaipeng commented Jul 30, 2024

Zzm0809 commented Jul 30, 2024 •

edited

Loading

chenhaipeng commented Jul 30, 2024 •

edited

Loading

chenhaipeng commented Jul 30, 2024 •

edited

Loading

Zzm0809 commented Jul 30, 2024

Zzm0809 commented Jul 30, 2024

chenhaipeng commented Jul 31, 2024

Zzm0809 commented Jul 31, 2024

chenhaipeng commented Aug 1, 2024

chenhaipeng commented Aug 1, 2024

Zzm0809 commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

gaoyan1998 commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

aiwenmo commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

[Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps #3680

[Bug] flink1.14 ,scala-2.11 java.lang.NoSuchMethodError: scala.Predef$.refArrayOps #3680

Comments

chenhaipeng commented Jul 30, 2024 • edited Loading

Search before asking

What happened

What you expected to happen

How to reproduce

Anything else

Version

Are you willing to submit PR?

Code of Conduct

github-actions bot commented Jul 30, 2024

chenhaipeng commented Jul 30, 2024

chenhaipeng commented Jul 30, 2024

Zzm0809 commented Jul 30, 2024 • edited Loading

chenhaipeng commented Jul 30, 2024 • edited Loading

chenhaipeng commented Jul 30, 2024 • edited Loading

Zzm0809 commented Jul 30, 2024

Zzm0809 commented Jul 30, 2024

chenhaipeng commented Jul 31, 2024

Zzm0809 commented Jul 31, 2024

chenhaipeng commented Aug 1, 2024

chenhaipeng commented Aug 1, 2024

Zzm0809 commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

gaoyan1998 commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

aiwenmo commented Aug 2, 2024

chenhaipeng commented Aug 2, 2024

chenhaipeng commented Jul 30, 2024 •

edited

Loading

Zzm0809 commented Jul 30, 2024 •

edited

Loading

chenhaipeng commented Jul 30, 2024 •

edited

Loading

chenhaipeng commented Jul 30, 2024 •

edited

Loading