[Bug] [Module Name] After restarting the Dinky service, kerberos authentication becomes invalid

[XXwhite]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1894)
at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1805)
at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1802)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1817)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1812)
at org.apache.hudi.util.StreamerUtil.tableExists(StreamerUtil.java:306)
... 175 more
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1629)
at org.apache.hadoop.ipc.Client.call(Client.java:1575)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:245)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:131)
at com.sun.proxy.$Proxy480.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:1008)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:435)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:170)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:162)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:100)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:366)
at com.sun.proxy.$Proxy481.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1892)
... 181 more

What you expected to happen

After restarting the Dinky service, kerberos authentication becomes invalid，I found that resubmitting the task after saving the cluster configuration again in the registration center can solve this problem。
重启Dinky服务后，kerberos身份验证无效，我发现在注册中心再次保存集群配置后重新提交任务可以解决这个问题，这应该是个bug，期望修复~

How to reproduce

在kerberos认证的环境下，重启服务后提交任务就后认证失败的报错。

Anything else

No response

Version

1.1.0

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Zzm0809]

社区无此环境, 你可以根据官网本地调试文档 进行本地调试寻找问题点及解决方案, 同时欢迎贡献您的 pr

[Zzm0809]

经过查看相关代码: 在运行过程中将认证信息存在了 jvm 中(详见 org.dinky.gateway.yarn.YarnGateway#init), 一旦进程释放, 所有都会销毁,可能造成以上问题

你可以对其进行优化并进行测试.
提供一个思路:

1. 在dinky 中有系统初始化操作, 你可以在系统初始化时,调用 clusterConfigurationService 中testGateway方法对每个集群配置重新走一遍认证的流程, 此思路仅供参考,更多细节可能需要考虑,如有其他方案请在此回复即可 @XXwhite