A flood of annoying messages from kubelet.

[moonek]

Describe the bug
Annoying messages from kubelet every few seconds while using trident.

Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.839371   13267 clientconn.go:106] parsed scheme: ""
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.839400   13267 clientconn.go:106] scheme "" not registered, fallback to default scheme
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.839468   13267 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins/csi.trident.netapp.io/csi.sock  <nil> 0 <nil>}] <nil> <nil>}
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.839478   13267 clientconn.go:948] ClientConn switching balancer to "pick_first"
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.839534   13267 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.840660   13267 clientconn.go:106] parsed scheme: ""
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.840675   13267 clientconn.go:106] scheme "" not registered, fallback to default scheme
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.840697   13267 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins/csi.trident.netapp.io/csi.sock  <nil> 0 <nil>}] <nil> <nil>}
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.840707   13267 clientconn.go:948] ClientConn switching balancer to "pick_first"
Oct 18 13:30:18 worker-02 kubelet[13267]: I1018 13:30:18.840769   13267 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.730361   13267 clientconn.go:106] parsed scheme: ""
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.730395   13267 clientconn.go:106] scheme "" not registered, fallback to default scheme
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.730462   13267 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins/csi.trident.netapp.io/csi.sock  <nil> 0 <nil>}] <nil> <nil>}
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.730473   13267 clientconn.go:948] ClientConn switching balancer to "pick_first"
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.730527   13267 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.731870   13267 clientconn.go:106] parsed scheme: ""
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.731892   13267 clientconn.go:106] scheme "" not registered, fallback to default scheme
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.731919   13267 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins/csi.trident.netapp.io/csi.sock  <nil> 0 <nil>}] <nil> <nil>}
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.731930   13267 clientconn.go:948] ClientConn switching balancer to "pick_first"
Oct 18 13:30:20 worker-02 kubelet[13267]: I1018 13:30:20.731966   13267 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick

Environment
Provide accurate information about the environment to help us reproduce the issue.

• Trident version: 21.04.0
• Container runtime: containerd://1.5.2
• Kubernetes version: v1.21.2
• OS: RHEL 7.9
• NetApp backend types: ontap-nas

[balaramesh]

Hello @moonek

This has also been reported upstream: kubernetes-csi/node-driver-registrar#88

Kubelet looks to be the place where the fix needs to be. This is observed by multiple storage provisioners.

[moonek]

Thank you. I will follow this issue upstream.