Support for Platform Type None where no External Load Balancers exist

[prb112]

Background

Installations with a Platform Type = None on OpenShift don't always have access to the External Load Balancer to create a Service with type LoadBalancer and generate an External IP. This keeps the TangServer CR from being populated with a URL.

❯ oc get svc -n nbde
NAME                                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service-tangserver-mini                            LoadBalancer   172.30.31.212    <pending>     7500:31696/TCP   116m
tang-operator-controller-manager-metrics-service   ClusterIP      172.30.246.254   <none>        8443/TCP         135m

A workaround of oc patch svc service-tangserver-mini --type=merge --patch '{"spec": { "type": "ClusterIP" } }' -n nbde
allows the service to have external access.

The Reconcile Loop https://github.com/latchset/tang-operator/blob/main/controllers/tangserver_controller.go#L459 and getService https://github.com/latchset/tang-operator/blob/main/controllers/tangserver_controller_service.go#L52 hard-code LoadBalancer.

We'd like to see support for Platform Type None.

We'd also be willing to contribute.

Many thanks, Paul

[sarroutbi]

Hello @prb112 . Can you please elaborate on this? What is your proposal? An specific entry in the CRD for this?

[prb112]

Hey @sarroutbi
Thanks for listening.

I've been thinking about this. We could use metallb in the cluster with Tang Operator.
However, by default we don't always have the LoadBalancers.

In this case, your thought of a specific entry in the CRD seems reasonable. something akin to loadBalancers=false would flip it on / off, and the default would be true.

Thanks, Paul

https://github.com/latchset/tang-operator/blob/main/api/v1alpha1/tangserver_types.go#L24

[sarroutbi]

Hello @prb112. I have started to work on this, but I don't see a ServiceType that corresonds to "None". Possible service types to configure are:

• "ClusterIP"
• "NodePort"
• "LoadBalancer"
• "ExternalName"

I don't see a "None" option to the ServiceType ... are you referring to the ClusterIP? According to the API:

       // clusterIP is the IP address of the service and is usually assigned                                                                                                        
        // randomly. If an address is specified manually, is in-range (as per                                                                                                        
        // system configuration), and is not in use, it will be allocated to the                                                                                                     
        // service; otherwise creation of the service will fail. This field may not                                                                                                  
        // be changed through updates unless the type field is also being changed                                                                                                    
        // to ExternalName (which requires this field to be blank) or the type                                                                                                       
        // field is being changed from ExternalName (in which case this field may                                                                                                    
        // optionally be specified, as describe above).  Valid values are "None",                                                                                                    
        // empty string (""), or a valid IP address. Setting this to "None" makes a                                                                                                  
        // "headless service" (no virtual IP), which is useful when direct endpoint                                                                                                  
        // connections are preferred and proxying is not required.  Only applies to                                                                                                  
        // types ClusterIP, NodePort, and LoadBalancer. If this field is specified                                                                                                   
        // when creating a Service of type ExternalName, creation will fail. This                                                                                                    
        // field will be wiped when updating a Service to type ExternalName.                                                                                                         
        // More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies                                                               
        // +optional                                                                                                                                                                 
        ClusterIP string `json:"clusterIP,omitempty" protobuf:"bytes,3,opt,name=clusterIP"`           

If I understand correctly, if you want to set None Cluster IP, you can set cluster IP to "None" and ServiceType to "ClusterIP" ... what do you think about being able to set the ClusterIP and the ServiceType manually in the CRD, so that it is configured externally and all the options are available?

[sarroutbi]

I have tested the patch provided in #273

If you apply the CRD in operator_configs/none-cluster-ip/:

$ cat operator_configs/none-cluster-ip/daemons_v1alpha1_tangserver.yaml 
---
apiVersion: daemons.redhat.com/v1alpha1
kind: TangServer
metadata:
  name: tangserver-none-cluster-ip
  namespace: nbde
  finalizers:
    - finalizer.daemons.tangserver.redhat.com
spec:
  replicas: 1
  image: "quay.io/sec-eng-special/fedora_tang_server"
  version: "latest"
  clusterIP: "None"
  serviceType: "ClusterIP"

you will get a None Cluster IP:

$ oc get services -n nbde
NAME                                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
service-tangserver-none-cluster-ip   ClusterIP   None         <none>        7500/TCP   3m41s

I guess this should be enough for what you are trying to get

[prb112]

Hi Sergio, yes, that should be fine. Thank you kindly, Paul