Expose additional sveltos settings for service deployment

[wahabmk]

Description

1. Exposes more of Sveltos settings that can use useful with deploying helm charts and templating values, like being able to refer to a Secret or ConfigMap in helm values.
2. Put all of the spec/settings related to deployment of services in ServiceSpec and use that for both ManagedCluster and MultiClusterService. So the CR will now become:

apiVersion: hmc.mirantis.com/v1alpha1
kind: ManagedCluster
. . .
spec:
  . . .
  serviceSpec:
    services:
      - template: kyverno-3-2-6
        name: kyverno
        namespace: kyverno
      - template: ingress-nginx-4-11-3
        name: ingress-nginx
        namespace: ingress-nginx
    priority: 100
    stopOnConflict: false
    . . .

Example

I have the following secret with dummy credentials:

apiVersion: v1
kind: Secret
metadata:
  name: my-creds
  namespace: hmc-system
data:
  username: bXl1c2VybmFtZQ==
  password: bXlwYXNzd29yZA==

If I want to refer to this secret to pluck these credentials while deploying my application, I can use the following:

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  labels:
    hmc.mirantis.com/managed: "true"
  name: myappz-oci
  namespace: hmc-system
spec:
  type: oci
  url: oci://registry-1.docker.io/wali90
---
apiVersion: hmc.mirantis.com/v1alpha1
kind: ServiceTemplate
metadata:
  name: myappz-0-6-0
  namespace: hmc-system
spec:
  helm:
    chartSpec:
      chart: myappz
      version: 0.6.0
      sourceRef:
        kind: HelmRepository
        name: myappz-oci
---
apiVersion: hmc.mirantis.com/v1alpha1
kind: ManagedCluster
metadata:
  name: wali-dev-1
  namespace: hmc-system
spec:
  config:
    clusterIdentity:
      name: aws-cluster-identity
      namespace: hmc-system
    controlPlane:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceType: t3.small
    controlPlaneNumber: 1
    publicIP: true
    region: ca-central-1
    worker:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceType: t3.small
    workersNumber: 1
  credential: aws-cluster-identity-cred
  serviceSpec:
    templateResourceRefs:
      - resource:
          apiVersion: v1
          kind: Secret
          name: my-creds
          namespace: hmc-system
        identifier: MyCreds
    services:
      - template: myappz-0-6-0
        name: myappz
        namespace: myappz
        values: |
          username: {{ (getResource "MyCreds").data.username }}
          password: {{ (getResource "MyCreds").data.password }}
    priority: 100
    stopOnConflict: false
  template: aws-standalone-cp-0-0-4

Now when looking at the kube deployment for myappz on the managed cluster, we can see that the values for USERNAME and PASSWORD env variables for the pod have been set:

➜  ~ kubectl -n myappz get deployments.apps myappz -o=jsonpath='{.spec.template.spec.containers[0].env}'
[{"name":"USERNAME","value":"bXl1c2VybmFtZQ=="},{"name":"PASSWORD","value":"bXlwYXNzd29yZA=="}]%  

[BROngineer]

What are advantages of this implementation over the adding of valuesFrom field similar to the one used in sveltos types?

From my perspective:

• despite adding flexibility, current implementation increases complexity of hmc's types
• user still must fulfil values field instead of just referring to configMap/secret