adding kustomize transformers: causes breakage

[benjamin-tucker]

Hello - firstly, thanks for awesome plugin. It solves big problems for us. Please can you help me solve this one? I am totally flummoxed.

I'm trying to add labels, following this https://github.com/roboll/helmfile/blob/master/docs/advanced-features.md#transformers

• Labels get added as expected, but if the chart takes parameters it nulls them and some stuff is removed(!)
• Occurs on multiple charts
• Works perfectly, if helmfile template | k apply -f - run locally on my mac (ofcourse, it always does!)

argo-cd = v2.7.6+00c914a.dirty
plugin = travisghansen/argo-cd-helmfile:v0.3.6

---

Specific example

This works:

$ cat helmfile.yaml
---

repositories:
  - name: prometheus-community
    url: https://prometheus-community.github.io/helm-charts

releases:
  - name: "prometheus-blackbox-exporter"
    chart: "prometheus-community/prometheus-blackbox-exporter"
    namespace: "kube-system-monitoring"

    values:
      - ./values/values-common.yaml

    set:
      - name: ingress.hosts[0].host
        value: prometheus-blackbox.private.{{ requiredEnv "CLUSTER" }}.{{ requiredEnv "TOP_LEVEL_DOMAIN" }}

This adds the label, but it unwinds the set, and removes the psp, role and rolebind. ❓ 🤷 🧠

$ cat helmfile.yaml
---

repositories:
  - name: prometheus-community
    url: https://prometheus-community.github.io/helm-charts

releases:
  - name: "prometheus-blackbox-exporter"
    chart: "prometheus-community/prometheus-blackbox-exporter"
    namespace: "kube-system-monitoring"

    values:
      - ./values/values-common.yaml

    set:
      - name: ingress.hosts[0].host
        value: prometheus-blackbox.private.{{ requiredEnv "CLUSTER" }}.{{ requiredEnv "TOP_LEVEL_DOMAIN" }}

    transformers:
      - apiVersion: builtin
        kind: LabelTransformer
        metadata:
          name: notImportantHere
        labels:
          foo: bar
        fieldSpecs:
          - kind: Deployment
            path: spec/template/metadata/labels
            create: true

$ argocd app diff argocd/prometheus-blackbox-exporter

===== apps/Deployment kube-system-monitoring/prometheus-blackbox-exporter ======
193a194
>         foo: bar

===== networking.k8s.io/Ingress kube-system-monitoring/prometheus-blackbox-exporter ======
54c54
<   - host: prometheus-blackbox.private.<redacted>
---
>   - host: CHANGE_ME

===== policy/PodSecurityPolicy /prometheus-blackbox-exporter-psp ======
1,67d0
< apiVersion: policy/v1beta1
< kind: PodSecurityPolicy
< metadata:
<   annotations:
<     kubectl.kubernetes.io/last-applied-configuration: |
<       {"apiVersion":"policy/v1beta1","kind":"PodSecurityPolicy","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"prometheus-blackbox-exporter","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"prometheus-blackbox-exporter","app.kubernetes.io/version":"v0.24.0","argocd.argoproj.io/instance":"prometheus-blackbox-exporter","helm.sh/chart":"prometheus-blackbox-exporter-7.10.0"},"name":"prometheus-blackbox-exporter-psp"},"spec":{"allowPrivilegeEscalation":false,"fsGroup":{"ran
===== rbac.authorization.k8s.io/Role kube-system-monitoring/prometheus-blackbox-exporter ======
1,53d0
< apiVersion: rbac.authorization.k8s.io/v1
< kind: Role
< metadata:
<   annotations:
<     kubectl.kubernetes.io/last-applied-configuration: |
<       {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"prometheus-blackbox-exporter","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"prometheus-blackbox-exporter","app.kubernetes.io/version":"v0.24.0","argocd.argoproj.io/instance":"prometheus-blackbox-exporter","helm.sh/chart":"prometheus-blackbox-exporter-7.10.0"},"name":"prometheus-blackbox-exporter","namespace":"kube-system-monitoring"},"rules":[{"apiGroups":
===== rbac.authorization.k8s.io/RoleBinding kube-system-monitoring/prometheus-blackbox-exporter ======
1,52d0
< apiVersion: rbac.authorization.k8s.io/v1
< kind: RoleBinding
< metadata:
<   annotations:
<     kubectl.kubernetes.io/last-applied-configuration: |
<       {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"RoleBinding","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"prometheus-blackbox-exporter","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"prometheus-blackbox-exporter","app.kubernetes.io/version":"v0.24.0","argocd.argoproj.io/instance":"prometheus-blackbox-exporter","helm.sh/chart":"prometheus-blackbox-exporter-7.10.0"},"name":"prometheus-blackbox-exporter","namespace":"kube-system-monitoring"},"roleRef":{"api%

My values file

$ cat values/values-common.yaml
---
ingress:
  enabled: true
  className: nginx-private
  hosts:
    - host: CHANGE_ME
      paths:
        - path: /
          pathType: Prefix

local run (x86 mac)
helmfile = v0.153.1
helm = v3.11.3
kustomize = v5.1.0

Deploy the application using argo-cd  (without the transformer) - first helmfile
Add the transformer - 2nd helmfile

$ helmfile template | k apply -f -
Adding repo prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories

Templating release=prometheus-blackbox-exporter, chart=/var/folders/9h/5kdwd1zd3bjchvwqpcg1t8fw0000gq/T/chartify3204551211/kube-system-monitoring/prometheus-blackbox-exporter/prometheus-blackbox-exporter
serviceaccount/prometheus-blackbox-exporter configured
configmap/prometheus-blackbox-exporter configured
service/prometheus-blackbox-exporter configured
deployment.apps/prometheus-blackbox-exporter configured
ingress.networking.k8s.io/prometheus-blackbox-exporter configured

$ argocd app diff argocd/prometheus-blackbox-exporter

===== /ConfigMap kube-system-monitoring/prometheus-blackbox-exporter ======
23a24
>     argocd.argoproj.io/instance: prometheus-blackbox-exporter

===== /Service kube-system-monitoring/prometheus-blackbox-exporter ======
11a12
>     argocd.argoproj.io/instance: prometheus-blackbox-exporter

===== /ServiceAccount kube-system-monitoring/prometheus-blackbox-exporter ======
11a12
>     argocd.argoproj.io/instance: prometheus-blackbox-exporter

===== apps/Deployment kube-system-monitoring/prometheus-blackbox-exporter ======
15a16
>     argocd.argoproj.io/instance: prometheus-blackbox-exporter
205d205
<         foo: bar

===== networking.k8s.io/Ingress kube-system-monitoring/prometheus-blackbox-exporter ======
12a13
>     argocd.argoproj.io/instance: prometheus-blackbox-exporter

[travisghansen]

Can you send to full output of the helmfile template command with/without the transformers? I'm interested to see what the product is.

I've never used transformers but on the surface I don't see any reason why they wouldn't work. Any chance of differences of versions of helm, helmfile, kustomize, etc?

[jjungnickel]

I'm currently in the process of investigating a similar issue. This appears to be caused by the --api-versions that are passed to helm template by way of helmfile template --args <...> not being passed to chartify which is only being involved once you specify patches/transformers.

If the templates use .Capabilities.APIVersions the resources will not be in included.

It appears that those versions defined in the state file via apiVersions will get passed on, so I was able to work around this by defining in my helmfile:

apiVersions:
  {{ env "KUBE_API_VERSIONS" | splitList "," | toYaml | nindent 2 }}

[travisghansen]

Wow good find. Here is a base file all my files include.

# note that helm currently *appends* these values vs replaces them
# https://github.com/helm/helm/pull/10108
apiVersions:
- networking.k8s.io/v1/IngressClass
#- networking.k8s.io/v1beta1/IngressClass
{{- if (env "KUBE_API_VERSIONS") }}
{{- range $apiV := ( requiredEnv "KUBE_API_VERSIONS" | split "," ) }}
- {{ $apiV }}
{{- end }}
{{- end }}