Next attempt to install Gardener in the mini-lab

[Gerrit91]

Description

I am trying to do this in order to develop an automation for migrating our Gardener deployment from metal-roles to the Gardener Operator. This is PR prepares for it by allowing to deploy the Gardener Control Plane through the existing automation.

I did not take care of any shoot deployments for now. This would be a stretch goal. For the sole purpose of developing the Operator migration, this implementation should be sufficient.

Depends on:

• Changes required for running Gardener in the mini-lab. metal-roles#365

References:

• Migration to Gardener Operator metal-roles#360

[Gerrit91]

Started with the provider-local and had to build it locally upfront. But I found the network setup too opinionated for our use-case to just reach the goal of having a running control plane. Therefore, I went for the power dns and backup s3 extension from @robertvolkmann.

Seed now reconciles to 100%, but two things have to be done manually:

1. Remove the pod anti affinity rule in order to successfully deploy the power dns extension
2. Patch 172.17.0.1 into the ingress status of istio-ingressgateway service type load balancer (kubectl patch -n istio-ingress svc istio-ingressgateway --type=merge --subresource status --patch 'status: {loadBalancer: {ingress: [{ip: "172.17.0.1"}]}}')
   • Automating this might be a bit difficult because patching the status resource does not work through the existing Ansible module. We could implement our own and use: https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/CoreV1Api.md#patch_namespaced_service_status

❯ k --context virtual-garden get seed
NAME   STATUS   LAST OPERATION               PROVIDER   REGION   AGE   VERSION    K8S VERSION
test   Ready    Reconcile Succeeded (100%)   metal      local    86m   v1.100.2   v1.30.6

❯ k get dnsrecords.extensions.gardener.cloud -A
NAMESPACE   NAME           TYPE       DOMAIN NAME                    RECORD TYPE   STATUS      AGE
garden      seed-ingress   powerdns   *.gardener.172.17.0.1.nip.io   A             Succeeded   73m