Skip to content

Latest commit

 

History

History
59 lines (56 loc) · 3.52 KB

Upgrade-Kubernetes-ControlPlane.md

File metadata and controls

59 lines (56 loc) · 3.52 KB

Kubernetes control plane upgrade procedure

  1. Build the new Kubernetes control plane template:
  • move to the packer directory
  • run packer: packer build -var-file vars.hcl exoscale-kube-controlplane.pkr.hcl
  • grab the template ID from the output: --> exoscale.base: Kubernetes 1.24.1 control plane @ de-fra-1 (b8f9c7ea-054f-45d6-951e-c6d921dafb0c)
  1. Update the Kubernetes control plane instance-pool:
  • Update the value for platform_components.kubernetes.templates.control_plane in your locals.tf file with the template ID from step 1. Keep the older value for the cleanup step
  • move to the terraform-kubernetes directory
  • run terraform apply, this should update the instance-pool template ID
  1. With exo CLI, or by other means, get the list of current instances. These instances are using the old template: 
    exo compute instance list |grep kube
# 【output】
# | e7678152-d7a5-4b0e-9a5e-65665b9bebee | paas-staging-kubernetes-4976a-iznkw       | de-fra-1 | standard.tiny  | 89.145.161.16   | running |
# | 0e30ea46-477c-4025-8044-e20c09c9caa6 | paas-staging-kubernetes-4976a-imfbp       | de-fra-1 | standard.tiny  | 89.145.161.6    | running |
  2. Stop the API-server service on an instance:
    • Move to the artifacts directory
    • Connect to the instance (ssh -i id_ed25519 [email protected])
    • Stop the nginx service (sudo systemctl stop nginx)
    • Disconnect (exit or logout)
  3. Ensure the apiserver traffic is now handled by another node:
    • Move to the artifacts directory
    • Run export KUBECONFIG=./admin.kubeconfig
    • Run kubectl get nodes to ensure the api server is still available from the failover node
  4. With exo CLI, or by other means, delete the instance. 
    exo compute instance delete paas-staging-kubernetes-4976a-iznkw
# 【output】
# [+] Are you sure you want to delete instance "paas-staging-kubernetes-4976a-iznkw"? [yN]: y
#  ✔ Deleting instance "paas-staging-kubernetes-4976a-iznkw"... 12s
  5. A new instance will be automatically created to replace the one you just deleted:
    • Wait for the new instance to be running and reachable through SSH.
    • Copy this new instance's hostname for later
  6. Connect to the instance and ensure every services are running correctly, in particular:
    • kube-scheduler.service
    • exoscale-cloud-controller-manager.service
    • kube-apiserver.service
    • kube-controller-manager.service
    • kube-vault-agent.service
    • cluster-autoscaler.service
    • konnectivity.service
  7. Repeat operations from step 4 for other cluster members found in step 3.
  8. Delete the older template (value before the step 2 update) as it's not used anymore: 
    exo compute template delete -z de-fra-1 f229708b-ce5f-4cd4-a9fa-bd96c89c1a49
# 【output】
# [+] Are you sure you want to delete template f229708b-ce5f-4cd4-a9fa-bd96c89c1a49 ("Kubernetes 1.24.1 control plane")? [yN]: y
# ✔ Deleting template f229708b-ce5f-4cd4-a9fa-bd96c89c1a49... 3s

Additional step: Update the Konnectivity agent deployment

At least one instance of Konnectivity agent needs to connect directly to a Konnectivity server which is hosted on each control-plane node. This means once you remove a control plane node, Konnectivity agent cannot connect anymore to it. To solve this issue, you have to go to the terraform-kubernete directory and run terraform apply again in order to update core deployments parameters (that include api-server direct ip addresses).