Enable high availability (HA) configuration for ASO #4445
Conversation
bingikarthik
requested review from
davefellows,
theunrepentantgeek,
matthchr,
babbageclunk and
super-harsh
as code owners
There was a problem hiding this comment.
How will things behave when upgrading ASO to the next version?
I'm worried about the scenario where the version N+1 of ASO deploys new CRD versions which (still running) version N doesn't understand.
In this scenario, since we use the newest resource version as the storage (hub) version of the resource, any resource that has been touched by the version N+1 will be unintelligible to version N, resulting in a panic/crash.
...zure-service-operator/templates/policy_v1_ poddisruptionbudget_azureserviceoperator-pdb.yaml
Outdated
Show resolved
Hide resolved
|
@matthchr I've made the requested changes. When you have a moment, could you kindly review the PR? |
|
@bingikarthik - thanks! I think we need to make a deployment rollout change to make this safe (as @theunrepentantgeek called out). I'll send a separate PR for that and link it here, and then once that merges we can make sure this change is safe w/ it, and assuming it is, merge this too. Thanks for your patience |
|
Does ASO support "leader" kind of approach? If multiple replicas are running (and reconciling independently), can the same request can be processed by multiple instances of ASO resulting in duplicate calls to Azure? |
Yes, it was indeed. Please check: https://github.com/Azure/azure-service-operator/blob/main/v2/charts/azure-service-operator/templates/apps_v1_deployment_azureserviceoperator-controller-manager.yaml#L59 |
| metadata: | ||
| name: controller-manager | ||
| namespace: system | ||
| labels: |
There was a problem hiding this comment.
This needs app.kubernetes.io/name: azure-service-operator label too, to match Helm
| apiVersion: policy/v1 | ||
| kind: PodDisruptionBudget | ||
| metadata: | ||
| name: controller-manager |
There was a problem hiding this comment.
Should be pdb not controller-manager.
|
had a few comments - once they're fixed I'll kick off CI and we can merge this. |
What this PR does
Closes #4215
As we adopt Azure Service Operator (ASO), it's essential to run multiple replicas for production workloads, especially during cluster upgrade operations. This PR enables high availability (HA) for ASO by increasing the replica count and implementing a Pod Disruption Budget (PDB) to maintain consistent uptime and resilience during potential disruptions.
How does this PR make you feel?
Checklist