discussion: affinity for scheduler deployment

[vsoch]

I'm looking at the scheduler for "Koordinator" and noticed they have affinity for the pod spec in their deployment:

https://github.com/koordinator-sh/charts/blob/95b88c4df3a7c6ad14020add854d5df4d48c836d/versions/v1.3.0/templates/koord-scheduler.yaml#L67-L78

I seem to remember we had a manual strategy to ensure some topology, but it might be a cool idea to have an affinity proper. Here is our deployment yaml for the similar idea:

https://github.com/flux-framework/flux-k8s/blob/main/sig-scheduler-plugins/manifests/fluence/deploy.yaml

Note that I haven't looked at the code that builds that container yet in detail - they could be very different (and then nullify this issue) but I wanted to open it for a future thing to think about (so I don't forget).