efficient airflow worker provisioning

[7yl4r]

Right now our workers are manually set up, but there are more efficient ways to provision workers:

1. virsh virt-install to spinup virtual machine(s)
2. docker to set up container(s) like puckel/docker-airflow
3. Azure powershell new-AzureRmVm

At our current scale it may not be worth it to dig into these, but if we want to scale well beyond a dozen workers it will save a lot of time to explore these.

The main challenges to automating worker provisioning are going to be:

• hostname configuration
• network configuration
• making puppet work within the VM/container

[7yl4r]

Setup checklist(s) should be here