feat: change how runner provisioning works #6751

Job	Run time
Test README	1m 8s
Test Scripts	7s
Console	1m 1s
Build Docs	10s
Ensure Frozen Migrations	9s
VSCode Extension	58s
Lint	2m 22s
Proto Breaking Change Check	6s
SQL	42s
Test Go	3m 35s
Lint Architecture	13s
Rebuild All	0s
Intellij Plugin	0s
Build Controller Docker Image	0s
Build Runner Docker Image	0s
Console e2e	0s
Build Box Docker Image	0s
Shard Integration Tests	0s
Integration Test	0s
Integration Success	1s
	10m 32s

Provide feedback