feat: change how runner provisioning works #6750

Job	Run time
Test Go	1m 13s
Test README	1m 4s
SQL	50s
Test Scripts	6s
Ensure Frozen Migrations	5s
Lint	1m 12s
Console	33s
VSCode Extension	59s
Intellij Plugin	0s
Proto Breaking Change Check	7s
Lint Architecture	11s
Rebuild All	0s
Build Docs	11s
Build Controller Docker Image	0s
Build Runner Docker Image	0s
Console e2e	0s
Build Box Docker Image	0s
Shard Integration Tests	0s
Integration Test	0s
Integration Success	1s
	6m 32s

Provide feedback