Sometimes vector_halo_update hangs

[rheacangeo]

fv3ser uses the halo update functions provided in this repo and they work really well. However, in some situations (tbd), the vector_halo_update command seems to get into a race condition and hangs indefinitely. The work around is to use start_vector_halo_update and place the wait() later in the code. This can be reproduced in fvser.git/fv3/stencils/dyn_core.py. reqs_vector starts a halo update, and later in the code the wait() completes it. If we either put the wait() right after the start, or remove the reqs_vector and instead call vector_halo_update (which functionally does the same thing), the race condition is encountered when we run it.
make tests_mpi TEST_ARGS="-vv -s --which_modules=DynCore"
There are other cases where a race condition is not encountered. e.g. CubedToLatLon has a vector_halo_update and runs with no problems.