MPI error with kernel_launching.jl

[simone-silvestri]

Discussed in #3981

^{Originally posted by francispoulin December 12, 2024}
We (@jakob-braga and @francispoulin ) are trying to run distributed_nonhydrostatic_turbulence.jl and getting an error.

We have tried two different servers and found the error in both is due to kernel_launching.jl.

@glwagner @simone-silvestri ?

ERROR: LoadError: ERROR: LoadError: MethodError: no method matching interior_work_layout(::RectilinearGrid{Float64, Periodic, Periodic, Flat, Oceananigans.Grids.StaticVerticalCoordinate{Nothing, Float64}, Float64, Float64, OffsetArrays.OffsetVector{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}}, OffsetArrays.OffsetVector{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}}, Distributed{CPU, false, Partition{Nothing, Nothing, Nothing}, Tuple{Int64, Int64, Int64}, Int64, Tuple{Int64, Int64, Int64}, Oceananigans.DistributedComputations.RankConnectivity{Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, MPI.Comm, Vector{MPI.Request}, Base.RefValue{Int64}}}, ::Tuple{}, ::Tuple{DataType, DataType, DataType})

Closest candidates are:
  interior_work_layout(::Any, !Matched::Symbol, ::Any)
   @ Oceananigans ~/software/Oceananigans.jl/src/Utils/kernel_launching.jl:133
```</div>

[glwagner]

Can we see more of the stack trace?