Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks for individual dynamics functions #577

Merged
merged 6 commits into from
Sep 26, 2024
Merged

Benchmarks for individual dynamics functions #577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
42b68c7
benchmarks for individual dynamics functions
maximilian-gelbrecht Sep 25, 2024
c889317
updated changelog
maximilian-gelbrecht Sep 25, 2024
955c9c5
removed redundant fields for BenchmarkSuiteDynamics
maximilian-gelbrecht Sep 26, 2024
16aaf3a
nlayers instead of nlev kwarg
maximilian-gelbrecht Sep 26, 2024
9563b5b
Merge branch 'main' into mg/benchmark-dynamics
maximilian-gelbrecht Sep 26, 2024
9b84304
version bump v0.11, and benchmark minimum time
maximilian-gelbrecht Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Unreleased

- Extend `set!` with `orography` keyword argument to set the orography with `set!` [#578](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/578)
- Added a new benchmark suite for the dynamics functions [#577](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/577)
- Introduced a new `set!` function that allows to set `PrognosticVariables` to new values with keyword arguments
- Restructured dynamical core with prognostic/diagnostic variables array-agnostic and 3-dimensional [#525](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/525)
- Modularised NetCDF output [#573](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/573)
Expand Down
2 changes: 1 addition & 1 deletion Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "SpeedyWeather"
uuid = "9e226e20-d153-4fed-8a5b-493def4f21a9"
authors = ["Milan Klöwer and SpeedyWeather contributors"]
version = "0.10.0"
version = "0.11.0"

[deps]
AbstractFFTs = "621f4979-c628-5d54-868e-fcf4e3e8185c"
Expand Down
3 changes: 3 additions & 0 deletions benchmark/Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
SpeedyWeather = "9e226e20-d153-4fed-8a5b-493def4f21a9"
107 changes: 63 additions & 44 deletions benchmark/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Benchmarks

created for SpeedyWeather.jl v0.9.0 on Tue, 02 Apr 2024 14:05:51.
created for SpeedyWeather.jl v0.11.0 on Thu, 26 Sep 2024 13:52:34.

All simulations have been benchmarked over several seconds (wallclock time) without output. Benchmarking excludes initialization and is started just before the main time loop and finishes right after. The benchmarking results here are not very robust, timings that change with +-50% are not uncommon. Proper benchmarking for performance optimization uses the minimum or median of many executions, while we run a simulation for several time steps which effectively represents the mean, susceptible to outliers that slow down the simulation. However, this is what a user will experience in most situations anyway and the following therefore presents a rough idea of how fast a SpeedyWeather simulation will run, and how much memory it requires.

Expand All @@ -9,19 +9,17 @@ All simulations have been benchmarked over several seconds (wallclock time) with
All benchmark simulation were single-threaded on a CPU:
```julia
julia> versioninfo()
Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Julia Version 1.10.4
Commit 48d4fd48430 (2024-06-04 10:41 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (x86_64-apple-darwin22.4.0)
CPU: 8 × Intel(R) Core(TM) i5-1030NG7 CPU @ 1.10GHz
OS: macOS (arm64-apple-darwin22.4.0)
CPU: 8 × Apple M3
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, icelake-client)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
Environment:
LD_LIBRARY_PATH = /Users/milan/.julia/conda/3/lib:
LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
```

### Explanation
Expand Down Expand Up @@ -49,73 +47,94 @@ inside `the SpeedyWeather.jl/benchmark` folder. It will create this `README.md`

| Model | T | L | Physics | Δt | SYPD | Memory|
| - | - | - | - | - | - | - |
| BarotropicModel | 31 | 1 | false | 1800 | 19349 | 1.22 MB |
| ShallowWaterModel | 31 | 1 | false | 1800 | 10195 | 1.24 MB |
| PrimitiveDryModel | 31 | 8 | true | 1800 | 530 | 3.95 MB |
| PrimitiveWetModel | 31 | 8 | true | 1800 | 399 | 4.28 MB |
| BarotropicModel | 31 | 1 | false | 1800 | 62943 | 1.17 MB |
| ShallowWaterModel | 31 | 1 | false | 1800 | 33470 | 1.19 MB |
| PrimitiveDryModel | 31 | 8 | true | 1800 | 2007 | 4.11 MB |
| PrimitiveWetModel | 31 | 8 | true | 1800 | 1247 | 4.47 MB |

## Grids

| Model | T | L | Grid | Rings | Δt | SYPD | Memory|
| - | - | - | - | - | - | - | - |
| PrimitiveWetModel | 63 | 8 | FullGaussianGrid | 96 | 900 | 35 | 22.50 MB |
| PrimitiveWetModel | 63 | 8 | FullClenshawGrid | 95 | 900 | 34 | 22.29 MB |
| PrimitiveWetModel | 63 | 8 | OctahedralGaussianGrid | 96 | 900 | 45 | 15.34 MB |
| PrimitiveWetModel | 63 | 8 | OctahedralClenshawGrid | 95 | 900 | 54 | 15.12 MB |
| PrimitiveWetModel | 63 | 8 | HEALPixGrid | 95 | 900 | 77 | 11.46 MB |
| PrimitiveWetModel | 63 | 8 | OctaHEALPixGrid | 95 | 900 | 56 | 13.67 MB |
| PrimitiveWetModel | 63 | 8 | FullGaussianGrid | 96 | 900 | 110 | 23.40 MB |
| PrimitiveWetModel | 63 | 8 | FullClenshawGrid | 95 | 900 | 139 | 23.19 MB |
| PrimitiveWetModel | 63 | 8 | OctahedralGaussianGrid | 96 | 900 | 198 | 15.91 MB |
| PrimitiveWetModel | 63 | 8 | OctahedralClenshawGrid | 95 | 900 | 184 | 15.68 MB |
| PrimitiveWetModel | 63 | 8 | HEALPixGrid | 95 | 900 | 201 | 11.86 MB |
| PrimitiveWetModel | 63 | 8 | OctaHEALPixGrid | 95 | 900 | 190 | 14.16 MB |

## Primitive wet model, resolution

| Model | T | L | Rings | Δt | SYPD | Memory|
| - | - | - | - | - | - | - |
| PrimitiveWetModel | 31 | 8 | 48 | 1800 | 378 | 4.28 MB |
| PrimitiveWetModel | 42 | 8 | 64 | 1350 | 161 | 7.22 MB |
| PrimitiveWetModel | 63 | 8 | 96 | 900 | 47 | 15.34 MB |
| PrimitiveWetModel | 85 | 8 | 128 | 675 | 20 | 26.73 MB |
| PrimitiveWetModel | 127 | 8 | 192 | 450 | 6 | 59.10 MB |
| PrimitiveWetModel | 170 | 8 | 256 | 338 | 2 | 105.37 MB |
| PrimitiveWetModel | 31 | 8 | 48 | 1800 | 1103 | 4.47 MB |
| PrimitiveWetModel | 42 | 8 | 64 | 1350 | 604 | 7.51 MB |
| PrimitiveWetModel | 63 | 8 | 96 | 900 | 199 | 15.91 MB |
| PrimitiveWetModel | 85 | 8 | 128 | 675 | 71 | 27.66 MB |
| PrimitiveWetModel | 127 | 8 | 192 | 450 | 19 | 60.99 MB |
| PrimitiveWetModel | 170 | 8 | 256 | 338 | 6 | 108.55 MB |

## PrimitiveWetModel: Physics or dynamics only

| Model | T | L | Dynamics | Physics | Δt | SYPD | Memory|
| - | - | - | - | - | - | - | - |
| PrimitiveWetModel | 31 | 8 | true | true | 1800 | 322 | 4.28 MB |
| PrimitiveWetModel | 31 | 8 | true | false | 1800 | 587 | 4.28 MB |
| PrimitiveWetModel | 31 | 8 | false | true | 1800 | 527 | 4.28 MB |
| PrimitiveWetModel | 31 | 8 | true | true | 1800 | 1097 | 4.47 MB |
| PrimitiveWetModel | 31 | 8 | true | false | 1800 | 3345 | 4.47 MB |
| PrimitiveWetModel | 31 | 8 | false | true | 1800 | 2011 | 4.47 MB |

## Individual dynamics functions


### PrimitiveWetModel | Float32 | T31 L8 | OctahedralGaussianGrid | 48 Rings

| Function | Time | Memory | Allocations |
| - | - | - | - |
| pressure_gradient_flux! | 27.792 μs| 12.88 KiB| 260 |
| linear_virtual_temperature! | 952.870 ns| 0 bytes| 0 |
| temperature_anomaly! | 3.266 μs| 0 bytes| 0 |
| geopotential! | 2.995 μs| 0 bytes| 0 |
| vertical_integration! | 12.417 μs| 0 bytes| 0 |
| surface_pressure_tendency! | 10.750 μs| 6.44 KiB| 130 |
| vertical_velocity! | 4.250 μs| 0 bytes| 0 |
| linear_pressure_gradient! | 933.333 ns| 0 bytes| 0 |
| vertical_advection! | 86.417 μs| 2.81 KiB| 24 |
| vordiv_tendencies! | 178.500 μs| 98.38 KiB| 1940 |
| temperature_tendency! | 257.750 μs| 147.56 KiB| 2910 |
| humidity_tendency! | 254.125 μs| 147.56 KiB| 2910 |
| bernoulli_potential! | 86.917 μs| 49.19 KiB| 970 |

## Shallow water model, resolution

| Model | T | L | Rings | Δt | SYPD | Memory|
| - | - | - | - | - | - | - |
| ShallowWaterModel | 31 | 1 | 48 | 1800 | 9402 | 1.24 MB |
| ShallowWaterModel | 42 | 1 | 64 | 1350 | 4034 | 2.13 MB |
| ShallowWaterModel | 63 | 1 | 96 | 900 | 1132 | 4.71 MB |
| ShallowWaterModel | 85 | 1 | 128 | 675 | 415 | 8.45 MB |
| ShallowWaterModel | 127 | 1 | 192 | 450 | 105 | 19.60 MB |
| ShallowWaterModel | 170 | 1 | 256 | 338 | 39 | 36.41 MB |
| ShallowWaterModel | 255 | 1 | 384 | 225 | 9 | 89.44 MB |
| ShallowWaterModel | 31 | 1 | 48 | 1800 | 36300 | 1.19 MB |
| ShallowWaterModel | 42 | 1 | 64 | 1350 | 14941 | 2.02 MB |
| ShallowWaterModel | 63 | 1 | 96 | 900 | 3816 | 4.40 MB |
| ShallowWaterModel | 85 | 1 | 128 | 675 | 1466 | 7.86 MB |
| ShallowWaterModel | 127 | 1 | 192 | 450 | 391 | 18.16 MB |
| ShallowWaterModel | 170 | 1 | 256 | 338 | 119 | 33.76 MB |
| ShallowWaterModel | 255 | 1 | 384 | 225 | 26 | 83.28 MB |

## Primitive Equation, Float32 vs Float64

| Model | NF | T | L | Δt | SYPD | Memory|
| - | - | - | - | - | - | - |
| PrimitiveWetModel | Float32 | 31 | 8 | 1800 | 317 | 4.28 MB |
| PrimitiveWetModel | Float64 | 31 | 8 | 1800 | 343 | 8.03 MB |
| PrimitiveWetModel | Float32 | 31 | 8 | 1800 | 1261 | 4.47 MB |
| PrimitiveWetModel | Float64 | 31 | 8 | 1800 | 1260 | 8.36 MB |

## PrimitiveDryModel: Physics or dynamics only

| Model | T | L | Dynamics | Physics | Δt | SYPD | Memory|
| - | - | - | - | - | - | - | - |
| PrimitiveDryModel | 31 | 8 | true | true | 1800 | 462 | 3.95 MB |
| PrimitiveDryModel | 31 | 8 | true | false | 1800 | 657 | 3.95 MB |
| PrimitiveDryModel | 31 | 8 | false | true | 1800 | 683 | 3.95 MB |
| PrimitiveDryModel | 31 | 8 | true | true | 1800 | 2007 | 4.11 MB |
| PrimitiveDryModel | 31 | 8 | true | false | 1800 | 5105 | 4.11 MB |
| PrimitiveDryModel | 31 | 8 | false | true | 1800 | 2521 | 4.11 MB |

## Number of vertical layers

| Model | T | L | Δt | SYPD | Memory|
| - | - | - | - | - | - |
| PrimitiveWetModel | 31 | 4 | 1800 | 583 | 2.92 MB |
| PrimitiveWetModel | 31 | 8 | 1800 | 345 | 4.28 MB |
| PrimitiveWetModel | 31 | 12 | 1800 | 217 | 5.65 MB |
| PrimitiveWetModel | 31 | 16 | 1800 | 186 | 7.03 MB |
| PrimitiveWetModel | 31 | 4 | 1800 | 1833 | 3.00 MB |
| PrimitiveWetModel | 31 | 8 | 1800 | 1257 | 4.47 MB |
| PrimitiveWetModel | 31 | 12 | 1800 | 1087 | 5.94 MB |
| PrimitiveWetModel | 31 | 16 | 1800 | 819 | 7.42 MB |
Loading