Performance Tweaks

[willtebbutt]

This currently just ensures that some inlining happens as required.

I also want to bump the dependency on MistyClosures to 2.0.0, which relies on compintell/MistyClosures.jl#9 being merged, before merging this. The performance improvement on the Turing.jl benchmark in the tests will be most apparent once this is happens.

edit: the MistyClosures version has now been bumped. The performance improvements in the Turing.jl problem from the benchmarks should be apparent when the current CI run has completed.

[codecov]

Codecov Report

Attention: Patch coverage is 28.57143% with 15 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/interpreter/s2s_reverse_mode_ad.jl	28.57%	15 Missing ⚠️

Files with missing lines	Coverage Δ
src/interpreter/s2s_reverse_mode_ad.jl	92.59% <28.57%> (ø)

[github-actions]

Performance Ratio:
Ratio of time to compute gradient and time to compute function.
Warning: results are very approximate! See here for more context.

┌────────────────────────────┬──────────┬─────────┬─────────────┬─────────┐
│                      Label │ Mooncake │  Zygote │ ReverseDiff │  Enzyme │
│                     String │   String │  String │      String │  String │
├────────────────────────────┼──────────┼─────────┼─────────────┼─────────┤
│                   sum_1000 │     70.4 │     1.0 │        5.61 │ missing │
│                  _sum_1000 │     6.71 │  1260.0 │        34.8 │ missing │
│               sum_sin_1000 │     3.02 │    1.69 │        10.6 │ missing │
│              _sum_sin_1000 │     2.62 │   243.0 │        13.5 │ missing │
│                   kron_sum │     81.8 │    4.45 │       265.0 │ missing │
│              kron_view_sum │     66.9 │    8.88 │       209.0 │ missing │
│      naive_map_sin_cos_exp │     2.55 │ missing │        7.15 │ missing │
│            map_sin_cos_exp │     2.84 │    1.45 │        6.15 │ missing │
│      broadcast_sin_cos_exp │     2.66 │    2.28 │        1.46 │ missing │
│                 simple_mlp │     6.52 │    3.08 │        9.75 │ missing │
│                     gp_lml │     11.2 │    3.65 │     missing │ missing │
│ turing_broadcast_benchmark │     3.82 │ missing │        29.6 │ missing │
│         large_single_block │     4.16 │  4420.0 │        31.8 │ missing │
└────────────────────────────┴──────────┴─────────┴─────────────┴─────────┘