Investigate why CI takes 50% longer for Python 3.12

[corneliusroemer]

I've noticed that our CI takes around 50% longer for Python 3.12 compared to earlier versions, that's quite strange. If anything, things should get faster for more recent versions.

It's the cram tests that are slower:

I can reproduce locally that py12 based env runs cram slower than py11