Merge branch 'master' into crosshair-in-ci

.github/workflows/main.yml

@@ -48,9 +48,9 @@ jobs:
           # - check-py313t-cover
           # - check-py313t-nocover
           # - check-py313t-niche
-          - check-py314-cover
-          - check-py314-nocover
-          - check-py314-niche
+          # - check-py314-cover
+          # - check-py314-nocover
+          # - check-py314-niche
           # - check-py314t-cover
           # - check-py314t-nocover
           # - check-py314t-niche
@@ -69,6 +69,7 @@ jobs:
           - check-py39-pytest46
           - check-py39-pytest54
           - check-pytest62
+          - check-django51
           - check-django50
           - check-django42
           - check-pandas22
@@ -77,8 +78,9 @@ jobs:
           - check-pandas15
           - check-pandas14
           - check-pandas13
-          - check-py39-pandas12
-          - check-py39-pandas11
+          ## FIXME: actions update means Python builds without eg _bz2, which was required
+          # - check-py39-pandas12
+          # - check-py39-pandas11
           ## `-cover` is too slow under crosshair; use a custom split
           - check-crosshair-custom-cover/test_[a-d]*
           - check-crosshair-custom-cover/test_[e-i]*
@@ -228,8 +230,8 @@ jobs:
       NODE_VERSION: 18
       # Note that the versions below must be updated in sync; we've automated
       # that with `update_pyodide_versions()` in our weekly cronjob.
-      PYODIDE_VERSION: 0.26.2
-      PYTHON_VERSION: 3.12.1
+      PYODIDE_VERSION: 0.27.0
+      PYTHON_VERSION: 3.12.7
       EMSCRIPTEN_VERSION: 3.1.58
     steps:
       - uses: actions/checkout@v3
@@ -247,7 +249,6 @@ jobs:
         uses: mymindstorm/setup-emsdk@6ab9eb1bda2574c4ddb79809fc9247783eaf9021 # v14
         with:
           version: ${{ env.EMSCRIPTEN_VERSION }}
-          actions-cache-folder: emsdk-cache
       - name: Build
         run: |
           pip install pyodide-build==$PYODIDE_VERSION

.github/workflows/update-deps.yml

@@ -18,7 +18,7 @@ jobs:
       - name: Update pinned dependencies
         run: ./build.sh upgrade-requirements
       - name: Open pull request
-        uses: peter-evans/create-pull-request@v4
+        uses: peter-evans/create-pull-request@v7
         with:
           token: ${{secrets.GH_TOKEN}}
           delete-branch: true

CODEOWNERS

@@ -1,6 +1,6 @@
 # Engine changes need to be approved by Zac-HD, as per
 # https://github.com/HypothesisWorks/hypothesis/blob/master/guides/review.rst#engine-changes
-/hypothesis-python/src/hypothesis/internal/conjecture/ @Zac-HD
+/hypothesis-python/src/hypothesis/internal/conjecture/ @DRMacIver @Zac-HD 
 
 # Changes to the paper also need to be approved by DRMacIver or Zac, as authors
 /paper.md @DRMacIver @Zac-HD

build.sh

@@ -25,8 +25,8 @@ if [ -n "${GITHUB_ACTIONS-}" ] || [ -n "${CODESPACES-}" ] ; then
 else
     # Otherwise, we install it from scratch
     # NOTE: tooling keeps this version in sync with ci_version in tooling
-    "$SCRIPTS/ensure-python.sh" 3.10.15
-    PYTHON=$(pythonloc 3.10.15)/bin/python
+    "$SCRIPTS/ensure-python.sh" 3.10.16
+    PYTHON=$(pythonloc 3.10.16)/bin/python
 fi
 
 TOOL_REQUIREMENTS="$ROOT/requirements/tools.txt"

hypothesis-python/benchmark/README.md

@@ -1,14 +1,12 @@
-This directory contains code for benchmarking Hypothesis' shrinking. This was written for [pull/3962](https://github.com/HypothesisWorks/hypothesis/pull/3962) and is a manual process at the moment, though we may eventually integrate it more closely with ci for automated benchmarking.
+This directory contains plotting code for our shrinker benchmarking. The code for collecting the data is in `conftest.py`. This directory handles plotting the results.
 
-To run a benchmark:
-
-* Add the contents of `conftest.py` to the bottom of `hypothesis-python/tests/conftest.py`
-* In `hypothesis-python/tests/common/debug.py`, change `derandomize=True` to `derandomize=False` (if you are running more than one trial)
-* Run the tests: `pytest hypothesis-python/tests/`
-  * Note that the benchmarking script does not currently support xdist, so do not use `-n 8` or similar.
+The plotting script (but not collecting benchmark data) requires additional dependencies: `pip install scipy vl-convert-python`.
 
-When pytest finishes the output will contain a dictionary of the benchmarking results. Add that as a new entry in `data.json`. Repeat for however many trials you want; n=5 seems reasonable.
+To run a benchmark:
 
-Also repeat for both your baseline ("old") and your comparison ("new") code.
+- `pytest tests/ --hypothesis-benchmark-shrinks new --hypothesis-benchmark-output data.json` (starting on the newer version)
+- `pytest tests/ --hypothesis-benchmark-shrinks old --hypothesis-benchmark-output data.json` (after switching to the old version)
+  - Use the same `data.json` path, the benchmark will append data. You can append `-k ...` for both commands to subset the benchmark.
+- `python benchmark/graph.py data.json shrinking.png`
 
-Then run `python graph.py` to generate a graph comparing the old and new results.
+This hooks any `minimal()` calls any reports the number of shrinks. Default (and currently unchangeable) number of iterations is 5 per test.