testTypedPkgSimpleEditable is failing on master

[JelleZijlstra]

e.g. https://github.com/python/mypy/actions/runs/5279780403/jobs/9550964906

__________________________ testTypedPkgSimpleEditable __________________________
[gw1] linux -- Python 3.11.4 /home/runner/work/mypy/mypy/.tox/py/bin/python
data: /home/runner/work/mypy/mypy/test-data/unit/pep561.test:84:
Failed: Invalid output (/home/runner/work/mypy/mypy/test-data/unit/pep561.test, line 84)
----------------------------- Captured stderr call -----------------------------
Expected:
  testTypedPkgSimpleEditable.py:5: note: Revealed type is "builtins.tuple[builtins.str, ...]" (diff)
Actual:
  testTypedPkgSimpleEditable.py:2: error: Cannot find implementation or library stub for module named "typedpkg.sample" (diff)
  testTypedPkgSimpleEditable.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports (diff)
  testTypedPkgSimpleEditable.py:3: error: Cannot find implementation or library stub for module named "typedpkg" (diff)
  testTypedPkgSimpleEditable.py:5: note: Revealed type is "Any" (diff)

Alignment of first line difference:
  E: ...kgSimpleEditable.py:5: note: Revealed type is "builtins.tuple[builtin...
  A: ...kgSimpleEditable.py:2: error: Cannot find implementation or library s...
                            ^

Seems to have started today. Cross-ref #15444 (comment)

[ilevkivskyi]

Yeah, I also noticed this test is being really flaky recently on Python 3.11. Here is an example run https://github.com/python/mypy/actions/runs/5261122966/jobs/9508885603

@ethanhs Since you are an expert on PEP 561, could you please check?

[emmatyping]

Yep! I'll take a look at this later today! It looks like the package is failing to install, which might be related to a change in pip or one of its dependencies? I'll have to dig into it.

[JelleZijlstra]

Thanks! I also suspect it's some third-party package being updated, but none of pip, setuptools, tox, and wheel had updates in the last few days. Python 3.11.4 just came out though, so maybe something changed there?

[cdce8p]

Python 3.11.4 just came out though, so maybe something changed there?

👍🏻 Looked though the logs. The Github action runner started using 3.11.4 today. 3.11.3 was fine.

The issue also seems to be related to setuptools recent editable mode changes.
https://setuptools.pypa.io/en/latest/userguide/development_mode.html

Adding something here, does work. Just not sure that's correct. @ethanhs Would appreciate if you could take a closer look at it.

mypy/mypy/test/testpep561.py

Lines 55 to 59 in cfec717

	if use_pip:
	install_cmd = [python_executable, "-m", "pip", "install"]
	if editable:
	install_cmd.append("-e")
	install_cmd.append(".")

      if editable: 
          install_cmd.append("-e") 
      install_cmd.append(".")
+     if editable:
+         install_cmd.extend(["--config-settings", "editable_mode=compat"])

pip==23.1.2
setuptools==67.7.2

[cdce8p]

The last good run from this morning: https://github.com/python/mypy/actions/runs/5276828348/jobs/9544075850

[emmatyping]

@cdce8p good find!

I don't think I want to rely on the compat mode because they seem to want to remove it. So I think this is basically #13392.

For now I think we should just disable this test and I'm going to leave a comment in the other issue that we should stop supporting editable installs.

[hauntsaninja]

If it's just the setuptools issue again, we could just change what build backend we use in CI. Most other build backends don't do the magical import hooks thing for editable installs...

[cdce8p]

Some more background. I bisected the issue to the pip update in Python 3.11.4 python/cpython@a59dc1f. A bit surprising IMO since I believed the venv pip version would be used, but that somehow doesn't seem to be the case. 3.11.3 was fine although the venv contained pip==23.1.2.

Pip 23.1 has a removal which might be relevant here

Remove setup.py install fallback when building a wheel failed for projects without pyproject.toml. ([#8368](https://github.com/pypa/pip/issues/8368))

https://pip.pypa.io/en/stable/news/#deprecations-and-removals

[cdce8p]

If we want to preserve testing the old behavior with setuptools, adding a pyproject.toml file would resolve it as well.

# test-data/packages/typedpkg/pyproject.toml

[build-system]
requires = ["setuptools~=62.3", "wheel"]
build-backend = "setuptools.build_meta"

Note the old setuptools version which still supports the editable installs.

[JukkaL]

I merged #15451 without a detailed review to unbreak master. I hope somebody else with more context on PEP 561 packages will take another look at the PR.

[hauntsaninja]

@cdce8p A little off topic, but just wanted to say thanks for all your work on mypy. I couldn't find any contact information for you on your profile, but there's a mypy contributor discord and would love for you to be in it if you're interested. Let me know at {my username}@gmail.com and I can send you an invite.