PyLance not recognizing imports from PEP-660 editable installs #3473
Comments
Can you try this instead of pip install -e . --config-settings editable_mode=compatAlso, for context, see our docs on PEP 660 editable installs. |
debonte
added
waiting for user response
|
Perfect, |
|
@rzats, that's good to know, because setuptools is planning to remove compat mode in the near future and I was under the impression until today that it was equivalent to I'll open an issue with setuptools asking them to keep compat mode around longer, ideally permanently. They indicated that they would be willing to do that if there was a strong argument for it. I'll keep this issue open until I file the issue with setuptools and then provide a link here. |
|
fwiw, we ended up selecting a slightly different mode, |
|
Ok, but to be clear, both compat mode and strict mode solve your problem? |
|
Just strict mode - sorry, that's my oversight. I assumed it was identical to compat mode, but in fact it's a separate option that falls back to the non-PEP-660 implementation; and this runs into |
debonte
added
by design
and removed
waiting for user response
|
Can I ask why this issue was closed, not supporting the default PEP-660 editable install mode is going to confuse a lot of users who can't find this page. Is there a fundamental reason why pylance can't use these libraries when installed in the default mode. |
Unfortunately PEP 660 was approved without discussion with the Python typing community. We cannot run import hooks. But there has been discussion about alternative ways to solve this problem. See this typing-sig thread. If a solution is found, agreed upon by typing-sig, and standardized, Pylance will consider supporting it. There's more about this in our docs on PEP 660 editable installs. |
|
Thanks for the further info, hopefully a decent resolution comes out of it as trying to remember this issue everytime I do an editable install of a package has bitten me a few times. |
|
It took me a fair amount of searching after a lot of mucking with setting interpreters to find this issue and the I wonder if Pylance could add some text or a link to the I just want to add that this solution is not currently sufficient for me. If I do "Go to Definition F12", it takes me to the I found that I'm using |
@qci-amos, can you open a separate issue on this?
There have been several similar issues filed and it doesn't surprise me that the user's chosen solution might differ in each. Our primary reference on PEP 660 import resolution is this doc page, which mentions both |
|
I don't understand why PyLance cannot use a simple logic like this to handle PEP660-style editable install import importlib
import re
import sys
from itertools import chain
from pathlib import Path
def get_editable_package_mappings():
# XXX not using `ast` to parse the import from the pth file since it's way easier with a simple regex
mappings = [
importlib.import_module(m.group(1)).MAPPING
for f in chain.from_iterable(
Path(p).glob('__editable__.*.pth') for p in sys.path
)
if (m := re.search(r'import (__editable___\S+_finder)', f.read_text()))
]
return mappingsThis works perfectly fine to map package names to their source locations. Sure, it makes certain assumptions about the structure of those files, but since it's probably unlikely that those files will significantly chance any time soon (even more so if popular tools like PyLance use them), but it does the job. In the typing mailing list discussion linked by @debonte a year ago, there's a mention of not being able to use anything but certain stdlib modules. My example above uses some more stdlib modules, but it would be fairly trivial to adapt the code and e.g. avoid importing the module but instead uses regex (and/or the ast module) to extract the mapping dict. |
1. Temporarily avoid recent asyncssh version that breaks the tests 2. Workaround pylance issue with pyproject.toml related changes and pip editable modules install format (#768) See Also: - microsoft/pylance-release#3473 May also affect `mypy`: - python/mypy#16988 - python/mypy#12313
1. Temporarily avoid recent asyncssh version that breaks the tests 2. Workaround pylance issue with pyproject.toml related changes and pip editable modules install format (microsoft#768) See Also: - microsoft/pylance-release#3473 May also affect `mypy`: - python/mypy#16988 - python/mypy#12313
1. Temporarily avoid recent asyncssh version that breaks the tests 2. Workaround pylance issue with pyproject.toml related changes and pip editable modules install format (microsoft#768) See Also: - microsoft/pylance-release#3473 May also affect `mypy`: - python/mypy#16988 - python/mypy#12313
|
|
Does compat mode work for you? See https://microsoft.github.io/pyright/#/import-resolution?id=editable-installs for links to the If neither compat mode nor strict mode works, please file a new issue and we can investigate. Please include links to a repo that reproduces the issue you are seeing. |
|
@debonte compat works! |
Annoyance (PyLance) and editable installs microsoft/pylance-release#3473
|
This still appears to be broken as of September 2024. |
|
TBH I think the easiest solution, if it's your own package, is to just drop setuptools in favor of hatchling. AFAICT it only has advantages :) |
Lots of things on the internet presume that tools work as advertised. I don't just need projects I control to work. |
|
The most reliable thing is just to stop expecting that any of it works and use a |
|
We currently have no plans to make changes here. See my comment above for an explanation of how we got where we are today. If the packaging and typing communities can come to an agreement on a way that static analyzers (emphasis on static) can work better here, we would likely adopt that approach. Also see Eric Traut's recent comment here: #6384 (reply in thread) |
For anyone else who would like to skip this research. From hatchling's readme: "If building extension modules is required then it is recommended that you continue using setuptools, or even other backends that specialize in interfacing with compilers." |
|
I ended up just writing a script that I can run in any repo to introspect the python environment and produce a Such a thing could be better if the IDE had some config options that let me set a list of package_prefix->path mappings for a project vs the single flat list of paths. This all seems like options that the static analyzer should support and then that the ide plug-in should be pragmatic about attempting to auto detect during environment setup. Sure, I would have done the packaging specification differently, but I have little sympathy for the argument here that has produced a user hostile outcome for years from an IDE. |
I realize that this style of solution can offend certain sensibilities, but if it were turned into a real workaround and recommended in a faq, it would have saved many people countless hours. If there were a vscode setting that just took a json mapping of package prefix/path mappings, a small script like this could generate that. Then copy/paste, done. Then maybe a couple of years from now the conversation resolves some declarative way to... and everyone is happy. I know this packaging stuff is a mess. Better workarounds and do-it-myself config options are the soup-du-joure. FTR, there are significant usability hurdles to the strict mode recommendation -- so much so that I have spent a lot of time living with the orange squiggles as the better option. I've spent many hours trying to use it, and it is just better to slog on without ide integration. |
|
other easy workarounds are either just setting |
|
Yeah, I think I've given up on this working in any sane way by default and am just sticking to .env files. Would be really nice if the IDE itself, as part of python environment selection could advertise a list of package/paths to plugins. |
|
@stellaraccident: may I know how you managed to solve this ? I have a Project Namespace in a In my case, either I use: So, for (b) as long as other Folders (besides last There seem to be other Issues related to "Editable Installs" (I usually just run I also tried these two but nothing changes:
Is there something that could make both work correctly ? Is seems to me like there is a "Package" merging that is not working correctly (at all). |
|
The other dirty workaround is to write a helper Script / hook to:
Any other Ideas ? |
Environment data
Code Snippet
I'm currently trying to get static analysis working for a project with a nonstandard directory structure (more at #3454). Turning it into an editable install seem like the best option so far.
https://github.com/rzats/setuptools-repro is a small repo that mirrors the structure of that project. It contains this
pyproject.toml:setup.cfg:as well as a Python file at
root/package/src/test.py, and amain.pythat tries to importtest.py.Repro Steps
Expected behavior
root.package.testshould show up as a valid import in VSCode too.Actual behavior
That doesn't happen:
Unfortunately, reverting to legacy editable packages as suggested by #3265 (comment) does not work for me, since that has an unrelated issue where advanced
package_dirbindings don't work at all - the newer PEP-660 implementation does support that.Logs
Python
Python Language Server
The text was updated successfully, but these errors were encountered: