Pyright fails to check types assigned to variables

[Pwuts]

_AgentGraphCompoundPrimaryKey is reported as "Unbound" by Pylance, in a usage directly below the definition:

This isn't just an issue with Pylance, because Pyright also fails to resolve and check uses of AgentGraphWhereUniqueInput when run through the CLI.

Example

_AgentGraphCompoundPrimaryKey with invalid arguments = error:

$ poetry run pyright
/home/reinier/code/agpt/AutoGPT/autogpt_platform/backend/backend/server/v2/library/db.py
  /home/reinier/code/agpt/AutoGPT/autogpt_platform/backend/backend/server/v2/library/db.py:119:29 - error: No overloads for "__init__" match the provided arguments
    Argument types: (str, int) (reportCallIssue)
1 error, 0 warnings, 0 informations

AgentGraphWhereUniqueInput, no errors despite invalid arguments:

$ poetry run pyright
0 errors, 0 warnings, 0 informations

Package versions

Pyright: v1.1.392 (latest)
Prisma: v5.17.0

[erictraut]

Please provide a minimal, self-contained code sample in text form that demonstrates your issue. Diagnosing type evaluation issues from partial screen shots is not really possible. If your code samples depend on imports from other libraries, please include the versions of those libraries or stubs that you're using.

[Pwuts]

The Prisma statement from the example:
https://github.com/Significant-Gravitas/AutoGPT/blob/ede7fb40c8df43b72732eefbec63a4e1bdcef741/autogpt_platform/backend/backend/server/v2/library/db.py#L115-L137

git clone [email protected]:Significant-Gravitas/AutoGPT.git
git checkout pwuts/pyright-issue-9709-example
cd AutoGPT/autogpt_platform/backend
poetry install && poetry run prisma generate
poetry run pyright

and you should see the lack of type error on the referenced lines in backend/server/v2/library/db.py.

[erictraut]

Thanks, I'm able to repro. The problem here is that the types.py source file in the prisma library is extremely large — about 75,000 lines of code. The code appears to be auto-generated, which makes sense because it's unlikely that such a file would be manually created and maintained.

The code within this file's global (module) scope is exceeding pyright's internal threshold for cyclomatic complexity and statement count. Beyond this threshold, pyright will refuse to perform any sort of analysis on the code flow graph. Doing so will result in long (perhaps infinite) hangs. In this case, the cyclomatic complexity isn't very high (since there are very few loops in this file), but the statement count — the number of nodes in the control flow graph — is very large.

There is a bug here, in that pyright should evaluate the type as Unknown in this situation rather than Unbound. I'll fix that, but it won't really solve the problem you're reporting.

You might wonder why a file with 75K lines would be problematic when the Python interpreter can handle it without issues. When the Python interpreter executes a large module like this, it follows only one path through the code flow graph. When a static analysis tool analyzes the same file, it needs to consider all possible paths through the code flow graph. That's much more complex and costly than what the interpreter does.

There will always be some limit beyond which code flow analysis isn't viable, and this source file exceeds this limit by a fairly large margin.

This limit wouldn't be an issue if the file contained only class definitions because pyright can evaluate a class definition without performing any code flow analysis. However, the prisma module is defining _AgentGraphCompoundPrimaryKey using the "alternate functional syntax" rather than using a standard class statement and then it's aliasing it to another variable AgentGraphWhereUniqueInput. I'm not sure why this approach was chosen given that most of the TypedDict classes in this file are defined using a class statement. The alternative syntax and the type alias definition are syntactically indistinguishable from a normal variable assignment, so pyright needs to perform code flow analysis to determine the variables' types. That's why you see this problem specifically for AgentGraphWhereUniqueInput but not for other TypedDict classes that are defined using a class statement.

You may want to report the issue to the maintainers of prisma. Since this file is auto-generated, I'm hoping that it's straightforward for them to break it up into a more manageable submodules. Alternately, if they could generate class statements for all of their TypedDicts rather than using the alternate functional syntax, that would mitigate the issue.

I don't think there's anything I can do in pyright to address this issue without a change in prisma. The code complexity threshold is in place for good reasons, and it has been tuned over many years. Increasing it to a larger number — especially one as large as would be required for the prisma module — would result in a bad experience for many pyright and pylance users. As an experiment, I tried changing the this threshold to accommodate the prisma module, and it resulted in a 20-second hang on my (very fast) machine. That means on a more typical computer, it would hang for the better part of a minute, and that would occur on every keystroke while editing.

[Pwuts]

Thank you very much for taking the time to repro and for explaining the issue in detail. I have learned something useful today.

I'll raise this issue with the Prisma project and see if I can put in a PR to fix it. The "alternate functional syntax" for defining that specific TypedDict also raised my eyebrows as it's harder for humans to read to begin with.

The code within this file's global (module) scope is exceeding pyright's internal threshold for cyclomatic complexity and statement count. Beyond this threshold, pyright will refuse to perform any sort of analysis on the code flow graph.

Would it be possible to add an info message or a warning when this happens? Either globally, or on types/statements/modules that aren't checked?
The silent failure/refusal to check certain statements let a bug slip through our pre-commit hooks and our CI into our staging environment. In this case it caused a significant issue which is why we spotted it, but I can imagine smaller issues could go unnoticed and end up in production.

[Pwuts]

Apparently there is a reason for defining TypedDicts like that:
https://github.com/RobertCraigie/prisma-client-py/blob/b12ab3a7bf7af5360ff5ba479f43a6b5e5adfdd8/src/prisma/generator/templates/types.py.jinja#L57-L67

[erictraut]

Would it be possible to add an info message or a warning when this happens? Either globally, or on types/statements/modules that aren't checked?

Pyright already emits an error in that module, but you would need to open it in your editor to see it. It would be confusing to emit the error in other modules. This is a rare issue (the first time I've seen it in a library), so I'm going to leave it as is for now.

[Pwuts]

Pyright already emits an error in that module, but you would need to open it in your editor to see it.

Understood. I was thinking of a message like "module {module_name} uses unchecked types: import {import_name} exceeds maximum complexity".