Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add type signatures to lru_cache using overloads (not ready to merge) #13033

Closed
wants to merge 2 commits into from
Closed

Add type signatures to lru_cache using overloads (not ready to merge) #13033

wants to merge 2 commits into from

Conversation

alwaysmpe
Copy link

@alwaysmpe alwaysmpe commented Nov 18, 2024 •
edited
Loading

Undecided if this is a good way to go, but marked as for review to see what others say. I've added some notes on output of mypy_primer at the end, there are a couple of unexpected errors but mostly good.

Change lru_cache callable signature to use ParamSpec. This has previously been tried unsuccessfully (#11662) due to compatibility with the classmethod decorator. I've attempted to solve that compatibility by using overloads that do/don't include the necessary cls and self parameter when it appearrs that a method/classmethod is being cached.

This removes the Hashable constraint on parameters.

Instead, this uses the function signatures (almost) correctly (methods and class methods have 2 overloads, one with and without cls and self, but I think this would be less error prone than currently, could possibly be refined).

As a trivial example, the below type checks correctly

from functools import cache
@cache
def foo(arg: str):
    ...
foo(1) # correctly raises type error
foo("1")
foo.cache_clear()

There is a more comprehensive example that I've been using to test in #12952 which (with 2 minor exceptions) type checks perfectly.

@brianschubert
Copy link
Contributor

(I'm guessing this follows from #12952 for context)

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@alwaysmpe
Copy link
Author

alwaysmpe commented Nov 18, 2024 •
edited
Loading

(I'm guessing this follows from #12952 for context)

Yup, this is (another) attempt, but quite a bit simpler than the previous approaches, it feels like it's fixable (and this caused me a few bugs).

The previous approach I used was to basically make the cache object a descriptor that (in the type system only) mirrored the behaviour of classmethod/staticmethod or a plain method descriptor as appropriate. This is needed because combining descriptors doesn't work well. classmethod and staticmethod are such a deep part of type checkers that they can't easily be changed (per discussion at microsoft/pyright#9384 ).

This approach instead defines an overload with and without the self/cls parameter, so probably less robust from a type checking perspective but simpler, more maintainable and probably less prone to breaking.

Not sure how common this pattern is for others, it works like SFINAE, overloads are enabled depending on the template parameters to the self type so eg the method overloads aren't enabled for what looks like classmethods. I think it's more common for them to be chosen depending on the argument types.

@alwaysmpe
Copy link
Author

alwaysmpe commented Nov 18, 2024 •
edited
Loading

Looking at primer output:

I think dacite might be erroring because they're using a TypeVar bound to Callable without any type parameters, so the implicit type arguments are Never (the empty type), so I lean towards true positive error

prefect error is lost because there's nolonger a Hashable requirement

Not sure why psycopg errors, to investigate

twine - mypy looks to be hitting an assert expecting something to be a FunctionLike, so guessing the new cache form doesn't meet the expected requirement. It's possible that it only triggers here because they're using lru_cache with property instead of cached_property.

mkdocs is erroring because the cached function is annotated with type without any type arguments, so I lean towards true positive

schemathesis is erroring because both parent and child methods are cached and have different signatures (different class so different self type) so not implausible that this is a reasonable error.

pydantic - nolonger a Hashable requirement

home-assistant/core I need to investigate, some of the error: Argument 1 [...] has incompatible type seem to be false positives.
One of the errors (homeassistant/helpers/llm.py:408) seems to be caused by combining cache and partial (easy to solve, add call specialization for that, partial does weird things to the signature).

Other errors look to be mostly good.

@alwaysmpe alwaysmpe marked this pull request as ready for review November 18, 2024 01:35
@Daverball
Copy link
Contributor

This solution appears to depend on methods/classmethods following the convention of naming the first parameter self/cls respectively, so I feel like this will break as soon as you violate this convention or have a positional only self/cls because one or more of your other arguments are positional only. Can you add some test cases to verify that this solution still works correctly with positional only self/cls or when you break this naming convention?

As soon as you stop relying on the parameter name you can no longer write a Protocol that properly distinguishes between a regular method and a classmethod (you would have to rely on the type checker always preferring the type[_T] overload for classmethod and always preferring _T for regular methods, which does not seem possible with metaclasses), and that's usually where earlier attempts failed.

alwaysmpe
alwaysmpe commented Nov 18, 2024
View reviewed changes
stdlib/functools.pyi
Comment on lines +56 to +63
class _Function(Protocol[_P, _R]): # type: ignore[misc] # pyright: ignore reportInvalidTypeVarUse
def __call__(self, *args: _P.args, **kwds: _P.kwargs) -> _R: ...

class _Method(Protocol[_T, _P, _R]): # type: ignore[misc] # pyright: ignore reportInvalidTypeVarUse
def __call__(_self, self: _T, *args: _P.args, **kwds: _P.kwargs) -> _R: ...

class _Classmethod(Protocol[_T, _P, _R]): # type: ignore[misc] # pyright: ignore reportInvalidTypeVarUse
def __call__(_self, cls: type[_T], *args: _P.args, **kwds: _P.kwargs) -> _R: ...
Copy link
Author

@alwaysmpe alwaysmpe Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think combine these and use Concatenate instead, avoid needing to explicitly name a parameter, partially fix issue mentioned by @Daverball

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's pretty much what the closest to working earlier approach did, yes. But it still had its edge cases where it didn't work unfortunately. Maybe things are a little bit more forgiving now.

If I'm not mistaken what you're doing in addition to earlier approaches is to always accept both signatures for methods/classmethods (i.e. the one where you have to explicitly provide self/cls and the one where you don't), so you give up some type safety on instance.method() vs. Cls.method() since it would not be able to detect that you're missing a required parameter.

That might be a reasonable trade-off for the overall better type safety, although I fear that language servers will not know what to do with these overloads, so instead of no parameter hints you may get confusing/difficult to read hints, which could be seen as a regression in ergonomics.

@alwaysmpe
Copy link
Author

alwaysmpe commented Nov 18, 2024 •
edited
Loading

you would have to rely on the type checker always preferring the type[_T] overload for classmethod and always preferring _T for regular methods

I'm hoping that shouldn't be a problem because overloads are only usable if it's possible to substitute in all parameters and if self generic argument isn't a Callable[[type[T], ...], Any] then the classmethod overload isn't selectable, but I'm putting together some tests, probably not until after work.

Thankyou for the feedback.

@Daverball
Copy link
Contributor

Daverball commented Nov 18, 2024 •
edited
Loading

I'm hoping that shouldn't be a problem because overloads are only usable if it's possible to substitute in all parameters and if self generic argument isn't a Callable[[type[T]], Any] then the classmethod overload isn't selectable, but I'm putting together some tests, probably not until after work.

That wouldn't be the case for metaclasses or instances of classes that define a __call__ decorated with classmethod, so there are edge cases here unfortunately. Also the overload resolution can always select the regular method case over the classmethod one once the named parameter stops helping you differentiate, they will always overlap for classmethods.

It might be worth just reducing this to the core of your idea and use Concatenate to split off the first parameter, if there is one, and then accept calls with or without it. It seems like that might always work, albeit not fully type safe.

@alwaysmpe alwaysmpe changed the title Add type signatures to lru_cache Add type signatures to lru_cache using overload (not ready to merge) Nov 18, 2024
@alwaysmpe
Copy link
Author

alwaysmpe commented Nov 18, 2024 •
edited
Loading

Edit: broke something...

Tests are going to fail. There are some things that are there to show what is/isn't working, pyright ignores are correct and most of the typing behavior is as intended, there are a couple of false positives/negatives. I haven't got all of the mypy ignores correct.

I've also been doing some work to refine the descriptor approach on another branch, that is probably more robust from a typing perspective. Will create another PR to run primer for comparison (apologies for the PR noise, it's all well intentioned if a bit rushed sometimes). I've created #13043 for comparison

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

dacite (https://github.com/konradhalas/dacite)
+ dacite/types.py:19: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], type[Any]]"; expected "Never"  [arg-type]
+ dacite/types.py:27: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:29: error: "Never" not callable  [misc]
+ dacite/types.py:32: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[T] | type[None]], T]"; expected "Never"  [arg-type]
+ dacite/types.py:34: error: "Never" not callable  [misc]
+ dacite/types.py:41: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:46: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:48: error: "Never" not callable  [misc]
+ dacite/types.py:59: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:61: error: "Never" not callable  [misc]
+ dacite/types.py:64: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:69: error: "Never" not callable  [misc]
+ dacite/types.py:74: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:79: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], type[Any]]"; expected "Never"  [arg-type]
+ dacite/types.py:84: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:89: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], type[Any] | Any]"; expected "Never"  [arg-type]
+ dacite/types.py:106: error: "Never" not callable  [misc]
+ dacite/types.py:107: error: "Never" not callable  [misc]
+ dacite/types.py:108: error: "Never" not callable  [misc]
+ dacite/types.py:109: error: "Never" not callable  [misc]
+ dacite/types.py:112: error: "Never" not callable  [misc]
+ dacite/types.py:114: error: "Never" not callable  [misc]
+ dacite/types.py:115: error: "Never" not callable  [misc]
+ dacite/types.py:125: error: "Never" not callable  [misc]
+ dacite/types.py:130: error: "Never" not callable  [misc]
+ dacite/types.py:131: error: "Never" not callable  [misc]
+ dacite/types.py:132: error: "Never" not callable  [misc]
+ dacite/types.py:133: error: "Never" not callable  [misc]
+ dacite/types.py:134: error: "Never" not callable  [misc]
+ dacite/types.py:135: error: "Never" not callable  [misc]
+ dacite/types.py:136: error: "Never" not callable  [misc]
+ dacite/types.py:137: error: "Never" not callable  [misc]
+ dacite/types.py:138: error: "Never" not callable  [misc]
+ dacite/types.py:143: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:145: error: "Never" not callable  [misc]
+ dacite/types.py:147: error: "Never" not callable  [misc]
+ dacite/types.py:154: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any], tuple[Any, ...]], tuple[Any, ...]]"; expected "Never"  [arg-type]
+ dacite/types.py:166: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any], type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/types.py:168: error: "Never" not callable  [misc]
+ dacite/types.py:169: error: "Never" not callable  [misc]
+ dacite/types.py:176: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[Any]], bool]"; expected "Never"  [arg-type]
+ dacite/exceptions.py:6: error: "Never" not callable  [misc]
+ dacite/dataclasses.py:19: error: "Never" not callable  [misc]
+ dacite/dataclasses.py:24: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[T]], list[Field[Any]]]"; expected "Never"  [arg-type]
+ dacite/dataclasses.py:30: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[type[T]], bool]"; expected "Never"  [arg-type]
+ dacite/core.py:51: error: "Never" not callable  [misc]
+ dacite/core.py:51: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Callable[[object | Callable[..., Any] | FunctionType | BuiltinFunctionType | MethodType | Module | WrapperDescriptorType | MethodWrapperType | MethodDescriptorType, dict[str, Any] | None, Mapping[str, Any] | None, bool], dict[str, Any]]"; expected "Never"  [arg-type]
+ dacite/core.py:54: error: "Never" not callable  [misc]
+ dacite/core.py:79: error: "Never" not callable  [misc]
+ dacite/core.py:88: error: "Never" not callable  [misc]
+ dacite/core.py:89: error: "Never" not callable  [misc]
+ dacite/core.py:92: error: "Never" not callable  [misc]
+ dacite/core.py:94: error: "Never" not callable  [misc]
+ dacite/core.py:96: error: "Never" not callable  [misc]
+ dacite/core.py:98: error: "Never" not callable  [misc]
+ dacite/core.py:98: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type overloaded function; expected "Never"  [arg-type]
+ dacite/core.py:101: error: "Never" not callable  [misc]
+ dacite/core.py:102: error: "Never" not callable  [misc]
+ dacite/core.py:103: error: "Never" not callable  [misc]
+ dacite/core.py:111: error: "Never" not callable  [misc]
+ dacite/core.py:112: error: "Never" not callable  [misc]
+ dacite/core.py:140: error: "Never" not callable  [misc]
+ dacite/core.py:141: error: "Never" not callable  [misc]
+ dacite/core.py:143: error: "Never" not callable  [misc]
+ dacite/core.py:146: error: "Never" not callable  [misc]
+ dacite/core.py:152: error: "Never" not callable  [misc]
+ dacite/core.py:153: error: "Never" not callable  [misc]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/settings/legacy.py:105: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[PrefectBaseSettings]"; expected "Hashable"  [arg-type]
- src/prefect/settings/legacy.py:105: note: Following member(s) of "PrefectBaseSettings" have conflicts:
- src/prefect/settings/legacy.py:105: note:     Expected:
- src/prefect/settings/legacy.py:105: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:105: note:     Got:
- src/prefect/settings/legacy.py:105: note:         def __hash__(self: object) -> int
- src/prefect/settings/legacy.py:105: note:     Expected:
- src/prefect/settings/legacy.py:105: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:105: note:     Got:
- src/prefect/settings/legacy.py:105: note:         def __hash__(self: object) -> int
- src/prefect/settings/legacy.py:136: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[PrefectBaseSettings]"; expected "Hashable"  [arg-type]
- src/prefect/settings/legacy.py:136: note: Following member(s) of "PrefectBaseSettings" have conflicts:
- src/prefect/settings/legacy.py:136: note:     Expected:
- src/prefect/settings/legacy.py:136: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:136: note:     Got:
- src/prefect/settings/legacy.py:136: note:         def __hash__(self: object) -> int
- src/prefect/settings/legacy.py:136: note:     Expected:
- src/prefect/settings/legacy.py:136: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:136: note:     Got:
- src/prefect/settings/legacy.py:136: note:         def __hash__(self: object) -> int

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/rows.py:143: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "*Generator[bytes | None, None, None]"; expected "bytes"  [arg-type]
+ psycopg/psycopg/types/enum.py:153: error: Need type annotation for "loader"  [var-annotated]
+ psycopg/psycopg/types/enum.py:153: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:153: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[E]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:153: error: Argument 3 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[tuple[bytes, E], ...]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:156: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:156: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[E]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:156: error: Argument 3 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[tuple[bytes, E], ...]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:161: error: Need type annotation for "dumper"  [var-annotated]
+ psycopg/psycopg/types/enum.py:161: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[E]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:161: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "int"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:161: error: Argument 3 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[tuple[E, bytes], ...]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:164: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[E]"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:164: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "int"; expected "Never"  [arg-type]
+ psycopg/psycopg/types/enum.py:164: error: Argument 3 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[tuple[E, bytes], ...]"; expected "Never"  [arg-type]

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/tests/test_temporal.py:2162: error: Argument "exclude" to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[str, str]"; expected "tuple[str]"  [arg-type]
+ ibis/backends/tests/test_temporal.py:2180: error: Argument "exclude" to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[str, str, str, str, str, str]"; expected "tuple[str]"  [arg-type]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/dtypes/cast.py:591: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/apply.py:1004: error: Unused "type: ignore" comment  [unused-ignore]

twine (https://github.com/pypa/twine)
- Warning: unused section(s) in mypy.ini: [mypy-tests.*]
+ twine/auth.py:93: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 1.13.0
+ twine/auth.py:93: : note: use --pdb to drop into pdb
+ twine/auth.py:93: error: Invalid self argument "Resolver" to attribute function "username" with type "Callable[[VarArg(Never), KwArg(Never)], Never]"  [misc]
+ Traceback (most recent call last):
+   File "mypy/checkexpr.py", line 5848, in accept
+   File "mypy/nodes.py", line 1893, in accept
+   File "mypy/checkexpr.py", line 3269, in visit_member_expr
+   File "mypy/checkexpr.py", line 3292, in analyze_ordinary_member_access
+   File "mypy/checkmember.py", line 206, in analyze_member_access
+   File "mypy/checkmember.py", line 225, in _analyze_member_access
+   File "mypy/checkmember.py", line 353, in analyze_instance_member_access
+   File "mypy/checkmember.py", line 532, in analyze_member_var_access
+   File "mypy/checkmember.py", line 818, in analyze_var
+ AssertionError: 

jinja (https://github.com/pallets/jinja)
+ src/jinja2/environment.py:1204: error: Unused "type: ignore" comment  [unused-ignore]

jax (https://github.com/google/jax)
+ jax/_src/dtypes.py:366: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/dtypes.py:367: error: Unused "type: ignore" comment  [unused-ignore]

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/tests/plugin_tests.py:137: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[_DummyPluginConfig]"; expected "type[Never]"  [arg-type]

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/utils/asyncio_utils.py:62: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Any | str"; expected "tuple[Any, ...] | None"  [arg-type]
+ mitmproxy/tools/console/common.py:811: error: Argument "error_message" to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "str"  [arg-type]
+ mitmproxy/tools/console/common.py:848: error: Incompatible types in assignment (expression has type "_lru_cache_wrapper[Callable[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(str | None, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]]", variable has type "_lru_cache_wrapper[Callable[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(bool, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]]")  [assignment]
+ mitmproxy/tools/console/common.py:853: error: Argument "is_replay" to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "bool"  [arg-type]
+ mitmproxy/addons/clientplayback.py:98: error: Need type annotation for "mode"  [var-annotated]
+ mitmproxy/master.py:135: error: Need type annotation for "mode"  [var-annotated]

rich (https://github.com/Textualize/rich)
+ rich/progress_bar.py:145: error: Argument 3 to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "str"  [arg-type]

schemathesis (https://github.com/schemathesis/schemathesis)
+ src/schemathesis/stateful/state_machine.py: note: In class "APIStateMachine":
+ src/schemathesis/stateful/state_machine.py:68: error: Signature of "_to_test_case" incompatible with supertype "RuleBasedStateMachine"  [override]
+ src/schemathesis/stateful/state_machine.py:68: note:      Superclass:
+ src/schemathesis/stateful/state_machine.py:68: note:          _lru_cache_wrapper[Callable[[Type[RuleBasedStateMachine]], Any]]
+ src/schemathesis/stateful/state_machine.py:68: note:      Subclass:
+ src/schemathesis/stateful/state_machine.py:68: note:          _lru_cache_wrapper[Callable[[Type[APIStateMachine]], type]]
+ src/schemathesis/specs/openapi/stateful/__init__.py: note: In function "create_state_machine":
+ src/schemathesis/specs/openapi/stateful/__init__.py:79: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "Tuple[Any, ...]"; expected "Iterator[str]"  [arg-type]
+ src/schemathesis/specs/openapi/stateful/__init__.py:79: note: "tuple" is missing following "Iterator" protocol member:
+ src/schemathesis/specs/openapi/stateful/__init__.py:79: note:     __next__
+ src/schemathesis/specs/openapi/stateful/__init__.py: note: At top level:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/index/package_finder.py:902: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "str"  [arg-type]

pydantic (https://github.com/pydantic/pydantic)
+ pydantic/v1/tools.py:37: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/networks.py:107: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[_BaseUrl]"; expected "Hashable"  [arg-type]
- pydantic/networks.py:107: note: Following member(s) of "_BaseUrl" have conflicts:
- pydantic/networks.py:107: note:     Expected:
- pydantic/networks.py:107: note:         def __hash__() -> int
- pydantic/networks.py:107: note:     Got:
- pydantic/networks.py:107: note:         def __hash__(self: object) -> int
- pydantic/networks.py:291: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[_BaseMultiHostUrl]"; expected "Hashable"  [arg-type]
- pydantic/networks.py:291: note: Following member(s) of "_BaseMultiHostUrl" have conflicts:
- pydantic/networks.py:291: note:     Expected:
- pydantic/networks.py:291: note:         def __hash__() -> int
- pydantic/networks.py:291: note:     Got:
- pydantic/networks.py:291: note:         def __hash__(self: object) -> int

core (https://github.com/home-assistant/core)
+ homeassistant/util/unit_conversion.py:387: error: Signature of "converter_factory" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:387: note:      Superclass:
+ homeassistant/util/unit_conversion.py:387: note:          _lru_cache_wrapper[Callable[[type[BaseUnitConverter], str | None, str | None], Callable[[float], float]]]
+ homeassistant/util/unit_conversion.py:387: note:      Subclass:
+ homeassistant/util/unit_conversion.py:387: note:          _lru_cache_wrapper[Callable[[type[SpeedConverter], str | None, str | None], Callable[[float], float]]]
+ homeassistant/util/unit_conversion.py:401: error: Signature of "converter_factory_allow_none" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:401: note:      Superclass:
+ homeassistant/util/unit_conversion.py:401: note:          _lru_cache_wrapper[Callable[[type[BaseUnitConverter], str | None, str | None], Callable[[float | None], float | None]]]
+ homeassistant/util/unit_conversion.py:401: note:      Subclass:
+ homeassistant/util/unit_conversion.py:401: note:          _lru_cache_wrapper[Callable[[type[SpeedConverter], str | None, str | None], Callable[[float | None], float | None]]]
+ homeassistant/util/unit_conversion.py:467: error: Signature of "converter_factory" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:467: note:      Superclass:
+ homeassistant/util/unit_conversion.py:467: note:          _lru_cache_wrapper[Callable[[type[BaseUnitConverter], str | None, str | None], Callable[[float], float]]]
+ homeassistant/util/unit_conversion.py:467: note:      Subclass:
+ homeassistant/util/unit_conversion.py:467: note:          _lru_cache_wrapper[Callable[[type[TemperatureConverter], str | None, str | None], Callable[[float], float]]]
+ homeassistant/util/unit_conversion.py:481: error: Signature of "converter_factory_allow_none" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:481: note:      Superclass:
+ homeassistant/util/unit_conversion.py:481: note:          _lru_cache_wrapper[Callable[[type[BaseUnitConverter], str | None, str | None], Callable[[float | None], float | None]]]
+ homeassistant/util/unit_conversion.py:481: note:      Subclass:
+ homeassistant/util/unit_conversion.py:481: note:          _lru_cache_wrapper[Callable[[type[TemperatureConverter], str | None, str | None], Callable[[float | None], float | None]]]
+ homeassistant/core.py:1511: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "EventType[_DataT] | str"; expected "Never"  [arg-type]
+ homeassistant/core.py:1529: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "EventType[_DataT] | str"; expected "Never"  [arg-type]
+ homeassistant/auth/__init__.py:655: error: Invalid self argument "_lru_cache_wrapper[partial[dict[str, Any]]]" to attribute function "__call__" with type "Callable[[_lru_cache_wrapper[_Method[_T, _P, _R]], **_P], _R]"  [misc]
+ homeassistant/components/xiaomi_aqara/config_flow.py:216: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "str"  [arg-type]
+ homeassistant/helpers/llm.py:408: error: Invalid self argument "_lru_cache_wrapper[partial[str]]" to attribute function "__call__" with type "Callable[[_lru_cache_wrapper[_Method[_T, _P, _R]], **_P], _R]"  [misc]
+ homeassistant/components/mqtt_room/sensor.py:199: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Any | None"; expected "str"  [arg-type]
+ homeassistant/components/esphome/update.py:63: error: "Never" has no attribute "get_entry_data"  [attr-defined]
+ homeassistant/components/esphome/light.py:178: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:187: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:198: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:207: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:208: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:213: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:225: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:242: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:247: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:261: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:342: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:61: error: Need type annotation for "domain_data"  [var-annotated]
+ homeassistant/components/esphome/__init__.py:89: error: "Never" has no attribute "get_or_create_store"  [attr-defined]

@alwaysmpe alwaysmpe changed the title Add type signatures to lru_cache using overload (not ready to merge) Add type signatures to lru_cache using overloads (not ready to merge) Nov 18, 2024
@alwaysmpe alwaysmpe mentioned this pull request Nov 18, 2024
Closed
@alwaysmpe alwaysmpe marked this pull request as draft November 19, 2024 00:28
@alwaysmpe
Copy link
Author

Closing this. The correct approach is for builtin descriptor type definitions to match their runtime descriptor protocol behaviour but mypy (and I believe other type checkers) don't currently understand it correctly. So I'll try to fix that first. If interested I discuss this on typing forum

@alwaysmpe alwaysmpe closed this Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Daverball Daverball Daverball left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alwaysmpe @brianschubert @Daverball