Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1304211: Pylance throws errors when using copy_options with copy_into_location method #1363

Closed
Yoshi-Egawa opened this issue Apr 4, 2024 · 0 comments · Fixed by #1372
Labels
bug Something isn't working needs triage Initial RCA is required

Comments

@Yoshi-Egawa
Copy link
Contributor

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

    Python 3.11.8

  2. What operating system and processor architecture are you using?

    linux 64

  3. What are the component versions in the environment (pip freeze)?

    snowflake-snowpark-python==1.11.1

  4. What did you do?

    Let's say you have a custom class like the one at the bottom (a minimal example to check for errors).

from snowflake.snowpark.session import Session

connection_parameters: dict[str, int | str] = {
    "user": "<user_name>",
    "password": "<password>",
    "account": "<account_name>",
    "role": "<role_name>",
    "warehouse": "<warehouse_name>",
    "database": "<database_name>",
    "schema": "<schema_name>",
}

session = Session.builder.configs(connection_parameters).create()

df = session.create_dataframe(
    [["John", "Berry"], ["Rick", "Berry"], ["Anthony", "Davis"]],
    schema=["FIRST_NAME", "LAST_NAME"],
)

remote_file_path = f"{session.get_session_stage()}/names.parquet"

copy_result = df.write.copy_into_location(
    remote_file_path,
    file_format_type="parquet",
    header=True,
    overwrite=True,
    single=True,
)

When linting this file (Pylance v2024.4.1) errors appear.

No overloads for "copy_into_location" match the provided arguments

Argument of type "Literal[True]" cannot be assigned to parameter "overwrite" of type "str | None" in function "copy_into_location"
  Type "Literal[True]" cannot be assigned to type "str | None"
    "Literal[True]" is incompatible with "str"
    "Literal[True]" is incompatible with "None"

Argument of type "Literal[True]" cannot be assigned to parameter "single" of type "str | None" in function "copy_into_location"
  Type "Literal[True]" cannot be assigned to type "str | None"
    "Literal[True]" is incompatible with "str"
    "Literal[True]" is incompatible with "None"

image

  1. What did you expect to see?

    The type of copy_options in the copy_into_location method is Optional[str], but the type of copy_options in the CopyIntoLocationNode class used in the copy_into_location method is Dict[str, Any]. Consequently, to align the types, the copy_options in the copy_into_location method should be of type Optional[Any].

  2. Can you set logging to DEBUG and collect the logs?

    This is not a RunTime error, so I think this will not give us any extra information.

@Yoshi-Egawa Yoshi-Egawa added bug Something isn't working needs triage Initial RCA is required labels Apr 4, 2024
@github-actions github-actions bot changed the title Pylance throws errors when using copy_options with copy_into_location method SNOW-1304211: Pylance throws errors when using copy_options with copy_into_location method Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Initial RCA is required
Projects
None yet
1 participant