Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1644950: make use_logical_type option more explicit #2190

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

#### Improvements

- Added support for `ln` in `snowflake.snowpark.functions`
- Improved documentation for `Session.write_pandas` by making `use_logical_type` option more explicit.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed ln function since it is already added above.

- Added support for specifying the following to `DataFrameWriter.save_as_table`:
- `enable_schema_evolution`
- `data_retention_time`
Expand Down
16 changes: 11 additions & 5 deletions src/snowflake/snowpark/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -2393,6 +2393,7 @@ def write_pandas(
create_temp_table: bool = False,
overwrite: bool = False,
table_type: Literal["", "temp", "temporary", "transient"] = "",
use_logical_type: Optional[bool] = None,
**kwargs: Dict[str, Any],
) -> Table:
"""Writes a pandas DataFrame to a table in Snowflake and returns a
Expand Down Expand Up @@ -2429,6 +2430,11 @@ def write_pandas(
table_type: The table type of table to be created. The supported values are: ``temp``, ``temporary``,
and ``transient``. An empty string means to create a permanent table. Learn more about table types
`here <https://docs.snowflake.com/en/user-guide/tables-temp-transient.html>`_.
use_logical_type: Boolean that specifies whether to use Parquet logical types when reading the parquet files
for the unloaded pandas dataframe. With this file format option, Snowflake can interpret Parquet logical
sfc-gh-aalam marked this conversation as resolved.
Show resolved Hide resolved
types during data loading. To enable Parquet logical types, set use_logical_type as True. Set to None to
use Snowflakes default. For more information, see:
`file format options: <https://docs.snowflake.com/en/sql-reference/sql/create-file-format#type-parquet>`_.

Example::

Expand Down Expand Up @@ -2506,11 +2512,11 @@ def write_pandas(
+ (table_name)
)
signature = inspect.signature(write_pandas)
if not ("use_logical_type" in signature.parameters):
# do not pass use_logical_type if write_pandas does not support it
use_logical_type_passed = kwargs.pop("use_logical_type", None)

if use_logical_type_passed is not None:
use_logical_type_supported = "use_logical_type" in signature.parameters
if use_logical_type_supported:
kwargs["use_logical_type"] = use_logical_type
else:
if use_logical_type is not None:
# raise warning to upgrade python connector
warnings.warn(
"use_logical_type will be ignored because current python "
Expand Down
Loading