Release

1.15.0 (2024-04-24)

New Features

• Added truncate save mode in DataFrameWrite to overwrite existing tables by truncating the underlying table instead of dropping it.
• Added telemetry to calculate query plan height and number of duplicate nodes during collect operations.
• Added the functions below to unload data from a DataFrame into one or more files in a stage:
  • DataFrame.write.json
  • DataFrame.write.csv
  • DataFrame.write.parquet
• Added distributed tracing using open telemetry APIs for action functions in DataFrame and DataFrameWriter:
  • snowflake.snowpark.DataFrame:
    • collect
    • collect_nowait
    • to_pandas
    • count
    • show
  • snowflake.snowpark.DataFrameWriter:
    • save_as_table
• Added support for snow:// URLs to snowflake.snowpark.Session.file.get and snowflake.snowpark.Session.file.get_stream
• Added support to register stored procedures and UDxFs with a comment.
• UDAF client support is ready for public preview. Please stay tuned for the Snowflake announcement of UDAF public preview.
• Added support for dynamic pivot. This feature is currently in private preview.

Improvements

• Improved the generated query performance for both compilation and execution by converting duplicate subqueries to Common Table Expressions (CTEs). It is still an experimental feature not enabled by default, and can be enabled by setting session.cte_optimization_enabled to True.

Bug Fixes

• Fixed a bug where statement_params was not passed to query executions that register stored procedures and user defined functions.
• Fixed a bug causing snowflake.snowpark.Session.file.get_stream to fail for quoted stage locations.
• Fixed a bug that an internal type hint in utils.py might raise AttributeError in case the underlying module can not be found.

Local Testing Updates

New Features

• Added support for registering UDFs and stored procedures.
• Added support for the following APIs:
  • snowflake.snowpark.Session:
    • file.put
    • file.put_stream
    • file.get
    • file.get_stream
    • read.json
    • add_import
    • remove_import
    • get_imports
    • clear_imports
    • add_packages
    • add_requirements
    • clear_packages
    • remove_package
    • udf.register
    • udf.register_from_file
    • sproc.register
    • sproc.register_from_file
  • snowflake.snowpark.functions
    • current_database
    • current_session
    • date_trunc
    • object_construct
    • object_construct_keep_null
    • pow
    • sqrt
    • udf
    • sproc
• Added support for StringType, TimestampType and VariantType data conversion in the mocked function to_time.

Bug Fixes

• Fixed a bug that null filled columns for constant functions.
• Fixed a bug that implementation of to_object, to_array and to_binary to better handle null inputs.
• Fixed a bug that timestamp data comparison can not handle year beyond 2262.
• Fixed a bug that Session.builder.getOrCreate should return the created mock session.