Skip to content

Release
Compare
Choose a tag to compare
@snowflake-connectors-app snowflake-connectors-app released this 28 May 23:33
v1.18.0
cbfec22

1.18.0 (2024-05-28)

Snowpark pandas API Updates

New Features

  • Added DataFrame.cache_result and Series.cache_result methods for users to persist DataFrames and Series to a temporary table lasting the duration of the session to improve latency of subsequent operations.

Improvements

  • Added partial support for DataFrame.pivot_table with no index parameter, as well as for margins parameter.
  • Updated the signature of DataFrame.shift/Series.shift/DataFrameGroupBy.shift/SeriesGroupBy.shift to match pandas 2.2.1. Snowpark pandas does not yet support the newly-added suffix argument, or sequence values of periods.
  • Re-added support for Series.str.split.

Bug Fixes

  • Fixed how we support mixed columns for string methods (Series.str.*).

Snowpark Local Testing Updates

New Features

  • Added support for the following DataFrameReader read options to file formats csv and json:
    • PURGE
    • PATTERN
    • INFER_SCHEMA with value being False
    • ENCODING with value being UTF8
  • Added support for DataFrame.analytics.moving_agg and DataFrame.analytics.cumulative_agg_agg.
  • Added support for if_not_exists parameter during UDF and stored procedure registration.

Bug Fixes

  • Fixed a bug that when processing time format, fractional second part is not handled properly.
  • Fixed a bug that caused function calls on * to fail.
  • Fixed a bug that prevented creation of map and struct type objects.
  • Fixed a bug that function date_add was unable to handle some numeric types.
  • Fixed a bug that TimestampType casting resulted in incorrect data.
  • Fixed a bug that caused DecimalType data to have incorrect precision in some cases.
  • Fixed a bug where referencing missing table or view raises confusing IndexError.
  • Fixed a bug that mocked function to_timestamp_ntz can not handle None data.
  • Fixed a bug that mocked UDFs handles output data of None improperly.
  • Fixed a bug where DataFrame.with_column_renamed ignores attributes from parent DataFrames after join operations.
  • Fixed a bug that integer precision of large value gets lost when converted to pandas DataFrame.
  • Fixed a bug that the schema of datetime object is wrong when create DataFrame from a pandas DataFrame.
  • Fixed a bug in the implementation of Column.equal_nan where null data is handled incorrectly.
  • Fixed a bug where DataFrame.drop ignore attributes from parent DataFrames after join operations.
  • Fixed a bug in mocked function date_part where Column type is set wrong.
  • Fixed a bug where DataFrameWriter.save_as_table does not raise exceptions when inserting null data into non-nullable columns.
  • Fixed a bug in the implementation of DataFrameWriter.save_as_table where
    • Append or Truncate fails when incoming data has different schema than existing table.
    • Truncate fails when incoming data does not specify columns that are nullable.

Improvements

  • Removed dependency check for pyarrow as it is not used.
  • Improved target type coverage of Column.cast, adding support for casting to boolean and all integral types.
  • Aligned error experience when calling UDFs and stored procedures.
  • Added appropriate error messages for is_permanent and anonymous options in UDFs and stored procedures registration to make it more clear that those features are not yet supported.
  • File read operation with unsupported options and values now raises NotImplementedError instead of warnings and unclear error information.
Assets 2
Loading