v0.8.6

Added

• pw.io.bigquery.write connector for writing Pathway tables into Google BigQuery.
• parameter filepath_globpattern to query method in VectorStoreClient for specifying which files should be considered in the query.
• Improved compatibility of pw.Json with standard methods such as len(), int(), float(), bool(), iter(), reversed() when feasible.

Changed

• pw.io.postgres.write can now parallelize writes to several threads if several workers are configured.
• Pathway now checks types of pointers rigorously. Indexing table with mismatched number/types of columns vs what was used to create index will now result in a TypeError.
• pw.Json.as_float() method now supports integer JSON values.

v0.8.5

Added

• New function answer_with_geometric_rag_strategy_from_index, which allows to use answer_with_geometric_rag_strategy without the need to first retrieve documents from index.
• Added support for custom state serialization to udf_reducer.
• Introduced instance parameter in AsyncTransformer. All calls with a given (instance, processing_time) pair are returned at the same processing time. Ordering is preserved within a single instance.
• Added successful, failed, finished properties to AsyncTransformer. They return tables with successful calls, failed calls and all finished calls, respectively.

Changed

• Property result of AsyncTransformer is deprecated. Property successful should be used instead.
• pw.io.csv.read, pw.io.jsonlines.read, pw.io.fs.read, pw.io.plaintext.read now handle path as a glob pattern and read all matched files and directories recursively.

v0.8.4

Fixed

• Pathway will only require LiteLLM package, if you use one of the wrappers for LiteLLM.
• Retries are implemented in pw.io.airbyte.read.
• State processing protocol is updated in pw.io.airbyte.read.

v0.8.3

Added

• New parameters of pw.UDF class and pw.udf decorator: return_type, deterministic, propagate_none, executor, cache_strategy.
• The LLM Xpack now provides integrations with LlamaIndex and LangChain for running the Pathway VectorStore server.

Changed

• Subclassing UDFSync and UDFAsync is deprecated. UDF should be subclassed to create a new UDF.
• Passing keyword arguments to pw.apply, pw.apply_with_type, pw.apply_async is deprecated. In the future, they'll be used for configuration, not passing data to the function.

Fixed

• Fixed a minor bug with Table.groupby() method which sometimes prevented of accessing certain columns in the following reduce().
• Fixed warnings from using OpenAI Async embedding model in the VectorStore in Colab.

v0.8.2

Added

• %:z timezone format code to strptime.
• Support for Airbyte connectors pw.io.airbyte.

v0.8.1

Added

• Introduced the send_alerts function in the pw.io.slack namespace, enabling users to send messages from a specified column directly to a Slack channel.
• Enhanced the pw.io.http.rest_connector by introducing an additional argument called request_validator. This feature empowers users to validate payloads and raise an HTTP 400 error if necessary.

Fixed

• Addressed an issue in pw.io.xpacks.llm.VectorStoreServer where the computation of the last modification timestamp for an indexed document was incorrect.

Changed

• Improved the behavior of pw.io.kafka.write. It now includes retries when sending data to the output topic encounters failures.

v0.8.0

Added

• pw.io.http.rest_connector now supports multiple HTTP request types.
• pw.io.http.PathwayWebserver now allows Cross-Origin Resource Sharing (CORS) to be enabled on newly added endpoints
• Wrappers for LiteLLM and HuggingFace chat services and SentenceTransformers embedding service are now added to Pathway xpack for LLMs.

Changed

• pw.run now includes an additional parameter runtime_typechecking that enables strict type checking at runtime.
• Embedders in pathway.xpacks.llm.embedders now correctly process empty strings as queries.
• BREAKING: pw.run and pw.run_all now only accept keyword arguments.

Fixed

• pw.Duration can now be returned from User-Defined Functions (UDFs) or used as a constant value without resulting in errors.
• pw.io.debezium.read now correctly handles tables that do not have a primary key.

v0.7.10

Added

• pw.io.http.rest_connector can now generate Open API 3.0.3 schema that will be returned by the route /_schema.
• Wrappers for OpenAI Chat and Embedding services are now added to Pathway xpack for LLMs.
• A vector indexing pipeline that allows querying for the most similar documents. It is available as class VectorStore as part of Pathway xpack for LLMs.

Fixed

• pw.debug.table_from_markdown now uses schema parameter (when set) to properly assign simple types (int, bool, float, str, bytes) and optional simple types to columns.

v0.7.9

Changed

• pw.io.http.rest_connector now also accepts port as a string for backwards compatibility.

v0.7.8

Added

• Support for comparisons of tuples has been added.
• Standalone versions of methods such as pw.groupby, pw.join, pw.join_inner, pw.join_left, pw.join_right, and pw.join_outer are now available.
• The abs function from Python can now be used on Pathway expressions.
• The asof_join method now has configurable temporal behavior. The behavior parameter can be used to pass the configuration.
• The state of the deduplicate operator can now be persisted.

Changed

• interval_join can now work with intervals of zero length.
• The pw.io.http.rest_connector can now open multiple endpoints on the same port using a new pw.io.http.PathwayWebserver class.
• The pw.xpacks.connectors.sharepoint.read and pw.io.gdrive.read methods now support the size limit for a single object. If set, it will exclude too large files and won't read them.