Skip to content

Commit

Permalink
document delta table partitioning support
Browse files Browse the repository at this point in the history
  • Loading branch information
jorritsandbrink committed Aug 15, 2024
1 parent 53999c4 commit 8912509
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions docs/website/docs/dlt-ecosystem/destinations/filesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,24 @@ def my_delta_resource():

> `dlt` always uses `parquet` as `loader_file_format` when using the `delta` table format. Any setting of `loader_file_format` is disregarded.
#### Delta table partitioning
A Delta table can be partitioned ([Hive-style partitioning](https://delta.io/blog/pros-cons-hive-style-partionining/)) by specifying one or more `partition` column hints. This example partitions the Delta table by the `foo` column:

```py
@dlt.resource(
table_format="delta",
columns={"foo": {"partition": True}}
)
def my_delta_resource():
...
```

:::caution

It is **not** possible to change partition columns after the Delta table has been created. Trying to do so causes an error stating that the partition columns don't match.
:::


#### Storage options
You can pass storage options by configuring `destination.filesystem.deltalake_storage_options`:

Expand Down

0 comments on commit 8912509

Please sign in to comment.