Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
steinitzu committed May 24, 2024
1 parent 55ec9d9 commit 93c0f7a
Showing 1 changed file with 12 additions and 4 deletions.
16 changes: 12 additions & 4 deletions docs/website/docs/dlt-ecosystem/destinations/athena.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,18 @@ You can choose the following file formats:

You can use the `athena_adapter` to add partitioning to Athena tables. This is currently only supported for Iceberg tables.

Here is how you use it:
Iceberg tables support a few transformation functions for partitioning. Info on all supported functions in the [AWS documentation](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html#querying-iceberg-creating-tables-query-editor).

Use the `athena_partition` helper to generate the partitioning hints for these functions:

* `athena_partition.year(column_name: str)`: Partition by year of date/datetime column.
* `athena_partition.month(column_name: str)`: Partition by month of date/datetime column.
* `athena_partition.day(column_name: str)`: Partition by day of date/datetime column.
* `athena_partition.hour(column_name: str)`: Partition by hour of date/datetime column.
* `athena_partition.bucket(n: int, column_name: str)`: Partition by hashed value to `n` buckets
* `athena_partition.truncate(length: int, column_name: str)`: Partition by truncated value to `length` (or width for numbers)

Here is an example of how to use the adapter to partition a table:

```py
from datetime import date
Expand Down Expand Up @@ -207,8 +218,5 @@ pipeline = dlt.pipeline("athena_example")
pipeline.run(partitioned_data)
```

See the AWS Athena documentation for more information on [available partitioning options](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html#querying-iceberg-creating-tables-query-editor).


<!--@@@DLT_TUBA athena-->

0 comments on commit 93c0f7a

Please sign in to comment.