From 93c0f7a69a6fd63765f0fcb9d91052d162f262d5 Mon Sep 17 00:00:00 2001 From: Steinthor Palsson Date: Fri, 24 May 2024 11:02:40 -0400 Subject: [PATCH] Update docs --- .../docs/dlt-ecosystem/destinations/athena.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/website/docs/dlt-ecosystem/destinations/athena.md b/docs/website/docs/dlt-ecosystem/destinations/athena.md index 6d33a1ccab..629c19e5f2 100644 --- a/docs/website/docs/dlt-ecosystem/destinations/athena.md +++ b/docs/website/docs/dlt-ecosystem/destinations/athena.md @@ -166,7 +166,18 @@ You can choose the following file formats: You can use the `athena_adapter` to add partitioning to Athena tables. This is currently only supported for Iceberg tables. -Here is how you use it: +Iceberg tables support a few transformation functions for partitioning. Info on all supported functions in the [AWS documentation](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html#querying-iceberg-creating-tables-query-editor). + +Use the `athena_partition` helper to generate the partitioning hints for these functions: + +* `athena_partition.year(column_name: str)`: Partition by year of date/datetime column. +* `athena_partition.month(column_name: str)`: Partition by month of date/datetime column. +* `athena_partition.day(column_name: str)`: Partition by day of date/datetime column. +* `athena_partition.hour(column_name: str)`: Partition by hour of date/datetime column. +* `athena_partition.bucket(n: int, column_name: str)`: Partition by hashed value to `n` buckets +* `athena_partition.truncate(length: int, column_name: str)`: Partition by truncated value to `length` (or width for numbers) + +Here is an example of how to use the adapter to partition a table: ```py from datetime import date @@ -207,8 +218,5 @@ pipeline = dlt.pipeline("athena_example") pipeline.run(partitioned_data) ``` -See the AWS Athena documentation for more information on [available partitioning options](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html#querying-iceberg-creating-tables-query-editor). - -