Skip to content

Commit

Permalink
[Doc](partition) Refine some auto partition behaviours (apache#488)
Browse files Browse the repository at this point in the history
  • Loading branch information
@zclllyybb
zclllyybb authored Mar 29, 2024
1 parent 4e4fecb commit f907490
Show file tree
Hide file tree
Showing 4 changed files with 40 additions and 112 deletions.
37 changes: 10 additions & 27 deletions docs/advanced/partition/auto-partition.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ When building a table, use the following syntax to populate [CREATE-TABLE](../..
1. AUTO RANGE PARTITION:

```sql
AUTO PARTITION BY RANGE FUNC_CALL_EXPR
AUTO PARTITION BY RANGE (FUNC_CALL_EXPR)
(
)
```
Expand All @@ -97,6 +97,8 @@ When building a table, use the following syntax to populate [CREATE-TABLE](../..
FUNC_CALL_EXPR ::= date_trunc ( <partition_column>, '<interval>' )
```

Note: In version 2.1.0, `FUNC_CALL_EXPR` does not need to be surrounded by parentheses.

2. AUTO LIST PARTITION:

```sql
Expand All @@ -114,7 +116,7 @@ When building a table, use the following syntax to populate [CREATE-TABLE](../..
`TIME_STAMP` datev2 NOT NULL COMMENT 'Date of collection'
) ENGINE=OLAP
DUPLICATE KEY(`TIME_STAMP`)
AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
AUTO PARTITION BY RANGE (date_trunc(`TIME_STAMP`, 'month'))
(
)
DISTRIBUTED BY HASH(`TIME_STAMP`) BUCKETS 10
Expand Down Expand Up @@ -183,9 +185,7 @@ mysql> select * from auto_null_list partition(pX);
1 row in set (0.20 sec)
```

2. For AUTO RANGE PARTITION, the storage of NULL values requires the manual creation of specific partitions.

In Doris' RANGE PARTITION, we do not use a separate NULL PARTITION to contain NULL values, but rather group them in the **minium LESS THAN partition**. For logical clarity, the AUTO RANGE PARTITION does not automatically create such a partition when it encounters a NULL value. If necessary, you can create such a partition yourself to contain NULL values.
2. For AUTO RANGE PARTITION, **NULLABLE columns are not supported as partition columns**.

```sql
mysql> CREATE TABLE `range_table_nullable` (
Expand All @@ -194,31 +194,14 @@ mysql> CREATE TABLE `range_table_nullable` (
-> `k3` DATETIMEV2(6)
-> ) ENGINE=OLAP
-> DUPLICATE KEY(`k1`)
-> AUTO PARTITION BY RANGE date_trunc(`k2`, 'day')
-> AUTO PARTITION BY RANGE (date_trunc(`k2`, 'day'))
-> (
-> )
-> DISTRIBUTED BY HASH(`k1`) BUCKETS 16
-> PROPERTIES (
-> "replication_allocation" = "tag.location.default: 1"
-> );
Query OK, 0 rows affected (0.09 sec)

mysql> insert into range_table_nullable values (0, null, null);
ERROR 1105 (HY000): errCode = 2, detailMessage = [CANCELLED]TStatus: errCode = 2, detailMessage = Can't create partition for NULL Range
mysql> alter table range_table_nullable add partition pX VALUES LESS THAN ("1970-01-01");
Query OK, 0 rows affected (0.11 sec)
mysql> insert into range_table_nullable values (0, null, null);
Query OK, 1 row affected (0.18 sec)
mysql> select * from range_table_nullable;
+------+------+------+
| k1 | k2 | k3 |
+------+------+------+
| 0 | NULL | NULL |
+------+------+------+
1 row in set (0.18 sec)
ERROR 1105 (HY000): errCode = 2, detailMessage = AUTO RANGE PARTITION doesn't support NULL column
```
## Sample Scenarios
Expand All @@ -233,7 +216,7 @@ CREATE TABLE `DAILY_TRADE_VALUE`
......
)
UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
AUTO PARTITION BY RANGE date_trunc(`TRADE_DATE`, 'year')
AUTO PARTITION BY RANGE (date_trunc(`TRADE_DATE`, 'year'))
(
)
DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
Expand Down Expand Up @@ -276,7 +259,7 @@ CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year') ())
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down Expand Up @@ -308,7 +291,7 @@ mysql > CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year') ())
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down
37 changes: 10 additions & 27 deletions ...-CN/docusaurus-plugin-content-docs/current/advanced/partition/auto-partition.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ PROPERTIES (
1. AUTO RANGE PARTITION:

```sql
AUTO PARTITION BY RANGE FUNC_CALL_EXPR
AUTO PARTITION BY RANGE (FUNC_CALL_EXPR)
(
)
```
Expand All @@ -97,6 +97,8 @@ PROPERTIES (
FUNC_CALL_EXPR ::= date_trunc ( <partition_column>, '<interval>' )
```

注意：在 2.1.0 版本，`FUNC_CALL_EXPR` 外围不需要被括号包围。

2. AUTO LIST PARTITION:

```sql
Expand All @@ -114,7 +116,7 @@ PROPERTIES (
`TIME_STAMP` datev2 NOT NULL COMMENT '采集日期'
) ENGINE=OLAP
DUPLICATE KEY(`TIME_STAMP`)
AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
AUTO PARTITION BY RANGE (date_trunc(`TIME_STAMP`, 'month'))
(
)
DISTRIBUTED BY HASH(`TIME_STAMP`) BUCKETS 10
Expand Down Expand Up @@ -183,9 +185,7 @@ mysql> select * from auto_null_list partition(pX);
1 row in set (0.20 sec)
```

2. 对于 AUTO RANGE PARTITION，NULL 值的存储需要手动创建特定的分区。

在 Doris 的 RANGE PARTITION 中，我们并不使用单独的 NULL PARTITION 来包含 NULL 值，而是将其归属于**最小的 LESS THAN 分区内**。为保持逻辑清晰，AUTO RANGE PARTITION 在遇到 NULL 值时不会自动创建这样的分区。如有需要，可以自行创建这样的分区，以包含 NULL 值。
2. 对于 AUTO RANGE PARTITION，**不支持 NULLABLE 列作为分区列**。

```sql
mysql> CREATE TABLE `range_table_nullable` (
Expand All @@ -194,31 +194,14 @@ mysql> CREATE TABLE `range_table_nullable` (
-> `k3` DATETIMEV2(6)
-> ) ENGINE=OLAP
-> DUPLICATE KEY(`k1`)
-> AUTO PARTITION BY RANGE date_trunc(`k2`, 'day')
-> AUTO PARTITION BY RANGE (date_trunc(`k2`, 'day'))
-> (
-> )
-> DISTRIBUTED BY HASH(`k1`) BUCKETS 16
-> PROPERTIES (
-> "replication_allocation" = "tag.location.default: 1"
-> );
Query OK, 0 rows affected (0.09 sec)

mysql> insert into range_table_nullable values (0, null, null);
ERROR 1105 (HY000): errCode = 2, detailMessage = [CANCELLED]TStatus: errCode = 2, detailMessage = Can't create partition for NULL Range
mysql> alter table range_table_nullable add partition pX VALUES LESS THAN ("1970-01-01");
Query OK, 0 rows affected (0.11 sec)
mysql> insert into range_table_nullable values (0, null, null);
Query OK, 1 row affected (0.18 sec)
mysql> select * from range_table_nullable;
+------+------+------+
| k1 | k2 | k3 |
+------+------+------+
| 0 | NULL | NULL |
+------+------+------+
1 row in set (0.18 sec)
ERROR 1105 (HY000): errCode = 2, detailMessage = AUTO RANGE PARTITION doesn't support NULL column
```
## 场景示例
Expand All @@ -233,7 +216,7 @@ CREATE TABLE `DAILY_TRADE_VALUE`
......
)
UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
AUTO PARTITION BY RANGE date_trunc(`TRADE_DATE`, 'year')
AUTO PARTITION BY RANGE (date_trunc(`TRADE_DATE`, 'year'))
(
)
DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
Expand Down Expand Up @@ -276,7 +259,7 @@ CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year')) ()
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down Expand Up @@ -308,7 +291,7 @@ mysql > CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year')) ()
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down
39 changes: 10 additions & 29 deletions ...docusaurus-plugin-content-docs/version-2.1/advanced/partition/auto-partition.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ PROPERTIES (
1. AUTO RANGE PARTITION:

```sql
AUTO PARTITION BY RANGE FUNC_CALL_EXPR
AUTO PARTITION BY RANGE (FUNC_CALL_EXPR)
(
)
```
Expand All @@ -97,6 +97,8 @@ PROPERTIES (
FUNC_CALL_EXPR ::= date_trunc ( <partition_column>, '<interval>' )
```

注意：在 2.1.0 版本，`FUNC_CALL_EXPR` 外围不需要被括号包围。

2. AUTO LIST PARTITION:

```sql
Expand All @@ -114,7 +116,7 @@ PROPERTIES (
`TIME_STAMP` datev2 NOT NULL COMMENT '采集日期'
) ENGINE=OLAP
DUPLICATE KEY(`TIME_STAMP`)
AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
AUTO PARTITION BY RANGE (date_trunc(`TIME_STAMP`, 'month'))
(
)
DISTRIBUTED BY HASH(`TIME_STAMP`) BUCKETS 10
Expand Down Expand Up @@ -148,8 +150,6 @@ PROPERTIES (

### NULL 值分区

> 从 2.1.2 开始，Doris 支持以下功能。
当开启 session variable `allow_partition_column_nullable` 后，LIST 和 RANGE 分区都支持 NULL 列作为分区列。当分区列实际遇到 NULL 值的插入时：

1. 对于 AUTO LIST PARTITION，会自动创建对应的 NULL 值分区：
Expand Down Expand Up @@ -185,9 +185,7 @@ mysql> select * from auto_null_list partition(pX);
1 row in set (0.20 sec)
```

2. 对于 AUTO RANGE PARTITION，NULL 值的存储需要手动创建特定的分区。

在 Doris 的 RANGE PARTITION 中，我们并不使用单独的 NULL PARTITION 来包含 NULL 值，而是将其归属于**最小的 LESS THAN 分区内**。为保持逻辑清晰，AUTO RANGE PARTITION 在遇到 NULL 值时不会自动创建这样的分区。如有需要，可以自行创建这样的分区，以包含 NULL 值。
2. 对于 AUTO RANGE PARTITION，**不支持 NULLABLE 列作为分区列**。

```sql
mysql> CREATE TABLE `range_table_nullable` (
Expand All @@ -196,31 +194,14 @@ mysql> CREATE TABLE `range_table_nullable` (
-> `k3` DATETIMEV2(6)
-> ) ENGINE=OLAP
-> DUPLICATE KEY(`k1`)
-> AUTO PARTITION BY RANGE date_trunc(`k2`, 'day')
-> AUTO PARTITION BY RANGE (date_trunc(`k2`, 'day'))
-> (
-> )
-> DISTRIBUTED BY HASH(`k1`) BUCKETS 16
-> PROPERTIES (
-> "replication_allocation" = "tag.location.default: 1"
-> );
Query OK, 0 rows affected (0.09 sec)

mysql> insert into range_table_nullable values (0, null, null);
ERROR 1105 (HY000): errCode = 2, detailMessage = [CANCELLED]TStatus: errCode = 2, detailMessage = Can't create partition for NULL Range
mysql> alter table range_table_nullable add partition pX VALUES LESS THAN ("1970-01-01");
Query OK, 0 rows affected (0.11 sec)
mysql> insert into range_table_nullable values (0, null, null);
Query OK, 1 row affected (0.18 sec)
mysql> select * from range_table_nullable;
+------+------+------+
| k1 | k2 | k3 |
+------+------+------+
| 0 | NULL | NULL |
+------+------+------+
1 row in set (0.18 sec)
ERROR 1105 (HY000): errCode = 2, detailMessage = AUTO RANGE PARTITION doesn't support NULL column
```
## 场景示例
Expand All @@ -235,7 +216,7 @@ CREATE TABLE `DAILY_TRADE_VALUE`
......
)
UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
AUTO PARTITION BY RANGE date_trunc(`TRADE_DATE`, 'year')
AUTO PARTITION BY RANGE (date_trunc(`TRADE_DATE`, 'year'))
(
)
DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
Expand Down Expand Up @@ -278,7 +259,7 @@ CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year')) ()
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down Expand Up @@ -310,7 +291,7 @@ mysql > CREATE TABLE tbl3
k1 DATETIME NOT NULL,
col1 int
)
AUTO PARTITION BY RANGE date_trunc(`k1`, 'year') ()
AUTO PARTITION BY RANGE (date_trunc(`k1`, 'year')) ()
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
Expand Down
Loading

0 comments on commit f907490

Please sign in to comment.