forked from greenplum-db/gpdb-archive
-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix queryId calculation for queries with grouping clause (#665)
Problem description: if 'pg_stat_statements' extension is enabled and a query containing GROUP BY clause with ROLLUP, CUBE or GROUPING SETS or with GROUPING function is executed, queryId calculation doesn't take into account these grouping clause parameters and GROUPING functions. So, semantically different queries are treated by pg_stat_statements as equivalent queries. Plus warning messages appear during jumbling. Also, jumbling logic doesn't handle properly queries with group_id() function call and calls of functions with 'anytable' parameter. First type of queries causes warnings. Second type of query causes error message "unrecognized RTE kind: 7". Expected correct behavior: queries with different grouping clauses or different usage of the GROUPING function are treated by pg_stat_statements as different queries, and no warning messages appear during the query execution. Cause: In JumbleExpr there is no handling for node tag and T_TableValueExpr and T_GroupingSet is handled incorrectly. Jumble hashing is used as queryId. Error message "unrecognized RTE kind: 7" caused by not handled range table type RTE_TABLEFUNCTION in JumbleRangeTable. Fix: Handling logic for missed node tags was added. According to comments in queryjumble.c, main guideline how to handle query tree is: "Rule of thumb for what to include is that we should ignore anything not semantically significant (such as alias names) as well as anything that can be deduced from child nodes (else we'd just be double- hashing that piece of information)." For T_GroupingSet we append to the jumble the grouping type and list of groupsets. Field 'location' is not appended to the jumble because it is the textual location from parser and is not semantically significant. For RTE_TABLEFUNCTION - 'functions' field of RangeTblEntry was added to the jumble. RangeTblEntry in case of RTE_TABLEFUNCTION uses 'functions' and 'subquery' fields (other significant fields are null). But 'subquery' is duplicated in TableValueExpr below, so do not jumble it. For T_TableValueExpr - 'subquery' field of TableValueExpr (the subquery that is inside "TABLE()" statement) was added to the jumble. Changes from original commit: 1. Cases for T_GroupId and T_GroupingFunc are already present, no fix needed. 2. Remove case for T_Integer because it is not necessary since GPDB 7 uses T_IntList instead. 3. T_GroupingClause is now named T_GroupingSet. Its case was not correct in GPDB 7, change to match implementation from GPDB 6. 4. Rename test to jumble since there were already tests present in GPDB 7. 5. The tests in pg_stat_statements were disabled before because they do not set up shared_preload_libraries and they don't disable optimizer. Add files enable_pg_stat_statements and disable_pg_stat_statements to adapt pg_stat_statements tests to current testing utilities. This allows to remove some initialization and cleanup from the new test. 6. Fix test output for GPDB 7. Notably, ROLLUP now outputs 1 row even if the table has no rows in it. (cherry picked from commit fa44e50)
- Loading branch information
1 parent
ec404f5
commit e8aa067
Showing
11 changed files
with
232 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| CREATE TABLE t(a int, b text) DISTRIBUTED BY (a); | ||
| -- Known issue: query is not added to pg_stat_statements statistics in | ||
| -- case it is planned by GPORCA. So disable GPORCA during tests. | ||
| SET optimizer=off; | ||
| SELECT pg_stat_statements_reset(); | ||
| pg_stat_statements_reset | ||
| -------------------------- | ||
|
|
||
| (1 row) | ||
|
|
||
| SELECT GROUPING (a) FROM t GROUP BY ROLLUP(a, b); | ||
| grouping | ||
| ---------- | ||
| 1 | ||
| (1 row) | ||
|
|
||
| -- launch not equivalent query | ||
| SELECT GROUPING (b) FROM t GROUP BY ROLLUP(a, b); | ||
| grouping | ||
| ---------- | ||
| 1 | ||
| (1 row) | ||
|
|
||
| -- check group_id() in a query | ||
| SELECT group_id() FROM t GROUP BY ROLLUP(a, b); | ||
| group_id | ||
| ---------- | ||
| 0 | ||
| (1 row) | ||
|
|
||
| -- check that queries have separate entries | ||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
| query | calls | ||
| --------------------------------------------------+------- | ||
| SELECT group_id() FROM t GROUP BY ROLLUP(a, b) | 1 | ||
| SELECT GROUPING (a) FROM t GROUP BY ROLLUP(a, b) | 1 | ||
| SELECT GROUPING (b) FROM t GROUP BY ROLLUP(a, b) | 1 | ||
| SELECT pg_stat_statements_reset() | 1 | ||
| (4 rows) | ||
|
|
||
| SELECT pg_stat_statements_reset(); | ||
| pg_stat_statements_reset | ||
| -------------------------- | ||
|
|
||
| (1 row) | ||
|
|
||
| -- check that different grouping options result in separate entries | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b); | ||
| count | ||
| ------- | ||
| 0 | ||
| (1 row) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY CUBE(a, b); | ||
| count | ||
| ------- | ||
| 0 | ||
| (1 row) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS(a, b); | ||
| count | ||
| ------- | ||
| (0 rows) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS((a), (a, b)); | ||
| count | ||
| ------- | ||
| (0 rows) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY a, b; | ||
| count | ||
| ------- | ||
| (0 rows) | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
| query | calls | ||
| -------------------------------------------------------------+------- | ||
| SELECT COUNT (*) FROM t GROUP BY a, b | 1 | ||
| SELECT COUNT (*) FROM t GROUP BY CUBE(a, b) | 1 | ||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS((a), (a, b)) | 1 | ||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS(a, b) | 1 | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b) | 1 | ||
| SELECT pg_stat_statements_reset() | 1 | ||
| (6 rows) | ||
|
|
||
| -- check several parameters options in ROLLUP | ||
| -- all should result in separate entries | ||
| SELECT pg_stat_statements_reset(); | ||
| pg_stat_statements_reset | ||
| -------------------------- | ||
|
|
||
| (1 row) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b); | ||
| count | ||
| ------- | ||
| 0 | ||
| (1 row) | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(b); | ||
| count | ||
| ------- | ||
| 0 | ||
| (1 row) | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
| query | calls | ||
| -----------------------------------------------+------- | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b) | 1 | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(b) | 1 | ||
| SELECT pg_stat_statements_reset() | 1 | ||
| (3 rows) | ||
|
|
||
| --- check anytable parameter for a function | ||
| SELECT pg_stat_statements_reset(); | ||
| pg_stat_statements_reset | ||
| -------------------------- | ||
|
|
||
| (1 row) | ||
|
|
||
| -- call of anytable_out will cause an error, | ||
| -- thus prevent actual call by adding FALSE condition | ||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t)) WHERE 1 = 0; | ||
| anytable_out | ||
| -------------- | ||
| (0 rows) | ||
|
|
||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t WHERE a=0)) WHERE 1 = 0; | ||
| anytable_out | ||
| -------------- | ||
| (0 rows) | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
| query | calls | ||
| -----------------------------------------------------------------------------+------- | ||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t)) WHERE $1 = $2 | 1 | ||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t WHERE a=$1)) WHERE $2 = $3 | 1 | ||
| SELECT pg_stat_statements_reset() | 1 | ||
| (3 rows) | ||
|
|
||
| DROP TABLE t; | ||
| RESET optimizer; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
contrib/pg_stat_statements/sql/disable_pg_stat_statements.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| -- start_ignore | ||
| \! gpconfig -r shared_preload_libraries; | ||
| \! gpstop -raq -M fast; | ||
| -- end_ignore |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| -- start_ignore | ||
| \! gpconfig -c shared_preload_libraries -v 'pg_stat_statements'; | ||
| \! gpstop -raq -M fast; | ||
| -- end_ignore |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| -- start_ignore | ||
| CREATE EXTENSION IF NOT EXISTS pg_stat_statements; | ||
| DROP TABLE IF EXISTS t; | ||
| -- end_ignore | ||
| CREATE TABLE t(a int, b text) DISTRIBUTED BY (a); | ||
|
|
||
| -- Known issue: query is not added to pg_stat_statements statistics in | ||
| -- case it is planned by GPORCA. So disable GPORCA during tests. | ||
| SET optimizer=off; | ||
|
|
||
| SELECT pg_stat_statements_reset(); | ||
|
|
||
| SELECT GROUPING (a) FROM t GROUP BY ROLLUP(a, b); | ||
| -- launch not equivalent query | ||
| SELECT GROUPING (b) FROM t GROUP BY ROLLUP(a, b); | ||
| -- check group_id() in a query | ||
| SELECT group_id() FROM t GROUP BY ROLLUP(a, b); | ||
|
|
||
| -- check that queries have separate entries | ||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
|
|
||
| SELECT pg_stat_statements_reset(); | ||
|
|
||
| -- check that different grouping options result in separate entries | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b); | ||
| SELECT COUNT (*) FROM t GROUP BY CUBE(a, b); | ||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS(a, b); | ||
| SELECT COUNT (*) FROM t GROUP BY GROUPING SETS((a), (a, b)); | ||
| SELECT COUNT (*) FROM t GROUP BY a, b; | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
|
|
||
| -- check several parameters options in ROLLUP | ||
| -- all should result in separate entries | ||
| SELECT pg_stat_statements_reset(); | ||
|
|
||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(a, b); | ||
| SELECT COUNT (*) FROM t GROUP BY ROLLUP(b); | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
|
|
||
| --- check anytable parameter for a function | ||
| SELECT pg_stat_statements_reset(); | ||
|
|
||
| -- call of anytable_out will cause an error, | ||
| -- thus prevent actual call by adding FALSE condition | ||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t)) WHERE 1 = 0; | ||
| SELECT * FROM anytable_out(TABLE(SELECT * FROM t WHERE a=0)) WHERE 1 = 0; | ||
|
|
||
| SELECT query, calls FROM pg_stat_statements ORDER BY query; | ||
|
|
||
| DROP TABLE t; | ||
|
|
||
| RESET optimizer; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters