Skip to content

Commit

Permalink
[flink] fix
Browse files Browse the repository at this point in the history
  • Loading branch information
LinMingQiang committed Nov 26, 2024
1 parent 8eda6ce commit d7e2290
Show file tree
Hide file tree
Showing 5 changed files with 12 additions and 3 deletions.
2 changes: 1 addition & 1 deletion docs/content/maintenance/dedicated-compaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Example: compact table
--catalog_conf s3.secret-key=*****
```
* `--compact_strategy` Determines how to pick files to be merged, the default is determined by the runtime execution mode, streaming-mode use `minor` strategy and batch-mode use `full` strategy.
* `full` : Only support in batch mode. All files will be picked up for merging.
* `full` : Only supports batch mode. All files will be selected for merging.
* `minor` : Pick the set of files that need to be merged based on specified conditions.
You can use `-D execution.runtime-mode=batch` or `-yD execution.runtime-mode=batch` (for the ON-YARN scenario) to control batch or streaming mode. If you submit a batch job, all
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ private void buildForTraditionalCompaction(
} else {
Preconditions.checkArgument(
!(fullCompaction && isStreaming),
"full compact strategy is only supported in batch mode. Please add -Dexecution.runtime-mode=BATCH.");
"The full compact strategy is only supported in batch mode. Please add -Dexecution.runtime-mode=BATCH.");
}
if (isStreaming) {
// for completely asynchronous compaction
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ public void printHelp() {
System.out.println(
" compact --warehouse hdfs:///path/to/warehouse --database test_db --table test_table "
+ "--partition_idle_time 10s");
System.out.println(
"--compact_strategy determines how to pick files to be merged, the default is determined by the runtime execution mode. "
+ "`full` : Only supports batch mode. All files will be selected for merging."
+ "`minor`: Pick the set of files that need to be merged based on specified conditions.");
System.out.println(
" compact --warehouse s3:///path/to/warehouse "
+ "--database test_db "
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ private void buildForTraditionalCompaction(

Preconditions.checkArgument(
!(fullCompaction && isStreaming),
"full compact strategy is only supported in batch mode. Please add -Dexecution.runtime-mode=BATCH.");
"The full compact strategy is only supported in batch mode. Please add -Dexecution.runtime-mode=BATCH.");

if (isStreaming) {
// for completely asynchronous compaction
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ public void printHelp() {
System.out.println(
"--partition_idle_time is used to do a full compaction for partition which had not receive any new data for 'partition_idle_time' time. And only these partitions will be compacted.");
System.out.println("--partition_idle_time is only supported in batch mode. ");
System.out.println(
"--compact_strategy determines how to pick files to be merged, the default is determined by the runtime execution mode. "
+ "`full` : Only supports batch mode. All files will be selected for merging."
+ "`minor`: Pick the set of files that need to be merged based on specified conditions.");

System.out.println();

System.out.println("Examples:");
Expand Down

0 comments on commit d7e2290

Please sign in to comment.