Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[improve] add multi table sink to DorisSink (apache#224)
DorisSink supports multi-table import. Example: ```java Configuration config = new Configuration(); // config.setString("execution.savepoint.path","/tmp/checkpoint/chk-6"); StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(config); env.setParallelism(1); env.getCheckpointConfig().setCheckpointStorage("file:///tmp/checkpoint/"); env.getCheckpointConfig().enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION); env.setRestartStrategy(RestartStrategies.fixedDelayRestart(5, Time.milliseconds(10000))); env.enableCheckpointing(10000); DorisSink.Builder<RecordWithMeta> builder = DorisSink.builder(); final DorisReadOptions.Builder readOptionBuilder = DorisReadOptions.builder(); Properties properties = new Properties(); properties.setProperty("column_separator", ","); properties.setProperty("line_delimiter", "\n"); properties.setProperty("format", "csv"); DorisOptions.Builder dorisBuilder = DorisOptions.builder(); dorisBuilder.setFenodes("127.0.0.1:8030") .setTableIdentifier("") .setUsername("root") .setPassword(""); DorisExecutionOptions.Builder executionBuilder = DorisExecutionOptions.builder(); executionBuilder.setLabelPrefix("xxx12") .setStreamLoadProp(properties) .setDeletable(false).enable2PC(); builder.setDorisReadOptions(readOptionBuilder.build()) .setDorisExecutionOptions(executionBuilder.build()) .setDorisOptions(dorisBuilder.build()); RecordWithMeta record = new RecordWithMeta("test", "test_flink_tmp1", "wangwu,1"); RecordWithMeta record1 = new RecordWithMeta("test", "test_flink_tmp", "wangwu,1"); DataStreamSource<RecordWithMeta> stringDataStreamSource = env.fromCollection( Arrays.asList(record, record1)); stringDataStreamSource.sinkTo(builder.build()); ``` For details, please refer to `org.apache.doris.flink.DorisSinkStreamMultiTableExample.java`
- Loading branch information