Consul HA backend for Apache Flink
In order to start an HA-cluster with flink-consul add the following configuration keys to conf/flink-conf.yaml:
high-availability: com.espro.flink.consul.ConsulHaServicesFactory
high-availability.consul.host: https://my-consul-server
high-availability.consul.port: 8550
high-availability.storageDir: hdfs:///flink/recovery
| Property | Default | Description |
|---|---|---|
| high-availability.consul.host | localhost | Address of the Consul server/agent |
| high-availability.consul.port | 8550 | Port to use to reach the Consul server/agent |
| Property | Default | Description |
|---|---|---|
| high-availability.consul.path.root | flink/ | Used to define the root/base path in Consul KV store. |
| high-availability.consul.path.jobstatus | jobstatus/ | Consul path relative to the root path for storing job states |
| high-availability.consul.path.jobgraphs | jobgraphs/ | Consul path relative to the root path for storing job graph |
| high-availability.consul.path.checkpoint-counter | checkpoint-counter/ | Consul path relative to the root path for storing checkpoint counter |
| high-availability.consul.path.checkpoints | checkpoints/ | Consul path relative to the root path for storing information for completed checkpoints |
| high-availability.consul.path.leader | leader/ | Consul path relative to the root path for storing leader information |
| Property | Default | Description |
|---|---|---|
| high-availability.consul.tls.enabled | false | To enabled tls secured http communication |
| high-availability.consul.tls.algorithm | TLS | Algorithm used for creating ssl context |
| high-availability.consul.tls.keystore.path | none | Path to the keystore file, e.g. file:/path/to/keystore/consul.p12 |
| high-availability.consul.tls.keystore.password | none | The password to use to read the keystore |
| high-availability.consul.tls.keystore.type | PKCS12 | Type of the keystore |
| high-availability.consul.tls.truststore.path | none | Path to the truststore file, e.g. file:/path/to/truststore/consul-ca.p12 |
| high-availability.consul.tls.truststore.password | none | The password to use to read the truststore |
| high-availability.consul.tls.truststore.type | PKCS12 | Type of the truststore |
| high-availability.consul.session.ttl | 10 | Consul session ttl in seconds |
| high-availability.consul.client.connect-timeout | 10 | Connect timeout for the consul client in seconds |
| high-availability.consul.client.connection-request-timeout | 10 | connection request timeout for the consul client in seconds |
| high-availability.consul.client.socket-timeout | 30 | socket timeout for the consul client in seconds |