diff --git a/docs/reference/supported-environments.md b/docs/reference/supported-environments.md index 419f22183d2..ca4b5f21fcc 100644 --- a/docs/reference/supported-environments.md +++ b/docs/reference/supported-environments.md @@ -36,6 +36,7 @@ For example, 1.2+ means support for the minor version 2, and any higher minors ( ## Clients - **Zeebe Java Client**: OpenJDK 8+ +- **Zeebe Spring SDK**: OpenJDK 17+ - **Zeebe Go Client**: Go 1.13+ - **zbctl**: Windows, macOS, and Linux (latest) - **Connector SDK**: OpenJDK 17+ diff --git a/docs/self-managed/setup/deploy/openshift/redhat-openshift.md b/docs/self-managed/setup/deploy/openshift/redhat-openshift.md index d228a6d27f5..f3470473bf8 100644 --- a/docs/self-managed/setup/deploy/openshift/redhat-openshift.md +++ b/docs/self-managed/setup/deploy/openshift/redhat-openshift.md @@ -157,7 +157,7 @@ If you deploy Camunda 8 (and related infrastructure) with permissive SCCs out of ## Ingress Configuration -Before exposing services outside the cluster, we need an ingress component. Here's how you can configure it: +Before exposing services outside the cluster, we need an Ingress component. Here's how you can configure it: @@ -174,7 +174,7 @@ For guidance on installing an Ingress controller, you can refer to the [Ingress Do not confuse the [ingress-nginx controller](https://github.com/kubernetes/ingress-nginx) with the [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift) that is endorsed by Red Hat for usage with OpenShift. Despite very similar names, they are two different products. -If you should decide to use the Red Hat endorsed [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift), you would require additional adjustments done to the Camunda 8 ingress objects and the NGINX Ingress Controller itself to make `gRPC` and `HTTP/2` connections work. In that case, please refer to the [example and the prerequisites](https://github.com/nginxinc/kubernetes-ingress/blob/main/examples/ingress-resources/grpc-services/README.md). +If you should decide to use the Red Hat endorsed [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift), you would require additional adjustments done to the Camunda 8 Ingress objects and the NGINX Ingress Controller itself to make `gRPC` and `HTTP/2` connections work. In that case, please refer to the [example and the prerequisites](https://github.com/nginxinc/kubernetes-ingress/blob/main/examples/ingress-resources/grpc-services/README.md). ::: @@ -193,25 +193,39 @@ To use these routes for the Zeebe Gateway, configure this through Ingress as wel As the Zeebe Gateway also uses `gRPC` (which relies on `HTTP/2`), [HTTP/2 Ingress Connectivity has to be enabled](https://docs.openshift.com/container-platform/latest/networking/ingress-operator.html#nw-http2-haproxy_configuring-ingress). +Additionally, the Zeebe Gateway should be configured to use an encrypted connection with TLS. In OpenShift, the connection from HAProxy to the Zeebe Gateway service can use HTTP/2 only for re-encryption or pass-through routes, and not for edge-terminated or insecure routes. + #### Required Steps -1. Provide [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway. The [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html) might be helpful here: +1. Provide two [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway. + + - The first TLS secret is issued to the Zeebe Gateway Service Name. This must use the [PKCS #8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [PKCS #1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as `camunda-platform-internal-service-certificate`. + + In the example below, a TLS certificate is generated for the Zeebe Gateway service with an [annotation](https://docs.openshift.com/container-platform/latest/security/certificates/service-serving-certificate.html). The generated certificate will be in the form of a secret. - - One issued to the Zeebe Gateway Service Name. This must use the [pkcs8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [pkcs1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as **Service Certificate Secret** or ``. For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes). + ```yaml + zeebeGateway: + service: + annotations: + service.beta.openshift.io/serving-cert-secret-name: camunda-platform-internal-service-certificate + ``` -
- pkcs8, pkcs1 syntax + Another option is [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html). For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes). - > PKCS1 private key encoding. PKCS1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS8. +
+ PKCS #8, PKCS #1 syntax - > PKCS8 private key encoding. PKCS8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header. + > PKCS #1 private key encoding. PKCS #1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS #8. - [pkcs1, pkcs8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding) -
+ > PKCS #8 private key encoding. PKCS #8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header. - - One that is used on the exposed route, referenced as **External URL Certificate Secret** or ``. + [PKCS #1, PKCS #8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding) -2. Configure your Zeebe Gateway Ingress to create a [Re-encrypt Route](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration): +
+ + - The second TLS secret is used on the exposed route, referenced as `camunda-platform-external-certificate`. For example, this would be the same TLS secret used for Ingress. + +1. Configure your Zeebe Gateway Ingress to create a [Re-encrypt Route](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration): ```yaml zeebeGateway: @@ -219,11 +233,11 @@ zeebeGateway: grpc: annotations: route.openshift.io/termination: reencrypt - route.openshift.io/destination-ca-certificate-secret: + route.openshift.io/destination-ca-certificate-secret: camunda-platform-internal-service-certificate className: openshift-default tls: enabled: true - secretName: + secretName: camunda-platform-external-certificate ``` 3. Mount the **Service Certificate Secret** to the Zeebe Gateway Pod: @@ -247,14 +261,14 @@ zeebeGateway: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt defaultMode: 420 - name: key secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.key path: tls.key @@ -272,6 +286,8 @@ operate: value: "true" - name: CAMUNDA_OPERATE_ZEEBE_CERTIFICATEPATH value: /usr/local/operate/config/tls.crt + - name: CAMUNDA_OPERATE_ZEEBE_BROKERCONTACTPOINT + value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500 extraVolumeMounts: - name: certificate mountPath: /usr/local/operate/config/tls.crt @@ -279,7 +295,7 @@ operate: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt @@ -297,6 +313,8 @@ tasklist: value: "true" - name: CAMUNDA_TASKLIST_ZEEBE_CERTIFICATEPATH value: /usr/local/tasklist/config/tls.crt + - name: CAMUNDA_TASKLIST_ZEEBE_BROKERCONTACTPOINT + value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500 extraVolumeMounts: - name: certificate mountPath: /usr/local/tasklist/config/tls.crt @@ -304,7 +322,7 @@ tasklist: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt @@ -313,7 +331,61 @@ tasklist: The actual configuration properties can be reviewed [in the Tasklist configuration documentation](/self-managed/tasklist-deployment/tasklist-configuration.md#zeebe-broker-connection). -5. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS. +5. For Connectors: + :::note + +The following will no longer be required when [the Connectors component supports PKCS #1 and PKCS #8](https://github.com/camunda/connectors/issues/2806). + +::: + +The Connectors component only accepts a `jks` (Java KeyStore) certificate. +If you have followed our previous recommendation of generating a TLS certificate using the OpenShift annotation, you will have a `PKCS #1` certificate the Connectors component will not accept. + +Below are a number of commands that convert the `PKCS #1` certificate generated by OpenShift to a `jks` format the Connectors component accepts: + +```bash +# Grab OpenShift generated TLS certificate. +kubectl get secret -n camunda camunda-zeebe-gateway -o jsonpath="{.data['tls\.crt']}" | base64 --decode > tls.crt +# Grab OpenShift generated TLS key. +kubectl get secret -n camunda camunda-zeebe-gateway -o jsonpath="{.data['tls\.key']}" | base64 --decode > zeebe-key.key +# Convert Zeebe Gateway unencrypted TLS key to an encrypted key. You will be prompted to enter a password when running this command. Note down the password: +openssl pkcs8 -topk8 -inform PEM -outform PEM -in ./zeebe-key.key -out ./zeebe-encrypted-key-gen.pem -v2 des3 +# Convert PKCS #1 certificate to PKCS #12. Again, you will be prompted to enter the password. +openssl pkcs12 -export -in tls.crt -inkey zeebe-encrypted-key-gen.pem -out zeebe-p12-certificate.p12 -name "certificate" +# Convert PKCS #12 certificate to jks cert. +keytool -importkeystore -srckeystore zeebe-p12-certificate.p12 -srcstoretype pkcs12 -destkeystore keystore.jks +``` + +Create a generic TLS secret from the `jks` file: + +```bash +kubectl create secret generic keystore -n camunda --from-file keystore.jks +``` + +Once the secret is created, follow the below example `values.yaml` config: + +```yaml +connectors: + inbound: + mode: oauth + env: + - name: ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS + value: "camunda-zeebe-gateway.camunda.svc.cluster.local:26500" + - name: ZEEBE_CLIENT_SECURITY_PLAINTEXT + value: "false" + - name: JAVA_TOOL_OPTIONS + value: "-Djavax.net.ssl.trustStore=/usr/local/certificates/keystore.jks -Djavax.net.ssl.trustStorePassword=changeit" + extraVolumeMounts: + - name: keystore + readOnly: true + mountPath: /usr/local/certificates + extraVolumes: + - name: keystore + secret: + secretName: keystore +``` + +6. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS. diff --git a/docs/self-managed/zeebe-deployment/configuration/broker.md b/docs/self-managed/zeebe-deployment/configuration/broker.md index 5de42ac803a..7ab569f9751 100644 --- a/docs/self-managed/zeebe-deployment/configuration/broker.md +++ b/docs/self-managed/zeebe-deployment/configuration/broker.md @@ -512,6 +512,160 @@ threads: ioThreadCount: 2 ``` +### zeebe.broker.flowControl.request + +Replaces the deprecated `zeebe.broker.backpressure` configuration. + +| Field | Description | Example Value | +| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| enabled | Set this to enable flow control for user requests. When enabled the broker rejects user requests when the number of inflight requests is greater than than the "limit". The value of the "limit" is determined based on the configured algorithm. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_ENABLED`. | true | +| algorithm | The algorithm configures which algorithm to use for the flow control. It should be one of vegas, aimd, fixed, gradient, or gradient2. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_ALGORITHM`. | aimd | +| useWindowed | If enabled, will use the average latencies over a window as the current latency to update the limit. It is not recommended to enable this when the algorithm is aimd. This setting is not applicable to fixed limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_USEWINDOWED`. | false | + +#### YAML snippet + +```yaml +flowControl: + request: + enabled: true + algorithm: aimd + useWindowed: false +``` + +### zeebe.broker.flowControl.request.aimd + +| Field | Description | Example Value | +| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| requestTimeout | The limit will be reduced if the observed latency is greater than the requestTimeout. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_AIMD_REQUESTTIMEOUT`. | 200ms | +| initialLimit | The initial limit to be used when the broker starts. The limit will be reset to this value when the broker restarts. This setting can also be overridden using the environment `ZEEBE_BROKER_FLOWCONTROL_REQUEST_AIMD_INITIALLIMIT`. | 100 | +| minLimit | The minimum limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_AIMD_MINLIMIT`. | 1 | +| maxLimit | The maximum limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL__REQUEST_AIMD_MAXLIMIT`. | 1000 | +| backoffRatio | The backoffRatio is a double value x such that x is between 0 and 1. It determines the factor by which the limit is decreased. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_AIMD_BACKOFFRATIO`. | 0.9 | + +#### YAML snippet + +```yaml +request: + algorithm: aimd + aimd: + requestTimeout: 200ms + initialLimit: 100 + minLimit: 1 + maxLimit: 1000 + backoffRatio: 0.9 +``` + +### zeebe.broker.flowControl.request.fixed + +| Field | Description | Example Value | +| ----- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| limit | Set a fixed limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_FIXED_LIMIT`. | 20 | + +#### YAML snippet + +```yaml +request: + algorithm: fixed + fixed: + limit: 20 +``` + +### zeebe.broker.flowControl.request.vegas + +| Field | Description | Example Value | +| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| initialLimit | The initial limit to be used when the broker starts. The limit will be reset to this value when the broker restarts. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_VEGAS_INITIALLIMIT`. | 20 | +| alpha | The limit is increased if the queue size is less than this value. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_VEGAS_ALPHA`. | 3 | +| beta | The limit is decreased if the queue size is greater than this value. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_VEGAS_BETA`. | 6 | + +#### YAML snippet + +```yaml +request: + algorithm: vegas + vegas: + initialLimit: 20 + alpha: 3 + beta: 6 +``` + +### zeebe.broker.flowControl.request.gradient + +| Field | Description | Example Value | +| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| minLimit | The minimum limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT_MINLIMIT`. | 10 | +| initialLimit | The initial limit to be used when the broker starts. The limit will be reset to this value when the broker restarts. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT_INITIALLIMIT`. | 20 | +| rttTolerance | Tolerance for changes from minimum latency. A value >= 1.0 indicating how much change from minimum latency is acceptable before reducing the limit. For example, a value of 2.0 means that a 2x increase in latency is acceptable. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT_RTTTOLERANCE` | 2.0 | + +#### YAML snippet + +```yaml +request: + algorithm: gradient + gradient: + minLimit: 10 + initialLimit: 20 + rttTolerance: 2.0 +``` + +### zeebe.broker.flowControl.request.gradient2 + +| Field | Description | Example Value | +| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| minLimit | The minimum limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT2_MINLIMIT`. | 10 | +| initialLimit | The initial limit to be used when the broker starts. The limit will be reset to this value when the broker restarts. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT2_INITIALLIMIT`. | 20 | +| rttTolerance | Tolerance for changes from minimum latency. A value >= 1.0 indicating how much change from minimum latency is acceptable before reducing the limit. For example, a value of 2.0 means that a 2x increase in latency is acceptable. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT2_RTTTOLERANCE`. | 2.0 | +| longWindow | longWindow is the length of the window (the number of samples) to calculate the exponentially smoothed average latency. This setting can also be overridden using the environment `ZEEBE_BROKER_FLOWCONTROL_REQUEST_GRADIENT2_LONGWINDOW`. | 600 | + +#### YAML snippet + +```yaml +request: + algorithm: gradient2 + gradient2: + minLimit: 10 + initialLimit: 20 + rttTolerance: 2.0 + longWindow: 600 +``` + +### zeebe.broker.flowControl.write + +| Field | Description | Example Value | +| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| enabled | Set this to enable or disable flow control for all writes. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_ENABLED`. | false | +| rampUp | Time period during which the write limit gradually increases to the configured limit. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_RAMPUP`. | 10s | +| limit | The maximum number of record that can be written per second. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_LIMIT`. | 1000 | + +#### YAML snippet + +```yaml +write: + enabled: false + rampUp: 10s + limit: 1000 +``` + +### zeebe.broker.flowControl.write.throttling + +| Field | Description | Example Value | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------- | +| enabled | Set this to enable or disable write throttling based on the exporting backlog. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_THROTTLING_ENABLED`. | false | +| acceptableBacklog | The number of records that can be in the exporting backlog. The write rate is throttled such that the backlog stabilizes around this value when exporting is a bottleneck. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_THROTTLING_ACCEPTABLEBACKLOG`. | 100000 | +| minimumLimit | The minimum write limit that is guaranteed even when throttling. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_THROTTLING_MINIMUMLIMIT`. | 100 | +| resolution | The frequency at which the throttling is updated. This setting can also be overridden using the environment variable `ZEEBE_BROKER_FLOWCONTROL_WRITE_THROTTLING_RESOLUTION`. | 15s | + +#### YAML snippet + +```yaml +write: + throttling: + enabled: false + acceptableBacklog: 100000 + minimumLimit: 100 + resolution: 15s +``` + ### zeebe.broker.backpressure | Field | Description | Example Value | diff --git a/howtos/documentation-guidelines.md b/howtos/documentation-guidelines.md index a58649e2516..501f7c704cb 100644 --- a/howtos/documentation-guidelines.md +++ b/howtos/documentation-guidelines.md @@ -12,6 +12,10 @@ PRs can be labeled “hold” by engineering teams for any reason, including ong PRs labeled “bug” will be merged and released as immediately as possible. +### Preview sites + +Adding a “deploy” label to a PR will trigger a deployment of the PR to a preview site. The preview site will be linked in the PR description. + ## Not all PRs require an issue In the spirit of "Always Progress", if you are confident you know what change needs to be made a separate issue on the topic is not required. However, if you have an idea that needs to go into the backlog, creating an issue instead of a PR is the best way to go. diff --git a/versioned_docs/version-8.5/reference/supported-environments.md b/versioned_docs/version-8.5/reference/supported-environments.md index b039a7e6c3e..072871a68de 100644 --- a/versioned_docs/version-8.5/reference/supported-environments.md +++ b/versioned_docs/version-8.5/reference/supported-environments.md @@ -36,6 +36,7 @@ For example, 1.2+ means support for the minor version 2, and any higher minors ( ## Clients - **Zeebe Java Client**: OpenJDK 8+ +- **Zeebe Spring SDK**: OpenJDK 17+ - **Zeebe Go Client**: Go 1.13+ - **zbctl**: Windows, macOS, and Linux (latest) - **Connector SDK**: OpenJDK 17+ diff --git a/versioned_docs/version-8.5/self-managed/setup/deploy/openshift/redhat-openshift.md b/versioned_docs/version-8.5/self-managed/setup/deploy/openshift/redhat-openshift.md index d228a6d27f5..f3470473bf8 100644 --- a/versioned_docs/version-8.5/self-managed/setup/deploy/openshift/redhat-openshift.md +++ b/versioned_docs/version-8.5/self-managed/setup/deploy/openshift/redhat-openshift.md @@ -157,7 +157,7 @@ If you deploy Camunda 8 (and related infrastructure) with permissive SCCs out of ## Ingress Configuration -Before exposing services outside the cluster, we need an ingress component. Here's how you can configure it: +Before exposing services outside the cluster, we need an Ingress component. Here's how you can configure it: @@ -174,7 +174,7 @@ For guidance on installing an Ingress controller, you can refer to the [Ingress Do not confuse the [ingress-nginx controller](https://github.com/kubernetes/ingress-nginx) with the [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift) that is endorsed by Red Hat for usage with OpenShift. Despite very similar names, they are two different products. -If you should decide to use the Red Hat endorsed [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift), you would require additional adjustments done to the Camunda 8 ingress objects and the NGINX Ingress Controller itself to make `gRPC` and `HTTP/2` connections work. In that case, please refer to the [example and the prerequisites](https://github.com/nginxinc/kubernetes-ingress/blob/main/examples/ingress-resources/grpc-services/README.md). +If you should decide to use the Red Hat endorsed [NGINX Ingress Controller](https://www.redhat.com/en/blog/using-nginx-ingress-controller-red-hat-openshift), you would require additional adjustments done to the Camunda 8 Ingress objects and the NGINX Ingress Controller itself to make `gRPC` and `HTTP/2` connections work. In that case, please refer to the [example and the prerequisites](https://github.com/nginxinc/kubernetes-ingress/blob/main/examples/ingress-resources/grpc-services/README.md). ::: @@ -193,25 +193,39 @@ To use these routes for the Zeebe Gateway, configure this through Ingress as wel As the Zeebe Gateway also uses `gRPC` (which relies on `HTTP/2`), [HTTP/2 Ingress Connectivity has to be enabled](https://docs.openshift.com/container-platform/latest/networking/ingress-operator.html#nw-http2-haproxy_configuring-ingress). +Additionally, the Zeebe Gateway should be configured to use an encrypted connection with TLS. In OpenShift, the connection from HAProxy to the Zeebe Gateway service can use HTTP/2 only for re-encryption or pass-through routes, and not for edge-terminated or insecure routes. + #### Required Steps -1. Provide [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway. The [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html) might be helpful here: +1. Provide two [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway. + + - The first TLS secret is issued to the Zeebe Gateway Service Name. This must use the [PKCS #8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [PKCS #1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as `camunda-platform-internal-service-certificate`. + + In the example below, a TLS certificate is generated for the Zeebe Gateway service with an [annotation](https://docs.openshift.com/container-platform/latest/security/certificates/service-serving-certificate.html). The generated certificate will be in the form of a secret. - - One issued to the Zeebe Gateway Service Name. This must use the [pkcs8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [pkcs1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as **Service Certificate Secret** or ``. For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes). + ```yaml + zeebeGateway: + service: + annotations: + service.beta.openshift.io/serving-cert-secret-name: camunda-platform-internal-service-certificate + ``` -
- pkcs8, pkcs1 syntax + Another option is [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html). For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes). - > PKCS1 private key encoding. PKCS1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS8. +
+ PKCS #8, PKCS #1 syntax - > PKCS8 private key encoding. PKCS8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header. + > PKCS #1 private key encoding. PKCS #1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS #8. - [pkcs1, pkcs8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding) -
+ > PKCS #8 private key encoding. PKCS #8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header. - - One that is used on the exposed route, referenced as **External URL Certificate Secret** or ``. + [PKCS #1, PKCS #8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding) -2. Configure your Zeebe Gateway Ingress to create a [Re-encrypt Route](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration): +
+ + - The second TLS secret is used on the exposed route, referenced as `camunda-platform-external-certificate`. For example, this would be the same TLS secret used for Ingress. + +1. Configure your Zeebe Gateway Ingress to create a [Re-encrypt Route](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration): ```yaml zeebeGateway: @@ -219,11 +233,11 @@ zeebeGateway: grpc: annotations: route.openshift.io/termination: reencrypt - route.openshift.io/destination-ca-certificate-secret: + route.openshift.io/destination-ca-certificate-secret: camunda-platform-internal-service-certificate className: openshift-default tls: enabled: true - secretName: + secretName: camunda-platform-external-certificate ``` 3. Mount the **Service Certificate Secret** to the Zeebe Gateway Pod: @@ -247,14 +261,14 @@ zeebeGateway: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt defaultMode: 420 - name: key secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.key path: tls.key @@ -272,6 +286,8 @@ operate: value: "true" - name: CAMUNDA_OPERATE_ZEEBE_CERTIFICATEPATH value: /usr/local/operate/config/tls.crt + - name: CAMUNDA_OPERATE_ZEEBE_BROKERCONTACTPOINT + value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500 extraVolumeMounts: - name: certificate mountPath: /usr/local/operate/config/tls.crt @@ -279,7 +295,7 @@ operate: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt @@ -297,6 +313,8 @@ tasklist: value: "true" - name: CAMUNDA_TASKLIST_ZEEBE_CERTIFICATEPATH value: /usr/local/tasklist/config/tls.crt + - name: CAMUNDA_TASKLIST_ZEEBE_BROKERCONTACTPOINT + value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500 extraVolumeMounts: - name: certificate mountPath: /usr/local/tasklist/config/tls.crt @@ -304,7 +322,7 @@ tasklist: extraVolumes: - name: certificate secret: - secretName: + secretName: camunda-platform-internal-service-certificate items: - key: tls.crt path: tls.crt @@ -313,7 +331,61 @@ tasklist: The actual configuration properties can be reviewed [in the Tasklist configuration documentation](/self-managed/tasklist-deployment/tasklist-configuration.md#zeebe-broker-connection). -5. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS. +5. For Connectors: + :::note + +The following will no longer be required when [the Connectors component supports PKCS #1 and PKCS #8](https://github.com/camunda/connectors/issues/2806). + +::: + +The Connectors component only accepts a `jks` (Java KeyStore) certificate. +If you have followed our previous recommendation of generating a TLS certificate using the OpenShift annotation, you will have a `PKCS #1` certificate the Connectors component will not accept. + +Below are a number of commands that convert the `PKCS #1` certificate generated by OpenShift to a `jks` format the Connectors component accepts: + +```bash +# Grab OpenShift generated TLS certificate. +kubectl get secret -n camunda camunda-zeebe-gateway -o jsonpath="{.data['tls\.crt']}" | base64 --decode > tls.crt +# Grab OpenShift generated TLS key. +kubectl get secret -n camunda camunda-zeebe-gateway -o jsonpath="{.data['tls\.key']}" | base64 --decode > zeebe-key.key +# Convert Zeebe Gateway unencrypted TLS key to an encrypted key. You will be prompted to enter a password when running this command. Note down the password: +openssl pkcs8 -topk8 -inform PEM -outform PEM -in ./zeebe-key.key -out ./zeebe-encrypted-key-gen.pem -v2 des3 +# Convert PKCS #1 certificate to PKCS #12. Again, you will be prompted to enter the password. +openssl pkcs12 -export -in tls.crt -inkey zeebe-encrypted-key-gen.pem -out zeebe-p12-certificate.p12 -name "certificate" +# Convert PKCS #12 certificate to jks cert. +keytool -importkeystore -srckeystore zeebe-p12-certificate.p12 -srcstoretype pkcs12 -destkeystore keystore.jks +``` + +Create a generic TLS secret from the `jks` file: + +```bash +kubectl create secret generic keystore -n camunda --from-file keystore.jks +``` + +Once the secret is created, follow the below example `values.yaml` config: + +```yaml +connectors: + inbound: + mode: oauth + env: + - name: ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS + value: "camunda-zeebe-gateway.camunda.svc.cluster.local:26500" + - name: ZEEBE_CLIENT_SECURITY_PLAINTEXT + value: "false" + - name: JAVA_TOOL_OPTIONS + value: "-Djavax.net.ssl.trustStore=/usr/local/certificates/keystore.jks -Djavax.net.ssl.trustStorePassword=changeit" + extraVolumeMounts: + - name: keystore + readOnly: true + mountPath: /usr/local/certificates + extraVolumes: + - name: keystore + secret: + secretName: keystore +``` + +6. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS.