From 7537908bb89cbd5484e97203abc638d6176a68ce Mon Sep 17 00:00:00 2001 From: Langleu Date: Tue, 17 Dec 2024 10:11:02 +0100 Subject: [PATCH] docs(ref/manual): address review feedback --- .../reference-architecture/manual/manual.md | 8 +-- .../reference-architecture.md | 18 +++---- .../setup/deploy/amazon/aws-ec2.md | 51 +++++++++---------- 3 files changed, 35 insertions(+), 42 deletions(-) diff --git a/docs/self-managed/reference-architecture/manual/manual.md b/docs/self-managed/reference-architecture/manual/manual.md index cafa841daf..f7a8982b37 100644 --- a/docs/self-managed/reference-architecture/manual/manual.md +++ b/docs/self-managed/reference-architecture/manual/manual.md @@ -28,11 +28,11 @@ This section includes deployment reference architectures for manual setups: Before starting with the self-managed single JAR setup, consider evaluating your deployment platform options (Kubernetes / Docker) based on your familiarity with these platforms. Self-managing your deployment requires a solid understanding of infrastructure, networking, and application management. If you prefer a simpler and more managed solution, you may want to explore [our SaaS offerings](https://camunda.com/platform/). SaaS can significantly reduce maintenance efforts, allowing you to focus on your core business needs. -## Limitations +## Considerations -- The focus is on the orchestration cluster. This includes the single JAR compromised of Identity, Operate, Optimize, Tasklist, and Zeebe, as well as the Connectors runtime. +- This overview page focuses on deploying the [orchestration cluster](/self-managed/reference-architecture/reference-architecture.md#orchestration-cluster), the single JAR compromised of Identity, Operate, Optimize, Tasklist, and Zeebe, as well as the Connectors runtime. Web Modeler and Console deployments are not included. - General guidance and examples focuses on **unix** users, but can be adapted by Windows users with options like [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) or included `batch` files. -- The deployment of `Optimize` is still based on the old architecture and must run on a single node. +- The Optimize importer is not highly available and must only run once within the whole setup. ## Architecture @@ -40,7 +40,7 @@ Before starting with the self-managed single JAR setup, consider evaluating your This above diagram illustrates a single-machine deployment using the single JAR package. While simple and effective for lightweight setups, scaling to multiple machines requires careful planning. -Compared to the generalized architecture depicted in the [reference architecture](../reference-architecture.md#architecture), the `Optimize importer` can be enabled as part of the single JAR. +Compared to the generalized architecture depicted in the [reference architecture](/self-managed/reference-architecture/reference-architecture.md#architecture), the `Optimize importer` can be enabled as part of the single JAR. ### High Availability (HA) diff --git a/docs/self-managed/reference-architecture/reference-architecture.md b/docs/self-managed/reference-architecture/reference-architecture.md index f33dfa999a..a0441e3215 100644 --- a/docs/self-managed/reference-architecture/reference-architecture.md +++ b/docs/self-managed/reference-architecture/reference-architecture.md @@ -31,20 +31,14 @@ Deviations from the reference architecture are unavoidable. However, such change ## Architecture -### Orchestration cluster - -![Orchestration Cluster](./img/orchestration-cluster.jpg) - -### Web Modeler and Console - -![Web Modeler and Console](./img/management-cluster.jpg) - ### Orchestration cluster vs Web Modeler and Console When designing a reference architecture, it's essential to understand the differences between an orchestration cluster and Web Modeler and Console Self-Managed. These components play crucial roles in the deployment and operation of processes, but they serve different purposes and include distinct components. #### Orchestration Cluster +![Orchestration Cluster](./img/orchestration-cluster.jpg) + The orchestration cluster is the core of Camunda. The included components are: @@ -56,11 +50,15 @@ The included components are: - [Identity](/self-managed/identity/what-is-identity.md): A service for managing user authentication and authorization. - [Connectors](/components/connectors/introduction.md): Pre-built integrations for connecting Zeebe with external systems and services. -The orchestration cluster is central to Camunda's workflow automation, which focuses on managing and executing processes through interconnected components. Each component within the orchestration cluster is part of an integrated system that works together to provide end-to-end process orchestration. Zeebe handles workflow execution, while Operate provides monitoring, and other components such as Tasklist and Optimize add functionality like human task management and analytics. These components form a unified cluster that is tightly integrated to ensure seamless communication and data flow. This design ensures that all components are in sync, working collectively to maintain consistent state management, data integrity, and smooth process orchestration across the entire cluster. This architecture ensures reliable process execution with clear boundaries between each workflow engine's operation. +Each component within the orchestration cluster is part of an integrated system that works together to provide end-to-end process orchestration. These components form a unified cluster that is tightly integrated to ensure seamless communication and data flow. + +This design ensures that all components are in sync, working collectively to maintain consistent state management, data integrity, and smooth process orchestration across the entire cluster. This architecture ensures reliable process execution with clear boundaries between each workflow engine's operation. #### Web Modeler and Console -Web Modeler and Console are designed to oversee and manage multiple orchestration clusters. It offers tools and interfaces for administrators and developers to monitor clusters and create BPMN models. The management cluster operates independently of the orchestration cluster and can function without requiring an orchestration cluster. +![Web Modeler and Console](./img/management-cluster.jpg) + +Web Modeler and Console are designed to interact with multiple orchestration clusters. Console offers tools and interfaces for administrators to monitor clusters, and Web Modeler allows developers to create and deploy BPMN models. - [Console](/components/console/introduction-to-console.md): A central management interface for monitoring and managing multiple orchestration clusters. - [Web Modeler](/self-managed/modeler/web-modeler/installation.md): A web-based tool for designing and deploying workflow models to any available orchestration cluster. diff --git a/docs/self-managed/setup/deploy/amazon/aws-ec2.md b/docs/self-managed/setup/deploy/amazon/aws-ec2.md index 6e2bca2d39..f6af0f628d 100644 --- a/docs/self-managed/setup/deploy/amazon/aws-ec2.md +++ b/docs/self-managed/setup/deploy/amazon/aws-ec2.md @@ -8,12 +8,10 @@ This guide provides a detailed walkthrough for installing the Camunda 8 single J This guide focuses on setting up the [orchestration cluster](/self-managed/reference-architecture/reference-architecture.md#orchestration-cluster-vs-web-modeler-and-console) for Camunda 8. The Web Modeler and Console are not covered in this manual deployment approach. These components are supported on Kubernetes and should be [deployed using Kubernetes](/self-managed/setup/install.md/#install-web-modeler). -## Disclaimer - :::note Using other Cloud provider -This guide is not limited to AWS but is built around the available tools and services that AWS offers. The scripts and ideas behind it can easily be adjusted for any other cloud provider and use case. +This guide is built around the available tools and services that AWS offers, but is not limited to AWS. The scripts and ideas included can be adjusted for any other cloud provider and use case. -However, if you wish to use this guide with a different cloud provider, please note that you will be responsible for configuring and maintaining the resulting infrastructure. Our support is limited to questions related to the guide itself, not to the specific tools and services of the chosen cloud provider. +When using this guide with a different cloud provider, note that you will be responsible for configuring and maintaining the resulting infrastructure. Our support is limited to questions related to the guide itself, not to the specific tools and services of the chosen cloud provider. ::: :::warning Cost management @@ -53,7 +51,7 @@ Both types of subnets are distributed over three availability zones of a single Alternatively, the same setup can run with a single AWS EC2 instance, but be aware that in case of a zone failure, the whole setup would be unreachable. ::: -## Prerequisites +## Requirements - An AWS account to create any resources within AWS. - On a high level, permissions are required on the **ec2**, **iam**, **elasticloadbalancing**, **kms**, **logs**, and **es** level. @@ -64,7 +62,6 @@ Alternatively, the same setup can run with a single AWS EC2 instance, but be awa ### Considerations -- TODO: reference official limitations if there are any - The Optimize importer is not highly available and must only run once within the whole setup. ### Outcome @@ -72,7 +69,12 @@ Alternatively, the same setup can run with a single AWS EC2 instance, but be awa The outcome is a fully working Camunda orchestration cluster running in a high availability setup using AWS EC2 and utilizing a managed OpenSearch domain. The EC2 instances come with an extra disk meant for Camunda to ensure that the content is separated from the operating system. -## Get started +## Create required infrastructure + +:::note Terraform infrastructure example +We do not recommend using the below Terraform related infrastructure as module as we do not guarantee compatibility. +Therefore, we recommend extending or reusing some elements of the Terraform example to ensure compatibility with your environments. +::: 1. **Download the reference architecture GitHub repository** to directly reuse and extend the existing Terraform example base. The idea is to provide an easy to use example implementation that's flexible to extend to your own needs without the potential limitations of a Terraform module. @@ -81,20 +83,13 @@ The EC2 instances come with an extra disk meant for Camunda to ensure that the c wget https://github.com/camunda/camunda-deployment-references/archive/refs/heads/main.zip ``` -### Infrastructure - -:::note Terraform infrastructure example -We do not recommend using the below Terraform related infrastructure as module as we do not guarantee compatibility. -Therefore, we recommend extending or reusing some elements of the Terraform example to ensure compatibility with your environments. -::: - -1. **Go to directory** +2. **Go to directory** ```sh cd camunda-deployment-references-main/aws/ec2/terraform ``` -2. **Configure Variables** +3. **Configure Variables** Edit the `variables.tf` file to customize the settings such as the prefix for resource names and CIDR blocks. Example: @@ -108,7 +103,7 @@ variable "cidr_blocks" { } ``` -3. **Configure alternative Terraform backend** +4. **Configure alternative Terraform backend** It's recommended to use a different backend than `local` as it is just meant for testing and development purposes. The state would be saved locally and does not allow to easily share it with colleagues. @@ -116,7 +111,7 @@ More information on alternatives can be found in the [Terraform documentation](h - In the `config.tf` change the `backend "local"` to e.g. [AWS 3](https://developer.hashicorp.com/terraform/language/backend/s3) or any other non `local` one that fits your organization. -4. **Configure Terraform AWS provider** +5. **Configure Terraform AWS provider** 1. **Add the [Terraform AWS provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) in the `config.tf`** @@ -148,7 +143,7 @@ More information on alternatives can be found in the [Terraform documentation](h We strongly recommend managing sensitive information using a secure secrets management solution like HashiCorp Vault. For details on how to inject secrets directly into Terraform via Vault, see the [Terraform Vault Secrets Injection Guide](https://developer.hashicorp.com/terraform/tutorials/secrets/secrets-vault). ::: -5. **Initialize Terraform** +6. **Initialize Terraform** Initialize the Terraform working directory. This step downloads the necessary provider plugins. @@ -156,7 +151,7 @@ Initialize the Terraform working directory. This step downloads the necessary pr terraform init ``` -6. **Deploy the Infrastructure** +7. **Deploy the Infrastructure** Plan the configuration files: @@ -172,7 +167,7 @@ terraform apply infrastructure.plan # apply the creation The execution takes roughly 30 minutes of which the majority of time ~ 25 minutes is solely the creation of a managed highly available OpenSearch cluster. -7. **Access outputs** +8. **Access outputs** After the infrastructure is created, you can access the outputs defined in `outputs.tf`. For example, to get the OpenSearch endpoint: @@ -180,7 +175,7 @@ After the infrastructure is created, you can access the outputs defined in `outp terraform output aws_opensearch_domain ``` -8. **(optional) Connect to remote machines via Bastion host** +9. **(optional) Connect to remote machines via Bastion host** The EC2 instances are not public and have to be reached via a Bastion host. Alternatively one can utilize the [AWS VPN Client](https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html) to connect securely to a private VPC. This is not covered here as the setup requires a lot more manual user interaction. @@ -193,7 +188,7 @@ export CAMUNDA_IP=$(tf output -json camunda_ips | jq -r '.[0]') ssh -J admin@${BASTION_HOST} admin@${CAMUNDA_IP} ``` -### Deploy Camunda 8 +## Deploy Camunda 8 1. **Go to the script directory** @@ -267,7 +262,7 @@ If you've used the bastion host for access, you can turn it off when it's no lon To do this, set the `enable_jump_host` variable to `false` in the `variables.tf` file and reapply Terraform. -### Verify connectivity to Camunda 8 +## Verify connectivity to Camunda 8 Using Terraform, you can obtain the HTTP endpoint of the Application Load Balancer and interact with Camunda through the [REST API](/apis-tools/camunda-api-rest/camunda-api-rest-overview.md). @@ -289,7 +284,7 @@ terraform output -raw alb_endpoint With the endpoint you can follow the example as outlined in [the documentation](/apis-tools/camunda-api-rest/camunda-api-rest-authentication.md) to authenticate and retrieve the cluster topology. -### Upgrade Camunda 8 +## Upgrade Camunda 8 :::info Direct upgrade not supported Upgrading directly from a Camunda 8.6 release to 8.7 is not supported and cannot be performed. @@ -309,13 +304,13 @@ The update process can be automated using the `all-in-one-install.sh` script, wh - Regenerates configuration files. - Restarts the application to apply the updates. -### Monitoring +## Monitoring Our default way of exposing metrics is in the Prometheus format, please conduct the general [metrisc related documentation](/self-managed/zeebe-deployment/operations/metrics.md) to learn more how to scrape Camunda 8. In an AWS environment, you can leverage CloudWatch not only for log collection but also for gathering [Prometheus metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-Prometheus-metrics.html). It's important to note that while Camunda natively supports Grafana and Prometheus, integrating CloudWatch for metric visualization is possible but requires additional configuration. -### Backups +## Backups Please conduct the general topic of backups in the [documentation](/self-managed/operational-guides/backup-restore/backup-and-restore.md). @@ -323,7 +318,7 @@ With AWS as chosen platform you can utilize [S3](https://aws.amazon.com/s3/) for If you are using a managed OpenSearch domain instead, you should check out the [official documentation](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshots.html) on creating backups and snapshots in OpenSearch. -### Troubleshooting +## Troubleshooting Please conduct the general topic of troubleshooting in the [documentation](/self-managed/operational-guides/troubleshooting/troubleshooting.md).