Terraform Installer For Seqera Platform Enterprise (Docker-Compose)

To launch pipelines with Seqera Platform Enterprise, you first need to deploy a Platform instance. A successful deployment requires you to make decisions about networking, security posture, container orchestration, permissions, etc. These decisions can be overwhelming to a client administrator who just wants to get their users launching Nextflow pipelines from the platform.

The Seqera CX team has developed a field tool to help simplify these initial efforts in two ways:

1. Reduce the infrastructure and configuration burden, so that a first-time deployment of Seqera Platform Enterprise requires less than 1 hour of work and only a few minutes for subsequent redeployments.

2. Provide an Infrastructure-As-Code (IaC) solution so activities are repeatable and artifacts can be checked into source-control.

Disclaimer - Use at your own discretion!!

This is an unofficial field tool.

The solution is delivered on a best-effort basis, but provides no guarantees of appropriateness for your specific scenario. Please conduct your due diligence prior to execution within your environment.

• Information about project structure, design decisions, and assumptions made can be found in Design Decisions.
• Information on existing deficiencies and gotchas can be found in Deficiencies & Gotchas.

Appropriateness criteria

The out-of-the-box solution depends on you meeting the following criteria:

1. You are a client of Seqera Labs.
2. You will run Seqera Platform in AWS.
3. Your corporate policies allow you to store secrets in AWS Systems Manager Parameter Store (SSM).
4. You have access to a local Linux-based terminal (Mac supported, with caveats).
5. You can use Terraform to provision infrastructure.
6. You use OpenSSH and can maintain a ~/.ssh/config file.

Tool dependencies

1. Install terraform v1.3.7 or later:

   • https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli

2. Install and configure the latest aws cli (version 2):

   • https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

3. Install a modern version of openssh:

   • https://www.openssh.com/

4. Install git v2.25.1 or later.

5. Install GNU Make 4.2.1 or later.

6. Install python 3.8.10 or later.

Tool permissions

Please see Permissions for a full list of necessary AWS IAM permissions required to run the tool.

Tool security scanning

Please see Security for details on how we scan for / mitigate tool security vulnerabilities.

Configuration steps

The tool relies on four core configuration files, all stored within the templates folder. These files must be prepared prior to project execution:

1. terraform.tfvars
   Used to set flags and values that control how your infrastructure and the Seqera Platform Enterprise instance is built and configured.

   This file must be configured and stored in the project folder root.

2. ssm_sensitive_values_tower.json
   Used to define secrets like credentials and passwords which the Seqera Platform needs to fulfill its duties.

   This file exists to make SSM data population more convenient and should not be stored in source-control.

3. ssm_sensitive_values_seqerakit.json (Optional)
   Used to define secrets like tokens and credentials which the Seqerakit tool needs to fulfill its duties.

   This file exists to make SSM data population more convenient and should not be stored in source-control.

4. ssm_sensitive_values_groundswell.json (Optional)
   Used to define credentials for the Groundswell optimization service.

   This file exists to make SSM data population more convenient and should not be stored in source-control.

01 - Clone the repository

1. Download a copy of the repository to your local workstation:

   git clone <TODO_path_to_offical_repo> && cd <TODO_name_of_directory>

02 - Prepare secrets

1. Select a name for your Seqera Platform application (default: tower-dev).

   This is a namespace isolator which prevents concurrent instances (e.g., dev and prod) from accidentally sharing configurations.

2. Modify the ssm_sensitive_values_tower.json in the templates folder:

   1. Perform a find-and-replace on tower-dev if you select a different application name.

   2. Replace every instance of CHANGE_ME in the file with the values supplied to you during your Seqera onboarding.

   3. Replace every instance of CHANGE_ME_SEE_SEQERA_DOCS with your own values. Configuration value guidance can be found at Seqera Platform Enterprise configuration.

   4. Review each instance of CHANGE_ME_IF_NECESSARY and replace as needed.

   WARNING:

   1. Do not change ssm_key entries apart from the application name. Path changes will prevent the application from booting.
   2. SSM entries have a limit of 4096 characters. Very long client-supplied values may result in errors when you try to create the entry.

3. Modify the ssm_sensitive_values_seqerakit.json in the templates folder:

   1. Execute a find-and-replace on tower-dev if you select a different application name.

   2. Replace every instance of CHANGE_ME in the file with the values appropriate for your organization.

   3. Review each instance of CHANGE_ME_IF_NECESSARY and replace as needed.

      An AWS credential set is a mandatory minimum. The terraform.tfvars file has flags that can enable/disable expectations regarding the AWS IAM role, Github credential, and Docker credential.

      WARNING:

      1. Do not change ssm_key entries apart from the application name. Path changes will prevent the application from booting.

4. Modify the ssm_sensitive_values_groundswell.json in the templates folder:

   1. Execute a find-and-replace on tower-dev if you select a different application name.

   2. Replace default values if necessary.

5. Decide on the AWS Parameter Store prefixes where you will store the payloads of your now-populated ssm_* files.

   Default values in terraform.tfvars are:

   • /seqera/sensitive_values/tower_dev/tower
   • /seqera/sensitive_values/tower_dev/seqerakit
   • /seqera/sensitive_values/tower_dev/groundswell

6. Create the entries and paste the appropriate JSON payload for each. Ensure that:

   1. Type is set to SecureString.
   2. Data type is text.

7. Remember your application name and SSM prefixes, as these need to be supplied during the configuration of the terrform.tfvars file.

03 - Prepare the terraform.tfvars file

NOTE: Fulsome field-level documentation exists inside the terraform.tfvars file. Consult there for details beyond the high-level instructions here.

1. Create a working copy of the terraform.tfvars file in the project root.

   # In project root
   cp templates/TEMPLATE_terraform.tfvars terraform.tfvars

2. Update the app_name, secrets_bootstrap_tower, secrets_bootstrap_seqerakit, and secrets_bootstrap_groundswell with the values you selected during the secrets preparation phase.

3. Replace every instance of REPLACE_ME with values appropriate for your organization.

4. Set flag_* variables to true / false to match your desired implementation architecture.

5. Review each instance of REPLACE_ME_IF_NECESSARY and modify as required (these decisions will be driven by the flag activations).

6. Review and modify placehoder values as necessary. Do not modify commented fields with DO_NOT_UNCOMMENT_ME values. These are in the file solely to provide a visual reminder that these keys are being set behind the scene via SSM secrets values.

04 - Create an AWS IAM Role with the necessary permissions

1. Modify the JSON block in the documentation folder:

   1. Perform a find-and-replace on APP_NAME_REPLACE_ME (suggested default is tower-dev).

   2. Replace every AWS_REGION_REPLACE_ME with your desired AWS region.

   3. Replace every AWS_ACCOUNT_REPLACE_ME instance with the id of the AWS Account which the Terraform installer will interact with.

2. Create an AWS IAM Role / User and attach these permissions.

3. Configure this identity into your AWS CLI and ensure it is active when executing the installer (specified in var.aws_profile).

05 - Modify your OpenSSH config

During the installation process, a SSH config file is created and used to connect to the EC2 instance hosting the Seqera Platform instance. To use the SSH config file successfully, you need a minor modification to your openssh configuration:

# Add the following entry AT THE TOP of your ~/.ssh/config file
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT/ssh_config

# NOTE: IF you intend to run multiple instances on the same host (e.g. dev and staging), add an Include to each project folder.
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT_DEV/ssh_config
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT_STAGING/ssh_config

06 - Update your Git repo settings

The project ships with a .githooks folder, which contains a Python script that wil scan your terraform.tfvars file for configuration mismatches which cause your deployment to fail.

To automatically invoke the script prior to a commit to your git repository, execute the following in the root of your project: git config core.hooksPath .githooks.

07 - Review your Terraform state storage strategy

By default, the installer writes the Terraform state to a local folder (<PROJECT_ROOT>/DONTDELETE). This is convenient for initial testing but likely not a good long-term solution for most clients.

You can change the state management strategy at the top of the 000-main.tf file. We have provided a commented-out example that demonstrates how to write state to an S3 bucket.

Note: Bulletproof state management can become complex and is beyond the scope of this initiative. Consult Terraform's official docs on Remote State and modify your project as necessary to meet your organization's needs.

Execution steps

Deployment

1. Within your terminal, navigate to the project root and initialize the project:

   terraform init

2. Create and review an execution plan:

   # Recommended approach. Execute the Seqera-supplied Python script to check your `terraform.tfvars` file for known configuration conflicts prior to terraform binary invocation.
   make plan
   
   # Alternative approach to execute plan without Python script verification execution.
   terraform plan

3. Execute the actions reviewed in the Terraform plan:

   # Recommended approach. Execute the Seqera-supplied Python script to check your  `terraform.tfvars` file for known configuration conflicts prior to terraform binary invocation.
   make apply
   
   # Alternative approach to execute plan without Python script verification execution.
   # Note: You can append `--auto-append` to the end of the command to avoid the need to type 'yes' to approve the deployment.
   terraform apply

Teardown

1. To destroy the infrastructure created via Terraform, execute the following command:

   terraform destroy

Logging

The tool offers the ability to configure the specific docker logging driver used to capture Tower container logs. By default, the local file logging driver is configured.

WARNINGS

1. If a database (regardless if container or RDS) was created as part of the deployment, teardown will destroy it and all data within.
   Prior to deletion, consider backing up your database if the data may be needed in future.

2. Terraform is not aware of actions executed within the Seqera Platform (i.e., invocation of Tower Forge to create compute environments by users / automation tools like Seqerakit). Executing terraform destroy without first conducting a purge of objects within Seqera Platform will result in orphaned assets in your AWS Account.

Multiple Deployment Consideration

Given client environment variability, Seqera offers no official guidance re: how best to run multiple concurrent implementations (e.g. entirely separate repositories, different branches in a monorepo, Terraform workspaces, git submodules, etc). Each site must decide what is best for them and implement accordingly.

With that said, for design purposes, this tool assumes that multiple project instances will live in the same filesystem (each within its own exclusive namespace). Each project's ssh_config file uses an alias matching the unique app_name from your tfvars file, making it possible to add multiple non-conflicting Include statements in your ~/.ssh/config.

Terraform asset details

Providers

Name	Version
aws	5.12.0
random	3.6.0
null	3.2.2
tls	4.0.5
template	2.2.0

Modules

Name	Source	Version
vpc	terraform-aws-modules/s3-bucket/aws	5.1.2
tower_eice_ingress_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_eice_egress_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_ec2_ssh_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_ec2_egress_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_ec2_direct_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_alb_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_db_sg	terraform-aws-modules/security-group/aws	5.1.0
tower_batch_sg	terraform-aws-modules/security-group/aws	5.1.0
rds	terraform-aws-modules/rds/aws	6.1.1
alb	terraform-aws-modules/alb/aws	8.7.0

Outputs

Name	Description
aws_account_id	AWS Account ID
aws_caller_arn	Assumed role used to deploy the project
aws_caller_user	User used to deploy the project
ec2_ssh_key	SSH key attached to the EC2 instance
tower_server_url	URL to check your Tower intance
route53_record_status	Identifies if a Route53 record was created or not
aws_ec2_private_ip	Private IP of the EC2 Instance
aws_ec2_public_ip	Public IP of the EC2 Instance (if applicable)
tower_api_endpoint	Platform API endpoint
seqera_configuration	Path to the SeqeraKit setup file
redis_endpoint	URL for your Redis instance