Skip to content

Commit

Permalink
Backends functions docs update (#240)
Browse files Browse the repository at this point in the history
* Revise the K8s unit docs, add the unit description

* Update backend and functions docs
  • Loading branch information
anastyakulyk authored Nov 15, 2023
1 parent e2f888f commit 680ac85
Show file tree
Hide file tree
Showing 2 changed files with 111 additions and 15 deletions.
56 changes: 44 additions & 12 deletions docs/stack-templates-functions.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Functions

You can use [basic Go template language](https://golang.org/pkg/text/template/#hdr-Functions) and [Sprig](https://masterminds.github.io/sprig/) functions to modify the text of a stack template.
You can use [basic Go template language](https://golang.org/pkg/text/template/#hdr-Functions) and [Sprig functions](https://masterminds.github.io/sprig/) to modify a stack template.

Additionally, you can use some enhanced functions that are listed below. These functions are integrated with the `yaml` syntax and can't be used everywhere.
Additionally, you can use some enhanced functions that are listed below.

## `insertYAML`

Allows for passing `yaml` block as a value of target `yaml` template.
Pass `yaml` block as a value of target `yaml` template.

**Argument**: data to pass, any value or reference to a block.
**Allowed use**: only as full `yaml` value, in unit `inputs`. Example:

The `insertYAML` function is integrated with the `yaml` syntax and can be used only as a full `yaml` value in units input. Example:

Source `yaml`:

Expand Down Expand Up @@ -51,21 +51,21 @@ Rendered stack template:

## `remoteState`

Allows for passing data across units and stacks, can be used in pre/post hooks.
Pass data across units and stacks, can be used in pre/post hooks.

**Argument**: string, path to remote state consisting of 3 parts separated by a dot: `"stack_name.unit_name.output_name"`. Since the name of the stack is unknown inside the stack template, you can use "this" instead:`"this.unit_name.output_name"`.

**Allowed use**:

* all units types: in `inputs`;
The `remoteState` function is integrated with the `yaml` syntax and can be used in following cases:

* In units' inputs (all types of units)

* all units types: in units pre/post hooks;
* In units' pre/post hooks (all types of units)

* in Kubernetes modules: in Kubernetes manifests.
* In Kubernetes manifests (Kubernetes units)

## `cidrSubnet`

Calculates a subnet address within given IP network address prefix. Same as [Terraform function](https://www.terraform.io/docs/language/functions/cidrsubnet.html). Example:
Calculate a subnet address within given IP network address prefix. Same as [Terraform function](https://www.terraform.io/docs/language/functions/cidrsubnet.html). Example:

Source:
```bash
Expand All @@ -76,3 +76,35 @@ Calculates a subnet address within given IP network address prefix. Same as [Ter
```bash
172.18.0.0/16
```

## `readFile`

Read the passed file and return its contents as a string.

**Argument**: path. The `readFile` function supports both absolute `readFile /path/to/file.txt` and relative paths `readFile ./files/data.yaml`.

A relative path must refer to a location where the function is used. When it is used in a template, the path's base folder will be the template directory. When it is used in one of the project files, the path will begin with the project directory.

!!! Note

The file is read as is; templating is not applied.

## `workDir`

Return absolute path to a working directory where a project runs, and the Terraform code is generated.

!!! Warning

Use the function with care since the absolute path that it returns varies depending on running conditions. This can affect the Cluster.dev's state.

## `reqEnv`

Return an environment variable required for a system to run. Using the `reqEnv` function without specifying the variable will result in failing `cdev apply` with an error message.

## `bcrypt`

Apply the bcrypt encryption algorithm to a passed string.

!!! Warning

Use the function with care since it returns a unique hash with each calling, which can affect the Cluster.dev’s state.
70 changes: 67 additions & 3 deletions docs/structure-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Remote backend uses remote cloud services to store the cluster state, making it

### `s3`

Stores the cluster state in AWS S3 bucket. The S3 backend supports all options of [Terraform S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3) backend.
Stores the cluster state in AWS S3 bucket. The Cluster.dev S3 backend supports some options of [Terraform S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3) backend. The list of supported options is referred below.

```yaml
name: aws-backend
Expand All @@ -39,9 +39,55 @@ spec:
region: {{ .project.variables.region }}
```

#### Options

* `bucket` - *required*. The name of the S3 bucket.

* `region` - *required*. AWS Region of the S3 Bucket and DynamoDB Table (if used). This can also be sourced from the `AWS_DEFAULT_REGION` and `AWS_REGION` environment variables.

* `access_key` - *optional*. AWS access key. If configured, must also configure `secret_key`. This can also be sourced from the `AWS_ACCESS_KEY_ID` environment variable, AWS shared credentials file (e.g. ~/.aws/credentials), or AWS shared configuration file (e.g. ~/.aws/config).

* `secret_key` - *optional*. AWS access key. If configured, must also configure `access_key`. This can also be sourced from the `AWS_SECRET_ACCESS_KEY` environment variable, AWS shared credentials file (e.g. ~/.aws/credentials), or AWS shared configuration file (e.g. ~/.aws/config).

* `profile` - *optional*. Name of AWS profile in AWS shared credentials file (e.g. ~/.aws/credentials) or AWS shared configuration file (e.g. ~/.aws/config) to use for credentials and/or configuration. This can also be sourced from the `AWS_PROFILE` environment variable.

* `token` - *optional*. Multi-Factor Authentication (MFA) token. This can also be sourced from the `AWS_SESSION_TOKEN` environment variable.

* `endpoint` - *optional*. Custom endpoint URL for the AWS S3 API.

* `skip_metadata_api_check` - *optional*. Skip usage of EC2 Metadata API.

* `skip_credentials_validation` - *optional*. Skip credentials validation via the STS API. Useful for testing and for AWS API implementations that do not have STS available.

* `max_retries` - *optional*. The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5.

* `shared_credentials_file` - *optional*, *deprecated*, use `shared_credentials_files` instead. Path to the AWS shared credentials file. Defaults to ~/.aws/credentials.

* `skip_region_validation` - *optional*. Skip validation of provided region name.

* `sts_endpoint` - *optional*, *deprecated*. Custom endpoint URL for the AWS Security Token Service (STS) API. Use `endpoints.sts` instead.

* `iam_endpoint` - *optional*, *deprecated*. Custom endpoint URL for the AWS Identity and Access Management (IAM) API. Use `endpoints.iam` instead.

* `force_path_style` - *optional*, *deprecated*. Enable path-style S3 URLs (https://<HOST>/<BUCKET> instead of https://<BUCKET>.<HOST>).

* `assume_role_policy` - *optional*. IAM Policy JSON describing further restricting permissions for the IAM Role being assumed. Use `assume_role.policy` instead.

* `assume_role_policy_arns` - *optional*. Set of Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed. Use `assume_role.policy_arns` instead.

* `assume_role_tags` - *optional*. Map of assume role session tags. Use `assume_role.tags` instead.

* `assume_role_transitive_tag_keys` - *optional*. Set of assume role session tag keys to pass to any subsequent sessions. Use `assume_role.transitive_tag_keys` instead.

* `external_id` - *optional*. External identifier to use when assuming the role. Use `assume_role.external_id` instead.

* `role_arn` - *optional*. Amazon Resource Name (ARN) of the IAM Role to assume. Use `assume_role.role_arn` instead.

* `session_name` - *optional*. Session name to use when assuming the role. Use `assume_role.session_name` instead.

### `azurerm`

Stores the cluster state in Microsoft Azure cloud. The `azurerm` backend supports all options of [Terraform azurerm](https://www.terraform.io/language/settings/backends/azurerm) backend.
Stores the cluster state in Microsoft Azure cloud. The `azurerm` backend supports the options of [Terraform azurerm](https://www.terraform.io/language/settings/backends/azurerm) backend.

```yaml
name: azurerm-b
Expand All @@ -55,7 +101,7 @@ spec:

### `gcs`

Stores the cluster state in Google Cloud service. The `gcs` backend supports all options of [Terraform gcs](https://www.terraform.io/language/settings/backends/gcs) backend.
Stores the cluster state in Google Cloud service. The `gcs` backend supports some options of [Terraform gcs](https://www.terraform.io/language/settings/backends/gcs) backend. The list of supported options is referred below.

```yaml
name: gcs-b
Expand All @@ -66,6 +112,24 @@ spec:
prefix: pref
```

#### Options

* `bucket` - *required*. The name of the GCS bucket. This name must be globally unique. For more information, see [Bucket Naming Guidelines](https://cloud.google.com/storage/docs/buckets#naming).

* `credentials` / `GOOGLE_BACKEND_CREDENTIALS` / `GOOGLE_CREDENTIALS` - *optional*. Local path to Google Cloud Platform account credentials in JSON format. If unset, the path uses [Google Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials). The provided credentials must have the Storage Object Admin role on the bucket. **Warning**: if using the Google Cloud Platform provider as well, it will also pick up the `GOOGLE_CREDENTIALS` environment variable.

* `impersonate_service_account` / GOOGLE_BACKEND_IMPERSONATE_SERVICE_ACCOUNT / GOOGLE_IMPERSONATE_SERVICE_ACCOUNT - *optional*. The service account to impersonate for accessing the State Bucket. You must have `roles/iam.serviceAccountTokenCreator` role on that account for the impersonation to succeed. If you are using a delegation chain, you can specify that using the `impersonate_service_account_delegates` field.

* `impersonate_service_account_delegates` - *optional*. The delegation chain for an impersonating a service account as described [here](https://cloud.google.com/iam/docs/create-short-lived-credentials-direct#sa-credentials-delegated).

* `access_token` - *optional*. A temporary [OAuth 2.0 access token] obtained from the Google Authorization server, i.e. the `Authorization: Bearer` token used to authenticate HTTP requests to GCP APIs. This is an alternative to credentials. If both are specified, `access_token` will be used over the credentials field.

* `prefix` - *optional*. GCS prefix inside the bucket. Named states for workspaces are stored in an object called `<prefix>/<name>.tfstate`.

* `encryption_key` / GOOGLE_ENCRYPTION_KEY - *optional*. A 32 byte base64 encoded 'customer-supplied encryption key' used when reading and writing state files in the bucket. For more information see [Customer-supplied Encryption Keys](https://cloud.google.com/storage/docs/encryption/customer-supplied-keys).

* `storage_custom_endpoint` / GOOGLE_BACKEND_STORAGE_CUSTOM_ENDPOINT / GOOGLE_STORAGE_CUSTOM_ENDPOINT - *optional*. A URL containing three parts: the protocol, the DNS name pointing to a Private Service Connect endpoint, and the path for the Cloud Storage API (`/storage/v1/b`, see [here](https://cloud.google.com/storage/docs/json_api/v1/buckets/get#http-request)). You can either use [a DNS name automatically made by the Service Directory](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#configure-p-dns) or a [custom DNS name](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#configure-dns-default) made by you. For example, if you create an endpoint called `xyz` and want to use the automatically-created DNS name, you should set the field value as `https://storage-xyz.p.googleapis.com/storage/v1/b`. For help creating a Private Service Connect endpoint using Terraform, see [this guide](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#terraform_1).

### Digital Ocean Spaces and MinIO

To use DO spaces or MinIO object storage as a backend, use `s3` backend provider with additional options. See details:
Expand Down

0 comments on commit 680ac85

Please sign in to comment.