From 4558f28ca2682e05c6e75f89f771e0601e14ad90 Mon Sep 17 00:00:00 2001 From: Noah Stride Date: Wed, 21 Aug 2024 18:03:21 +0100 Subject: [PATCH] Machine ID: Document Kubernetes Workload Attestation (#45369) (#45584) * Add KWA features into the machine id config reference * Add example manifests for tbot service for workload attestation * Update docs/pages/enroll-resources/machine-id/workload-identity/workload-attestation.mdx * Add more detail to explanation of workload attestation * Add table * Update cspell * Fix spelling of connect * Fix spelling of attested --------- Co-authored-by: Paul Gottschling --- docs/cspell.json | 2 + .../machine-id/workload-identity.mdx | 1 + .../workload-attestation.mdx | 239 ++++++++++++++++++ .../reference/machine-id/configuration.mdx | 77 ++++-- 4 files changed, 300 insertions(+), 19 deletions(-) create mode 100644 docs/pages/enroll-resources/machine-id/workload-identity/workload-attestation.mdx diff --git a/docs/cspell.json b/docs/cspell.json index 9d5fb1b66b94c..6efcb0d625837 100644 --- a/docs/cspell.json +++ b/docs/cspell.json @@ -121,6 +121,7 @@ "Keycloak", "Keyspaces", "knownhosts", + "Kubelet", "Kubernetes", "Kubes", "LDAPS", @@ -292,6 +293,7 @@ "argoproj", "armv", "atburke", + "attested", "attrname", "attobj", "auditctl", diff --git a/docs/pages/enroll-resources/machine-id/workload-identity.mdx b/docs/pages/enroll-resources/machine-id/workload-identity.mdx index d8f78125ec175..af56023caad1f 100644 --- a/docs/pages/enroll-resources/machine-id/workload-identity.mdx +++ b/docs/pages/enroll-resources/machine-id/workload-identity.mdx @@ -148,6 +148,7 @@ Teleport's Workload Identity currently only supports issuing X.509-SVIDs. - [Getting Started](./workload-identity/getting-started.mdx): How to configure Teleport for Workload Identity. - [Best Practices](./workload-identity/best-practices.mdx): Best practices for using Workload Identity in Production. +- [Workload Attestation](./workload-identity/workload-attestation.mdx): Learn about using Workload Attestation to securely issue SVIDs to specific workloads. - [TSH Support](./workload-identity/tsh.mdx): How to use `tsh` with Workload Identity to issue SVIDs to users. - [AWS Roles Anywhere](./workload-identity/aws-roles-anywhere.mdx): Configuring AWS to accept Workload ID certificates as authentication using AWS Roles Anywhere. diff --git a/docs/pages/enroll-resources/machine-id/workload-identity/workload-attestation.mdx b/docs/pages/enroll-resources/machine-id/workload-identity/workload-attestation.mdx new file mode 100644 index 0000000000000..8bc4846432628 --- /dev/null +++ b/docs/pages/enroll-resources/machine-id/workload-identity/workload-attestation.mdx @@ -0,0 +1,239 @@ +--- +title: Workload Attestation +description: An overview of the Teleport Workload Identity Workload Attestation feature. +--- + + +Teleport Workload Identity is currently in Preview. This means that some +features may be missing. We're actively looking for design partners to help us +shape the future of Workload Identity and would love to +[hear your feedback](mailto:product@goteleport.com). + + +Workload Attestation is the process completed by `tbot` to assert the identity +of a workload that has connected to the Workload API and requested certificates. +The information gathered during attestation is used to decide which, if any, +SPIFFE IDs should be encoded into an SVID and issued to the workload. + +Workload Attestors are the individual components that perform this attestation. +They use the process ID of the workload to gather information about the workload +from platform-specific APIs. For example, the Kubernetes Workload Attestor +queries the local Kubelet API to determine which Kubernetes pod the process +belongs to. + +The result of the attestation process is known as attestation metadata. This +attestation metadata is referred to by the rules you configured for `tbot`'s +Workload API service. For example, you may state that only workloads running in +a specific Kubernetes namespace should be issued a specific SPIFFE ID. + +Additionally, this metadata is included in the log messages output by `tbot` +when it issues an SVID. This allows you to audit the issuance of SVIDs and +understand why a specific SPIFFE ID was issued to a workload. + +## Unix + +The Unix Workload Attestor is the most basic attestor and allows you to restrict +the issuance of SVIDs to specific Unix processes based on a range of criteria. + +### Attestation Metadata + +The following metadata is produced by the Unix Workload Attestor and is +available to be used when configuring rules for `tbot`'s Workload API service: + +| Field | Description | +|-------------------|------------------------------------------------------------------------------| +| `unix.attested` | Indicates that the workload has been attested by the Unix Workload Attestor. | +| `unix.pid` | The process ID of the attested workload. | +| `unix.uid` | The effective user ID of the attested workload. | +| `unix.gid` | The effective primary group ID of the attested workload. | + +## Kubernetes + +The Kubernetes Workload Attestor allows you to restrict the issuance of SVIDs +to specific Kubernetes workloads based on a range of criteria. + +It works by first determining the pod ID for a given process ID and then by +querying the local kubelet API for details about that pod. + +### Attestation Metadata + +The following metadata is produced by the Kubernetes Workload Attestor and is +available to be used when configuring rules for `tbot`'s Workload API service: + +| Field | Description | +|------------------------------|------------------------------------------------------------------------------------| +| `kubernetes.attested` | Indicates that the workload has been attested by the Kubernetes Workload Attestor. | +| `kubernetes.namespace` | The namespace of the Kubernetes Pod. | +| `kubernetes.service_account` | The service account of the Kubernetes Pod. | +| `kubernetes.pod_name` | The name of the Kubernetes Pod. | + +### Deployment Guidance + +To use Kubernetes Workload Attestation, `tbot` must be deployed as a daemon +set. This is because the unix domain socket can only be accessed by pods on the +same node as the agent. Additionally, the daemon set must have the `hostPID` +property set to `true` to allow the agent to access information about +processes within other containers. + +The daemon set must also have a service account assigned that allows it to query +the Kubelet API. This is an example role with the required RBAC: + +```yaml +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: tbot +rules: + - resources: ["pods","nodes","nodes/proxy"] + apiGroups: [""] + verbs: ["get"] +``` + +Mapping the Workload API Unix domain socket into the containers of workloads +can be done in two ways: + +- Directly configuring a hostPath volume for the `tbot` daemonset and workloads + which will need to connect to it. +- Using [spiffe-csi-driver](https://github.com/spiffe/spiffe-csi). + +Example manifests for required Kubernetes resources: + +```yaml +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: tbot +rules: + - resources: ["pods","nodes","nodes/proxy"] + apiGroups: [""] + verbs: ["get"] +--- +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: tbot +subjects: + - kind: ServiceAccount + name: tbot + namespace: default +roleRef: + kind: ClusterRole + name: tbot + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: tbot + namespace: default +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: tbot-config + namespace: default +data: + tbot.yaml: | + version: v2 + onboarding: + join_method: kubernetes + # replace with the name of a join token you have created. + token: example-token + storage: + type: memory + # ensure this is configured to the address of your Teleport Proxy Service. + proxy_server: example.teleport.sh:443 + services: + - type: spiffe-workload-api + listen: unix:///run/tbot/sockets/workload.sock + attestor: + kubernetes: + enabled: true + kubelet: + # skip verification of the Kubelet API certificate as this is not + # usually issued by the cluster CA. + skip_verify: true + # replace the svid entries with the SPIFFE IDs that you wish to issue, + # using the `rules` blocks to restrict these to specific Kubernetes + # workloads. + svids: + - path: /my-service + rules: + - kubernetes: + namespace: default + service_account: example-sa +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: tbot +spec: + selector: + matchLabels: + app: tbot + template: + metadata: + labels: + app: tbot + spec: + securityContext: + runAsUser: 0 + runAsGroup: 0 + hostPID: true + containers: + - name: tbot + image: public.ecr.aws/gravitational/tbot-distroless:(=teleport.version=) + imagePullPolicy: IfNotPresent + securityContext: + privileged: true + args: + - start + - -c + - /config/tbot.yaml + - --log-format + - json + volumeMounts: + - mountPath: /config + name: config + - mountPath: /var/run/secrets/tokens + name: join-sa-token + - name: tbot-sockets + mountPath: /run/tbot/sockets + readOnly: false + env: + - name: TELEPORT_NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: KUBERNETES_TOKEN_PATH + value: /var/run/secrets/tokens/join-sa-token + serviceAccountName: tbot + volumes: + - name: tbot-sockets + hostPath: + path: /run/tbot/sockets + type: DirectoryOrCreate + - name: config + configMap: + name: tbot-config + - name: join-sa-token + projected: + sources: + - serviceAccountToken: + path: join-sa-token + # 600 seconds is the minimum that Kubernetes supports. We + # recommend this value is used. + expirationSeconds: 600 + # `example.teleport.sh` must be replaced with the name of + # your Teleport cluster. + audience: example.teleport.sh +``` + +## Next steps + +- [Workload Identity Overview](../workload-identity.mdx): Overview of Teleport +Workload Identity. +- [Best Practices](./best-practices.mdx): Best practices for using Workload +Identity in Production. +- Read the [configuration reference](../../../reference/machine-id/configuration.mdx) to explore +all the available configuration options. diff --git a/docs/pages/reference/machine-id/configuration.mdx b/docs/pages/reference/machine-id/configuration.mdx index 3ba87bf005797..f17189443858a 100644 --- a/docs/pages/reference/machine-id/configuration.mdx +++ b/docs/pages/reference/machine-id/configuration.mdx @@ -21,8 +21,6 @@ artifacts include configuration files, certificates, and cryptographic key material. Usually, artifacts are files, but this term is explicitly avoided because a destination isn't required to be a filesystem. -## Configuration v2 - From Teleport 14, `tbot` supports the v2 configuration version. ```yaml @@ -174,7 +172,7 @@ outputs: path: ./tbot-user ``` -### Outputs +## Outputs Outputs define what actions `tbot` should take when it runs. They describe the format of the certificates to be generated, the roles used to generate the certificates, and the @@ -183,7 +181,7 @@ destination where they should be written. There are multiple types of output. Select the one that is most appropriate for your intended use-case. -#### `identity` +### `identity` The `identity` output can be used to authenticate: @@ -224,7 +222,7 @@ app_name: grafana (!docs/pages/includes/machine-id/common-output-config.yaml!) ``` -#### `database` +### `database` The `database` output is used to generate credentials that can be used to access databases that have been configured with Teleport. @@ -255,7 +253,7 @@ format: tls (!docs/pages/includes/machine-id/common-output-config.yaml!) ``` -##### Supported formats +#### Supported formats You can provide the following values to the `format` configuration field in the `database` output type: @@ -267,7 +265,7 @@ the `database` output type: | `cockroach` | Provides `cockroach/node.key`, `cockroach/node.crt`, and `cockroach/ca.crt`. This is designed to be used with CockroachDB clients. | | `tls` | Provides `tls.key`, `tls.crt`, and `tls.cas`. This is for generic clients that require the specific file extensions. | -#### `kubernetes` +### `kubernetes` The `kubernetes` output is used to generate credentials that can be used to access Kubernetes clusters that have been configured with Teleport. @@ -290,7 +288,7 @@ kubernetes_cluster: my-cluster (!docs/pages/includes/machine-id/common-output-config.yaml!) ``` -#### `ssh_host` +### `ssh_host` The `ssh_host` output is used to generate the artifacts required to configure an OpenSSH server with Teleport in order to allow Teleport users to connect to @@ -313,7 +311,7 @@ principals: (!docs/pages/includes/machine-id/common-output-config.yaml!) ``` -#### `spiffe-svid` +### `spiffe-svid` The `spiffe-svid` output is used to generate a SPIFFE X509 SVID and write this to a configured destination. @@ -346,14 +344,14 @@ svid: (!docs/pages/includes/machine-id/common-output-config.yaml!) ``` -### Services +## Services Services are configurable long-lived components that run within `tbot`. Unlike Outputs, they may not necessarily generate artifacts. Typically, services provide supporting functionality for machine to machine access, for example, opening tunnels or providing APIs. -#### `spiffe-workload-api` +### `spiffe-workload-api` The `spiffe-workload-api` service opens a listener for a service that implements the SPIFFE Workload API. This service is used to provide SPIFFE SVIDs to @@ -372,6 +370,47 @@ type: spiffe-workload-api # - TCP: `tcp://
:` # - Unix socket: `unix:///` listen: unix:///opt/machine-id/workload.sock +# attestors allows Workload Attestation to be configured for this Workload +# API. +attestors: + # kubernetes is configuration for the Kubernetes Workload Attestor. See + # the Kubernetes Workload Attestor section for more information. + kubernetes: + # enabled specifies whether the Kubernetes Workload Attestor should be + # enabled. If unspecified, this defaults to false. + enabled: true + # kubelet holds configuration relevant to the Kubernetes Workload Attestors + # interaction with the Kubelet API. + kubelet: + # read_only_port is the port on which the Kubelet API is exposed for + # read-only operations. Since Kubernetes 1.16, the read-only port is + # typically disabled by default and secure_port should be used instead. + read_only_port: 10255 + # secure_port is the port on which the attestor should connect to the + # Kubelet secure API. If unspecified, this defaults to `10250`. This is + # mutually exclusive with ReadOnlyPort. + secure_port: 10250 + # token_path is the path to the token file that the Kubelet API client + # should use to authenticate with the Kubelet API. If unspecified, this + # defaults to `/var/run/secrets/kubernetes.io/serviceaccount/token`. + token_path: "/var/run/secrets/kubernetes.io/serviceaccount/token" + # ca_path is the path to the CA file that the Kubelet API client should + # use to validate the Kubelet API server's certificate. If unspecified, + # this defaults to `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`. + ca_path: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" + # skip_verify is used to disable verification of the Kubelet API server's + # certificate. If unspecified, this defaults to false. + # + # If specified, the value specified in ca_path is ignored. + # + # This is useful in cases where the Kubelet API server has not been issued + # with a certificate signed by the Kubernetes cluster's CA. This is fairly + # common with a number of Kubernetes distributions. + skip_verify: true + # anonymous is used to disable authentication with the Kubelet API. If + # unspecified, this defaults to false. If set, the token_path field is + # ignored. + anonymous: false # svids specifies the SPIFFE SVIDs that the Workload API should provide. svids: # path specifies what the path element should be requested for the SPIFFE @@ -421,7 +460,7 @@ svids: gid: 50 ``` -#### `database-tunnel` +### `database-tunnel` The `database-tunnel` service opens a listener for a service that tunnels connections to a database server. @@ -453,7 +492,7 @@ database: postgres username: postgres ``` -#### `application-tunnel` +### `application-tunnel` The `application-tunnel` service opens a listener that tunnels connections to an application in Teleport. It supports both HTTP and TCP applications. This is @@ -481,7 +520,7 @@ listen: tcp://127.0.0.1:8084 app_name: my-application ``` -#### `ssh-multiplexer` +### `ssh-multiplexer` The `ssh-multiplexer` service opens a listener for a high-performance local SSH multiplexer. This is designed for use-cases which create a large number @@ -546,7 +585,7 @@ destination: - `v1.sock`: the Unix socket that the multiplexer listens on. - `agent.sock`: the Unix socket that the SSH agent listens on. -##### Using the SSH multiplexer programmatically +#### Using the SSH multiplexer programmatically To use the SSH multiplexer programmatically, your SSH client library will need to support one of two things: @@ -690,7 +729,7 @@ func main() { ``` -### Destinations +## Destinations A destination is somewhere that `tbot` can read and write artifacts. @@ -702,7 +741,7 @@ Destinations are used in two places in the `tbot` configuration: Destinations come in multiple types. Usually, the `directory` type is the most appropriate. -#### `directory` +### `directory` The `directory` destination type stores artifacts as files in a specified directory. @@ -739,7 +778,7 @@ symlinks: try-secure acls: try ``` -#### `memory` +### `memory` The `memory` destination type stores artifacts in the process memory. When the process exits, nothing is persisted. This destination type @@ -753,7 +792,7 @@ Configuration: type: memory ``` -#### `kubernetes_secret` +### `kubernetes_secret` The `kubernetes_secret` destination type stores artifacts in a Kubernetes secret. This allows them to be mounted into other containers deployed in