docs: document EKS Auto discovery self-bootstrap (#42107)

* docs: document EKS Auto discovery self-bootstrap This PR documents EKS auto-discovery self bootstrap using AccessEntries to gain access to the cluster. Signed-off-by: Tiago Silva <[email protected]> * Update kubernetes.mdx Co-authored-by: Steven Martin <[email protected]> * Update kubernetes.mdx Co-authored-by: Steven Martin <[email protected]> * fix lint * add how it works section * handle code review * Update kubernetes.mdx Co-authored-by: Paul Gottschling <[email protected]> --------- Signed-off-by: Tiago Silva <[email protected]> Co-authored-by: Steven Martin <[email protected]> Co-authored-by: Paul Gottschling <[email protected]>
gravitational · May 30, 2024 · d86c25e · d86c25e
1 parent bfe1e5f
commit d86c25e
Show file tree

Hide file tree

Showing 2 changed files with 136 additions and 26 deletions.
diff --git a/docs/pages/auto-discovery/kubernetes.mdx b/docs/pages/auto-discovery/kubernetes.mdx
@@ -1,7 +1,6 @@
 ---
 title: Kubernetes Clusters Discovery
 description: Detailed guides for configuring Kubernetes Clusters Discovery.
-layout: tocless-doc
 ---
 
 Kubernetes Clusters Discovery allows Kubernetes clusters
@@ -80,6 +79,17 @@ discovery_service:
        # Optional section: Defaults to "*":"*"
        tags:
          "env": "prod"
+       # AWS role to assume when discovering resources in the AWS Account.
+       # This value is an optional AWS role ARN to assume when polling EKS clusters
+       assume_role_arn: arn:aws:iam::123456789012:role/iam-discovery-role
+       # External ID is an optional value that should be set when accessing
+	     # your AWS account from a third-party service (delegated access).
+       external_id: "example-external-id"
+       # Specifies the role for which the Discovery Service should create the EKS access entry.
+       # This is an optional parameter. If not set, the Discovery Service will attempt to create
+       # the access entry using its own identity.
+       # If used, the role must match the role configured for the Kubernetes Service.
+       setup_access_for_arn: arn:aws:iam::123456789012:role/kube-service-role
     # Matchers for discovering Azure-hosted resources.
     azure:
       # Azure resource types. Valid options are:
@@ -136,7 +146,38 @@ kubernetes_service:
     resources:
     - labels:
         "*": "*" # can be configured to limit the clusters to watched by this service.
+      aws:
+       # AWS role to assume when accessing EKS clusters in the AWS Account.
+       # This value is an optional AWS role ARN to assume when forwarding requests
+       # to EKS clusters.
+       assume_role_arn: arn:aws:iam::123456789012:role/iam-discovery-role
+       # External ID is an optional value that should be set when accessing
+	     # your AWS account from a third-party service (delegated access).
+       external_id: "example-external-id"
 ```
 
+
+When configuring the `kubernetes_service.resources` parameter, the Teleport
+Kubernetes Service is set to monitor EKS clusters that are discovered by
+the Teleport Discovery Service. The monitoring process involves a label
+matching mechanism to identify and manage the EKS clusters.
+
+1. **Discovery and Label Matching:** The Discovery Service identifies available
+   EKS clusters within the AWS environment. The Teleport Kubernetes Service
+   checks the labels of these clusters against the labels specified in the
+   `kubernetes_service.resources[].labels` configuration, which is the
+   **selector array**.
+1. **Role Selection:** The first selector in the array that matches the labels
+   of an EKS cluster determines the role that the Kubernetes Service will
+   assume. This role is essential for the Teleport Kubernetes Service to
+   retrieve necessary cluster details from the AWS API. If no match is found,
+   the Kubernetes Service defaults to its own identity.
+1. **Interaction with AWS and Kubernetes APIs:** Once a match is found and a
+   role is assumed, the Teleport Kubernetes Service uses this role to access the
+   AWS API. It retrieves information about the EKS cluster, such as
+   configuration and status. Subsequently, the Teleport Kubernetes Service
+   forwards requests to the Kubernetes API, enabling interaction with the
+   cluster.
+
 Both services — Discovery and Kubernetes — can be configured in the same
-Teleport process or separate processes.
+Teleport process or separate processes.
diff --git a/docs/pages/auto-discovery/kubernetes/aws.mdx b/docs/pages/auto-discovery/kubernetes/aws.mdx
@@ -9,37 +9,33 @@ configured labels.
 
 (!docs/pages/includes/discovery/step-description.mdx!)
 
+## How it works
+
+The Teleport Discovery Service scans configured cloud providers, including AWS,
+for Kubernetes clusters that match specified filtering labels, creating dynamic
+resources within Teleport for any new clusters identified. The Teleport Kubernetes
+Service monitors these dynamic resources, forwarding requests to the corresponding
+Kubernetes clusters. Both services require access to the AWS API to perform their
+functions.
+
+Additionally, the Kubernetes Service needs direct access to the target clusters
+and the necessary permissions to forward requests.
+
 ## Prerequisites
 
 (!docs/pages/includes/edition-prereqs-tabs.mdx!)
 
 - An AWS account with permissions to create and attach IAM policies.
-- An AWS account with `system:masters` RBAC access to EKS clusters.
 - A host to run the Teleport Discovery and Kubernetes services.
 - One or more EKS clusters running.
 
-## Known limitations
-
-Due to the authorization method that AWS has chosen for EKS clusters, it is not
-possible for the Teleport process to configure the necessary permissions to forward
-requests to EKS clusters.
-This limitation exists because the authorization method on EKS clusters requires that the IAM role -
-used for authentication - exists in the
-[`aws-auth`](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) `ConfigMap`.
-
-In this `ConfigMap`, IAM roles are mapped to Kubernetes RBAC users or groups that are then
-used by Kubernetes RBAC to grant access permissions. If the mapping does not exist, it results in
-unauthorized requests. Therefore, Teleport cannot edit the `ConfigMap`
-without first having access to the cluster.
-
-If Teleport is not running with the same IAM identity that creates all clusters,
-please ensure you configure the required permissions as described in Step 2.
-
 <Admonition type="note">
 
-The AWS EKS team is working on a feature request to introduce an external API to manage access to the cluster without manually editing the Configmap ([aws/containers-roadmap#185](https://github.com/aws/containers-roadmap/issues/185)).
-
-Hopefully, once the feature is available, Teleport can leverage it to configure its access to the cluster.
+Starting with Teleport v15.3.8, the Discovery Service can self-bootstrap
+access to EKS clusters by automatically creating and managing Access Entries
+for each discovered cluster. This contrasts with earlier versions of EKS
+Auto-Discovery, where agents could not access a cluster without having
+pre-existing access configured.
 
 </Admonition>
 
@@ -56,8 +52,12 @@ describe EKS clusters:
         {
             "Effect": "Allow",
             "Action": [
-                "eks:DescribeCluster",
-                "eks:ListClusters"
+              "eks:DescribeCluster",
+              "eks:ListClusters",
+              "eks:DescribeAccessEntry",
+              "eks:CreateAccessEntry",
+              "eks:DeleteAccessEntry",
+              "eks:AssociateAccessPolicy"
             ],
             "Resource": "<Var name="arn:aws:eks:*:123456789012:cluster/*" />"
         }
@@ -69,6 +69,13 @@ Update `Resource` entries to match your AWS account's EKS resources, which have
 
 ## Step 2/3. Configure EKS cluster authorization
 
+<Notice type="warning">
+If you are running Teleport Discovery v15.3.8 or later and the IAM role
+used by the Discovery Service has the necessary permissions to create and
+update Access Entries, you can skip this section. The service can self-bootstrap
+the required permissions automatically.
+</Notice>
+
 When the Kubernetes Service uses an IAM role that is different from the one that
 creates the clusters, you need to configure the mapping between the Teleport IAM
 Role and the Kubernetes RBAC permissions by editing the `aws-auth` `Configmap` on
@@ -149,6 +156,28 @@ subjects:
 
 ### IAM mapping
 
+<Tabs>
+  <TabItem label="EKS access entry" >
+
+Create an EKS access entry to link the <Var name="arn:aws:iam::222222222222:role/teleport-role" />
+to the Kubernetes Group `teleport` we created in the previous step.
+
+```shell
+$ aws eks create-access-entry \
+  --cluster-name <Var name="eks-cluster" /> \
+  --region <Var name="eu-west-1" /> \
+  --principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" /> \
+  --kubernetes-groups teleport
+
+{
+  ...
+}
+```
+
+  </TabItem>
+
+  <TabItem label="aws-auth Configmap" >
+
 Finally, edit the `configmap/aws-auth` in the `kube-system` namespace and append
 the following to `mapRoles`. Replace `{teleport_aws_iam_role}` with the
 appropriate IAM role that Teleport Kubernetes Service will use.
@@ -164,6 +193,8 @@ data:
       rolearn: {teleport_aws_iam_role} # e.g. arn:aws:iam::222222222222:role/teleport-role
       username: teleport
 ```
+  </TabItem>
+</Tabs>
 
 At this point, the Teleport IAM role already has the minimum permissions
 to forward requests to the cluster.
@@ -203,6 +234,41 @@ service means granting administrator-level permissions on the Kubernetes cluster
 To follow least privilege principle we do not recommend using this method in production.
   </Notice>
 
+
+
+<Tabs>
+  <TabItem label="EKS access entry" >
+
+Create an EKS access entry and Access Policy to link the <Var name="arn:aws:iam::222222222222:role/teleport-role" />
+to the cluster wide policy `arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy` (equivalent of `cluster-admin` `ClusterRole`).
+
+```shell
+$ aws eks create-access-entry \
+  --cluster-name <Var name="eks-cluster" /> \
+  --region <Var name="eu-west-1" /> \
+  --principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" />
+
+{
+  ...
+}
+
+$ aws eks associate-access-policy \
+  --cluster-name <Var name="eks-cluster" /> \
+  --region <Var name="eu-west-1" /> \
+  --principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" />
+  --policy-arn "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy" \
+  --access-scope type=cluster
+
+{
+  ...
+}
+
+```
+
+  </TabItem>
+
+  <TabItem label="aws-auth Configmap" >
+
 To associate the Teleport IAM role with the `system:masters` RBAC group,
 edit the `configmap/aws-auth` in the `kube-system` namespace and append
 the following to `mapRoles`.
@@ -220,6 +286,9 @@ data:
 
 Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
 Teleport Kubernetes Service is using.
+  </TabItem>
+</Tabs>
+
 
 At this point, the Teleport IAM role already has the minimum permissions
 to forward requests to the cluster.
@@ -229,7 +298,7 @@ to forward requests to the cluster.
 
 <Notice type="tip">
 If you provision your EKS clusters using tools such as `terraform`, `eksctl` or
-`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap`
+`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap` or access entry
 and create the `ClusterRole` and `ClusterRoleBinding` resources during cluster provisioning.
 </Notice>