Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: document EKS Auto discovery self-bootstrap #42107

Merged
merged 7 commits into from
May 30, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions docs/pages/auto-discovery/kubernetes.mdx
tigrato marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
title: Kubernetes Clusters Discovery
description: Detailed guides for configuring Kubernetes Clusters Discovery.
layout: tocless-doc
---

Kubernetes Clusters Discovery allows Kubernetes clusters
Expand Down Expand Up @@ -80,6 +79,17 @@ discovery_service:
# Optional section: Defaults to "*":"*"
tags:
"env": "prod"
# AWS role to assume when discovering resources in the AWS Account.
# This value is an optional AWS role ARN to assume when polling EKS clusters
assume_role_arn: arn:aws:iam::123456789012:role/iam-discovery-role
# External ID is an optional value that should be set when accessing
# your AWS account from a third-party service (delegated access).
external_id: "example-external-id"
# Specifies the role for which the Discovery Service should create the EKS access entry.
# This is an optional parameter. If not set, the Discovery Service will attempt to create
# the access entry using its own identity.
# If used, the role must match the role configured for the Kubernetes Service.
setup_access_for_arn: arn:aws:iam::123456789012:role/kube-service-role
# Matchers for discovering Azure-hosted resources.
azure:
# Azure resource types. Valid options are:
Expand Down Expand Up @@ -136,7 +146,38 @@ kubernetes_service:
resources:
- labels:
"*": "*" # can be configured to limit the clusters to watched by this service.
aws:
# AWS role to assume when accessing EKS clusters in the AWS Account.
# This value is an optional AWS role ARN to assume when forwarding requests
# to EKS clusters.
assume_role_arn: arn:aws:iam::123456789012:role/iam-discovery-role
# External ID is an optional value that should be set when accessing
# your AWS account from a third-party service (delegated access).
external_id: "example-external-id"
```


When configuring the `kubernetes_service.resources` parameter, the Teleport
Kubernetes Service is set to monitor EKS clusters that are discovered by
the Teleport Discovery Service. The monitoring process involves a label
matching mechanism to identify and manage the EKS clusters.

1. **Discovery and Label Matching**:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is a little difficult for me to read. Would it be possible to use subheadings in place of the numbered list items and regular body text instead of the bulleted list items? Since these are full sentences, I don't think bullets make them more readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by this? I tried to split them in operations that would make sense

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make a suggestion!

- The Discovery Service identifies available EKS clusters within the AWS environment.
- The Teleport Kubernetes Service checks the labels of these discovered EKS clusters
against the labels specified in the `kubernetes_service.resources[].labels` configuration.
- The labels in the `kubernetes_service.resources[].labels` array serve as selectors.
2. **Role Selection**:
- The first selector in the array that matches the labels of an EKS cluster determines
the role that the Kubernetes Service will assume.
- This role is essential for the Teleport Kubernetes Service to retrieve necessary cluster details from the AWS API.
- If no match is found, the Kubernetes Service defaults to its own identity.
3. **Interaction with AWS and Kubernetes API**:
- Once a match is found and a role is assumed, the Teleport Kubernetes Service uses
this role to access the AWS API.
- It retrieves detailed information about the EKS cluster, such as configuration and status.
- Subsequently, the Teleport Kubernetes Service forwards requests to the Kubernetes
API, enabling interaction with the cluster.
tigrato marked this conversation as resolved.
Show resolved Hide resolved

Both services — Discovery and Kubernetes — can be configured in the same
Teleport process or separate processes.
Teleport process or separate processes.
117 changes: 93 additions & 24 deletions docs/pages/auto-discovery/kubernetes/aws.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,37 +9,33 @@ configured labels.

(!docs/pages/includes/discovery/step-description.mdx!)

## How it works

The Teleport Discovery Service scans configured cloud providers, including AWS,
for Kubernetes clusters that match specified filtering labels, creating dynamic
resources within Teleport for any new clusters identified. The Teleport Kubernetes
Service monitors these dynamic resources, forwarding requests to the corresponding
Kubernetes clusters. Both services require access to the AWS API to perform their
functions.

Additionally, the Kubernetes Service needs direct access to the target clusters
and the necessary permissions to forward requests.

## Prerequisites

(!docs/pages/includes/edition-prereqs-tabs.mdx!)

- An AWS account with permissions to create and attach IAM policies.
- An AWS account with `system:masters` RBAC access to EKS clusters.
- A host to run the Teleport Discovery and Kubernetes services.
- One or more EKS clusters running.

## Known limitations

Due to the authorization method that AWS has chosen for EKS clusters, it is not
possible for the Teleport process to configure the necessary permissions to forward
requests to EKS clusters.
This limitation exists because the authorization method on EKS clusters requires that the IAM role -
used for authentication - exists in the
[`aws-auth`](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) `ConfigMap`.

In this `ConfigMap`, IAM roles are mapped to Kubernetes RBAC users or groups that are then
used by Kubernetes RBAC to grant access permissions. If the mapping does not exist, it results in
unauthorized requests. Therefore, Teleport cannot edit the `ConfigMap`
without first having access to the cluster.

If Teleport is not running with the same IAM identity that creates all clusters,
please ensure you configure the required permissions as described in Step 2.

<Admonition type="note">

The AWS EKS team is working on a feature request to introduce an external API to manage access to the cluster without manually editing the Configmap ([aws/containers-roadmap#185](https://github.com/aws/containers-roadmap/issues/185)).

Hopefully, once the feature is available, Teleport can leverage it to configure its access to the cluster.
Starting with Teleport v15.3.8, the Discovery Service can self-bootstrap
access to EKS clusters by automatically creating and managing Access Entries
for each discovered cluster. This contrasts with earlier versions of EKS
Auto-Discovery, where agents could not access a cluster without having
pre-existing access configured.

</Admonition>

Expand All @@ -56,8 +52,12 @@ describe EKS clusters:
{
"Effect": "Allow",
"Action": [
"eks:DescribeCluster",
"eks:ListClusters"
"eks:DescribeCluster",
"eks:ListClusters",
"eks:DescribeAccessEntry",
"eks:CreateAccessEntry",
"eks:DeleteAccessEntry",
"eks:AssociateAccessPolicy"
],
"Resource": "<Var name="arn:aws:eks:*:123456789012:cluster/*" />"
}
Expand All @@ -69,6 +69,13 @@ Update `Resource` entries to match your AWS account's EKS resources, which have

## Step 2/3. Configure EKS cluster authorization

<Notice type="warning">
If you are running Teleport Discovery v15.3.8 or later and the IAM role
used by the Discovery Service has the necessary permissions to create and
update Access Entries, you can skip this section. The service can self-bootstrap
the required permissions automatically.
</Notice>

When the Kubernetes Service uses an IAM role that is different from the one that
creates the clusters, you need to configure the mapping between the Teleport IAM
Role and the Kubernetes RBAC permissions by editing the `aws-auth` `Configmap` on
Expand Down Expand Up @@ -149,6 +156,28 @@ subjects:

### IAM mapping

<Tabs>
<TabItem label="EKS access entry" >

Create an EKS access entry to link the <Var name="arn:aws:iam::222222222222:role/teleport-role" />
to the Kubernetes Group `teleport` we created in the previous step.

```shell
$ aws eks create-access-entry \
--cluster-name <Var name="eks-cluster" /> \
--region <Var name="eu-west-1" /> \
--principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" /> \
--kubernetes-groups teleport

{
...
}
```

</TabItem>

<TabItem label="aws-auth Configmap" >

Finally, edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`. Replace `{teleport_aws_iam_role}` with the
appropriate IAM role that Teleport Kubernetes Service will use.
Expand All @@ -164,6 +193,8 @@ data:
rolearn: {teleport_aws_iam_role} # e.g. arn:aws:iam::222222222222:role/teleport-role
username: teleport
```
</TabItem>
</Tabs>

At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.
Expand Down Expand Up @@ -203,6 +234,41 @@ service means granting administrator-level permissions on the Kubernetes cluster
To follow least privilege principle we do not recommend using this method in production.
</Notice>



<Tabs>
<TabItem label="EKS access entry" >

Create an EKS access entry and Access Policy to link the <Var name="arn:aws:iam::222222222222:role/teleport-role" />
to the cluster wide policy `arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy` (equivalent of `cluster-admin` `ClusterRole`).

```shell
$ aws eks create-access-entry \
--cluster-name <Var name="eks-cluster" /> \
--region <Var name="eu-west-1" /> \
--principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" />

{
...
}

$ aws eks associate-access-policy \
--cluster-name <Var name="eks-cluster" /> \
--region <Var name="eu-west-1" /> \
--principal-arn <Var name="arn:aws:iam::222222222222:role/teleport-role" />
--policy-arn "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy" \
--access-scope type=cluster

{
...
}

```

</TabItem>

<TabItem label="aws-auth Configmap" >

To associate the Teleport IAM role with the `system:masters` RBAC group,
edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`.
Expand All @@ -220,6 +286,9 @@ data:

Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
Teleport Kubernetes Service is using.
</TabItem>
</Tabs>


At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.
Expand All @@ -229,7 +298,7 @@ to forward requests to the cluster.

<Notice type="tip">
If you provision your EKS clusters using tools such as `terraform`, `eksctl` or
`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap`
`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap` or access entry
and create the `ClusterRole` and `ClusterRoleBinding` resources during cluster provisioning.
</Notice>

Expand Down
Loading