forked from cloudposse/terraform-aws-eks-cluster
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.yaml
271 lines (244 loc) · 14.2 KB
/
README.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
name: terraform-aws-eks-cluster
license: APACHE2
github_repo: cloudposse/terraform-aws-eks-cluster
badges:
- name: Latest Release
image: https://img.shields.io/github/release/cloudposse/terraform-aws-eks-cluster.svg
url: https://github.com/cloudposse/terraform-aws-eks-cluster/releases/latest
- name: Slack Community
image: https://slack.cloudposse.com/badge.svg
url: https://slack.cloudposse.com
related:
- name: terraform-aws-eks-workers
description: Terraform module to provision an AWS AutoScaling Group, IAM Role, and
Security Group for EKS Workers
url: https://github.com/cloudposse/terraform-aws-eks-workers
- name: terraform-aws-ec2-autoscale-group
description: Terraform module to provision Auto Scaling Group and Launch Template
on AWS
url: https://github.com/cloudposse/terraform-aws-ec2-autoscale-group
- name: terraform-aws-ecs-container-definition
description: Terraform module to generate well-formed JSON documents (container
definitions) that are passed to the aws_ecs_task_definition Terraform resource
url: https://github.com/cloudposse/terraform-aws-ecs-container-definition
- name: terraform-aws-ecs-alb-service-task
description: Terraform module which implements an ECS service which exposes a web
service via ALB
url: https://github.com/cloudposse/terraform-aws-ecs-alb-service-task
- name: terraform-aws-ecs-web-app
description: Terraform module that implements a web app on ECS and supports autoscaling,
CI/CD, monitoring, ALB integration, and much more
url: https://github.com/cloudposse/terraform-aws-ecs-web-app
- name: terraform-aws-ecs-codepipeline
description: Terraform module for CI/CD with AWS Code Pipeline and Code Build for
ECS
url: https://github.com/cloudposse/terraform-aws-ecs-codepipeline
- name: terraform-aws-ecs-cloudwatch-autoscaling
description: Terraform module to autoscale ECS Service based on CloudWatch metrics
url: https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling
- name: terraform-aws-ecs-cloudwatch-sns-alarms
description: Terraform module to create CloudWatch Alarms on ECS Service level metrics
url: https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms
- name: terraform-aws-ec2-instance
description: Terraform module for providing a general purpose EC2 instance
url: https://github.com/cloudposse/terraform-aws-ec2-instance
- name: terraform-aws-ec2-instance-group
description: Terraform module for provisioning multiple general purpose EC2 hosts
for stateful applications
url: https://github.com/cloudposse/terraform-aws-ec2-instance-group
description: Terraform module to provision an [EKS](https://aws.amazon.com/eks/) cluster
on AWS.
introduction: |-
The module provisions the following resources:
- EKS cluster of master nodes that can be used together with the [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers),
[terraform-aws-eks-node-group](https://github.com/cloudposse/terraform-aws-eks-node-group) and
[terraform-aws-eks-fargate-profile](https://github.com/cloudposse/terraform-aws-eks-fargate-profile)
modules to create a full-blown cluster
- IAM Role to allow the cluster to access other AWS services
- Security Group which is used by EKS workers to connect to the cluster and kubelets and pods to receive communication from the cluster control plane
- The module creates and automatically applies an authentication ConfigMap to allow the wrokers nodes to join the cluster and to add additional users/roles/accounts
__NOTE:__ The module works with [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html).
__NOTE:__ In `auth.tf`, we added `ignore_changes = [data["mapRoles"]]` to the `kubernetes_config_map` for the following reason:
- We provision the EKS cluster and then the Kubernetes Auth ConfigMap to map additional roles/users/accounts to Kubernetes groups
- Then we wait for the cluster to become available and for the ConfigMap to get provisioned (see `data "null_data_source" "wait_for_cluster_and_kubernetes_configmap"` in `examples/complete/main.tf`)
- Then we provision a managed Node Group
- Then EKS updates the Auth ConfigMap and adds worker roles to it (for the worker nodes to join the cluster)
- Since the ConfigMap is modified outside of Terraform state, Terraform wants to update it (remove the roles that EKS added) on each `plan/apply`
If you want to modify the Node Group (e.g. add more Node Groups to the cluster) or need to map other IAM roles to Kubernetes groups,
set the variable `kubernetes_config_map_ignore_role_changes` to `false` and re-provision the module. Then set `kubernetes_config_map_ignore_role_changes` back to `true`.
usage: |2-
For a complete example, see [examples/complete](examples/complete).
For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](test).
Other examples:
- [terraform-root-modules/eks](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks) - Cloud Posse's service catalog of "root module" invocations for provisioning reference architectures
- [terraform-root-modules/eks-backing-services-peering](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks-backing-services-peering) - example of VPC peering between the EKS VPC and backing services VPC
```hcl
provider "aws" {
region = var.region
}
module "label" {
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=master"
namespace = var.namespace
name = var.name
stage = var.stage
delimiter = var.delimiter
attributes = compact(concat(var.attributes, list("cluster")))
tags = var.tags
}
locals {
# The usage of the specific kubernetes.io/cluster/* resource tags below are required
# for EKS and Kubernetes to discover and manage networking resources
# https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking
tags = merge(var.tags, map("kubernetes.io/cluster/${module.label.id}", "shared"))
# Unfortunately, most_recent (https://github.com/cloudposse/terraform-aws-eks-workers/blob/34a43c25624a6efb3ba5d2770a601d7cb3c0d391/main.tf#L141)
# variable does not work as expected, if you are not going to use custom AMI you should
# enforce usage of eks_worker_ami_name_filter variable to set the right kubernetes version for EKS workers,
# otherwise the first version of Kubernetes supported by AWS (v1.11) for EKS workers will be used, but
# EKS control plane will use the version specified by kubernetes_version variable.
eks_worker_ami_name_filter = "amazon-eks-node-${var.kubernetes_version}*"
}
module "vpc" {
source = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
cidr_block = "172.16.0.0/16"
tags = local.tags
}
module "subnets" {
source = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=master"
availability_zones = var.availability_zones
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
vpc_id = module.vpc.vpc_id
igw_id = module.vpc.igw_id
cidr_block = module.vpc.vpc_cidr_block
nat_gateway_enabled = false
nat_instance_enabled = false
tags = local.tags
}
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
instance_type = var.instance_type
eks_worker_ami_name_filter = local.eks_worker_ami_name_filter
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
oidc_provider_enabled = false
workers_security_group_ids = [module.eks_workers.security_group_id]
workers_role_arns = [module.eks_workers.workers_role_arn]
}
```
Module usage with two worker groups:
```hcl
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "small"
attributes = var.attributes
tags = var.tags
instance_type = "t3.small"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_workers_2" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "medium"
attributes = var.attributes
tags = var.tags
instance_type = "t3.medium"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
oidc_provider_enabled = false
workers_role_arns = [module.eks_workers.workers_role_arn, module.eks_workers_2.workers_role_arn]
workers_security_group_ids = [module.eks_workers.security_group_id, module.eks_workers_2.security_group_id]
}
```
include:
- docs/targets.md
- docs/terraform.md
contributors:
- name: Erik Osterman
homepage: https://github.com/osterman
avatar: http://s.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb?s=144
github: osterman
- name: Andriy Knysh
homepage: https://github.com/aknysh/
avatar: https://avatars0.githubusercontent.com/u/7356997?v=4&u=ed9ce1c9151d552d985bdf5546772e14ef7ab617&s=144
github: aknysh
- name: Igor Rodionov
homepage: https://github.com/goruha/
avatar: http://s.gravatar.com/avatar/bc70834d32ed4517568a1feb0b9be7e2?s=144
github: goruha
- name: Oscar
homepage: https://github.com/osulli/
avatar: https://avatars1.githubusercontent.com/u/46930728?v=4&s=144
github: osulli