Skip to content

Commit

Permalink
build: Terraform Modules
Browse files Browse the repository at this point in the history
  • Loading branch information
jshlbrd committed Sep 17, 2023
1 parent d154a71 commit 4ca4142
Show file tree
Hide file tree
Showing 21 changed files with 198 additions and 156 deletions.
21 changes: 3 additions & 18 deletions build/terraform/aws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,6 @@ This module is used to create Event Bridge rules that trigger a Lambda.

Read more about Event Bridge [here](https://aws.amazon.com/eventbridge/).

### IAM

This module is used to provide default Identity and Access Management (IAM) policies for the most commonly used permissions. We use this naming convention: [AWS service]\_[read|write|modify]\_policy. For example, the `kinesis_read_policy` grants all the permissions required to read from a provided Kinesis stream.

Read more about IAM policies [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html).

### IAM Attachment

This module is used to attach policies from the IAM module to resources used in a pipeline (such as Kinesis streams, KMS keys, DynamoDB tables, etc.). Separating policies and policy attachment allows for granular permission control. We recommend [granting least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege) whenever possible.

### Kinesis

This module is used to create new Kinesis Data Streams (KDS) and accompanying CloudWatch alarms. The streams created by this module are intended to be used with Substation's autoscaling application -- this feature provides stream autoscaling at a significantly reduced cost compared to Kinesis Firehose.
Expand All @@ -77,14 +67,9 @@ Read more about the Key Management Service [here](https://aws.amazon.com/kms/).

### Lambda

This module is used to create and manage Lambda, which is the recommended service for data processing. At release, the Lambda Substation app ( `cmd/aws/lambda/substation` ) supports these Lambda triggers:

* API Gateway
* Kinesis Data Streams
* SNS via S3
* S3
This module is used to create and manage Lambda, which is the recommended service for data processing.

This module is flexible enough to deploy supporting apps (such as `cmd/aws/lambda/kinesis_autoscaling` ) and custom apps (such as apps that provide data enrichment functionality). When new Lambda are created with this module, an accompanying AppConfig configuration profile is created under the `substation` application.
This module is flexible enough to deploy supporting apps (such as `cmd/aws/lambda/kinesis_autoscaling`) and custom apps (such as apps that provide data enrichment functionality). When new Lambda are created with this module, an accompanying AppConfig configuration profile is created under the `substation` application.

Read more about AWS Lambda [here](https://aws.amazon.com/lambda/).

Expand Down Expand Up @@ -114,4 +99,4 @@ Read more about SQS [here](https://aws.amazon.com/sqs/).

This module can be used to create a custom VPC with outbound connectivity via a NAT gateway to a public subnet that contains an IGW. This allows for connectivity with VPC only services.

Read more about VPCs [here](https://aws.amazon.com/vpc/).
Read more about VPCs [here](https://aws.amazon.com/vpc/).
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
variable "config" {
type = object({
name = string
stream = string
name = string
stream = string
timeout = optional(number, 1000)
})
})
description = "Configuration for the API Gateway Kinesis Data Stream integration."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}
6 changes: 4 additions & 2 deletions build/terraform/aws/api_gateway/lambda/_variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ variable "config" {
arn = string
})
})
description = "Configuration for the API Gateway Lambda integration."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}
18 changes: 11 additions & 7 deletions build/terraform/aws/dynamodb/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
variable "kms" {
type = object({
arn = string
id = string
arn = string
id = string
})
description = "KMS key used to encrypt the table."
}

variable "config" {
type = object({
name = string
name = string
hash_key = string
attributes = list(object({
name = string
Expand All @@ -30,15 +31,18 @@ variable "config" {
# https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html
stream_view_type = optional(string, "NEW_AND_OLD_IMAGES")
})

description = "Configuration for the DynamoDB table."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}

variable "access" {
type = list(string)
default = []
type = list(string)
default = []
description = "List of IAM ARNs that are granted access to the resource."
}
8 changes: 4 additions & 4 deletions build/terraform/aws/dynamodb/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,14 @@ resource "aws_dynamodb_table" "table" {

# Applies the policy to each role in the access list.
resource "aws_iam_role_policy_attachment" "access" {
for_each = toset(var.access)
role = each.value
for_each = toset(var.access)
role = each.value
policy_arn = aws_iam_policy.access.arn
}

resource "aws_iam_policy" "access" {
name = var.config.name
description = "Policy for the ${var.config.name} DynamoDB table"
name = "${var.config.name}-access"
description = "Policy for the ${var.config.name} DynamoDB table."
policy = data.aws_iam_policy_document.access.json
}

Expand Down
19 changes: 11 additions & 8 deletions build/terraform/aws/ecr/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
variable "config" {
variable "kms" {
type = object({
name = string
})
arn = string
id = string
})
description = "KMS key used to encrypt the repository."
}

variable "kms" {
variable "config" {
type = object({
arn = string
id = string
name = string
})
description = "Configuration for the ECR repository."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}
14 changes: 8 additions & 6 deletions build/terraform/aws/event_bridge/lambda/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
variable "config" {
type = object({
name = string
name = string
description = string
schedule = string
schedule = string
function = object({
arn = string
arn = string
name = string
})
})
})
description = "Configuration for the EventBridge Lambda rule."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}
27 changes: 15 additions & 12 deletions build/terraform/aws/kinesis_data_stream/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
variable "config" {
variable "kms" {
type = object({
name = string
autoscaling_topic = string
shards = optional(number, 2)
retention = optional(number, 24)
arn = string
id = string
})
description = "KMS key used to encrypt the stream."
}

variable kms {
variable "config" {
type = object({
arn = string
id = string
name = string
autoscaling_topic = string
shards = optional(number, 2)
retention = optional(number, 24)
})
description = "Configuration for the Kinesis stream."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}

variable "access" {
type = list(string)
default = []
type = list(string)
default = []
description = "List of IAM ARNs that are granted access to the resource."
}
8 changes: 4 additions & 4 deletions build/terraform/aws/kinesis_data_stream/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ resource "aws_kinesis_stream" "stream" {

# Applies the policy to each role in the access list.
resource "aws_iam_role_policy_attachment" "access" {
for_each = toset(var.access)
role = each.value
for_each = toset(var.access)
role = each.value
policy_arn = aws_iam_policy.access.arn
}

resource "aws_iam_policy" "access" {
name = var.config.name
description = "Policy for the ${var.config.name} Kinesis Data Stream"
name = "${var.config.name}-access"
description = "Policy for the ${var.config.name} Kinesis Data Stream."
policy = data.aws_iam_policy_document.access.json
}

Expand Down
12 changes: 7 additions & 5 deletions build/terraform/aws/kms/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
variable config {
variable "config" {
type = object({
name = string
policy = optional(string, "")
name = string
policy = optional(string, null)
})
description = "Configuration for the KMS key."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}
41 changes: 24 additions & 17 deletions build/terraform/aws/lambda/_variables.tf
Original file line number Diff line number Diff line change
@@ -1,46 +1,53 @@
variable "appconfig" {
type = object({
arn = string
id = string
id = string
})
description = "AppConfig application used for the Lambda configuration."
}

variable "kms" {
type = object({
arn = string
id = string
id = string
})
description = "KMS key used to encrypt the Lambda."
}

variable "config" {
type = object({
name = string
name = string
description = string
image_uri = string
architectures = optional(list(string), ["x86_64"])
timeout = optional(number, 300)
memory = optional(number, 1024)
env = optional(map(any), null)
secret = optional(bool, false)
image_uri = string
image_arm = bool
timeout = optional(number, 300)
memory = optional(number, 1024)
env = optional(map(any), null)
secret = optional(bool, false)
vpc_config = optional(object({
subnet_ids = list(string)
subnet_ids = list(string)
security_group_ids = list(string)
}), null)
}), {
subnet_ids = []
security_group_ids = []
})
iam_statements = optional(list(object({
sid = string
actions = list(string)
sid = string
actions = list(string)
resources = list(string)
})), [])
})
description = "Configuration for the Lambda function."
}

variable "tags" {
type = map(any)
default = {}
type = map(any)
default = {}
description = "Tags to apply to all resources."
}

variable "access" {
type = list(string)
default = []
type = list(string)
default = []
description = "List of IAM ARNs that are granted access to the resource."
}
Loading

0 comments on commit 4ca4142

Please sign in to comment.