Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ Feature ] Arquitetura de containers do servidor "legacy" #4

Merged
merged 70 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
6588486
feat(instances): add start terraform plan to deploy legacy-server
igr-santos Nov 6, 2024
8f99522
chore(instances): add infra to create legacy and sites server in dev
igr-santos Nov 6, 2024
110e510
fix(instances): remove duplicate instances
igr-santos Nov 6, 2024
aaf989d
feat(instances): add bootstrap and persist terraform-state on S3 e Dy…
igr-santos Nov 6, 2024
5b13899
chore(instances): add workflow to deploy aws
igr-santos Nov 6, 2024
935d9ed
chore(instances): add workflow to deploy aws
igr-santos Nov 6, 2024
4267896
Merge remote-tracking branch 'refs/remotes/origin/feature/terraform' …
sergiomario Nov 7, 2024
61d592f
feat(instances): use terraform workspaces
sergiomario Nov 7, 2024
5c02b06
chore(docs): update readme with todo
igr-santos Nov 7, 2024
5d3ca57
feat(ci/cd): Add terraform infra deploy to stage environment
igr-santos Nov 7, 2024
44b4811
fix(ci/cd): working directory to run command steps
igr-santos Nov 7, 2024
8f4aa16
fix(ci/cd): add secret to setup SSH PRIVATE KEY
igr-santos Nov 7, 2024
132e8fe
fix(ci/cd): Add submodules download to use monitoring files in setup …
igr-santos Nov 7, 2024
a00df05
Update README.md
sergiomario Nov 7, 2024
8f18146
chore(ci/cd): become monitoring scripts public
igr-santos Nov 7, 2024
78814ba
fix(ci/cd): add variable to setup influxdb token in servers
igr-santos Nov 7, 2024
6ad6cc6
fix(monitoring): add braces to var used in telegraf.conf
igr-santos Nov 7, 2024
074e55a
fix(instances): set influxdb token by variable to telegraf
igr-santos Nov 7, 2024
bccfc1d
feat(instances): allocate elastic IP and settings production instances
igr-santos Nov 8, 2024
bc30967
feat(legacy): add traefik router connections
igr-santos Nov 8, 2024
a6149ea
fix(legacy): rename traefik service to use same root domain others se…
igr-santos Nov 8, 2024
e13c6c1
fix(legacy): remove middleware to etcd
igr-santos Nov 8, 2024
755082b
feat(legacy): add remote-schemas stack with api-accounts
igr-santos Nov 8, 2024
ba381bb
fix(legacy): use same network between files
igr-santos Nov 8, 2024
27fb5b5
fix(legacy): create manually network web in docker
igr-santos Nov 8, 2024
4334664
feat(instances): add command to attach server in portainer
igr-santos Nov 12, 2024
b22f22a
feat(remote-schemas): add rule to default domain in api-accounts
igr-santos Nov 12, 2024
5368ac3
feat(remote-schemas): add in stack activists, data, domains, payments…
igr-santos Nov 12, 2024
dda3d53
feat(clients): add stack with accounts, app, admin canary and redes c…
igr-santos Nov 12, 2024
4cbdaed
fix(instances): add volume size to prod legacy server
igr-santos Nov 12, 2024
9cc8c23
feat(common): add hasura api-graphql in stack
igr-santos Nov 13, 2024
905e8f2
fix(common): add default domain rule to common services
igr-santos Nov 13, 2024
92be75c
feat(common): add n8n updated version in stack
igr-santos Nov 21, 2024
855452a
feat(common): setup rule in n8n with env
igr-santos Nov 21, 2024
711dd85
feat(data): add new stack with metabase service
igr-santos Nov 22, 2024
f78b530
feat(data): update metabase to version 0.51
igr-santos Nov 22, 2024
1c6853b
fix(data): change rule to domain metabase service
igr-santos Nov 25, 2024
c867f9d
fix(common): add generic envs to database urls
igr-santos Nov 26, 2024
dfdbb54
feat(common): add api-rest service in stack
igr-santos Nov 26, 2024
472f497
feat(common): add default domain to api-rest service
igr-santos Nov 26, 2024
99e7b8c
feat(legacy): add services to mapa do acolhimento stack
igr-santos Nov 28, 2024
f71ad18
feat(legacy): update rules to router mapa acolhimento stack
igr-santos Dec 2, 2024
52a772e
fix(legacy): fixed version to moodle in mapa acolhimento stack
igr-santos Dec 2, 2024
7a99f37
Update meta base docker-compose.yml
sarahsoaresc Dec 10, 2024
e51f6ee
Merge pull request #3 from sarahsoaresc/patch-1
sergiomario Dec 10, 2024
e17b4c7
feat(terraform): add sites server to instances
igr-santos Dec 18, 2024
2cd7e37
Merge pull request #2 from nossas/feature/terraform
igr-santos Dec 18, 2024
b45d5ef
feat(legacy): add traefik router connections
igr-santos Nov 8, 2024
1c2070b
fix(legacy): rename traefik service to use same root domain others se…
igr-santos Nov 8, 2024
82d99d5
fix(legacy): remove middleware to etcd
igr-santos Nov 8, 2024
13eb7bd
feat(legacy): add remote-schemas stack with api-accounts
igr-santos Nov 8, 2024
94938af
fix(legacy): use same network between files
igr-santos Nov 8, 2024
d3d904d
fix(legacy): create manually network web in docker
igr-santos Nov 8, 2024
c724ebd
feat(remote-schemas): add rule to default domain in api-accounts
igr-santos Nov 12, 2024
ce9911e
feat(remote-schemas): add in stack activists, data, domains, payments…
igr-santos Nov 12, 2024
825f4c1
feat(clients): add stack with accounts, app, admin canary and redes c…
igr-santos Nov 12, 2024
8a78f73
feat(common): add hasura api-graphql in stack
igr-santos Nov 13, 2024
79cd96e
fix(common): add default domain rule to common services
igr-santos Nov 13, 2024
2442334
feat(common): add n8n updated version in stack
igr-santos Nov 21, 2024
b9193c2
feat(common): setup rule in n8n with env
igr-santos Nov 21, 2024
23c24bc
feat(data): add new stack with metabase service
igr-santos Nov 22, 2024
62e4317
feat(data): update metabase to version 0.51
igr-santos Nov 22, 2024
2dd45d6
fix(data): change rule to domain metabase service
igr-santos Nov 25, 2024
d384d89
fix(common): add generic envs to database urls
igr-santos Nov 26, 2024
7ef8559
feat(common): add api-rest service in stack
igr-santos Nov 26, 2024
b870853
feat(common): add default domain to api-rest service
igr-santos Nov 26, 2024
5c091c6
feat(legacy): add services to mapa do acolhimento stack
igr-santos Nov 28, 2024
f2876b4
feat(legacy): update rules to router mapa acolhimento stack
igr-santos Dec 2, 2024
b993244
fix(legacy): fixed version to moodle in mapa acolhimento stack
igr-santos Dec 2, 2024
8b47df9
Merge branch 'feature/legacy-architecture' of github.com:nossas/devop…
igr-santos Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .github/workflows/terraform-stage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: 'Stage deploy using Terraform'

# Garante que esse workflow irá executar apenas quando alterados arquivos
# da pasta instances, que é o local onde está concentrado os arquivos de infraestrura
# declarados com Terraform.
on:
push:
branches:
- "feature/*"
- "hotfix/*"
paths:
- "instances/**"
- ".github/workflows/terraform-stage.yml"

jobs:
terraform:
name: 'Terraform'
runs-on: ubuntu-latest

steps:
# Confira o repositório para o GitHub Actions runner
- name: Checkout
uses: actions/checkout@v3
# with:
# submodules: true # Clona os submódulos junto com o repositório principal
# fetch-depth: 0 # Garante que o histórico completo seja baixado, necessário para alguns submódulos
# token: ${{ secrets.GITHUB_TOKEN }} # Usa o Github token para acessar submódulos privados

- name: Setup SSH key
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
run: |
mkdir -p ~/.ssh
echo "$SSH_PRIVATE_KEY" > ~/.ssh/ailton-krenak.pem

# Instale a versão mais recente do Terraform CLI e defina a configuração do Terraform CLI.
- name: Install Terraform
run: |
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

# Inicia uma serie de comandos para publicar nossa infraestrura.
# Como estamos publicando uma infraestrutura na AWS é necessário garantir
# a configuração dos secrets AWS_ACCESS_KEY_ID e AWS_SECRET_ACCESS_KEY no repositório do Github.
- name: Terraform Initialize
working-directory: ./instances
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_VAR_influxdb_token: ${{ secrets.INFLUXDB_TOKEN }}
run: terraform init

- name: Terraform Validate
working-directory: ./instances
run: terraform validate

# Define que iremos trabalhar no ambiente de stage
- name: Terraform Select Workspace
working-directory: ./instances
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_VAR_influxdb_token: ${{ secrets.INFLUXDB_TOKEN }}
run: terraform workspace select stage

- name: Terraform Plan
working-directory: ./instances
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_VAR_influxdb_token: ${{ secrets.INFLUXDB_TOKEN }}
run: terraform plan"

- name: Terraform Apply
working-directory: ./instances
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_VAR_influxdb_token: ${{ secrets.INFLUXDB_TOKEN }}
run: terraform apply -auto-approve"
145 changes: 145 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# Devops da infraestrutura Nosssas

Este projeto utiliza `awscli` e `terraform` para gerenciar infraestrutura como código. A configuração do estado da infraestrutura é persistida em um bucket S3 e o controle de lock é gerenciado por uma tabela no DynamoDB.

## Dependências

Para executar este projeto, você precisará instalar as seguintes ferramentas:

- `awscli`: Ferramenta de linha de comando da AWS para gerenciar e interagir com serviços da AWS.
- `terraform`: Ferramenta de infraestrutura como código para gerenciar e provisionar recursos em várias plataformas.

### Instalação no MacOS
aws-cli:
```bash
brew install awscli
```

terraform:
```bash
brew tap hashicorp/tap
brew install hashicorp/tap/terraform
brew update
brew upgrade hashicorp/tap/terraform
```

### Instalação no Debian/Ubuntu
aws-cli:
```bash
sudo apt update
sudo apt install awscli -y
```

terraform:
```bash
sudo apt update
sudo apt install -y software-properties-common
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform
```

### Variáveis de ambiente
É necessário inserir duas chaves no seu env para rodar os comandos terraform.
```bash
export AWS_ACCESS_KEY_ID=<inserir chave aqui>
export AWS_SECRET_ACCESS_KEY=<inserir chave aqui>
```

### Chave ssh da AWS
Para fins de criação e manutenção de instâncias na AWS é necessário incluir a chave pública `ailton-krenak.pem` no seguinte path do seu ambiente de desenvolvimento `˜/.ssh/ailton-krenak.pem`.

## Como executar

### Configurando a infraestrutura com Terraform

O estado do terraform está sendo compartilhado no S3 com gerenciamento de lock em uma tabela no DynamoDB.

Se o bucket configurado `bonde-terraform-up-and-running-state` não existir na lista de buckets da sua conta Amazon (região: us-east-1), você deve executar os seguintes comandos na pasta `instances/boostrap`:

```bash
terraform init
terraform plan
terraform init -auto-approve
```

Essa sequência de comandos acima irá criar a infraestrutura não persistente responsável por cuidar do estado da nossa infraestrutura persistente.

### Executando a infraestrutura com esquema de workspaces do terraform

Este projeto utiliza *workspaces* do Terraform para gerenciar múltiplos ambientes (como `dev`, `stage` e `prod`) dentro de uma única configuração de infraestrutura. Cada *workspace* permite isolar o estado e os recursos entre diferentes ambientes.

Dentro da estrutura do nosso código, temos o arquivo `main.tf` onde nós definimos todas as variáveis *default*, como a seguir:

```
locals {
# Tipo de imagem para o servidor legado (APIS e Clientes Bonde)
ami = "ami-0866a3c8686eaeeba"

# Nome da chave SSH
key_name = "custom-host"

# Caminho para a chave privada SSH
private_key_path = "~/.ssh/ailton-krenak.pem"

# Ambiente (dev, staging, production)
env = terraform.workspace

# Tipo de instância para o servidor legado (APIS e Clientes Bonde)
legacy_server_instance_type = terraform.workspace == "stage" ? "t3.small" : "t3.micro"

# Tipo de instância para o servidor de sites (Bonde Público e CMS)
sites_server_instance_type = terraform.workspace == "stage" ? "t3.micro" : "t2.micro"
}
```

1. **Criando ou Selecionando um Workspace**:
- Para criar um novo *workspace*, execute:
```bash
terraform workspace new nome-do-workspace
```
- Para alternar para um *workspace* existente, execute:
```bash
terraform workspace select nome-do-workspace
```

2. **Executando a Configuração no Workspace Selecionado**:
Com o *workspace* adequado selecionado, você pode executar os comandos do Terraform normalmente. O estado será armazenado separadamente para cada *workspace*, mantendo os recursos de cada ambiente isolados.

```bash
terraform init
terraform apply -auto-approve
```

### Configuração do ambiente

Você deve configurar as seguintes váriaveis no seu ambiente de execução do terraform. Lembre-se você deve utilizar os mesmos valores de váriaveis que outros ambientes que também executam o terraform, exemplo dev local e github actions.

```
TF_VAR_influxdb_token=
TF_VAR_ami=
# legacy envs
TF_VAR_legacy_elastic_ip_allocation_id=
TF_VAR_legacy_server_instance_type=
TF_VAR_legacy_portainer_edge_id=
TF_VAR_legacy_portainer_edge_key=
# sites envs
TF_VAR_sites_elastic_ip_allocation_id=
TF_VAR_sites_server_instance_type=
TF_VAR_sites_portainer_edge_id=
TF_VAR_sites_portainer_edge_key=
```

NOTE: As váriaveis com suffix `_elastic_ip_allocation_id` não são obrigatórias, caso você não configure os scripts não vão associar um IP Elástico ao seus servidores.

## Sites


## TODO

- [X] Persistir o estado no S3 e lock com DynamoDB
- [ ] Criar fluxo de trabalho para publicação automatizada no Github
- [X] Resolver caminho da chave privada
- [X] Documentar o uso do workspace
- [X] Documentar o uso da chave privada `custom-host / ailton-krenak`
- [X] Configuração da awscli e terraform e das variaveis de ambiente que precisam ser configuradas para acessar a conta AWS
48 changes: 48 additions & 0 deletions instances/bootstrap/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
provider "aws" {
region = "us-east-1"
}

resource "aws_s3_bucket" "terraform_state" {
bucket = "bonde-terraform-up-and-running-state"

# Prevent accidental deletion of this S3 bucket
lifecycle {
prevent_destroy = true
}
}

resource "aws_s3_bucket_versioning" "enabled" {
bucket = aws_s3_bucket.terraform_state.id
versioning_configuration {
status = "Enabled"
}
}

resource "aws_s3_bucket_server_side_encryption_configuration" "default" {
bucket = aws_s3_bucket.terraform_state.id

rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}

resource "aws_s3_bucket_public_access_block" "public_access" {
bucket = aws_s3_bucket.terraform_state.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}

resource "aws_dynamodb_table" "terraform_locks" {
name = "terraform-up-and-running-locks"
billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"

attribute {
name = "LockID"
type = "S"
}
}
6 changes: 6 additions & 0 deletions instances/envs/dev.tfvars
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
legacy_server_instance_type="t2.micro"
sites_server_instance_type="t3.micro"
ami="ami-0866a3c8686eaeeba"
env="dev"
key_name="custom-host"
private_key_path="~/Repositories/nossas/keys/ailton-krenak.pem"
6 changes: 6 additions & 0 deletions instances/envs/production.tfvars
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
legacy_server_instance_type="t2.micro"
sites_server_instance_type="t3.micro"
ami="ami-0866a3c8686eaeeba"
env="production"
key_name="custom-host"
private_key_path="~/Repositories/nossas/keys/ailton-krenak.pem"
67 changes: 67 additions & 0 deletions instances/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.16"
}
}

required_version = ">= 1.2.0"

backend "s3" {
# Replace this with your bucket name!
bucket = "bonde-terraform-up-and-running-state"
key = "global/s3/terraform.tfstate"
region = "us-east-1"

# Replace this with your DynamoDB table name!
dynamodb_table = "terraform-up-and-running-locks"
encrypt = true
}
}

provider "aws" {
region = "us-east-1"
}

locals {
# Nome da chave SSH
key_name = "custom-host"

# Caminho para a chave privada SSH
private_key_path = "~/.ssh/ailton-krenak.pem"

# Ambiente (dev, staging, production)
env = terraform.workspace
}

# Módulo para o servidor web
module "legacy_server" {
source = "./modules/common"
ami = var.ami
instance_type = var.legacy_server_instance_type
instance_name = "legacy-server-${local.env}"
volume_size = terraform.workspace == "prod" ? 100 : 30
key_name = local.key_name
private_key_path = local.private_key_path
monitoring_files_path = "./monitoring"
influxdb_token = var.influxdb_token
elastic_ip_allocation_id = var.legacy_elastic_ip_allocation_id
portainer_edge_id = var.legacy_portainer_edge_id
portainer_edge_key = var.legacy_portainer_edge_key
}

module "sites_server" {
source = "./modules/common"
ami = var.ami
instance_type = var.sites_server_instance_type
instance_name = "sites-server-${local.env}"
volume_size = terraform.workspace == "prod" ? 100 : 30
key_name = local.key_name
private_key_path = local.private_key_path
monitoring_files_path = "./monitoring"
influxdb_token = var.influxdb_token
elastic_ip_allocation_id = var.sites_elastic_ip_allocation_id
portainer_edge_id = var.sites_portainer_edge_id
portainer_edge_key = var.sites_portainer_edge_key
}
Loading
Loading