Skip to content

Commit

Permalink
feat: Scaffolding for using OpenTofu
Browse files Browse the repository at this point in the history
Deploys a small DigitalOcean project using a thin abstraction layer
  • Loading branch information
adityahase committed Aug 26, 2024
1 parent 6bdb9f2 commit 87389ef
Show file tree
Hide file tree
Showing 6 changed files with 438 additions and 5 deletions.
53 changes: 53 additions & 0 deletions pilot/pilot/design-wip/provision.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Provision
There's no way to do something like tofu.deploy() right now.

This goes through two levels of abstractions?. JSII

### CDKTF
We write TerraformStack subclass and define our configration in init. It should something like this
```python
from cdktf import App, Fn, TerraformStack, TerraformVariable
from cdktf_cdktf_provider_digitalocean.provider import DigitaloceanProvider
from cdktf_cdktf_provider_digitalocean.vpc import Vpc
from constructs import Construct


class MyStack(TerraformStack):
def __init__(self, scope: Construct, id: str):
super().__init__(scope, id)
do_token_variable = TerraformVariable(self,"do_token", type="string")
DigitaloceanProvider(self, "digitalocean", token=do_token_variable.string_value)
vpc = Vpc(self, "example_vpc", name="vpc-1", region="blr-1", ip_range="ip_range")

```

Unfortunately, Pilot config isn't static. But good news is, this is actually implemented as following

1. Define TerraformStack subclass, like we did before
This is equivalent of writing a HCL file

2. Define an app and call `synth` on it

```python
from cdktf import App, Fn, TerraformStack, TerraformVariable


app = App()
MyStack(app, "cdktf-demo")
app.synth()
```


3. Apply generated plan
`cdktf deploy`
We can open up the implemntation and see what happens underneath.

1. If the implementation is complicated then we can run `cdktf deploy` ourselves


---

We need to put some dynamic logic on Step 1.
We can't write a class everytime.
Generating Python code is basically the same as writing HCL
What we can do is build a class implementation and app implementation on the fly
40 changes: 40 additions & 0 deletions pilot/provision/README.wtf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
We want to do as much work in Python / JSON as possible

Luckily Tofu helps out.

Every plan / state can be stored as JSON or converted to JSON.

Here's the rough idea

```mermaid
stateDiagram
code: Python Declaration
plan: Plan
json: Synthesized JSON
infra: Infrastructure
code --> json: app.synth()
json --> plan: tofu plan
plan --> infra: tofu deploy
```


Same thing in words

1. Write Python code in `TerraformStack.__init__()` that describes the infra we need
2. "Synthesize" this TerraformStack object `app.synth()`

The synth actually executes the `__init__` so we can do whatever we want in Python (loops, conditionals etc)

`synth` generates a `cdktf.out/stacks/<stack-name>/cdktf.json` file in the working directory (this is `frappe-bench/sites`)

We will use `sites/<site>/stacks` for now. So the state moves with the site without any special handling.

TODO: Include this directory in the file backups.

3. Store this synthesized JSON in some DocType Provision


Note: Our Stack can have bugs or the code that defines what we need can have bugs. Have a way to prevent catastrophies at this stage. We need sanity checks in Production to guard against
- Don't trigger anything that can cause data loss
- Don't trigger massive changes ( More than n resources at a time)
- Cross stack changes ?! (Don't delete someone other region?!)
87 changes: 84 additions & 3 deletions pilot/provision/doctype/region/region.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,17 @@
"engine": "InnoDB",
"field_order": [
"title",
"cloud_provider"
"cloud_provider",
"status",
"column_break_quxt",
"region_slug",
"provision",
"destroy",
"credentials_section",
"access_token",
"networking_section",
"vpc_cidr_block",
"vpc_id"
],
"fields": [
{
Expand All @@ -30,11 +40,82 @@
"options": "Cloud Provider",
"reqd": 1,
"set_only_once": 1
},
{
"fieldname": "status",
"fieldtype": "Select",
"label": "Status",
"options": "Draft\nPending\nActive\nArchived"
},
{
"fieldname": "networking_section",
"fieldtype": "Section Break",
"label": "Networking"
},
{
"fieldname": "vpc_cidr_block",
"fieldtype": "Data",
"label": "VPC CIDR Block"
},
{
"fieldname": "vpc_id",
"fieldtype": "Data",
"label": "VPC ID"
},
{
"fieldname": "credentials_section",
"fieldtype": "Section Break",
"label": "Credentials"
},
{
"fieldname": "access_token",
"fieldtype": "Password",
"label": "Access Token"
},
{
"fieldname": "column_break_quxt",
"fieldtype": "Column Break"
},
{
"fieldname": "provision",
"fieldtype": "Button",
"label": "Provision",
"options": "provision"
},
{
"fieldname": "region_slug",
"fieldtype": "Data",
"label": "Region Slug",
"reqd": 1,
"set_only_once": 1
},
{
"fieldname": "destroy",
"fieldtype": "Button",
"label": "Destroy",
"options": "destroy"
}
],
"index_web_pages_for_search": 1,
"links": [],
"modified": "2024-07-18 15:42:03.445128",
"links": [
{
"link_doctype": "Provision Action",
"link_fieldname": "region"
},
{
"link_doctype": "Provision Declaration",
"link_fieldname": "region"
},
{
"link_doctype": "Provision Plan",
"link_fieldname": "region"
},
{
"link_doctype": "Provision State",
"link_fieldname": "region"
}
],
"modified": "2024-08-09 11:54:46.764057",
"modified_by": "Administrator",
"module": "Provision",
"name": "Region",
Expand Down
18 changes: 16 additions & 2 deletions pilot/provision/doctype/region/region.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# Copyright (c) 2024, Frappe and contributors
# For license information, please see license.txt

# import frappe
import frappe
from frappe.model.document import Document

from pilot.provision.opentofu import OpenTofu


class Region(Document):
# begin: auto-generated types
Expand All @@ -16,10 +18,22 @@ class Region(Document):

access_token: DF.Password | None
cloud_provider: DF.Link
region_slug: DF.Data
status: DF.Literal["Draft", "Pending", "Active", "Archived"]
title: DF.Data
vpc_cidr_block: DF.Data | None
vpc_id: DF.Data | None
# end: auto-generated types

pass
@frappe.whitelist()
def provision(self) -> None:
OpenTofu(self).provision()

@frappe.whitelist()
def destroy(self) -> None:
OpenTofu(self).destroy()

def on_trash(self) -> None:
zones = frappe.get_all("Zone", filters={"region": self.name})
for zone in zones:
frappe.delete_doc("Zone", zone.name)
Loading

0 comments on commit 87389ef

Please sign in to comment.