Skip to content

Latest commit

 

History

History

anthos-bm-gcp-terraform

README.md

This is a terraform script to provision the GCE infrastructure in which Anthos clusters on bare metal (ABM) will be installed. The cluster installation is done using bash scripts. This does NOT use the google_gkeonprem_* terraform resources of the google provider to create the ABM cluster. If you are looking for a sample that is fully based off of the terraform provider, see the anthos-onprem-terraform directory.

Anthos Baremetal on Google Compute Engine VMs with Terraform

Read the dosclaimer on top of this README before you continue.

This repository shows you how to use Terraform to try Anthos clusters on bare metal in High Availability (HA) mode using Virtual Machines (VMs) running on Compute Engine. For information about how to use the gcloud command-line tool to try this, see Try Anthos clusters on bare metal on Compute Engine VMs.

Pre-requisites

  • A workstation with access to internet (i.e. Google Cloud APIs) with the following installed

  • A Google Cloud Project (in which the resources for the setup will be provisioned)

  • A Service Account in the project that satisfies one of the following requirements and its key file downloaded to the workstation:

    • The Service Account has Owner permissions
    • The Service Account has both Editor and Project IAM Admin permissions

Bare metal infrastructure on Google Cloud using Compute Engine VMs

The Quick starter guide sets up the following infrastructure in Google Cloud using Compute Engine VMs. The diagram assumes that the none of the default values for the variables were changed other than the ones mentioned in the quick starter.

Bare metal infrastructure on Google Cloud using Compute Engine VMs

Getting started

  • Terraform Module Information (includes variables definitions)

  • Quick start guide:

    • The terraform script sets up the GCE VM environment. The output of the script prints out the commands to follow to install Anthos on bare metal in the provisioned GCE VMs.

  • All in one install:

    • The terraform script sets up the GCE VM environment and also triggers the Anthos on bare metal installation on the provisioned GCE VMs. The output of the script prints out the commands to SSH into the admin workstation VM and monitor the Anthos on bare metal installation process.

  • Manual LB install:

    • The terraform script sets up the GCE environment and triggers the Anthos on bare metal installation similar to the all-in-one install. However, in this mode Anthos on bare metal is installed with a Manual Loadbalancer instead of the default Bundled LB. We use Google Cloud Loadbalancer as the manual loadbalancer for the Anthos on bare metal cluster. The output of the script prints out the same instructions as the all-in-one install; additionally it also prints out the Public IP addresses of the loadbalancers.

  • NFS Shared Storage:

    • An optional NFS server is provisioned in conjunction with any of the install methods above to provide shared storage to the Anthos on bare metal cluster.

Inputs

Name Description Type Default Required
abm_cluster_id Unique id to represent the Anthos Cluster to be created string "cluster1" no
abm_version Version of Anthos Bare Metal string "1.14.1" no
anthos_service_account_name Name given to the Service account that will be used by the Anthos cluster components string "baremetal-gcr" no
as_sub_module This script is being run as a sub module; thus output extra variables bool false no
boot_disk_size Size of the primary boot disk to be attached to the Compute Engine VMs in GBs number 200 no
boot_disk_type Type of the boot disk to be attached to the Compute Engine VMs string "pd-ssd" no
credentials_file Path to the Google Cloud Service Account key file.
This is the key that will be used to authenticate the provider with the Cloud APIs		 string n/a yes
enable_nested_virtualization Enable nested virtualization on the Compute Engine VMs are to be scheduled string "true" no
gcp_login_accounts GCP account email addresses that must be allowed to login to the cluster using Google Cloud Identity. list(string) [] no
gpu GPU information to be attached to the provisioned GCE instances.
See https://cloud.google.com/compute/docs/gpus for supported types		 object({ type = string, count = number }) 
{
  "count": 0,
  "type": ""
}
 no
image The source image to use when provisioning the Compute Engine VMs.
Use 'gcloud compute images list' to find a list of all available images		 string "ubuntu-2004-focal-v20221018" no
image_family Source image to use when provisioning the Compute Engine VMs.
The source image should be one that is in the selected image_project		 string "ubuntu-2004-lts" no
image_project Project name of the source image to use when provisioning the Compute Engine VMs string "ubuntu-os-cloud" no
instance_count Number of instances to provision per layer (Control plane and Worker nodes) of the cluster map(any) 
{
  "controlplane": 3,
  "worker": 2
}
 no
machine_type Google Cloud machine type to use when provisioning the Compute Engine VMs string "n1-standard-8" no
min_cpu_platform Minimum CPU architecture upon which the Compute Engine VMs are to be scheduled string "Intel Haswell" no
mode Indication of the execution mode. By default the terraform execution will end
after setting up the GCE VMs where the Anthos bare metal clusters can be deployed.

setup: create and initialize the GCE VMs required to install Anthos bare metal.

install: everything up to 'setup' mode plus automatically run Anthos bare metal installation steps as well.

manuallb: similar to 'install' mode but Anthos on bare metal is installed with ManualLB mode.		 string "setup" no
network VPC network to which the provisioned Compute Engine VMs is to be connected to string "default" no
nfs_server Provision a Google Filestore instance for NFS shared storage bool false no
primary_apis List of primary Google Cloud APIs to be enabled for this deployment list(string) 
[
  "cloudresourcemanager.googleapis.com"
]
 no
project_id Unique identifer of the Google Cloud Project that is to be used string n/a yes
region Google Cloud Region in which the Compute Engine VMs should be provisioned string "us-central1" no
resources_path Path to the resources folder with the template files string n/a yes
secondary_apis List of secondary Google Cloud APIs to be enabled for this deployment list(string) 
[
  "anthos.googleapis.com",
  "anthosgke.googleapis.com",
  "container.googleapis.com",
  "gkeconnect.googleapis.com",
  "gkehub.googleapis.com",
  "serviceusage.googleapis.com",
  "stackdriver.googleapis.com",
  "monitoring.googleapis.com",
  "logging.googleapis.com",
  "iam.googleapis.com",
  "compute.googleapis.com",
  "anthosaudit.googleapis.com",
  "opsconfigmonitoring.googleapis.com",
  "file.googleapis.com",
  "connectgateway.googleapis.com"
]
 no
tags List of tags to be associated to the provisioned Compute Engine VMs list(string) 
[
  "http-server",
  "https-server"
]
 no
username The name of the user to be created on each Compute Engine VM to execute the init script string "tfadmin" no
zone Zone within the selected Google Cloud Region that is to be used string "us-central1-b" no

Outputs

Name Description
abm_version Version of Anthos Bare Metal
admin_vm_ssh Run the following command to provision the anthos cluster.
admin_workstation_ip This is the IP address of your admin workstation. You may access the Anthos
on bare metal cluster by accessing this IP address.
admin_workstation_ssh_key This is the local file path for the SSH key of the admin workstation. You
may use this to SSH into the admin workstation.
controlplane_ip You may access the control plane nodes of the Anthos on bare metal cluster
by accessing this IP address. You need to copy the kubeconfig file for the
cluster from the admin workstation to access using the kubectl CLI.
ingress_ip You may access the application deployed in the Anthos on bare metal cluster
by accessing this IP address
installation_check Run the following command to check the Anthos bare metal installation status.

Contributing

Pre-requisites

  • The same pre-requisites to run this sample is required for testing as well

Pull requests

  • For improvements to this sample submit your pull requests to the main branch

Testing

  • Ensure that the improvements have unit/integration tests where appropriate
  • To run the existing tests you have to set two environment variables
export GOOGLE_CLOUD_PROJECT="<YOUR_GOOGLE_CLOUD_PROJECT>"
export GOOGLE_APPLICATION_CREDENTIALS="<PATH_TO_THE_SERVICE_ACCOUNT_KEY_FILE>"
  • Move into the test directory and recursively execute the tests
cd anthos-bm-gcp-terraform/test
go test -v -timeout 30m ./...