Overall

This module will deploy OpenShift with Cilium on the KVM hypervisor through Terraform Libvirt and aicli (https://github.com/karmab/aicli). It currently works with https://github.com/isovalent/terraform-equinix-infra modules to create the OpenShift VMs on Equinix Metal. In theory, it should work with any KVM hypervisor and may need more tweaks.

A lot of predefined configurations from the terraform-equinix-infra module on the router are for this module to deliver a seamless user experience when consuming OpenShift.

Example to deploy OpenShift on Equinix

Please consider this example as your starting point to get OpenShift running on Equinix:

module "infra" {
  source           = "git::https://github.com/isovalent/terraform-equinix-infra"
  api_key          = var.api_key
  infra_name       = "test"
  k8s_cluster_name = "liyi-ocp"
}

module "ocp" {
  source                         = "git::https://github.com/isovalent/terraform-libvirt-openshift"
  ocp_offline_token              = var.ocp_offline_token
  path_to_ocp_pull_secret        = "./pull-secret.txt"
  ocp_base_domain                = module.infra.dns_base_domain
  ocp_cluster_name               = module.infra.k8s_cluster_name
  ocp_masters_count              = module.infra.k8s_master_count
  ocp_workers_count              = module.infra.k8s_worker_count
  kvm_host_ip_address            = module.infra.host-public-ip-address
  path_to_kvm_host_login_ssh_key = module.infra.ssh_private_key_file_path
  path_to_ocp_setup_private_key  = module.infra.ssh_private_key_file_path
  path_to_cilium_config          = "./cilium-config.yaml"
  cilium_version                 = "1.15.6"
  libvirt_volume_main_name       = module.infra.libvirt_pool_main_name
  libvirt_private_network_id     = module.infra.libvirt_private_network_id
  private_network_ipv4_cidr      = module.infra.private_network_ipv4_cidr
  private_network_ipv6_cidr      = module.infra.private_network_ipv6_cidr
  ocp_master_ip_mac_hostname_map = module.infra.k8s_master_ip_mac_hostname_map
  ocp_worker_ip_mac_hostname_map = module.infra.k8s_worker_ip_mac_hostname_map
  kube_api_server_ip             = module.infra.router-public-ip-address
}

You need an account at https://console.redhat.com/ to get the pull secrets and the ocp_offline_token.

You need to run terraform apply --target=module.infra and terraform apply in 2 separate steps to avoid the dependency issue between the 2 modules.

After running it, it will take around 60 minutes to get the cluster up and running, and you can check https://console.redhat.com/openshift for the deployment status.

Notes

• OpenShift VMs are on the private network and use the router from the terraform-equinix-infra module as the gateway. If you would like to access the OpenShift VM through SSH, you will need to SSH to the testbox from the terraform-equinix-infra module as the jumpbox.

• The SSH key to the OpenShift VM and OpenShift setup manifest are on the hypervisor in the /aicli/ directory.

• The router in the terraform-equinix-infra module has preconfigured the HA proxy for ports 6443, 443, and 80, so it will forward the traffic to OpenShift without any configuration from you.

• Because we have a private FQDN which is only valid on the private network, we have a post_install_script to override your local /etc/hosts file to route the traffic to the router if you use the variable kube_api_server_ip with the router's public IP address.

Output of this modules

After running this module, the kubeconfig path will be output and you can access the API server through kubectl. In the same directory, you can also find the kubeadmin password so you can access the GUI of OpenShift.

Requirements

Name	Version
terraform	>=1.6.5
libvirt	>=0.7.6
remote	0.1.2

Providers

Name	Version
libvirt	>=0.7.6
local	n/a
null	n/a
remote	0.1.2

Modules

No modules.

Resources

Name	Type
libvirt_domain.ocp_masters	resource
libvirt_domain.ocp_workers	resource
libvirt_volume.ocp_masters	resource
libvirt_volume.ocp_masters_base	resource
libvirt_volume.ocp_workers	resource
libvirt_volume.ocp_workers_base	resource
local_file.aicli_ocp_config	resource
local_file.ocp_kubeadmin_password	resource
local_file.ocp_kubeconfig	resource
null_resource.aicli_cilium_setup	resource
null_resource.ocp_install	resource
null_resource.post_install_script	resource
remote_file.ocp_kubeadmin_password	data source
remote_file.ocp_kubeconfig	data source

Inputs

Name	Description	Type	Default	Required
aicli_version	aicli version	string	"99.0.202403282009"	no
cilium_version	cilium version for the ocp please check https://github.com/isovalent/cilium-ee-olm since olm has a different life cycle from helm release	string	n/a	yes
kube_api_server_ip	kube api server ip address, since we use the private FQDN, setting this will run post_install_script resource to override the /etct/hosts files	string	""	no
kvm_host_ip_address	kvm host ip address for OCP VM deployment	string	n/a	yes
kvm_host_username	kvm host username for OCP VM deployment	string	"root"	no
libvirt_private_network_id	libvirt private network id	any	n/a	yes
libvirt_volume_main_name	libvirt main pool name	any	n/a	yes
ocp_base_domain	base domain name for ocp cluster	string	n/a	yes
ocp_cluster_name	ocp cluster name	string	"default-ocp-name"	no
ocp_master_ip_mac_hostname_map	master_ip_mac_host_map from terraform-equinix-infra module. If provided, it will get the static IP address and FQDN mapping predefined in terraform-equinix-infra's router	string	""	no
ocp_masters_count	the number of the OCP master VM, 3 or 5 shoud be enough for most cases.	number	3	no
ocp_offline_token	the assistant installer token gather an offline token at https://cloud.redhat.com/openshift/token	string	""	no
ocp_version	openshift version	string	4.14	no
ocp_worker_ip_mac_hostname_map	worker_ip_mac_host_map from terraform-equinix-infra module. If provided, it will get the static IP address and FQDN mapping predefined in terraform-equinix-infra's router	string	""	no
ocp_workers_count	the number of the OCP worker VM.	number	2	no
path_to_cilium_config	path to cilium config for ocp	string	n/a	yes
path_to_kvm_host_login_ssh_key	private ssh key to login the kvm host	string	n/a	yes
path_to_ocp_pull_secret	the ocp pull secret, the pull secret should match your redhat account for the offline_token. you can get it from here https://console.redhat.com/openshift/install/pull-secret	string	""	no
path_to_ocp_setup_private_key	private key to setup the ocp cluster	string	n/a	yes
private_network_ipv4_cidr	private VM networks where the OCP VM sit	any	n/a	yes
private_network_ipv6_cidr	private VM ipv6 networks where the OCP VM sit	string	""	no

Outputs

Name	Description
ocp_path_to_kubeconfig_file	ocp kubeconfig location to access the k8s api server