Red Hat OpenShift VPC cluster on IBM Cloud module

Use this module to provision an IBM Cloud Red Hat OpenShift cluster on VPC Gen2. The module either creates the required Cloud Object Storage instance or uses an existing instance. The module also supports optionally passing a key management configuration for secret encryption and boot volume encryption.

Before you begin

• Make sure that you have a recent version of the IBM Cloud CLI.
• Make sure that you have a recent version of the IBM Cloud Kubernetes service CLI.

Overview

• terraform-ibm-base-ocp-vpc
• Submodules
  • fscloud
• Examples
  • 2 MZR clusters in same VPC example
  • Advanced example (mzr, auto-scale, kms, taints)
  • Basic single zone example
  • Cluster security group rules example
  • Cross account KMS encryption example
  • Financial Services compliant example
• Contributing

terraform-ibm-base-ocp-vpc

Usage

module "ocp_base" {
  source               = "terraform-ibm-modules/base-ocp-vpc/ibm"
  version              = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  ibmcloud_api_key     = "XXXXXXXXXXXXXXXXXXX"
  cluster_name         = "example-cluster-name"
  resource_group_id    = "xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX"
  region               = "us-south"
  force_delete_storage = true
  vpc_id               = "79cxxxx-xxxx-xxxx-xxxx-xxxxxXX8667"
  # obtain the below values from the targeted VPC and adjust to the number of zones, subnets, subnet name, cidr_block, id, zone
  vpc_subnets          = {
    zone-1    = [
        {
            cidr_block = "192.168.32.0/22"
            id         = "0717-afc29fbb-0dbe-493a-a5b9-f3c5899cb8b9"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.36.0/22"
            id         = "0727-d65c1eda-9e38-4200-8452-cb8ff5bb3140"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.40.0/22"
            id         = "0737-9a823cd3-16bf-4ba4-a429-9e1fc7db74b8"
            zone       = "us-south-3"
        }
    ]
    zone-2 = [
        {
            cidr_block = "192.168.0.0/22"
            id         = "0717-846b9490-34ae-4a6c-8288-28112dca1ba3"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.4.0/22"
            id         = "0727-ef8db7f6-ffa5-4d8b-a317-4631741a45ee"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.8.0/22"
            id         = "0737-c9a6d871-d95b-4914-abf5-82c22f4161d1"
            zone       = "us-south-3"
        }
    ]
    zone-3 = [
        {
            cidr_block = "192.168.16.0/22"
            id         = "0717-d46e227c-89d4-4b02-9008-d03907a275b6"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.20.0/22"
            id         = "0727-93b1edcb-966c-4517-a7af-6ac63cd93adf"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.24.0/22"
            id         = "0737-807ec4f1-4d84-484e-b2f4-62dd5e431065"
            zone       = "us-south-3"
        }
    ]
  }
}

Troubleshooting

New kube_version message

• When you run a terraform plan command, you might get a message about a new version of Kubernetes, as in the following example:

  kube_version = "4.12.16_openshift" -> "4.12.20_openshift"
  
  Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

  A new version is detected because the Kubernetes master node is updated outside of Terraform, and the Terraform state is out of date with that version.

  The Kubernetes version is ignored in the module code, so the infrastructure will not be changed. The message identifies only that drift exists in the versions, and after you run a terraform apply command, the state will be refreshed.

Required IAM access policies

You need the following permissions to run this module.

• Account Management
  • All Identity and Access Enabled service
    • Viewer platform access
  • All Resource Groups service
    • Viewer platform access
• IAM Services
  • Cloud Object Storage service
    • Editor platform access
    • Manager service access
  • Kubernetes service
    • Administrator platform access
    • Manager service access
  • VPC Infrastructure service
    • Administrator platform access
    • Manager service access

Optionally, you need the following permissions to attach Access Management tags to resources in this module.

• IAM Services
  • Tagging service
    • Administrator platform access

Note :

• One worker pool should always be named as default. Refer issue 2849 for further details.

Requirements

Name	Version
terraform	>= 1.3.0, < 1.6.0
ibm	>= 1.60.0, < 2.0.0
kubernetes	>= 2.16.1
null	>= 3.2.1
time	>= 0.9.1

Modules

Name	Source	Version
cos_instance	terraform-ibm-modules/cos/ibm	7.1.4

Resources

Name	Type
ibm_container_addons.addons	resource
ibm_container_vpc_cluster.autoscaling_cluster	resource
ibm_container_vpc_cluster.cluster	resource
ibm_container_vpc_worker_pool.autoscaling_pool	resource
ibm_container_vpc_worker_pool.pool	resource
ibm_resource_tag.cluster_access_tag	resource
ibm_resource_tag.cos_access_tag	resource
kubernetes_config_map_v1_data.set_autoscaling	resource
null_resource.confirm_network_healthy	resource
null_resource.reset_api_key	resource
time_sleep.wait_operators	resource
ibm_container_addons.existing_addons	data source
ibm_container_cluster_config.cluster_config	data source
ibm_container_cluster_versions.cluster_versions	data source

Inputs

Name	Description	Type	Default	Required
access_tags	A list of access tags to apply to the resources created by the module, see https://cloud.ibm.com/docs/account?topic=account-access-tags-tutorial for more details	list(string)	[]	no
addons	Map of OCP cluster add-on versions to install (NOTE: The 'vpc-block-csi-driver' add-on is installed by default for VPC clusters, however you can explicitly specify it here if you wish to choose a later version than the default one). For full list of all supported add-ons and versions, see https://cloud.ibm.com/docs/containers?topic=containers-supported-cluster-addon-versions	object({ debug-tool = optional(string) image-key-synchronizer = optional(string) openshift-data-foundation = optional(string) vpc-file-csi-driver = optional(string) static-route = optional(string) cluster-autoscaler = optional(string) vpc-block-csi-driver = optional(string) })	null	no
cluster_config_endpoint_type	Specify which type of endpoint to use for for cluster config access: 'default', 'private', 'vpe', 'link'. 'default' value will use the default endpoint of the cluster.	string	"default"	no
cluster_name	The name that will be assigned to the provisioned cluster	string	n/a	yes
cluster_ready_when	The cluster is ready when one of the following: MasterNodeReady (not recommended), OneWorkerNodeReady, Normal, IngressReady	string	"IngressReady"	no
cos_name	Name of the COS instance to provision for OpenShift internal registry storage. New instance only provisioned if 'enable_registry_storage' is true and 'use_existing_cos' is false. Default: '<cluster_name>_cos'	string	null	no
disable_public_endpoint	Whether access to the public service endpoint is disabled when the cluster is created. Does not affect existing clusters. To change a public endpoint to private, create another cluster with the variable set to true or see Switching to the private endpoint.	bool	false	no
enable_registry_storage	Set to true to enable IBM Cloud Object Storage for the Red Hat OpenShift internal image registry. Set to false only for new cluster deployments in an account that is allowlisted for this feature.	bool	true	no
existing_cos_id	The COS id of an already existing COS instance to use for OpenShift internal registry storage. Only required if 'enable_registry_storage' and 'use_existing_cos' are true	string	null	no
force_delete_storage	Flag indicating whether or not to delete attached storage when destroying the cluster - Default: false	bool	false	no
ibmcloud_api_key	APIkey that's associated with the account to use, set via environment variable TF_VAR_ibmcloud_api_key	string	n/a	yes
ignore_worker_pool_size_changes	Enable if using worker autoscaling. Stops Terraform managing worker count	bool	false	no
kms_config	Use to attach a KMS instance to the cluster. If account_id is not provided, defaults to the account in use.	object({ crk_id = string instance_id = string private_endpoint = optional(bool, true) # defaults to true account_id = optional(string) # To attach KMS instance from another account })	null	no
manage_all_addons	Instructs Terraform to manage all cluster addons, even if addons were installed outside of the module. If set to 'true' this module will destroy any addons that were installed by other sources.	bool	false	no
ocp_entitlement	Value that is applied to the entitlements for OCP cluster provisioning	string	"cloud_pak"	no
ocp_version	The version of the OpenShift cluster that should be provisioned (format 4.x). This is only used during initial cluster provisioning, but ignored for future updates. Supports passing the string 'latest' (current latest available version) or 'default' (current IKS default recommended version). If no value is passed, it will default to 'default'.	string	null	no
region	The IBM Cloud region where the cluster will be provisioned.	string	n/a	yes
resource_group_id	The Id of an existing IBM Cloud resource group where the cluster will be grouped.	string	n/a	yes
tags	Metadata labels describing this cluster deployment, i.e. test	list(string)	[]	no
use_existing_cos	Flag indicating whether or not to use an existing COS instance for OpenShift internal registry storage. Only applicable if 'enable_registry_storage' is true	bool	false	no
verify_worker_network_readiness	By setting this to true, a script will run kubectl commands to verify that all worker nodes can communicate successfully with the master. If the runtime does not have access to the kube cluster to run kubectl commands, this should be set to false.	bool	true	no
vpc_id	Id of the VPC instance where this cluster will be provisioned	string	n/a	yes
vpc_subnets	Metadata that describes the VPC's subnets. Obtain this information from the VPC where this cluster will be created	map(list(object({ id = string zone = string cidr_block = string })))	n/a	yes
worker_pools	List of worker pools	list(object({ subnet_prefix = optional(string) vpc_subnets = optional(list(object({ id = string zone = string cidr_block = string }))) pool_name = string machine_type = string workers_per_zone = number resource_group_id = optional(string) labels = optional(map(string)) minSize = optional(number) maxSize = optional(number) enableAutoscaling = optional(bool) boot_volume_encryption_kms_config = optional(object({ crk = string kms_instance_id = string kms_account_id = optional(string) })) }))	n/a	yes
worker_pools_taints	Optional, Map of lists containing node taints by node-pool name	map(list(object({ key = string, value = string, effect = string })))	null	no

Outputs

Name	Description
cluster_crn	CRN for the created cluster
cluster_id	ID of cluster created
cluster_name	Name of the created cluster
cos_crn	CRN of the COS instance
ingress_hostname	Ingress hostname
kms_config	KMS configuration details
ocp_version	Openshift Version of the cluster
private_service_endpoint_url	Private service endpoint URL
public_service_endpoint_url	Public service endpoint URL
region	Region cluster is deployed in
resource_group_id	Resource group ID the cluster is deployed in
vpc_id	ID of the clusters VPC
workerpools	Worker pools created

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.