Terraform Intersight Kubernetes Service Module

The Cisco Intersight Terraform Provider is available in the Terraform Registry at https://registry.terraform.io/providers/CiscoDevNet/intersight/latest. This repository contains example modules that use the provider to create

A terraform module to create a managed Kubernetes clusters using Intersight Kubernetes Service (IKS). Available through the Terraform registry.

and its source code.

CAVEAT

• You cannot assign the cluster action as "Deploy" and "wait_for_completion" as TRUE at the same time.

Assumptions

• You want to create an IKS cluster on your on-premises infrastructure using Intersight.
• These resources will be provided using Intersight and VMware vCenter 6.7.
• You've claimed vCenter using the Intersight Assist Appliance.

Details

This module creates all of the resources required for IKS. Those resources are identitified below. It is designed as a quickstart/example of how to get an IKS cluster running. More customization is being enabled but currently there are some caveats:

Reusing prebuilt policies is supported. Each object block has a variable for doing this. Set

use_existing = true

If existing objects are not available this module will create those objects for you where required. Set

use_existing = false

For the runtime_policies and the Trusted registry, if you DO NOT want to use this policy in your cluster build you need to set the following variable combination in EACH object block.

  use_existing         = false
  create_new           = false

Usage

See the Examples ---> Complete directory for usage of this module.

There are 4 example files below that are needed to use this module. Create these files in the same directory, run terraform init. You will then be ready to run terraform plan or terraform apply.

Change the variables in the terraform.tfvars file and the main.tf as needed. See the above Examples folder for more information.

Sample main.tf file.

provider "intersight" {
  apikey    = var.apikey
  secretkey = var.secretkey
  endpoint  = var.endpoint
}

module "terraform-intersight-iks" {

  source  = "terraform-cisco-modules/iks/intersight//"
  version = "2.1.0"

# Kubernetes Cluster Profile  Adjust the values as needed.
  cluster = {
    name                = "new_cluster"
    action              = "Unassign"
    wait_for_completion = false
    worker_nodes        = 5
    load_balancers      = 5
    worker_max          = 20
    control_nodes       = 1
    ssh_user            = var.ssh_user
    ssh_public_key      = var.ssh_key
  }


# IP Pool Information (To create new change "use_existing" to 'false' uncomment variables and modify them to meet your needs.)
  ip_pool = {
    use_existing        = true
    name                = "10-239-21-0"
    # ip_starting_address = "10.239.21.220"
    # ip_pool_size        = "20"
    # ip_netmask          = "255.255.255.0"
    # ip_gateway          = "10.239.21.1"
    # dns_servers         = ["10.101.128.15","10.101.128.16"]
  }

# Sysconfig Policy (UI Reference NODE OS Configuration) (To create new change "use_existing" to 'false' uncomment variables and modify them to meet your needs.)
  sysconfig = {
    use_existing = true
    name         = "richfield"
    # domain_name  = "rich.ciscolabs.com"
    # timezone     = "America/New_York"
    # ntp_servers  = ["10.101.128.15"]
    # dns_servers  = ["10.101.128.15"]
  }

# Kubernetes Network CIDR (To create new change "use_existing" to 'false' uncomment variables and modify them to meet your needs.)
  k8s_network = {
    use_existing = true
    name         = "default"

    ######### Below are the default settings.  Change if needed. #########
    # pod_cidr     = "100.65.0.0/16"
    # service_cidr = "100.64.0.0/24"
    # cni          = "Calico"
  }
# Version policy (To create new change "useExisting" to 'false' uncomment variables and modify them to meet your needs.)
  versionPolicy = {
    useExisting = true
    policyName     = "1-19-15-iks.3"
    iksVersionName = "1.19.15-iks.3"
  }
# Trusted Registry Policy (To create new change "use_existing" to 'false' and set "create_new' to 'true' uncomment variables and modify them to meet your needs.)
# Set both variables to 'false' if this policy is not needed.
  tr_policy = {
    use_existing = false
    create_new   = false
    name         = "trusted-registry"
  }
# Runtime Policy (To create new change "use_existing" to 'false' and set "create_new' to 'true' uncomment variables and modify them to meet your needs.)
# Set both variables to 'false' if this policy is not needed.
  runtime_policy = {
    use_existing = false
    create_new   = false
    # name                 = "runtime"
    # http_proxy_hostname  = "t"
    # http_proxy_port      = 80
    # http_proxy_protocol  = "http"
    # http_proxy_username  = null
    # http_proxy_password  = null
    # https_proxy_hostname = "t"
    # https_proxy_port     = 8080
    # https_proxy_protocol = "https"
    # https_proxy_username = null
    # https_proxy_password = null
  }

# Infrastructure Configuration Policy (To create new change "use_existing" to 'false' and uncomment variables and modify them to meet your needs.)
  infraConfigPolicy = {
    use_existing = true
    # platformType = "iwe"
    # targetName   = "falcon"
    policyName   = "dev"
    # description  = "Test Policy"
    # interfaces   = ["iwe-guests"]
    # vcTargetName   = optional(string)
    # vcClusterName      = optional(string)
    # vcDatastoreName     = optional(string)
    # vcResourcePoolName = optional(string)
    # vcPassword      = optional(string)
  }

# Addon Profile and Policies (To create new change "createNew" to 'true' and uncomment variables and modify them to meet your needs.)
# This is an Optional item.  Comment or remove to not use.  Multiple addons can be configured.
  addons       = [
    {
    createNew = true
    addonPolicyName = "smm-tf"
    addonName            = "smm"
    description       = "SMM Policy"
    upgradeStrategy  = "AlwaysReinstall"
    installStrategy  = "InstallOnly"
    releaseVersion = "1.7.4-cisco4-helm3"
    overrides = yamlencode({"demoApplication":{"enabled":true}})
    },
    # {
    # createNew = true
    # addonName            = "ccp-monitor"
    # description       = "monitor Policy"
    # # upgradeStrategy  = "AlwaysReinstall"
    # # installStrategy  = "InstallOnly"
    # releaseVersion = "0.2.61-helm3"
    # # overrides = yamlencode({"demoApplication":{"enabled":true}})
    # }
  ]

# Worker Node Instance Type (To create new change "use_existing" to 'false' and uncomment variables and modify them to meet your needs.)
  instance_type = {
    use_existing = true
    name         = "small"
    # cpu          = 4
    # memory       = 16386
    # disk_size    = 40
  }

# Organization and Tag Information
  organization = var.organization
  tags         = var.tags
}

Sample terraform.tfvars file.

apikey       = ""
secretkey    = "../../.secret"
organization = "default"
ssh_user = "iksadmin"
ssh_key  = ""
tags = [
  {
    "key" : "managed_by"
    "value" : "Terraform"
  },
  {
    "key" : "owner"
    "value" : "jb"
  }
]
organization = "default" # Change this if a different org is required.  Default org is set to "default"

Sample versions.tf file

terraform {
  required_version = ">=0.14.5"

  required_providers {
    intersight = {
      source  = "CiscoDevNet/intersight"
      version = ">=1.0.18"
    }
  }
}

Sample variables.tf file.

variable "apikey" {
  type        = string
  description = "API Key"
}
variable "secretkey" {
  type        = string
  description = "Secret Key or file location"
}
variable "endpoint" {
  type        = string
  description = "API Endpoint URL"
  default     = "https://www.intersight.com"
}
variable "organization" {
  type        = string
  description = "Organization Name"
  default     = "default"
}
variable "ssh_user" {
  type        = string
  description = "SSH Username for node login."
}
variable "ssh_key" {
  type        = string
  description = "SSH Public Key to be used to node login."
}
variable "tags" {
  type    = list(map(string))
  default = []
}

Always check Kubernetes Release Notes before updating the major version.

Requirements

Name	Version
terraform	>=0.14.5
intersight	>=1.0.18

Providers

Name	Version
intersight	>=1.0.18

Modules

Name	Source	Version
addons	terraform-cisco-modules/iks/intersight//modules/addon_policy	n/a
cluster_addon_profile	terraform-cisco-modules/iks/intersight//modules/cluster_addon_profile	n/a
cluster_profile	terraform-cisco-modules/iks/intersight//modules/cluster	n/a
control_profile	terraform-cisco-modules/iks/intersight//modules/node_profile	n/a
control_provider	terraform-cisco-modules/iks/intersight//modules/infra_provider	n/a
infra_config_policy	terraform-cisco-modules/iks/intersight//modules/infra_config_policy	n/a
instance_type	terraform-cisco-modules/iks/intersight//modules/worker_profile	n/a
ip_pool_policy	terraform-cisco-modules/iks/intersight//modules/ip_pool	n/a
k8s_network	terraform-cisco-modules/iks/intersight//modules/k8s_network	n/a
k8s_sysconfig	terraform-cisco-modules/iks/intersight//modules/k8s_sysconfig	n/a
k8s_version	terraform-cisco-modules/iks/intersight//modules/version	n/a
runtime_policy	terraform-cisco-modules/iks/intersight//modules/runtime_policy	n/a
trusted_registry	terraform-cisco-modules/iks/intersight//modules/trusted_registry	n/a
worker_profile	terraform-cisco-modules/iks/intersight//modules/node_profile	n/a
worker_provider	terraform-cisco-modules/iks/intersight//modules/infra_provider	n/a

Resources

Name	Type
intersight_ippool_pool.this	data source
intersight_kubernetes_container_runtime_policy.this	data source
intersight_kubernetes_network_policy.this	data source
intersight_kubernetes_sys_config_policy.this	data source
intersight_kubernetes_trusted_registries_policy.this	data source
intersight_kubernetes_version_policy.this	data source
intersight_kubernetes_virtual_machine_infra_config_policy.this	data source
intersight_kubernetes_virtual_machine_instance_type.this	data source

Inputs

Name	Description	Type	Default	Required
addons	n/a	list(object({ createNew = bool addonPolicyName = optional(string) addonName = optional(string) description = optional(string) upgradeStrategy = optional(string) installStrategy = optional(string) overrideSets = optional(list(map(string))) overrides = optional(string) releaseName = optional(string) releaseNamespace = optional(string) releaseVersion = optional(string) }))	[]	no
cluster	n/a	object({ name = string action = string wait_for_completion = bool worker_nodes = number load_balancers = number worker_max = number control_nodes = number ssh_user = string ssh_public_key = string })	n/a	yes
infraConfigPolicy	n/a	object({ use_existing = bool platformType = optional(string) targetName = optional(string) policyName = string description = optional(string) interfaces = optional(list(string)) diskMode = optional(string) vcTargetName = optional(string) vcClusterName = optional(string) vcDatastoreName = optional(string) vcResourcePoolName = optional(string) vcPassword = optional(string) })	n/a	yes
infra_config_policy_name	Name of existing infra config policy (if it exists) to be used.	string	""	no
instance_type	n/a	object({ use_existing = bool name = string cpu = optional(number) memory = optional(number) disk_size = optional(number) })	n/a	yes
ip_pool	n/a	object({ use_existing = bool name = string ip_starting_address = optional(string) ip_pool_size = optional(string) ip_netmask = optional(string) ip_gateway = optional(string) dns_servers = optional(list(string)) })	n/a	yes
k8s_network	n/a	object({ use_existing = bool name = optional(string) pod_cidr = optional(string) service_cidr = optional(string) cni = optional(string) })	n/a	yes
k8s_network_policy_name	Name of existing K8s Network Policy (if it exists) to be used.	string	""	no
organization	Organization Name	string	"default"	no
runtime_policy	n/a	object({ use_existing = bool create_new = bool name = optional(string) http_proxy_hostname = optional(string) http_proxy_port = optional(number) http_proxy_protocol = optional(string) http_proxy_username = optional(string) http_proxy_password = optional(string) https_proxy_hostname = optional(string) https_proxy_port = optional(number) https_proxy_protocol = optional(string) https_proxy_username = optional(string) https_proxy_password = optional(string) docker_no_proxy = optional(list(string)) })	n/a	yes
sysconfig	n/a	object({ use_existing = bool name = string ntp_servers = optional(list(string)) dns_servers = optional(list(string)) timezone = optional(string) domain_name = optional(string) })	n/a	yes
tags	n/a	list(map(string))	[]	no
tr_policy	n/a	object({ use_existing = bool create_new = bool name = optional(string) root_ca_registries = optional(list(string)) unsigned_registries = optional(list(string)) })	n/a	yes
versionPolicy	n/a	object({ useExisting = bool policyName = string iksVersionName = optional(string) description = optional(string) versionName = optional(string) })	n/a	yes

Outputs

Name	Description
k8s_cluster_moid	n/a