By default, the cluster is accessible over the public internet. The firewall is automatically configured to use the IPv4 address and /64 IPv6 CIDR of the machine running this module. To disable this automatic configuration, set the following variables to false:

firewall_use_current_ipv4 = false
firewall_use_current_ipv6 = false

To manually specify source networks for the Talos API and Kube API, configure the firewall_talos_api_source and firewall_kube_api_source variables as follows:

firewall_talos_api_source = [
  "1.2.3.0/32",
  "1:2:3::/64"
]
firewall_kube_api_source = [
  "1.2.3.0/32",
  "1:2:3::/64"
]

This allows explicit control over which networks can access your APIs, overriding the default behavior when set.

If your internal network is routed and accessible, you can directly access the cluster using internal IPs by setting:

cluster_access = "private"

For integrating Talos nodes with an internal network, configure a default route (0.0.0.0/0) in the Hetzner Network to point to your router or gateway. Additionally, add specific routes on the Talos nodes to encompass your entire network CIDR:

talos_extra_routes = ["10.0.0.0/8"]

# Optionally, disable NAT for your globally routed CIDR
network_native_routing_cidr = "10.0.0.0/8"

# Optionally, use an existing Network
hcloud_network_id = 123456789

This setup ensures that the Talos nodes can route traffic appropriately across your internal network.

Optionally, a hostname can be configured to direct access to the Kubernetes API through a node IP, load balancer, or Virtual IP (VIP):

kube_api_hostname = "kube-api.example.com"

For accessing the Kubernetes API from the public internet, choose one of the following options based on your needs:

1. Use a Load Balancer (Recommended):
   Deploy a load balancer to manage API traffic, enhancing availability and load distribution.

   kube_api_load_balancer_enabled = true

2. Use a Virtual IP (Floating IP):
   A Floating IP is configured to automatically move between control plane nodes in case of an outage, ensuring continuous access to the Kubernetes API.

   control_plane_public_vip_ipv4_enabled = true
   
   # Optionally, specify an existing Floating IP
   control_plane_public_vip_ipv4_id = 123456789

When accessing the Kubernetes API via an internal network, an internal Virtual IP (Alias IP) is utilized by default to route API requests within the network. This feature can be disabled with the following configuration:

control_plane_private_vip_ipv4_enabled = false

To enhance internal availability, a load balancer can be used:

kube_api_load_balancer_enabled = true

This setup ensures secure and flexible access to the Kubernetes API, accommodating different networking environments.

Example kubernetes.tf snippet:

# Configuration for cluster autoscaler node pools
cluster_autoscaler_nodepools = [
  {
    name     = "autoscaler"
    type     = "cax11"
    location = "fsn1"
    min      = 0
    max      = 6
    labels   = { "autoscaler-node" = "true" }
    taints   = [ "autoscaler-node=true:NoExecute" ]
  }
]

Optionally, pass additional Helm values to the cluster autoscaler configuration:

cluster_autoscaler_helm_values = {
  extraArgs = {
    enforce-node-group-min-size   = true
    scale-down-delay-after-add    = "45m"
    scale-down-delay-after-delete = "4m"
    scale-down-unneeded-time      = "5m"
  }
}

Cilium offers an Egress Gateway to ensure network compatibility with legacy systems and firewalls requiring fixed IPs. The use of Cilium Egress Gateway does not provide high availability and increases latency due to extra network hops and tunneling. Consider this configuration only as a last resort.

Example kubernetes.tf snippet:

# Enable Cilium Egress Gateway
cilium_egress_gateway_enabled = true

# Define worker nodepools including an egress-specific node pool
worker_nodepools = [
  # ... (other node pool configurations)
  {
    name     = "egress"
    type     = "cax11"
    location = "fsn1"
    labels   = { "egress-node" = "true" }
    taints   = [ "egress-node=true:NoSchedule" ]
  }
]

Example Egress Gateway Policy:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: sample-egress-policy
spec:
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: sample-namespace
          app: sample-app

  destinationCIDRs:
    - "0.0.0.0/0"

  egressGateway:
    nodeSelector:
      matchLabels:
        egress-node: "true"

Please visit the Cilium documentation for more details.

Each rule is defined with the following properties:

• description: A brief description of the rule.
• direction: The direction of traffic (in for inbound, out for outbound).
• source_ips: A list of source IP addresses for outbound rules.
• destination_ips: A list of destination IP addresses for inbound rules.
• protocol: The protocol used (valid options: tcp, udp, icmp, gre, esp).
• port: The port number (required for tcp and udp protocols, must not be specified for icmp, gre, and esp).

Example kubernetes.tf snippet:

firewall_extra_rules = [
  {
    description = "Custom UDP Rule"
    direction   = "in"
    source_ips  = ["0.0.0.0/0", "::/0"]
    protocol    = "udp"
    port        = "12345"
  },
  {
    description = "Custom TCP Rule"
    direction   = "in"
    source_ips  = ["1.2.3.4", "1:2:3:4::"]
    protocol    = "tcp"
    port        = "8080-9000"
  },
  {
    description = "Allow ICMP"
    direction   = "in"
    source_ips  = ["0.0.0.0/0", "::/0"]
    protocol    = "icmp"
  }
]

For access to Talos and the Kubernetes API, please refer to the Cluster Access configuration section.

The ingress controller uses a default load balancer service to manage external traffic. For geo-redundancy and high availability, ingress_load_balancer_pools can be configured as an alternative, replacing the default load balancer with the specified pool of load balancers.

To replace the default load balancer, use ingress_load_balancer_pools in the Terraform configuration. This setup ensures high availability and geo-redundancy by distributing traffic from various locations across all targets in all regions.

Example kubernetes.tf configuration:

ingress_load_balancer_pools = [
  {
    name     = "lb-nbg"
    location = "nbg1"
    type     = "lb11"
  },
  {
    name     = "lb-fsn"
    location = "fsn1"
    type     = "lb11"
  }
]

Configuring local traffic handling enhances network efficiency by reducing latency. Processing traffic closer to its source eliminates unnecessary routing delays, ensuring consistent performance for low-latency or region-sensitive applications.

Example kubernetes.tf configuration:

ingress_nginx_kind = "DaemonSet"
ingress_nginx_service_external_traffic_policy = "Local"

ingress_load_balancer_pools = [
  {
    name          = "regional-lb-nbg"
    location      = "nbg1"
    local_traffic = true
  },
  {
    name          = "regional-lb-fsn"
    location      = "fsn1"
    local_traffic = true
  }
]

Key settings in this configuration:

• local_traffic: Limits load balancer targets to nodes in the same geographic location as the load balancer, reducing data travel distances and keeping traffic within the region.
• ingress_nginx_service_external_traffic_policy set to Local: Ensures external traffic is handled directly on the local node, avoiding extra network hops.
• ingress_nginx_kind set to DaemonSet: Deploys an ingress controller instance on every node, enabling requests to be handled locally for faster response times.

Topology-aware routing in ingress-nginx can optionally be enabled by setting the ingress_nginx_topology_aware_routing variable to true. This functionality routes traffic to the nearest upstream endpoints, enhancing efficiency for supported services. Note that this feature is only applicable to services that support topology-aware routing. For more information, refer to the Kubernetes documentation.

By default, this module calculates optimal subnets based on the provided network CIDR (network_ipv4_cidr). The network is segmented automatically as follows:

• 1st Quarter: Reserved for other uses such as classic VMs.
• 2nd Quarter:
  • 1st Half: Allocated for Node Subnets (network_node_ipv4_cidr)
  • 2nd Half: Allocated for Service IPs (network_service_ipv4_cidr)
• 3rd and 4th Quarters:
  • Full Span: Allocated for Pod Subnets (network_pod_ipv4_cidr)

Each Kubernetes node requires a /24 subnet within network_pod_ipv4_cidr. To support this configuration, the optimal node subnet size (network_node_ipv4_subnet_mask_size) is calculated using the formula:
32 - (24 - subnet_mask_size(network_pod_ipv4_cidr)).

With the default 10.0.0.0/16 network CIDR (network_ipv4_cidr), the following values are calculated:

• Node Subnet Size: /25 (Max. 128 Nodes per Subnet)
• Node Subnets: 10.0.64.0/19 (Max. 64 Subnets, each with /25)
• Service IPs: 10.0.96.0/19 (Max. 8192 Services)
• Pod Subnet Size: /24 (Max. 256 Pods per Node)
• Pod Subnets: 10.0.128.0/17 (Max. 128 Nodes, each with /24)

Please consider the following Hetzner Cloud limits:

• Up to 100 servers can be attached to a network.
• Up to 100 routes can be created per network.
• Up to 50 subnets can be created per network.
• A project can have up to 50 placement groups.

A /16 Network CIDR is sufficient to fully utilize Hetzner Cloud's scaling capabilities. It supports:

• Up to 100 nodes, each with its own /24 Pod subnet route.
• Configuration of up to 50 nodepools, one nodepool per subnet, each with at least one placement group.

Here is a table with more example calculations:

This module natively supports Hcloud Object Storage. Below is an example of how to configure backups with MinIO Client (mc) and Hcloud Object Storage. While it's possible to create the bucket through the Hcloud Console, this method does not allow for the configuration of automatic retention policies.

Create an alias for the endpoint using the following command:

mc alias set <alias> \
  https://<location>.your-objectstorage.com \
  <access-key> <secret-key> \
  --api "s3v4" \
  --path "off"

Create a bucket with automatic retention policies to protect your backups:

mc mb --with-lock --region <location> <alias>/<bucket>
mc retention set GOVERNANCE 14d --default <alias>/<bucket>

Configure your kubernetes.tf file:

talos_backup_s3_hcloud_url = "https://<bucket>.<location>.your-objectstorage.com"
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"

# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"

# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"

For users of other object storage providers, configure kubernetes.tf as follows:

talos_backup_s3_region   = "<region>"
talos_backup_s3_endpoint = "<endpoint>"
talos_backup_s3_bucket   = "<bucket>"
talos_backup_s3_prefix   = "<prefix>"

# Use path-style URLs (set true if required by your provider)
talos_backup_s3_path_style = true

# Access credentials
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"

# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"

# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"

To recover from a snapshot, please refer to the Talos Disaster Recovery section in the Documentation.