Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Airgap Support #253

Merged
merged 6 commits into from
Nov 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,7 @@ k3s_cluster:
If needed, you can also edit `vars` section at the bottom to match your environment.

If multiple hosts are in the server group the playbook will automatically setup k3s in HA mode with embedded etcd.
An odd number of server nodes is required (3,5,7). Read the offical documentation below for more information and options.
https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/
An odd number of server nodes is required (3,5,7). Read the [official documentation](https://docs.k3s.io/datastore/ha-embedded) for more information.

Setting up a loadbalancer or VIP beforehand to use as the API endpoint is possible but not covered here.

Expand All @@ -72,6 +71,26 @@ A playbook is provided to upgrade K3s on all nodes in the cluster. To use it, up
ansible-playbook playbook/upgrade.yml -i inventory.yml
```

## Airgap Install

Airgap installation is supported via the `airgap_dir` variable. This variable should be set to the path of a directory containing the K3s binary and images. The release artifacts can be downloaded from the [K3s Releases](https://github.com/k3s-io/k3s/releases). You must download the appropriate images for you architecture (any of the compression formats will work).

An example folder for an x86_64 cluster:
```bash
$ ls ./playbook/my-airgap/
total 248M
-rwxr-xr-x 1 $USER $USER 58M Nov 14 11:28 k3s
-rw-r--r-- 1 $USER $USER 190M Nov 14 11:30 k3s-airgap-images-amd64.tar.gz

$ cat inventory.yml
...
airgap_dir: ./my-airgap # Paths are relative to the playbook directory
```

Additionally, if deploying on a OS with SELinux, you will also need to download the latest [k3s-selinux RPM](https://github.com/k3s-io/k3s-selinux/releases/latest) and place it in the airgap folder.


It is assumed that the control node has access to the internet. The playbook will automatically download the k3s install script on the control node, and then distribute all three artifacts to the managed nodes.

## Kubeconfig

Expand Down
2 changes: 1 addition & 1 deletion Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ NETWORK_PREFIX = "10.10.10"
def provision(vm, role, node_num)
vm.box = NODE_BOXES[node_num]
vm.hostname = role
# We use a private network because the default IPs are dynamicly assigned
# We use a private network because the default IPs are dynamically assigned
# during provisioning. This makes it impossible to know the server-0 IP when
# provisioning subsequent servers and agents. A private network allows us to
# assign static IPs to each node, and thus provide a known IP for the API endpoint.
Expand Down
5 changes: 3 additions & 2 deletions inventory-sample.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,16 @@
extra_server_args: ""
extra_agent_args: ""

# Optional vars

Check warning on line 22 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

yaml[comments-indentation]

Comment not indented like content

Check warning on line 22 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

22:3 [comments-indentation] comment not indented like content
# api_port: 6443

Check warning on line 23 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

yaml[comments-indentation]

Comment not indented like content

Check warning on line 23 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

23:5 [comments-indentation] comment not indented like content
# k3s_server_location: /var/lib/rancher/k3s
# systemd_dir: /etc/systemd/system
# extra_service_envs: [ 'ENV_VAR1=VALUE1', 'ENV_VAR2=VALUE2' ]
# Manifests or Airgap should be either full paths or relative to the playbook directory.
# List of locally available manifests to apply to the cluster, useful for PVCs or Traefik modifications.
# Manifests should be either full paths or relative to the playbook directory.
# extra_manifests: [ '/path/to/manifest1.yaml', '/path/to/manifest2.yaml' ]
# airgap_dir: /tmp/k3s-airgap-images
# config_yaml: |
# This is now an inner yaml file. Mantain the indentation.
# This is now an inner yaml file. Maintain the indentation.

Check warning on line 32 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

yaml[comments-indentation]

Comment not indented like content

Check warning on line 32 in inventory-sample.yml

View workflow job for this annotation

GitHub Actions / Lint

32:7 [comments-indentation] comment not indented like content
# YAML here will be placed as the content of /etc/rancher/k3s/config.yaml
# See https://docs.k3s.io/installation/configuration#configuration-file
129 changes: 127 additions & 2 deletions roles/download/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,126 @@
---
- name: Check for Airgap
when: airgap_dir is defined
block:
- name: Download k3s install script [Airgap]
delegate_to: localhost
ansible.builtin.get_url:
url: https://get.k3s.io/
timeout: 120
dest: "{{ airgap_dir }}/k3s-install.sh"
mode: 0755

- name: Distribute K3s install script [Airgap]
ansible.builtin.copy:
src: "{{ airgap_dir }}/k3s-install.sh"
dest: /usr/local/bin/k3s-install.sh
owner: root
group: root
mode: 0755

- name: Distribute K3s binary [Airgap]
ansible.builtin.copy:
src: "{{ airgap_dir }}/k3s"
dest: /usr/local/bin/k3s
owner: root
group: root
mode: 0755

- name: Distribute K3s SELinux RPM [Airgap]
ansible.builtin.copy:
src: "{{ item }}"
dest: /tmp/
owner: root
group: root
mode: 0755
with_fileglob:
- "{{ airgap_dir }}/k3s-selinux*.rpm"
register: selinux_copy
ignore_errors: true

- name: Install K3s SELinux RPM [Airgap]
when:
- ansible_os_family == 'RedHat'
- selinux_copy.skipped is false
ansible.builtin.yum:
name: "{{ selinux_copy.results[0].dest }}"
state: present
disable_gpg_check: true

- name: Make images directory [Airgap]
ansible.builtin.file:
path: "/var/lib/rancher/k3s/agent/images/"
mode: 0755
state: directory

- name: Determine Architecture [Airgap]
ansible.builtin.set_fact:
k3s_arch: "{{ ansible_architecture }}"

- name: Distribute K3s amd64 images [Airgap]
when: ansible_architecture == 'x86_64'
ansible.builtin.copy:
src: "{{ item }}"
dest: /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar
owner: root
group: root
mode: 0755
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-amd64.tar.zst"
- "{{ airgap_dir }}/k3s-airgap-images-amd64.tar.gz"
- "{{ airgap_dir }}/k3s-airgap-images-amd64.tar"
skip: true

- name: Distribute K3s arm64 images [Airgap]
when: ansible_architecture == 'aarch64'
ansible.builtin.copy:
src: "{{ item }}"
dest: /var/lib/rancher/k3s/agent/images/k3s-airgap-images-arm64.tar
owner: root
group: root
mode: 0755
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-arm64.tar.zst"
- "{{ airgap_dir }}/k3s-airgap-images-arm64.tar.gz"
- "{{ airgap_dir }}/k3s-airgap-images-arm64.tar"
skip: true

- name: Distribute K3s arm images [Airgap]
when: ansible_architecture == 'armv7l'
ansible.builtin.copy:
src: "{{ item }}"
dest: /var/lib/rancher/k3s/agent/images/k3s-airgap-images-arm.tar
owner: root
group: root
mode: 0755
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-arm.tar.zst"
- "{{ airgap_dir }}/k3s-airgap-images-arm.tar.gz"
- "{{ airgap_dir }}/k3s-airgap-images-arm.tar"
skip: true

- name: Run K3s Install [server][Airgap]
ansible.builtin.command:
cmd: /usr/local/bin/k3s-install.sh
environment:
INSTALL_K3S_SKIP_START: "true"
INSTALL_K3S_SKIP_DOWNLOAD: "true"
changed_when: true

- name: Run K3s Install [agent][Airgap]
ansible.builtin.command:
cmd: /usr/local/bin/k3s-install.sh
environment:
INSTALL_K3S_SKIP_START: "true"
INSTALL_K3S_SKIP_DOWNLOAD: "true"
INSTALL_K3S_EXEC: "agent"
changed_when: true

- name: Download k3s install script
when: airgap_dir is undefined
ansible.builtin.get_url:
url: https://get.k3s.io/
timeout: 120
Expand All @@ -9,7 +130,9 @@
mode: 0755

- name: Download k3s binary [server]
when: "'server' in group_names"
when:
- "'server' in group_names"
- airgap_dir is undefined
ansible.builtin.command:
cmd: /usr/local/bin/k3s-install.sh
environment:
Expand All @@ -18,7 +141,9 @@
changed_when: true

- name: Download k3s binary [agent]
when: "'agent' in group_names"
when:
- "'agent' in group_names"
- airgap_dir is undefined
ansible.builtin.command:
cmd: /usr/local/bin/k3s-install.sh
environment:
Expand Down
2 changes: 1 addition & 1 deletion roles/k3s/server/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
group: root
mode: 0644

- name: Add service enviorment variables
- name: Add service environment variables
when: extra_service_envs is defined
ansible.builtin.lineinfile:
path: "{{ systemd_dir }}/k3s.service.env"
Expand Down
11 changes: 9 additions & 2 deletions roles/prereq/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@
- name: Populate service facts
ansible.builtin.service_facts:

- name: Assign api_port if not defined
when: api_port is undefined
ansible.builtin.set_fact:
api_port: 6443

- name: Allow UFW Exceptions
when:
- ansible_facts.services['ufw'] is defined
Expand Down Expand Up @@ -131,17 +136,19 @@

- name: Install Apparmor Parser [Suse]
when:
- apparmor_status.stdout == "Y"
- ansible_os_family == 'Suse'
- apparmor_status is defined
- apparmor_status.stdout == "Y"
ansible.builtin.package:
name: apparmor-parser
state: present

- name: Install Apparmor Parser [Debian]
when:
- apparmor_status.stdout == "Y"
- ansible_distribution == 'Debian'
- ansible_facts['distribution_major_version'] == "11"
- apparmor_status is defined
- apparmor_status.stdout == "Y"
ansible.builtin.package:
name: apparmor
state: present
Expand Down
Loading