Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove restund #740

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Remove restund #740

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5144e53
remove ansible-restund submodules
amitsagtani97 Oct 10, 2024
7828462
remove ansible.yml playbook
amitsagtani97 Oct 10, 2024
6d40c7c
remove restund references and code
amitsagtani97 Oct 10, 2024
47e8368
update docs.md
amitsagtani97 Oct 10, 2024
94af95a
update docs
amitsagtani97 Oct 10, 2024
3535c89
remove restund calling network config from docs
amitsagtani97 Oct 30, 2024
ba8fc9b
update docs
amitsagtani97 Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@
[submodule "ansible/roles-external/ansible-minio"]
path = ansible/roles-external/ansible-minio
url = https://github.com/wireapp/ansible-minio.git
[submodule "ansible/roles-external/ansible-restund"]
path = ansible/roles-external/ansible-restund
url = https://github.com/wireapp/ansible-restund.git
[submodule "ansible/roles-external/ansible-tinc"]
path = ansible/roles-external/ansible-tinc
url = https://github.com/wireapp/ansible-tinc.git
Expand Down
24 changes: 0 additions & 24 deletions ansible/inventory/offline/99-static
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@
# You could add more if capacity is needed
# kubenode4 ....

# restund1 ansible_host=XXXX
# restund2 ansible_host=XXXX

# cassandra1 ansible_host=XXXX
# cassandra2 ansible_host=XXXX
# cassandra3 ansible_host=XXXX
Expand Down Expand Up @@ -96,22 +93,6 @@
[rmq-cluster:vars]
# rabbitmq_network_interface = enp1s0

[restund:vars]
# Uncomment if your public IP is not on the default gateway
# restund_network_interface = enp1s0
# Uncomment and set to the true public IP if you are behind 1:1 NAT
# restund_peer_udp_advertise_addr = a.b.c.d
#
# Uncomment to create firewall exception for private networks
# restund_allowed_private_network_cidrs = a.b.c.d/24
# If you install restund together with other services on the same machine
# you need to restund_allowed_private_network_cidrs to allow these services
# to communicate on the private network. E.g. If your private network is 172.16.0.0/24
# restund_allowed_private_network_cidrs = '["172.16.0.0/24"]'

# Explicitely specify the restund user id to be "root" to override the default of "997"
restund_uid = root

# For the following groups, add all nodes defined above to the sections below.
# Define any additional variables that should be set for these nodes.

Expand Down Expand Up @@ -156,11 +137,6 @@ restund_uid = root
kube-master
kube-node

[restund]

# restund1
# restund2

# Add all cassandra nodes here
[cassandra]
# cassandra1
Expand Down
17 changes: 0 additions & 17 deletions ansible/inventory/prod/hosts.example.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,6 @@ minio01 ansible_host=X.X.X.X
minio02 ansible_host=X.X.X.X
minio03 ansible_host=X.X.X.X

# * 'ansible_host' is the IP to ssh into
# * set restund_network_interface to the interface that you want the process to bind to in the [all:vars] section
# * Optional: 'restund_peer_udp_advertise_addr' is the public IP to advertise for other turn servers if different than the ip on the 'restund_network_interface'
# If using 'restund_peer_udp_advertise_addr', make sure that UDP (!) traffic from any restund server (including itself)
# can reach that IP (for restund->restund communication)
restund01 ansible_host=X.X.X.X
restund02 ansible_host=X.X.X.X

# * 'ansible_host' is the IP to ssh into
# * 'ip' is the IP to bind to (if multiple network interfaces are in use)
Expand Down Expand Up @@ -65,14 +58,6 @@ prefix = "example-"
domain = "example.com"
deeplink_title = "example.com environment"

[restund]
restund01
restund02

[restund:vars]
## Set the network interface name for restund to bind to if you have more than one network interface
## If unset, defaults to the ansible_default_ipv4 (if defined) otherwise to eth0
# restund_network_interface = eth0

### KUBERNETES ###

Expand Down Expand Up @@ -141,8 +126,6 @@ is_aws_environment = False
## Set this to a name of a network interface (e.g. 'eth0'), on which you wish minio processes to talk to each other.
# minio_network_interface = "ens123"

### RESTUND section ###
# restund_network_interface = "..."

### KUBERNETES section (see kubespray documentation for details) ###

Expand Down
27 changes: 0 additions & 27 deletions ansible/restund.yml
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion ansible/roles-external/ansible-restund
Open in desktop
Submodule ansible-restund deleted from 8feeb7
11 changes: 0 additions & 11 deletions ansible/seed-offline-containerd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,3 @@
# sudo ctr -n=k8s.io images tag registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343 registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
# sudo ctr -n=k8s.io images tag registry.k8s.io/ingress-nginx/controller:v1.6.4 registry.k8s.io/ingress-nginx/controller:v1.6.4@sha256:15be4666c53052484dd2992efacf2f50ea77a78ae8aa21ccd91af6baaa7ea22f
####################################################################################################


- name: Download restund container
hosts: restund
tags: restund-containers
tasks:
- name: load restund container
shell: |
for container in $(curl -q {{ assethost_host }}/containers-other/index.txt);do
curl -q "{{ assethost_host }}/containers-other/$container" | ctr -n=k8s.io images import -
done
10 changes: 0 additions & 10 deletions ansible/seed-offline-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,3 @@
for container in $(curl -q {{ assethost_host }}/containers-helm/index.txt);do
curl -q "{{ assethost_host }}/containers-helm/$container" | docker load
done

- name: Download restund container
hosts: restund
tags: restund-containers
tasks:
- name: load containers
shell: |
for container in $(curl -q {{ assethost_host }}/containers-other/index.txt);do
curl -q "{{ assethost_host }}/containers-other/$container" | docker load
done
9 changes: 1 addition & 8 deletions ansible/setup-offline-sources.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,6 @@
tags:
- containers-helm
- containers
- name: Copy other containers
unarchive:
src: ../containers-other.tar
dest: /opt/assets
tags:
- containers-other
- containers
- copy:
src: files/serve-assets.service
dest: /etc/systemd/system/serve-assets.service
Expand All @@ -47,7 +40,7 @@
daemon-reload: yes

- name: Set up offline repositories and remove online ones
hosts: k8s-cluster:etcd:restund:cassandra:elasticsearch:minio:rmq-cluster
hosts: k8s-cluster:etcd:cassandra:elasticsearch:minio:rmq-cluster
tasks:
- name: Bail if GPG is not installed or installable.
apt:
Expand Down
4 changes: 2 additions & 2 deletions bin/autodeploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,8 +306,8 @@ ufw allow 25672/tcp;

cp values/wire-server/prod-values.example.yaml values/wire-server/values.yaml
sed -i "s/example.com/$TARGET_SYSTEM/g" values/wire-server/values.yaml
sed -i "s/# - \"turn:<IP of restund1>:80\"/- \"turn:$HOST_IP:3478\"/g" values/wire-server/values.yaml
sed -i "s/# - \"turn:<IP of restund1>:80?transport=tcp\"/- \"turn:$HOST_IP:3478?transport=tcp\"/g" values/wire-server/values.yaml
sed -i "s/# - \"turn:<IP of coturn1>:3478\"/- \"turn:$HOST_IP:3478\"/g" values/wire-server/values.yaml
sed -i "s/# - \"turn:<IP of coturn1>:3478?transport=tcp\"/- \"turn:$HOST_IP:3478?transport=tcp\"/g" values/wire-server/values.yaml

d helm install wire-server ./charts/wire-server --timeout=15m0s --values ./values/wire-server/values.yaml --values ./values/wire-server/secrets.yaml

Expand Down
4 changes: 0 additions & 4 deletions bin/offline-cluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,6 @@ ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/setup-offline-sources.yml
# are part of the offline bundle
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/kubernetes.yml --tags bastion,bootstrap-os,preinstall,container-engine

# Install docker on the restund nodes
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/restund.yml --tags docker

# With ctr being installed on all nodes that need it, seed all container images:
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/seed-offline-containerd.yml

Expand All @@ -53,7 +50,6 @@ ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/kubernetes.yml --skip-tags boot
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/cassandra.yml
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/elasticsearch.yml
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/minio.yml
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/restund.yml

# create helm values that tell our helm charts what the IP addresses of cassandra, elasticsearch and minio are:
ansible-playbook -i $INVENTORY_FILE $ANSIBLE_DIR/helm_external.yml --skip-tags=rabbitmq-external
1 change: 0 additions & 1 deletion bin/offline-secrets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ fi
if [[ ! -f $ANSIBLE_DIR/inventory/offline/group_vars/all/secrets.yaml ]]; then
echo "Writing $ANSIBLE_DIR/inventory/offline/group_vars/all/secrets.yaml"
cat << EOT > $ANSIBLE_DIR/inventory/offline/group_vars/all/secrets.yaml
restund_zrest_secret: "$zrest"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need a zrest secret for coturn?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are directly setting it here -

brig:
  secrets:
    smtpPassword: dummyPassword
    zAuth:
      publicKeys: "$zauth_public"
      privateKeys: "$zauth_private"
    turn:
      secret: "$zrest"

minio_access_key: "$minio_access_key"
minio_secret_key: "$minio_secret_key"
EOT
Expand Down
8 changes: 1 addition & 7 deletions offline/ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,6 @@ list-system-containers | create-container-dump containers-system
tar cf containers-system.tar containers-system
[[ "$INCREMENTAL" -eq 0 ]] && rm -r containers-system

# Used for ansible-restund role
echo "quay.io/wire/restund:v0.6.0-rc.2" | create-container-dump containers-other
tar cf containers-other.tar containers-other
[[ "$INCREMENTAL" -eq 0 ]] && rm -r containers-other

legacy_chart_release() {
# Note: if you want to ship from the develop branch, replace 'repo' url below
# repo=https://s3-eu-west-1.amazonaws.com/public.wire.com/charts-develop
Expand Down Expand Up @@ -143,7 +138,6 @@ legacy_chart_release() {

calling_charts=(
sftd
restund
coturn
)
for chartName in "${calling_charts[@]}"; do
Expand Down Expand Up @@ -245,6 +239,6 @@ tar cf containers-helm.tar containers-helm

echo "docker_ubuntu_repo_repokey: '${fingerprint}'" > ansible/inventory/offline/group_vars/all/key.yml

tar czf assets.tgz debs-jammy.tar binaries.tar containers-adminhost containers-helm.tar containers-other.tar containers-system.tar ansible charts values bin
tar czf assets.tgz debs-jammy.tar binaries.tar containers-adminhost containers-helm.tar containers-system.tar ansible charts values bin

echo "Done"
90 changes: 37 additions & 53 deletions offline/coturn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This document explains how to install Coturn on a newly deployed Wire-Server ins
This presumes you already have:

* Followed the [single Hetzner machine installation](single_hetzner_machine_installation.md) guide or otherwise have a machine ready to accept a Wire-Server deployment.
* Have followed the [Wire-Server installation](docs_ubuntu_22.04.md) guide and have Wire-Server deployed and working (with Restund as the TURN server, which is currently the default, and will be replaced by Coturn as part of this process).
* Have followed the [Wire-Server installation](docs_ubuntu_22.04.md) guide and have Wire-Server deployed and working.

## Plan.

Expand All @@ -22,8 +22,7 @@ To setup Coturn, we will:
* Configure the Coturn labels to select on which machine(s) it will run.
* Configure the SFT labels for Coturn and SFT to share a port range.
* Configure the port redirection in Nftables.
* Change the Wire-Server configuration to use Coturn instead of Restund.
* Disable Restund.
* Change the Wire-Server configuration to use Coturn.
* Install Coturn using Helm.
* Verify that Coturn is working.

Expand Down Expand Up @@ -239,7 +238,7 @@ Note: This section is only relevant if you are running Wire-Server/Coturn/SFT be

We must configure the port redirection in Nftables to allow traffic to reach Coturn and SFT.

Calling and TURN services (Coturn, Restund, SFT) require being reachable on a range of ports used to transmit the calling data.
Calling and TURN services (Coturn, SFT) require being reachable on a range of ports used to transmit the calling data.

Both SFT and Coturn both want to use the same port range, therefore predicting which node is using which port range ahead of time requires dividing/configuring port ranges in advance.

Expand Down Expand Up @@ -335,9 +334,9 @@ sudo systemctl restart nftables

```

## Change the Wire-Server configuration to use Coturn instead of Restund.
## Change the Wire-Server configuration to use Coturn.

We must change the Wire-Server configuration to use Coturn instead of Restund.
We must change the Wire-Server configuration to use Coturn.

First, we must locate what the "external" IP address of the machine is.

Expand Down Expand Up @@ -368,26 +367,10 @@ You will find a section that looks like this (default):
turnStatic:
v1: []
v2:
# - "turn:<IP of restund1>:80"
# - "turn:<IP of restund2:80"
# - "turn:<IP of restund1>:80?transport=tcp"
# - "turn:<IP of restund2>:80?transport=tcp"
# - "turns:<IP of restund1>:443?transport=tcp"
# - "turns:<IP of restund2>:443?transport=tcp"

```

Or if you have already configured Restund, something like this:

```yaml

turnStatic:
v1: []
v2:
- "turn:<IP of restund1>:80"
- "turn:<IP of restund2>:80"
- "turn:<IP of restund1>:80?transport=tcp"
- "turn:<IP of restund2>:80?transport=tcp"
# - "turn:<IP of coturn1>:3478"
# - "turn:<IP of coturn2>:3478"
# - "turn:<IP of coturn1>:3478?transport=tcp"
# - "turn:<IP of coturn2>:3478?transport=tcp"

```

Expand All @@ -409,35 +392,10 @@ d helm upgrade --install wire-server ./charts/wire-server --timeout=15m0s --valu

```

## Disable Restund.

As we are no longer using Restund, we should now disable it entirely.

We do this by editing the `hosts.ini` file:

Edit `ansible/inventory/offline/hosts.ini`, and comment out the restund section by adding `#` at the beginning of each line :

```
[restund]
# ansnode1
# ansnode2
```

Then connect to each ansnode and do:

```bash
sudo service restund stop
```

And check it is stopped with:

```bash
sudo service restund status
```

## Install Coturn with Helm.

We have now configured our Coturn `value` and `secret` files, configured `wire-server` to use Coturn, and disabled Restund.
We have now configured our Coturn `value` and `secret` files, configured `wire-server` to use Coturn.

It is time to actually deploy Coturn.

Expand Down Expand Up @@ -515,4 +473,30 @@ These are the additional steps to ensure a smooth transition:
2. Change the `turnStatic` call configuration in the `values/wire-server/values.yaml` file to use the Coturn IPs instead of the Restund IPs.
3. Re-deploy the Wire-Server chart to apply the new configuration.
4. Wait at least 24 hours for all clients to retrieve the new configuration.
5. Once you are sure all clients have migrated to Coturn, you can disable Restund as described in this guide.
5. Once you are sure all clients have migrated to Coturn, you can disable Restund as described in this guide below.

## Disable Restund.

As we are no longer using Restund, we should now disable it entirely.

We do this by editing the `hosts.ini` file:

Edit `ansible/inventory/offline/hosts.ini`, and comment out the restund section by adding `#` at the beginning of each line :

```
[restund]
# ansnode1
# ansnode2
```

Then connect to each ansnode and do:

```bash
sudo service restund stop
```

And check it is stopped with:

```bash
sudo service restund status
```
Loading
Loading