Skip to content

Commit

Permalink
working on upgrading keycloak
Browse files Browse the repository at this point in the history
  • Loading branch information
chrisdjscott committed Sep 4, 2023
1 parent 6f7009a commit 8cbd152
Show file tree
Hide file tree
Showing 6 changed files with 164 additions and 174 deletions.
273 changes: 134 additions & 139 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,139 +1,134 @@
# flexi training environment

This repo sets up a training environment using Open OnDemand within the FlexiHPC platform using Terraform and Ansible.

[Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) and
[Ansible](https://www.ansible.com/) need to be installed on your system to run this.

## Configure terraform

First copy the config file:

```
cp terraform/terraform.tfvars.example terraform/terraform.tfvars
```

Inside the `terraform/terraform.tfvars` file is some user configuration required.

```
key_pair = "FLEXIHPC_KEYPAIR_NAME"
key_file = "/path/to/flexi/private_key"
```

where

- `FLEXIHPC_KEYPAIR_NAME` is your `Key Pair` name that is setup in FlexiHPC
- `FLEXIHPC_KEYFILE` is the local location for your ssh key

Set environment variables for authenticating with OpenStack and object store (for the state file), e.g.

```
export OS_USERNAME="FLEXIHPC_USER"
export OS_PASSWORD="FLEXIHPC_PASSWORD"
export AWS_ACCESS_KEY_ID="EC2_User_Access_Token"
export AWS_SECRET_KEY="EC2_User_Secret_Token"
```

where

- `FLEXIHPC_USER` is set to your username for the FlexiHPC Platform
- `FLEXIHPC_PASSWORD` is set to your password for the FlexiHPC Platform
- `EC2_User_Access_Token` is set to your EC2 access token
- `EC2_User_Secret_Token` is set to your EC2 secret token

If you don't have any EC2 credentials then use the following CLI command to generate new ones:

```
openstack ec2 credentials create
```

## Configure ansible

Install dependencies:

```
ansible-galaxy install -r requirements.yml
```

Copy example secrets file and edit:

```
cp vars/secrets.yml.example vars/secrets.yml
```

Copy ondemand config:

```
cp vars/ondemand-config.yml.example vars/ondemand-config.yml
```

and edit, in particular set `oidc_settings.OIDCCryptoPassphrase` with a randomly
generated password, e.g. the output of `openssl rand -hex 40`.

## Note about terraform workspaces

The terraform workspace must have already been created before running the below command.
This will always be the case for the "default" workspace but if you want to create another
workspace you should do it manually by running:

```
cd terraform
terraform init
terraform workspace select -or-create=true <workspace_name>
```

Then continuing with the `ansible-playbook` command below, substituting in the name
of your workspace instead of "default".

## Destroy environment

To destroy a previously created environment run:

```
ansible-playbook setup-infra.yml -e operation=destroy -e terraform_workspace=default
```

## Create the environment

First, create the terraform resources:

```
ansible-playbook setup-infra.yml -e operation=create -e terraform_workspace=default
```

Then configure the environment:

```
export ANSIBLE_HOST_KEY_CHECKING=False
ansible-playbook -i host.ini -u ubuntu --key-file ~/.ssh/flexi-private-key setup-training-environment.yml
```

By default 2 users will be created, `training1` and `training2`. Passwords for these users will be
stored in the *users* sub-directory:

```
$ ls users/
password_training1.txt password_training2.txt
```

More users can be added by overriding the `num_users_create` variable, e.g.

```
ansible-playbook -i host.ini -u ubuntu --key-file ~/.ssh/flexi-private-key \
--extra-vars "num_users_create=5" setup-training-environment.yml
```

You will need to modify your hosts file with the IP addresses from *host.ini*, on Linux this file is
*/etc/hosts*, on Windows it is XXXX.

```
# /etc/hosts snippet
# this one should be the IP for webnode from host.ini
1.2.3.4 ood.flexi.nesi
# this one should be the IP for servicesnode from host.ini
5.6.7.8 ood-idp.flexi.nesi
```

Connect via [https://ood.flexi.nesi](https://ood.flexi.nesi).
# flexi training environment

This repo sets up a training environment using Open OnDemand within the FlexiHPC platform using Terraform and Ansible.

[Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) and
[Ansible](https://www.ansible.com/) need to be installed on your system to run this.

## Configure terraform

Set some variables via environment variables:

```
export TF_VAR_key_pair="FLEXIHPC_KEYPAIR_NAME"
export TF_VAR_key_file="/path/to/flexi/private_key"
export TF_VAR_vm_user="ubuntu"
```

where

- `FLEXIHPC_KEYPAIR_NAME` is your `Key Pair` name that is setup in FlexiHPC
- `/path/to/flexi/private_key` is the local path to the private key

Set environment variables for authenticating with OpenStack and object store (for the state file), e.g.

```
export OS_USERNAME="FLEXIHPC_USER"
export OS_PASSWORD="FLEXIHPC_PASSWORD"
export AWS_ACCESS_KEY_ID="EC2_User_Access_Token"
export AWS_SECRET_KEY="EC2_User_Secret_Token"
```

where

- `FLEXIHPC_USER` is set to your username for the FlexiHPC Platform
- `FLEXIHPC_PASSWORD` is set to your password for the FlexiHPC Platform
- `EC2_User_Access_Token` is set to your EC2 access token
- `EC2_User_Secret_Token` is set to your EC2 secret token

If you don't have any EC2 credentials then use the following CLI command to generate new ones:

```
openstack ec2 credentials create
```

## Configure ansible

Install dependencies:

```
ansible-galaxy install -r requirements.yml
```

Copy example secrets file and edit:

```
cp vars/secrets.yml.example vars/secrets.yml
```

Copy ondemand config:

```
cp vars/ondemand-config.yml.example vars/ondemand-config.yml
```

and edit, in particular set `oidc_settings.OIDCCryptoPassphrase` with a randomly
generated password, e.g. the output of `openssl rand -hex 40`.

## Note about terraform workspaces

The terraform workspace must have already been created before running the below command.
This will always be the case for the "default" workspace but if you want to create another
workspace you should do it manually by running:

```
cd terraform
terraform init
terraform workspace select -or-create=true <workspace_name>
```

Then continuing with the `ansible-playbook` command below, substituting in the name
of your workspace instead of "default".

## Destroy environment

To destroy a previously created environment run:

```
ansible-playbook setup-infra.yml -e operation=destroy -e terraform_workspace=default
```

## Create the environment

First, create the terraform resources:

```
ansible-playbook setup-infra.yml -e operation=create -e terraform_workspace=default
```

Then configure the environment:

```
export ANSIBLE_HOST_KEY_CHECKING=False
ansible-playbook -i host.ini -u ubuntu --key-file ~/.ssh/flexi-private-key setup-training-environment.yml
```

By default 2 users will be created, `training1` and `training2`. Passwords for these users will be
stored in the *users* sub-directory:

```
$ ls users/
password_training1.txt password_training2.txt
```

More users can be added by overriding the `num_users_create` variable, e.g.

```
ansible-playbook -i host.ini -u ubuntu --key-file ~/.ssh/flexi-private-key \
--extra-vars "num_users_create=5" setup-training-environment.yml
```

You will need to modify your hosts file with the IP addresses from *host.ini*, on Linux this file is
*/etc/hosts*, on Windows it is XXXX.

```
# /etc/hosts snippet
# this one should be the IP for webnode from host.ini
1.2.3.4 ood.flexi.nesi
# this one should be the IP for servicesnode from host.ini
5.6.7.8 ood-idp.flexi.nesi
```

Connect via [https://ood.flexi.nesi](https://ood.flexi.nesi).
2 changes: 1 addition & 1 deletion deployment.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ case $1 in
;;
"create")
ansible-playbook setup-infra.yml -e operation=create -e terraform_workspace=${2:-default}
ansible-playbook -i host.ini setup-training-environment.yml -u ${var.vm_user} --key-file '${var.key_file}'
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i host.ini setup-training-environment.yml -u ${TF_VAR_vm_user} --key-file '${TF_VAR_key_file}'
;;
esac
3 changes: 2 additions & 1 deletion roles/keycloak/files/keycloak.service
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ After=network.target
Type=idle
User=keycloak
Group=keycloak
ExecStart=/opt/keycloak-9.0.0/bin/standalone.sh -b 0.0.0.0
ExecStart=/opt/keycloak-22.0.1/bin/kc.sh start --hostname-strict=false --http-enabled=true
EnvironmentFile=/etc/default/keycloak
TimeoutStartSec=600
TimeoutStopSec=600

Expand Down
54 changes: 23 additions & 31 deletions roles/keycloak/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
- name: Install dependencies for keycloak
apt:
name:
- openjdk-8-jdk
- openjdk-17-jdk
state: present
- name: Download keycloak archive
ansible.builtin.get_url:
url: https://downloads.jboss.org/keycloak/9.0.0/keycloak-9.0.0.tar.gz
dest: /opt/keycloak-9.0.0.tar.gz
checksum: "sha256:3b2f050b166b07dfe54aff94b99649f290bd2ad834d844a71e0c1b1ea9add98c"
url: https://github.com/keycloak/keycloak/releases/download/22.0.1/keycloak-22.0.1.tar.gz
dest: /opt/keycloak-22.0.1.tar.gz
checksum: "sha1:eea6076b99a78f88104e3a08618f4422d0ff16e2"
- name: Extract keycloak archive
ansible.builtin.unarchive:
src: /opt/keycloak-9.0.0.tar.gz
src: /opt/keycloak-22.0.1.tar.gz
dest: /opt
creates: /opt/keycloak-9.0.0
creates: /opt/keycloak-22.0.1
remote_src: yes
- name: Create keycloak group
ansible.builtin.group:
Expand All @@ -31,31 +31,23 @@
state: present
- name: Set ownership of keycloak dir
file:
path: /opt/keycloak-9.0.0
path: /opt/keycloak-22.0.1
owner: keycloak
group: keycloak
recurse: true
- name: Restrict access to standalone
file:
path: /opt/keycloak-9.0.0/standalone
mode: 0700
- name: Add keycloak admin user
command: "sudo -u keycloak /opt/keycloak-9.0.0/bin/add-user-keycloak.sh --user admin --password {{ keycloak_admin_password }} --realm master"
args:
creates: /opt/keycloak-9.0.0/.admin_user_added
register: create_keycloak_admin
- file:
path: /opt/keycloak-9.0.0/.admin_user_added
state: touch
when: create_keycloak_admin is changed
- name: Copy proxy config file
copy:
src: keycloak-proxy-config.cli
dest: /opt/keycloak-9.0.0/.keycloak-proxy-config.cli
register: keycloak_proxy_config
- name: Allow proxying in keycloak
shell: "sudo -u keycloak /opt/keycloak-9.0.0/bin/jboss-cli.sh --file=/opt/keycloak-9.0.0/.keycloak-proxy-config.cli"
when: keycloak_proxy_config.changed
- name: Install keycloak defaults file
template:
src: keycloak_defaults.j2
dest: /etc/default/keycloak
owner: root
group: root
mode: '600'
- name: Keycloak proxy configuration
lineinfile:
path: /opt/keycloak-22.0.1/conf/keycloak.conf
line: proxy=edge
owner: keycloak
group: keycloak
- name: Copy keycloak system service file
copy:
src: keycloak.service
Expand Down Expand Up @@ -133,7 +125,7 @@
- name: Create ondemand Keycloak realm
community.general.keycloak_realm:
auth_client_id: admin-cli
auth_keycloak_url: http://127.0.0.1:8080/auth
auth_keycloak_url: http://127.0.0.1:8080
auth_realm: master
auth_username: admin
auth_password: "{{ keycloak_admin_password }}"
Expand All @@ -146,7 +138,7 @@
- name: Create LDAP user federation
community.general.keycloak_user_federation:
auth_client_id: admin-cli
auth_keycloak_url: http://127.0.0.1:8080/auth
auth_keycloak_url: http://127.0.0.1:8080
auth_realm: master
auth_username: admin
auth_password: "{{ keycloak_admin_password }}"
Expand All @@ -170,7 +162,7 @@
- name: Add ondemand as a client
community.general.keycloak_client:
auth_client_id: admin-cli
auth_keycloak_url: http://127.0.0.1:8080/auth
auth_keycloak_url: http://127.0.0.1:8080
auth_realm: master
auth_username: admin
auth_password: "{{ keycloak_admin_password }}"
Expand Down
2 changes: 2 additions & 0 deletions roles/keycloak/templates/keycloak_defaults.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
KEYCLOAK_ADMIN="admin"
KEYCLOAK_ADMIN_PASSWORD="{{ keycloak_admin_password }}"
Loading

0 comments on commit 8cbd152

Please sign in to comment.