Skip to content

Commit

Permalink
Add automated scripts to install M2K and MTA
Browse files Browse the repository at this point in the history
Signed-off-by: gabriel-farache <[email protected]>
  • Loading branch information
gabriel-farache authored and masayag committed Sep 24, 2024
1 parent abb2e15 commit e1b4144
Show file tree
Hide file tree
Showing 4 changed files with 110 additions and 18 deletions.
30 changes: 21 additions & 9 deletions charts/move2kube/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,19 @@ The list of the overridable values can be found in our [git repository](https://

You can also view the [Move2Kube README on GitHub](https://github.com/parodos-dev/serverless-workflows-config/blob/main/charts/move2kube/README.md)

## Prerequisites
## Automated installation
Run the [installation script](install_m2k.sh):
```console
PRIV_ID_RSA_PATH=${HOME}/.ssh/id_rsa PUB_ID_RSA_PATH=${HOME}/.ssh/id_rsa.pub ./install_m2k.sh
```
You can override the helm repo to use by setting `M2K_HELM_REPO`. By default `orchestrator-workflows/move2kube` is used and the helm repository `orchestrator-workflows` is installed from `https://parodos.dev/serverless-workflows-config`

To use the local file, set `M2K_HELM_REPO` to `.`:
```console
M2K_HELM_REPO=. PRIV_ID_RSA_PATH=${HOME}/.ssh/id_rsa PUB_ID_RSA_PATH=${HOME}/.ssh/id_rsa.pub ./install_m2k.sh
```
## Manual installation
### Prerequisites
Set `TARGET_NS` to the target namespace:
```console
TARGET_NS=sonataflow-infra
Expand All @@ -16,7 +28,7 @@ Set `M2K_INSTANCE_NS` to the namespace hosting the move2kube instance:
M2K_INSTANCE_NS=move2kube
```

### For Knative
#### For Knative
We need to use `initContainers` and `securityContext` in our Knative services to allow SSH key exchange in move2kube workflow, we have to tell Knative to enable that feature:
```bash
oc patch configmap/config-features \
Expand All @@ -25,7 +37,7 @@ We need to use `initContainers` and `securityContext` in our Knative services to
-p '{"data":{"kubernetes.podspec-init-containers": "enabled", "kubernetes.podspec-securitycontext": "enabled"}}'

```
### For move2kube instance
#### For move2kube instance
Also, `move2kube` instance runs as root so we need to allow the `default` service account to use `runAsUser`:
```console
oc -n ${TARGET_NS} adm policy add-scc-to-user anyuid -z default
Expand All @@ -45,17 +57,17 @@ Note that those ssh keys need to be added to your git repository as well. For bi

View the [Move2Kube README](https://github.com/parodos-dev/serverless-workflows-config/blob/main/charts/move2kube/README.md) on GitHub.

## Installation
### Installation

Run
```console
helm repo add orchestrator-workflows https://parodos.dev/serverless-workflows-config
helm install move2kube orchestrator-workflows/move2kube -n ${TARGET_NS} --set instance.namespace=${M2K_INSTANCE_NS}
```

## Post-installation
### Post-installation

### Configure move2kube instance
#### Configure move2kube instance
To create SSH Keys secret for move2kube instance and connfigure SCC, run:
```console
oc -n ${M2K_INSTANCE_NS} adm policy add-scc-to-user anyuid -z default
Expand All @@ -67,7 +79,7 @@ Then force the pod to be recreated:
oc -n ${M2K_INSTANCE_NS} scale deploy move2kube --replicas=0 && oc -n ${M2K_INSTANCE_NS} scale deploy move2kube --replicas=1
```

### Set `M2K_ROUTE` and `BROKER_URL` for the Knative service
#### Set `M2K_ROUTE` and `BROKER_URL` for the Knative service
As the Knative service cannot be updated, we need to delete if first and then re-create it with the helm command.

Run the following command or follow the steps prompted at the end of the workflow installation to apply it to the `move2kubeURL` parameter:
Expand All @@ -77,7 +89,7 @@ oc -n ${TARGET_NS} delete ksvc m2k-save-transformation-func &&
helm upgrade move2kube orchestrator-workflows/move2kube -n ${TARGET_NS} --set workflow.move2kubeURL=https://${M2K_ROUTE}
```

### Edit the `${WORKFLOW_NAME}-creds` Secret
#### Edit the `${WORKFLOW_NAME}-creds` Secret
The token for sending notifications from the m2k workflow to RHDH notifications service needs to be provided to the workflow.

Edit the secret `${WORKFLOW_NAME}-creds` and set the value of `NOTIFICATIONS_BEARER_TOKEN`:
Expand All @@ -93,7 +105,7 @@ Note that the modification of the secret does not currently restart the pod, the

Note that when you run the `helm upgrade` command, the values of the secret are reseted.

### Set `M2K_ROUTE` and `K_SINK` for the Sonataflow CR
#### Set `M2K_ROUTE` and `K_SINK` for the Sonataflow CR

Run the following to set `K_SINK` and `MOVE2KUBE_URL` environment variable in the workflow:
```console
Expand Down
42 changes: 42 additions & 0 deletions charts/move2kube/install_m2k.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash

if [[ -z "${PRIV_ID_RSA_PATH}" ]]; then
echo 'PRIV_ID_RSA_PATH env variable must be set to the path of the private id_rsa file to use. I.e: ${HOME}/.ssh/id_rsa'
exit -1
fi

if [[ -z "${PUB_ID_RSA_PATH}" ]]; then
echo 'PUB_ID_RSA_PATH env variable must be set to the path of the public id_rsa file to use. I.e: ${HOME}/.ssh/id_rsa'
exit -1
fi

if [[ -z "${M2K_HELM_REPO}" ]]; then
M2K_HELM_REPO=orchestrator-workflows/move2kube
echo "M2K_HELM_REPO not set, using default helm m2k helm repository ${M2K_HELM_REPO}"
helm repo add orchestrator-workflows https://parodos.dev/serverless-workflows-config
fi

TARGET_NS=sonataflow-infra
M2K_INSTANCE_NS=move2kube
WORKFLOW_NAME=m2k
oc patch configmap/config-features \
-n knative-serving \
--type merge \
-p '{"data":{"kubernetes.podspec-init-containers": "enabled", "kubernetes.podspec-securitycontext": "enabled"}}'
oc -n ${TARGET_NS} create secret generic sshkeys --from-file=id_rsa=${PRIV_ID_RSA_PATH} --from-file=id_rsa.pub=${PUB_ID_RSA_PATH}
helm install move2kube ${M2K_HELM_REPO} -n ${TARGET_NS} --set instance.namespace=${M2K_INSTANCE_NS}
if [ $? -ne 0 ]; then
echo "move2kube chart already installed, run helm delete move2kube -n ${TARGET_NS} to remove it"
exit -1
fi
oc -n ${TARGET_NS} adm policy add-scc-to-user $(oc -n ${TARGET_NS} get deployments m2k-save-transformation-func-v1-deployment -oyaml | oc adm policy scc-subject-review --no-headers -o yaml --filename - | yq -r .status.allowedBy.name) -z default
oc -n ${M2K_INSTANCE_NS} adm policy add-scc-to-user $(oc -n ${M2K_INSTANCE_NS} get deployments move2kube -oyaml | oc adm policy scc-subject-review --no-headers -o yaml --filename - | yq -r .status.allowedBy.name) -z default
oc -n ${M2K_INSTANCE_NS} create secret generic sshkeys --from-file=id_rsa=${PRIV_ID_RSA_PATH} --from-file=id_rsa.pub=${PUB_ID_RSA_PATH}
oc -n ${M2K_INSTANCE_NS} scale deploy move2kube --replicas=0 && oc -n ${M2K_INSTANCE_NS} scale deploy move2kube --replicas=1
M2K_ROUTE=$(oc -n ${M2K_INSTANCE_NS} get routes move2kube-route -o yaml | yq -r .spec.host)
oc -n ${TARGET_NS} delete ksvc m2k-save-transformation-func
helm upgrade move2kube ${M2K_HELM_REPO} -n ${TARGET_NS} --set workflow.move2kubeURL=https://${M2K_ROUTE}

oc -n ${TARGET_NS} patch secret "${WORKFLOW_NAME}-creds" --type merge -p '{"data": { "NOTIFICATIONS_BEARER_TOKEN": "'$(oc get secrets -n rhdh-operator backstage-backend-auth-secret -o go-template='{{ .data.BACKEND_SECRET }}')'"}}'
BROKER_URL=$(oc -n ${TARGET_NS} get broker -o yaml | yq -r .items[0].status.address.url)
oc -n ${TARGET_NS} patch sonataflow m2k --type merge -p '{"spec": { "podTemplate": { "container": { "env": [{"name": "K_SINK", "value": "'${BROKER_URL}'"}, {"name": "MOVE2KUBE_URL", "value": "https://'${M2K_ROUTE}'"}]}}}}'
38 changes: 29 additions & 9 deletions charts/mta-v7.x/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,44 @@ At the end of a successful assessment workflow, a link to the report will be ava
## Configuration
View the [MTA v7.X README on GitHub](https://github.com/parodos-dev/serverless-workflows-config/blob/main/charts/mta-v7.x/README.md)

## Installation
## Automated installation
Run the [installation script](install-mta-v7.sh):
```console
./install-mta-v7.sh
```
You can override the helm repo to use by setting `MTA_HELM_REPO`. By default `orchestrator-workflows/mta-v7` is used and the helm repository `orchestrator-workflows` is installed from `https://parodos.dev/serverless-workflows-config`

To use the local file, set `MTA_HELM_REPO` to `.`:
```console
MTA_HELM_REPO=. ./install-mta-v7.sh
```
## Manual installation

### Prerequisites
Set `TARGET_NS` to the target namespace:
```console
TARGET_NS=sonataflow-infra
```


### Installation
- Run
```console
helm repo add orchestrator-workflows https://parodos.dev/serverless-workflows-config
helm install mta orchestrator-workflows/mta-v7 -n sonataflow-infra
helm install mta orchestrator-workflows/mta-v7 -n ${TARGET_NS}
```

## Post-installation
### Set up the MTA instance with a Jira Connection
### Post-installation
#### Set up the MTA instance with a Jira Connection
Define a Jira instance in MTA and establish a connection to it, by following the [Creating and configuring a Jira connection](https://access.redhat.com/documentation/en-us/migration_toolkit_for_applications/7.0/html/user_interface_guide/creating-configuring-jira-connection#doc-wrapper) guide.

### Edit the `${WORKFLOW_NAME}-creds` Secret
#### Edit the `${WORKFLOW_NAME}-creds` Secret
The token for sending notifications from the MTA-v7.x workflow to RHDH notifications service needs to be provided to the workflow.

Edit the secret `${WORKFLOW_NAME}-creds` and set the value of `NOTIFICATIONS_BEARER_TOKEN`:
```
WORKFLOW_NAME=mta-analysis-v7
oc -n sonataflow-infra patch secret "${WORKFLOW_NAME}-creds" --type merge -p '{"data": { "NOTIFICATIONS_BEARER_TOKEN": "'$(oc get secrets -n rhdh-operator backstage-backend-auth-secret -o go-template='{{ .data.BACKEND_SECRET }}')'"}}'
oc -n ${TARGET_NS} patch secret "${WORKFLOW_NAME}-creds" --type merge -p '{"data": { "NOTIFICATIONS_BEARER_TOKEN": "'$(oc get secrets -n rhdh-operator backstage-backend-auth-secret -o go-template='{{ .data.BACKEND_SECRET }}')'"}}'
```

This secret is used in the `sonataflow` CR to inject the token as an environment variable that will be used by the workflow.
Expand All @@ -35,7 +55,7 @@ Note that the modification of the secret does not currently restart the pod, the

Note that if you run the `helm upgrade` command, the values of the secret are reseted.

### Edit the `mta-analysis-v7` Sontaflow CR:
#### Edit the `mta-analysis-v7` Sontaflow CR:

There is one variable required to be set in the `mta-analysis-v7-props` ConfigMap:
* **mta.url** - The URL to the MTA application
Expand All @@ -51,12 +71,12 @@ MTA_ROUTE=$(oc -n openshift-mta get route mta -o yaml | yq -r .spec.host)
oc -n ${TARGET_NS} patch sonataflow mta-analysis-v7 --type merge -p '{"spec": { "podTemplate": { "container": { "env": [{"name": "MTA_URL", "value": "https://'${MTA_ROUTE}'"}]}}}}'
```

### Validate instalation
## Validate instalation

- Verify MTA resources and workflow are ready:
```console
sleep 120s # to wait until the MTA operator has created all requested resources
oc wait --for=jsonpath='{.status.phase}=Succeeded' -n openshift-mta csv/mta-operator.v7.0.3 --timeout=2m
oc wait --for=condition=Ready=true pods -l "app.kubernetes.io/name=mta-ui" -n openshift-mta --timeout=2m
oc wait -n sonataflow-infra sonataflow/mta-analysis-v7 --for=condition=Running --timeout=2m
oc wait -n ${TARGET_NS} sonataflow/mta-analysis-v7 --for=condition=Running --timeout=2m
```
18 changes: 18 additions & 0 deletions charts/mta-v7.x/install-mta-v7.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
TARGET_NS=sonataflow-infra

if [[ -z "${MTA_HELM_REPO}" ]]; then
MTA_HELM_REPO=orchestrator-workflows/mta-v7
echo "MTA_HELM_REPO not set, using default helm mta v7 helm repository ${MTA_HELM_REPO}"
helm repo add orchestrator-workflows https://parodos.dev/serverless-workflows-config
fi

helm install mta ${MTA_HELM_REPO} -n ${TARGET_NS}
WORKFLOW_NAME=mta-analysis-v7
oc -n ${TARGET_NS} patch secret "${WORKFLOW_NAME}-creds" --type merge -p '{"data": { "NOTIFICATIONS_BEARER_TOKEN": "'$(oc get secrets -n rhdh-operator backstage-backend-auth-secret -o go-template='{{ .data.BACKEND_SECRET }}')'"}}'
while [[ $retry_count -lt 5 ]]; do
oc -n openshift-mta get route mta && break || sleep 60
retry_count=$((retry_count + 1))
done
MTA_ROUTE=$(oc -n openshift-mta get route mta -o yaml | yq -r .spec.host)
oc -n ${TARGET_NS} patch sonataflow mta-analysis-v7 --type merge -p '{"spec": { "podTemplate": { "container": { "env": [{"name": "MTA_URL", "value": "https://'${MTA_ROUTE}'"}]}}}}'

0 comments on commit e1b4144

Please sign in to comment.