diff --git a/content/explanation/console.md b/content/explanation/console.md index b20c3007..e3d7f50a 100644 --- a/content/explanation/console.md +++ b/content/explanation/console.md @@ -11,20 +11,18 @@ The dashboard serves as a centralized monitoring hub, offering insights into the By default, MTO Console will be disabled and has to be enabled by setting the below configuration in IntegrationConfig. ```yaml -provision: +components: console: true ingress: + ingressClassName: console: host: tenant-operator-console. - ingressClassName: tlsSecretName: gateway: host: tenant-operator-gateway. - ingressClassName: tlsSecretName: keycloak: host: tenant-operator-keycloak. - ingressClassName: tlsSecretName: showback: true trustedRootCert: diff --git a/content/faq.md b/content/faq.md index f4f7ffd8..c96961a8 100644 --- a/content/faq.md +++ b/content/faq.md @@ -8,7 +8,7 @@ unable to find annotation openshift.io/sa.scc.uid-range ``` -**Answer.** OpenShift recently updated its process of handling SCC, and it's now managed by annotations like `openshift.io/sa.scc.uid-range` on the namespaces. Absence of them wont let pods schedule. The fix for the above error is to make sure ServiceAccount `system:serviceaccount:openshift-infra.` regex is always mentioned in `PrivilegedServiceAccounts` section of `IntegrationConfig`. This regex will allow operations from all `ServiceAccounts` present in `openshift-infra` namespace. More info at [Privileged Service Accounts](./how-to-guides/integration-config.md#privileged-serviceaccounts) +**Answer.** OpenShift recently updated its process of handling SCC, and it's now managed by annotations like `openshift.io/sa.scc.uid-range` on the namespaces. Absence of them wont let pods schedule. The fix for the above error is to make sure ServiceAccount `system:serviceaccount:openshift-infra.` regex is always mentioned in `Privileged.serviceAccounts` section of `IntegrationConfig`. This regex will allow operations from all `ServiceAccounts` present in `openshift-infra` namespace. More info at [Privileged Service Accounts](./how-to-guides/integration-config.md#privileged-serviceaccounts) ## Namespace Admission Webhook @@ -47,11 +47,11 @@ The fix is to create namespaces with `kubectl create` instead. ### Q. How do I deploy cluster-scoped resource via the ArgoCD integration? -**Answer.** Multi-Tenant Operator's ArgoCD Integration allows configuration of which cluster-scoped resources can be deployed, both globally and on a per-tenant basis. For a global allow-list that applies to all tenants, you can add both resource `group` and `kind` to the [IntegrationConfig's](./how-to-guides/integration-config.md#argocd) `spec.argocd.clusterResourceWhitelist` field. Alternatively, you can set this up on a tenant level by configuring the same details within a [Tenant's](./how-to-guides/tenant.md) `spec.argocd.appProject.clusterResourceWhitelist` field. For more details, check out the [ArgoCD integration use cases](./tutorials/argocd/enabling-multi-tenancy-argocd.md#allow-argocd-to-sync-certain-cluster-wide-resources) +**Answer.** Multi-Tenant Operator's ArgoCD Integration allows configuration of which cluster-scoped resources can be deployed, both globally and on a per-tenant basis. For a global allow-list that applies to all tenants, you can add both resource `group` and `kind` to the [IntegrationConfig's](./how-to-guides/integration-config.md#argocd) `spec.integrations.argocd.clusterResourceWhitelist` field. Alternatively, you can set this up on a tenant level by configuring the same details within a [Tenant's](./how-to-guides/tenant.md) `spec.integrations.argocd.appProject.clusterResourceWhitelist` field. For more details, check out the [ArgoCD integration use cases](./tutorials/argocd/enabling-multi-tenancy-argocd.md#allow-argocd-to-sync-certain-cluster-wide-resources) ## Q. InvalidSpecError: application repo \ is not permitted in project \ -**Answer.** The above error can occur if the ArgoCD Application is syncing from a source that is not allowed the referenced AppProject. To solve this, verify that you have referred to the correct project in the given ArgoCD Application, and that the repoURL used for the Application's source is valid. If the error still appears, you can add the URL to the relevant Tenant's `spec.argocd.sourceRepos` array. +**Answer.** The above error can occur if the ArgoCD Application is syncing from a source that is not allowed the referenced AppProject. To solve this, verify that you have referred to the correct project in the given ArgoCD Application, and that the repoURL used for the Application's source is valid. If the error still appears, you can add the URL to the relevant Tenant's `spec.integrations.argocd.sourceRepos` array. ## Q. Why are there `mto-showback-*` pods failing in my cluster? diff --git a/content/how-to-guides/integration-config.md b/content/how-to-guides/integration-config.md index f18135b8..e3346500 100644 --- a/content/how-to-guides/integration-config.md +++ b/content/how-to-guides/integration-config.md @@ -3,12 +3,257 @@ IntegrationConfig is used to configure settings of multi-tenancy for Multi Tenant Operator. ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator -spec: +Spec: + components: + console: true + showback: true + ingress: + IngressClassName: 'nginx' + Keycloak: + Host: tenant-operator-keycloak.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + Console: + Host: tenant-operator-console.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + Gateway: + Host: tenant-operator-gateway.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + trustedRootCert: my-custom-cert + accessControl: + rbac: + tenantRoles: + default: + owner: + clusterRoles: + - admin + editor: + clusterRoles: + - edit + viewer: + clusterRoles: + - view + - viewer + custom: + - labelSelector: + matchExpressions: + - key: stakater.com/kind + operator: In + values: + - build + matchLabels: + stakater.com/kind: dev + owner: + clusterRoles: + - custom-owner + editor: + clusterRoles: + - custom-editor + viewer: + clusterRoles: + - custom-viewer + - custom-view + namespaceAccessPolicy: + deny: + privilegedNamespaces: + users: + - system:serviceaccount:openshift-argocd:argocd-application-controller + - adam@stakater.com + groups: + - cluster-admins + privileged: + namespaces: + - ^default$ + - ^openshift-* + - ^kube-* + serviceAccounts: + - ^system:serviceaccount:openshift-* + - ^system:serviceaccount:kube-* + users: + - '' + groups: + - cluster-admins + metadata: + groups: + labels: + role: customer-reader + annotations: + openshift.io/node-selector: node-role.kubernetes.io/worker= + namespaces: + labels: + stakater.com/workload-monitoring: "true" + annotations: + openshift.io/node-selector: node-role.kubernetes.io/worker= + sandboxes: + labels: + stakater.com/kind: sandbox + annotations: + openshift.io/node-selector: node-role.kubernetes.io/worker= + integrations: + argocd: + enabled: bool + clusterResourceWhitelist: + - group: tronador.stakater.com + kind: EnvironmentProvisioner + namespaceResourceBlacklist: + - group: '' # all groups + kind: ResourceQuota + namespace: openshift-operators + vault: + enabled: true + authMethod: kubernetes #enum: {kubernetes:default, Token} + accessInfo: + accessorPath: oidc/ + address: https://vault.apps.prod.abcdefghi.kubeapp.cloud/ + roleName: mto + secretRef: + name: '' + namespace: '' + config: + ssoClient: vault +``` + +Following are the different components that can be used to configure multi-tenancy in a cluster via Multi Tenant Operator. + +## Components + +```yaml + components: + console: true + showback: true + ingress: + IngressClassName: nginx + Keycloak: + Host: tenant-operator-keycloak.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + Console: + Host: tenant-operator-console.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + Gateway: + Host: tenant-operator-gateway.apps.mycluster-ams.abcdef.cloud + TLSSecretName: tenant-operator-tls + trustedRootCert: my-custom-cert +``` + +- `components.console:` Enables or disables the console GUI for MTO. +- `components.showback:` Enables or disables the showback feature on the console. +- `components.ingress:` Configures the ingress settings for various components: + - `ingressClassName:` Ingress class to be used for the ingress. + - `console:` Settings for the console's ingress. + - `host:` hostname for the console's ingress. + - `tlsSecretName:` Name of the secret containing the TLS certificate and key for the console's ingress. + - `gateway:` Settings for the gateway's ingress. + - `host:` hostname for the gateway's ingress. + - `tlsSecretName:` Name of the secret containing the TLS certificate and key for the gateway's ingress. + - `keycloak:` Settings for the Keycloak's ingress. + - `host:` hostname for the Keycloak's ingress. + - `tlsSecretName:` Name of the secret containing the TLS certificate and key for the Keycloak's ingress. +- `components.trustedRootCert:` Name of the secret containing the root CA certificate. + +Here's an example of how to generate the secrets required to configure MTO: + +**TLS Secret for Ingress:** + +Create a TLS secret containing your SSL/TLS certificate and key for secure communication. This secret will be used for the Console, Gateway, and Keycloak ingresses. + +```bash +kubectl -n multi-tenant-operator create secret tls --key= --cert= +``` + +**Trusted Root Certificate Secret:** + +If using a custom certificate authority (CA) or self-signed certificates, create a Kubernetes secret containing your root CA certificate. This is required in order to ensure MTO Components trust the custom certificates. + +```bash +kubectl -n multi-tenant-operator create secret generic --from-file= +``` + +>Note: `trustedRootCert` and `tls-secret-name` are optional. If not provided, MTO will use the default root CA certificate and secrets respectively. + +Integration config will be managing the following resources required for console GUI: + +- `MTO Postgresql` resources. +- `MTO Prometheus` resources. +- `MTO Opencost` resources. +- `MTO Console, Gateway, Keycloak` resources. +- `Showback` cronjob. + +Details on console GUI and showback can be found [here](../explanation/console.md) + +## Access Control + +```yaml +accessControl: + rbac: + tenantRoles: + default: + owner: + clusterRoles: + - admin + editor: + clusterRoles: + - edit + viewer: + clusterRoles: + - view + - viewer + custom: + - labelSelector: + matchExpressions: + - key: stakater.com/kind + operator: In + values: + - build + matchLabels: + stakater.com/kind: dev + owner: + clusterRoles: + - custom-owner + editor: + clusterRoles: + - custom-editor + viewer: + clusterRoles: + - custom-viewer + - custom-view + namespaceAccessPolicy: + deny: + privilegedNamespaces: + users: + - system:serviceaccount:openshift-argocd:argocd-application-controller + - adam@stakater.com + groups: + - cluster-admins + privileged: + namespaces: + - ^default$ + - ^openshift-* + - ^kube-* + serviceAccounts: + - ^system:serviceaccount:openshift-* + - ^system:serviceaccount:kube-* + users: + - '' + groups: + - cluster-admins +``` + +### RBAC + +RBAC is used to configure the roles that will be applied to each Tenant namespace. The field allows optional custom roles, that are then used to create RoleBindings for namespaces that match a labelSelector. + +#### TenantRoles + +TenantRoles are required within the IntegrationConfig, as they are used for defining what roles will be applied to each Tenant namespace. The field allows optional custom roles, that are then used to create RoleBindings for namespaces that match a labelSelector. + +> ⚠️ If you do not configure roles in any way, then the default OpenShift roles of `owner`, `edit`, and `view` will apply to Tenant members. Their details can be found [here](../reference-guides/custom-roles.md) + +```yaml +rbac: tenantRoles: default: owner: @@ -40,124 +285,13 @@ spec: clusterRoles: - custom-viewer - custom-view - openshift: - project: - labels: - stakater.com/workload-monitoring: "true" - annotations: - openshift.io/node-selector: node-role.kubernetes.io/worker= - group: - labels: - role: customer-reader - sandbox: - labels: - stakater.com/kind: sandbox - clusterAdminGroups: - - cluster-admins - privilegedNamespaces: - - ^default$ - - ^openshift-* - - ^kube-* - privilegedServiceAccounts: - - ^system:serviceaccount:openshift-* - - ^system:serviceaccount:kube-* - namespaceAccessPolicy: - deny: - privilegedNamespaces: - users: - - system:serviceaccount:openshift-argocd:argocd-application-controller - - adam@stakater.com - groups: - - cluster-admins - argocd: - namespace: openshift-operators - namespaceResourceBlacklist: - - group: '' # all groups - kind: ResourceQuota - clusterResourceWhitelist: - - group: tronador.stakater.com - kind: EnvironmentProvisioner - provision: - console: true - ingress: - console: - host: tenant-operator-console.apps.mycluster-ams.abcdef.cloud - tlsSecretName: tenant-operator-tls - ingressClassName: nginx - gateway: - host: tenant-operator-gateway.apps.mycluster-ams.abcdef.cloud - tlsSecretName: tenant-operator-tls - ingressClassName: nginx - keycloak: - host: tenant-operator-keycloak.apps.mycluster-ams.abcdef.cloud - tlsSecretName: tenant-operator-tls - ingressClassName: nginx - showback: true - trustedRootCert: my-custom-cert - rhsso: - enabled: true - realm: customer - endpoint: - url: https://iam-keycloak-auth.apps.prod.abcdefghi.kubeapp.cloud/ - secretReference: - name: auth-secrets - namespace: openshift-auth - vault: - enabled: true - accessorPath: oidc/ - address: 'https://vault.apps.prod.abcdefghi.kubeapp.cloud/' - roleName: mto - sso: - clientName: vault -``` - -Following are the different components that can be used to configure multi-tenancy in a cluster via Multi Tenant Operator. - -## TenantRoles - -TenantRoles are required within the IntegrationConfig, as they are used for defining what roles will be applied to each Tenant namespace. The field allows optional custom roles, that are then used to create RoleBindings for namespaces that match a labelSelector. - -> ⚠️ If you do not configure roles in any way, then the default OpenShift roles of `owner`, `edit`, and `view` will apply to Tenant members. Their details can be found [here](../reference-guides/custom-roles.md) - -```yaml -tenantRoles: - default: - owner: - clusterRoles: - - admin - editor: - clusterRoles: - - edit - viewer: - clusterRoles: - - view - - viewer - custom: - - labelSelector: - matchExpressions: - - key: stakater.com/kind - operator: In - values: - - build - matchLabels: - stakater.com/kind: dev - owner: - clusterRoles: - - custom-owner - editor: - clusterRoles: - - custom-editor - viewer: - clusterRoles: - - custom-viewer - - custom-view ``` -### Default +##### Default This field contains roles that will be used to create default roleBindings for each namespace that belongs to tenants. These roleBindings are only created for a namespace if that namespace isn't already matched by the `custom` field below it. Therefore, it is required to have at least one role mentioned within each of its three subfields: `owner`, `editor`, and `viewer`. These 3 subfields also correspond to the member fields of the [Tenant CR](./tenant.md#tenant) -### Custom +##### Custom An array of custom roles. Similar to the `default` field, you can mention roles within this field as well. However, the custom roles also require the use of a `labelSelector` for each iteration within the array. The roles mentioned here will only apply to the namespaces that are matched by the labelSelector. If a namespace is matched by 2 different labelSelectors, then both roles will apply to it. Additionally, roles can be skipped within the labelSelector. These missing roles are then inherited from the `default` roles field . For example, if the following custom roles arrangement is used: @@ -178,58 +312,91 @@ custom: Then the `editor` and `viewer` roles will be taken from the `default` roles field, as that is required to have at least one role mentioned. -## OpenShift +### Namespace Access Policy + +Namespace Access Policy is used to configure the namespaces that are allowed to be created by tenants. It also allows the configuration of namespaces that are ignored by MTO. + +```yaml +namespaceAccessPolicy: + deny: + privilegedNamespaces: + groups: + - cluster-admins + users: + - system:serviceaccount:openshift-argocd:argocd-application-controller + - adam@stakater.com + privileged: + namespaces: + - ^default$ + - ^openshift-* + - ^kube-* + serviceAccounts: + - ^system:serviceaccount:openshift-* + - ^system:serviceaccount:kube-* + users: + - '' + groups: + - cluster-admins +``` + +#### Deny + +`namespaceAccessPolicy.Deny:` Can be used to restrict privileged *users/groups* CRUD operation over managed namespaces. + +#### Privileged + +##### Namespaces + +`privileged.namespaces:` Contains the list of `namespaces` ignored by MTO. MTO will not manage the `namespaces` in this list. Treatment for privileged namespaces does not involve further integrations or finalizers processing as with normal namespaces. Values in this list are regex patterns. + +For example: + +- To ignore the `default` namespace, we can specify `^default$` +- To ignore all namespaces starting with the `openshift-` prefix, we can specify `^openshift-.*`. +- To ignore any namespace containing `stakater` in its name, we can specify `^stakater.`. (A constant word given as a regex pattern will match any namespace containing that word.) + +##### ServiceAccounts -``` yaml -openshift: - project: +`privileged.serviceAccounts:` Contains the list of `ServiceAccounts` ignored by MTO. MTO will not manage the `ServiceAccounts` in this list. Values in this list are regex patterns. For example, to ignore all `ServiceAccounts` starting with the `system:serviceaccount:openshift-` prefix, we can use `^system:serviceaccount:openshift-.*`; and to ignore a specific service account like `system:serviceaccount:builder` service account we can use `^system:serviceaccount:builder$.` + +!!! note + `stakater`, `stakater.` and `stakater.*` will have the same effect. To check out the combinations, go to [Regex101](https://regex101.com/), select Golang, and type your expected regex and test string. + +##### Users + +`privileged.users:` Contains the list of `users` ignored by MTO. MTO will not manage the `users` in this list. Values in this list are regex patterns. + +##### Groups + +`privileged.groups:` Contains names of the groups that are allowed to perform CRUD operations on namespaces present on the cluster. Users in the specified group(s) will be able to perform these operations without MTO getting in their way. MTO does not interfere even with the deletion of privilegedNamespaces. + +!!! note + User `kube:admin` is bypassed by default to perform operations as a cluster admin, this includes operations on all the namespaces. + +> ⚠️ If you want to use a more complex regex pattern (for the `privileged.namespaces` or `privileged.serviceAccounts` field), it is recommended that you test the regex pattern first - either locally or using a platform such as . + +## Metadata + +```yaml +metadata: + groups: + labels: + role: customer-reader + annotations: {} + namespaces: labels: stakater.com/workload-monitoring: "true" annotations: openshift.io/node-selector: node-role.kubernetes.io/worker= - group: - labels: - role: customer-reader - sandbox: + sandboxes: labels: stakater.com/kind: sandbox - clusterAdminGroups: - - cluster-admins - privilegedNamespaces: - - ^default$ - - ^openshift-* - - ^kube-* - privilegedServiceAccounts: - - ^system:serviceaccount:openshift-* - - ^system:serviceaccount:kube-* - namespaceAccessPolicy: - deny: - privilegedNamespaces: - users: - - system:serviceaccount:openshift-argocd:argocd-application-controller - - adam@stakater.com - groups: - - cluster-admins + annotations: {} ``` -### Project, group and sandbox - -We can use the `openshift.project`, `openshift.group` and `openshift.sandbox` fields to automatically add `labels` and `annotations` to the **Projects** and **Groups** managed via MTO. +### Namespaces, group and sandbox -```yaml - openshift: - project: - labels: - stakater.com/workload-monitoring: "true" - annotations: - openshift.io/node-selector: node-role.kubernetes.io/worker= - group: - labels: - role: customer-reader - sandbox: - labels: - stakater.com/kind: sandbox -``` +We can use the `metadata.namespaces`, `metadata.group` and `metadata.sandbox` fields to automatically add `labels` and `annotations` to the **Namespaces** and **Groups** managed via MTO. If we want to add default *labels/annotations* to sandbox namespaces of tenants than we just simply add them in `openshift.project.labels`/`openshift.project.annotations` respectively. @@ -263,188 +430,101 @@ users: - andrew@stakater.com ``` -### Cluster Admin Groups - -### Cluster Admin Groups +## Integrations -`clusterAdminGroups:` Contains names of the groups that are allowed to perform CRUD operations on namespaces present on the cluster. Users in the specified group(s) will be able to perform these operations without MTO getting in their way. MTO does not interfere even with the deletion of privilegedNamespaces. - -!!! note - User `kube:admin` is bypassed by default to perform operations as a cluster admin, this includes operations on all the namespaces. - -### Privileged Namespaces - -`privilegedNamespaces:` Contains the list of `namespaces` ignored by MTO. MTO will not manage the `namespaces` in this list. Treatment for privileged namespaces does not involve further integrations or finalizers processing as with normal namespaces. Values in this list are regex patterns. - -For example: - -- To ignore the `default` namespace, we can specify `^default$` -- To ignore all namespaces starting with the `openshift-` prefix, we can specify `^openshift-.*`. -- To ignore any namespace containing `stakater` in its name, we can specify `^stakater.`. (A constant word given as a regex pattern will match any namespace containing that word.) - -### Privileged ServiceAccounts - -`privilegedServiceAccounts:` Contains the list of `ServiceAccounts` ignored by MTO. MTO will not manage the `ServiceAccounts` in this list. Values in this list are regex patterns. For example, to ignore all `ServiceAccounts` starting with the `system:serviceaccount:openshift-` prefix, we can use `^system:serviceaccount:openshift-.*`; and to ignore a specific service account like `system:serviceaccount:builder` service account we can use `^system:serviceaccount:builder$.` - -!!! note - `stakater`, `stakater.` and `stakater.*` will have the same effect. To check out the combinations, go to [Regex101](https://regex101.com/), select Golang, and type your expected regex and test string. - -### Namespace Access Policy - -`namespaceAccessPolicy.Deny:` Can be used to restrict privileged *users/groups* CRUD operation over managed namespaces. +Integrations are used to configure the integrations that MTO has with other tools. Currently, MTO supports the following integrations: ```yaml -namespaceAccessPolicy: - deny: - privilegedNamespaces: - groups: - - cluster-admins - users: - - system:serviceaccount:openshift-argocd:argocd-application-controller - - adam@stakater.com +integrations: + argocd: + enabled: bool + clusterResourceWhitelist: + - group: tronador.stakater.com + kind: EnvironmentProvisioner + namespaceResourceBlacklist: + - group: '' # all groups + kind: ResourceQuota + namespace: openshift-operators + vault: + enabled: true + authMethod: kubernetes #enum: {kubernetes:default, Token} + accessInfo: + accessorPath: oidc/ + address: https://vault.apps.prod.abcdefghi.kubeapp.cloud/ + roleName: mto + secretRef: + name: '' + namespace: '' + config: + ssoClient: vault ``` -> ⚠️ If you want to use a more complex regex pattern (for the `openshift.privilegedNamespaces` or `openshift.privilegedServiceAccounts` field), it is recommended that you test the regex pattern first - either locally or using a platform such as . - -## ArgoCD +### ArgoCD -### Namespace +[ArgoCD](https://argoproj.github.io/argo-cd/) is a declarative, GitOps continuous delivery tool for Kubernetes. It follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. ArgoCD uses Kubernetes manifests and configures the applications on the cluster. -`argocd.namespace` is an optional field used to specify the namespace where ArgoCD Applications and AppProjects are deployed. The field should be populated when you want to create an ArgoCD AppProject for each tenant. - -### NamespaceResourceBlacklist - -```yaml -argocd: - namespaceResourceBlacklist: - - group: '' # all resource groups - kind: ResourceQuota - - group: '' - kind: LimitRange - - group: '' - kind: NetworkPolicy -``` - -`argocd.namespaceResourceBlacklist` prevents ArgoCD from syncing the listed resources from your GitOps repo. - -### ClusterResourceWhitelist +If `argocd` is configured on a cluster, then ArgoCD configuration can be enabled. ```yaml argocd: + enabled: bool clusterResourceWhitelist: - - group: tronador.stakater.com - kind: EnvironmentProvisioner -``` - -`argocd.clusterResourceWhitelist` allows ArgoCD to sync the listed cluster scoped resources from your GitOps repo. - -## Provision - -```yaml -provision: - console: true - ingress: - console: - host: tenant-operator-console.apps.mycluster-ams.abcdef.cloud - ingressSecretName: tenant-operator-tls - ingressClassName: nginx - gateway: - host: tenant-operator-gateway.apps.mycluster-ams.abcdef.cloud - ingressSecretName: tenant-operator-tls - ingressClassName: nginx - keycloak: - host: tenant-operator-keycloak.apps.mycluster-ams.abcdef.cloud - ingressSecretName: tenant-operator-tls - ingressClassName: nginx - showback: true - trustedRootCert: my-custom-cert -``` - -`provision.console:` Enables or disables the console GUI for MTO. -`provision.ingress:` Configures the ingress settings for various components: - `console:` Settings for the console's ingress. - `gateway:` Settings for the gateway's ingress. - `keycloak:` Settings for the Keycloak's ingress. - (including host, TLS secret, and ingress class) -`provision.showback:` Enables or disables the showback feature on the console. -`provision.trustedRootCert:` Name of the secret containing the root CA certificate. - -Here's an example of how to generate the secrets required to configure MTO: - -**TLS Secret for Ingress:** - -Create a TLS secret containing your SSL/TLS certificate and key for secure communication. This secret will be used for the Console, Gateway, and Keycloak ingresses. - -```bash -kubectl -n multi-tenant-operator create secret tls --key= --cert= -``` - -**Trusted Root Certificate Secret:** - -If using a custom certificate authority (CA) or self-signed certificates, create a Kubernetes secret containing your root CA certificate. This is required in order to ensure MTO Components trust the custom certificates. - -```bash -kubectl -n multi-tenant-operator create secret generic --from-file= + - group: tronador.stakater.com + kind: EnvironmentProvisioner + namespaceResourceBlacklist: + - group: '' # all groups + kind: ResourceQuota + namespace: openshift-operators ``` ->Note: `trustedRootCert` and `tls-secret-name` are optional. If not provided, MTO will use the default root CA certificate and secrets respectively. - -Integration config will be managing the following resources required for console GUI: - -- `MTO Postgresql` resources. -- `MTO Prometheus` resources. -- `MTO Opencost` resources. -- `MTO Console, Gateway, Keycloak` resources. -- `Showback` cronjob. - -Details on console GUI and showback can be found [here](../explanation/console.md) +- `argocd.clusterResourceWhitelist` allows ArgoCD to sync the listed cluster scoped resources from your GitOps repo. +- `argocd.namespaceResourceBlacklist` prevents ArgoCD from syncing the listed resources from your GitOps repo. +- `argocd.namespace` is an optional field used to specify the namespace where ArgoCD Applications and AppProjects are deployed. The field should be populated when you want to create an ArgoCD AppProject for each tenant. -## RHSSO (Red Hat Single Sign-On) +### Vault -Red Hat Single Sign-On [RHSSO](https://access.redhat.com/products/red-hat-single-sign-on) is based on the Keycloak project and enables you to secure your web applications by providing Web single sign-on (SSO) capabilities based on popular standards such as SAML 2.0, OpenID Connect and OAuth 2.0. +[Vault](https://www.vaultproject.io/) is used to secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. -If `RHSSO` is configured on a cluster, then RHSSO configuration can be enabled. +If `vault` is configured on a cluster, then Vault configuration can be enabled. ```yaml -rhsso: +vault: enabled: true - realm: customer - endpoint: - secretReference: - name: auth-secrets - namespace: openshift-auth - url: https://iam-keycloak-auth.apps.prod.abcdefghi.kubeapp.cloud/ + authMethod: kubernetes #enum: {kubernetes:default, token} + accessInfo: + accessorPath: oidc/ + address: https://vault.apps.prod.abcdefghi.kubeapp.cloud/ + roleName: mto + secretRef: + name: '' + namespace: '' + config: + ssoClient: vault ``` -If enabled, then admins have to provide secret and URL of RHSSO. +If enabled, then admins have to specify the `authMethod` to be used for authentication. MTO supports two authentication methods: -- `secretReference.name:` Will contain the name of the secret. -- `secretReference.namespace:` Will contain the namespace of the secret. -- `realm:` Will contain the realm name which is configured for users. -- `url:` Will contain the URL of RHSSO. +- `kubernetes`: This is the default authentication method. It uses the Kubernetes authentication method to authenticate with Vault. +- `token`: This method uses a Vault token to authenticate with Vault. -## Vault +#### AuthMethod - Kubernetes -[Vault](https://www.vaultproject.io/) is used to secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. +If `authMethod` is set to `kubernetes`, then admins have to specify the following fields: -If `vault` is configured on a cluster, then Vault configuration can be enabled. +- `accessorPath:` Accessor Path within Vault to fetch SSO accessorID +- `address:` Valid Vault address reachable within cluster. +- `roleName:` Vault's Kubernetes authentication role +- `sso.clientName:` SSO client name. -```yaml -Vault: - enabled: true - accessorPath: oidc/ - address: 'https://vault.apps.prod.abcdefghi.kubeapp.cloud/' - roleName: mto - sso: - clientName: vault -``` +#### AuthMethod - Token -If enabled, then admins have to provide following details: +If `authMethod` is set to `token`, then admins have to specify the following fields: - `accessorPath:` Accessor Path within Vault to fetch SSO accessorID - `address:` Valid Vault address reachable within cluster. -- `roleName:` Vault's Kubernetes authentication role -- `sso.clientName:` SSO client name. +- `secretRef:` Secret containing Vault token. + - `name:` Name of the secret containing Vault token. + - `namespace:` Namespace of the secret containing Vault token. For more details around enabling Kubernetes auth in Vault, visit [here](https://developer.hashicorp.com/vault/docs/auth/kubernetes) diff --git a/content/reference-guides/configuring-multitenant-network-isolation.md b/content/reference-guides/configuring-multitenant-network-isolation.md index 0d508d18..5d1bc6ef 100644 --- a/content/reference-guides/configuring-multitenant-network-isolation.md +++ b/content/reference-guides/configuring-multitenant-network-isolation.md @@ -51,14 +51,14 @@ resources: Once the template has been created, Bill edits the [IntegrationConfig](../how-to-guides/integration-config.md) to add unique label to all tenant projects: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - openshift: - project: + metadata: + namespaces: labels: stakater.com/workload-monitoring: "true" tenant-network-policy: "true" @@ -67,11 +67,12 @@ spec: sandbox: labels: stakater.com/kind: sandbox - privilegedNamespaces: + privileged: + namespaces: - default - ^openshift-* - ^kube-* - privilegedServiceAccounts: + serviceAccounts: - ^system:serviceaccount:openshift-* - ^system:serviceaccount:kube-* ``` diff --git a/content/reference-guides/custom-roles.md b/content/reference-guides/custom-roles.md index 1af4184f..d9c8a36b 100644 --- a/content/reference-guides/custom-roles.md +++ b/content/reference-guides/custom-roles.md @@ -5,23 +5,25 @@ This feature allows the cluster admins to change the default roles assigned to T For example, if Bill as the cluster admin wants to reduce the privileges that tenant owners have, so they cannot create or edit Roles or bind them. As an admin of an OpenShift cluster, Bill can do this by assigning the `edit` role to all tenant owners. This is easily achieved by modifying the [IntegrationConfig](../how-to-guides/integration-config.md): ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - tenantRoles: - default: - owner: - clusterRoles: - - edit - editor: - clusterRoles: - - edit - viewer: - clusterRoles: - - view + accessControl: + rbac: + tenantRoles: + default: + owner: + clusterRoles: + - edit + editor: + clusterRoles: + - edit + viewer: + clusterRoles: + - view ``` Once all namespaces reconcile, the old `admin` RoleBindings should get replaced with the `edit` ones for each tenant owner. @@ -31,42 +33,44 @@ Once all namespaces reconcile, the old `admin` RoleBindings should get replaced Bill now wants the owners of the tenants `bluesky` and `alpha` to have `admin` permissions over their namespaces. Custom roles feature will allow Bill to do this, by modifying the IntegrationConfig like this: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - tenantRoles: - default: - owner: - clusterRoles: - - edit - editor: - clusterRoles: - - edit - viewer: - clusterRoles: - - view - custom: - - labelSelector: - matchExpressions: - - key: stakater.com/tenant - operator: In - values: - - alpha - owner: - clusterRoles: - - admin - - labelSelector: - matchExpressions: - - key: stakater.com/tenant - operator: In - values: - - bluesky - owner: - clusterRoles: - - admin + accessControl: + rbac: + tenantRoles: + default: + owner: + clusterRoles: + - edit + editor: + clusterRoles: + - edit + viewer: + clusterRoles: + - view + custom: + - labelSelector: + matchExpressions: + - key: stakater.com/tenant + operator: In + values: + - alpha + owner: + clusterRoles: + - admin + - labelSelector: + matchExpressions: + - key: stakater.com/tenant + operator: In + values: + - bluesky + owner: + clusterRoles: + - admin ``` New Bindings will be created for the Tenant owners of `bluesky` and `alpha`, corresponding to the `admin` Role. Bindings for editors and viewer will be inherited from the `default roles`. All other Tenant owners will have an `edit` Role bound to them within their namespaces diff --git a/content/reference-guides/integrationconfig.md b/content/reference-guides/integrationconfig.md index ce2ff231..1bf7b382 100644 --- a/content/reference-guides/integrationconfig.md +++ b/content/reference-guides/integrationconfig.md @@ -14,14 +14,14 @@ Error from server (Cannot Create namespace stakater-test without label stakater. Bill is trying to create a namespace without the `stakater.com/tenant` label. Creating a namespace without this label is only allowed if the namespace is privileged. Privileged namespaces will be ignored by MTO and do not require the said label. Therefore, Bill will add the required regex in the IntegrationConfig, along with any other namespaces which are privileged and should be ignored by MTO - like `default`, or namespaces with prefixes like `openshift`, `kube`: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - openshift: - privilegedNamespaces: + privileged: + namespaces: - ^default$ - ^openshift-.* - ^kube-.* @@ -40,14 +40,14 @@ MTO will also disallow all users which are not tenant owners to perform CRUD ope If Bill wants MTO to ignore Service Accounts, then he would simply have to add them in the IntegrationConfig: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - openshift: - privilegedServiceAccounts: + privileged: + serviceAccounts: - system:serviceaccount:openshift - system:serviceaccount:stakater - system:serviceaccount:kube @@ -58,14 +58,14 @@ spec: Bill can also use regex patterns to ignore a set of service accounts: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - openshift: - privilegedServiceAccounts: + privileged: + serviceAccounts: - ^system:serviceaccount:openshift-.* - ^system:serviceaccount:stakater-.* ``` @@ -81,19 +81,25 @@ MTO automatically creates Vault secret paths for tenants, where tenant members c Bill would first have to integrate Vault with MTO by adding the details in IntegrationConfig. For more [details](../how-to-guides/integration-config.md#vault) ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: - vault: - enabled: true - accessorPath: oidc/ - address: 'https://vault.apps.prod.abcdefghi.kubeapp.cloud/' - roleName: mto - sso: - clientName: vault + integrations: + vault: + enabled: true + authMethod: kubernetes + accessInfo: + accessorPath: oidc/ + address: https://vault.apps.prod.abcdefghi.kubeapp.cloud/ + roleName: mto + secretRef: + name: '' + namespace: '' + config: + ssoClient: vault ``` Bill then creates a tenant for Anna and John: @@ -117,24 +123,3 @@ spec: Now Bill goes to `Vault` and sees that a path for `tenant` has been made under the name `bluesky/kv`, confirming that Tenant members with the Owner or Edit roles now have access to the tenant's Vault path. Now if Anna sign's in to the Vault via OIDC, she can see her tenants path and secrets. Whereas if John sign's in to the Vault via OIDC, he can't see his tenants path or secrets as he doesn't have the access required to view them. - -## Configuring RHSSO (Red Hat Single Sign-On) in IntegrationConfig - -Red Hat Single Sign-On [RHSSO](https://access.redhat.com/products/red-hat-single-sign-on) is based on the Keycloak project and enables you to secure your web applications by providing Web single sign-on (SSO) capabilities based on popular standards such as SAML 2.0, OpenID Connect and OAuth 2.0. - -If Bill the cluster admin has RHSSO configured in his cluster, then he can take benefit from MTO's integration with RHSSO and Vault. - -MTO automatically allows tenant members to access Vault via OIDC(RHSSO authentication and authorization) to access secret paths for tenants where tenant members can securely save their secrets. - -Bill would first have to integrate RHSSO with MTO by adding the details in IntegrationConfig. [Visit here](../how-to-guides/integration-config.md#rhsso-red-hat-single-sign-on) for more details. - -```yaml -rhsso: - enabled: true - realm: customer - endpoint: - secretReference: - name: auth-secrets - namespace: openshift-auth - url: https://iam-keycloak-auth.apps.prod.abcdefghi.kubeapp.cloud/ -``` diff --git a/content/tutorials/argocd/enabling-multi-tenancy-argocd.md b/content/tutorials/argocd/enabling-multi-tenancy-argocd.md index 06d86c90..780cd461 100644 --- a/content/tutorials/argocd/enabling-multi-tenancy-argocd.md +++ b/content/tutorials/argocd/enabling-multi-tenancy-argocd.md @@ -27,7 +27,7 @@ We have set a default ArgoCD configuration in Multi Tenant Operator that fulfils Bill wants each tenant to also have their own ArgoCD AppProjects. To make sure this happens correctly, Bill will first specify the ArgoCD namespace in the [IntegrationConfig](../../how-to-guides/integration-config.md): ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config @@ -134,20 +134,21 @@ Users belonging to the Sigma group will now only see applications created by the Bill wants tenants to not be able to sync `ResourceQuota` and `LimitRange` resources to their namespaces. To do this correctly, Bill will specify these resources to blacklist in the ArgoCD portion of the [IntegrationConfig](../../how-to-guides/integration-config.md): ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: ... - argocd: - namespace: openshift-operators - namespaceResourceBlacklist: - - group: "" - kind: ResourceQuota - - group: "" - kind: LimitRange + integrations: + argocd: + namespace: openshift-operators + namespaceResourceBlacklist: + - group: "" + kind: ResourceQuota + - group: "" + kind: LimitRange ... ``` @@ -174,18 +175,19 @@ spec: Bill now wants tenants to be able to sync the `Environment` cluster scoped resource to the cluster. To do this correctly, Bill will specify the resource to allow-list in the ArgoCD portion of the Integration Config's Spec: ```yaml -apiVersion: tenantoperator.stakater.com/v1alpha1 +apiVersion: tenantoperator.stakater.com/v1beta1 kind: IntegrationConfig metadata: name: tenant-operator-config namespace: multi-tenant-operator spec: ... - argocd: - namespace: openshift-operators - clusterResourceWhitelist: - - group: "" - kind: Environment + integrations: + argocd: + namespace: openshift-operators + clusterResourceWhitelist: + - group: "" + kind: Environment ... ``` diff --git a/content/tutorials/installation.md b/content/tutorials/installation.md index 1d72e321..043b4fd6 100644 --- a/content/tutorials/installation.md +++ b/content/tutorials/installation.md @@ -112,7 +112,7 @@ To enable console GUI for MTO, go to `Search` -> `IntegrationConfig` -> `tenant- ```yaml spec: - provision: + components: console: true showback: true ```