diff --git a/docs/glossary.md b/docs/glossary.md index 6215f4d..4536f45 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -65,6 +65,10 @@ OpenStack, etc.). ### Managed cluster A Kubernetes cluster created and managed by Project 2A. +### Mutli Cluster Service +The `MultiClusterService` is a custom resource used to manage deployment of beach-head services +across multiple clusters. + ### Management cluster The Kubernetes cluster where 2A is installed and from which all other managed -clusters are managed. \ No newline at end of file +clusters are managed. diff --git a/docs/quick-start/2a-installation.md b/docs/quick-start/2a-installation.md index 9dbc827..a0dec86 100644 --- a/docs/quick-start/2a-installation.md +++ b/docs/quick-start/2a-installation.md @@ -1,4 +1,4 @@ -### Requirements +## Requirements Project 2A requires a Kubernetes cluster. It can be of any type and will become the 2A _management cluster_. @@ -12,7 +12,7 @@ The following instructions assume: - You have [Helm](https://helm.sh/docs/intro/install/) installed. - You have [kubectl](https://kubernetes.io/docs/tasks/tools/) installed. -#### Helpful Tools +### Helpful Tools It may be helpful to have the following tools installed: @@ -20,26 +20,26 @@ It may be helpful to have the following tools installed: - [Mirantis Lens](https://k8slens.dev/) - [k9s](https://k9scli.io/) -### Installation via Helm +## Installation via Helm ```bash -helm install hmc oci://ghcr.io/mirantis/hmc/charts/hmc --version 0.0.3 -n hmc-system --create-namespace +helm install hmc oci://ghcr.io/mirantis/hmc/charts/hmc --version 0.0.4 -n hmc-system --create-namespace ``` -### Verification +## Verification -The installation takes a couple of minutes until 2A and its subcomponents are +The installation takes a few minutes until 2A and its subcomponents are fully installed and configured. -Verify the installation by checking all the pods in the `hmc-system` namespace -with the following command: +### Verify Core Components are running + +Check pods are running in the `hmc-system` namespace with the following command: ```bash kubectl get pods -n hmc-system ``` -The output is similar to: - +The output should be similar to: ```bash NAME READY STATUS azureserviceoperator-controller-manager-86d566cdbc-rqkt9 1/1 Running @@ -59,33 +59,83 @@ k0smotron-controller-manager-infrastructure-7f77f55675-tv8vb 2/2 Running source-controller-5f648d6f5d-7mhz5 1/1 Running ``` +Checking pods are running in the `projectsveltos` namespace with the following command: +```sh +kubectl get pods -n projectsveltos +``` + +The output should be similar to: +```sh +NAME READY STATUS RESTARTS AGE +access-manager-cd49cffc9-c4q97 1/1 Running 0 16m +addon-controller-64c7f69796-whw25 1/1 Running 0 16m +classifier-manager-574c9d794d-j8852 1/1 Running 0 16m +conversion-webhook-5d78b6c648-p6pxd 1/1 Running 0 16m +event-manager-6df545b4d7-mbjh5 1/1 Running 0 16m +hc-manager-7b749c57d-5phkb 1/1 Running 0 16m +sc-manager-f5797c4f8-ptmvh 1/1 Running 0 16m +shard-controller-767975966-v5qqn 1/1 Running 0 16m +sveltos-agent-manager-56bbf5fb94-9lskd 1/1 Running 0 15m +``` + If you have fewer pods, give 2A more time to reconcile all the pods. -As a second verification, check that the example `ClusterTemplate` objects have -been installed and are valid: +### Verify 2A templates have been successfully reconciled + +For additional verification, check that the example templates packaged with 2A have +been installed and are valid. + +Check `ProviderTemplate` objects with: + +```sh +kubectl get providertemplate -n hmc-system +``` + +The output should be similar to: + +```sh +NAME VALID +cluster-api-0-0-3 true +cluster-api-provider-aws-0-0-3 true +cluster-api-provider-azure-0-0-3 true +cluster-api-provider-vsphere-0-0-3 true +hmc-0-0-4 true +k0smotron-0-0-3 true +projectsveltos-0-42-0 true +``` + +Check `ClusterTemplate` objects with: ```bash kubectl get clustertemplate -n hmc-system ``` -The output is similar to: +The output should be similar to: ```bash NAME VALID -aws-eks-0-0-1 true -aws-hosted-cp-0-0-2 true -aws-standalone-cp-0-0-1 true -aws-standalone-cp-0-1-0 true +aws-eks-0-0-2 true +aws-hosted-cp-0-0-3 true +aws-standalone-cp-0-0-3 true azure-hosted-cp-0-0-3 true -azure-standalone-cp-0-0-1 true -azure-standalone-cp-0-1-0 true -remote-single-standalone-cp-0-1-4 true -remote-single-standalone-cp-0-1-5 true -remote-single-standalone-cp-0-1-6 true -remote-single-standalone-cp-0-1-7 true -remote-single-standalone-cp-0-1-8 true -vsphere-hosted-cp-0-0-2 true -vsphere-standalone-cp-0-0-2 true +azure-standalone-cp-0-0-3 true +vsphere-hosted-cp-0-0-3 true +vsphere-standalone-cp-0-0-3 true +``` + +Check `ServiceTemplate` objects with: + +```sh +kubectl get servicetemplate -n hmc-system +``` + +The output should be similar to: + +```sh +NAME VALID +ingress-nginx-4-11-0 true +ingress-nginx-4-11-3 true +kyverno-3-2-6 true ``` ### Next Step diff --git a/docs/smc/servicetemplate_usage_guide.md b/docs/smc/servicetemplate_usage_guide.md deleted file mode 100644 index 8c6bc28..0000000 --- a/docs/smc/servicetemplate_usage_guide.md +++ /dev/null @@ -1,232 +0,0 @@ -# ServiceTemplate Usage Guide - -MES version21112024 - -## Overview - -ServiceTemplates define reusable service configurations that can be deployed across multiple Kubernetes clusters managed by Project 2A. They support different deployment methods, including Helm charts and raw manifests. - -## Usage Patterns - -### 1\. Direct Template Usage - -``` -apiVersion: hmc.mirantis.com/v1alpha1 -kind: ServiceTemplate -metadata: - name: ingress-nginx-v1.11.2 -spec: - helm: - chartName: mirantis-ha-ingress-nginx - chartVersion: 1.11.2 -``` - -### 2\. Multi-Cluster Service Deployment - -``` -apiVersion: hmc.mirantis.com/v1alpha1 -kind: MultiClusterService -metadata: - name: global-nginx -spec: - services: - - name: ingress-nginx-v1.11.2 - config: - controller: - watchIngressWithoutClass: true - clusterSelector: - matchLabels: - hmc.mirantis.com/managed-by: hmc - servicesPriority: 100 -``` - -## Template Management - -### Version Control - -Templates follow semantic versioning (e.g., v1.11.2). Version upgrades can be managed through: - -1. Template chains -2. Direct template updates -3. Multi-cluster service configurations - -### Access Control - -``` -apiVersion: hmc.mirantis.com/v1alpha1 -kind: TemplateManagement -metadata: - name: template-access -spec: - allowedNamespaces: - selector: - matchLabels: - environment: production - serviceTemplateChains: - - name: ingress-nginx-chain - -``` - -## Priority and Conflict Resolution - -* Use `servicesPriority` to manage template precedence -* Lower numbers have higher priority -* Set `stopOnConflict: true` to prevent conflicting deployments - -## Status Monitoring - -Monitor service deployment status through: - -``` -kubectl get multiclusterservice -o yaml -``` - -Status conditions include: - -* Ready -* SveltosClusterProfileReady -* FetchServicesStatusSuccess - -## Best Practices - -* **TBD** - -## Common Operations - -### Create New Template - -``` -kubectl apply -f service-template.yaml -``` - -### Update Existing Service - -``` -kubectl patch multiclusterservice --patch '{"spec":{"services":[{"name":"new-template"}]}}' --type=merge -``` - -### Remove Service - -``` -kubectl delete multiclusterservice -``` - -# ServiceTemplate Additional Notes - -## Template Lifecycle Management - -### ServiceTemplateVersion - -``` -apiVersion: templates.hmc.mirantis.com/v1alpha1 -kind: ServiceTemplateVersion -metadata: - name: example-service-v1 -spec: - template: - name: example-service - version: "1.0.0" - parentVersion: "" - configuration: - helm: - repository: "https://example.com/charts" - chart: "example-service" - version: "1.0.0" - parameters: - required: - - name: replicaCount - type: integer - description: "Number of replicas" - default: 1 - -``` - -### ServiceTemplateUpgrade - -``` -apiVersion: templates.hmc.mirantis.com/v1alpha1 -kind: ServiceTemplateUpgrade -metadata: - name: upgrade-to-v2 -spec: - templateName: example-service - targetVersion: "2.0.0" - strategy: - type: Rolling - rolling: - maxUnavailable: 1 - maxSurge: 1 - validation: - timeout: 5m - rollback: - automatic: true - timeout: 10m -``` - -## Template Chains - -### Chain Definition - -``` -apiVersion: hmc.mirantis.com/v1alpha1 -kind: ServiceTemplateChain -metadata: - name: service-chain -spec: - templates: - - name: service-v1.0.0 - availableUpgrades: - - name: service-v1.1.0 - - name: service-v1.2.0 -``` - -## Integration with ManagedClusters - -### Adding Services to ManagedCluster - -``` -apiVersion: hmc.mirantis.com/v1alpha1 -kind: ManagedCluster -metadata: - name: example-cluster -spec: - template: aws-standalone-cp-v1.1.2 - serviceTemplates: - - name: ingress-nginx-v1.11.2 - config: - controller: - watchIngressWithoutClass: true -``` - -### - -``` - -``` - -## Operational Guidelines (needs further testing) - -### Monitoring Template Status - -``` -# View template status -kubectl get servicetemplate -n hmc-system - -# View upgrade status -kubectl get servicetemplateupgrade -n hmc-system - -# View deployment status across clusters -kubectl get clustersummary -A -``` - -### Health Checks - -``` -healthChecks: - liveness: - initialDelaySeconds: 30 - periodSeconds: 10 - readiness: - initialDelaySeconds: 10 - periodSeconds: 5 -``` diff --git a/docs/template/main.md b/docs/template/main.md index a690f74..74eae96 100644 --- a/docs/template/main.md +++ b/docs/template/main.md @@ -1,16 +1,161 @@ # Templates system -By default, 2A delivers a set of default `ProviderTemplate`, `ClusterTemplate` and `SystemTemplate` objects: +By default, 2A delivers a set of default `ProviderTemplate`, `ClusterTemplate` and `ServiceTemplate` objects: * `ProviderTemplate` - The template containing the configuration of the provider, ex. k0smotron. Cluster-scoped. + The template containing the configuration of the provider (e.g., k0smotron). Cluster-scoped. * `ClusterTemplate` The template containing the configuration of the cluster objects. Namespace-scoped. * `ServiceTemplate` The template containing the configuration of the service to be installed on the managed cluster. Namespace-scoped. All Templates are immutable. You can also build your own templates and use them for deployment along with the -Templates shipped with 2A. +templates shipped with 2A. Below are some examples for each of the templates. + +> EXAMPLE: An example of a `ProviderTemplate` with its status. +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ProviderTemplate +> metadata: +> name: cluster-api-0-0-4 +> spec: +> helm: +> chartSpec: +> chart: cluster-api +> interval: 10m0s +> reconcileStrategy: ChartVersion +> sourceRef: +> kind: HelmRepository +> name: hmc-templates +> version: 0.0.4 +> status: +> capiContracts: +> v1alpha3: "" +> v1alpha4: "" +> v1beta1: "" +> chartRef: +> kind: HelmChart +> name: cluster-api-0-0-4 +> namespace: hmc-system +> config: +> airgap: false +> config: {} +> configSecret: +> create: false +> name: "" +> namespace: "" +> description: A Helm chart for Cluster API core components +> observedGeneration: 1 +> valid: true +> ``` + +> EXAMPLE: An example of a `ClusterTemplate` with its status. +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ClusterTemplate +> metadata: +> name: aws-standalone-cp-0-0-3 +> namespace: hmc-system +> spec: +> helm: +> chartSpec: +> chart: aws-standalone-cp +> interval: 10m0s +> reconcileStrategy: ChartVersion +> sourceRef: +> kind: HelmRepository +> name: hmc-templates +> version: 0.0.3 +> status: +> chartRef: +> kind: HelmChart +> name: aws-standalone-cp-0-0-3 +> namespace: hmc-system +> config: +> bastion: +> allowedCIDRBlocks: [] +> ami: "" +> disableIngressRules: false +> enabled: false +> instanceType: t2.micro +> clusterIdentity: +> kind: AWSClusterStaticIdentity +> name: "" +> clusterNetwork: +> pods: +> cidrBlocks: +> - 10.244.0.0/16 +> services: +> cidrBlocks: +> - 10.96.0.0/12 +> controlPlane: +> amiID: "" +> iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io +> imageLookup: +> baseOS: "" +> format: amzn2-ami-hvm*-gp2 +> org: "137112412989" +> instanceType: "" +> rootVolumeSize: 8 +> controlPlaneNumber: 3 +> extensions: +> chartRepository: "" +> imageRepository: "" +> k0s: +> version: v1.31.1+k0s.1 +> publicIP: false +> region: "" +> sshKeyName: "" +> worker: +> amiID: "" +> iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io +> imageLookup: +> baseOS: "" +> format: amzn2-ami-hvm*-gp2 +> org: "137112412989" +> instanceType: "" +> rootVolumeSize: 8 +> workersNumber: 2 +> description: 'An HMC template to deploy a k0s cluster on AWS with bootstrapped control +> plane nodes. ' +> observedGeneration: 1 +> providerContracts: +> bootstrap-k0smotron: v1beta1 +> control-plane-k0smotron: v1beta1 +> infrastructure-aws: v1beta2 +> providers: +> - bootstrap-k0smotron +> - control-plane-k0smotron +> - infrastructure-aws +> valid: true +> ``` + +> EXAMPLE: An example of a `ServiceTemplate` with its status. +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ServiceTemplate +> metadata: +> name: kyverno-3-2-6 +> namespace: hmc-system +> spec: +> helm: +> chartSpec: +> chart: kyverno +> interval: 10m0s +> reconcileStrategy: ChartVersion +> sourceRef: +> kind: HelmRepository +> name: hmc-templates +> version: 3.2.6 +> status: +> chartRef: +> kind: HelmChart +> name: kyverno-3-2-6 +> namespace: hmc-system +> description: A Helm chart to refer the official kyverno helm chart +> observedGeneration: 1 +> valid: true +> ``` ## Template Life Cycle Management diff --git a/docs/usage/create-multiclusterservice.md b/docs/usage/create-multiclusterservice.md new file mode 100644 index 0000000..837b47b --- /dev/null +++ b/docs/usage/create-multiclusterservice.md @@ -0,0 +1,299 @@ +# Deploy beach-head services using MultiClusterService + +The `MultiClusterService` object is used to deploy beach-head services on multiple matching clusters. + +## Creation + +The `MultiClusterService` object can be created with the following YAML: + +```yaml +apiVersion: hmc.mirantis.com/v1alpha1 +kind: MultiClusterService +metadata: + name: +spec: + clusterSelector: + matchLabels: + : + : + . . . + services: + - template: + name: + namespace: + servicesPriority: 100 + stopOnConflict: false +``` + +## Matching Multiple Clusters + +Consider the following example where 2 clusters have been deployed using ManagedCluster objects: +```sh +➜ ~ kubectl get managedclusters.hmc.mirantis.com -n hmc-system +NAME READY STATUS +dev-cluster-1 True ManagedCluster is ready +dev-cluster-2 True ManagedCluster is ready +➜ ~ +➜ ~ +➜ ~ kubectl get cluster -n hmc-system --show-labels +NAME CLUSTERCLASS PHASE AGE VERSION LABELS +dev-cluster-1 Provisioned 2h41m app.kubernetes.io/managed-by=Helm,helm.toolkit.fluxcd.io/name=dev-cluster-1,helm.toolkit.fluxcd.io/namespace=hmc-system,sveltos-agent=present +dev-cluster-2 Provisioned 3h10m app.kubernetes.io/managed-by=Helm,helm.toolkit.fluxcd.io/name=dev-cluster-2,helm.toolkit.fluxcd.io/namespace=hmc-system,sveltos-agent=present +``` + +> EXAMPLE: +> Spec for `dev-cluster-1` ManagedCluster (only sections relevant to beach-head services): +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> name: dev-cluster-1 +> namespace: hmc-system +> spec: +> . . . +> services: +> - name: kyverno +> namespace: kyverno +> template: kyverno-3-2-6 +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-0 +> servicesPriority: 100 +> stopOnConflict: false +> . . . +> ``` +> +> Spec for `dev-cluster-2` ManagedCluster (only sections relevant to beach-head services): +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> name: dev-cluster-2 +> namespace: hmc-system +> spec: +> . . . +> services: +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-0 +> servicesPriority: 500 +> stopOnConflict: false +> . . . +> ``` + +> NOTE: See [Deploy beach-head Services using Managed Cluster](deploy-services-managedcluster.md) for how to use beach-head services with ManagedCluster. + +Now the following `global-ingress` MultiClusterService object is created with the following spec: + +```yaml +apiVersion: hmc.mirantis.com/v1alpha1 +kind: MultiClusterService +metadata: + name: global-ingress +spec: + clusterSelector: + matchLabels: + app.kubernetes.io/managed-by: Helm + services: + - name: ingress-nginx + namespace: ingress-nginx + template: ingress-nginx-4-11-3 + servicesPriority: 300 + stopOnConflict: false +``` + + +This MultiClusterService will match any CAPI cluster with the label `app.kubernetes.io/managed-by: Helm` and deploy +version 4.11.3 of ingress-nginx service on it. + +### Configuring Custom Values + +Refer to "Configuring Custom Values" in [Deploy beach-head Services using Managed Cluster](./deploy-services-managedcluster.md). + +### Templating Custom Values + +Refer to "Templating Custom Values" in [Deploy beach-head Services using Managed Cluster](./deploy-services-managedcluster.md). + +### Services Priority and Conflict + +The `.spec.servicesPriority` field is used to specify the priority for the services managed by a ManagedCluster or MultiClusterService object. +Considering the example above: + +1. ManagedCluster `dev-cluster-1` manages deployment of kyverno (v3.2.6) and ingress-nginx (v4.11.0) with `servicesPriority=100` on its cluster. +2. ManagedCluster `dev-cluster-2` manages deployment of ingress-nginx (v4.11.0) with `servicesPriority=500` on its cluster. +3. MultiClusterService `global-ingress` manages deployment of ingress-nginx (v4.11.3) with `servicesPriority=300` on both clusters. + +This scenario presents a conflict on both the clusters as the MultiClusterService is attempting to deploy v4.11.3 of ingress-nginx +on both whereas the ManagedCluster for each is attempting to deploy v4.11.0 of ingress-nginx. + +This is where `.spec.servicesPriority` can be used to specify who gets the priority. Higher number means higer priority and vice versa. In this example: +1. ManagedCluster `dev-cluster-1` would lose and its ingress-nginx (v4.11.3) would be deployed on its cluster. +2. ManagedCluster `dev-cluster-2` would win and ingress-nginx (v4.11.0) would be deployed on its cluster. + +> NOTE: If servicesPriority are equal, the first one to reach the cluster wins and deploys its beach-head services. + +## Checking Status + +The status for the MultiClusterService object will show the deployment status for the beach-head services managed +by it on each of the CAPI target clusters that it matches. Consider the same example where 2 ManagedClusters +and 1 MultiClusterService is deployed. + +> EXAMPLE: Status for `global-ingress` MultiClusterService +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: MultiClusterService +> metadata: +> . . . +> name: global-ingress +> resourceVersion: "38146" +> . . . +> spec: +> clusterSelector: +> matchLabels: +> app.kubernetes.io/managed-by: Helm +> services: +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-3 +> servicesPriority: 300 +> stopOnConflict: false +> status: +> conditions: +> - lastTransitionTime: "2024-10-25T08:36:24Z" +> message: "" +> reason: Succeeded +> status: "True" +> type: SveltosClusterProfileReady +> - lastTransitionTime: "2024-10-25T08:36:24Z" +> message: MultiClusterService is ready +> reason: Succeeded +> status: "True" +> type: Ready +> observedGeneration: 1 +> services: +> - clusterName: dev-cluster-2 +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-10-25T08:36:35Z" +> message: | +> cannot manage chart ingress-nginx/ingress-nginx. ClusterSummary p--dev-cluster-2-capi-dev-cluster-2 managing it. +> reason: Failed +> status: "False" +> type: Helm +> - lastTransitionTime: "2024-10-25T08:36:25Z" +> message: 'Release ingress-nginx/ingress-nginx: ClusterSummary p--dev-cluster-2-capi-dev-cluster-2 +> managing it' +> reason: Conflict +> status: "False" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> - clusterName: dev-cluster-1 +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-10-25T08:36:24Z" +> message: "" +> reason: Provisioned +> status: "True" +> type: Helm +> - lastTransitionTime: "2024-10-25T08:36:25Z" +> message: Release ingress-nginx/ingress-nginx +> reason: Managing +> status: "True" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> ``` + +The status under `.status.services` shows a conflict for `dev-cluster-2` as expected because the MultiClusterService has a lower priority. +Whereas, it shows provisioned for `dev-cluster-1` because the MultiClusterService has a higher priority. + +> EXAMPLE: Status for `dev-cluster-1` ManagedCluster (only sections relevant to beach-head services): +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> name: dev-cluster-1 +> namespace: hmc-system +> . . . +> spec: +> . . . +> services: +> - name: kyverno +> namespace: kyverno +> template: kyverno-3-2-6 +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-0 +> servicesPriority: 100 +> stopOnConflict: false +> . . . +> status: +> . . . +> services: +> - clusterName: dev-cluster-1 +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-10-25T08:36:35Z" +> message: | +> cannot manage chart ingress-nginx/ingress-nginx. ClusterSummary global-ingress-capi-dev-cluster-1 managing it. +> reason: Provisioning +> status: "False" +> type: Helm +> - lastTransitionTime: "2024-10-25T07:44:43Z" +> message: Release kyverno/kyverno +> reason: Managing +> status: "True" +> type: kyverno.kyverno/SveltosHelmReleaseReady +> - lastTransitionTime: "2024-10-25T08:36:25Z" +> message: 'Release ingress-nginx/ingress-nginx: ClusterSummary global-ingress-capi-dev-cluster-1 +> managing it' +> reason: Conflict +> status: "False" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> ``` + +The status under `.status.services` for ManagedCluster `dev-cluster-1` shows that it is managing kyverno but unable to manage ingress-nginx because +another object with higher priority is managing it, so it shows a conflict instead. + +> EXAMPLE: Status for `dev-cluster-2` ManagedCluster (only sections relevant to beach-head services): +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> name: dev-cluster-2 +> namespace: hmc-system +> resourceVersion: "30889" +> . . . +> spec: +> . . . +> services: +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-0 +> servicesPriority: 500 +> stopOnConflict: false +> . . . +> status: +> . . . +> services: +> - clusterName: dev-cluster-2 +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-10-25T08:18:22Z" +> message: "" +> reason: Provisioned +> status: "True" +> type: Helm +> - lastTransitionTime: "2024-10-25T08:18:22Z" +> message: Release ingress-nginx/ingress-nginx +> reason: Managing +> status: "True" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> ``` + +The status under `.status.services` for ManagedCluster `dev-cluster-2` shows that it is managing ingress-nginx as expected since it has a higher priority. + +## Parameter List + +Refer to "Parameter List" in [Deploy beach-head Services using Managed Cluster](./deploy-services-managedcluster.md). diff --git a/docs/usage/deploy-services-managedcluster.md b/docs/usage/deploy-services-managedcluster.md new file mode 100644 index 0000000..f6e8ea1 --- /dev/null +++ b/docs/usage/deploy-services-managedcluster.md @@ -0,0 +1,291 @@ +# Deploy beach-head Services using Managed Cluster + +## Deployment + +Beach-head services can be deployed on a managed cluster (i.e., target cluster) using the `ManagedCluster` object. +Consider the following example: + +> EXAMPLE: `ManagedCluster` object for AWS Infrastructure Provider with beach-head services +> +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> name: my-managed-cluster +> namespace: hmc-system +> spec: +> config: +> clusterNetwork: +> pods: +> cidrBlocks: +> - 10.244.0.0/16 +> services: +> cidrBlocks: +> - 10.96.0.0/12 +> controlPlane: +> iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io +> instanceType: "" +> controlPlaneNumber: 3 +> k0s: +> version: v1.27.2+k0s.0 +> publicIP: false +> region: "" +> sshKeyName: "" +> worker: +> amiID: "" +> iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io +> instanceType: "" +> workersNumber: 2 +> template: aws-standalone-cp-0-0-3 +> credential: aws-credential +> services: +> - template: kyverno-3-2-6 +> name: kyverno +> namespace: kyverno +> - template: ingress-nginx-4-11-3 +> name: ingress-nginx +> namespace: ingress-nginx +> servicesPriority: 100 +> stopOnConflict: false +> ``` + +In the example above the following fields are relevant to the deployment of beach-head services: + +```yaml + . . . + services: + - template: kyverno-3-2-6 + name: kyverno + namespace: kyverno + - template: ingress-nginx-4-11-3 + name: ingress-nginx + namespace: ingress-nginx + servicesPriority: 100 + stopOnConflict: false + template: aws-standalone-cp-0-0-3 +``` + +> NOTE: +> Refer to [Parameter List](#parameter-list) for more detail about these fields. + +This example `ManagedCluster` object will deploy kyverno and ingress-nginx referred to by their +service templates respectively on the target cluster it is managing. See the example below for +the service template for kyverno. + +> EXAMPLE: `ServiceTemplate` object for kyverno version 3.2.6 +> The `ServiceTemplate` for kyverno: +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ServiceTemplate +> metadata: +> name: kyverno-3-2-6 +> namespace: hmc-system +> spec: +> helm: +> chartSpec: +> chart: kyverno +> interval: 10m0s +> reconcileStrategy: ChartVersion +> sourceRef: +> kind: HelmRepository +> name: hmc-templates +> version: 3.2.6 +> ``` + +The `hmc-templates` helm repository hosts the actual chart for kyverno version 3.2.6. +For more details see the [Bring your own Templates](../template/byo-templates.md) guide. + +### Configuring Custom Values + +Helm values can be passed to each beach-head services with the `.spec.services[].values` field in the `ManagedCluster` or `MultiClusterService` object. + +EXAMPLE: +```yaml +apiVersion: hmc.mirantis.com/v1alpha1 +kind: ManagedCluster +metadata: + . . . + name: my-managedcluster + namespace: hmc-system + . . . +spec: + . . . + services: + . . . + - name: cert-manager + namespace: cert-manager + template: cert-manager-1-16-1 + values: | + crds: + enabled: true + - name: motel-regional + namespace: motel + template: motel-regional-0-1-1 + values: | + victoriametrics: + vmauth: + ingress: + host: vmauth.hmc0.example.net + credentials: + username: motel + password: motel + grafana: + ingress: + host: grafana.hmc0.example.net + cert-manager: + email: mail@example.net + - template: ingress-nginx-4-11-3 + name: ingress-nginx + namespace: ingress-nginx + . . . +``` + +The example above shows how custom values are specified for each beach-head service. + +### Templating Custom Values + +Using Sveltos templating feature, we can also write templates which can be useful for automatically fetching pre-existing information within the cluster. + +EXAMPLE: +```yaml +apiVersion: hmc.mirantis.com/v1alpha1 +kind: ManagedCluster +metadata: + name: my-managedcluster + namespace: hmc-system +spec: + . . . + servicesPriority: 100 + services: + - template: motel-0-1-0 + name: motel + namespace: motel + - template: myappz-0-3-0 + name: myappz + namespace: myappz + values: | + controlPlaneEndpointHost: {{ .Cluster.spec.controlPlaneEndpoint.host }} + controlPlaneEndpointPort: "{{ .Cluster.spec.controlPlaneEndpoint.port }}" +``` + +In the example above the host and port information is being fetched from the spec of the CAPI cluster that belongs to this `ManagedCluster`. + +## Checking status + +The `.status.services` field of the `ManagedCluster` object shows the status for each of the beach-head services. + +> EXAMPLE: Status for beach-head services deployed with `ManagedCluster` +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> generation: 1 +> name: wali-aws-dev +> namespace: hmc-system +> . . . +> spec: +> . . . +> services: +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-3 +> servicesPriority: 100 +> stopOnConflict: false +> status: +> . . . +> observedGeneration: 1 +> services: +> - clusterName: my-managed-cluster +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-12-11T23:03:05Z" +> message: "" +> reason: Provisioned +> status: "True" +> type: Helm +> - lastTransitionTime: "2024-12-11T23:03:05Z" +> message: Release kyverno/kyverno +> reason: Managing +> status: "True" +> type: kyverno.kyverno/SveltosHelmReleaseReady +> - lastTransitionTime: "2024-12-11T23:03:05Z" +> message: Release ingress-nginx/ingress-nginx +> reason: Managing +> status: "True" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> ``` + +Based on the status above both kyverno and ingress-nginx should be installed in their respective namespaces on the target cluster. + +```sh +➜ ~ kubectl get pod -n kyverno +NAME READY STATUS RESTARTS AGE +kyverno-admission-controller-96c5d48b4-sg5ts 1/1 Running 0 2m39s +kyverno-background-controller-65f9fd5859-tm2wm 1/1 Running 0 2m39s +kyverno-cleanup-controller-848b4c579d-ljrj5 1/1 Running 0 2m39s +kyverno-reports-controller-6f59fb8cd6-s8jc8 1/1 Running 0 2m39s +➜ ~ +➜ ~ +➜ ~ kubectl get pod -n ingress-nginx +NAME READY STATUS RESTARTS AGE +ingress-nginx-controller-cbcf8bf58-zhvph 1/1 Running 0 24m +``` + +> NOTE: +> * Refer to Step 7 of [Create Managed Cluster](../usage/create-managed-cluster.md/step-7-retrieve-kubernetes-configuration-of-your-managed-cluster) guide for how to access the target cluster. +> * Refer to [Service Templates](../servicetemplates.md) for more detail on what statuses are reported. + +## Removing beach-head services + +To remove a beach-head service simply remove its entry from `.spec.services`. +The example below removes `kyverno-3-2-6` so its status also removed from `.status.services`. + +> EAMPLE: Showing removal of `kyverno-3-2-6` from `ManagedCluster` +> ```yaml +> apiVersion: hmc.mirantis.com/v1alpha1 +> kind: ManagedCluster +> metadata: +> . . . +> generation: 2 +> name: wali-aws-dev +> namespace: hmc-system +> . . . +> spec: +> . . . +> services: +> - name: ingress-nginx +> namespace: ingress-nginx +> template: ingress-nginx-4-11-3 +> servicesPriority: 100 +> stopOnConflict: false +> status: +> . . . +> observedGeneration: 2 +> services: +> - clusterName: wali-aws-dev +> clusterNamespace: hmc-system +> conditions: +> - lastTransitionTime: "2024-12-11T23:15:45Z" +> message: "" +> reason: Provisioned +> status: "True" +> type: Helm +> - lastTransitionTime: "2024-12-11T23:15:45Z" +> message: Release ingress-nginx/ingress-nginx +> reason: Managing +> status: "True" +> type: ingress-nginx.ingress-nginx/SveltosHelmReleaseReady +> ``` + +## Parameter List + +| Parameter | Example | Description | +|------------------------------|------------------------|-----------------------------------------------------------------------------------------------| +| `.spec.servicesPriority` | `100` | Sets the priority for the beach-head services defined in this spec (default: `100`) | +| `.spec.stopOnConflict` | `false` | Stops deployment of beach-head services upon first encounter of a conflict (default: `false`) | +| `.spec.services[].template` | `kyverno-3-2-6` | Name of the `ServiceTemplate` object located in the same namespace | +| `.spec.services[].name` | `my-kyverno-release` | Release name for the beach-head service | +| `.spec.services[].namespace` | `my-kyverno-namespace` | Release namespace for the beach-head service (default: `.spec.services[].name`) | +| `.spec.services[].values` | `replicas: 3` | Helm values to be used with the template while deployed the beach-head services | +| `.spec.services[].disable` | `false` | Disable handling of this beach-head service (default: `false`) |