BOSH Release for Helm and Kubectl

Purpose

The purpose of this bosh release is to offer a bosh deployment for Helm chart and Kubectl product You can declare in your deployment helm repositories and helm charts, a default storage class and ingress rules. This bosh release should be use as an errand to apply charts. It uses Helm V3.

Usage

see web site: https://orange-cloudfoundry.github.io/helm-kubectl-boshrelease/ These bosh release is composed by 1 jobs

• action
  • it creates namespace
  • it applies kubectl command
  • it adds helm repository
  • it creates helm chart instance, or any helm command
  • it creates secret
  • it creates basic auth secret
  • it can execute any shell

During undeploy of the bosh release every thing created by action will be deleted.

Upload the last release

To use this bosh release, first upload it to your bosh: Note: change the index the helm-kubectl-[index].yml to the last version of the bosh release

bosh target BOSH_HOST
git clone https://github.com/orange-cloudfoundry/helm-kubectl-boshrelease
cd helm-kubectl-boshrelease
bosh upload release releases/helm-kubectl/helm-kubectl-1.yml

Base deployment

#Deployment Identification
name: cfcr-addon

#Features Block

#Releases Block
releases:
- name: helm-kubectl
  version: latest

#Stemcells Block
stemcells:
- alias: default
  os: ubuntu-xenial
  version: latest

#Update Block
update:
  canaries: 1
  max_in_flight: 2
  canary_watch_time: 15000-30000
  update_watch_time: 15000-300000

#Instance Groups Block
instance_groups:
- name: cfcr-helm-addons
  vm_type: small
  stemcell: default
  networks:
  - name: ((network))
  azs: [z1]
  instances: 1
  jobs:
  - name: action
    release: helm-kubectl
    properties:
      kubernetes:
        host: ((kubernetes.host))
        port: ((kubernetes.port))
        cluster_ca_certificate: ((kubernetes.cluster_ca_certificate))
        password: ((kubernetes-password))
        default_storageclass: ((default_storageclass))
      proxy:
        https: ((https_proxy))
        http: ((http_proxy))
        noproxy: ((no_proxy))
      repository_mirror:
        enabled: true
        url: https://((helm_mirror_url))
      actions:
      - type: helm_repo
        name: stable
        url: https://kubernetes-charts.storage.googleapis.com/
      - type: helm_repo
        name: incubator
        url: https://kubernetes-charts-incubator.storage.googleapis.com/

Action job

Action job provide an array of action. They are apply during bosh errand usage or on each deploy in case of run_on_each_deploy=true How it works internally: Each action will be converted into kubectl or helm command

add namespace

As helm_V3 doesn't create namespace, you can create namespace by using this kind of operator.

basic example:

 - type: replace
   path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
   value:
     type: namespace
     name: my-namespace    

example with annotations and labels:

 - type: replace
   path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
   value:
     type: namespace
     name: my-namespace
     annotations:
     - name: myannotation
       value: hello
     labels:
     - name: mylabel
       value: hello
         

Caution: During bosh delete-deployment the created namespace will be deleted. So be careful do not create kube-system namespace with this kind of operator.

add helm repository

Some time the chart need to be loaded from a specific helm repository. You can do that with this operator.

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: helm_repo
    name: gitlab
    url: https://charts.gitlab.io                          

add helm chart

Helm chart deployment can be customize by properties or by value file

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: helm_chart
    name: gitlab
    chart:  gitlab/gitlab
    namespace: gitlab
    version: ((gitlab-version))
    properties:
    - name: gitlab.unicorn.ingress.tls.secretName
      value: release-gitlab-tls
    - name: unicorn.ingress.enabled
      value: false

    values_file_content:
      global:
        ## GitLab operator is Alpha. Not for production use.
        operator:
          enabled: false
        ## doc/installation/deployment.md#deploy-the-community-edition
        edition: ce

        ## doc/charts/globals.md#gitlab-version
        # gitlabVersion: master

        ## doc/charts/globals.md#application-resource
        application:
          create: false
        ...
                          

By default the helm type will perform

Caution: During bosh delete-deployment the created instance of chart will be deleted.

Helm sample using custom command:

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: helm_chart
    name: gitlab
    chart:  gitlab/gitlab
    namespace: gitlab
    version: ((gitlab-version))
    cmd: test # override default helm command (ie: 'upgrade') 
    options: # override default options (ie: '--install --atomic --cleanup-on-fail') and reset it

add kubectl cmd

example of use with an apply deployment

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: kubectl
    name: "deploy-k8sdash"
    cmd: "apply"
    options: ""
    content:
      kind: Deployment
      apiVersion: apps/v1
      metadata:
        name: k8dash
        namespace: kube-system
      spec:
        replicas: 1
        selector:
          matchLabels:
            k8s-app: k8dash
        template:
          metadata:
            labels:
              k8s-app: k8dash
          spec:
            containers:
            - name: k8dash
              image: herbrandson/k8dash:latest
              ports:
              - containerPort: 4654
              livenessProbe:
                httpGet:
                  scheme: HTTP
                  path: /
                  port: 4654
                initialDelaySeconds: 30
                timeoutSeconds: 30
            nodeSelector:
              'beta.kubernetes.io/os': linux

example of use with direct apply on content from internet :

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: kubectl
    name: "crd-for-cert-manager"
    cmd: "apply"
    options: "-f https://github.com/jetstack/cert-manager/releases/download/v((cert-manager-version))/cert-manager-no-webhook.yaml"

example of use to produce a config map with very large content:

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=kubectl/properties/commands/-
  value:
    name: "cm-grafana-k8s-master-node-exporter-dashboard"
    cmd: "replace"
    options: " --force --save-config=false "
    apply:
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: dash-k8s-all-node-exporter
        namespace: monitoring
        labels:
          grafana_dashboard: '1'
      data:
        grafana_k8d_all_node_exporter_dashboard.json: |
          {
            "annotations": {
              "list": [
                {
                  "builtIn": 1,
                  ....

add secret

This action will encode in base64 the content of value and create a K8S secret in the namespace. By default the type of the secret is generic but it can be override by secret_type

example of use:

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: secret
    name: cloud-credentials
    namespace: velero
    data:
    - name: cloud
      value: |
        [default]
        aws_access_key_id = backup_remote_s3_access_key_id
        aws_secret_access_key = ((backup_remote_s3_secret_access_key))

add secret for basic auth

This action will encode in base64 the content of value and create a K8S secret in the namespace.

example of use:

- type: replace
  path: /instance_groups/name=cfcr-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: basic_auth_secret
    name: mybasicauth
    namespace: traefik
    user: admin
    password: ((mypassword))

add exec action

This action let user to use kubelet or helm or kustomise in shell to perform any shell script.

example:

- type: replace
  path: /instance_groups/name=k8s-helm-addons/jobs/name=action/properties/actions/-
  value:
    type: exec
    cmd: |
      cat << EOF > /tmp/coredns.yml
      ((coredns_clusterrole))
      ---
      ((coredns_clusterrolebinding))
      ---
      ((coredns_configmap))
      ---
      ((coredns_deployment))
      ---
      ((coredns_service))
      EOF
      kubectl apply -f  /tmp/coredns.yml

Development

As a developer of this release, create new releases and upload them:

bosh create release --force && bosh -n upload release

Final releases

To share final releases:

bosh create release --final

By default the version number will be bumped to the next major number. You can specify alternate versions:

bosh create release --final --version 2.1

After the first release you need to contact Dmitriy Kalinin to request your project is added to https://bosh.io/releases (as mentioned in README above).