EDN based templating for Kubernetes

Manage kubernetes resources declaratively with edn.

Pre-alpha status - API subject to change at any time.

Experimental project using GraalVM to compile Clojure into a native executable to achieve tolerable performace as a command line tool.

Motivation

Distributing services with kubernetes eventually requires some form of templating and helm is fast becoming a popular choice for this task. Helm uses a text-based templating system similar to mustache to render yaml files. By text-based I mean that the templates are treated as large strings and processed before being read as yaml. The templating also supports some functions and a unix pipe-like syntax for composing them. This is known as an external domain specific language (DSL) becuase the templating language uses a completely custom language (not YAML).

Edn is a data format with many features that make it uniquely suited to writing domain specific languages. Edn supports more types than json and yaml as well as a tag system for user extension.

Kubeedn is an experiment. It is a thin wrapper around kubectl that enables authoring kubernetes configurations via edn.

Building

You need the following tools installed:

• Java
• Clojure tools cli
• GraalVM (native-image must be available in $PATH)
• make
• kubectl (only required at runtime)

Build the binary with

make native-image

You can also run the code using the clojure cli.

clojure -m kubeedn.main --help

NAME:
 kubeedn - Write kubernetes manifests with edn

USAGE:
 kubeedn [global-options] command [command options] [arguments...]

VERSION:
 0.0.1-SNAPSHOT

COMMANDS:
   transform, xf        Transforms edn to yaml
   apply, a             Kubectl apply after preprocessing with kubeedn

GLOBAL OPTIONS:
   -?, --help

Usage

Currely only supports running on a single files. Directories will likely be supported in the future.

Transform

These examples assume they are run from this directory. They also assume you have kubectl available and configured to talk to a kubernetes cluster. The example manifest also requires support for services of type “LoadBalancer”.

The simplest usage is to view generated yaml for an edn manifest. In manifests/nginx.edn you will find some resources similar to this:

{:apiVersion "apps/v1"
 :kind "Deployment"
 :metadata {:name "kubeedn-nginx"
            :labels {:app "kubeedn-nginx"}}
 :spec {:selector {:matchLabels {:app "kubeedn-nginx"}}
        :minReadySeconds 5
        :template {:metadata {:labels {:app "kubeedn-nginx"}}
                   :spec {:containers [{:name "nginx"
                                        :image "nginx:1.7.9"
                                        :ports [{:containerPort 80}]}]}}}}

I’ve shown a single object, but you can have multiple by placing them inside an edn vector.

[
 {;; ...
  :apiVersion "apps/v1"
  :kind "Deployment"},
 {;; ...
  :apiVersion "v1"
  :kind "Service"}
 ]

This will translate into multiple yaml objects separated by “\n—\n”.

kubeedn transform -f manifests/nginx.edn

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubeedn-nginx
  labels: {app: kubeedn-nginx}
spec:
  selector:
    matchLabels: {app: kubeedn-nginx}
  minReadySeconds: 5
  template:
    metadata:
      labels: {app: kubeedn-nginx}
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - {containerPort: 80}

---
apiVersion: v1
kind: Service
metadata:
  name: kubeedn-nginx
  labels: {app: kubeedn-nginx}
spec:
  type: LoadBalancer
  ports:
  - {port: 80, protocol: TCP}
  selector: {app: kubeedn-nginx}

Ok, lets redirect the output to a file and create the objects in a cluster with kubectl. xf is a shorter alias for transform.

kubeedn xf -f manifests/nginx.edn > nginx.yaml
kubectl apply -f nginx.yaml

deployment.apps/kubeedn-nginx created
service/kubeedn-nginx created

We’ll delete the resources to reset our state for the next example.

kubectl delete -f nginx.yaml

deployment.apps "kubeedn-nginx" deleted
service "kubeedn-nginx" deleted

You can also pipe the output from kubeedn directly to kubectl apply -f since kubectl supports reading from stdin.

kubeedn xf -f manifests/nginx.edn | kubectl apply -f -

deployment.apps/kubeedn-nginx created
service/kubeedn-nginx created

And deleting them again.

kubeedn xf -f manifests/nginx.edn | kubectl delete -f -

deployment.apps "kubeedn-nginx" deleted
service "kubeedn-nginx" deleted

Apply

Wrapper for `kubectl apply`.

kubeedn apply -f manifests/nginx.edn

deployment.apps/kubeedn-nginx created
service/kubeedn-nginx created

And deleting them. Using transform becuase delete is not implemented yet.

kubeedn xf -f manifests/nginx.edn | kubectl delete -f -

deployment.apps "kubeedn-nginx" deleted
service "kubeedn-nginx" deleted

You can pass extra flags directly to kubectl using the --kubectl flag.

kubeedn apply -f manifests/nginx.edn --kubectl '--prune -l app=kubeedn-nginx'

deployment.apps/kubeedn-nginx created
service/kubeedn-nginx created

One lovely feature of edn is the prefix tag #_ that comments out an entire form. You can use this to easily comment out an entire resource. Combined with the --prune flag you can make kubectl behave similarly to terraform where it will “clean up” any resources that are no longer required.

The service is commented out in ./manifests/nginx-no-service.edn.

[{:apiVersion "apps/v1"
  :kind "Deployment"
  :metadata {:name "kubeedn-nginx"
             :labels {:app "kubeedn-nginx"}}
  :spec {:selector {:matchLabels {:app "kubeedn-nginx"}}
         :minReadySeconds 5
         :template {:metadata {:labels {:app "kubeedn-nginx"}}
                    :spec {:containers [{:name "nginx"
                                         :image "nginx:1.7.9"
                                         :ports [{:containerPort 80}]}]}}}}
 #_{:apiVersion "v1"
  :kind "Service"
  :metadata {:name "kubeedn-nginx"
             :labels {:app "kubeedn-nginx"}}
  :spec {:type "LoadBalancer"
         :ports [{:port 80 :protocol "TCP"}]
         :selector {:app "kubeedn-nginx"}}}]

kubeedn apply -f manifests/nginx-no-service.edn --kubectl '--prune -l app=kubeedn-nginx'

deployment.apps/kubeedn-nginx unchanged
service/kubeedn-nginx pruned

Roadmap

Some things I’m considering implementing:

• [ ] directories of edn manifests (only single file supported currently)
• [X] pass through commands to kubectl (e.g. apply)
• [ ] chart-like “packages” configured with edn
• [ ] functionality via edn tags and embedded lispy scripting language

License

The MIT License (MIT)

Copyright © 2018 Jedidiah T Clinger

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN