-
Notifications
You must be signed in to change notification settings - Fork 15
Copper DSL
Copper DSL is a simple language that's focused on fetching values from configuration files and checking their validity. It has built-in functionality to deal with IP Addresses, Semantic versioning of components and basic string manipulation.
Copper files contain the Copper DSL script. They have text files and have a .cop extension. You can use any text editor to edit them.
There is an extension for VisualStudio Code that provides syntax highlighting for Copper files. This extension is under active development and doesn't support Copper DSL's full syntax.
A rule is like a test you would like to run against your configuration file. Just like code unit tests, it's better to keep the rules focused on one specific area of the configuration file and give then relevant names. A Copper file can contain as many rules as you like.
A rule must have a name. Names should begin with a letter and can contain any alphanumerical characters.
A rule must have an Action. Action is what should happen if the rule fails: An ensure action means failure of the rule will fail the validation. A warn action means a warning is shown next to the failed rule but the validations will pass.
A condition is a boolean logic that should be true for the rule to pass.
rule NAME (warn | ensure) {
CONDITION
}
Example
rule foo ensure {
1 = 1
}
Defines a rule called foo which ensures the statement 1 = 1 returns true
You can define variables in Copper files as a way to avoid repeating the same things over and over again. For example, you can keep your the valid range of ports in a variable and use that variable in different rules. In Copper DSL, variables are more like constants in other languages and cannot be changed once set.
var VARIABLE = VALUE
Example
var foo = 1 var bar = "hello" var valid_ports = (8000..9000)
Using variables in a rule
rule bar warn {
8050 in valid_ports == true
}
Copper files can be commented. Copper supports the Java comment syntax:
rule foo warn { // this is a single inline comment
1 = 1 /* this is a single line block comment */
}
/* we are going to comment this part off
rule bar ensure {
false = true
}*/
The condition inside of a rule is usually made up of a value compared against another value. The result of this comparison is either true or false.
The comparison operation can be one of the following:
| Operation | Meaning |
|---|---|
= or ==
|
Left side is equal the right side |
> |
Left side is greater than the right side |
< |
Left side is less than the right side |
>= |
Left side is greater than or equal the right side |
<= |
Left side is less than or equal the right side |
!= |
Left side is not equal the right side |
in |
Right side is included in the left side (only for sets and ranges) |
Comparisons can be combined with and and or to make up more complex conditions.
Example
rule ComplexRulesAreUs warn {
2 > 1 and 3 == 3 or 2 != 8 and
8 in [1,2,3,4]
}
| Boolean Operand | Meaning |
|---|---|
and, & or &&
|
Boolean AND |
or, | or ||
|
Boolean OR |
Copper DSL supports the following data types:
A number is integer or decimal.
Example
var my_int = 12
rule foo ensure {
my_int > 11.43
}
Strings are wrapped in double quotes ".
Example
var a_string = "foo"
Strings have the following attributes:
Returns the length of the string: "foo".count will return 3.
Replaces text in the string using the regular expression pattern given.
For example "abc".gsub("b", "!") will return "a!c".
Returns the character at the given index: "abc".at(2) returns "b".
Splits the string into an array: "foo/bar/baz".split("/") returns ["foo", "bar", "baz"].
Arrays can contain any number of values. Arrays can hold values of different types. An array is wrapped in [ and ] and each item is separated by a ,.
Example
var my_array = [1,2,3,4]
Arrays have the following attributes:
Returns the number of items in the array: [1,2,3].count will return 3.
Returns the first item of the array: ["foo","bar",45].first will return "foo".
Returns the last item of the array: ["foo","bar",45].first will return 45.
Returns the item at the given index: [1, "item 2", "third item", 4].at(2) returns "item 2".
Returns true if the given item can be found in the array: [1,2,"foo"].contains(2) will return true.
Removes duplicates from the array and returns a new array: [1,2,3,2,1].unique will return [1,2,3]
Runs each item of a string only array through a regular expression and returns the item with the given index of the regexp:
["name1:tag1", "name2:tag2", "name3:tag3"].extract(".*:(.*)", 1) will return ["tag1", "tag2", "tag3"]. The number 1 in this case refers to the regexp group.
Another example
["path1/image1:tag1", "path2/image2:tag2"].extract(".*\/(.*):.*", 2) will return ["image1", "image2", "image3"].
Converts each element of an array into a different data type. For example this can be used to convert an array of strings into Image data type.
["quay.io/mysql:1.2.3", "ubuntu:3.2.1"].as(:image) returns an array of Image data type (see below).
Returns an array by picking an attributes off of each item of the array. For example this can be used to pick the tag attribute of an array of Image.
The example below returns the length of each element of an array:
["a", "xo", "foo"].pick(:count) returns [1, 2, 3]
pick takes in the name of the attribute to pick in the form of a : followed by the attribute name. For example to pick the tag attribute you can use pick(:tag).
You can use = or == to compare two arrays. This will return true if both arrays contain the same items but ignores the ordering of the items. For example:
[1,2,3] == [2,3,1] while [1,2,3] != [1,2,3,4].
You can use the in comparison for arrays: 1 in [1,2,3] is true and "foo" in ["bar", "fuzz"] is false.
Range contains all the numbers between two numbers. Ranges are wrapped in ( and ) with .. between the low and the high numbers. Range is inclusive of both ends.
Example
var the_range = (1..10)
Returns true if the given item can be found in the range: (1..10).contains(1) will return true.
You can use the in comparison for ranges: 10 in (1..10) is true and 13 in (23..45) is false.
Copper DSL supports a growing set of configuration specific data types. Currently this includes the following:
An IPAddress can hold an IP address and/or subnet. You can use IPAddress to check various things about an IPAddress, like it's range, inclusion of other IP addresses, its class and more.
Example
var internal = ipaddress("62.0.0.0/24")
var front_end = ipaddress("62.0.2.45")
IPAddress has the following attributes:
Returns the first IP address in a range: ipaddress("10.0.0.0/24").first will return ipaddress("10.0.0.1")
Returns the last IP address in a range: ipaddress("10.0.0.0/24").last will return ipaddress("10.0.0.254")
Returns the IP address and the prefix: ipaddress("10.0.0.1").full_address will return "10.0.0.1/32"
Returns the IP address without the prefix: ipaddress("172.16.10.1/24").address will return "172.16.10.1"
Returns the IP netmask: ipaddress("10.0.0.0/8").netmask will return "255.0.0.0"
Returns an array of IP address octets: ipaddress("172.16.10.1").octets will return [172, 16, 10, 1]
Returns the IP address prefix without the address: ipaddress("172.16.10.1/24").prefix will return 8
Returns true if the given IP address is a network address. ipaddress("10.0.0.0/24").is_network will return true while ipaddress("10.0.0.1/32").is_network returns false.
Returns true if the given IP address is a local loopback address. ipaddress("127.0.0.1").is_loopback will return true.
Returns true if the given IP address is a multicast address. ipaddress("224.0.0.1/32").is_multicast will return true.
Returns true if the given IP address is a class A IP address. ipaddress("10.0.0.1/24").is_class_a will return true.
Inclusion of an IP address in a network IP range can be checked using the in comparison. For example ipaddress("10.1.1.32") in ipaddress("10.1.1.0/24") returns true.
Returns true if the given IP address is a class A IP address. ipaddress("172.16.10.1/24").is_class_b will return true.
Returns true if the given IP address is a class A IP address. ipaddress("192.168.1.1/30").is_class_c will return true.
Semver holds and parses a string as a Semantic version. This allows support of semantic versioning and checks.
Example
var mysql_version = semver("6.5.0")
var web_server = semver("1.2.4-pre")
Returns the major part of the version number: semver("6.5.7").major returns "6".
Returns the minor part of the version number: semver("6.5.7").major returns "5".
Returns the patch part of the version number: semver("6.5.7").major returns "7".
Returns the build part of the version number if available: semver("3.7.9-pre.1+revision.15723").major returns "revision.15623".
Returns the pre part of the version number if available: semver("3.7.9-pre.1+revision.15723").major returns "pre.1".
Checks if a semver satisfies a Pessimistic version comparison: semver("1.6.5").satisfies("~> 1.5") returns true.
You can use <, >, =, ==, <=, >= and != comparisons between two semvers.
Image holds a Docker image path and lets you access its different parts. It also understands some of the particular attributes of docker images (like no registry name means DockerHub or no tag means latest).
For example, you can parse a string containing an image name to an Image like this:
var i = image("quay.io/cloud66/mysql:5.6.1") this will let you access the image name constituents:
i.registry or i.tag. The Image type, combined with as and pick will make a powerful tool for inspecting images used in a configuration file.
Returns the registry name of the image. It will return index.docker.io if no registry is available in the image name.
Returns the name of the image. It will append library/ to the beginning of the image name if no namespace is available on the image name (DockerHub image names). For example, ubuntu:1.2.3 will return library/ubuntu as name.
Returns the tag of the image. It will return latest if no tag is available on the image. For example mysql will return latest as the tag.
Returns the URL for the registry, including the scheme. For example, quay.io/ubuntu:1.2.3 returns https://quay.io as registry_url.
Returns the Fully Qualified Image Name. This includes the scheme. For example ubuntu will return https://index.docker.io/library/ubuntu:latest.
In most cases, values read from a configuration file are strings. In order for them to be usable with Copper DSL's complex data types, you can read them as different types using the as function.
As function takes in a type name which is a : followed by the type name. For example to convert a string into a Semver use :semver in the as function: "1.2.3".as(:semver)
Example
Here, we are assuming the value of the `mysql_version` variable is a string `"5.6.7"`:
mysql_version.as(:semver).satisfies("~> 5.6")
The full name of the configuration file used in a check is available in the Copper DSL as filename. filename is a Filename data type with the following attributes:
Returns the path to the configuration file (excluding the filename). For example samples/test.yml returns samples.
Returns the filename of the configuration file (excluding the path). For example samples/test.yml returns test.
Returns the file extension of the configuration file (including the leading .). For example samples/test.yml returns .yml.
Returns the full filename of the configuration file. For example samples/test.yml returns samples/test.yml.
Returns the full expanded file path of the configuration file. For example samples/test.yml will return (depending on the absolute location of the file) something like /Users/john/projects/tests/kubernetes/samples/test.yml.
Copper DSL uses JSONPath format to read values from a configuration file. For any configuration file format, the content is first read and converted in to JSON which makes it possible to use JSONPath to find nodes and attributes in the configuration file.
The fetch function accepts the JSONPath and returns an array of all matching nodes and attributes in the configuration.
This is a YAML configuration file used in the following examples
apiVersion: v1
kind: Service
metadata:
namespace: foobar
name: foo-svc
annotations:
cloud66.com/snapshot-uid: 123-456-789
cloud66.com/snapshot-gitref: abcd
labels:
app: foo
tier: bar
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8090
- port: 8100
targetPort: 8100
- port: 5000
selector:
app: foo
tier: bar
To fetch the value of type under spec (which is NodePort in the file above), we can use the following JSONPath format:
fetch("$.spec.type") // will return ["NodePort"]
To return all the targetPort values under spec.port you can use attribute selectors:
fetch("$.spec.ports..targetPort") // will return [8090, 8100]
To return the value of targetPort for the 8080 port (8090 in the example above) you can use the filters:
fetch("$.spect.ports[?(@.port == 8080)]") // will return [8090]
You can use the JSONPath reference as a syntax guideline. Copper DSL implements a subset of JSONPath, listed below. You can also use the Online JSONPath evaluator for testing or refer to the debugging section below:
| Operation | Meaning |
|---|---|
$ |
The root element to query. This starts all path expressions. |
@ |
The current node being processed by a filter predicate. |
* |
Wildcard. Available anywhere a name or numeric are required. |
.. |
Deep scan. Available anywhere a name is required. |
.<name> |
Dot-notated child |
['<name>' (, '<name>')] |
Bracket-notated child or children |
[<number> (, <number>)] |
Array index or indexes |
[start:end] |
Array slice operator |
[?(<expression>)]<name> |
Filter expression. Expression must evaluate to a boolean value. |
| Operator | Description |
|---|---|
== |
left is equal to right (note that 1 is not equal to '1') |
!= |
left is not equal to right |
< |
left is less than right |
<= |
left is less or equal to right |
> |
left is greater than right |
>= |
left is greater than or equal to right |
Another example
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
namespace: foobar
name: foo
spec:
template:
metadata:
labels:
app: foo
tier: bar
spec:
containers:
- name: foo
image: index.docker.io/library/ubuntu:latest
ports:
- containerPort: 8080
- name: mysql
image: quay.io/mysql:2.3.0
ports:
- containerPort: 3306
- name: buzz
image: quay.io/pg:latest
ports:
- containerPort: 8080
imagePullSecrets:
- name: registry-pull-secret
Get the image tag of all containers
fetch("$.spec.spec.containers..image").extract(".*:(.*)", 1) // returns ["latest", "2.3.0", "latest"]
Get the image name of the container named mysql
fetch("$.spec.spec.containers[?(@.name == 'mysql')].image") // will return ["quay.io/mysql:2.3.0"]
You can dump the results of variables and comparisons to the console using the -> console directive.
$ copper check --rules rule.cop --file config.yml --debug
Example
Write the value of variable `mysql_version` to the console
rule foo warn {
mysql_variable -> console
}
Write the result of a comparison to the console
rule foo warn {
fetch("$.spec.template.images").contain("ubuntu") -> console
}
Write the result of a fetch to console
rule foo warn {
fetch("$.spec.ports..targetPort") -> console
}