When a user wants to leverage parseLab, they will input data to parseLab through the medium of a specification file. This file is responsible for informing parseLab of the characteristics of the target protocol. A guide for leveraging specification files can be found here and here for the UDP and MAVLink protocols, respectively. Although the connotation throughout these guides is that these files are hand-written, the ideal situation is that all protocol specification files are generated by another system. The JSON format was chosen as a format that is easily human-readable, but more importantly easily machine-writable. Ideally, there is a process that can read a database that knows all of the ins-and-outs of the target protocol which can then format the data in this JSON format which can be processed by parseLab.
parseLab focuses on the characteristics that define the message formats within the protocol. These characteristics can be boiled down into 4 components: Messages, Fields, Data Types, and Value Constraints.
- Messages
- Fields
- Data Types
- Value Constraints
Since the protocol specification file's job is to provide the formats of messages to parseLab, the outer-most layer of the component architecture is the Message class.
This class essentially just holds the name of the message type and the fields that it contains.
# Simplified representation of the Message class
class Message:
def __init__(self, name: str, fields: list):
self.name = name
self.fields = fieldsThe field component contains the necessary information to define a field within a message format.
We have defined a field with the FieldDef class. This class is mostly composed of: a name, a data type, and a value constraint.
# Simplified representation of the FieldDef class
class FieldDef:
def __init__(self):
self.name = name
self.value_def = value_def
self.dtype = dtypeData types define what format that a field can be described with. In parseLab, data types can only be defined as integers or floats. These data types can also be defined as lists of integers/floats. Lastly, they can be defined as big or little endian.
Integer data types are either signed or unsigned, denoted by an I or U, respectively.
Integer data types can be of any size, denoted by the number of bits following the signedness symbol.
For example, I8 would be an signed 8-bit integer, where as a U13 would be an unsigned 8-bit integer.
For data types whose value is larger than one byte, the endianness is big by default, but can be changed to little by prepending the data type string with a < symbol (same as the python struct library symbol)
At this time, there is only one way to represent a floating point value in a field and that is with an F.
Similarly, the only floats accepted by parseLab are 32-bit floats in the IEEE 754 format.
This means, to denote a float data type, you would use the string F32.
List data types are simply the base data type string with a C-style array brackets appended on to define the number of elements in the array.
For example, U8[9] is a list of 9 unsigned 8-bit integers.
However, this does not need to be a constant value.
If the protocol message requires a dependency across two values (which tends to happen when a data field is tied to a length field), then you can reference the dependee in the array size.
For the example of a data field being tied to the length field, it would look like U8[LENGTH], meaning that the data field is LENGTH many unsigned 8-bit integers.
parseLab actually splits Value Constraint (Value) and the Value Definition (ValueDef) up into two objects. A Field will have single ValueDef component, and this ValueDef will have a specific value constraint type, or Value type.
The Value object is actually an abstract class, while the (currently) four Value Type classes are the usable, derived classes: ValueSingle, ValueRange, ValueChoice, and ValueList
All Value Types will have a reference to the data type that makes up the value constraint, and some reference to the data contents stored which define the actual constrained data.
The ValueSingle object is the basic building block of the Value Types. All other Value Types can be described as series of one or many ValueSingle objects.
For example, here is an example of a protocol specification which defines a single message, with a single field which has a ValueSingle as a value constraint:
{
"protocol_types": [
{
"msg_name": "EXAMPLE_MESSAGE_01",
"fields": [
{
"field_name": "VALUE_SINGLE_FIELD",
"value": "15",
"type": "U8"
}
]
}
]
}In the above example, we have a single message EXAMPLE_MESSAGE_01 being described as having a single field, named VALUE_SINGLE_FIELD.
The VALUE_SINGLE_FIELD has a data type "U8" which means that this field is of size 8 bits, and is an unsigned integer.
The value label defines the value constraint to be a single 15 which tells parseLab that when a parser (or testcase) is generated, the required value for this field has to be an unsigned, 8-bit, 15. Or in C terms: uint8_t val = 15;
A Value Single can be denoted as base 10, base 8, base 2, or as characters (for 8-bit types only).
| Base | Example |
|---|---|
| 10 | 65 |
| 8 | 0x41 |
| 2 | 0b1000001 |
| char | A |
The ValueRange constraint type is defined as two ValueSingle constraints being set as the lower and upper, inclusive, bounds of a range of values.
This can be helpful if your field can be any value from 0 to N, for example. The following json block describes how this might be used.
{
"name": "VALUE_RANGE_FIELD",
"value": "(0, 50)",
"type": "I8"
}This example demonstrates a ValueRange constraint with a I8, or signed 8-bit integer, as the data type.
This ValueRange take the values available to the data type [-128,127] and constrain the set down to a range of [0, 50]
The ValueChoice constraint type is defined as N many ValueSingle constraints or N many ValueRange constraints, in series.
The ValueChoice type will allow you to specify that a field can have a set of discrete values that are disjoint and cannot be defined with a single ValueRange constraint.
The following json block describes how this may be used for series of singles or series of ranges. NOTE: At this time, you cannot mix and match the types of Values that are defined with the choice operator.
{
"name": "VALUE_CHOICE_SINGLE_FIELD",
"value": "1|3|5|7|9|11|13|15",
"type": "U4"
}
{
"name": "VALUE_CHOICE_RANGE_FIELD",
"value": "(0,5)|(10,15)|(20,25)|(30,35)|(40,45)|(50,55)|(60,64)",
"type": "U7"
}The first field block describes a way of defining the field to only allow the odd numbers that are available in the set of unsigned 4-bit integer representation.
The second field block describes a way of defining the field to only allow the first 6 numbers of each interval of 10 within the set of unsigned 7-bit integer representation.
The ValueList constraint type allows specification of fields which are best represented as arrays of values.
The ValueList consists of N ValueSingle constraints that are all of the specified data type.
The following json block will depict two ways of leveraging ValueList constraints:
{
"name": "VALUE_LIST_CONST",
"value": "[h, e, l, l, o]",
"type": "U8[5]"
}
{
"name": "VALUE_LIST_DEPENDEE",
"value": "15",
"type": "32",
"dependee": "True"
}
{
"name": "VALUE_LIST_DEPENDENT",
"value": "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]"
"type": "U8[VALUE_LIST_DEPENDEE]"
}The field VALUE_LIST_CONST defines a ValueList constraint where the data type is a 5-long unsigned 8-bit integer array which will be constrained to the following series of bytes: 0x68, 0x65, 0x6c, 0x6c, 0x6f, which resolves to the following string: hello.
The field VALUE_LIST_DEPENDEE is a ValueSingle which is being used by the following field as a reference for defining the length of the array defined in the VALUE_LIST_DEPENDENT field.
By specifying the depnedee=True attribute, parseLab will know to store the information found in the VALUE_LIST_DEPNEDEE value constraint for later use in another value constraint.
The field VALUE_LIST_DEPENDENT has a value constraint on it that is actually linked to the previous field.
This is observed with the array size declaration in the data type which references the name of a previous field.