mview (Mask viewer) is a program in the style of Unix coreutils that lets you view raw data though a mask. The mask is configurable and supports the common data types. Of course mview is written in Rust and therefore 🚀🚀🚀 blazingly fast 🚀🚀🚀.
One of mview’s original applications was viewing messages that come from network. E.g. when you have an embedded device that sends it status via network over and over, Mview can read those messages and present them in a human readable way. But mview is very versatile and has many other applications, see examples below.
Let’s say we have a socket that listens on port 3000. This can be simulated with netcat. We can read from that socket and pipe the output to mview:
nc -l -p 3000 | mview -c ./src/example_config --pause 100Where --pause 100 just tells mview to wait with updating its output for 100ms.
And the example_config passed to mview is this file:
MyData1:String:2
SomeOtherData:u8
SomeMoreData:bool8
EvenMoreData:String:3(See below for the different datatypes.)
The first String before the first : is an arbitrary identifier. You can name that field however you want, for the sake of your own confusion you can also give different fields the same name.
Now we connect to that socket using netcat again: (in another terminal)
nc localhost 3000We can now send data to that socket in stdin:
~ > nc localhost 3000
ab3456 # I typed this in and pressed returnmview will output
MyData1: ab SomeOtherData: 51 SomeMoreData: true EvenMoreData: 56
Mind that mview is trying to output the third byte it finds in the data it received as a u8. But since we typed characters into nc it will output the ASCII-value of the character 3, which is 51.
The type bool8 will interpret a whole byte like a boolean. Like in C anything other than 0 is interpreted as true.
If we typed more data in that socket, mview would start over and display the next few bytes through that mask.
Data from tcpdump and such is usually captured in a file format called PCAP. Such files can be read by mview and the output of tcpdump can also be piped into mview.
To do so, mview has the flag --pcap.
tcpdump -i eth0 -w - -U | mview -c ./myconfig --pcapOr reading from a file
mview -c ./myconfig --pcap -i mycapture.pcapThe arguments --rawhex, --stats and --bitpos can then be used to get an idea of the length of a chunk, where it starts end ends, how long jumps over eventual IP headers should be etc.
Using --pcap and the argument --timestamp the timestamps that are extracted from the PCAP input file or the PCAP input stream are used. (Without --pcap the argument --timestamp prints the current time the chunk is written to the output as timestamp. --timestamp does not make much sense when reading from a raw file rather than a raw stream or PCAP file.)
Lets say you have a super secret header in a binary file and you are tired of reading raw hexadecimal values. Lets make such a file first:
echo hello345test0000000 > ~/myFileHere we are only interested at the data before the 0s start. The 0s represent the rest of the file that comes after the header. Here is the according config to that:
Shouldbehello:String:5 SomeOtherData:u8 SomeMoreData:bool8 SomeMoreData:u8 Shouldbetest:String:4
We know the header is 12 bytes long, so lets pipe that into mview:
~ > head -c 12 ~/myFile | mview -c ./src/example_config
Shouldbehello: hello
SomeOtherData: 51
SomeMoreData: true
SomeMoreData: 53
Shouldbetest: testInstead of using the head command to get the first 12 bytes of the file, we could use the build in --head argument with the --infile to read directly from a file instead of stdin as well:
~ > mview -c /home/pascal/dev/mview/src/example_config --infile ~/myFile --head 12 # head makes mview read 12 bytes, then exit
Shouldbehello: hello
SomeOtherData: 51
SomeMoreData: true
SomeMoreData: 53
Shouldbetest: testThe tell mview what the mask is, we need to pass a configuration file to it with the -c / --config argument. The following points are the possible contents of that file.
Lines that start with a # are not evaluated by mview and can be used as comments.
Also everything that comes after an # in a line is not evaluated.
Example:
Myfieldname:String:3 # this line is evaluated, this comment not # A comment that is not evaluated
Evaluates a single bit in the chunk as true or false.
Myfieldname:bool1
Evaluates a whole byte to true or false in C style: Everything except 0 is true.
Myfieldname:bool8
Evaluates 8, 16, 32 bits into an integer. I think the type names are self-explanatory, if not look them up here.
Myfieldname:i16 Myfieldname:i32:h # can also be displayed in hexadecimal Myfieldname:i32:hex # works as well Myfieldname:i32:hexadecimal # works as well Myfieldname:u8:b # can also be displayed in binary Myfieldname:u8:binary # works as well
Evaluates a 32 bits / 64 bits into a floating point number.
Myfieldname:f32
In order to save a few bits of space sometimes integers don’t use full bytes in network messages. Therefore there is this type. The length operator (the number at the end in the config line below) represents the length of that field in bits, not bytes!
Myfieldname:iarb:7
Read a few bytes of the chunk and display them as ASCII characters. The length operator represents the length in bytes, not bits! Only one-byte characters are supported like standard strings in C.
Myfieldname:String:4
Sometimes you want to skip a few bytes and don’t display them in the output. You could just fill those with bool8 and bool1, but for tidiness sake there are those types. Mview will jump the bytes and bits ahead in a chunk and continue evaluation with the next config line. Bytegap takes a length operator in bytes and bitgap takes a length operator in bits.
Myfieldname:bytegap:2 # a 2 byte wide gap Myfieldname:bitgap:4 # a 4 bit wide gap
Because mview is primarily used for decoding network messages, integers that consist of several bytes are evaluated in network byte order (big-endian/ motorola order) by default.
However, the expected byte order can be changed to little-endian (intel) order with the --le flag.
mview receives messages from stdin or a file. It then divides a received message into chunks, where the size of a chunk is determined by the config. (The length of the datatypes added up.) If a datagram socket is read, usually the chunksize is the same like the messages size:
+--------------+ +--------------+ +------------------+ | Message 1 | | Chunk 1 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | +--------------+ +--------------+ +------------------+ +--------------+ +--------------+ +------------------+ | Message 2 | | Chunk 1 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | +--------------+ +--------------+ +------------------+
But if a stream socket is read, or a socket was recorded and that record is fed into mview, mview has no possible way to determine where one message ends and another starts. In this case (when the message is longer than chunk size) mview will take the length of a chunk in bytes from the message and display it fields, then continue to take the next length of a chunk from the message and display that until the whole message is processed.
+--------------+ +--------------+ +------------------+ | Message 1 | | Chunk 1 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | | | +--------------+ +------------------+ | | +--------------+ +------------------+ | | | Chunk 2 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | +--------------+ +--------------+ +------------------+ +--------------+ +--------------+ +------------------+ | Message 2 | | Chunk 1 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | | | +--------------+ +------------------+ | | +--------------+ +------------------+ | | | Chunk 2 |------->| Data field 1 | | | | | +------------------+ | | | | +------------------+ | |-->| |------->| Data field 2 | | | | | +------------------+ | | | | +------------------+ | | | |------->| Data field 3 | +--------------+ +--------------+ +------------------+
In case the message size is not a whole multiple of chunk size, the data fields of the last message will be cut off: (this will be printed in mviews output as “values size is bigger than what is left of that data chunk”)
+--------------+ +--------------+ +------------------+
| Message 1 | | Chunk 1 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 3 |
| | +--------------+ +------------------+
| | +--------------+ +------------------+
| | | Chunk 2 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
|______________| | | +------------------+
| |
| |
+--------------+
+--------------+ +--------------+ +------------------+
| Message 2 | | Chunk 1 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 3 |
| | +--------------+ +------------------+
| | +--------------+ +------------------+
| | | Chunk 2 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
|______________| | | +------------------+
| |
| |
+--------------+
In the same way the data fields will be cut off and a message displayed for the fields that have no data will be displayed if the chunksize is bigger than message size:
+--------------+ +--------------+ +------------------+
| Message 1 | | Chunk 1 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 3 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 4 |
|______________| | | +------------------+
| |
| |
| |
| |
| |
+--------------+
+--------------+ +--------------+ +------------------+
| Message 2 | | Chunk 1 |------->| Data field 1 |
| | | | +------------------+
| | | | +------------------+
| |-->| |------->| Data field 2 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 3 |
| | | | +------------------+
| | | | +------------------+
| | | |------->| Data field 4 |
|______________| | | +------------------+
| |
| |
| |
| |
| |
+--------------+
Usually the chunk size is calculated from the config file, but it can be set manually with the argument --chunksize. (In bytes) In the case that the sum of the length of the fields in config is bigger than the size from the argument, the message “values size is bigger than what is left of that data chunk” will be displayed for the fields that have no data. If the chunk size from the argument is bigger, the remaining bytes from that chunk will not be evaluated.
Please note that writing to a file is different that piping stdout of mview into a file like this:
mview -c ./src/example_config --stats > loggingfile.txtMview resets the cursor position for every chunk, therefore the output file would only have the last chunk printed in it like this:
Message no: 3 Message length: 30 bytes Current chunk in this message: 2 Shouldbehello: oehus SomeOtherData: 111 SomeMoreData: true SomeMoreData: 115 Shouldbetest: hutn EvenMoreData: eu
When using the ---outfile argument, mview will not reset the cursor position and the outfile will look like this:
Message no: 1 Message length: 37 bytes Current chunk in this message: 1 Shouldbehello: otnes SomeOtherData: 117 SomeMoreData: true SomeMoreData: 115 Shouldbetest: oent EvenMoreData: hsh Message no: 1 Message length: 37 bytes Current chunk in this message: 2 Shouldbehello: usneu SomeOtherData: 116 SomeMoreData: true SomeMoreData: 111 Shouldbetest: euho EvenMoreData: euh Message no: 1 Message length: 37 bytes Current chunk in this message: 3 Shouldbehello: sotne SomeOtherData: 117 SomeMoreData: true SomeMoreData: values size is bigger than what is left of that data chunk Shouldbetest: values size is bigger than what is left of that data chunk EvenMoreData: values size is bigger than what is left of that data chunk
However, it is also possible to use the --nojump argument instead. This makes mview print to stdout like it does to files.
Normally mview resets the cursor position for every chunk. It therefore print chunks ‘on top of each other’, so to speak. This behavior is hard to implement because different terminal emulators have different features and react differently to the same control messages.
Therefore it can happen, that mview deletes to many or to few lines, or it does not delete the lines at all. For this mview can also clear the terminal instead with the --clear flag. With this flag an output always starts at top of the terminal and the terminal is cleared before a chunk is printed.