Compression Example on the ESP32

[vangalvin]

Hi Martin, I was wondering if you by any chance had a compression example for the ESP32.
I have quite a few sensors and would like to be able to reduce the size of the overall transmitted data if possible before it is sent to a remote web site and feed in to a javascript decompression routine then stored in a database.

Regards
Andrew

[martinberlin]

I tried so far but still getting an exception in the looptask. I asked for support in Brotli main library but I guess first need to test the same in a C compiled file out of ESP32 environment to see if my input makes sense. Will update it here if I can get it to work

[vangalvin]

thanks @martinberlin :)

[martinberlin]

Try the feature/compress branch

Reading file '/sensors.json' from FS

---

349 bytes read into inBuffer from: /sensors.json
Calling BrotliEncoderCompress
Compression params:
1 quality
22 lgwin
status: 1
2131 microseconds spend compressing
219 bytes after compression
decompressBuffer() - - - - - - - - - - - - - - - - - - - - - - - - - -

status: 1
Decompress output: - - - - - - - - - - - - - - - - - - - - - - - - - -
{
"DS1": [-16.931, -16.931, -37.938, 29.703, 47.683, 100.362, 74.003, 107.58, -16.549, 70.011, 85.483, 118.478, -49.773, 54.442, 114.613, 99.228, 16.555, 43.453, 10.91, 33.173, 14.56, -54.855, 93.051, 109.894, 60.431, 88.935, 5.121, -46.954, 59.805, -29.47, 87.34, -54.071, 8.601, 104.404, 30.632, 45.635, 77.765, 93.987, 26.215, 50.569, -28.651]
}

However, trying to increase quality will fail. Anyways it's great to have it working.
Please contact me per E-Mail if you need further help with your implementation and feel free to close this if it's working for you.

[vangalvin]

Awesome, I will give that a go shortly and report on how it works.

[martinberlin]

Note if you want to go to the lowest transmission protocol, json is definitively not the way to go.
But if you do not have any other alternative than json, reduce the floats to integers, and try to compress again. Also, the DS1 could be 1 character if you decide so. All little reductions here and there will help you to get a smaller size.

[vangalvin]

Thanks Martin,
Yes I have changed DS1 to be a value of 01 (there will eventually be about 20 units in place) Also I am now using the raw value from the sensor (16 bit value) this did drop the size down and will be dumping the JSON formatting as I decided that signed 16bit's tailed on to each other can be read back and converted at the other end to the values and the json can be reconstructed there.