From 94e99380f88cf7e854edd95f2f4ff8ddd7c808cc Mon Sep 17 00:00:00 2001 From: Jasem Mutlaq Date: Mon, 30 Jan 2023 09:05:24 +0300 Subject: [PATCH] Add more documentation on fast blobs that was written by Ludovic in INDI forum --- drivers/binary-transfers.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/drivers/binary-transfers.md b/drivers/binary-transfers.md index e0d22fc..e820bb0 100644 --- a/drivers/binary-transfers.md +++ b/drivers/binary-transfers.md @@ -55,3 +55,30 @@ For client, since the existing semantic allows them to modify the blob data and `camera_client->enableDirectBlobAccess(MYCCD, nullptr);` There are further optimisations possible to avoid more memory copies, on the driver side (like producing the camera frame directly in the memory buffer instead of copying). + +The fast-blob protocol is mostly the same than default one, with the following deviations: + ++ The client must connect to unix domain socket (only supported under Linux, MacOS lacks some important feature there...). ++ the client has to reply to "pingRequest" messages by "pingReply" (this implements the back-pressure, see below why it's important). The uid attribute from the request must be repeated in the reply: +``` + + +``` +on UNIX domain connection, the server can pass the blob content as an [anciliary file descriptor](http://man7.org/linux/man-pages/man7/unix.7.html). The file descriptor can later be mmapped. In that case, the xml message will have the attached property set to true (and no enclen by the way): + +``` + + + +``` +1. It is guaranteed that the file descriptor(s) for attached blob(s) are received by the end of the xml message. The exact position probably depends on the OS. +2. The server may also decide to not used shared buffer so the client must still support the use existing base64 layout (for example for small blobs) +3. The transport of buffers as filedescriptor can be very very fast, so your client may receive GB of data without much processing. Without backpressure, memory in the client side may grow uncontrolled (ultimately leading to Out of memory error, possibly system wide invoking the OOM killer). For this reason, it is strongly advised that when a blob arrive, the client stops processing incoming messages (especially replying to pingRequests) until it has finished processing the blob. +4. To actually free the memory used by a filedescriptor, the client must close the fd, and munmap the memory area. + +Some example of low level functions for these tasks can be found in the indi source file integs/test.cpp (integration tests), which is somewhat decoupled from indi so probably easier to read for a start: ++ integs/test.cpp: the unixSocketConnect function - to connect to a local socket by path name ++ integs/ConnectionMock.cpp, method ConnectionMock::read, shows how to read the input channel, collecting received fds at the same time ++ integs/SharedBuffer.cpp, shows how to access memory from a fd. It is in the write direction, while you client will want to read insted. So your client should use PROT_READ only and remove PROT_WRITE) + +Tests codes in the integs directory may also be useful as they show the plain text version of the messages for various cases.