From e4510113c1c6ae748102f8ac7963fc73c05046f0 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sat, 7 Dec 2024 21:41:27 +0200 Subject: [PATCH 01/24] add design.md --- DESIGN.md | 241 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 241 insertions(+) create mode 100644 DESIGN.md diff --git a/DESIGN.md b/DESIGN.md new file mode 100644 index 0000000..e35f774 --- /dev/null +++ b/DESIGN.md @@ -0,0 +1,241 @@ +# Open Cyphal Vehicle System Management Daemon for GNU/Linux + +This project implements a user-facing C++14 library backed by a GNU/Linux daemon used to asynchronously perform certain common operations on an OpenCyphal network. Being based on LibCyphal, the solution can theoretically support all transport protocols supported by LibCyphal, notably Cyphal/UDP and Cyphal/CAN. + +The implementation will be carried out in multiple stages. The milestones achieved at every stage are described here along with the overall longer-term vision. + +The design of the C++ API is inspired by the [`ravemodemfactory`](https://github.com/aleksander0m/ravemodemfactory) project (see `src/librmf/rmf-operations.h`). + +[Yakut](https://github.com/OpenCyphal/yakut) is a dispantly related project with the following key differences: + +- Yakut is a developer tool, while OCVSMD is a well-packaged component intended for deployment in production systems. + +- Yakut is a user-interactive tool with a CLI interface, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI interface as well, but it will always come secondary to the well-formalized C++ API. + +- Yakut is entirely written in Python, and thus it tends to be resource-heavy when used in embedded computers. + +- Yakut is not designed to be highly robust. + +## Long-term vision + +Not all of the listed items will be implemented the way they are seen at the time of writing this document, but the current description provides a general direction things are expected to develop in. + +OCVSMD is focused on solving problems that are pervasive in intra-vehicular OpenCyphal networks with minimal focus on any application-specific details. This list may eventually include: + +- Publish/subscribe on Cyphal subjects with arbitrary DSDL data types loaded at runtime, with the message objects represented as dynamically typed structures. More on this below. +- RPC client for invoking arbitrarily-typed RPC servers with DSDL types loaded at runtime. +- Support for the common Cyphal network services out of the box, configurable via the daemon API: + - File server running with the specified set of root directories (see Yakut). + - Firmware update on a directly specified remote node with a specified filename. + - Automatic firmware update as implemented in Yakut. + - Centralized (eventually could be distributed for fault tolerance) plug-and-play node-ID allocation server. +- Depending on how the named topics project develops (many an intern has despaired over it), the Cyphal resource name server may also be implemented as part of OCVSMD at some point. + +Dynamic DSDL loading is proposed to be implemented by creating serializer objects whose behavior is defined by the DSDL definition ingested at runtime. The serialization method is to accept a byte stream and to produce a DSDL object model providing named field accessors, similar to what one would find in a JSON serialization library; the deserialization method is the inverse of that. Naturally, said model will heavily utilize PMR for storage. The API could look like: + +```c++ +namespace ocvsmd::dsdl +{ +/// Represents a DSDL object of any type. +class Object +{ + friend class Type; +public: + /// Field accessor by name. Empty if no such field. + std::optional operator[](const std::string_view field_name) const; + + /// Array element accessor by index. Empty if out of range. + std::optional> operator[](const std::size_t array_index); + std::optional> operator[](const std::size_t array_index) const; + + /// Coercion to primitives (implicit truncation or the loss of precision are possible). + operator std::optional() const; + operator std::optional() const; + operator std::optional() const; + + /// Coercion from primitives (implicit truncation or the loss of precision are possible). + Object& operator=(const std::int64_t value); + Object& operator=(const std::uint64_t value); + Object& operator=(const double value); + + const class Type& get_type() const noexcept; + + std::expected serialize(const std::span output) const; + std::expected deserialize(const std::span input); +}; + +/// Represents a parsed DSDL definition. +class Type +{ + friend std::pmr::unordered_map read_namespaces(directories, pmr, ...); +public: + /// Constructs a default-initialized Object of this Type. + Object instantiate() const; + ... +}; + +using TypeNameAndVersion = std::tuple; + +/// Reads all definitions from the specified namespaces and returns mapping from the full type name +/// and version to its type model. +std::pmr::unordered_map read_namespaces(directories, pmr, ...); +} +``` + +One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers like Boost Interprocess or specialized PMR. + +Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: + +- Ability to operate from a read-only filesystem. +- Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. +- Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. + +The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. + +```c++ +namespace ocvsmd +{ +/// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. +/// Once instantiated, the published may live on until the daemon process is terminated. +class Publisher +{ +public: + /// True on success, false on timeout. + virtual std::expected publish(const std::span data, + const std::chrono::microseconds timeout) = 0; + std::expected publish(const dsdl::Object& obj, const std::chrono::microseconds timeout) + { + // obj.serialize() and this->publish() ... + } +}; + +/// The daemon will implicitly instantiate a subscriber on the specified port-ID the first time it is requested. +/// Once instantiated, the subscriber may live on until the daemon is terminated. +/// +/// The daemon will associate an independent IPC queue with each client-side subscriber and push every received +/// message into the queues. Queues whose client has died are removed. +/// +/// The client has to keep its Subscriber instance alive to avoid losing messages. +class Subscriber +{ +public: + /// Empty if no message received within the timeout. + virtual std::expected, Error> receive(const std::chrono::microseconds timeout) = 0; + + /// TODO: add methods for setting the transfer-ID timeout. +}; + +/// The daemon will implicitly instantiate a client on the specified port-ID and server node when it is requested. +/// Once instantiated, the client may live on until the daemon is terminated. +class RPCClient +{ +public: + /// Returns the response object on success, empty on timeout. + virtual std::expected, Error> call(const dsdl::Object& obj, + const std::chrono::microseconds timeout) = 0; +}; + +/// The daemon always has the standard file server running. +/// This interface can be used to configure it. +class FileServerController +{ +public: + /// When the file server handles a request, it will attempt to locate the path relative to each of its root + /// directories. See Yakut for a hands-on example. + /// The daemon will canonicalize the path and resolve symlinks. + virtual std::expected add_root(const std::string_view directory); + + /// Does nothing if such root does not exist. + /// The daemon will canonicalize the path and resolve symlinks. + virtual std::expected remove_root(const std::string_view directory); +}; + +/// A helper for invoking the uavcan.node.ExecuteCommand service on the specified remote nodes. +/// The daemon always has a set of uavcan.node.ExecuteCommand clients ready. +class NodeCommandClient +{ +public: + /// TODO: add constants for the response status codes. + + struct Response + { + std::uint8_t status; + std::pmr::string output; + }; + + /// Empty option indicates that the corresponding node did not return a response on time. + virtual std::expected>, Error> + call(const std::span node_ids, + const std::uint16_t command, + const std::string_view parameter = {}, + const std::chrono::microseconds timeout = 1s) = 0; + + // TODO: add methods for the standard commands, like firmware update. +}; + +/// A helper for manipulating registers on the specified remote nodes. +class RegisterClient +{ +public: + // TODO: partial results + virtual std::expected< + std::pmr::unordered_map>>, + Error> + list(const std::span node_ids) = 0; + + // TODO +}; + +class Monitor +{ +public: + // TODO +}; + +/// An abstract factory for the specialized interfaces. +class Daemon +{ +public: + virtual std::expected, Error> make_publisher(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_subscriber(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_client(const dsdl::Type& type, + const std::uint16_t service_id) = 0; + + virtual FileServerController& get_file_server_controller() = 0; + + virtual NodeCommandClient& get_node_command_client() = 0; + + virtual RegisterClient& get_register_client() = 0; + + virtual Monitor& get_monitor() = 0; + + // TODO: PnP node-ID allocator controls (start/stop, read table, reset table) +}; + +/// A factory for the abstract factory that connects to the daemon. +/// Returns nullptr if the daemon cannot be connected to (not running). +std::unique_ptr connect(); +} +``` + +TODO: configuration file format -- tab-separated values; first column contains register name, second column contains the value. + +TODO: CLI interface + +## Milestone 0 + +SystemV, not systemd. + +File server running continuously. + +API: integers + +## Milestone 1 + +systemd integration along SystemV. + +## Milestone 2 From 579c18082ebd903e124a62b305c2992380015b75 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 17:52:08 +0200 Subject: [PATCH 02/24] Updates --- DESIGN.md | 289 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 199 insertions(+), 90 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index e35f774..60b31d9 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -39,45 +39,46 @@ namespace ocvsmd::dsdl /// Represents a DSDL object of any type. class Object { - friend class Type; + friend class Type; public: - /// Field accessor by name. Empty if no such field. - std::optional operator[](const std::string_view field_name) const; - - /// Array element accessor by index. Empty if out of range. - std::optional> operator[](const std::size_t array_index); - std::optional> operator[](const std::size_t array_index) const; - - /// Coercion to primitives (implicit truncation or the loss of precision are possible). - operator std::optional() const; - operator std::optional() const; - operator std::optional() const; - - /// Coercion from primitives (implicit truncation or the loss of precision are possible). - Object& operator=(const std::int64_t value); - Object& operator=(const std::uint64_t value); - Object& operator=(const double value); - - const class Type& get_type() const noexcept; - - std::expected serialize(const std::span output) const; - std::expected deserialize(const std::span input); + /// Field accessor by name. Empty if no such field. + std::optional operator[](const std::string_view field_name) const; + + /// Array element accessor by index. Empty if out of range. + std::optional> operator[](const std::size_t array_index); + std::optional> operator[](const std::size_t array_index) const; + + /// Coercion to primitives (implicit truncation or the loss of precision are possible). + operator std::optional() const; + operator std::optional() const; + operator std::optional() const; + + /// Coercion from primitives (implicit truncation or the loss of precision are possible). + Object& operator=(const std::int64_t value); + Object& operator=(const std::uint64_t value); + Object& operator=(const double value); + + const class Type& get_type() const noexcept; + + std::expected serialize(const std::span output) const; + std::expected deserialize(const std::span input); }; /// Represents a parsed DSDL definition. class Type { - friend std::pmr::unordered_map read_namespaces(directories, pmr, ...); + friend std::pmr::unordered_map read_namespaces(directories, pmr, ...); public: - /// Constructs a default-initialized Object of this Type. - Object instantiate() const; - ... + /// Constructs a default-initialized Object of this Type. + Object instantiate() const; + ... }; using TypeNameAndVersion = std::tuple; /// Reads all definitions from the specified namespaces and returns mapping from the full type name /// and version to its type model. +/// Optionally, the function should cache the results per namespace, with an option to disable the cache. std::pmr::unordered_map read_namespaces(directories, pmr, ...); } ``` @@ -90,23 +91,24 @@ Being a daemon designed for unattended operation in deeply-embedded vehicular co - Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. - Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. -The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. +The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. The API is intentionally designed to not hide the structure of the Cyphal protocol itself; that is to say that it is intentionally low-level. Higher-level abstractions can be built on top of it on the client side rather than the daemon side to keep the IPC protocol stable. The `Error` type used in the API definition here is a placeholder for the actual algebraic type listing all possible error states per API entity. ```c++ namespace ocvsmd { /// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. /// Once instantiated, the published may live on until the daemon process is terminated. +/// Internally, published messages may be transferred to the daemon via an IPC message queue. class Publisher { public: - /// True on success, false on timeout. - virtual std::expected publish(const std::span data, - const std::chrono::microseconds timeout) = 0; - std::expected publish(const dsdl::Object& obj, const std::chrono::microseconds timeout) - { - // obj.serialize() and this->publish() ... - } + /// True on success, false on timeout. + virtual std::expected publish(const std::span data, + const std::chrono::microseconds timeout) = 0; + std::expected publish(const dsdl::Object& obj, const std::chrono::microseconds timeout) + { + // obj.serialize() and this->publish() ... + } }; /// The daemon will implicitly instantiate a subscriber on the specified port-ID the first time it is requested. @@ -119,10 +121,10 @@ public: class Subscriber { public: - /// Empty if no message received within the timeout. - virtual std::expected, Error> receive(const std::chrono::microseconds timeout) = 0; - - /// TODO: add methods for setting the transfer-ID timeout. + /// Empty if no message received within the timeout. + virtual std::expected, Error> receive(const std::chrono::microseconds timeout) = 0; + + /// TODO: add methods for setting the transfer-ID timeout. }; /// The daemon will implicitly instantiate a client on the specified port-ID and server node when it is requested. @@ -130,24 +132,30 @@ public: class RPCClient { public: - /// Returns the response object on success, empty on timeout. - virtual std::expected, Error> call(const dsdl::Object& obj, - const std::chrono::microseconds timeout) = 0; + /// Returns the response object on success, empty on timeout. + virtual std::expected, Error> call(const dsdl::Object& obj, + const std::chrono::microseconds timeout) = 0; }; /// The daemon always has the standard file server running. /// This interface can be used to configure it. +/// It is not possible to stop the server; the closest alternative is to remove all root directories. class FileServerController { public: - /// When the file server handles a request, it will attempt to locate the path relative to each of its root - /// directories. See Yakut for a hands-on example. - /// The daemon will canonicalize the path and resolve symlinks. - virtual std::expected add_root(const std::string_view directory); - - /// Does nothing if such root does not exist. - /// The daemon will canonicalize the path and resolve symlinks. - virtual std::expected remove_root(const std::string_view directory); + /// When the file server handles a request, it will attempt to locate the path relative to each of its root + /// directories. See Yakut for a hands-on example. + /// The daemon will canonicalize the path and resolve symlinks. + /// The same path may be added multiple times to avoid interference across different clients. + virtual std::expected add_root(const std::string_view directory); + + /// Does nothing if such root does not exist (no error reported). + /// If such root is listed more than once, only one copy is removed. + /// The daemon will canonicalize the path and resolve symlinks. + virtual std::expected remove_root(const std::string_view directory); + + /// The returned paths are canonicalized. The entries are not unique. + virtual std::expected, Error> list_roots() const; }; /// A helper for invoking the uavcan.node.ExecuteCommand service on the specified remote nodes. @@ -155,65 +163,166 @@ public: class NodeCommandClient { public: - /// TODO: add constants for the response status codes. - - struct Response - { - std::uint8_t status; - std::pmr::string output; - }; - - /// Empty option indicates that the corresponding node did not return a response on time. - virtual std::expected>, Error> - call(const std::span node_ids, - const std::uint16_t command, - const std::string_view parameter = {}, - const std::chrono::microseconds timeout = 1s) = 0; - - // TODO: add methods for the standard commands, like firmware update. + /// TODO: add constants for the response status codes. + + struct Response + { + std::uint8_t status; + std::pmr::string output; + }; + + /// Empty response indicates that the associated node did not respond in time. + using Result = std::expected>, Error>; + + /// Empty option indicates that the corresponding node did not return a response on time. + /// All requests are sent concurrently and the call returns when the last response has arrived, + /// or the timeout has expired. + virtual Result send_custom_command(const std::span node_ids, + const std::uint16_t command, + const std::string_view parameter = {}, + const std::chrono::microseconds timeout = 1s) = 0; + + /// A convenience method for invoking send_custom_command() with COMMAND_RESTART. + Result restart(const std::span node_ids, const std::chrono::microseconds timeout = 1s) + { + return send_custom_command(node_ids, 65535, "", timeout); + } + + /// A convenience method for invoking send_custom_command() with COMMAND_BEGIN_SOFTWARE_UPDATE. + /// The file_path is relative to one of the roots configured in the file server. + Result begin_software_update(const std::span node_ids, + const std::string_view file_path, + const std::chrono::microseconds timeout = 1s) + { + return send_custom_command(node_ids, 65533, file_path, timeout); + } + + // TODO: add convenience methods for the other standard commands. }; +/// A long operation may fail with partial results being available. The conventional approach to error handling +/// prescribes that the partial results are to be discarded and the error returned. However, said partial +/// results may occasionally be useful, if only to provide the additional context for the error itself. +/// +/// This type allows one to model the normal results obtained from querying multiple remote nodes along with +/// non-exclusive error states both per-node and shared. +/// +/// One alternative is to pass the output container or a callback as an out-parameter to the method, +/// so that the method does not return a new container but updates one in-place. +template +struct MulticastResult +{ + struct PerNode final + { + PerNodeResult result{}; + PerNodeError error{}; + }; + std::pmr::unordered_map result; + std::optional error; +}; + + +template +struct Wrapper { T value; }; + +using Float16 = Wrapper; + +/// Here, we should be using fixed-capacity containers instead! Perhaps CETL needs helpers for this. +/// The "empty" is represented with an optional wrapper. +using RegisterValue = std::variant< + std::pmr::string, + std::pmr::vector, + std::pmr::vector, + + std::pmr::vector, + std::pmr::vector, + std::pmr::vector, + std::pmr::vector, + + std::pmr::vector, + std::pmr::vector, + std::pmr::vector, + std::pmr::vector, + + std::pmr::vector, // Add fixed-width float aliases to CETL (C++23 polyfill) + std::pmr::vector, + std::pmr::vector, +>; +using MaybeRegisterValue = std::optional; + /// A helper for manipulating registers on the specified remote nodes. class RegisterClient { public: - // TODO: partial results - virtual std::expected< - std::pmr::unordered_map>>, - Error> - list(const std::span node_ids) = 0; + /// This method may return partial results. + virtual MulticastResult, Error, Error> + list(const std::span node_ids) = 0; + + /// Alternative definition of the list method using callbacks instead of partial results. + /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; + /// instead, they can be postponed until the blocking IPC call is finished. + using ListCallback = cetl::function), ...>; + virtual std::expected list(const std::span node_ids, const ListCallback& cb) = 0; + + /// This method may return partial results. + virtual MulticastResult, Error, Error> + read(const std::span node_ids, + const std::pmr::vector& names) = 0; + + /// This method may return partial results. + virtual MulticastResult, Error, Error> + write(const std::span node_ids, + const std::pmr::unordered_map& values) = 0; + + /// Alternative definitions of the read/write methods using callbacks instead of partial results. + /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; + /// instead, they can be postponed until the blocking IPC call is finished. + using ValueCallback = cetl::function), ...>; + virtual std::expected read(const std::span node_ids, + const std::pmr::vector& names, + const ValueCallback& cb) = 0; + virtual std::expected write(const std::span node_ids, + const std::pmr::unordered_map& values, + const ValueCallback& cb) = 0; - // TODO }; class Monitor { public: - // TODO + // TODO +}; + +class PnPNodeIDAllocatorController +{ +public: + // TODO: PnP node-ID allocator controls (start/stop, read table, reset table) }; /// An abstract factory for the specialized interfaces. class Daemon { public: - virtual std::expected, Error> make_publisher(const dsdl::Type& type, - const std::uint16_t subject_id) = 0; - - virtual std::expected, Error> make_subscriber(const dsdl::Type& type, - const std::uint16_t subject_id) = 0; - - virtual std::expected, Error> make_client(const dsdl::Type& type, - const std::uint16_t service_id) = 0; - - virtual FileServerController& get_file_server_controller() = 0; - - virtual NodeCommandClient& get_node_command_client() = 0; - - virtual RegisterClient& get_register_client() = 0; - - virtual Monitor& get_monitor() = 0; - - // TODO: PnP node-ID allocator controls (start/stop, read table, reset table) + virtual std::expected, Error> make_publisher(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_subscriber(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_client(const dsdl::Type& type, + const std::uint16_t service_id) = 0; + + virtual FileServerController& get_file_server_controller() = 0; + virtual const FileServerController& get_file_server_controller() const = 0; + + virtual NodeCommandClient& get_node_command_client() = 0; + + virtual RegisterClient& get_register_client() = 0; + + virtual Monitor& get_monitor() = 0; + virtual const Monitor& get_monitor() const = 0; + + virtual PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() = 0; }; /// A factory for the abstract factory that connects to the daemon. From bfc347dac1820ad6da97127371eda64e2d234763 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 20:36:37 +0200 Subject: [PATCH 03/24] Update the register interface --- DESIGN.md | 86 ++++++++++++++++++++++++------------------------------- 1 file changed, 38 insertions(+), 48 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 60b31d9..a21bd77 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -94,6 +94,12 @@ Being a daemon designed for unattended operation in deeply-embedded vehicular co The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. The API is intentionally designed to not hide the structure of the Cyphal protocol itself; that is to say that it is intentionally low-level. Higher-level abstractions can be built on top of it on the client side rather than the daemon side to keep the IPC protocol stable. The `Error` type used in the API definition here is a placeholder for the actual algebraic type listing all possible error states per API entity. ```c++ +#include +#include +#include +#include +#include + namespace ocvsmd { /// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. @@ -163,13 +169,8 @@ public: class NodeCommandClient { public: - /// TODO: add constants for the response status codes. - - struct Response - { - std::uint8_t status; - std::pmr::string output; - }; + using Request = uavcan::node::ExecuteCommand_1::Request; + using Response = uavcan::node::ExecuteCommand_1::Response; /// Empty response indicates that the associated node did not respond in time. using Result = std::expected>, Error>; @@ -178,14 +179,13 @@ public: /// All requests are sent concurrently and the call returns when the last response has arrived, /// or the timeout has expired. virtual Result send_custom_command(const std::span node_ids, - const std::uint16_t command, - const std::string_view parameter = {}, + const Request& request, const std::chrono::microseconds timeout = 1s) = 0; /// A convenience method for invoking send_custom_command() with COMMAND_RESTART. Result restart(const std::span node_ids, const std::chrono::microseconds timeout = 1s) { - return send_custom_command(node_ids, 65535, "", timeout); + return send_custom_command(node_ids, {65535, ""}, timeout); } /// A convenience method for invoking send_custom_command() with COMMAND_BEGIN_SOFTWARE_UPDATE. @@ -194,7 +194,7 @@ public: const std::string_view file_path, const std::chrono::microseconds timeout = 1s) { - return send_custom_command(node_ids, 65533, file_path, timeout); + return send_custom_command(node_ids, {65533, file_path}, timeout); } // TODO: add convenience methods for the other standard commands. @@ -222,69 +222,59 @@ struct MulticastResult }; -template -struct Wrapper { T value; }; - -using Float16 = Wrapper; - -/// Here, we should be using fixed-capacity containers instead! Perhaps CETL needs helpers for this. -/// The "empty" is represented with an optional wrapper. -using RegisterValue = std::variant< - std::pmr::string, - std::pmr::vector, - std::pmr::vector, - - std::pmr::vector, - std::pmr::vector, - std::pmr::vector, - std::pmr::vector, - - std::pmr::vector, - std::pmr::vector, - std::pmr::vector, - std::pmr::vector, - - std::pmr::vector, // Add fixed-width float aliases to CETL (C++23 polyfill) - std::pmr::vector, - std::pmr::vector, ->; -using MaybeRegisterValue = std::optional; - /// A helper for manipulating registers on the specified remote nodes. class RegisterClient { public: + using Name = uavcan::_register::Name_1; + using Value = uavcan::_register::Value_1; + /// This method may return partial results. - virtual MulticastResult, Error, Error> + virtual MulticastResult, Error, Error> list(const std::span node_ids) = 0; /// Alternative definition of the list method using callbacks instead of partial results. /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; /// instead, they can be postponed until the blocking IPC call is finished. - using ListCallback = cetl::function), ...>; + using ListCallback = cetl::function), ...>; virtual std::expected list(const std::span node_ids, const ListCallback& cb) = 0; /// This method may return partial results. - virtual MulticastResult, Error, Error> + virtual MulticastResult, Error, Error> read(const std::span node_ids, - const std::pmr::vector& names) = 0; + const std::pmr::vector& names) = 0; /// This method may return partial results. - virtual MulticastResult, Error, Error> + virtual MulticastResult, Error, Error> write(const std::span node_ids, - const std::pmr::unordered_map& values) = 0; + const std::pmr::unordered_map& values) = 0; /// Alternative definitions of the read/write methods using callbacks instead of partial results. /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; /// instead, they can be postponed until the blocking IPC call is finished. - using ValueCallback = cetl::function), ...>; + using ValueCallback = cetl::function&), ...>; virtual std::expected read(const std::span node_ids, - const std::pmr::vector& names, + const std::pmr::vector& names, const ValueCallback& cb) = 0; virtual std::expected write(const std::span node_ids, - const std::pmr::unordered_map& values, + const std::pmr::unordered_map& values, const ValueCallback& cb) = 0; + /// Helper wrappers for the above that operate on a single remote node only. + std::tuple, std::optional> list(const std::uint16_t node_id) + { + // non-virtual implementation + } + std::tuple, Error> + read(const std::uint16_t node_id, const std::pmr::vector& names) + { + // non-virtual implementation + } + std::tuple, Error> + write(const std::uint16_t node_id, const std::pmr::unordered_map& values) + { + // non-virtual implementation + } }; class Monitor From f8dd93aac37f308c9f50e7be1e3a8c4f7eca87fd Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 21:16:27 +0200 Subject: [PATCH 04/24] Add monitor and PnP allocator --- DESIGN.md | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 83 insertions(+), 3 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index a21bd77..2359629 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -277,16 +277,90 @@ public: } }; +/// The monitor continuously maintains a list of online nodes in the network. class Monitor { public: - // TODO + using Heartbeat = uavcan::node::Heartbeat_1; + using NodeInfo = uavcan::node::GetInfo_1::Response; + + /// An avatar represents the latest known state of the remote node. + /// The info struct is available only if the node responded to a uavcan.node.GetInfo request since last bootup. + /// GetInfo requests are sent continuously until a response is received. + /// If heartbeat publications cease, the corresponding node is marked as offline. + struct Avatar + { + std::uint16_t node_id; + + bool is_online; ///< If not online, the other fields contain the latest known information. + + std::chrono::system_clock::time_point last_heartbeat_at; + Heartbeat last_heartbeat; + + /// The info is automatically reset when the remote node is detected to have restarted. + /// It is automatically re-populated as soon as a GetInfo response is received. + struct Info final + { + std::chrono::system_clock::time_point received_at; + NodeInfo info; + }; + std::optional info; + + /// The port list is automatically reset when the remote node is detected to have restarted. + /// It is automatically re-populated as soon as an update is received. + struct PortList final + { + std::chrono::system_clock::time_point received_at; + std::bitset<65536> publishers; + std::bitset<65536> subscribers; + std::bitset<512> clients; + std::bitset<512> servers; + }; + std::optional port_list; + }; + + struct Snapshot final + { + /// If a node appears online at least once, it will be given a slot in the table permanently. + /// If it goes offline, it will be retained in the table but it's is_online field will be false. + /// The table is ordered by node-ID. Use binary search for fast lookup. + std::pmr::vector table; + std::tuple daemon; + bool has_anonymous; ///< If any anonymous nodes are online (e.g., someone is trying to get a PnP node-ID allocation) + }; + + /// Returns a snapshot of the current network state plus the daemon's own node state. + virtual Snapshot snap() const = 0; + + // TODO: Eventually, we could equip the monitor with snooping support so that we could also obtain: + // - Actual traffic per port. + // - Update node info and local register cache without sending separate requests. + // Yakut does that with the help of the snooping support in PyCyphal, but LibCyphal does not currently has that capability. }; +/// Implementation detail: internally, the PnP allocator uses the Monitor because the Monitor continuously +/// maintains the mapping between node-IDs and their unique-IDs. It needs to subscribe to notifications from the +/// monitor; this is not part of the API though. See pycyphal.application.plug_and_play.Allocator. class PnPNodeIDAllocatorController { public: - // TODO: PnP node-ID allocator controls (start/stop, read table, reset table) + /// Maps unique-ID <=> node-ID. + /// For some node-IDs there may be no unique-ID (at least temporarily until a GetInfo response is received). + /// The table includes the daemon's node as well. + using UID = std::array; + using Entry = std::tuple>; + using Table = std::pmr::vector; + + /// The method is infallible because the corresponding publishers/subscribers are always active; + /// when enabled==false, the allocator simply refuses to send responses. + virtual void set_enabled(const bool enabled) = 0; + virtual bool is_enabled() const = 0; + + /// The allocation table may or may not be persistent (retained across daemon restarts). + virtual Table get_table() const = 0; + + /// Forget all allocations; the table will be rebuilt from the Monitor state. + virtual void drop_table() = 0; }; /// An abstract factory for the specialized interfaces. @@ -312,7 +386,8 @@ public: virtual Monitor& get_monitor() = 0; virtual const Monitor& get_monitor() const = 0; - virtual PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() = 0; + virtual PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() = 0; + virtual const PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() const = 0; }; /// A factory for the abstract factory that connects to the daemon. @@ -321,6 +396,11 @@ std::unique_ptr connect(); } ``` +Normally, the daemon should have a node-ID of its own. It should be possible to run it without one, in the anonymous mode, with limited functionality: + +- The Monitor will not be able to query GetInfo. +- The RegisterClient, PnPNodeIDAllocatorController, FileServerController, NodeCommandClient, etc. will not be operational. + TODO: configuration file format -- tab-separated values; first column contains register name, second column contains the value. TODO: CLI interface From 32fc5c0a76e49837dad1e9ed7a02926e396374ac Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 21:49:12 +0200 Subject: [PATCH 05/24] updates --- DESIGN.md | 84 +++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 60 insertions(+), 24 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 2359629..b40dc58 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -31,6 +31,14 @@ OCVSMD is focused on solving problems that are pervasive in intra-vehicular Open - Centralized (eventually could be distributed for fault tolerance) plug-and-play node-ID allocation server. - Depending on how the named topics project develops (many an intern has despaired over it), the Cyphal resource name server may also be implemented as part of OCVSMD at some point. +Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: + +- Ability to operate from a read-only filesystem. +- Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. +- Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. + +### Dynamic DSDL loading + Dynamic DSDL loading is proposed to be implemented by creating serializer objects whose behavior is defined by the DSDL definition ingested at runtime. The serialization method is to accept a byte stream and to produce a DSDL object model providing named field accessors, similar to what one would find in a JSON serialization library; the deserialization method is the inverse of that. Naturally, said model will heavily utilize PMR for storage. The API could look like: ```c++ @@ -85,11 +93,7 @@ std::pmr::unordered_map read_namespaces(directories, p One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers like Boost Interprocess or specialized PMR. -Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: - -- Ability to operate from a read-only filesystem. -- Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. -- Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. +### C++ API The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. The API is intentionally designed to not hide the structure of the Cyphal protocol itself; that is to say that it is intentionally low-level. Higher-level abstractions can be built on top of it on the client side rather than the daemon side to keep the IPC protocol stable. The `Error` type used in the API definition here is a placeholder for the actual algebraic type listing all possible error states per API entity. @@ -146,19 +150,20 @@ public: /// The daemon always has the standard file server running. /// This interface can be used to configure it. /// It is not possible to stop the server; the closest alternative is to remove all root directories. -class FileServerController +class FileServer { public: /// When the file server handles a request, it will attempt to locate the path relative to each of its root /// directories. See Yakut for a hands-on example. /// The daemon will canonicalize the path and resolve symlinks. /// The same path may be added multiple times to avoid interference across different clients. - virtual std::expected add_root(const std::string_view directory); + /// The path may be that of a file rather than a directory. + virtual std::expected add_root(const std::string_view path); /// Does nothing if such root does not exist (no error reported). /// If such root is listed more than once, only one copy is removed. /// The daemon will canonicalize the path and resolve symlinks. - virtual std::expected remove_root(const std::string_view directory); + virtual std::expected remove_root(const std::string_view path); /// The returned paths are canonicalized. The entries are not unique. virtual std::expected, Error> list_roots() const; @@ -221,7 +226,6 @@ struct MulticastResult std::optional error; }; - /// A helper for manipulating registers on the specified remote nodes. class RegisterClient { @@ -341,7 +345,7 @@ public: /// Implementation detail: internally, the PnP allocator uses the Monitor because the Monitor continuously /// maintains the mapping between node-IDs and their unique-IDs. It needs to subscribe to notifications from the /// monitor; this is not part of the API though. See pycyphal.application.plug_and_play.Allocator. -class PnPNodeIDAllocatorController +class PnPNodeIDAllocator { public: /// Maps unique-ID <=> node-ID. @@ -376,8 +380,8 @@ public: virtual std::expected, Error> make_client(const dsdl::Type& type, const std::uint16_t service_id) = 0; - virtual FileServerController& get_file_server_controller() = 0; - virtual const FileServerController& get_file_server_controller() const = 0; + virtual FileServer& get_file_server() = 0; + virtual const FileServer& get_file_server() const = 0; virtual NodeCommandClient& get_node_command_client() = 0; @@ -386,8 +390,8 @@ public: virtual Monitor& get_monitor() = 0; virtual const Monitor& get_monitor() const = 0; - virtual PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() = 0; - virtual const PnPNodeIDAllocatorController& get_pnp_node_id_allocator_controller() const = 0; + virtual PnPNodeIDAllocator& get_pnp_node_id_allocator() = 0; + virtual const PnPNodeIDAllocator& get_pnp_node_id_allocator() const = 0; }; /// A factory for the abstract factory that connects to the daemon. @@ -396,25 +400,57 @@ std::unique_ptr connect(); } ``` +### Anonymous mode considerations + Normally, the daemon should have a node-ID of its own. It should be possible to run it without one, in the anonymous mode, with limited functionality: - The Monitor will not be able to query GetInfo. -- The RegisterClient, PnPNodeIDAllocatorController, FileServerController, NodeCommandClient, etc. will not be operational. +- The RegisterClient, PnPNodeIDAllocator, FileServer, NodeCommandClient, etc. will not be operational. -TODO: configuration file format -- tab-separated values; first column contains register name, second column contains the value. +### Configuration file format -TODO: CLI interface +The daemon configuration is stored in a TSV file, where each row contains a key, followed by at least one whitespace separator, followed by the value. The keys are register names. Example: -## Milestone 0 +```tsv +uavcan.node.id 123 +uavcan.node.description This is the OCVSMD +uavcan.udp.iface 192.168.1.33 192.168.2.33 +``` + +For the standard register names, refer to . + +### CLI interface + +TBD -SystemV, not systemd. +### Common use cases -File server running continuously. +#### Firmware update -API: integers +Per the design of the OpenCyphal's standard network services, the firmware update process is entirely driven by the node being updated (updatee) rather than the node providing the new firmware file (updater). While it is possible to indirectly infer the progress of the update process by observing the offset of the file reads done by the updatee, this solution is fragile because there is ultimately no guarantee that the updatee will read the file sequentially, or even read it in its entirety. Per the OpenCyphal design, the only relevant parameters of a remote node that can be identified robustly are: + +- Whether a firmware update is currently in progress or not. +- The version numbers, CRC, and VCS ID of the firmare that is currently being executed. + +The proposed API allows one to commence an update process and wait for its completion as follows: + +1. Identify the node that requires a firmware update, and locate a suitable firmware image file on the local machine. +2. `daemon.get_file_server().add_root(firmware_path)`, where `firmware_path` is the path to the new image. +3. `daemon.get_node_command_client().begin_software_update(node_id, firmware_name)`, where `firmware_name` is the last component of the `firmware_path`. +4. Using `daemon.get_monitor().snapshot()`, ensure that the node in question has entered the firmware update mode. Abort if not. +5. Using `daemon.get_monitor().snapshot()`, wait until the node has left the firmware update mode. +6. Using `daemon.get_monitor().snapshot()`, ensure that the firmware version numbers match those of the new image. + +It is possible to build a convenience method that manages the above steps. Said method will be executed on the client side as opposed the daemon side. + +## Milestone 0 -## Milestone 1 +This milestone includes the very barebones implementation, including only: -systemd integration along SystemV. +- The daemon itself, compatible with System V architecture only. Support for systemd will be introduced in a future milestone. +- Running a local Cyphal/UDP node. No support for other transports yet. +- Loading the configuration from the configuration file as defiend above. +- File server. +- Node command client. -## Milestone 2 +These items will be sufficient to perform firmware updates on remote nodes, but not to monitor the update progress. Progress monitoring will require the Monitor module. From e1eef4cdf1bda9473c3f0ec15f11e2160f5a97f2 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 21:52:34 +0200 Subject: [PATCH 06/24] typos --- DESIGN.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index b40dc58..da1e0a9 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -2,15 +2,15 @@ This project implements a user-facing C++14 library backed by a GNU/Linux daemon used to asynchronously perform certain common operations on an OpenCyphal network. Being based on LibCyphal, the solution can theoretically support all transport protocols supported by LibCyphal, notably Cyphal/UDP and Cyphal/CAN. -The implementation will be carried out in multiple stages. The milestones achieved at every stage are described here along with the overall longer-term vision. +The implementation is planned to proceed in multiple stages. The milestones achieved at every stage are described here along with the overall longer-term vision. The design of the C++ API is inspired by the [`ravemodemfactory`](https://github.com/aleksander0m/ravemodemfactory) project (see `src/librmf/rmf-operations.h`). -[Yakut](https://github.com/OpenCyphal/yakut) is a dispantly related project with the following key differences: +[Yakut](https://github.com/OpenCyphal/yakut) is a distantly related project with the following key differences: - Yakut is a developer tool, while OCVSMD is a well-packaged component intended for deployment in production systems. -- Yakut is a user-interactive tool with a CLI interface, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI interface as well, but it will always come secondary to the well-formalized C++ API. +- Yakut is a user-interactive tool with a CLI, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI as well, but it will always come secondary to the well-formalized C++ API. - Yakut is entirely written in Python, and thus it tends to be resource-heavy when used in embedded computers. @@ -339,7 +339,7 @@ public: // TODO: Eventually, we could equip the monitor with snooping support so that we could also obtain: // - Actual traffic per port. // - Update node info and local register cache without sending separate requests. - // Yakut does that with the help of the snooping support in PyCyphal, but LibCyphal does not currently has that capability. + // Yakut does that with the help of the snooping support in PyCyphal, but LibCyphal does not currently have that capability. }; /// Implementation detail: internally, the PnP allocator uses the Monitor because the Monitor continuously @@ -419,7 +419,7 @@ uavcan.udp.iface 192.168.1.33 192.168.2.33 For the standard register names, refer to . -### CLI interface +### CLI TBD @@ -430,7 +430,7 @@ TBD Per the design of the OpenCyphal's standard network services, the firmware update process is entirely driven by the node being updated (updatee) rather than the node providing the new firmware file (updater). While it is possible to indirectly infer the progress of the update process by observing the offset of the file reads done by the updatee, this solution is fragile because there is ultimately no guarantee that the updatee will read the file sequentially, or even read it in its entirety. Per the OpenCyphal design, the only relevant parameters of a remote node that can be identified robustly are: - Whether a firmware update is currently in progress or not. -- The version numbers, CRC, and VCS ID of the firmare that is currently being executed. +- The version numbers, CRC, and VCS ID of the firmware that is currently being executed. The proposed API allows one to commence an update process and wait for its completion as follows: @@ -449,7 +449,7 @@ This milestone includes the very barebones implementation, including only: - The daemon itself, compatible with System V architecture only. Support for systemd will be introduced in a future milestone. - Running a local Cyphal/UDP node. No support for other transports yet. -- Loading the configuration from the configuration file as defiend above. +- Loading the configuration from the configuration file as defined above. - File server. - Node command client. From c8d097a08a42263d5eb4f59b196ed2982b9751ee Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Sun, 8 Dec 2024 22:00:10 +0200 Subject: [PATCH 07/24] Split the API --- DESIGN.md | 456 --------------------------------- docs/DESIGN.md | 107 ++++++++ docs/daemon.hpp | 34 +++ docs/dsdl.hpp | 47 ++++ docs/file_server.hpp | 26 ++ docs/monitor.hpp | 68 +++++ docs/node_command_client.hpp | 42 +++ docs/pnp_node_id_allocator.hpp | 29 +++ docs/pubsub.hpp | 34 +++ docs/register_client.hpp | 83 ++++++ docs/rpc.hpp | 15 ++ 11 files changed, 485 insertions(+), 456 deletions(-) delete mode 100644 DESIGN.md create mode 100644 docs/DESIGN.md create mode 100644 docs/daemon.hpp create mode 100644 docs/dsdl.hpp create mode 100644 docs/file_server.hpp create mode 100644 docs/monitor.hpp create mode 100644 docs/node_command_client.hpp create mode 100644 docs/pnp_node_id_allocator.hpp create mode 100644 docs/pubsub.hpp create mode 100644 docs/register_client.hpp create mode 100644 docs/rpc.hpp diff --git a/DESIGN.md b/DESIGN.md deleted file mode 100644 index da1e0a9..0000000 --- a/DESIGN.md +++ /dev/null @@ -1,456 +0,0 @@ -# Open Cyphal Vehicle System Management Daemon for GNU/Linux - -This project implements a user-facing C++14 library backed by a GNU/Linux daemon used to asynchronously perform certain common operations on an OpenCyphal network. Being based on LibCyphal, the solution can theoretically support all transport protocols supported by LibCyphal, notably Cyphal/UDP and Cyphal/CAN. - -The implementation is planned to proceed in multiple stages. The milestones achieved at every stage are described here along with the overall longer-term vision. - -The design of the C++ API is inspired by the [`ravemodemfactory`](https://github.com/aleksander0m/ravemodemfactory) project (see `src/librmf/rmf-operations.h`). - -[Yakut](https://github.com/OpenCyphal/yakut) is a distantly related project with the following key differences: - -- Yakut is a developer tool, while OCVSMD is a well-packaged component intended for deployment in production systems. - -- Yakut is a user-interactive tool with a CLI, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI as well, but it will always come secondary to the well-formalized C++ API. - -- Yakut is entirely written in Python, and thus it tends to be resource-heavy when used in embedded computers. - -- Yakut is not designed to be highly robust. - -## Long-term vision - -Not all of the listed items will be implemented the way they are seen at the time of writing this document, but the current description provides a general direction things are expected to develop in. - -OCVSMD is focused on solving problems that are pervasive in intra-vehicular OpenCyphal networks with minimal focus on any application-specific details. This list may eventually include: - -- Publish/subscribe on Cyphal subjects with arbitrary DSDL data types loaded at runtime, with the message objects represented as dynamically typed structures. More on this below. -- RPC client for invoking arbitrarily-typed RPC servers with DSDL types loaded at runtime. -- Support for the common Cyphal network services out of the box, configurable via the daemon API: - - File server running with the specified set of root directories (see Yakut). - - Firmware update on a directly specified remote node with a specified filename. - - Automatic firmware update as implemented in Yakut. - - Centralized (eventually could be distributed for fault tolerance) plug-and-play node-ID allocation server. -- Depending on how the named topics project develops (many an intern has despaired over it), the Cyphal resource name server may also be implemented as part of OCVSMD at some point. - -Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: - -- Ability to operate from a read-only filesystem. -- Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. -- Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. - -### Dynamic DSDL loading - -Dynamic DSDL loading is proposed to be implemented by creating serializer objects whose behavior is defined by the DSDL definition ingested at runtime. The serialization method is to accept a byte stream and to produce a DSDL object model providing named field accessors, similar to what one would find in a JSON serialization library; the deserialization method is the inverse of that. Naturally, said model will heavily utilize PMR for storage. The API could look like: - -```c++ -namespace ocvsmd::dsdl -{ -/// Represents a DSDL object of any type. -class Object -{ - friend class Type; -public: - /// Field accessor by name. Empty if no such field. - std::optional operator[](const std::string_view field_name) const; - - /// Array element accessor by index. Empty if out of range. - std::optional> operator[](const std::size_t array_index); - std::optional> operator[](const std::size_t array_index) const; - - /// Coercion to primitives (implicit truncation or the loss of precision are possible). - operator std::optional() const; - operator std::optional() const; - operator std::optional() const; - - /// Coercion from primitives (implicit truncation or the loss of precision are possible). - Object& operator=(const std::int64_t value); - Object& operator=(const std::uint64_t value); - Object& operator=(const double value); - - const class Type& get_type() const noexcept; - - std::expected serialize(const std::span output) const; - std::expected deserialize(const std::span input); -}; - -/// Represents a parsed DSDL definition. -class Type -{ - friend std::pmr::unordered_map read_namespaces(directories, pmr, ...); -public: - /// Constructs a default-initialized Object of this Type. - Object instantiate() const; - ... -}; - -using TypeNameAndVersion = std::tuple; - -/// Reads all definitions from the specified namespaces and returns mapping from the full type name -/// and version to its type model. -/// Optionally, the function should cache the results per namespace, with an option to disable the cache. -std::pmr::unordered_map read_namespaces(directories, pmr, ...); -} -``` - -One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers like Boost Interprocess or specialized PMR. - -### C++ API - -The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. The API is intentionally designed to not hide the structure of the Cyphal protocol itself; that is to say that it is intentionally low-level. Higher-level abstractions can be built on top of it on the client side rather than the daemon side to keep the IPC protocol stable. The `Error` type used in the API definition here is a placeholder for the actual algebraic type listing all possible error states per API entity. - -```c++ -#include -#include -#include -#include -#include - -namespace ocvsmd -{ -/// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. -/// Once instantiated, the published may live on until the daemon process is terminated. -/// Internally, published messages may be transferred to the daemon via an IPC message queue. -class Publisher -{ -public: - /// True on success, false on timeout. - virtual std::expected publish(const std::span data, - const std::chrono::microseconds timeout) = 0; - std::expected publish(const dsdl::Object& obj, const std::chrono::microseconds timeout) - { - // obj.serialize() and this->publish() ... - } -}; - -/// The daemon will implicitly instantiate a subscriber on the specified port-ID the first time it is requested. -/// Once instantiated, the subscriber may live on until the daemon is terminated. -/// -/// The daemon will associate an independent IPC queue with each client-side subscriber and push every received -/// message into the queues. Queues whose client has died are removed. -/// -/// The client has to keep its Subscriber instance alive to avoid losing messages. -class Subscriber -{ -public: - /// Empty if no message received within the timeout. - virtual std::expected, Error> receive(const std::chrono::microseconds timeout) = 0; - - /// TODO: add methods for setting the transfer-ID timeout. -}; - -/// The daemon will implicitly instantiate a client on the specified port-ID and server node when it is requested. -/// Once instantiated, the client may live on until the daemon is terminated. -class RPCClient -{ -public: - /// Returns the response object on success, empty on timeout. - virtual std::expected, Error> call(const dsdl::Object& obj, - const std::chrono::microseconds timeout) = 0; -}; - -/// The daemon always has the standard file server running. -/// This interface can be used to configure it. -/// It is not possible to stop the server; the closest alternative is to remove all root directories. -class FileServer -{ -public: - /// When the file server handles a request, it will attempt to locate the path relative to each of its root - /// directories. See Yakut for a hands-on example. - /// The daemon will canonicalize the path and resolve symlinks. - /// The same path may be added multiple times to avoid interference across different clients. - /// The path may be that of a file rather than a directory. - virtual std::expected add_root(const std::string_view path); - - /// Does nothing if such root does not exist (no error reported). - /// If such root is listed more than once, only one copy is removed. - /// The daemon will canonicalize the path and resolve symlinks. - virtual std::expected remove_root(const std::string_view path); - - /// The returned paths are canonicalized. The entries are not unique. - virtual std::expected, Error> list_roots() const; -}; - -/// A helper for invoking the uavcan.node.ExecuteCommand service on the specified remote nodes. -/// The daemon always has a set of uavcan.node.ExecuteCommand clients ready. -class NodeCommandClient -{ -public: - using Request = uavcan::node::ExecuteCommand_1::Request; - using Response = uavcan::node::ExecuteCommand_1::Response; - - /// Empty response indicates that the associated node did not respond in time. - using Result = std::expected>, Error>; - - /// Empty option indicates that the corresponding node did not return a response on time. - /// All requests are sent concurrently and the call returns when the last response has arrived, - /// or the timeout has expired. - virtual Result send_custom_command(const std::span node_ids, - const Request& request, - const std::chrono::microseconds timeout = 1s) = 0; - - /// A convenience method for invoking send_custom_command() with COMMAND_RESTART. - Result restart(const std::span node_ids, const std::chrono::microseconds timeout = 1s) - { - return send_custom_command(node_ids, {65535, ""}, timeout); - } - - /// A convenience method for invoking send_custom_command() with COMMAND_BEGIN_SOFTWARE_UPDATE. - /// The file_path is relative to one of the roots configured in the file server. - Result begin_software_update(const std::span node_ids, - const std::string_view file_path, - const std::chrono::microseconds timeout = 1s) - { - return send_custom_command(node_ids, {65533, file_path}, timeout); - } - - // TODO: add convenience methods for the other standard commands. -}; - -/// A long operation may fail with partial results being available. The conventional approach to error handling -/// prescribes that the partial results are to be discarded and the error returned. However, said partial -/// results may occasionally be useful, if only to provide the additional context for the error itself. -/// -/// This type allows one to model the normal results obtained from querying multiple remote nodes along with -/// non-exclusive error states both per-node and shared. -/// -/// One alternative is to pass the output container or a callback as an out-parameter to the method, -/// so that the method does not return a new container but updates one in-place. -template -struct MulticastResult -{ - struct PerNode final - { - PerNodeResult result{}; - PerNodeError error{}; - }; - std::pmr::unordered_map result; - std::optional error; -}; - -/// A helper for manipulating registers on the specified remote nodes. -class RegisterClient -{ -public: - using Name = uavcan::_register::Name_1; - using Value = uavcan::_register::Value_1; - - /// This method may return partial results. - virtual MulticastResult, Error, Error> - list(const std::span node_ids) = 0; - - /// Alternative definition of the list method using callbacks instead of partial results. - /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; - /// instead, they can be postponed until the blocking IPC call is finished. - using ListCallback = cetl::function), ...>; - virtual std::expected list(const std::span node_ids, const ListCallback& cb) = 0; - - /// This method may return partial results. - virtual MulticastResult, Error, Error> - read(const std::span node_ids, - const std::pmr::vector& names) = 0; - - /// This method may return partial results. - virtual MulticastResult, Error, Error> - write(const std::span node_ids, - const std::pmr::unordered_map& values) = 0; - - /// Alternative definitions of the read/write methods using callbacks instead of partial results. - /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; - /// instead, they can be postponed until the blocking IPC call is finished. - using ValueCallback = cetl::function&), ...>; - virtual std::expected read(const std::span node_ids, - const std::pmr::vector& names, - const ValueCallback& cb) = 0; - virtual std::expected write(const std::span node_ids, - const std::pmr::unordered_map& values, - const ValueCallback& cb) = 0; - - /// Helper wrappers for the above that operate on a single remote node only. - std::tuple, std::optional> list(const std::uint16_t node_id) - { - // non-virtual implementation - } - std::tuple, Error> - read(const std::uint16_t node_id, const std::pmr::vector& names) - { - // non-virtual implementation - } - std::tuple, Error> - write(const std::uint16_t node_id, const std::pmr::unordered_map& values) - { - // non-virtual implementation - } -}; - -/// The monitor continuously maintains a list of online nodes in the network. -class Monitor -{ -public: - using Heartbeat = uavcan::node::Heartbeat_1; - using NodeInfo = uavcan::node::GetInfo_1::Response; - - /// An avatar represents the latest known state of the remote node. - /// The info struct is available only if the node responded to a uavcan.node.GetInfo request since last bootup. - /// GetInfo requests are sent continuously until a response is received. - /// If heartbeat publications cease, the corresponding node is marked as offline. - struct Avatar - { - std::uint16_t node_id; - - bool is_online; ///< If not online, the other fields contain the latest known information. - - std::chrono::system_clock::time_point last_heartbeat_at; - Heartbeat last_heartbeat; - - /// The info is automatically reset when the remote node is detected to have restarted. - /// It is automatically re-populated as soon as a GetInfo response is received. - struct Info final - { - std::chrono::system_clock::time_point received_at; - NodeInfo info; - }; - std::optional info; - - /// The port list is automatically reset when the remote node is detected to have restarted. - /// It is automatically re-populated as soon as an update is received. - struct PortList final - { - std::chrono::system_clock::time_point received_at; - std::bitset<65536> publishers; - std::bitset<65536> subscribers; - std::bitset<512> clients; - std::bitset<512> servers; - }; - std::optional port_list; - }; - - struct Snapshot final - { - /// If a node appears online at least once, it will be given a slot in the table permanently. - /// If it goes offline, it will be retained in the table but it's is_online field will be false. - /// The table is ordered by node-ID. Use binary search for fast lookup. - std::pmr::vector table; - std::tuple daemon; - bool has_anonymous; ///< If any anonymous nodes are online (e.g., someone is trying to get a PnP node-ID allocation) - }; - - /// Returns a snapshot of the current network state plus the daemon's own node state. - virtual Snapshot snap() const = 0; - - // TODO: Eventually, we could equip the monitor with snooping support so that we could also obtain: - // - Actual traffic per port. - // - Update node info and local register cache without sending separate requests. - // Yakut does that with the help of the snooping support in PyCyphal, but LibCyphal does not currently have that capability. -}; - -/// Implementation detail: internally, the PnP allocator uses the Monitor because the Monitor continuously -/// maintains the mapping between node-IDs and their unique-IDs. It needs to subscribe to notifications from the -/// monitor; this is not part of the API though. See pycyphal.application.plug_and_play.Allocator. -class PnPNodeIDAllocator -{ -public: - /// Maps unique-ID <=> node-ID. - /// For some node-IDs there may be no unique-ID (at least temporarily until a GetInfo response is received). - /// The table includes the daemon's node as well. - using UID = std::array; - using Entry = std::tuple>; - using Table = std::pmr::vector; - - /// The method is infallible because the corresponding publishers/subscribers are always active; - /// when enabled==false, the allocator simply refuses to send responses. - virtual void set_enabled(const bool enabled) = 0; - virtual bool is_enabled() const = 0; - - /// The allocation table may or may not be persistent (retained across daemon restarts). - virtual Table get_table() const = 0; - - /// Forget all allocations; the table will be rebuilt from the Monitor state. - virtual void drop_table() = 0; -}; - -/// An abstract factory for the specialized interfaces. -class Daemon -{ -public: - virtual std::expected, Error> make_publisher(const dsdl::Type& type, - const std::uint16_t subject_id) = 0; - - virtual std::expected, Error> make_subscriber(const dsdl::Type& type, - const std::uint16_t subject_id) = 0; - - virtual std::expected, Error> make_client(const dsdl::Type& type, - const std::uint16_t service_id) = 0; - - virtual FileServer& get_file_server() = 0; - virtual const FileServer& get_file_server() const = 0; - - virtual NodeCommandClient& get_node_command_client() = 0; - - virtual RegisterClient& get_register_client() = 0; - - virtual Monitor& get_monitor() = 0; - virtual const Monitor& get_monitor() const = 0; - - virtual PnPNodeIDAllocator& get_pnp_node_id_allocator() = 0; - virtual const PnPNodeIDAllocator& get_pnp_node_id_allocator() const = 0; -}; - -/// A factory for the abstract factory that connects to the daemon. -/// Returns nullptr if the daemon cannot be connected to (not running). -std::unique_ptr connect(); -} -``` - -### Anonymous mode considerations - -Normally, the daemon should have a node-ID of its own. It should be possible to run it without one, in the anonymous mode, with limited functionality: - -- The Monitor will not be able to query GetInfo. -- The RegisterClient, PnPNodeIDAllocator, FileServer, NodeCommandClient, etc. will not be operational. - -### Configuration file format - -The daemon configuration is stored in a TSV file, where each row contains a key, followed by at least one whitespace separator, followed by the value. The keys are register names. Example: - -```tsv -uavcan.node.id 123 -uavcan.node.description This is the OCVSMD -uavcan.udp.iface 192.168.1.33 192.168.2.33 -``` - -For the standard register names, refer to . - -### CLI - -TBD - -### Common use cases - -#### Firmware update - -Per the design of the OpenCyphal's standard network services, the firmware update process is entirely driven by the node being updated (updatee) rather than the node providing the new firmware file (updater). While it is possible to indirectly infer the progress of the update process by observing the offset of the file reads done by the updatee, this solution is fragile because there is ultimately no guarantee that the updatee will read the file sequentially, or even read it in its entirety. Per the OpenCyphal design, the only relevant parameters of a remote node that can be identified robustly are: - -- Whether a firmware update is currently in progress or not. -- The version numbers, CRC, and VCS ID of the firmware that is currently being executed. - -The proposed API allows one to commence an update process and wait for its completion as follows: - -1. Identify the node that requires a firmware update, and locate a suitable firmware image file on the local machine. -2. `daemon.get_file_server().add_root(firmware_path)`, where `firmware_path` is the path to the new image. -3. `daemon.get_node_command_client().begin_software_update(node_id, firmware_name)`, where `firmware_name` is the last component of the `firmware_path`. -4. Using `daemon.get_monitor().snapshot()`, ensure that the node in question has entered the firmware update mode. Abort if not. -5. Using `daemon.get_monitor().snapshot()`, wait until the node has left the firmware update mode. -6. Using `daemon.get_monitor().snapshot()`, ensure that the firmware version numbers match those of the new image. - -It is possible to build a convenience method that manages the above steps. Said method will be executed on the client side as opposed the daemon side. - -## Milestone 0 - -This milestone includes the very barebones implementation, including only: - -- The daemon itself, compatible with System V architecture only. Support for systemd will be introduced in a future milestone. -- Running a local Cyphal/UDP node. No support for other transports yet. -- Loading the configuration from the configuration file as defined above. -- File server. -- Node command client. - -These items will be sufficient to perform firmware updates on remote nodes, but not to monitor the update progress. Progress monitoring will require the Monitor module. diff --git a/docs/DESIGN.md b/docs/DESIGN.md new file mode 100644 index 0000000..b3c89a0 --- /dev/null +++ b/docs/DESIGN.md @@ -0,0 +1,107 @@ +# Open Cyphal Vehicle System Management Daemon for GNU/Linux + +This project implements a user-facing C++14 library backed by a GNU/Linux daemon used to asynchronously perform certain common operations on an OpenCyphal network. Being based on LibCyphal, the solution can theoretically support all transport protocols supported by LibCyphal, notably Cyphal/UDP and Cyphal/CAN. + +The implementation is planned to proceed in multiple stages. The milestones achieved at every stage are described here along with the overall longer-term vision. + +The design of the C++ API is inspired by the [`ravemodemfactory`](https://github.com/aleksander0m/ravemodemfactory) project (see `src/librmf/rmf-operations.h`). + +[Yakut](https://github.com/OpenCyphal/yakut) is a distantly related project with the following key differences: + +- Yakut is a developer tool, while OCVSMD is a well-packaged component intended for deployment in production systems. + +- Yakut is a user-interactive tool with a CLI, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI as well, but it will always come secondary to the well-formalized C++ API. + +- Yakut is entirely written in Python, and thus it tends to be resource-heavy when used in embedded computers. + +- Yakut is not designed to be highly robust. + +## Long-term vision + +Not all of the listed items will be implemented the way they are seen at the time of writing this document, but the current description provides a general direction things are expected to develop in. + +OCVSMD is focused on solving problems that are pervasive in intra-vehicular OpenCyphal networks with minimal focus on any application-specific details. This list may eventually include: + +- Publish/subscribe on Cyphal subjects with arbitrary DSDL data types loaded at runtime, with the message objects represented as dynamically typed structures. More on this below. +- RPC client for invoking arbitrarily-typed RPC servers with DSDL types loaded at runtime. +- Support for the common Cyphal network services out of the box, configurable via the daemon API: + - File server running with the specified set of root directories (see Yakut). + - Firmware update on a directly specified remote node with a specified filename. + - Automatic firmware update as implemented in Yakut. + - Centralized (eventually could be distributed for fault tolerance) plug-and-play node-ID allocation server. +- Depending on how the named topics project develops (many an intern has despaired over it), the Cyphal resource name server may also be implemented as part of OCVSMD at some point. + +Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: + +- Ability to operate from a read-only filesystem. +- Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. +- Local node configuration ((redundant) transport configuration, node-ID, node description, etc) is loaded from a file, which is common for daemons. + +### Dynamic DSDL loading + +Dynamic DSDL loading is proposed to be implemented by creating serializer objects whose behavior is defined by the DSDL definition ingested at runtime. The serialization method is to accept a byte stream and to produce a DSDL object model providing named field accessors, similar to what one would find in a JSON serialization library; the deserialization method is the inverse of that. Naturally, said model will heavily utilize PMR for storage. An API mockup is given in `dsdl.hpp`. + +One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers like Boost Interprocess or specialized PMR. + +### C++ API + +The API will consist of several well-segregated C++ interfaces, each dedicated to a particular feature subset. The interface-based design is chosen to simplify testing in client applications. The API is intentionally designed to not hide the structure of the Cyphal protocol itself; that is to say that it is intentionally low-level. Higher-level abstractions can be built on top of it on the client side rather than the daemon side to keep the IPC protocol stable. + +The `Error` type used in the API definition here is a placeholder for the actual algebraic type listing all possible error states per API entity. + +The main file of the C++ API is the `daemon.hpp`, which contains the abstract factory `Daemon` for the specialized interfaces, as well as the static factory factory (sic) `connect() -> Daemon`. + +### Anonymous mode considerations + +Normally, the daemon should have a node-ID of its own. It should be possible to run it without one, in the anonymous mode, with limited functionality: + +- The Monitor will not be able to query GetInfo. +- The RegisterClient, PnPNodeIDAllocator, FileServer, NodeCommandClient, etc. will not be operational. + +### Configuration file format + +The daemon configuration is stored in a TSV file, where each row contains a key, followed by at least one whitespace separator, followed by the value. The keys are register names. Example: + +```tsv +uavcan.node.id 123 +uavcan.node.description This is the OCVSMD +uavcan.udp.iface 192.168.1.33 192.168.2.33 +``` + +For the standard register names, refer to . + +### CLI + +TBD + +### Common use cases + +#### Firmware update + +Per the design of the OpenCyphal's standard network services, the firmware update process is entirely driven by the node being updated (updatee) rather than the node providing the new firmware file (updater). While it is possible to indirectly infer the progress of the update process by observing the offset of the file reads done by the updatee, this solution is fragile because there is ultimately no guarantee that the updatee will read the file sequentially, or even read it in its entirety. Per the OpenCyphal design, the only relevant parameters of a remote node that can be identified robustly are: + +- Whether a firmware update is currently in progress or not. +- The version numbers, CRC, and VCS ID of the firmware that is currently being executed. + +The proposed API allows one to commence an update process and wait for its completion as follows: + +1. Identify the node that requires a firmware update, and locate a suitable firmware image file on the local machine. +2. `daemon.get_file_server().add_root(firmware_path)`, where `firmware_path` is the path to the new image. +3. `daemon.get_node_command_client().begin_software_update(node_id, firmware_name)`, where `firmware_name` is the last component of the `firmware_path`. +4. Using `daemon.get_monitor().snapshot()`, ensure that the node in question has entered the firmware update mode. Abort if not. +5. Using `daemon.get_monitor().snapshot()`, wait until the node has left the firmware update mode. +6. Using `daemon.get_monitor().snapshot()`, ensure that the firmware version numbers match those of the new image. + +It is possible to build a convenience method that manages the above steps. Said method will be executed on the client side as opposed the daemon side. + +## Milestone 0 + +This milestone includes the very barebones implementation, including only: + +- The daemon itself, compatible with System V architecture only. Support for systemd will be introduced in a future milestone. +- Running a local Cyphal/UDP node. No support for other transports yet. +- Loading the configuration from the configuration file as defined above. +- File server. +- Node command client. + +These items will be sufficient to perform firmware updates on remote nodes, but not to monitor the update progress. Progress monitoring will require the Monitor module. diff --git a/docs/daemon.hpp b/docs/daemon.hpp new file mode 100644 index 0000000..2844d71 --- /dev/null +++ b/docs/daemon.hpp @@ -0,0 +1,34 @@ +namespace ocvsmd +{ + +/// An abstract factory for the specialized interfaces. +class Daemon +{ +public: + virtual std::expected, Error> make_publisher(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_subscriber(const dsdl::Type& type, + const std::uint16_t subject_id) = 0; + + virtual std::expected, Error> make_client(const dsdl::Type& type, + const std::uint16_t service_id) = 0; + + virtual FileServer& get_file_server() = 0; + virtual const FileServer& get_file_server() const = 0; + + virtual NodeCommandClient& get_node_command_client() = 0; + + virtual RegisterClient& get_register_client() = 0; + + virtual Monitor& get_monitor() = 0; + virtual const Monitor& get_monitor() const = 0; + + virtual PnPNodeIDAllocator& get_pnp_node_id_allocator() = 0; + virtual const PnPNodeIDAllocator& get_pnp_node_id_allocator() const = 0; +}; + +/// A factory for the abstract factory that connects to the daemon. +/// Returns nullptr if the daemon cannot be connected to (not running). +std::unique_ptr connect(); +} diff --git a/docs/dsdl.hpp b/docs/dsdl.hpp new file mode 100644 index 0000000..098965b --- /dev/null +++ b/docs/dsdl.hpp @@ -0,0 +1,47 @@ +namespace ocvsmd::dsdl +{ +/// Represents a DSDL object of any type. +class Object +{ + friend class Type; +public: + /// Field accessor by name. Empty if no such field. + std::optional operator[](const std::string_view field_name) const; + + /// Array element accessor by index. Empty if out of range. + std::optional> operator[](const std::size_t array_index); + std::optional> operator[](const std::size_t array_index) const; + + /// Coercion to primitives (implicit truncation or the loss of precision are possible). + operator std::optional() const; + operator std::optional() const; + operator std::optional() const; + + /// Coercion from primitives (implicit truncation or the loss of precision are possible). + Object& operator=(const std::int64_t value); + Object& operator=(const std::uint64_t value); + Object& operator=(const double value); + + const class Type& get_type() const noexcept; + + std::expected serialize(const std::span output) const; + std::expected deserialize(const std::span input); +}; + +/// Represents a parsed DSDL definition. +class Type +{ + friend std::pmr::unordered_map read_namespaces(directories, pmr, ...); +public: + /// Constructs a default-initialized Object of this Type. + Object instantiate() const; + ... +}; + +using TypeNameAndVersion = std::tuple; + +/// Reads all definitions from the specified namespaces and returns mapping from the full type name +/// and version to its type model. +/// Optionally, the function should cache the results per namespace, with an option to disable the cache. +std::pmr::unordered_map read_namespaces(directories, pmr, ...); +} diff --git a/docs/file_server.hpp b/docs/file_server.hpp new file mode 100644 index 0000000..5d4cd29 --- /dev/null +++ b/docs/file_server.hpp @@ -0,0 +1,26 @@ +namespace ocvsmd +{ + +/// The daemon always has the standard file server running. +/// This interface can be used to configure it. +/// It is not possible to stop the server; the closest alternative is to remove all root directories. +class FileServer +{ +public: + /// When the file server handles a request, it will attempt to locate the path relative to each of its root + /// directories. See Yakut for a hands-on example. + /// The daemon will canonicalize the path and resolve symlinks. + /// The same path may be added multiple times to avoid interference across different clients. + /// The path may be that of a file rather than a directory. + virtual std::expected add_root(const std::string_view path); + + /// Does nothing if such root does not exist (no error reported). + /// If such root is listed more than once, only one copy is removed. + /// The daemon will canonicalize the path and resolve symlinks. + virtual std::expected remove_root(const std::string_view path); + + /// The returned paths are canonicalized. The entries are not unique. + virtual std::expected, Error> list_roots() const; +}; + +} diff --git a/docs/monitor.hpp b/docs/monitor.hpp new file mode 100644 index 0000000..26303cc --- /dev/null +++ b/docs/monitor.hpp @@ -0,0 +1,68 @@ +#include +#include + +namespace ocvsmd +{ + +/// The monitor continuously maintains a list of online nodes in the network. +class Monitor +{ +public: + using Heartbeat = uavcan::node::Heartbeat_1; + using NodeInfo = uavcan::node::GetInfo_1::Response; + + /// An avatar represents the latest known state of the remote node. + /// The info struct is available only if the node responded to a uavcan.node.GetInfo request since last bootup. + /// GetInfo requests are sent continuously until a response is received. + /// If heartbeat publications cease, the corresponding node is marked as offline. + struct Avatar + { + std::uint16_t node_id; + + bool is_online; ///< If not online, the other fields contain the latest known information. + + std::chrono::system_clock::time_point last_heartbeat_at; + Heartbeat last_heartbeat; + + /// The info is automatically reset when the remote node is detected to have restarted. + /// It is automatically re-populated as soon as a GetInfo response is received. + struct Info final + { + std::chrono::system_clock::time_point received_at; + NodeInfo info; + }; + std::optional info; + + /// The port list is automatically reset when the remote node is detected to have restarted. + /// It is automatically re-populated as soon as an update is received. + struct PortList final + { + std::chrono::system_clock::time_point received_at; + std::bitset<65536> publishers; + std::bitset<65536> subscribers; + std::bitset<512> clients; + std::bitset<512> servers; + }; + std::optional port_list; + }; + + struct Snapshot final + { + /// If a node appears online at least once, it will be given a slot in the table permanently. + /// If it goes offline, it will be retained in the table but it's is_online field will be false. + /// The table is ordered by node-ID. Use binary search for fast lookup. + std::pmr::vector table; + std::tuple daemon; + bool has_anonymous; ///< If any anonymous nodes are online (e.g., someone is trying to get a PnP node-ID allocation) + }; + + /// Returns a snapshot of the current network state plus the daemon's own node state. + virtual Snapshot snap() const = 0; + + // TODO: Eventually, we could equip the monitor with snooping support so that we could also obtain: + // - Actual traffic per port. + // - Update node info and local register cache without sending separate requests. + // Yakut does that with the help of the snooping support in PyCyphal, but LibCyphal does not currently have that capability. +}; + +} diff --git a/docs/node_command_client.hpp b/docs/node_command_client.hpp new file mode 100644 index 0000000..f65983a --- /dev/null +++ b/docs/node_command_client.hpp @@ -0,0 +1,42 @@ +#include + +namespace ocvsmd +{ + +/// A helper for invoking the uavcan.node.ExecuteCommand service on the specified remote nodes. +/// The daemon always has a set of uavcan.node.ExecuteCommand clients ready. +class NodeCommandClient +{ +public: + using Request = uavcan::node::ExecuteCommand_1::Request; + using Response = uavcan::node::ExecuteCommand_1::Response; + + /// Empty response indicates that the associated node did not respond in time. + using Result = std::expected>, Error>; + + /// Empty option indicates that the corresponding node did not return a response on time. + /// All requests are sent concurrently and the call returns when the last response has arrived, + /// or the timeout has expired. + virtual Result send_custom_command(const std::span node_ids, + const Request& request, + const std::chrono::microseconds timeout = 1s) = 0; + + /// A convenience method for invoking send_custom_command() with COMMAND_RESTART. + Result restart(const std::span node_ids, const std::chrono::microseconds timeout = 1s) + { + return send_custom_command(node_ids, {65535, ""}, timeout); + } + + /// A convenience method for invoking send_custom_command() with COMMAND_BEGIN_SOFTWARE_UPDATE. + /// The file_path is relative to one of the roots configured in the file server. + Result begin_software_update(const std::span node_ids, + const std::string_view file_path, + const std::chrono::microseconds timeout = 1s) + { + return send_custom_command(node_ids, {65533, file_path}, timeout); + } + + // TODO: add convenience methods for the other standard commands. +}; + +} diff --git a/docs/pnp_node_id_allocator.hpp b/docs/pnp_node_id_allocator.hpp new file mode 100644 index 0000000..add84a7 --- /dev/null +++ b/docs/pnp_node_id_allocator.hpp @@ -0,0 +1,29 @@ +namespace ocvsmd +{ + +/// Implementation detail: internally, the PnP allocator uses the Monitor because the Monitor continuously +/// maintains the mapping between node-IDs and their unique-IDs. It needs to subscribe to notifications from the +/// monitor; this is not part of the API though. See pycyphal.application.plug_and_play.Allocator. +class PnPNodeIDAllocator +{ +public: + /// Maps unique-ID <=> node-ID. + /// For some node-IDs there may be no unique-ID (at least temporarily until a GetInfo response is received). + /// The table includes the daemon's node as well. + using UID = std::array; + using Entry = std::tuple>; + using Table = std::pmr::vector; + + /// The method is infallible because the corresponding publishers/subscribers are always active; + /// when enabled==false, the allocator simply refuses to send responses. + virtual void set_enabled(const bool enabled) = 0; + virtual bool is_enabled() const = 0; + + /// The allocation table may or may not be persistent (retained across daemon restarts). + virtual Table get_table() const = 0; + + /// Forget all allocations; the table will be rebuilt from the Monitor state. + virtual void drop_table() = 0; +}; + +} diff --git a/docs/pubsub.hpp b/docs/pubsub.hpp new file mode 100644 index 0000000..8983db4 --- /dev/null +++ b/docs/pubsub.hpp @@ -0,0 +1,34 @@ +namespace ocvsmd +{ +/// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. +/// Once instantiated, the published may live on until the daemon process is terminated. +/// Internally, published messages may be transferred to the daemon via an IPC message queue. +class Publisher +{ +public: + /// True on success, false on timeout. + virtual std::expected publish(const std::span data, + const std::chrono::microseconds timeout) = 0; + std::expected publish(const dsdl::Object& obj, const std::chrono::microseconds timeout) + { + // obj.serialize() and this->publish() ... + } +}; + +/// The daemon will implicitly instantiate a subscriber on the specified port-ID the first time it is requested. +/// Once instantiated, the subscriber may live on until the daemon is terminated. +/// +/// The daemon will associate an independent IPC queue with each client-side subscriber and push every received +/// message into the queues. Queues whose client has died are removed. +/// +/// The client has to keep its Subscriber instance alive to avoid losing messages. +class Subscriber +{ +public: + /// Empty if no message received within the timeout. + virtual std::expected, Error> receive(const std::chrono::microseconds timeout) = 0; + + /// TODO: add methods for setting the transfer-ID timeout. +}; + +} diff --git a/docs/register_client.hpp b/docs/register_client.hpp new file mode 100644 index 0000000..fc9fff1 --- /dev/null +++ b/docs/register_client.hpp @@ -0,0 +1,83 @@ +#include +#include + +namespace ocvsmd +{ + +/// A long operation may fail with partial results being available. The conventional approach to error handling +/// prescribes that the partial results are to be discarded and the error returned. However, said partial +/// results may occasionally be useful, if only to provide the additional context for the error itself. +/// +/// This type allows one to model the normal results obtained from querying multiple remote nodes along with +/// non-exclusive error states both per-node and shared. +/// +/// One alternative is to pass the output container or a callback as an out-parameter to the method, +/// so that the method does not return a new container but updates one in-place. +template +struct MulticastResult +{ + struct PerNode final + { + PerNodeResult result{}; + PerNodeError error{}; + }; + std::pmr::unordered_map result; + std::optional error; +}; + +/// A helper for manipulating registers on the specified remote nodes. +class RegisterClient +{ +public: + using Name = uavcan::_register::Name_1; + using Value = uavcan::_register::Value_1; + + /// This method may return partial results. + virtual MulticastResult, Error, Error> + list(const std::span node_ids) = 0; + + /// Alternative definition of the list method using callbacks instead of partial results. + /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; + /// instead, they can be postponed until the blocking IPC call is finished. + using ListCallback = cetl::function), ...>; + virtual std::expected list(const std::span node_ids, const ListCallback& cb) = 0; + + /// This method may return partial results. + virtual MulticastResult, Error, Error> + read(const std::span node_ids, + const std::pmr::vector& names) = 0; + + /// This method may return partial results. + virtual MulticastResult, Error, Error> + write(const std::span node_ids, + const std::pmr::unordered_map& values) = 0; + + /// Alternative definitions of the read/write methods using callbacks instead of partial results. + /// The callbacks do not have to be invoked in real-time to simplify the IPC interface; + /// instead, they can be postponed until the blocking IPC call is finished. + using ValueCallback = cetl::function&), ...>; + virtual std::expected read(const std::span node_ids, + const std::pmr::vector& names, + const ValueCallback& cb) = 0; + virtual std::expected write(const std::span node_ids, + const std::pmr::unordered_map& values, + const ValueCallback& cb) = 0; + + /// Helper wrappers for the above that operate on a single remote node only. + std::tuple, std::optional> list(const std::uint16_t node_id) + { + // non-virtual implementation + } + std::tuple, Error> + read(const std::uint16_t node_id, const std::pmr::vector& names) + { + // non-virtual implementation + } + std::tuple, Error> + write(const std::uint16_t node_id, const std::pmr::unordered_map& values) + { + // non-virtual implementation + } +}; + +} diff --git a/docs/rpc.hpp b/docs/rpc.hpp new file mode 100644 index 0000000..433129f --- /dev/null +++ b/docs/rpc.hpp @@ -0,0 +1,15 @@ + +namespace ocvsmd +{ + +/// The daemon will implicitly instantiate a client on the specified port-ID and server node when it is requested. +/// Once instantiated, the client may live on until the daemon is terminated. +class RPCClient +{ +public: + /// Returns the response object on success, empty on timeout. + virtual std::expected, Error> call(const dsdl::Object& obj, + const std::chrono::microseconds timeout) = 0; +}; + +} From 4c971988145bf7402537a4446bd6efd7d79866e4 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Tue, 10 Dec 2024 14:11:08 +0200 Subject: [PATCH 08/24] Feedback --- README.md | 5 +++-- docs/DESIGN.md | 13 ++++++++----- docs/daemon.hpp | 5 +++-- docs/monitor.hpp | 6 +++--- docs/pubsub.hpp | 2 +- 5 files changed, 18 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 53b72d8..120fecf 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,3 @@ -# yakut-native -A CLI tool for diagnostics and debugging of Cyphal networks, written in C++, suitable for embedded computers +# OpenCyphal Vehicle System Management Daemon + +👻 diff --git a/docs/DESIGN.md b/docs/DESIGN.md index b3c89a0..8fe04af 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -10,11 +10,11 @@ The design of the C++ API is inspired by the [`ravemodemfactory`](https://github - Yakut is a developer tool, while OCVSMD is a well-packaged component intended for deployment in production systems. -- Yakut is a user-interactive tool with a CLI, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSDM may be equipped with a CLI as well, but it will always come secondary to the well-formalized C++ API. +- Yakut is a user-interactive tool with a CLI, while OCVSMD is equipped with a machine-friendly interface -- a C++ API. Eventually, OCVSMD may be equipped with a CLI as well, but it will always come secondary to the well-formalized C++ API. -- Yakut is entirely written in Python, and thus it tends to be resource-heavy when used in embedded computers. +- OCVSMD will be suitable for embedded Linux systems including such systems running on single-core "cross-over" processors. -- Yakut is not designed to be highly robust. +- OCVSMD will be robust. ## Long-term vision @@ -30,8 +30,9 @@ OCVSMD is focused on solving problems that are pervasive in intra-vehicular Open - Automatic firmware update as implemented in Yakut. - Centralized (eventually could be distributed for fault tolerance) plug-and-play node-ID allocation server. - Depending on how the named topics project develops (many an intern has despaired over it), the Cyphal resource name server may also be implemented as part of OCVSMD at some point. +- A possible future node authentication protocol may also be implemented in this project. -Being a daemon designed for unattended operation in deeply-embedded vehicular computers, OCVSMD must meet the following requirements: +Being a daemon designed for unattended operation in embedded vehicular computers, OCVSMD must meet the following requirements: - Ability to operate from a read-only filesystem. - Startup time much faster than that of Yakut. This should not be an issue for a native application since most of the Yakut startup time is spent on the Python runtime initialization, compilation, and module importing. @@ -41,7 +42,9 @@ Being a daemon designed for unattended operation in deeply-embedded vehicular co Dynamic DSDL loading is proposed to be implemented by creating serializer objects whose behavior is defined by the DSDL definition ingested at runtime. The serialization method is to accept a byte stream and to produce a DSDL object model providing named field accessors, similar to what one would find in a JSON serialization library; the deserialization method is the inverse of that. Naturally, said model will heavily utilize PMR for storage. An API mockup is given in `dsdl.hpp`. -One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers like Boost Interprocess or specialized PMR. +One approach assumes that instances of `dsdl::Object` are not exchanged between the client and the daemon; instead, only their serialized representations are transferred between the processes; thus, the entire DSDL support machinery exists in the client's process only. This approach involves certain work duplication between clients, and may impair their ability to start up quickly if DSDL parsing needs to be done. Another approach is to use shared-memory-friendly containers, e.g., via specialized PMR. + +Irrespective of how the dynamic DSDL loading is implemented, the standard data types located in the `uavcan` namespace will be compiled into both the daemon and the clients, as they are used in the API definition -- more on this below. ### C++ API diff --git a/docs/daemon.hpp b/docs/daemon.hpp index 2844d71..d31d2ac 100644 --- a/docs/daemon.hpp +++ b/docs/daemon.hpp @@ -12,6 +12,7 @@ class Daemon const std::uint16_t subject_id) = 0; virtual std::expected, Error> make_client(const dsdl::Type& type, + const std::uint16_t server_node_id, const std::uint16_t service_id) = 0; virtual FileServer& get_file_server() = 0; @@ -29,6 +30,6 @@ class Daemon }; /// A factory for the abstract factory that connects to the daemon. -/// Returns nullptr if the daemon cannot be connected to (not running). -std::unique_ptr connect(); +/// The pointer is never null on success. +std::expected, Error> connect(); } diff --git a/docs/monitor.hpp b/docs/monitor.hpp index 26303cc..4600cdf 100644 --- a/docs/monitor.hpp +++ b/docs/monitor.hpp @@ -11,11 +11,11 @@ class Monitor using Heartbeat = uavcan::node::Heartbeat_1; using NodeInfo = uavcan::node::GetInfo_1::Response; - /// An avatar represents the latest known state of the remote node. + /// A shadow represents the latest known state of the remote node. /// The info struct is available only if the node responded to a uavcan.node.GetInfo request since last bootup. /// GetInfo requests are sent continuously until a response is received. /// If heartbeat publications cease, the corresponding node is marked as offline. - struct Avatar + struct Shadow final { std::uint16_t node_id; @@ -51,7 +51,7 @@ class Monitor /// If a node appears online at least once, it will be given a slot in the table permanently. /// If it goes offline, it will be retained in the table but it's is_online field will be false. /// The table is ordered by node-ID. Use binary search for fast lookup. - std::pmr::vector table; + std::pmr::vector table; std::tuple daemon; bool has_anonymous; ///< If any anonymous nodes are online (e.g., someone is trying to get a PnP node-ID allocation) }; diff --git a/docs/pubsub.hpp b/docs/pubsub.hpp index 8983db4..3f5be89 100644 --- a/docs/pubsub.hpp +++ b/docs/pubsub.hpp @@ -1,7 +1,7 @@ namespace ocvsmd { /// The daemon will implicitly instantiate a publisher on the specified port-ID the first time it is requested. -/// Once instantiated, the published may live on until the daemon process is terminated. +/// Once instantiated, the publisher may live on until the daemon process is terminated. /// Internally, published messages may be transferred to the daemon via an IPC message queue. class Publisher { From 240a0c00f3649dee1404b2ab30a0001f161aec92 Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Tue, 10 Dec 2024 14:37:57 +0200 Subject: [PATCH 09/24] Add notes about progress reporting --- docs/DESIGN.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 8fe04af..962f076 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -97,6 +97,18 @@ The proposed API allows one to commence an update process and wait for its compl It is possible to build a convenience method that manages the above steps. Said method will be executed on the client side as opposed the daemon side. +##### Progress monitoring + +To enable monitoring the progress of a firmware update process, the following solutions have been considered and rejected: + +- Add an additional general-purpose numerical field to `uavcan.node.ExecuteCommand.1` that returns the progress information when an appropriate command (a new standard command) is sent. This is rejected because an RPC-based solution is undesirable. + +- Report the progress via `uavcan.node.Heartbeat.1.vendor_specific_status_code`. This is rejected because the VSSC is vendor-specific, so it shouldn't be relied on by the standard. + +The plan that is tentatively agreed upon is to define a new standard message with a fixed port-ID for needs of progress reporting. The message will likely be placed in the diagnostics namespace as `uavcan.diagnostic.ProgressReport` with a fixed port-ID of 8183. The `uavcan.node.ExecuteCommand.1` RPC may return a flag indicating if the progress of the freshly launched process will be reported via the new message. + +If this message-based approach is chosen, the daemon will subscribe to the message and provide the latest received progress value per node via the monitor interface. + ## Milestone 0 This milestone includes the very barebones implementation, including only: From 2bec5038595ee81dee3f5c45221af1ef50611ac6 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Wed, 11 Dec 2024 17:36:33 +0200 Subject: [PATCH 10/24] initial source code structure --- .clang-format | 86 +++++++++++++++++++++++++ .clang-tidy | 37 +++++++++++ .cspell/custom-dictionary-workspace.txt | 2 + .github/workflows/tests.yml | 0 .gitignore | 15 +++++ .vscode/settings.json | 10 +++ CMakeLists.txt | 21 ++++++ src/CMakeLists.txt | 10 +++ src/cli/CMakeLists.txt | 6 ++ src/common/CMakeLists.txt | 6 ++ src/daemon/CMakeLists.txt | 8 +++ src/daemon/main.cpp | 8 +++ test/CMakeLists.txt | 10 +++ test/cli/CMakeLists.txt | 6 ++ test/common/CMakeLists.txt | 6 ++ test/daemon/CMakeLists.txt | 6 ++ 16 files changed, 237 insertions(+) create mode 100644 .clang-format create mode 100644 .clang-tidy create mode 100644 .cspell/custom-dictionary-workspace.txt create mode 100644 .github/workflows/tests.yml create mode 100644 .vscode/settings.json create mode 100644 CMakeLists.txt create mode 100644 src/CMakeLists.txt create mode 100644 src/cli/CMakeLists.txt create mode 100644 src/common/CMakeLists.txt create mode 100644 src/daemon/CMakeLists.txt create mode 100644 src/daemon/main.cpp create mode 100644 test/CMakeLists.txt create mode 100644 test/cli/CMakeLists.txt create mode 100644 test/common/CMakeLists.txt create mode 100644 test/daemon/CMakeLists.txt diff --git a/.clang-format b/.clang-format new file mode 100644 index 0000000..276b656 --- /dev/null +++ b/.clang-format @@ -0,0 +1,86 @@ +--- +Language: Cpp +BasedOnStyle: LLVM +AccessModifierOffset: -4 +AlignAfterOpenBracket: Align +AlignConsecutiveAssignments: true +AlignConsecutiveDeclarations: true +AlignEscapedNewlines: Left +AlignOperands: true +AlignTrailingComments: true +AllowAllParametersOfDeclarationOnNextLine: false +AllowShortBlocksOnASingleLine: Never +AllowShortCaseLabelsOnASingleLine: false +AllowShortFunctionsOnASingleLine: Empty +AllowShortIfStatementsOnASingleLine: false +AllowShortLoopsOnASingleLine: false +AlwaysBreakAfterDefinitionReturnType: None +AlwaysBreakAfterReturnType: None +AlwaysBreakBeforeMultilineStrings: false +AlwaysBreakTemplateDeclarations: Yes +BinPackArguments: false +BinPackParameters: false +BreakBeforeBinaryOperators: None +BreakBeforeBraces: Custom +BraceWrapping: + SplitEmptyRecord: false + AfterEnum: true + AfterStruct: true + AfterClass: true + AfterControlStatement: true + AfterFunction: true + AfterUnion: true + AfterNamespace: true + AfterExternBlock: true + BeforeElse: true + +BreakBeforeTernaryOperators: true +BreakConstructorInitializers: BeforeComma +BreakStringLiterals: true +ColumnLimit: 120 +CommentPragmas: '^ (coverity|NOSONAR|pragma:)' +CompactNamespaces: false +ConstructorInitializerAllOnOneLineOrOnePerLine: false +ConstructorInitializerIndentWidth: 4 +ContinuationIndentWidth: 4 +Cpp11BracedListStyle: true +DerivePointerAlignment: false +DisableFormat: false +FixNamespaceComments: true +ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH ] +IncludeBlocks: Preserve +IndentCaseLabels: false +IndentPPDirectives: AfterHash +IndentWidth: 4 +IndentWrappedFunctionNames: false +KeepEmptyLinesAtTheStartOfBlocks: false +MacroBlockBegin: '' +MacroBlockEnd: '' +MaxEmptyLinesToKeep: 1 +NamespaceIndentation: None +PenaltyBreakAssignment: 2 +PenaltyBreakBeforeFirstCallParameter: 10000 # Raised intentionally; prefer breaking all +PenaltyBreakComment: 300 +PenaltyBreakFirstLessLess: 120 +PenaltyBreakString: 1000 +PenaltyExcessCharacter: 1000000 +PenaltyReturnTypeOnItsOwnLine: 10000 # Raised intentionally because it hurts readability +PointerAlignment: Left +ReflowComments: true +SortIncludes: Never +SortUsingDeclarations: false +SpaceAfterCStyleCast: true +SpaceAfterTemplateKeyword: true +SpaceBeforeAssignmentOperators: true +SpaceBeforeParens: ControlStatements +SpaceInEmptyParentheses: false +SpacesBeforeTrailingComments: 2 +SpacesInAngles: false +SpacesInCStyleCastParentheses: false +SpacesInContainerLiterals: false +SpacesInParentheses: false +SpacesInSquareBrackets: false +Standard: c++14 +TabWidth: 8 +UseTab: Never +... diff --git a/.clang-tidy b/.clang-tidy new file mode 100644 index 0000000..99d9c05 --- /dev/null +++ b/.clang-tidy @@ -0,0 +1,37 @@ +Checks: >- + boost-*, + bugprone-*, + cert-*, + clang-analyzer-*, + cppcoreguidelines-*, + google-*, + hicpp-*, + llvm-*, + misc-*, + modernize-*, + performance-*, + portability-*, + readability-*, + -clang-analyzer-core.uninitialized.Assign, + -cppcoreguidelines-avoid-const-or-ref-data-members, + -cppcoreguidelines-use-default-member-init, + -google-readability-avoid-underscore-in-googletest-name, + -google-readability-todo, + -llvm-header-guard, + -modernize-concat-nested-namespaces, + -modernize-type-traits, + -modernize-use-constraints, + -modernize-use-default-member-init, + -modernize-use-nodiscard, + -readability-avoid-const-params-in-decls, + -readability-identifier-length, + -*-use-trailing-return-type, + -*-named-parameter, +CheckOptions: + - key: readability-function-cognitive-complexity.Threshold + value: '90' + - key: readability-magic-numbers.IgnoredIntegerValues + value: '1;2;3;4;5;8;10;16;20;32;60;64;100;128;256;500;512;1000' +WarningsAsErrors: '*' +HeaderFilterRegex: 'include/libcyphal/.*\.hpp' +FormatStyle: file diff --git a/.cspell/custom-dictionary-workspace.txt b/.cspell/custom-dictionary-workspace.txt new file mode 100644 index 0000000..c8eb7d9 --- /dev/null +++ b/.cspell/custom-dictionary-workspace.txt @@ -0,0 +1,2 @@ +# Custom Dictionary Words +ocvsmd diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 0000000..e69de29 diff --git a/.gitignore b/.gitignore index 259148f..3f4f3be 100644 --- a/.gitignore +++ b/.gitignore @@ -30,3 +30,18 @@ *.exe *.out *.app + +# Build folders +**/build/ +**/build_* + +# JetBrains +.idea/* +cmake-build-*/ + +# Python +.venv + +# Dumb OS crap +.DS_Store +*.bak diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 0000000..71e9b05 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,10 @@ +{ + "cSpell.customDictionaries": { + "custom-dictionary-workspace": { + "name": "custom-dictionary-workspace", + "path": "${workspaceFolder:opencyphal-vehicle-system-management-daemon}/.cspell/custom-dictionary-workspace.txt", + "addWords": true, + "scope": "workspace" + } + } +} \ No newline at end of file diff --git a/CMakeLists.txt b/CMakeLists.txt new file mode 100644 index 0000000..7643d3d --- /dev/null +++ b/CMakeLists.txt @@ -0,0 +1,21 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) + +project(ocvsmd + VERSION ${OCVSMD_VERSION} + LANGUAGES CXX + HOMEPAGE_URL https://github.com/OpenCyphal-Garage/opencyphal-vehicle-system-management-daemon +) + +set(CMAKE_CXX_STANDARD 14) +set(CMAKE_CXX_STANDARD_REQUIRED ON) + +# Set the output binary directory +set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin) + +add_subdirectory(src) +add_subdirectory(test) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt new file mode 100644 index 0000000..7acc51f --- /dev/null +++ b/src/CMakeLists.txt @@ -0,0 +1,10 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) + +add_subdirectory(common) +add_subdirectory(daemon) +add_subdirectory(cli) diff --git a/src/cli/CMakeLists.txt b/src/cli/CMakeLists.txt new file mode 100644 index 0000000..abc8902 --- /dev/null +++ b/src/cli/CMakeLists.txt @@ -0,0 +1,6 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file diff --git a/src/common/CMakeLists.txt b/src/common/CMakeLists.txt new file mode 100644 index 0000000..abc8902 --- /dev/null +++ b/src/common/CMakeLists.txt @@ -0,0 +1,6 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file diff --git a/src/daemon/CMakeLists.txt b/src/daemon/CMakeLists.txt new file mode 100644 index 0000000..c1a837f --- /dev/null +++ b/src/daemon/CMakeLists.txt @@ -0,0 +1,8 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) + +add_executable(ocvsmd main.cpp) diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp new file mode 100644 index 0000000..4e9a5f2 --- /dev/null +++ b/src/daemon/main.cpp @@ -0,0 +1,8 @@ +#include + +int main(const int argc, const char** const argv) +{ + (void) argc; + (void) argv; + return EXIT_SUCCESS; +} diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt new file mode 100644 index 0000000..7acc51f --- /dev/null +++ b/test/CMakeLists.txt @@ -0,0 +1,10 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) + +add_subdirectory(common) +add_subdirectory(daemon) +add_subdirectory(cli) diff --git a/test/cli/CMakeLists.txt b/test/cli/CMakeLists.txt new file mode 100644 index 0000000..abc8902 --- /dev/null +++ b/test/cli/CMakeLists.txt @@ -0,0 +1,6 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file diff --git a/test/common/CMakeLists.txt b/test/common/CMakeLists.txt new file mode 100644 index 0000000..abc8902 --- /dev/null +++ b/test/common/CMakeLists.txt @@ -0,0 +1,6 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file diff --git a/test/daemon/CMakeLists.txt b/test/daemon/CMakeLists.txt new file mode 100644 index 0000000..abc8902 --- /dev/null +++ b/test/daemon/CMakeLists.txt @@ -0,0 +1,6 @@ +# +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file From 39f3357cc0287f7e71c01bc1779619598228c96c Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Wed, 11 Dec 2024 17:53:22 +0200 Subject: [PATCH 11/24] added clang-format --- CMakeLists.txt | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CMakeLists.txt b/CMakeLists.txt index 7643d3d..882e9d6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -17,5 +17,19 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON) # Set the output binary directory set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin) +set(src_dir "${CMAKE_SOURCE_DIR}/src") +set(test_dir "${CMAKE_SOURCE_DIR}/test") +set(include_dir "${CMAKE_SOURCE_DIR}/include") + +# clang-format +find_program(clang_format NAMES clang-format) +if (NOT clang_format) + message(STATUS "Could not locate clang-format") +else () + file(GLOB format_files ${include_dir}/**/*.[ch]pp ${src_dir}/**/*.[ch]pp ${test_dir}/**/*.[ch]pp) + message(STATUS "Using clang-format: ${clang_format}; files: ${format_files}") + add_custom_target(format COMMAND ${clang_format} -i -fallback-style=none -style=file --verbose ${format_files}) +endif () + add_subdirectory(src) add_subdirectory(test) From c4b03efbd83d3c7d81fbd3514d5a33c904d1026a Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Wed, 11 Dec 2024 20:08:11 +0200 Subject: [PATCH 12/24] implemented steps 1...8 --- .gitignore | 33 -------------- init.d/ocvsmd | 57 +++++++++++++++++++++++ src/daemon/main.cpp | 109 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 166 insertions(+), 33 deletions(-) create mode 100755 init.d/ocvsmd diff --git a/.gitignore b/.gitignore index 3f4f3be..a12b348 100644 --- a/.gitignore +++ b/.gitignore @@ -1,36 +1,3 @@ -# Prerequisites -*.d - -# Compiled Object files -*.slo -*.lo -*.o -*.obj - -# Precompiled Headers -*.gch -*.pch - -# Compiled Dynamic libraries -*.so -*.dylib -*.dll - -# Fortran module files -*.mod -*.smod - -# Compiled Static libraries -*.lai -*.la -*.a -*.lib - -# Executables -*.exe -*.out -*.app - # Build folders **/build/ **/build_* diff --git a/init.d/ocvsmd b/init.d/ocvsmd new file mode 100755 index 0000000..4b998ca --- /dev/null +++ b/init.d/ocvsmd @@ -0,0 +1,57 @@ +#!/bin/sh +### BEGIN INIT INFO +# Provides: ocvsmd +# Required-Start: $remote_fs $syslog +# Required-Stop: $remote_fs $syslog +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: SysV script for ocvsmd +# Description: This service manages the OpenCyphal Vehicle System Management Daemon (OCVSMD). +### END INIT INFO + +DAEMON_NAME="ocvsmd" +DAEMON_PATH="/usr/local/bin/ocvsmd" +PIDFILE="/var/run/${DAEMON_NAME}.pid" + +case "$1" in + start) + echo "Starting $DAEMON_NAME..." + if [ -f $PIDFILE ]; then + echo "$DAEMON_NAME is already running." + exit 1 + fi + start-stop-daemon --start --quiet --background --make-pidfile --pidfile $PIDFILE --exec $DAEMON_PATH + echo "$DAEMON_NAME started." + ;; + stop) + echo "Stopping $DAEMON_NAME..." + if [ ! -f $PIDFILE ]; then + echo "$DAEMON_NAME is not running." + exit 1 + fi + start-stop-daemon --stop --quiet --pidfile $PIDFILE + rm -f $PIDFILE + echo "$DAEMON_NAME stopped." + ;; + restart) + $0 stop + $0 start + ;; + status) + if [ -f $PIDFILE ]; then + PID=$(cat $PIDFILE) + if kill -0 "$PID" >/dev/null 2>&1; then + echo "$DAEMON_NAME is running (PID $PID)." + else + echo "$DAEMON_NAME is not running, but pidfile exists." + fi + else + echo "$DAEMON_NAME is not running." + fi + ;; + *) + echo "Usage: $0 {start|stop|restart|status}" + exit 1 + ;; +esac +exit 0 diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index 4e9a5f2..3eb3743 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -1,8 +1,117 @@ +#include +#include #include +#include +#include +#include +#include + +namespace +{ + +void fork_and_exit_parent() +{ + // Fork off the parent process + const pid_t pid = fork(); + if (pid < 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to fork: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + if (pid > 0) + { + ::exit(EXIT_SUCCESS); + } +} + +void step_01_close_all_file_descriptors() +{ + rlimit rlimit_files{}; + if (getrlimit(RLIMIT_NOFILE, &rlimit_files) != 0) + { + const char* const err_txt = strerror(errno); + std::cerr << "Failed to getrlimit(RLIMIT_NOFILE): " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + constexpr int first_fd_to_close = 3; // 0, 1 & 2 for standard input, output, and error. + for (int fd = first_fd_to_close; fd <= rlimit_files.rlim_max; ++fd) + { + (void) ::close(fd); + } +} + +void step_02_reset_all_signal_handlers_to_default() +{ + for (int sig = 1; sig < _NSIG; ++sig) + { + (void) ::signal(sig, SIG_DFL); + } +} + +void step_03_reset_signal_mask() +{ + sigset_t sigset_all{}; + if (::sigfillset(&sigset_all) != 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to sigfillset(): " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + if (::sigprocmask(SIG_SETMASK, &sigset_all, nullptr) != 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to sigprocmask(SIG_SETMASK): " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } +} + +void step_04_sanitize_environment() +{ + // TODO: Implement this step. +} + +void step_05_fork_to_background() +{ + fork_and_exit_parent(); +} + +void step_06_create_new_session() +{ + if (::setsid() < 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to setsid: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } +} + +void step_07_08_fork_and_exit_again() +{ + fork_and_exit_parent(); +} + +/// Implements the daemonization procedure as described in the `man 7 daemon` manual page. +/// +void daemonize() +{ + step_01_close_all_file_descriptors(); + step_02_reset_all_signal_handlers_to_default(); + step_03_reset_signal_mask(); + step_04_sanitize_environment(); + step_05_fork_to_background(); + step_06_create_new_session(); + step_07_08_fork_and_exit_again(); +} + +} // namespace int main(const int argc, const char** const argv) { (void) argc; (void) argv; + + daemonize(); + return EXIT_SUCCESS; } From 963a8bf09b23d9a5631384a14349c68c01eb23dd Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Wed, 11 Dec 2024 21:59:18 +0200 Subject: [PATCH 13/24] implemented steps 9...15 --- README.md | 34 +++++++++- src/daemon/main.cpp | 159 ++++++++++++++++++++++++++++++++++++++------ 2 files changed, 170 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 120fecf..f58b447 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,35 @@ # OpenCyphal Vehicle System Management Daemon -👻 +### Build + +``` +cd ocvsmd +mkdir build && cd build +cmake .. +make ocvsmd +``` + +### Installing + +#### Installing the Daemon Binary: +``` +sudo cp bin/ocvsmd /usr/local/bin/ocvsmd +``` + +#### Installing the Init Script: +``` +sudo cp ../init.d/ocvsmd /etc/init.d/ocvsmd +sudo chmod +x /etc/init.d/ocvsmd +``` + +#### Enabling at Startup (on SysV-based systems): +``` +sudo update-rc.d ocvsmd defaults +``` + +### Usage +``` +sudo /etc/init.d/ocvsmd start +sudo /etc/init.d/ocvsmd status +sudo /etc/init.d/ocvsmd stop +``` diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index 3eb3743..ac913dd 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -1,14 +1,20 @@ +#include #include #include #include #include +#include #include #include +#include +#include #include namespace { +volatile bool s_running = true; + void fork_and_exit_parent() { // Fork off the parent process @@ -25,6 +31,19 @@ void fork_and_exit_parent() } } +void handle_signal(const int sig) +{ + switch (sig) + { + case SIGTERM: + case SIGINT: + s_running = false; + break; + default: + break; + } +} + void step_01_close_all_file_descriptors() { rlimit rlimit_files{}; @@ -41,54 +60,131 @@ void step_01_close_all_file_descriptors() } } -void step_02_reset_all_signal_handlers_to_default() +void step_02_03_setup_signal_handlers() { - for (int sig = 1; sig < _NSIG; ++sig) - { - (void) ::signal(sig, SIG_DFL); - } + // Catch termination signals + (void) ::signal(SIGTERM, handle_signal); + (void) ::signal(SIGINT, handle_signal); +} + +void step_04_sanitize_environment() +{ + // TODO: Implement this step. +} + +void step_05_fork_to_background() +{ + fork_and_exit_parent(); } -void step_03_reset_signal_mask() +void step_06_create_new_session() { - sigset_t sigset_all{}; - if (::sigfillset(&sigset_all) != 0) + if (::setsid() < 0) { const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to sigfillset(): " << err_txt << "\n"; + std::cerr << "Failed to setsid: " << err_txt << "\n"; ::exit(EXIT_FAILURE); } - if (::sigprocmask(SIG_SETMASK, &sigset_all, nullptr) != 0) +} + +void step_07_08_fork_and_exit_again() +{ + fork_and_exit_parent(); +} + +void step_09_redirect_stdio_to_devnull() +{ + const int fd = ::open("/dev/null", O_RDWR); + if (fd == -1) { const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to sigprocmask(SIG_SETMASK): " << err_txt << "\n"; + std::cerr << "Failed to open(/dev/null): " << err_txt << "\n"; ::exit(EXIT_FAILURE); } + + ::dup2(fd, STDIN_FILENO); + ::dup2(fd, STDOUT_FILENO); + ::dup2(fd, STDERR_FILENO); + + if (fd > 2) + { + ::close(fd); + } } -void step_04_sanitize_environment() +void step_10_reset_umask() { - // TODO: Implement this step. + ::umask(0); } -void step_05_fork_to_background() +void step_11_change_curr_dir() { - fork_and_exit_parent(); + if (::chdir("/") != 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to chdir(/): " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } } -void step_06_create_new_session() +void step_12_create_pid_file(const char* const pid_file_name) { - if (::setsid() < 0) + const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); + if (fd == -1) { const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to setsid: " << err_txt << "\n"; + std::cerr << "Failed to create on PID file: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + + if (::lockf(fd, F_TLOCK, 0) == -1) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to lock PID file: " << err_txt << "\n"; + ::close(fd); + ::exit(EXIT_FAILURE); + } + + if (::ftruncate(fd, 0) != 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to ftruncate PID file: " << err_txt << "\n"; + ::close(fd); + ::exit(EXIT_FAILURE); + } + + std::array buf{}; + const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); + if (::write(fd, buf.data(), len) != len) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to write to PID file: " << err_txt << "\n"; + ::close(fd); ::exit(EXIT_FAILURE); } + + // Keep the PID file open until the process exits. } -void step_07_08_fork_and_exit_again() +void step_13_drop_privileges() { - fork_and_exit_parent(); + // n the daemon process, drop privileges, if possible and applicable. + // TODO: Implement this step. +} + +void step_14_notify_init_complete() +{ + // From the daemon process, notify the original process started that initialization is complete. This can be + // implemented via an unnamed pipe or similar communication channel that is created before the first fork() and + // hence available in both the original and the daemon process. + // TODO: Implement this step. +} + +void step_15_exit_org_process() +{ + // Call exit() in the original process. The process that invoked the daemon must be able to rely on that this exit() + // happens after initialization is complete and all external communication channels are established and accessible. + // TODO: Implement this step. } /// Implements the daemonization procedure as described in the `man 7 daemon` manual page. @@ -96,12 +192,20 @@ void step_07_08_fork_and_exit_again() void daemonize() { step_01_close_all_file_descriptors(); - step_02_reset_all_signal_handlers_to_default(); - step_03_reset_signal_mask(); + step_02_03_setup_signal_handlers(); step_04_sanitize_environment(); step_05_fork_to_background(); step_06_create_new_session(); step_07_08_fork_and_exit_again(); + step_09_redirect_stdio_to_devnull(); + step_10_reset_umask(); + step_11_change_curr_dir(); + step_12_create_pid_file("/var/run/ocvsmd.pid"); + step_13_drop_privileges(); + step_14_notify_init_complete(); + step_15_exit_org_process(); + + ::openlog("ocvsmd", LOG_PID, LOG_DAEMON); } } // namespace @@ -113,5 +217,16 @@ int main(const int argc, const char** const argv) daemonize(); + ::syslog(LOG_NOTICE, "ocvsmd daemon started."); + + while (s_running) + { + // TODO: Insert daemon code here. + ::sleep(1); + } + + ::syslog(LOG_NOTICE, "ocvsmd daemon terminated."); + ::closelog(); + return EXIT_SUCCESS; } From ef6a78245577c4790baaec1548eb5447ee1f1040 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 13:24:01 +0200 Subject: [PATCH 14/24] minor lint fixes --- src/daemon/main.cpp | 43 ++++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index ac913dd..68dbde1 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -1,19 +1,33 @@ #include #include #include +#include #include #include #include #include #include #include -#include +#include #include namespace { -volatile bool s_running = true; +volatile int s_running = 1; // NOLINT + +extern "C" void handle_signal(const int sig) +{ + switch (sig) + { + case SIGTERM: + case SIGINT: + s_running = 0; + break; + default: + break; + } +} void fork_and_exit_parent() { @@ -31,19 +45,6 @@ void fork_and_exit_parent() } } -void handle_signal(const int sig) -{ - switch (sig) - { - case SIGTERM: - case SIGINT: - s_running = false; - break; - default: - break; - } -} - void step_01_close_all_file_descriptors() { rlimit rlimit_files{}; @@ -94,7 +95,7 @@ void step_07_08_fork_and_exit_again() void step_09_redirect_stdio_to_devnull() { - const int fd = ::open("/dev/null", O_RDWR); + const int fd = ::open("/dev/null", O_RDWR); // NOLINT if (fd == -1) { const char* const err_txt = ::strerror(errno); @@ -129,7 +130,7 @@ void step_11_change_curr_dir() void step_12_create_pid_file(const char* const pid_file_name) { - const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); + const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); // NOLINT if (fd == -1) { const char* const err_txt = ::strerror(errno); @@ -154,7 +155,7 @@ void step_12_create_pid_file(const char* const pid_file_name) } std::array buf{}; - const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); + const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); // NOLINT if (::write(fd, buf.data(), len) != len) { const char* const err_txt = ::strerror(errno); @@ -217,15 +218,15 @@ int main(const int argc, const char** const argv) daemonize(); - ::syslog(LOG_NOTICE, "ocvsmd daemon started."); + ::syslog(LOG_NOTICE, "ocvsmd daemon started."); // NOLINT - while (s_running) + while (s_running == 1) { // TODO: Insert daemon code here. ::sleep(1); } - ::syslog(LOG_NOTICE, "ocvsmd daemon terminated."); + ::syslog(LOG_NOTICE, "ocvsmd daemon terminated."); // NOLINT ::closelog(); return EXIT_SUCCESS; From aec45e656ea03bbc0b7652caf18097fc48fae9ef Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 13:41:59 +0200 Subject: [PATCH 15/24] try GH actions --- .github/workflows/tests.yml | 94 +++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e69de29..cacf613 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -0,0 +1,94 @@ +name: Test Workflow +on: [push, pull_request] +env: + LLVM_VERSION: 15 +jobs: + debug: + if: github.event_name == 'push' + runs-on: ubuntu-latest + strategy: + matrix: + toolchain: ['clang', 'gcc'] + include: + - toolchain: gcc + c-compiler: gcc + cxx-compiler: g++ + - toolchain: clang + c-compiler: clang + cxx-compiler: clang++ + steps: + - uses: actions/checkout@v4 + - run: | + wget https://apt.llvm.org/llvm.sh + chmod +x llvm.sh + sudo ./llvm.sh $LLVM_VERSION + sudo apt update -y && sudo apt upgrade -y + sudo apt-get -y install gcc-multilib g++-multilib clang-tidy-$LLVM_VERSION + sudo update-alternatives --install /usr/bin/clang-tidy clang-tidy /usr/bin/clang-tidy-$LLVM_VERSION 50 + clang-tidy --version + - run: > + cmake + -B ${{ github.workspace }}/build + -DCMAKE_BUILD_TYPE=Debug + -DCMAKE_C_COMPILER=${{ matrix.c-compiler }} + -DCMAKE_CXX_COMPILER=${{ matrix.cxx-compiler }} + ocvsmd + - working-directory: ${{github.workspace}}/build + run: | + make VERBOSE=1 + make test + - uses: actions/upload-artifact@v3 + if: always() + with: + name: ${{github.job}} + path: ${{github.workspace}}/**/* + retention-days: 2 + + optimizations: + if: github.event_name == 'push' + runs-on: ubuntu-latest + strategy: + matrix: + toolchain: ['clang', 'gcc'] + build_type: [Release, MinSizeRel] + include: + - toolchain: gcc + c-compiler: gcc + cxx-compiler: g++ + - toolchain: clang + c-compiler: clang + cxx-compiler: clang++ + steps: + - uses: actions/checkout@v4 + - run: | + sudo apt update -y && sudo apt upgrade -y + sudo apt install gcc-multilib g++-multilib + - run: > + cmake + -B ${{ github.workspace }}/build + -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} + -DCMAKE_C_COMPILER=${{ matrix.c-compiler }} + -DCMAKE_CXX_COMPILER=${{ matrix.cxx-compiler }} + -DNO_STATIC_ANALYSIS=1 + ocvsmd + - working-directory: ${{github.workspace}}/build + run: | + make VERBOSE=1 + make test + - uses: actions/upload-artifact@v3 + if: always() + with: + name: ${{github.job}} + path: ${{github.workspace}}/**/* + retention-days: 2 + + style_check: + if: github.event_name == 'push' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: DoozyX/clang-format-lint-action@v0.17 + with: + source: './include ./test ./src' + extensions: 'c,h,cpp,hpp' + clangFormatVersion: ${{ env.LLVM_VERSION }} From d5007bcd60e821692a3ef69e2e90c11995efdf5b Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 13:48:59 +0200 Subject: [PATCH 16/24] try GH actions # 2 --- .github/workflows/tests.yml | 6 +++--- src/daemon/main.cpp | 5 +++++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index cacf613..b8bb19c 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -37,7 +37,7 @@ jobs: run: | make VERBOSE=1 make test - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: always() with: name: ${{github.job}} @@ -75,7 +75,7 @@ jobs: run: | make VERBOSE=1 make test - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: always() with: name: ${{github.job}} @@ -89,6 +89,6 @@ jobs: - uses: actions/checkout@v4 - uses: DoozyX/clang-format-lint-action@v0.17 with: - source: './include ./test ./src' + source: './test ./src' extensions: 'c,h,cpp,hpp' clangFormatVersion: ${{ env.LLVM_VERSION }} diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index 68dbde1..5948d28 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -1,3 +1,8 @@ +// +// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +// SPDX-License-Identifier: MIT +// + #include #include #include From b275694ac6b9c25e511bd17d121dc98718bb9f42 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 15:05:49 +0200 Subject: [PATCH 17/24] try GH actions # 3 --- .github/workflows/tests.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index b8bb19c..05ab941 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -40,7 +40,7 @@ jobs: - uses: actions/upload-artifact@v4 if: always() with: - name: ${{github.job}} + name: ${{github.job}}_debug_${{matrix.toolchain}} path: ${{github.workspace}}/**/* retention-days: 2 @@ -78,7 +78,7 @@ jobs: - uses: actions/upload-artifact@v4 if: always() with: - name: ${{github.job}} + name: ${{github.job}}_optimizations_${{matrix.toolchain}}_${{matrix.build_type}} path: ${{github.workspace}}/**/* retention-days: 2 From e57d37f222b0ef682c3c444e9e3a03db6dc5bd15 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 15:15:59 +0200 Subject: [PATCH 18/24] static analysis --- .github/workflows/tests.yml | 8 ++------ CMakeLists.txt | 14 ++++++++++++++ 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 05ab941..8a51d88 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -30,9 +30,7 @@ jobs: cmake -B ${{ github.workspace }}/build -DCMAKE_BUILD_TYPE=Debug - -DCMAKE_C_COMPILER=${{ matrix.c-compiler }} -DCMAKE_CXX_COMPILER=${{ matrix.cxx-compiler }} - ocvsmd - working-directory: ${{github.workspace}}/build run: | make VERBOSE=1 @@ -40,7 +38,7 @@ jobs: - uses: actions/upload-artifact@v4 if: always() with: - name: ${{github.job}}_debug_${{matrix.toolchain}} + name: ${{github.job}}_${{matrix.toolchain}} path: ${{github.workspace}}/**/* retention-days: 2 @@ -67,10 +65,8 @@ jobs: cmake -B ${{ github.workspace }}/build -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} - -DCMAKE_C_COMPILER=${{ matrix.c-compiler }} -DCMAKE_CXX_COMPILER=${{ matrix.cxx-compiler }} -DNO_STATIC_ANALYSIS=1 - ocvsmd - working-directory: ${{github.workspace}}/build run: | make VERBOSE=1 @@ -78,7 +74,7 @@ jobs: - uses: actions/upload-artifact@v4 if: always() with: - name: ${{github.job}}_optimizations_${{matrix.toolchain}}_${{matrix.build_type}} + name: ${{github.job}}_${{matrix.toolchain}}_${{matrix.build_type}} path: ${{github.workspace}}/**/* retention-days: 2 diff --git a/CMakeLists.txt b/CMakeLists.txt index 882e9d6..b6ffc90 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,6 +11,8 @@ project(ocvsmd HOMEPAGE_URL https://github.com/OpenCyphal-Garage/opencyphal-vehicle-system-management-daemon ) +set(NO_STATIC_ANALYSIS OFF CACHE BOOL "disable static analysis") + set(CMAKE_CXX_STANDARD 14) set(CMAKE_CXX_STANDARD_REQUIRED ON) @@ -31,5 +33,17 @@ else () add_custom_target(format COMMAND ${clang_format} -i -fallback-style=none -style=file --verbose ${format_files}) endif () +# Use -DNO_STATIC_ANALYSIS=1 to suppress static analysis. +# If not suppressed, the tools used here shall be available, otherwise the build will fail. +if (NOT NO_STATIC_ANALYSIS) + # clang-tidy (separate config files per directory) + find_program(clang_tidy NAMES clang-tidy) + if (NOT clang_tidy) + message(FATAL_ERROR "Could not locate clang-tidy") + endif () + message(STATUS "Using clang-tidy: ${clang_tidy}") + set(CMAKE_CXX_CLANG_TIDY ${clang_tidy}) +endif() + add_subdirectory(src) add_subdirectory(test) From 12bff65781691477400bc0912eb9f9fdc1dd29e2 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 15:26:56 +0200 Subject: [PATCH 19/24] clang-tidy fixes --- CMakeLists.txt | 2 ++ src/daemon/main.cpp | 17 ++++++++++------- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index b6ffc90..34f0c4b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,6 +11,8 @@ project(ocvsmd HOMEPAGE_URL https://github.com/OpenCyphal-Garage/opencyphal-vehicle-system-management-daemon ) +enable_testing() + set(NO_STATIC_ANALYSIS OFF CACHE BOOL "disable static analysis") set(CMAKE_CXX_STANDARD 14) diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index 5948d28..a17f8d0 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -11,6 +11,7 @@ #include #include #include +#include // NOLINT *-deprecated-headers for `pid_t` type #include #include #include @@ -19,7 +20,8 @@ namespace { -volatile int s_running = 1; // NOLINT +// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) +volatile int s_running = 1; extern "C" void handle_signal(const int sig) { @@ -100,7 +102,7 @@ void step_07_08_fork_and_exit_again() void step_09_redirect_stdio_to_devnull() { - const int fd = ::open("/dev/null", O_RDWR); // NOLINT + const int fd = ::open("/dev/null", O_RDWR); // NOLINT *-vararg if (fd == -1) { const char* const err_txt = ::strerror(errno); @@ -135,7 +137,7 @@ void step_11_change_curr_dir() void step_12_create_pid_file(const char* const pid_file_name) { - const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); // NOLINT + const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); // NOLINT *-vararg if (fd == -1) { const char* const err_txt = ::strerror(errno); @@ -159,8 +161,9 @@ void step_12_create_pid_file(const char* const pid_file_name) ::exit(EXIT_FAILURE); } - std::array buf{}; - const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); // NOLINT + constexpr std::size_t max_pid_str_len = 32; + std::array buf{}; + const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); // NOLINT *-vararg if (::write(fd, buf.data(), len) != len) { const char* const err_txt = ::strerror(errno); @@ -223,7 +226,7 @@ int main(const int argc, const char** const argv) daemonize(); - ::syslog(LOG_NOTICE, "ocvsmd daemon started."); // NOLINT + ::syslog(LOG_NOTICE, "ocvsmd daemon started."); // NOLINT *-vararg while (s_running == 1) { @@ -231,7 +234,7 @@ int main(const int argc, const char** const argv) ::sleep(1); } - ::syslog(LOG_NOTICE, "ocvsmd daemon terminated."); // NOLINT + ::syslog(LOG_NOTICE, "ocvsmd daemon terminated."); // NOLINT *-vararg ::closelog(); return EXIT_SUCCESS; From b464cec58aa7dea80a66d9aab7253e9a1bc5a2a2 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 15:37:40 +0200 Subject: [PATCH 20/24] cmake tidy --- CMakeLists.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 34f0c4b..d40c0b4 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -6,7 +6,6 @@ cmake_minimum_required(VERSION 3.22.0) project(ocvsmd - VERSION ${OCVSMD_VERSION} LANGUAGES CXX HOMEPAGE_URL https://github.com/OpenCyphal-Garage/opencyphal-vehicle-system-management-daemon ) From 6230df8bab9b6832e97fa594fcd1b230d4ecd632 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 15:46:36 +0200 Subject: [PATCH 21/24] minor fixes --- src/cli/CMakeLists.txt | 2 +- src/common/CMakeLists.txt | 2 +- test/cli/CMakeLists.txt | 2 +- test/common/CMakeLists.txt | 2 +- test/daemon/CMakeLists.txt | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/cli/CMakeLists.txt b/src/cli/CMakeLists.txt index abc8902..1a0980c 100644 --- a/src/cli/CMakeLists.txt +++ b/src/cli/CMakeLists.txt @@ -3,4 +3,4 @@ # SPDX-License-Identifier: MIT # -cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file +cmake_minimum_required(VERSION 3.22.0) diff --git a/src/common/CMakeLists.txt b/src/common/CMakeLists.txt index abc8902..1a0980c 100644 --- a/src/common/CMakeLists.txt +++ b/src/common/CMakeLists.txt @@ -3,4 +3,4 @@ # SPDX-License-Identifier: MIT # -cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file +cmake_minimum_required(VERSION 3.22.0) diff --git a/test/cli/CMakeLists.txt b/test/cli/CMakeLists.txt index abc8902..1a0980c 100644 --- a/test/cli/CMakeLists.txt +++ b/test/cli/CMakeLists.txt @@ -3,4 +3,4 @@ # SPDX-License-Identifier: MIT # -cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file +cmake_minimum_required(VERSION 3.22.0) diff --git a/test/common/CMakeLists.txt b/test/common/CMakeLists.txt index abc8902..1a0980c 100644 --- a/test/common/CMakeLists.txt +++ b/test/common/CMakeLists.txt @@ -3,4 +3,4 @@ # SPDX-License-Identifier: MIT # -cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file +cmake_minimum_required(VERSION 3.22.0) diff --git a/test/daemon/CMakeLists.txt b/test/daemon/CMakeLists.txt index abc8902..1a0980c 100644 --- a/test/daemon/CMakeLists.txt +++ b/test/daemon/CMakeLists.txt @@ -3,4 +3,4 @@ # SPDX-License-Identifier: MIT # -cmake_minimum_required(VERSION 3.22.0) \ No newline at end of file +cmake_minimum_required(VERSION 3.22.0) From fc01f260a8cc0e9e95c3adb7f45df01b1b1e9b91 Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 18:18:44 +0200 Subject: [PATCH 22/24] minor fixes --- README.md | 1 + src/daemon/main.cpp | 132 ++++++++++++++++++++++++++++++++------------ 2 files changed, 97 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index f58b447..7888b82 100644 --- a/README.md +++ b/README.md @@ -31,5 +31,6 @@ sudo update-rc.d ocvsmd defaults ``` sudo /etc/init.d/ocvsmd start sudo /etc/init.d/ocvsmd status +sudo /etc/init.d/ocvsmd restart sudo /etc/init.d/ocvsmd stop ``` diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index a17f8d0..8e5e701 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -4,6 +4,7 @@ // #include +#include #include #include #include @@ -36,23 +37,7 @@ extern "C" void handle_signal(const int sig) } } -void fork_and_exit_parent() -{ - // Fork off the parent process - const pid_t pid = fork(); - if (pid < 0) - { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to fork: " << err_txt << "\n"; - ::exit(EXIT_FAILURE); - } - if (pid > 0) - { - ::exit(EXIT_SUCCESS); - } -} - -void step_01_close_all_file_descriptors() +void step_01_close_all_file_descriptors(std::array& pipe_fds) { rlimit rlimit_files{}; if (getrlimit(RLIMIT_NOFILE, &rlimit_files) != 0) @@ -66,6 +51,15 @@ void step_01_close_all_file_descriptors() { (void) ::close(fd); } + + // Create a pipe to communicate with the original process. + // + if (::pipe(pipe_fds.data()) == -1) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to create pipe: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } } void step_02_03_setup_signal_handlers() @@ -80,9 +74,31 @@ void step_04_sanitize_environment() // TODO: Implement this step. } -void step_05_fork_to_background() +bool step_05_fork_to_background(std::array& pipe_fds) { - fork_and_exit_parent(); + // Fork off the parent process + const pid_t parent_pid = fork(); + if (parent_pid < 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to fork: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + + if (parent_pid == 0) + { + // Close read end on the child side. + ::close(pipe_fds[0]); + pipe_fds[0] = -1; + } + else + { + // Close write end on the parent side. + ::close(pipe_fds[1]); + pipe_fds[1] = -1; + } + + return parent_pid == 0; } void step_06_create_new_session() @@ -95,9 +111,24 @@ void step_06_create_new_session() } } -void step_07_08_fork_and_exit_again() +void step_07_08_fork_and_exit_again(int& pipe_write_fd) { - fork_and_exit_parent(); + assert(pipe_write_fd != -1); + + // Fork off the parent process + const pid_t pid = fork(); + if (pid < 0) + { + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to fork: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); + } + if (pid > 0) + { + ::close(pipe_write_fd); + pipe_write_fd = -1; + ::exit(EXIT_SUCCESS); + } } void step_09_redirect_stdio_to_devnull() @@ -181,38 +212,67 @@ void step_13_drop_privileges() // TODO: Implement this step. } -void step_14_notify_init_complete() +void step_14_notify_init_complete(int& pipe_write_fd) { + assert(pipe_write_fd != -1); + // From the daemon process, notify the original process started that initialization is complete. This can be // implemented via an unnamed pipe or similar communication channel that is created before the first fork() and // hence available in both the original and the daemon process. - // TODO: Implement this step. + + // Closing the writing end of the pipe will signal the original process that the daemon is ready. + ::close(pipe_write_fd); + pipe_write_fd = -1; } -void step_15_exit_org_process() +void step_15_exit_org_process(int& pipe_read_fd) { // Call exit() in the original process. The process that invoked the daemon must be able to rely on that this exit() // happens after initialization is complete and all external communication channels are established and accessible. - // TODO: Implement this step. + + constexpr std::size_t buf_size = 16; + std::array buf{}; + if (::read(pipe_read_fd, buf.data(), buf.size()) > 0) + { + std::cout << "Child has finished initialization.\n"; + } + ::close(pipe_read_fd); + pipe_read_fd = -1; + ::exit(EXIT_SUCCESS); } /// Implements the daemonization procedure as described in the `man 7 daemon` manual page. /// void daemonize() { - step_01_close_all_file_descriptors(); + std::array pipe_fds{-1, -1}; + + step_01_close_all_file_descriptors(pipe_fds); step_02_03_setup_signal_handlers(); step_04_sanitize_environment(); - step_05_fork_to_background(); - step_06_create_new_session(); - step_07_08_fork_and_exit_again(); - step_09_redirect_stdio_to_devnull(); - step_10_reset_umask(); - step_11_change_curr_dir(); - step_12_create_pid_file("/var/run/ocvsmd.pid"); - step_13_drop_privileges(); - step_14_notify_init_complete(); - step_15_exit_org_process(); + if (step_05_fork_to_background(pipe_fds)) + { + // Child process. + assert(pipe_fds[0] == -1); + assert(pipe_fds[1] != -1); + + step_06_create_new_session(); + step_07_08_fork_and_exit_again(pipe_fds[1]); + step_09_redirect_stdio_to_devnull(); + step_10_reset_umask(); + step_11_change_curr_dir(); + step_12_create_pid_file("/var/run/ocvsmd.pid"); + step_13_drop_privileges(); + step_14_notify_init_complete(pipe_fds[1]); + } + else + { + // Original parent process. + assert(pipe_fds[0] != -1); + assert(pipe_fds[1] == -1); + + step_15_exit_org_process(pipe_fds[0]); + } ::openlog("ocvsmd", LOG_PID, LOG_DAEMON); } From 8a765a00fbbae406e9d63b45624f1105f65d43ef Mon Sep 17 00:00:00 2001 From: Sergei Shirokov Date: Thu, 12 Dec 2024 19:38:43 +0200 Subject: [PATCH 23/24] implemented passing children error to parent --- init.d/ocvsmd | 10 ++++- src/daemon/main.cpp | 90 +++++++++++++++++++++++---------------------- 2 files changed, 55 insertions(+), 45 deletions(-) diff --git a/init.d/ocvsmd b/init.d/ocvsmd index 4b998ca..ade5d09 100755 --- a/init.d/ocvsmd +++ b/init.d/ocvsmd @@ -20,8 +20,14 @@ case "$1" in echo "$DAEMON_NAME is already running." exit 1 fi - start-stop-daemon --start --quiet --background --make-pidfile --pidfile $PIDFILE --exec $DAEMON_PATH - echo "$DAEMON_NAME started." + start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON_PATH + ret=$? + if [ $ret -eq 0 ]; then + echo "$DAEMON_NAME started." + else + echo "$DAEMON_NAME failed with exit code $ret" + exit 1 + fi ;; stop) echo "Stopping $DAEMON_NAME..." diff --git a/src/daemon/main.cpp b/src/daemon/main.cpp index 8e5e701..71f7416 100644 --- a/src/daemon/main.cpp +++ b/src/daemon/main.cpp @@ -21,6 +21,8 @@ namespace { +const auto* const s_init_complete = "init_complete"; + // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) volatile int s_running = 1; @@ -37,6 +39,14 @@ extern "C" void handle_signal(const int sig) } } +void exit_with_failure(const int fd, const char* const msg) +{ + const char* const err_txt = strerror(errno); + ::write(fd, msg, strlen(msg)); + ::write(fd, err_txt, strlen(err_txt)); + ::exit(EXIT_FAILURE); +} + void step_01_close_all_file_descriptors(std::array& pipe_fds) { rlimit rlimit_files{}; @@ -101,13 +111,11 @@ bool step_05_fork_to_background(std::array& pipe_fds) return parent_pid == 0; } -void step_06_create_new_session() +void step_06_create_new_session(const int pipe_write_fd) { if (::setsid() < 0) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to setsid: " << err_txt << "\n"; - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to setsid: "); } } @@ -119,9 +127,7 @@ void step_07_08_fork_and_exit_again(int& pipe_write_fd) const pid_t pid = fork(); if (pid < 0) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to fork: " << err_txt << "\n"; - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to fork: "); } if (pid > 0) { @@ -131,14 +137,12 @@ void step_07_08_fork_and_exit_again(int& pipe_write_fd) } } -void step_09_redirect_stdio_to_devnull() +void step_09_redirect_stdio_to_devnull(const int pipe_write_fd) { const int fd = ::open("/dev/null", O_RDWR); // NOLINT *-vararg if (fd == -1) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to open(/dev/null): " << err_txt << "\n"; - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to open(/dev/null): "); } ::dup2(fd, STDIN_FILENO); @@ -156,40 +160,30 @@ void step_10_reset_umask() ::umask(0); } -void step_11_change_curr_dir() +void step_11_change_curr_dir(const int pipe_write_fd) { if (::chdir("/") != 0) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to chdir(/): " << err_txt << "\n"; - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to chdir(/): "); } } -void step_12_create_pid_file(const char* const pid_file_name) +void step_12_create_pid_file(const int pipe_write_fd, const char* const pid_file_name) { const int fd = ::open(pid_file_name, O_RDWR | O_CREAT, 0644); // NOLINT *-vararg if (fd == -1) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to create on PID file: " << err_txt << "\n"; - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to create on PID file: "); } if (::lockf(fd, F_TLOCK, 0) == -1) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to lock PID file: " << err_txt << "\n"; - ::close(fd); - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to lock PID file: "); } if (::ftruncate(fd, 0) != 0) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to ftruncate PID file: " << err_txt << "\n"; - ::close(fd); - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to ftruncate PID file: "); } constexpr std::size_t max_pid_str_len = 32; @@ -197,10 +191,7 @@ void step_12_create_pid_file(const char* const pid_file_name) const auto len = ::snprintf(buf.data(), buf.size(), "%ld\n", static_cast(::getpid())); // NOLINT *-vararg if (::write(fd, buf.data(), len) != len) { - const char* const err_txt = ::strerror(errno); - std::cerr << "Failed to write to PID file: " << err_txt << "\n"; - ::close(fd); - ::exit(EXIT_FAILURE); + exit_with_failure(pipe_write_fd, "Failed to write to PID file: "); } // Keep the PID file open until the process exits. @@ -217,10 +208,11 @@ void step_14_notify_init_complete(int& pipe_write_fd) assert(pipe_write_fd != -1); // From the daemon process, notify the original process started that initialization is complete. This can be - // implemented via an unnamed pipe or similar communication channel that is created before the first fork() and + // implemented via an unnamed pipe or similar communication channel created before the first fork() and // hence available in both the original and the daemon process. // Closing the writing end of the pipe will signal the original process that the daemon is ready. + (void) ::write(pipe_write_fd, s_init_complete, strlen(s_init_complete)); ::close(pipe_write_fd); pipe_write_fd = -1; } @@ -230,12 +222,22 @@ void step_15_exit_org_process(int& pipe_read_fd) // Call exit() in the original process. The process that invoked the daemon must be able to rely on that this exit() // happens after initialization is complete and all external communication channels are established and accessible. - constexpr std::size_t buf_size = 16; - std::array buf{}; - if (::read(pipe_read_fd, buf.data(), buf.size()) > 0) + constexpr std::size_t buf_size = 256; + std::array msg_from_child{}; + const auto res = ::read(pipe_read_fd, msg_from_child.data(), msg_from_child.size() - 1); + if (res == -1) { - std::cout << "Child has finished initialization.\n"; + const char* const err_txt = ::strerror(errno); + std::cerr << "Failed to read pipe: " << err_txt << "\n"; + ::exit(EXIT_FAILURE); } + + if (::strcmp(msg_from_child.data(), s_init_complete) != 0) + { + std::cerr << "Child init failed: " << msg_from_child.data() << "\n"; + ::exit(EXIT_FAILURE); + } + ::close(pipe_read_fd); pipe_read_fd = -1; ::exit(EXIT_SUCCESS); @@ -255,23 +257,25 @@ void daemonize() // Child process. assert(pipe_fds[0] == -1); assert(pipe_fds[1] != -1); + auto& pipe_write_fd = pipe_fds[1]; - step_06_create_new_session(); - step_07_08_fork_and_exit_again(pipe_fds[1]); - step_09_redirect_stdio_to_devnull(); + step_06_create_new_session(pipe_write_fd); + step_07_08_fork_and_exit_again(pipe_write_fd); + step_09_redirect_stdio_to_devnull(pipe_write_fd); step_10_reset_umask(); - step_11_change_curr_dir(); - step_12_create_pid_file("/var/run/ocvsmd.pid"); + step_11_change_curr_dir(pipe_write_fd); + step_12_create_pid_file(pipe_write_fd, "/var/run/ocvsmd.pid"); step_13_drop_privileges(); - step_14_notify_init_complete(pipe_fds[1]); + step_14_notify_init_complete(pipe_write_fd); } else { // Original parent process. assert(pipe_fds[0] != -1); assert(pipe_fds[1] == -1); + auto& pipe_read_fd = pipe_fds[0]; - step_15_exit_org_process(pipe_fds[0]); + step_15_exit_org_process(pipe_read_fd); } ::openlog("ocvsmd", LOG_PID, LOG_DAEMON); From 96b34fb0bc2a517fae40165020ba58e42146bd0c Mon Sep 17 00:00:00 2001 From: Pavel Kirienko Date: Mon, 16 Dec 2024 17:30:38 +0200 Subject: [PATCH 24/24] Update DESIGN.md: fw progress --- docs/DESIGN.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 962f076..fc0eb38 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -105,9 +105,7 @@ To enable monitoring the progress of a firmware update process, the following so - Report the progress via `uavcan.node.Heartbeat.1.vendor_specific_status_code`. This is rejected because the VSSC is vendor-specific, so it shouldn't be relied on by the standard. -The plan that is tentatively agreed upon is to define a new standard message with a fixed port-ID for needs of progress reporting. The message will likely be placed in the diagnostics namespace as `uavcan.diagnostic.ProgressReport` with a fixed port-ID of 8183. The `uavcan.node.ExecuteCommand.1` RPC may return a flag indicating if the progress of the freshly launched process will be reported via the new message. - -If this message-based approach is chosen, the daemon will subscribe to the message and provide the latest received progress value per node via the monitor interface. +The plan that is tentatively agreed upon is to use an existing primitive message for progress reporting. The `uavcan.node.ExecuteCommand.1.4` RPC will optionally accept the subject-ID to publish progress messages at. ## Milestone 0