Expose more SessionOptions from libbmq
#7
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Additionally: this PR contains the commits from #5, which are necessary to test these changes locally. That PR should be merged first. |
pniedzielski
force-pushed
the
expose-session-options
branch
2 times, most recently
from
eb31799 to
156efc0
Compare
hallfox
reviewed
Nov 30, 2023
pniedzielski
force-pushed
the
expose-session-options
branch
from
77e6024 to
da487a0
Compare
Since the creation of the BlazingMQ Python SDK, our underlying C++ SDK libbmq has gained more `SessionOptions` that a user can configure, including additional and more fine-grained timeout settings, new watermark configurations, and buffer sizes. In order to expose these new `SessionOptions` to users of the Python SDK, we need to incnlude them in the C++ library which our Cython layer calls, pybmq. This patch exposes more of libbmq’s `SessionOptions` via the constructor of pybmq’s `Session`. After applying this patch, the pybmq should be able to accept all `SessionOptions` from libbmq except those that are deprecated (e.g., `eventQueueSize`) and those that are related to distributed trace, which would require more substantial integration work. This work can be undertaken in the future without impact on the changes here. Signed-off-by: Patrick M. Niedzielski <[email protected]>
The BlazingMQ Python SDK is divided into three layers, with the lowest level being a pure C++ library pybmq that interfaces with the BlazingMQ C++ client library libbmq, with the highest level being exposed through the `_session.py` module’s `Session` Python class, and with a thin middle layer of Cython glue code in `_ext.pyx` connecting the two. This patch exposes the additional `SessionOptions` through the Cython glue layer that the previous patch exposed through pybmq’s `Session` class. Signed-off-by: Patrick M. Niedzielski <[email protected]>
Currently, the Python SDK does not allow users to specify different timeouts for each session operation, instead accepting a single floating-point value that sets open, configure, and close queue timeouts. This is in contrast to the C++ `libbmq` SDK, which allows users to configure timeouts for each operation separately using the `bmqt_sessionoptions` component. In order to preserve the behavior of existing clients, while enabling feature parity between the Python SDK and the C++ SDK with regard to selecting timeouts, this patch introduces a new value-semantic class, `Timeouts`, which allows users to independently set the timeouts for each of the five session operations whose timeouts are configureable: session connection, session disconnection, queue opening, queue configuration, and queue closing. It is intended that clients may either continue to provide a `float` value with the same semantics when creating a `Session` object, or may instead provide an instance of `Timeouts` for the same parameter value to configure each timeout separately. Signed-off-by: Patrick M. Niedzielski <[email protected]>
This patch completes the trio of patches that expose libbmq’s new `SessionOptions` to the users of the Python SDK through the latter’s three layers, by allowing users to build Python `Session`s that specify all the new session options. The changes in this patch should nonetheless maintain backwards compatibility with all existing clients of the Python SDK. All new session options that the `Session` constructor accepts are defaulted to `None` and as such are not set by pybmq, maintaining the existing behavior for callers who do not set any of these new arguments. New session options are also appended to the list of arguments, so clients who build a `Session` using positional arguments rather than named arguments will continue to function correctly. Finally, by overloading the `timeout` argument to taking either a simple float or an instance of the new value-semantic class `Timeouts`, users who currently set a timeout for the `Session` by passing in a single timeout will continue to see the same behavior, but users passing in a new `Timeouts` value have access to the more fine-grained timeout settings that libbmq’s `SessionOptions` provides. Signed-off-by: Patrick M. Niedzielski <[email protected]>
While users can continue to construct the `Session` directly using its constructor, adding the additional session options arguments to the constructor has made it somewhat unwieldly to call, especially if the user only wants to configure one of the session options. This patch provides an alternative, simpler way to construct a `Session` instance, using a new value-semantic builder class named `SessionOptions` that holds each of the new session options and a class method named `with_options`, which takes an instance of `SessionOptions` and returns a new `Session`. `with_options` is the recommended way to construct a new session if you plan to configure the session options. Signed-off-by: Patrick M. Niedzielski <[email protected]>
The `MockSession` class in the Python SDK is used by our test suite to ensure that session options are properly propagated through our Cython layer so they can be passed to libbmq. However, because have introduced new session options into the Python SDK, the `MockSession` class needs to be updated so it knows about these additional options. This patch extends `MockSession` with the new session options. Signed-off-by: Patrick M. Niedzielski <[email protected]>
Signed-off-by: Patrick M. Niedzielski <[email protected]>
Signed-off-by: Patrick M. Niedzielski <[email protected]>
This allows us to simplify the branching in `Session`’s constructor, and more explicitly shows the existing behavior of passing a `float` rather than a `Timeouts`. Signed-off-by: Patrick M. Niedzielski <[email protected]>
Signed-off-by: Patrick M. Niedzielski <[email protected]>
pniedzielski
force-pushed
the
expose-session-options
branch
from
da487a0 to
fe22261
Compare
pniedzielski
force-pushed
the
expose-session-options
branch
from
fe22261 to
15305c3
Compare
Signed-off-by: Patrick M. Niedzielski <[email protected]>
pniedzielski
force-pushed
the
expose-session-options
branch
from
15305c3 to
7999d4d
Compare
678098
reviewed
Dec 1, 2023
Signed-off-by: Patrick M. Niedzielski <[email protected]>
pniedzielski
force-pushed
the
expose-session-options
branch
from
73a7c08 to
da077b7
Compare
Signed-off-by: Patrick M. Niedzielski <[email protected]>
pniedzielski
force-pushed
the
expose-session-options
branch
from
da077b7 to
20d8f2b
Compare
hallfox
approved these changes
Dec 6, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR exposes all
SessionOptionsthat have been added to libbmq since the creation of this Python SDK, except those that are deprecated or that configure distributed trace (which will require additional work on the Python end to enable). Please see commit messages for details.The PR is broken into a series of patches for ease-of-review, but they should be squashed before merge, as each patch on its own will not pass the test suite.
Fixes #8.