Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: major rewrite #32

Merged
merged 1 commit into from
Jun 4, 2024
Merged

refactor!: major rewrite #32

merged 1 commit into from
Jun 4, 2024

Conversation

J-Loudet
Copy link
Contributor

@J-Loudet J-Loudet commented Jun 4, 2024

This commit introduces a major (short of complete) rewrite of the Python bindings for Zenoh-Flow.

The main motivation is to simplify the code base. This translates into less Python code and more Rust code. Which means that the Python bindings are more tightly coupled with Zenoh-Flow's code base which leaves less room for errors and misalignment.

This rewrite is however not complete as mentioned in #30.

This rewrite also introduces few improvements:

  • the bindings can now work in a Python virtual environment on macOS,
  • the log emitted in a Python node are now propagated to the Zenoh-Flow runtime and, thus, displayed.
  • 01-python.zfext: deleted and replaced with an example Zenoh-Flow configuration in the examples folder and an FAQ entry in the README.
  • Cargo.toml:
    • forced resolver to version 2,
    • removed no longer used zenoh-flow-python-commons crate,
    • removed [profile.dev] and [profile.release] sections,
    • added a [workspace.package] section,
    • removed the dependencies:
      • libloading
      • log - zenoh-flow
    • added or updated the dependencies: - anyhow - pyo3 - pyo3-asyncio - pyo3-pylogger - tracing - tracing-subscriber - zenoh-flow-nodes
  • NOTICE.md: removed as no longer appropriate.
  • README.md:
    • updated the content to reflect the new changes,
    • added a section "FAG and Troubleshooting".
  • examples/flow.yaml: example data flow that leverages a Zenoh built-in Source and a Python sink.
  • examples/sink.py: Sink node that prints the payload and the timestamp of the messages received by the Source.
  • examples/zenoh-flow-configuration.yaml: an example configuration for a Zenoh-Flow standalone daemon.
  • zenoh-flow-python-commons/Cargo.toml: deleted as no longer needed.
  • zenoh-flow-python-commons/src/lib.rs: deleted as no longer needed.
  • zenoh-flow-python-extension: deleted as no longer needed.
  • zenoh-flow-python-extension-plugin: deleted as no longer needed.
  • zenoh-flow-python-operator-wrapper/Cargo.toml:
    • added [package] section,
    • updated the [dependencies] section to point to the workspace,
    • removed the dependencies:
      • libloading
      • log - zenoh-flow-python-commons
    • added the dependencies: - tracing - tracing-subscriber
    • removed no longer needed build configuration,
    • removed [package.metadata.deb] section.
  • zenoh-flow-python-operator-wrapper/build.rs: deleted as no longer needed.
  • zenoh-flow-python-operator-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to load the libpython shared library.
  • zenoh-flow-python-sink-wrapper/Cargo.toml:
    • added [package] section,
    • updated the [dependencies] section to point to the workspace,
    • removed the dependencies:
      • libloading
      • log - zenoh-flow-python-commons
    • added the dependencies: - tracing - tracing-subscriber
    • removed no longer needed build configuration,
    • removed [package.metadata.deb] section.
  • zenoh-flow-python-sink-wrapper/build.rs: deleted as no longer needed.
  • zenoh-flow-python-sink-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the libpython shared library.
  • zenoh-flow-python-source-wrapper/Cargo.toml:
    • added [package] section,
    • updated the [dependencies] section to point to the workspace,
    • removed the dependencies:
      • libloading
      • log - zenoh-flow-python-commons
    • added the dependencies: - tracing - tracing-subscriber
    • removed no longer needed build configuration,
    • removed [package.metadata.deb] section.
  • zenoh-flow-python-source-wrapper/build.rs: deleted as no longer needed.
  • zenoh-flow-python-source-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the libpython shared library.
  • zenoh-flow-python/Cargo.toml:
    • added [package] section,
    • updated the [dependencies] section to point to the workspace,
    • added the dependencies:
      • anyhow
      • pyo3-asyncio - serde_json - tracing - zenoh-flow-nodes
    • removed the dependency to zenoh-flow-python-commons as it no longer (need to) exists.
    • removed the features:
      • abi-py37: it is enabled at the workspace level,
      • default: we do not want to activate the extension-module feature by default.
  • zenoh-flow-python/README.md: deleted as it was redundant with the README at the root of the project.
  • zenoh-flow-python/examples/loader-config.yml: deleted as no longer relevant.
  • zenoh-flow-python/examples/operator.py: deleted (will be replaced).
  • zenoh-flow-python/examples/py-operator.yml: deleted, no longer necessary.
  • zenoh-flow-python/examples/py-pipeline.yml: deleted, replaced.
  • zenoh-flow-python/examples/py-sink.yml: deleted, no longer necessary.
  • zenoh-flow-python/examples/py-source.yml: deleted, no longer necessary.
  • zenoh-flow-python/examples/sink.py: deleted, replaced.
  • zenoh-flow-python/examples/source.py: deleted (will be replaced).
  • zenoh-flow-python/pyproject.toml:
    • updated the version of maturin used,
    • updated the [project] section,
    • added a [tool.maturin] section to only enable the "pyo3/extension-module" feature when building the Python package of Zenoh-Flow,
    • updated the urls.
  • zenoh-flow-python/requirements-dev.txt:
    • updated the version of maturin to reflect what is written in pyproject.toml,
    • removed wheel.
  • zenoh-flow-python/src/lib.rs: almost complete rewrite, leveraging the "newtype pattern" to write the bindings.
  • zenoh-flow-python/zenoh_flow/init.py: updated to reflect the new changes.
  • zenoh-flow-python/zenoh_flow/interfaces/operator.py: deleted, replaced by nodes.py.
  • zenoh-flow-python/zenoh_flow/interfaces/sink.py: deleted, replaced by nodes.py
  • zenoh-flow-python/zenoh_flow/interfaces/source.py: deleted, replaced by nodes.py.
  • zenoh-flow-python/zenoh_flow/types/init.py: deleted as no longer needed.
  • zenoh-flow-python/zenoh_flow/interfaces/init.py -> zenoh-flow-python/zenoh_flow_python/init.py
  • zenoh-flow-python/zenoh_flow_python/nodes.py: definition of Python "Protocol" that users must implement to create Zenoh-Flow nodes.
  • zenoh-flow-python/zenoh_flow_python/py.typed: empty file to indicate that we are using a stub file to define our types.
  • zenoh-flow-python/zenoh_flow_python/zenoh_flow_python.pyi: stub file exposing the types of functions, classes and their methods.

This commit introduces a major (short of complete) rewrite of the Python
bindings for Zenoh-Flow.

The main motivation is to simplify the code base. This translates into
less Python code and more Rust code. Which means that the Python
bindings are more tightly coupled with Zenoh-Flow's code base which
leaves less room for errors and misalignment.

This rewrite is however not complete as mentioned in #30.

This rewrite also introduces few improvements:
- the bindings can now work in a Python virtual environment on macOS,
- the log emitted in a Python node are now propagated to the Zenoh-Flow
  runtime and, thus, displayed.

* 01-python.zfext: deleted and replaced with an example Zenoh-Flow
  configuration in the `examples` folder and an FAQ entry in the README.
* Cargo.toml:
  - forced resolver to version 2,
  - removed no longer used `zenoh-flow-python-commons` crate,
  - removed `[profile.dev]` and `[profile.release]` sections,
  - added a `[workspace.package]` section,
  - removed the dependencies:
    - libloading
    - log
    - zenoh-flow
  - added or updated the dependencies:
    - anyhow
    - pyo3
    - pyo3-asyncio
    - pyo3-pylogger
    - tracing
    - tracing-subscriber
    - zenoh-flow-nodes
* NOTICE.md: removed as no longer appropriate.
* README.md:
  - updated the content to reflect the new changes,
  - added a section "FAG and Troubleshooting".
* examples/flow.yaml: example data flow that leverages a Zenoh built-in
  Source and a Python sink.
* examples/sink.py: Sink node that prints the payload and the timestamp
  of the messages received by the Source.
* examples/zenoh-flow-configuration.yaml: an example configuration for
  a Zenoh-Flow standalone daemon.
* zenoh-flow-python-commons/Cargo.toml: deleted as no longer needed.
* zenoh-flow-python-commons/src/lib.rs: deleted as no longer needed.
* zenoh-flow-python-extension: deleted as no longer needed.
* zenoh-flow-python-extension-plugin: deleted as no longer needed.
* zenoh-flow-python-operator-wrapper/Cargo.toml:
  - added `[package]` section,
  - updated the `[dependencies]` section to point to the workspace,
  - removed the dependencies:
    - libloading
    - log
    - zenoh-flow-python-commons
  - added the dependencies:
    - tracing
    - tracing-subscriber
  - removed no longer needed build configuration,
  - removed `[package.metadata.deb]` section.
* zenoh-flow-python-operator-wrapper/build.rs: deleted as no longer
  needed.
* zenoh-flow-python-operator-wrapper/src/lib.rs: updated the code to the
  latest changes. In particular it is no longer needed to load the
  `libpython` shared library.
* zenoh-flow-python-sink-wrapper/Cargo.toml:
  - added `[package]` section,
  - updated the `[dependencies]` section to point to the workspace,
  - removed the dependencies:
    - libloading
    - log
    - zenoh-flow-python-commons
  - added the dependencies:
    - tracing
    - tracing-subscriber
  - removed no longer needed build configuration,
  - removed `[package.metadata.deb]` section.
* zenoh-flow-python-sink-wrapper/build.rs: deleted as no longer needed.
* zenoh-flow-python-sink-wrapper/src/lib.rs: updated the code to the
  latest changes. In particular it is no longer needed to manually load
  the `libpython` shared library.
* zenoh-flow-python-source-wrapper/Cargo.toml:
  - added `[package]` section,
  - updated the `[dependencies]` section to point to the workspace,
  - removed the dependencies:
    - libloading
    - log
    - zenoh-flow-python-commons
  - added the dependencies:
    - tracing
    - tracing-subscriber
  - removed no longer needed build configuration,
  - removed `[package.metadata.deb]` section.
* zenoh-flow-python-source-wrapper/build.rs: deleted as no longer
  needed.
* zenoh-flow-python-source-wrapper/src/lib.rs: updated the code to the
  latest changes. In particular it is no longer needed to manually load
  the `libpython` shared library.
* zenoh-flow-python/Cargo.toml:
  - added `[package]` section,
  - updated the `[dependencies]` section to point to the workspace,
  - added the dependencies:
    - anyhow
    - pyo3-asyncio
    - serde_json
    - tracing
    - zenoh-flow-nodes
  - removed the dependency to zenoh-flow-python-commons as it no longer
    (need to) exists.
  - removed the features:
    - abi-py37: it is enabled at the workspace level,
    - default: we do not want to activate the extension-module feature
      by default.
* zenoh-flow-python/README.md: deleted as it was redundant with the
  README at the root of the project.
* zenoh-flow-python/examples/loader-config.yml: deleted as no longer
  relevant.
* zenoh-flow-python/examples/operator.py: deleted (will be replaced).
* zenoh-flow-python/examples/py-operator.yml: deleted, no longer
  necessary.
* zenoh-flow-python/examples/py-pipeline.yml: deleted, replaced.
* zenoh-flow-python/examples/py-sink.yml: deleted, no longer
  necessary.
* zenoh-flow-python/examples/py-source.yml: deleted, no longer
  necessary.
* zenoh-flow-python/examples/sink.py: deleted, replaced.
* zenoh-flow-python/examples/source.py: deleted (will be replaced).
* zenoh-flow-python/pyproject.toml:
  - updated the version of maturin used,
  - updated the `[project]` section,
  - added a `[tool.maturin]` section to only enable the
    "pyo3/extension-module" feature when building the Python package of
    Zenoh-Flow,
  - updated the urls.
* zenoh-flow-python/requirements-dev.txt:
  - updated the version of maturin to reflect what is written in
    pyproject.toml,
  - removed wheel.
* zenoh-flow-python/src/lib.rs: almost complete rewrite, leveraging
  the "newtype pattern" to write the bindings.
* zenoh-flow-python/zenoh_flow/__init__.py: updated to reflect the new
  changes.
* zenoh-flow-python/zenoh_flow/interfaces/operator.py: deleted, replaced
  by `nodes.py`.
* zenoh-flow-python/zenoh_flow/interfaces/sink.py: deleted, replaced by
  `nodes.py`
* zenoh-flow-python/zenoh_flow/interfaces/source.py: deleted, replaced
  by `nodes.py`.
* zenoh-flow-python/zenoh_flow/types/__init__.py: deleted as no longer
  needed.
* zenoh-flow-python/zenoh_flow/interfaces/__init__.py -> zenoh-flow-python/zenoh_flow_python/__init__.py
* zenoh-flow-python/zenoh_flow_python/nodes.py: definition of Python
  "Protocol" that users must implement to create Zenoh-Flow nodes.
* zenoh-flow-python/zenoh_flow_python/py.typed: empty file to indicate
  that we are using a stub file to define our types.
* zenoh-flow-python/zenoh_flow_python/zenoh_flow_python.pyi: stub file
  exposing the types of our class, their methods and functions.

Signed-off-by: Julien Loudet <[email protected]>
@J-Loudet J-Loudet merged commit cfb4ad7 into main Jun 4, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant