Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move kiwipy/rmq related modules into a specific module #297

Draft
wants to merge 13 commits into
base: dev
Choose a base branch
from

Conversation

unkcpz
Copy link
Member

@unkcpz unkcpz commented Dec 12, 2024

The refactoring is targeting to decouple the dependencies of using kiwipy+rmq as the communicator for the process control.
One of the major change here is hand write the resolver of future by mimic how tho asyncio does for wrapping concurrent.futures.Future into asyncio.Future. I use the same way to convert asyncio.Future into concurent.futures.Future (which is the kiwipy.Future as alias).

  • move the aio_pika import lazily by moving the rmq exceptions to rmq module, this can increase the performance of import aiida; aiida.orm.
  • CancellableAction using composite to behave as a Future like object.
  • use asyncio.Future in favor of alias plumpy.Future and
  • use concurrent.futures.Future instead of alias kiwipy.Future.
  • Hand write _chain and _copy_future since we can not just rely on the API of asyncio that is not exposed.
  • Forming the coordinator/Communicator protocol.
  • Just forming the coordinator/Coordinator protocol and wrap rmq/communicator as a coordinator that not require changs in kiwipy.
  • Mock the coordinator for unit test.

@unkcpz unkcpz marked this pull request as draft December 12, 2024 10:58
@unkcpz unkcpz mentioned this pull request Dec 12, 2024
Copy link

codecov bot commented Dec 14, 2024

Codecov Report

Attention: Patch coverage is 80.25210% with 47 lines in your changes missing coverage. Please review.

Please upload report for BASE (dev@4611154). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/plumpy/rmq/futures.py 52.86% 33 Missing ⚠️
src/plumpy/message.py 93.27% 7 Missing ⚠️
src/plumpy/futures.py 68.75% 5 Missing ⚠️
src/plumpy/processes.py 96.16% 1 Missing ⚠️
src/plumpy/rmq/process_comms.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##             dev     #297   +/-   ##
======================================
  Coverage       ?   90.20%           
======================================
  Files          ?       26           
  Lines          ?     3051           
  Branches       ?        0           
======================================
  Hits           ?     2752           
  Misses         ?      299           
  Partials       ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@unkcpz unkcpz force-pushed the rmq-out branch 4 times, most recently from 156d81a to fc52fcd Compare December 14, 2024 23:22
@unkcpz unkcpz changed the base branch from dev to master December 15, 2024 00:22
@unkcpz unkcpz changed the base branch from master to dev December 15, 2024 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant