Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SIGTERM handling to geographic gatherer #157

Merged
merged 7 commits into from
Nov 13, 2024

Conversation

pnuu
Copy link
Member

@pnuu pnuu commented Nov 12, 2024

This PR adds SIGTERM handling to geographic gatherer.

After receiving the signal, the gatherer will wait until all active collections have been finished and then exit the program.

@pnuu pnuu self-assigned this Nov 12, 2024
@pnuu
Copy link
Member Author

pnuu commented Nov 12, 2024

Locally I have an issue with the tests never exiting. The tests pass, but then hang until CTRL+C:

======================================================== 14 passed, 22 warnings in 2.22s ========================================================
^CException ignored in: <module 'threading' from '/home/lahtinep/mambaforge/envs/py312/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/home/lahtinep/mambaforge/envs/py312/lib/python3.12/threading.py", line 1624, in _shutdown
    lock.acquire()
KeyboardInterrupt: 
/home/lahtinep/mambaforge/envs/py312/lib/python3.12/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /tmp/pytest-of-lahtinep/garbage-3a9a0511-3423-47e3-959b-19062ba25739
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-lahtinep/garbage-3a9a0511-3423-47e3-959b-19062ba25739'
  warnings.warn(

The actual functionality seems to work fine when running the main script. At least when there are no collecting going on, which I don't have an easy way to test.

@pnuu
Copy link
Member Author

pnuu commented Nov 12, 2024

Running the actual script in terminal and killing the process:

$ geographic_gatherer.py -p 40000 -n false -i 127.0.0.1:40001 /tmp/geographic_gatherer.ini 
/home/lahtinep/mambaforge/envs/py312/bin/geographic_gatherer.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('pytroll-collectors==0.16.0+0.gacc2631.dirty')
Setting timezone to UTC
[INFO: 2024-11-12 08:49:33 : posttroll.backends.zmq.publisher] Publisher for tcp://*:40000 started on port 40000.
[INFO: 2024-11-12 08:49:33 : posttroll.backends.zmq.subscriber] Subscriber adding address tcp://127.0.0.1:40001 with topics ['pytroll://ears/mersi/fy-3d/segment']
[INFO: 2024-11-12 08:49:47 : pytroll_collectors.geographic_gatherer] Caught SIGTERM, shutting down when all collections are finished.
[INFO: 2024-11-12 08:49:47 : pytroll_collectors.geographic_gatherer] Shutting down...
[INFO: 2024-11-12 08:49:47 : pytroll_collectors.geographic_gatherer] Ending the gathering of granules...
[INFO: 2024-11-12 08:49:47 : geographic_gatherer] GeographicGatherer has stopped.

@pnuu
Copy link
Member Author

pnuu commented Nov 13, 2024

In 962e875 I mock the trigger creation. I think I need to create a more involved mock or fake trigger creation with collectors to simulate the structure so that the checks in _keep_running() are there.

Copy link

codecov bot commented Nov 13, 2024

Codecov Report

Attention: Patch coverage is 88.46154% with 6 lines in your changes missing coverage. Please review.

Project coverage is 91.78%. Comparing base (3409378) to head (e866900).
Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...troll_collectors/tests/test_geographic_gatherer.py 88.46% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #157      +/-   ##
==========================================
- Coverage   91.82%   91.78%   -0.05%     
==========================================
  Files          28       28              
  Lines        3953     4005      +52     
==========================================
+ Hits         3630     3676      +46     
- Misses        323      329       +6     
Flag Coverage Δ
unittests 91.78% <88.46%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

coveralls commented Nov 13, 2024

Pull Request Test Coverage Report for Build 11817159508

Details

  • 52 of 71 (73.24%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.3%) to 90.775%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pytroll_collectors/tests/test_geographic_gatherer.py 46 52 88.46%
pytroll_collectors/geographic_gatherer.py 6 19 31.58%
Totals Coverage Status
Change from base Build 11815718287: -0.3%
Covered Lines: 4113
Relevant Lines: 4531

💛 - Coveralls

@pnuu
Copy link
Member Author

pnuu commented Nov 13, 2024

With e866900 I think the new feature is "fully" tested.

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mraspaud mraspaud merged commit 33c5786 into pytroll:main Nov 13, 2024
6 of 8 checks passed
@pnuu pnuu deleted the feature-sigterm-geographic-gatherer branch November 13, 2024 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants