Skip to content

Commit

Permalink
[Core] v1: Use atexit to handle engine core client shutdown (vllm-pro…
Browse files Browse the repository at this point in the history
…ject#11076)

Signed-off-by: Russell Bryant <[email protected]>
  • Loading branch information
russellb authored Dec 11, 2024
1 parent 9974fca commit 61b1d2f
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions vllm/v1/engine/core_client.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import atexit
import multiprocessing
from typing import List, Union

Expand Down Expand Up @@ -157,6 +158,7 @@ def __init__(
should_shutdown=self.should_shutdown,
**kwargs,
)
atexit.register(self.shutdown)

def shutdown(self):
# Send shutdown signal to background process.
Expand Down

0 comments on commit 61b1d2f

Please sign in to comment.