Skip to content

Commit

Permalink
[Python] Skip termination when engine is not initialized (#3040)
Browse files Browse the repository at this point in the history
This PR skips the MLCEngine termination when the engine is
not initialized at all (e.g., failed during JIT compilation).
Previously there will be an extra error message saying
`'MLCEngine' object has no attribute '_ffi'` which is confusing
and misleading. Skipping the termination can eliminate this error
message.
  • Loading branch information
MasterJH5574 authored Nov 21, 2024
1 parent 78b69f0 commit 77b7857
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions python/mlc_llm/serve/engine_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -659,6 +659,8 @@ def terminate(self):
if hasattr(self, "_terminated") and self._terminated:
return
self._terminated = True
if not hasattr(self, "_ffi"):
return
self._ffi["exit_background_loop"]()
if hasattr(self, "_background_loop_thread"):
self._background_loop_thread.join()
Expand Down

0 comments on commit 77b7857

Please sign in to comment.